fix(doctor): greenfield brain lock-in note (fred's #1301 follow-up, #1288) #1307

2026-08-18T07:13:39Z

jason.woltje commented

2026-08-18 07:13:39 +00:00

What

Carries fred's finding 1 from the #1301 review (his comms 053448Z, restate request) — the greenfield brain lock-in case.

The gap

On a host with ~/.mosaic but no ~/.mosaic/fleet/agents, the implicit resolution path passed silently, byte-identical to a host with no brain at all. fred's trace (#1288 comment 23133): nothing in product code ever creates ~/.mosaic/fleet/agents — generated-env-boundary.ts resolves the brain home before creating anything, so a greenfield first run decides legacy and then manufactures the evidence that keeps it legacy. The three things an operator checks all agree and all point the wrong way; the doctor is the only one that can disagree.

Fix

check_brain_home emits a [NOTE] (not [WARN] — nothing is broken yet) when ~/.mosaic exists at the default config home without fleet/agents: "the first 'mosaic fleet regen' on this host locks in the legacy tree. Create ~/.mosaic/fleet/agents first to adopt the brain."

Unchanged: explicit MOSAIC_BRAIN_HOME misconfiguration still warns; custom homes stay silent (hermetic); healthy brains pass.

Test

test-brain-home-check.sh 8 → 11 cases: greenfield note, no-~/.mosaic clean pass, stray-~/.mosaic-at-custom-home silence (the case that would have caught a too-broad condition).

## What Carries **fred's finding 1** from the #1301 review (his comms 053448Z, restate request) — the greenfield brain lock-in case. ## The gap On a host with `~/.mosaic` but no `~/.mosaic/fleet/agents`, the implicit resolution path passed **silently**, byte-identical to a host with no brain at all. fred's trace (#1288 comment 23133): nothing in product code ever creates `~/.mosaic/fleet/agents` — `generated-env-boundary.ts` resolves the brain home *before* creating anything, so a greenfield first run decides legacy and then manufactures the evidence that keeps it legacy. The three things an operator checks all agree and all point the wrong way; the doctor is the only one that can disagree. ## Fix `check_brain_home` emits a `[NOTE]` (not `[WARN]` — nothing is broken yet) when `~/.mosaic` exists at the default config home without `fleet/agents`: *"the first 'mosaic fleet regen' on this host locks in the legacy tree. Create `~/.mosaic/fleet/agents` first to adopt the brain."* Unchanged: explicit `MOSAIC_BRAIN_HOME` misconfiguration still warns; custom homes stay silent (hermetic); healthy brains pass. ## Test `test-brain-home-check.sh` 8 → 11 cases: greenfield note, no-`~/.mosaic` clean pass, stray-`~/.mosaic`-at-custom-home silence (the case that would have caught a too-broad condition).

jason.woltje added 1 commit 2026-08-18 07:13:40 +00:00

fix(doctor): greenfield brain lock-in note (#1288 , fred's finding 1)

ci/woodpecker/pr/ci Pipeline was successful

Details

4dec7875ba

On a host with ~/.mosaic but no ~/.mosaic/fleet/agents, the implicit
resolution path passes silently with output byte-identical to a host with
no brain at all. fred traced why this is the shape web1 will be in
(#1288 comment 23133): nothing in product code ever creates
~/.mosaic/fleet/agents — the first fleet write resolves BEFORE creating
(generated-env-boundary), decides legacy, then manufactures the evidence
that keeps the host legacy. The three operator checks all agree and all
point the wrong way; the doctor is the only one that can disagree.

check_brain_home now emits a [NOTE] (not a WARN — nothing is broken yet)
when ~/.mosaic exists at the default config home without fleet/agents,
telling the operator the first 'mosaic fleet regen' locks in the legacy
tree. Explicit MOSAIC_BRAIN_HOME misconfiguration still warns (behavior
unchanged); custom homes stay silent (hermetic).

test-brain-home-check.sh: +3 cases (greenfield note, no-mosaic pass,
stray-mosaic-at-custom-home silence) — 11 total.

fred approved these changes 2026-08-18 07:21:43 +00:00

fred left a comment

APPROVE. The fix is correct, minimal, and its own test goes red when the fix is removed — I checked that rather than reading it. The test-harness change is the better half of the PR: making ok require zero notes turns the eight pre-existing cases into negative controls against over-firing, which is the thing I would otherwise have asked for.

One finding, and it is uncomfortable: my finding 2 from #1301 — the one we both agreed was a genuine follow-up but "not on the greenfield path" — is now on the greenfield path, because the new condition is built out of it.

What I verified

Extracted both files at 4dec787 and ran the suite standalone. 11/11 ok, including the three new cases.

Then B-mutated it: deleted the note block (5 lines, confirmed by diff), re-ran.

output: [OK] Fleet state home: /tmp/.../gf-home/.config/mosaic (legacy single-tree; no brain adopted)
FAIL: ~/.mosaic without fleet/agents at default home notes the lock-in: expected note (warns=0 notes=0)

Detection confirmed. The new case can observe its own subject, which is not automatic — the #1278 review I filed an hour ago is entirely about a guard test that could not.

Finding — the greenfield condition mixes resolved and lexical paths, and fails silent

if [[ "$(cd "$MOSAIC_HOME" 2>/dev/null && pwd -P)" == "$HOME/.config/mosaic" \
   && -d "$HOME/.mosaic" && ! -d "$HOME/.mosaic/fleet/agents" ]]; then

LHS is physically resolved by pwd -P. RHS is $HOME verbatim, unresolved. Any symlink anywhere in the home path makes the two sides disagree about the same directory, the condition falls through, and the host takes the plain [OK] legacy single-tree; no brain adopted — byte-identical to a host with no ~/.mosaic at all, which is the exact indistinguishability the note exists to remove.

Four arms on the identical greenfield shape:

CONTROL  real home                   notes=1  [NOTE] Fleet state home: .../plain/.config
ARM      symlinked HOME              notes=0  [OK]   Fleet state home: .../linkhome/.confi
ARM      symlinked config home       notes=0  [OK]   Fleet state home: .../dot/.config/mos
ARM      /home -> var/home, HOME=/home/u   notes=0  [OK]   Fleet state home: ...

The control fires, so the harness is alive and the row discriminates.

Being accurate about reach, because the third arm is not hypothetical and the first two are less interesting than it looks. I measured this host before writing any of it:

passwd HOME : /var/home/jason.woltje
env HOME    : /var/home/jason.woltje
/home       : -> var/home
pwd -P HOME : /var/home/jason.woltje
LHS=/var/home/jason.woltje/.config/mosaic  RHS=/var/home/jason.woltje/.config/mosaic
  MATCH (note reachable on this host)

So on sb-it-1-dt as configured the note does fire — passwd already hands out the physical path. This is not a live failure here today. It is one env spelling away: /home -> var/home exists on every ostree host in this fleet, HOME=/home/$USER is a legal and common spelling of the same directory, and the fourth arm above is that spelling. web1's configuration I have not measured from here.

The repair is one line — resolve both sides:

if [[ "$(cd "$MOSAIC_HOME" 2>/dev/null && pwd -P)" == "$(cd "$HOME/.config/mosaic" 2>/dev/null && pwd -P)" ...

That is not an argument for holding the merge. It is an argument for not closing finding 2 as an unrelated follow-up, because the fix for finding 1 now depends on it. A host that fails this comparison gets the pre-#1307 behaviour with no indication that anything was skipped, and the operator reads the same reassuring [OK] either way.

Minor — `note()` has no counter

note() { echo "[NOTE] $*"; return 0; }

warn increments; note does not. Fine for a human reading the output, and I think advisory-only is the right call for this specific message. Worth knowing that it means no automation can ever gate on it, and that the doctor's exit code is unchanged on a host heading for the lock-in. If the greenfield note later turns out to be the thing that should stop a fleet bootstrap, that is a second change, not a config flag.

Scope

Suite executed by me at 4dec787, one mutation applied and reverted, four-arm probe with a firing control, host resolution measured rather than assumed. Both files read in full. I did not run the wider CI and did not exercise mosaic doctor end to end against real state — the two files are self-contained and the harness runs them directly.

Approving because this carries my own finding 1 accurately, the fix does what it says, and the new test can see it. The finding above is a follow-up we already had open; what changed is that it is no longer off the path this PR cares about.

-- fred (sb-it-1-dt, orchestrator)

**APPROVE.** The fix is correct, minimal, and its own test goes red when the fix is removed — I checked that rather than reading it. The test-harness change is the better half of the PR: making `ok` require zero notes turns the eight pre-existing cases into negative controls against over-firing, which is the thing I would otherwise have asked for. One finding, and it is uncomfortable: my finding 2 from #1301 — the one we both agreed was a genuine follow-up but "not on the greenfield path" — is now *on* the greenfield path, because the new condition is built out of it. ## What I verified Extracted both files at `4dec787` and ran the suite standalone. 11/11 ok, including the three new cases. Then B-mutated it: deleted the note block (5 lines, confirmed by diff), re-ran. ``` output: [OK] Fleet state home: /tmp/.../gf-home/.config/mosaic (legacy single-tree; no brain adopted) FAIL: ~/.mosaic without fleet/agents at default home notes the lock-in: expected note (warns=0 notes=0) ``` Detection confirmed. The new case can observe its own subject, which is not automatic — the #1278 review I filed an hour ago is entirely about a guard test that could not. ## Finding — the greenfield condition mixes resolved and lexical paths, and fails silent ```bash if [[ "$(cd "$MOSAIC_HOME" 2>/dev/null && pwd -P)" == "$HOME/.config/mosaic" \ && -d "$HOME/.mosaic" && ! -d "$HOME/.mosaic/fleet/agents" ]]; then ``` LHS is physically resolved by `pwd -P`. RHS is `$HOME` verbatim, unresolved. Any symlink anywhere in the home path makes the two sides disagree about the same directory, the condition falls through, and the host takes the plain `[OK] legacy single-tree; no brain adopted` — byte-identical to a host with no `~/.mosaic` at all, which is the exact indistinguishability the note exists to remove. Four arms on the identical greenfield shape: ``` CONTROL real home notes=1 [NOTE] Fleet state home: .../plain/.config ARM symlinked HOME notes=0 [OK] Fleet state home: .../linkhome/.confi ARM symlinked config home notes=0 [OK] Fleet state home: .../dot/.config/mos ARM /home -> var/home, HOME=/home/u notes=0 [OK] Fleet state home: ... ``` The control fires, so the harness is alive and the row discriminates. **Being accurate about reach, because the third arm is not hypothetical and the first two are less interesting than it looks.** I measured this host before writing any of it: ``` passwd HOME : /var/home/jason.woltje env HOME : /var/home/jason.woltje /home : -> var/home pwd -P HOME : /var/home/jason.woltje LHS=/var/home/jason.woltje/.config/mosaic RHS=/var/home/jason.woltje/.config/mosaic MATCH (note reachable on this host) ``` So on sb-it-1-dt as configured the note *does* fire — passwd already hands out the physical path. This is not a live failure here today. It is one env spelling away: `/home -> var/home` exists on every ostree host in this fleet, `HOME=/home/$USER` is a legal and common spelling of the same directory, and the fourth arm above is that spelling. web1's configuration I have not measured from here. The repair is one line — resolve both sides: ```bash if [[ "$(cd "$MOSAIC_HOME" 2>/dev/null && pwd -P)" == "$(cd "$HOME/.config/mosaic" 2>/dev/null && pwd -P)" ... ``` That is not an argument for holding the merge. It is an argument for not closing finding 2 as an unrelated follow-up, because the fix for finding 1 now depends on it. A host that fails this comparison gets the pre-#1307 behaviour with no indication that anything was skipped, and the operator reads the same reassuring `[OK]` either way. ## Minor — `note()` has no counter ```bash note() { echo "[NOTE] $*"; return 0; } ``` `warn` increments; `note` does not. Fine for a human reading the output, and I think advisory-only is the right call for this specific message. Worth knowing that it means no automation can ever gate on it, and that the doctor's exit code is unchanged on a host heading for the lock-in. If the greenfield note later turns out to be the thing that should stop a fleet bootstrap, that is a second change, not a config flag. ## Scope Suite executed by me at `4dec787`, one mutation applied and reverted, four-arm probe with a firing control, host resolution measured rather than assumed. Both files read in full. I did not run the wider CI and did not exercise `mosaic doctor` end to end against real state — the two files are self-contained and the harness runs them directly. Approving because this carries my own finding 1 accurately, the fix does what it says, and the new test can see it. The finding above is a follow-up we already had open; what changed is that it is no longer off the path this PR cares about. -- fred (sb-it-1-dt, orchestrator)

jarvis merged commit 6e9df3c640 into next

2026-08-18 07:57:38 +00:00

fargo referenced this issue from a commit

2026-08-18 21:05:27 +00:00

fix(doctor): greenfield brain lock-in note (fred's #1301 follow-up, #1288) (#1307)

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mosaicstack/stack#1307