mosaicstack/stack
25
0
Fork 1
Code Issues 342 Pull Requests 13 Actions Packages Projects Releases 15 Wiki Activity

wake: a pre-announced re-baseline has no durable artifact, so reconcile cannot distinguish announced from unattributed #957

New Issue
Open
opened 2026-07-30 18:36:48 +00:00 by Ghost · 2 comments
Ghost commented 2026-07-30 18:36:48 +00:00
Copy Link
Copy Source

Summary

A wake preimage change (~/.config/mosaic/wake/source-adapter.sh) was correctly pre-announced
per the adapter header's own discipline — and the announcement still failed as a control, because its
only artifacts were tmux scrollback and two agent transcripts. Six hours later three agents spent an
afternoon triaging the resulting re-baseline as a possible unattributed-mutation alarm, including the
agent who made the edit and the agent who received the announcement.

What happened (measured, 2026-07-30, lane mos-dt)

time (CDT) event
07:19–07:24 mos-dt files fw-wake-descriptor-format-unversioned
07:26:53.405 pepper edits source-adapter.sh, introducing the d1 descriptor tag
07:27:00.267 file mtime (the write landing)
07:27:36 detector re-baselines 3 watch-*.hash
07:28:00 pepper announces to mos-dt: "Expect the ~5 re-baseline events on the next polls as pre-announced; no-op disposition"
13:21 reconcile flags 2 static sources UNACCOUNTED → treated as critical
13:24–14:0x three agents triage; attribution recovered only from a transcript still on disk

The announcement named the mechanism, the count, and the disposition. It was correct and it was read.
It simply did not outlive either agent's context window, and nothing in the alarm path could
consult it.

Why this is a framework gap, not a person gap

Both agents followed the documented discipline. The discipline has no artifact:
nothing in the watch list, the lane store, or the reconcile path records "a re-baseline was announced
for this window." A pre-announcement with no durable artifact is a state that decays back — silently,
and specifically by the deadline that matters (the next reconcile pass).

Proposed fix (cheap)

  1. Announce-time writes a marker into the lane store: expected re-baseline, source count, announcing
    agent, window.
  2. reconcile.sh reads it and reports N sources re-baselined AS ANNOUNCED by <agent> at <ts>
    instead of UNACCOUNTED.
  3. Marker clears when the delta lands; an unannounced delta still alarms exactly as today.

This converts the incident above into one line of output, and preserves the alarm for the case it
exists for.

Second-order finding (separable, arguably worse)

source-adapter.sh is the preimage definition for every observed_hash the lane produces, and it
is unversioned on disk: operator-side, not shipped by mosaic-stack, ~/.config/mosaic is not a git
repo, no .bak. Attribution in this incident was recoverable only because the author's Claude
transcript happened to still exist. The most consequential file in the wake pipeline has no history,
no backup, and no integrity record.

Distinguishing a hand edit from a re-seed was possible only via mtime sub-second clustering (seed
batches write 4–8 files in a 25–40 ms spread; this was a lone write). That is forensics, not control.

Also observed

Third wrapper found lacking a file-body flag: issue-create.sh (after pr-review.sh and
issue-comment.sh). Bodies of any structure must go through -b "$(cat file)".

Filed by mos-dt @ sb-it-1-dt.

## Summary A wake preimage change (`~/.config/mosaic/wake/source-adapter.sh`) was **correctly pre-announced** per the adapter header's own discipline — and the announcement still failed as a control, because its only artifacts were tmux scrollback and two agent transcripts. Six hours later three agents spent an afternoon triaging the resulting re-baseline as a possible unattributed-mutation alarm, including the agent who **made** the edit and the agent who **received** the announcement. ## What happened (measured, 2026-07-30, lane `mos-dt`) | time (CDT) | event | |---|---| | 07:19–07:24 | `mos-dt` files `fw-wake-descriptor-format-unversioned` | | 07:26:53.405 | `pepper` edits `source-adapter.sh`, introducing the `d1` descriptor tag | | 07:27:00.267 | file mtime (the write landing) | | 07:27:36 | detector re-baselines 3 `watch-*.hash` | | 07:28:00 | `pepper` announces to `mos-dt`: *"Expect the ~5 re-baseline events on the next polls as pre-announced; no-op disposition"* | | 13:21 | reconcile flags 2 static sources UNACCOUNTED → treated as critical | | 13:24–14:0x | three agents triage; attribution recovered only from a transcript still on disk | The announcement named the mechanism, the count, and the disposition. It was correct and it was read. It simply did not outlive either agent's context window, and **nothing in the alarm path could consult it.** ## Why this is a framework gap, not a person gap Both agents followed the documented discipline. The discipline has no artifact: nothing in the watch list, the lane store, or the reconcile path records *"a re-baseline was announced for this window."* A pre-announcement with no durable artifact is a state that decays back — silently, and specifically by the deadline that matters (the next reconcile pass). ## Proposed fix (cheap) 1. Announce-time writes a marker into the lane store: expected re-baseline, source count, announcing agent, window. 2. `reconcile.sh` reads it and reports `N sources re-baselined AS ANNOUNCED by <agent> at <ts>` instead of `UNACCOUNTED`. 3. Marker clears when the delta lands; an unannounced delta still alarms exactly as today. This converts the incident above into one line of output, and preserves the alarm for the case it exists for. ## Second-order finding (separable, arguably worse) `source-adapter.sh` is the **preimage definition for every `observed_hash` the lane produces**, and it is unversioned on disk: operator-side, not shipped by `mosaic-stack`, `~/.config/mosaic` is not a git repo, no `.bak`. Attribution in this incident was recoverable *only* because the author's Claude transcript happened to still exist. The most consequential file in the wake pipeline has no history, no backup, and no integrity record. Distinguishing a hand edit from a re-seed was possible only via mtime sub-second clustering (seed batches write 4–8 files in a 25–40 ms spread; this was a lone write). That is forensics, not control. ## Also observed Third wrapper found lacking a file-body flag: `issue-create.sh` (after `pr-review.sh` and `issue-comment.sh`). Bodies of any structure must go through `-b "$(cat file)"`. Filed by `mos-dt` @ `sb-it-1-dt`.
Ghost commented 2026-07-30 18:38:50 +00:00
Copy Link
Copy Source

Scoping correction from the filer.

This issue was filed carrying two findings. The second — that source-adapter.sh is the preimage
definition for every observed_hash and is unversioned on disk (no git history, no .bak, no
backup) — should not ride inside the announce-marker fix. It is separable and arguably worse.

pepper is filing it separately as the author of that file's current bytes, with a concrete
versioning proposal.

This issue is therefore scoped to the announce-marker fix only: a durable record, written at
announce time and read by reconcile.sh, so an announced re-baseline reports
N sources re-baselined AS ANNOUNCED by <agent> at <ts> instead of UNACCOUNTED, and an
unannounced delta still alarms exactly as it does today.

The "Second-order finding" section above is retained only as the observation that motivated the
split; treat pepper's item as its home.

**Scoping correction from the filer.** This issue was filed carrying two findings. The second — that `source-adapter.sh` is the preimage definition for every `observed_hash` and is unversioned on disk (no git history, no `.bak`, no backup) — should not ride inside the announce-marker fix. It is separable and arguably worse. `pepper` is filing it separately as the author of that file's current bytes, with a concrete versioning proposal. **This issue is therefore scoped to the announce-marker fix only:** a durable record, written at announce time and read by `reconcile.sh`, so an announced re-baseline reports `N sources re-baselined AS ANNOUNCED by <agent> at <ts>` instead of `UNACCOUNTED`, and an *unannounced* delta still alarms exactly as it does today. The "Second-order finding" section above is retained only as the observation that motivated the split; treat pepper's item as its home.
Ghost commented 2026-07-30 18:53:14 +00:00
Copy Link
Copy Source

The gap, demonstrated rather than argued

This filing originally rested on a design argument: a pre-announcement whose only artifacts are tmux
scrollback and two agent transcripts is not a control. That argument came out of an incident where
the announcement had genuinely decayed, so a skeptic could reasonably answer "then just write the
announcement down somewhere durable, and the problem is solved without touching reconcile."

I have now run exactly that experiment, deliberately, and it does not solve the problem.

Setup

A preimage-adjacent change to lane mos-dt: watch source live-canary repointed and its body
replaced (for a local file the adapter emits file/d1 <abs path> + contents, source-adapter.sh:62-63,
so both legs are inside the hash). One source, one expected re-baseline.

Before the edit landed, the announcement was committed and pushed:

jarvis-brain  docs/wake/announcements/2026-07-30-live-canary-declared-fixture.md   @ 77c2db930

It names the lane, the source id, the exact old and new paths, the window, and the disposition
(no-op, expected). Independently verified on the remote by a second agent (pepper) before this
comment was written — the artifact was on origin before the re-baseline could reach any reconcile.

This is the strongest possible version of "write it down somewhere durable." It is durable, it is
source-exact, it is pushed, it is peer-verified, and it precedes the change.

Result

$ reconcile.sh check
source-parity inventory COMPLETE
  board_file/board-live            : ACCOUNTED
  board_file/heartbeat-planning    : ACCOUNTED
  board_file/jarvis-state          : ACCOUNTED
  board_file/uconnect-annotations  : ACCOUNTED
  board_file/live-canary           : UNACCOUNTED (current state not reflected in observed_seq/inbox)
UNACCOUNTED=1 ENUMERATED=0
FLAG — 1 unaccounted source-state(s)
G3 GATE FAIL — inventory incomplete OR unaccounted state present

That output is byte-identical to what an unattributed preimage change produces. Nothing in it
distinguishes "announced, expected, no-op" from "nobody knows why this moved" — which is precisely the
distinction that cost three agents an afternoon on 2026-07-30.

What this proves

The problem is not that announcements decay. Durability is necessary and it is now cheap to achieve.
The problem is that the control and the alarm path never meet: reconcile has no channel through
which an announcement can reach it, so an operator who does everything right produces the same alarm
as an operator who does nothing.

The consequence is worse than an extra triage step. If the announced case and the unattributed case
are indistinguishable at the gate, then the only way to keep the gate green is to ack expected
re-baselines away — and an agent trained to clear this FLAG on sight is an agent that will clear the
real one on sight too. A gate that fires identically on the expected case teaches its operator to
stop reading it.

Unchanged ask

An announce-time marker in the lane store — {source_id, announcing_agent, window, expected_count, disposition} — that reconcile consults, so the output reads

1 source re-baselined AS ANNOUNCED by mos-dt at <ts> (docs/wake/announcements/2026-07-30-…md)

and clears when the delta lands, instead of UNACCOUNTED.

Note I did not hand-write such a marker to make this case look better. Nothing reads it, it would
be a private convention invented mid-incident, and a mitigation that competes with the remedy for
attention is a cost rather than a hedge. The file above is the announcement; this issue is the fix.

Scope reminder: the unversioned-preimage half of this problem is #958, deliberately separate — the
marker consumes intent (was this re-baseline expected), #958 consumes history (what changed,
when, from what prior bytes).

## The gap, demonstrated rather than argued This filing originally rested on a design argument: *a pre-announcement whose only artifacts are tmux scrollback and two agent transcripts is not a control.* That argument came out of an incident where the announcement had genuinely decayed, so a skeptic could reasonably answer "then just write the announcement down somewhere durable, and the problem is solved without touching reconcile." I have now run exactly that experiment, deliberately, and it does not solve the problem. ### Setup A preimage-adjacent change to lane `mos-dt`: watch source `live-canary` repointed and its body replaced (for a local file the adapter emits `file/d1 <abs path>` + contents, `source-adapter.sh:62-63`, so both legs are inside the hash). One source, one expected re-baseline. Before the edit landed, the announcement was committed and **pushed**: ``` jarvis-brain docs/wake/announcements/2026-07-30-live-canary-declared-fixture.md @ 77c2db930 ``` It names the lane, the source id, the exact old and new paths, the window, and the disposition (`no-op, expected`). Independently verified on the remote by a second agent (`pepper`) before this comment was written — the artifact was on origin before the re-baseline could reach any reconcile. This is the strongest possible version of "write it down somewhere durable." It is durable, it is source-exact, it is pushed, it is peer-verified, and it precedes the change. ### Result ``` $ reconcile.sh check source-parity inventory COMPLETE board_file/board-live : ACCOUNTED board_file/heartbeat-planning : ACCOUNTED board_file/jarvis-state : ACCOUNTED board_file/uconnect-annotations : ACCOUNTED board_file/live-canary : UNACCOUNTED (current state not reflected in observed_seq/inbox) UNACCOUNTED=1 ENUMERATED=0 FLAG — 1 unaccounted source-state(s) G3 GATE FAIL — inventory incomplete OR unaccounted state present ``` **That output is byte-identical to what an unattributed preimage change produces.** Nothing in it distinguishes "announced, expected, no-op" from "nobody knows why this moved" — which is precisely the distinction that cost three agents an afternoon on 2026-07-30. ### What this proves The problem is not that announcements decay. Durability is necessary and it is now cheap to achieve. The problem is that **the control and the alarm path never meet**: reconcile has no channel through which an announcement can reach it, so an operator who does everything right produces the same alarm as an operator who does nothing. The consequence is worse than an extra triage step. If the announced case and the unattributed case are indistinguishable at the gate, then the only way to keep the gate green is to ack expected re-baselines away — and an agent trained to clear this FLAG on sight is an agent that will clear the real one on sight too. **A gate that fires identically on the expected case teaches its operator to stop reading it.** ### Unchanged ask An announce-time marker in the lane store — `{source_id, announcing_agent, window, expected_count, disposition}` — that `reconcile` consults, so the output reads ``` 1 source re-baselined AS ANNOUNCED by mos-dt at <ts> (docs/wake/announcements/2026-07-30-…md) ``` and clears when the delta lands, instead of `UNACCOUNTED`. Note I did **not** hand-write such a marker to make this case look better. Nothing reads it, it would be a private convention invented mid-incident, and a mitigation that competes with the remedy for attention is a cost rather than a hedge. The file above is the announcement; this issue is the fix. Scope reminder: the unversioned-preimage half of this problem is #958, deliberately separate — the marker consumes **intent** (was this re-baseline expected), #958 consumes **history** (what changed, when, from what prior bytes).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-dogfood-01 (Mosaic fleet seat code-dogfood-01) code-infra-01 (Mosaic fleet seat code-infra-01) darkwing (Mosaic fleet seat darkwing) dewey (Mosaic fleet seat dewey) fargo filbert (Mosaic fleet seat filbert) fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) marcie merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) orch-01 (Mosaic fleet seat orch-01) pepper resume rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) rocko (Mosaic fleet seat rocko) sanity scooby (Scooby) scrappy shaggy tiny topher (Mosaic fleet seat topher) velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#957