Files
jason.woltje 21aa1cab0b darkwing: re-pin window rows 177-185 packet, STATE.md (sha256 b586f11c9b6f1329), W.txt (edd4622c5c45ef3f), S1/S2/S2b/B snapshots, matrices, W10 browser check and P3 rehearsals (#1683; P19 386)
NEW 490e2771d live for mosaic-stack, activation 20261020T075800Z-490e2771d3d0, store 3i. W0-W10 pass;
stops at W9 (assistant idle stop, mos-0948) and S2b/B (old chief end rows in the backup, mos-0950).
2026-10-20 03:27:30 -05:00

13 KiB

Re-pin window rows 177-185, STATE report (W11)

Operator and tester: Darkwing. Plan: agents/sage/work/window-plan/repin-rows-177-185.md draft 4 at 9ddd8a92c, sha256 b05eb85e357ecb7d1c267c537a3a8c6fef82967368520101ce842d1677e871a1. Tool blob f8051d81cc (scripts/stack-release.sh), checked before W0, W5, W9 and W10. OLD 89af38a83a, NEW 490e2771d3 (the row 185 landing). Every stamp below comes from date -u or from the transcript W.txt in this directory. Written 2026-10-20 after W10 (mos-0952).

Outcome

NEW is live for mosaic-stack with its PM. The store stays on 3i and the tester listener is on 127.0.0.1:7333. W0 to W10 passed and nothing was rolled back. Mos accepted every step. The window had two stops, both reported to Mos before going on. The W9 assistant run was still up 10 minutes after its reply. The B snapshot differed from S1 by the old chief's end rows. Both had a known cause, explained below. Nobody acted as Jason. There was no --by jason, no pi, no Save and no mosaic name, and bus.tester was not touched.

Stamps and activation

Step Start (UTC) Result
P1 values, business-check OLD 07:19:33.013Z values as planned, LOAD_OK
P3 rehearsal run 3, limits rerun, proofs 07:32:23Z, 07:46:44Z to 07:49:16Z PASS, see p3/NOTES.md
P4, P5, P6 07:37:43Z, 07:37:43Z, 07:37:49Z P4 PASS, P5 PASS (HEAD 2ea05700b), matrix-old.txt
W0 copy NEW, business-check NEW 07:53:51.633Z copied, LOAD_OK
W0b business file copy 07:53:53Z sha256 1e49bdf0
W1 idle check, start stamp, S1 07:53:53Z chief idle; S1 2fd8b96c
W2 status 07:53:53.535Z pointer OLD, pm true, 3i, check pass, unit flags match
W3 plan 07:53:53.741Z exit 0, matches P3 run 3 except the root and live-flag lines (7, 61-63)
W4 unit read 07:53:53.879Z OLD units, no drop-ins, bus --pm, no quote or backslash
W5 activate (mos-0946 go 07:57:23Z) 07:58:00.006Z, end 07:58:02.738Z exit 0, gate passed, activate ok
W5 status 07:58:22.071Z pointer NEW, pm true, 3i, check pass, unit flags match
W9 checks and turns (mos-0947 go 08:00:50Z) 08:01:03Z pass (mos-0948)
W9 run end and S2 08:16:18.103Z, S2 08:16:22Z ended by the launcher; S2 58e864da
S2b and B (mos-0949) 08:18:46Z S2b a9e8d6df, B 42553b74; stop, then accepted (mos-0950)
W10 browser, matrix, P4 (mos-0950 go) 08:23:32.268Z to 08:23:54.592Z pass (mos-0952)
W11 reads after 08:25Z units, tunnel, business file sha

Activation id 20261020T075800Z-490e2771d3d0. Pointer at 2026-10-20T07:58:02Z.

Backups and hashes

  • Store backup: ~/.mosaic-dev/backups/stack/20261020T075800Z-490e2771d3d0. STAMP 2026-10-20T07:58:00.381Z, SCHEMA 3i, DATA_ROOT /home/jwoltje/.mosaic-dev. SHA256SUMS sha256 399f710540430750bf3f1fb488824b4239be841f2f91e4668a2d749ad2065683, sha256sum -c OK in bus/. bus.sqlite sha256 1729da1b4e87ec4ec7768590580a439b7507aa31b6b4591fd47d13bbb12c103d. It had no -wal or -shm before and after the S2b and B reads, and its hash didn't change across them.
  • Rollback set: ~/.mosaic-dev/state/stack-rollback/20261020T075800Z-490e2771d3d0.
  • Business file: 1e49bdf05e62607972958f5b7298cfd600c7825812370e26d7c2a842f2af9b93 at P1, at W0b and again at W11. The W0b copy is ~/.config/mosaic-dev/businesses/mosaic-stack.json.pre-window-20261020T075353Z with the same hash. Nothing in the window wrote it.
  • backups.txt holds both lines (sha256 427d7f55).

Pointer before and after

Before (W2): copy 89af38a83a, pm true, schema 3i, activation 20261020T014103Z-89af38a83afa, at 2026-10-20T01:41:05Z. Console flags --port 7332 --public-host mosaic.woltje.com --auth-header X-authentik-username --allow-user jason.woltje --send --tester-port 7333. Bus pid 1514133.

After (W5 and W9 status): copy 490e2771d3, pm true, schema 3i, activation 20261020T075800Z-490e2771d3d0, at 2026-10-20T07:58:02Z. The Console flags are the same, with no --proof-header or --proof-file. Bus pid 555397, since 07:58:01.329Z. Log line activate ... "backup":"20261020T075800Z-490e2771d3d0","result":"ok".

Unit ExecStart lines (no Environment values)

W4, before:

bus:   .../copies/89af38a83afa22a8a4f0d368a0a4030ad15a0efa/scripts/mosaic bus start mosaic-stack --pm
webui: /usr/bin/node .../copies/89af38a83afa22a8a4f0d368a0a4030ad15a0efa/packages/webui/src/cli.mjs serve --checkout /mnt/storage/src/mosaic-stack --port 7332 --public-host mosaic.woltje.com --auth-header X-authentik-username --allow-user jason.woltje --send --tester-port 7333

W11, after. Both units active, DropInPaths empty, WorkingDirectory the NEW copy:

bus:   .../copies/490e2771d3d057ab76991e2c7859cd251857e64c/scripts/mosaic bus start mosaic-stack --pm   (MainPID 555397)
webui: /usr/bin/node .../copies/490e2771d3d057ab76991e2c7859cd251857e64c/packages/webui/src/cli.mjs serve --checkout /mnt/storage/src/mosaic-stack --port 7332 --public-host mosaic.woltje.com --auth-header X-authentik-username --allow-user jason.woltje --send --tester-port 7333   (MainPID 555401)

The bus Environment has 2 assignments with no quote or backslash. The values are not recorded.

Tunnel (mos-0952): mosaic-preview-tunnel.service ActiveState=active, SubState=running, MainPID=1317, ActiveEnterTimestamp Tue 2026-10-13 08:40:19 CDT. The window didn't touch it. The pid=0 in both P4 reads is ExecStart's static field, not the running process.

S1, S2 and the per-row check

S1 is W1's held read at 07:53:53Z. S2 is the held read after the assistant run ended, at 08:16:22Z. S2b is a held read against the W5 backup, and B is the backup alone with --stopped. The schema is 3i in every one. SNAP (snapshot.mjs, 639fd28a) was checked before S1, S2, S2b and B.

Table S1 rows B rows S2 rows S2 vs S1
decisions, decision_events 20, 20 20, 20 20, 20 same sha256
task_snapshots, task_comment_cursors 359, 171 359, 171 359, 171 same sha256
meta, instance_names, thread_opens 3, 0, 0 3, 0, 0 3, 0, 0 same sha256
deliveries 1673 1673 1687 +14
events 1424 1425 1431 +7
role_claims 91 92 93 +2
messages 550 550 554 +4 (counts only)
message_threads 0 0 2 +2 (counts only)
usage_records 12 12 16 +4
projects, threads, thread_claims 0, 0, 0 0, 0, 0 1, 1, 2 new

For every grown table with a seq, maxSeq grew by exactly the row delta, so S2 lost no row. usage byRun keeps S1's four runs with the same sums and adds the chief r34520a581318 (2/111) and the assistant r7e99838f9f93 (2/44).

S2b. Every table with a seq has priorSame true, onlyInBackup is empty, meta keysChanged is empty, and message_threads went from 0 to 2. So no row at or below the backup's maxSeq changed in any table. Changed old rows: 0. stopTables lists only the tables with new rows.

B against S1. Twelve tables are equal. events (1425 against 1424) and role_claims (92 against 91) are not. Under mos-0949 that stopped the window before W10, and I reported it. bprefix.mjs hashed the backup's rows at or below S1's maxSeq the same way SNAP does. Both prefixes equal S1: events bfc78646, role_claims adc44bef. The extra rows are role_claims 92 (07:58:00.217Z, chief release, holder and by rc78c9700c8e9) and events 1425 (07:58:00.238Z, session.ended, chief rc78c9700c8e9). Both come from OLD's broker.mjs endLaunch (:484, release :499, event :505) when W5 stopped the old chief. The launches log shows the end at 07:58:00.246Z, before the backup STAMP at 07:58:00.381Z. mos-0950 accepted this and recorded that S1 and the backup are not equal by design. The chain holds row by row: S1 is a prefix of the backup, and the backup is a prefix of S2.

W9 output

  • status: pointer NEW, check pass. console-check NEW (repo copy d068fe02): CONSOLE_PASS. The journal since 07:58:00 UTC has 1 Proxy: proxy proof off line.
  • launches list: one running chief, r34520a581318, since 07:58:02.199Z. agents --tester: chief idle.
  • talk chief --tester: sent 580f50ed, RESULT 90fb6939 with one line and no staffing request.
  • project create window-20261020-0801 --tester: id 8fe35a6d-ebe8-4607-ba67-7625f2ef09f6.
  • thread create window-20261020-0801 w9 --tester: id d0ba7a66-aa3f-4918-9518-94132bce6b28, instance assistant.
  • talk --thread --wait 180 --tester: sent dc002b67, RESULT 48cdd10b. The reply arrived at 08:01:18Z.
  • thread read --json --tester: 2 messages, a tester to assistant REQUEST and an assistant to tester RESULT. threads --tester lists the thread. C_NEW names --tester --json: [].
  • usage --tester --json: new byRun entries for the chief and the assistant, as in S2.
  • Human-thread probe: skipped. S1 has 0 threads and 0 message_threads rows, so there was no human thread to read.
  • The assistant run r7e99838f9f93 (launched 08:01:14.335Z) was still up at 08:11:28Z. I stopped and reported (STOP 1). mos-0948 said not to stop it, since the launcher ends a thread run after threadIdleSeconds, 900 s by default (business.mjs:20, launcher threadPass). The launches log has thread-idle at 08:16:18.082Z and end reason=stopped exitCode=0 at 08:16:18.103Z. The bus journal has run r7e99838f9f93 (assistant) ended: stopped. Nothing was stopped by hand.

W10 output

  • Browser (w10-browser.mjs b653c2ce, harness packages/webui/tests/browser.mjs same as 490e2771d): 12 of 12 checks pass. Settings shows the tester banner and the Routing, Role instances and Seat names panels. It has no Token limits panel and no launch settings, assistant, limit reset or send. The Board reads, with the chief held and idle. #/chat lists thread w9 (assistant) at #/chat/threads/d0ba7a66-.... The thread page was never opened. There were 23 requests, all GET, none refused and none to the W9 thread. Log .w10-browser-run.log a60258ab; screenshots w10-settings.png 575ea467, w10-board.png 716efa0f, w10-chat.png 4e383493.
  • Matrix: matrix-new.txt is byte-identical to matrix-old.txt, both sha256 93c07e15:
a 200 ok=true
b 401 error="sign-in required"
c 401 error="sign-in required"
d 403 error="proxy Host refused"
e 403 error="tester listener required"
f 403 error="proxy header refused"
g 200 non-json 6454 bytes

index.html is the same in OLD and NEW, so g's size matches. No request carried jason.woltje or a credential. The proof value is a fixed 32-letter dummy.

  • P4 again (P4-W10.txt): PASS, the same as P4 apart from w.sh's exit line.

Stops

  1. W9, 08:11:28Z: the assistant run was still up 10 minutes after its reply, so a stop was Mos's call. mos-0948 ruled it a plan gap, not a fault. The plan's 10-minute rule doesn't account for the 900 s idle stop the launcher applies by design.
  2. S2b and B, 08:19:04Z: B didn't match S1. I stopped before W10 under mos-0949. mos-0950 confirmed the stop was correct and the cause is by design (above).

Deviations and notes

  • console-check.sh ran from the repo copy agents/sage/work/window-plan/repin-de352dfaf/ (d068fe02) after its SCRIPTS.sha256 check, not from the plan's $P. Darkwing doesn't read /mnt/storage/scratch/sage-*.
  • business-check.mjs (eece4d46) ran from the same repo copy, for the same reason.
  • The human-thread probe was skipped, as noted in W9.
  • bprefix.mjs (adae9dc7) is a script outside the plan. It opened only the backup, with immutable=1 and readOnly.
  • Mos's expectation in mos-0949 was wrong. The chief release is an appended role_claims row, not an in-place update, and deliveries showed no change to an old row (mos-0950).
  • by jason on launches list lines is the launcher's record of b.human. The tester started nothing as Jason. It is a display item (mos-0948).
  • mos-0938 time bounds: mos-0934 went out before 07:37:04Z, mos-0936 before 07:37:58Z and mos-0937 before 07:42:17Z. Mos's live limits GET on 7333 ran before 07:37:04Z.
  • P3 scratch deviations, none of which apply to live. The PM launch was emulated by a stand-in that holds no claim and whose session has no provider, so limits refused on both copies until proof 2 set an exact provider and model. The example business file has no thinking var, so the assistant thread refused no-route until proof 1 added one. Scratch has no session credential, so the assistant start refused no-session-credential on both copies once the route was set. The claude binary is a fixture. Detail and hashes are in p3/NOTES.md.
  • My slip: there was no interim report between 07:58:28Z and 08:11:39Z, which breaks Mos's 10-minute report rule. After mos-0948 no wait ran past 10 minutes.

Files

Transcript W.txt (final sha256 below). S1.json 2fd8b96c, S2.json 58e864da, S2b.json a9e8d6df, B.json 42553b74, W3-plan.txt fe61fae5, P4.txt b3479e0d, P4-W10.txt ff845006, P5.txt 0f4cfc1b, P6-run.txt e4a1d56b, matrix-old.txt and matrix-new.txt 93c07e15, .w9b-run.log, .w9c-run.log, .w9c-bprefix.txt 5cc5feec, .w10-browser-run.log a60258ab, and the per-step .w0-*, .w1-* and .w9-* captures. Scripts: w.sh a45baeb5, wsteps.sh 95b55320, w9.sh be39da24, w9b.sh 871aee2c, w9c.sh c4e08d78, bprefix.mjs adae9dc7, p1.mjs a19a3639, p1.sh 71ea61c6, p456.sh af6a9293, w10-browser.mjs b653c2ce, snapshot.mjs 639fd28a.

W.txt final sha256: edd4622c5c45ef3fd4d1a832d7484b14384f0001257e6d3a1bd2f3cfd403b59a