Files
stack/docs/remediation/MISSION.md
T
mos-dt-0andClaude Opus 5 d1e7ba19ca docs(remediation): promote observe-the-property principle to charter; bank D-13 dual credential registries
Charter gains a first-class principle per Mos: no write is done until the requested
PROPERTY is observed; a success exit code is not evidence. Success output is designed
to be believed — that is why the inert-gate class exists and why the P-WRAPPER
tri-state is not optional. Recorded with its provenance: the orchestrator committed
this exact error (D-12), and three of the session's twelve instances were its own.

D-13: diagnosing D-12 found two parallel credential registries that can disagree.
gitea-mosaicstack-mos-dt-0.token EXISTS, but tea has no mosaicstack login for that
identity — so raw-API paths work while tea-dependent wrapper paths silently degrade.
tea is not stale; the login does not exist. Capability declared authoritative by the
token-file set does not govern the tea path.

RM-04 must reconcile the registries (or assert agreement at startup, with a must-fail
control). RM-50's pre-dispatch check must verify capability for the path actually used.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-07-31 18:06:18 -05:00

9.4 KiB
Raw Blame History

Mosaic Stack Remediation — Mission Charter

Owner: project orchestrator mos-remediation (Claude, launched in /src/mosaic-stack). Origin: 2026-07-16..31 fleet lifecycle postmortem. Status: PLANNING (task decomposition). HOLD lifted for this workstream by Jason, 2026-07-31 — "begin full mosaic fleet operation on this."

Goal

Convert the 15 accepted postmortem remediation proposals into a working, dogfooded implementation. North star: anything with a deterministic right answer moves OUT of the LLM into a deterministic gate/program; the LLM handles only genuine judgment.

First-class principle — observe the property, not the exit code

No write is done until the requested PROPERTY is observed. A success exit code is not evidence.

Success output is designed to be believed. That is the whole reason the inert-gate class exists and why P-WRAPPER-001's tri-state (verified / written-unverified / failed) is not optional. The failure is not carelessness — a green is engineered to be trusted, so trusting it is the default behaviour of a competent operator, not a lapse.

Promoted to the charter by Mos (2026-07-31) after the orchestrator committed this exact error: a --draft flag was silently dropped by a wrapper fallback that still exited 0, and the PR was reported as a draft on the strength of the exit code rather than an observed draft: true (D-12). Twelve failure instances were banked in that session; three of them were the orchestrator's own. That ratio is the point — the mechanism must catch the mechanic too, or it is not a mechanism.

Operationally: after any write, read back the property you required. Applies to gates, wrappers, PR flags, commit authorship, file installs, and message delivery alike.

Decision record (authoritative, immutable)

  • 15/15 proposals decided: 13 accept, 2 modify (P-AUTHORITY-001, P-INBOX-001), 0 reject.
  • Site + annotations.json: jarvis-brain/docs/postmortem-spec/site/ (committed, origin/main).
  • Discussion checkpoint (rich rationale per proposal): jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md.
  • Postmortem report: mosaicstack/stack PR #107 (merged 88f4ee04).
  • MACP wiring scout (verdict c=STRANDED): /tmp/macp-wiring-investigation.md (copy into this dir — see TODO).

The plan — 15 proposals collapse to 4 builds + hygiene

Build Absorbs What it is
1. One choke-point service (mechanical enforcer) MISSION, STATE, AUDIT, WRAPPER, QUEUE Deterministic program every task/data mutation flows through. Wire the stranded @mosaicstack/macp in at mosaic_orchestrator.py::run_single_task — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes.
2. One durable spine + hot path (storage under everything) PG system-of-record + Redis hot queue (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here.
3. Rotation lifecycle (finish the Mission Control Plane) LIFECYCLE, CONTRACT, GUIDE, RECOVERY Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse packages/coord; existing PRD at docs/mission-control/.
4. Comms service AUTHORITY, INBOX (+ versioning roadmap) Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants.
+ Hygiene & proof FLEET, WORKFLOW, CONFORMANCE One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 14 hold.

The finding that sets the cost

Built-but-unwired disease. @mosaicstack/macp is stranded (nothing calls it); packages/coord primitives exist; the Mission Control PRD exists; PG + Redis already run in-stack. Three duplicate MACP islands, an orphaned context loader, a fail-open bypass. Work = wire + consolidate + retire, NOT greenfield. "Finish, don't re-spec."

Sequencing (skeleton — adversarial decomposition refines this)

  1. Spine + choke-point service (builds 1+2) — foundation; unlocks MISSION/STATE/AUDIT/WRAPPER/QUEUE at one integration point.
  2. Rotation daemon (build 3) on that spine — the drift fix proper.
  3. Comms service (build 4) — envelope → service → PG/Redis → adapters; retire direct-tmux.
  4. Hygiene + conformance (build 5) — fleet convergence, allowlist sync, dogfood harness.
  • Cross-cutting retirements: flat-file orchestration tracking (hard cutover to DB), the 3 duplicate MACP islands, the silent MOSAIC BYPASS.

Standing directives (Jason, 2026-07-31)

  • Dogfooding: validate EACH fix against the live fleet failure that motivated it. Seed acceptance tests: Pi brick (RECOVERY), scout-bounce (INBOX/FLEET), gate-6 inert + #1019 recursion (QUEUE), identity drift (WRAPPER), auto-sync sweep (WORKFLOW), #1018 stale-consumed (INBOX). The fleet is its own test bed.
  • Orchestration tracking → DB, hard cutover ("rip off the bandaid"), NO flat-file interim. jarvis-brain PDA flat-files untouched. Current flat-file tracking runs as-is/unhardened until DB tracking is real, then one clean replace.

The 15 decisions (one-line; full rationale in the checkpoint)

  1. P-ACTIVATION-001 accept — transactional CLI+hooks+broker+version release; block launch on skew, fail-SAFE.
  2. P-AUTHORITY-001 MODIFY — structured authenticated inbox; envelope carries comms-PROTOCOL version; version the protocol not participants; N-version window.
  3. P-LIFECYCLE-001 accept — rotation not recursive compaction; pre-empt at token threshold; enforcer = deterministic coordinator; = finish Mission Control Plane.
  4. P-MISSION-001 accept — bind lanes to mission+task ledger; convention exists, ENFORCEMENT is the gap; mission+tasks → DB spine (hard cutover).
  5. P-QUEUE-001 accept — repair queue transport + exit-asserting non-null-case tests (gate-6 was INERT fleet-wide; #1019 fix recursed the same bug).
  6. P-STATE-001 accept — typed claims (source/confidence/TTL) not prose blob; MACP typed record; integrity fail-closed HMAC; don't fork a 4th island.
  7. P-AUDIT-001 accept — MACPEvent lifecycle ledger; EXTEND enum to lifecycle events; runtime-neutral (executor-emitted); retire duplicate Python ledger.
  8. P-WRAPPER-001 accept — identity derives from seat name + survives respawn; tri-state write outcomes MANDATORY; name safe target metadata.
  9. P-CONTRACT-001 accept — bind session to contract hash; re-anchor on policy-change OR compaction-detected; stale generation loses authority MECHANICALLY.
  10. P-INBOX-001 MODIFY — sole-path comms SERVICE; PG durable SoR + Redis hot queue (outbox, reconciliation sweeper); pluggable adapters; protocol-first, PG-first-then-Redis.
  11. P-RECOVERY-001 accept — broker-independent bootstrap recovery; honest capability labeling; break-glass LOUD+AUDITED+TEMPORARY not silent permanent bypass.
  12. P-GUIDE-001 accept — delete /compact and continue from orchestrator path (keep for ephemeral); removal = substitution (wire rotation trigger).
  13. P-FLEET-001 accept — one roster-owned socket/host; quarantine unmanaged; stale-session GC; prerequisite for INBOX identity-addressing.
  14. P-WORKFLOW-001 accept — auto-sync ALLOWLIST not denylist; worktree/lease isolation for agent docs/source; DB-tracking obviates the flat-file-sweep criterion.
  15. P-CONFORMANCE-001 accept — fleet lifecycle harness on REAL runtime artifacts + fault injection; the 100-rotations-lossless bar is a test; target the DB substrate.

Fleet operating model

  • Project orchestrator mos-remediation (this seat) owns the mission; coordinates under Mos (lead).
  • Adversarial task decomposition: planner-opus (robustness) + planner-sol (pragmatic) each decompose the plan independently; orchestrator reconciles into TASKS.md/DB tasks. Oppositional by design.
  • Delivery gates (non-negotiable): author≠reviewer, PRE-REGISTERED diff-blind acceptance checks committed before reading the diff, CI terminal-green, completion = merged PR + closed issue. rev-974 = mosaicstack reviewer.
  • Compaction survival: see KICKSTART.md in this dir — the resume procedure. Persist typed state, not transcript.