Applied D-14's own interim rule immediately instead of waiting to be bitten again, and it caught a second instance within minutes: MISSION.md's standing directives still stated the DB hard-cutover with no mention of Mos's binding qualification that the spine must not be a single-point hard-stop (degraded mode + rollback artifact required). A seat reading the charter would have designed toward an availability posture the coordinator had explicitly rejected, with the superseded 'no DB means the fleet stops' recommendation nowhere contradicted. Two un-propagated rulings out of two that touched charter text. Without a mechanism, propagation failure is the default outcome, not an oversight — which is the argument for D-14 being a requirement rather than a discipline. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
104 lines
13 KiB
Markdown
104 lines
13 KiB
Markdown
# Mosaic Stack Remediation — Mission Charter
|
||
|
||
**Owner:** project orchestrator `mos-remediation` (Claude, launched in `/src/mosaic-stack`).
|
||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** EXECUTING (planning complete; RM-01 in flight).
|
||
**HOLD lifted** for this workstream by Jason, 2026-07-31 — "begin full mosaic fleet operation on this."
|
||
|
||
## Goal
|
||
|
||
Convert the 15 accepted postmortem remediation proposals into a working, **dogfooded** implementation.
|
||
**North star:** anything with a deterministic right answer moves OUT of the LLM into a deterministic
|
||
gate/program; the LLM handles only genuine judgment.
|
||
|
||
### First-class principle — observe the property, not the exit code
|
||
|
||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||
>
|
||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||
> and why P-WRAPPER-001's tri-state (`verified` / `written-unverified` / `failed`) is not optional. The
|
||
> failure is not carelessness — a green is _engineered_ to be trusted, so trusting it is the default
|
||
> behaviour of a competent operator, not a lapse.
|
||
>
|
||
> Promoted to the charter by Mos (2026-07-31) after the orchestrator committed this exact error: a
|
||
> `--draft` flag was silently dropped by a wrapper fallback that still exited 0, and the PR was reported
|
||
> as a draft on the strength of the exit code rather than an observed `draft: true` (D-12). Twelve
|
||
> failure instances were banked in that session; **three of them were the orchestrator's own.** That
|
||
> ratio is the point — the mechanism must catch the mechanic too, or it is not a mechanism.
|
||
>
|
||
> Operationally: after any write, read back the property you required. Applies to gates, wrappers, PR
|
||
> flags, commit authorship, file installs, and message delivery alike.
|
||
|
||
## Decision record (authoritative, immutable)
|
||
|
||
- **15/15 proposals decided: 13 accept, 2 modify (P-AUTHORITY-001, P-INBOX-001), 0 reject.**
|
||
- Site + `annotations.json`: `jarvis-brain/docs/postmortem-spec/site/` (committed, origin/main).
|
||
- Discussion checkpoint (rich rationale per proposal): `jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md`.
|
||
- Postmortem report: mosaicstack/stack PR #107 (merged 88f4ee04).
|
||
- MACP wiring scout (verdict c=STRANDED): [`MACP-WIRING-SCOUT.md`](./MACP-WIRING-SCOUT.md) (copied into this dir; TODO discharged). Its findings are sound; its _recommended wire-in point_ is superseded by DECISION-1.
|
||
|
||
## The plan — 15 proposals collapse to 4 builds + hygiene
|
||
|
||
| Build | Absorbs | What it is |
|
||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. ⚠ **Target CORRECTED 2026-07-31 (DECISION-1, Mos):** a new production Node `TaskExecutor` on the **live** dispatch path (`packages/mosaic` launch + `packages/coord`), which Coord/Forge/live-dispatch submit through. **NOT** `mosaic_orchestrator.py::run_single_task` — that controller is `"enabled": false` and references a dispatcher absent from this checkout; wiring it would strand the executor, reproducing this mission's own disease. The Python rail is **deleted**, not ported. Both planners reached this independently. |
|
||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||
|
||
## The finding that sets the cost
|
||
|
||
**Built-but-unwired disease.** `@mosaicstack/macp` is stranded (nothing calls it); `packages/coord` primitives
|
||
exist; the Mission Control PRD exists; PG + Redis already run in-stack. Three duplicate MACP islands, an
|
||
orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retire, NOT greenfield. "Finish, don't re-spec."**
|
||
|
||
## Sequencing (skeleton — adversarial decomposition refines this)
|
||
|
||
1. **Spine + choke-point service** (builds 1+2) — foundation; unlocks MISSION/STATE/AUDIT/WRAPPER/QUEUE at one integration point.
|
||
(Per DECISION-1, a P0 phase of provable-gate + activation work precedes this; see `TASKS.md` §3.)
|
||
2. **Rotation daemon** (build 3) on that spine — the drift fix proper.
|
||
3. **Comms service** (build 4) — envelope → service → PG/Redis → adapters; retire direct-tmux.
|
||
4. **Hygiene + conformance** (build 5) — fleet convergence, allowlist sync, dogfood harness.
|
||
|
||
- **Cross-cutting retirements:** flat-file orchestration tracking (hard cutover to DB), the 3 duplicate MACP islands, the silent `MOSAIC BYPASS`.
|
||
|
||
## Standing directives (Jason, 2026-07-31)
|
||
|
||
- **Dogfooding:** validate EACH fix against the live fleet failure that motivated it. Seed acceptance tests:
|
||
Pi brick (RECOVERY), scout-bounce (INBOX/FLEET), gate-6 inert + #1019 recursion (QUEUE), identity drift
|
||
(WRAPPER), auto-sync sweep (WORKFLOW), #1018 stale-consumed (INBOX). The fleet is its own test bed.
|
||
- **Orchestration tracking → DB**, hard cutover ("rip off the bandaid"), NO flat-file interim. jarvis-brain
|
||
PDA flat-files untouched. Current flat-file tracking runs as-is/unhardened until DB tracking is real, then one clean replace.
|
||
- ⚠ **QUALIFIED 2026-07-31 (DECISION-2, Mos):** the DB spine **must NOT be a single-point hard-stop.**
|
||
A broker-independent / degraded mode **and** a rehearsed rollback artifact are **design requirements**
|
||
(P-RECOVERY-001), binding now on RM-12, RM-13, RM-23, RM-36 and RM-53. This **supersedes** the earlier
|
||
orchestrator recommendation to pre-commit "no DB ⇒ the fleet stops" — that answer is _not_ on record.
|
||
Only the specific availability _target_ remains open, queued for Jason; it does **not** block current work.
|
||
|
||
## The 15 decisions (one-line; full rationale in the checkpoint)
|
||
|
||
1. **P-ACTIVATION-001** accept — transactional CLI+hooks+broker+version release; block launch on skew, fail-SAFE.
|
||
2. **P-AUTHORITY-001** MODIFY — structured authenticated inbox; envelope carries comms-PROTOCOL version; version the protocol not participants; N-version window.
|
||
3. **P-LIFECYCLE-001** accept — rotation not recursive compaction; pre-empt at token threshold; enforcer = deterministic coordinator; = finish Mission Control Plane.
|
||
4. **P-MISSION-001** accept — bind lanes to mission+task ledger; convention exists, ENFORCEMENT is the gap; mission+tasks → DB spine (hard cutover).
|
||
5. **P-QUEUE-001** accept — repair queue transport + exit-asserting non-null-case tests (gate-6 was INERT fleet-wide; #1019 fix recursed the same bug).
|
||
6. **P-STATE-001** accept — typed claims (source/confidence/TTL) not prose blob; MACP typed record; integrity fail-closed HMAC; don't fork a 4th island.
|
||
7. **P-AUDIT-001** accept — MACPEvent lifecycle ledger; EXTEND enum to lifecycle events; runtime-neutral (executor-emitted); retire duplicate Python ledger.
|
||
8. **P-WRAPPER-001** accept — identity derives from seat name + survives respawn; tri-state write outcomes MANDATORY; name safe target metadata.
|
||
9. **P-CONTRACT-001** accept — bind session to contract hash; re-anchor on policy-change OR compaction-detected; stale generation loses authority MECHANICALLY.
|
||
10. **P-INBOX-001** MODIFY — sole-path comms SERVICE; PG durable SoR + Redis hot queue (outbox, reconciliation sweeper); pluggable adapters; protocol-first, PG-first-then-Redis.
|
||
11. **P-RECOVERY-001** accept — broker-independent bootstrap recovery; honest capability labeling; break-glass LOUD+AUDITED+TEMPORARY not silent permanent bypass.
|
||
12. **P-GUIDE-001** accept — delete `/compact and continue` from orchestrator path (keep for ephemeral); removal = substitution (wire rotation trigger).
|
||
13. **P-FLEET-001** accept — one roster-owned socket/host; quarantine unmanaged; stale-session GC; prerequisite for INBOX identity-addressing.
|
||
14. **P-WORKFLOW-001** accept — auto-sync ALLOWLIST not denylist; worktree/lease isolation for agent docs/source; DB-tracking obviates the flat-file-sweep criterion.
|
||
15. **P-CONFORMANCE-001** accept — fleet lifecycle harness on REAL runtime artifacts + fault injection; the 100-rotations-lossless bar is a test; target the DB substrate.
|
||
|
||
## Fleet operating model
|
||
|
||
- **Project orchestrator** `mos-remediation` (this seat) owns the mission; coordinates under Mos (lead).
|
||
- **Adversarial task decomposition:** `planner-opus` (robustness) + `planner-sol` (pragmatic) each decompose
|
||
the plan independently; orchestrator reconciles into `TASKS.md`/DB tasks. Oppositional by design.
|
||
- **Delivery gates (non-negotiable):** author≠reviewer, PRE-REGISTERED diff-blind acceptance checks committed
|
||
before reading the diff, CI terminal-green, completion = merged PR + closed issue. rev-974 = mosaicstack reviewer.
|
||
- **Compaction survival:** see `KICKSTART.md` in this dir — the resume procedure. Persist typed state, not transcript.
|