175 lines
18 KiB
Markdown
175 lines
18 KiB
Markdown
# Mosaic Stack Remediation — Mission Charter
|
||
|
||
**Owner:** project orchestrator `mos-remediation` (Claude, launched in `/src/mosaic-stack`).
|
||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** EXECUTING (planning complete; RM-01 in flight).
|
||
**HOLD lifted** for this workstream by Jason, 2026-07-31 — "begin full mosaic fleet operation on this."
|
||
|
||
## Goal
|
||
|
||
Convert the 15 accepted postmortem remediation proposals into a working, **dogfooded** implementation.
|
||
**North star:** anything with a deterministic right answer moves OUT of the LLM into a deterministic
|
||
gate/program; the LLM handles only genuine judgment.
|
||
|
||
### First-class principle — observe the property, not the exit code
|
||
|
||
<!-- GATE-CLAIM:OBSERVE-PROPERTY -->
|
||
|
||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||
>
|
||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||
> and why P-WRAPPER-001's tri-state (`verified` / `written-unverified` / `failed`) is not optional. The
|
||
> failure is not carelessness — a green is _engineered_ to be trusted, so trusting it is the default
|
||
> behaviour of a competent operator, not a lapse.
|
||
>
|
||
> Promoted to the charter by Mos (2026-07-31) after the orchestrator committed this exact error: a
|
||
> `--draft` flag was silently dropped by a wrapper fallback that still exited 0, and the PR was reported
|
||
> as a draft on the strength of the exit code rather than an observed `draft: true` (D-12). Twelve
|
||
> failure instances were banked in that session; **three of them were the orchestrator's own.** That
|
||
> ratio is the point — the mechanism must catch the mechanic too, or it is not a mechanism.
|
||
>
|
||
> Operationally: after any write, read back the property you required. Applies to gates, wrappers, PR
|
||
> flags, commit authorship, file installs, and message delivery alike.
|
||
|
||
### First-class principle — pre-registration prevents retrofitting, and nothing else
|
||
|
||
<!-- GATE-CLAIM:PREREGISTRATION-BOUNDARY -->
|
||
|
||
> **A pre-registered check set can fail in three distinct ways:**
|
||
>
|
||
> | mode | the set is… | found as |
|
||
> | --------------------------- | ------------------------------------------------- | -------- |
|
||
> | **WRONG** | a check does not test what it claims | D-8 |
|
||
> | **INCOMPLETE** | green while a criterion's requirement is untested | D-17 |
|
||
> | **INTERNALLY INCONSISTENT** | two criteria cannot both hold | D-18 |
|
||
>
|
||
> **Pre-registration protects against exactly one thing: retrofitting a check to fit the implementation
|
||
> it is supposed to judge.** It confers neither correctness, nor coverage, nor consistency. "We
|
||
> pre-registered the checks" has been treated as though it settled the question — it settles one of
|
||
> three.
|
||
>
|
||
> Promoted to the charter by Mos (2026-07-31). All three modes were found on this mission's own **first
|
||
> delivery**, by the machinery applied to its own work — not by inspection, and not by looking for them.
|
||
>
|
||
> **Enforceable form — RM-02's four clauses.** The registry must establish that: (1) each check is
|
||
> **right** — proven red for its own stated reason before its green counts; (2) the set **covers** —
|
||
> every criterion bound to a case that actually exercises it; (3) no two criteria **conflict** —
|
||
> mutual unsatisfiability is a registry defect discoverable by construction; (4) when a criterion's
|
||
> meaning changes, the registry **retains original text, restatement, and reason**, so evolution stays
|
||
> auditable. A criterion with no case that can fail for its own reason is unregistered in substance,
|
||
> however it reads in the manifest.
|
||
|
||
### Corollary — never ship an integrity claim dressed as a property
|
||
|
||
<!-- GATE-CLAIM:ARTIFACT-INTEGRITY-BOUNDARY -->
|
||
|
||
> A verification artifact that can be forged by whoever it is meant to catch verifies nothing. If a
|
||
> manifest, marker, ledger, or receipt is writable by the same actor whose behaviour it certifies, it
|
||
> **certifies the attack.** Such an artifact must sit inside the integrity envelope it belongs to,
|
||
> publish atomically, and carry a **tamper negative-control observed red** — otherwise its integrity is
|
||
> a _claim_, not a _property_.
|
||
>
|
||
> **If it cannot be made tamper-evident, say so and reconsider the approach.** Laundering foreign
|
||
> content as certified is the only unacceptable outcome; an honest "this cannot be verified" is always
|
||
> available and always preferable.
|
||
|
||
### First-class principle — when a property cannot exist at the layer it was specified
|
||
|
||
<!-- GATE-CLAIM:IMPOSSIBLE-LAYER-BOUNDARY -->
|
||
|
||
> Some required properties are **impossible at the layer that asked for them** — not hard, impossible.
|
||
> A local check cannot defend against an actor who can rewrite the check itself. When that happens,
|
||
> there are exactly three honest moves, and all three are mandatory:
|
||
>
|
||
> 1. **Implement what the layer _can_ guarantee.** Partial protection against the class it was actually
|
||
> born from is worth having.
|
||
> 2. **State the boundary precisely, in BOTH directions.** What it does _not_ defend, **and** beside it
|
||
> what it _does_. A reader who sees only the negative dismisses the check as worthless; one who sees
|
||
> only the positive over-trusts it. **Both together is the honest artifact** — either alone misleads.
|
||
> 3. **Record where the real guarantee will come from — as a TRACKED DEPENDENCY, not prose.** It must
|
||
> name a task that someone must close. _A documented gap with no owner becomes a permanent gap that
|
||
> reads as intentional._
|
||
>
|
||
> **A written-down gap is acceptable engineering. An implied-fixed gap is this mission's core failure in
|
||
> a new costume** — a verification artifact that verifies nothing, with a green to prove it.
|
||
>
|
||
> Promoted to the charter by Mos (2026-07-31) from D-19. Origin: the RM-01 symlink manifest could not be
|
||
> made tamper-evident against a same-UID actor (CWE-345), because the manifest and its marker share one
|
||
> writable tree. The implementing seat **escalated rather than relabelling self-authentication as
|
||
> tamper-resistance** — the corollary above firing on its first real adversarial test, on the cheapest
|
||
> seat in the loop. Residual risk bound to **RM-59** (`depends_on: RM-12, RM-21, RM-25`), where the
|
||
> choke-point executor and spine verify from _outside_ the worktree's authority.
|
||
|
||
## Decision record (authoritative, immutable)
|
||
|
||
- **15/15 proposals decided: 13 accept, 2 modify (P-AUTHORITY-001, P-INBOX-001), 0 reject.**
|
||
- Site + `annotations.json`: `jarvis-brain/docs/postmortem-spec/site/` (committed, origin/main).
|
||
- Discussion checkpoint (rich rationale per proposal): `jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md`.
|
||
- Postmortem report: mosaicstack/stack PR #107 (merged 88f4ee04).
|
||
- MACP wiring scout (verdict c=STRANDED): [`MACP-WIRING-SCOUT.md`](./MACP-WIRING-SCOUT.md) (copied into this dir; TODO discharged). Its findings are sound; its _recommended wire-in point_ is superseded by DECISION-1.
|
||
|
||
## The plan — 15 proposals collapse to 4 builds + hygiene
|
||
|
||
| Build | Absorbs | What it is |
|
||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. ⚠ **Target CORRECTED 2026-07-31 (DECISION-1, Mos):** a new production Node `TaskExecutor` on the **live** dispatch path (`packages/mosaic` launch + `packages/coord`), which Coord/Forge/live-dispatch submit through. **NOT** `mosaic_orchestrator.py::run_single_task` — that controller is `"enabled": false` and references a dispatcher absent from this checkout; wiring it would strand the executor, reproducing this mission's own disease. The Python rail is **deleted**, not ported. Both planners reached this independently. |
|
||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||
|
||
## The finding that sets the cost
|
||
|
||
**Built-but-unwired disease.** `@mosaicstack/macp` is stranded (nothing calls it); `packages/coord` primitives
|
||
exist; the Mission Control PRD exists; PG + Redis already run in-stack. Three duplicate MACP islands, an
|
||
orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retire, NOT greenfield. "Finish, don't re-spec."**
|
||
|
||
## Sequencing (skeleton — adversarial decomposition refines this)
|
||
|
||
1. **Spine + choke-point service** (builds 1+2) — foundation; unlocks MISSION/STATE/AUDIT/WRAPPER/QUEUE at one integration point.
|
||
(Per DECISION-1, a P0 phase of provable-gate + activation work precedes this; see `TASKS.md` §3.)
|
||
2. **Rotation daemon** (build 3) on that spine — the drift fix proper.
|
||
3. **Comms service** (build 4) — envelope → service → PG/Redis → adapters; retire direct-tmux.
|
||
4. **Hygiene + conformance** (build 5) — fleet convergence, allowlist sync, dogfood harness.
|
||
|
||
- **Cross-cutting retirements:** flat-file orchestration tracking (hard cutover to DB), the 3 duplicate MACP islands, the silent `MOSAIC BYPASS`.
|
||
|
||
## Standing directives (Jason, 2026-07-31)
|
||
|
||
- **Dogfooding:** validate EACH fix against the live fleet failure that motivated it. Seed acceptance tests:
|
||
Pi brick (RECOVERY), scout-bounce (INBOX/FLEET), gate-6 inert + #1019 recursion (QUEUE), identity drift
|
||
(WRAPPER), auto-sync sweep (WORKFLOW), #1018 stale-consumed (INBOX). The fleet is its own test bed.
|
||
- **Orchestration tracking → DB**, hard cutover ("rip off the bandaid"), NO flat-file interim. jarvis-brain
|
||
PDA flat-files untouched. Current flat-file tracking runs as-is/unhardened until DB tracking is real, then one clean replace.
|
||
- ⚠ **QUALIFIED 2026-07-31 (DECISION-2, Mos):** the DB spine **must NOT be a single-point hard-stop.**
|
||
A broker-independent / degraded mode **and** a rehearsed rollback artifact are **design requirements**
|
||
(P-RECOVERY-001), binding now on RM-12, RM-13, RM-23, RM-36 and RM-53. This **supersedes** the earlier
|
||
orchestrator recommendation to pre-commit "no DB ⇒ the fleet stops" — that answer is _not_ on record.
|
||
Only the specific availability _target_ remains open, queued for Jason; it does **not** block current work.
|
||
|
||
## The 15 decisions (one-line; full rationale in the checkpoint)
|
||
|
||
1. **P-ACTIVATION-001** accept — transactional CLI+hooks+broker+version release; block launch on skew, fail-SAFE.
|
||
2. **P-AUTHORITY-001** MODIFY — structured authenticated inbox; envelope carries comms-PROTOCOL version; version the protocol not participants; N-version window.
|
||
3. **P-LIFECYCLE-001** accept — rotation not recursive compaction; pre-empt at token threshold; enforcer = deterministic coordinator; = finish Mission Control Plane.
|
||
4. **P-MISSION-001** accept — bind lanes to mission+task ledger; convention exists, ENFORCEMENT is the gap; mission+tasks → DB spine (hard cutover).
|
||
5. **P-QUEUE-001** accept — repair queue transport + exit-asserting non-null-case tests (gate-6 was INERT fleet-wide; #1019 fix recursed the same bug).
|
||
6. **P-STATE-001** accept — typed claims (source/confidence/TTL) not prose blob; MACP typed record; integrity fail-closed HMAC; don't fork a 4th island.
|
||
7. **P-AUDIT-001** accept — MACPEvent lifecycle ledger; EXTEND enum to lifecycle events; runtime-neutral (executor-emitted); retire duplicate Python ledger.
|
||
8. **P-WRAPPER-001** accept — identity derives from seat name + survives respawn; tri-state write outcomes MANDATORY; name safe target metadata.
|
||
9. **P-CONTRACT-001** accept — bind session to contract hash; re-anchor on policy-change OR compaction-detected; stale generation loses authority MECHANICALLY.
|
||
10. **P-INBOX-001** MODIFY — sole-path comms SERVICE; PG durable SoR + Redis hot queue (outbox, reconciliation sweeper); pluggable adapters; protocol-first, PG-first-then-Redis.
|
||
11. **P-RECOVERY-001** accept — broker-independent bootstrap recovery; honest capability labeling; break-glass LOUD+AUDITED+TEMPORARY not silent permanent bypass.
|
||
12. **P-GUIDE-001** accept — delete `/compact and continue` from orchestrator path (keep for ephemeral); removal = substitution (wire rotation trigger).
|
||
13. **P-FLEET-001** accept — one roster-owned socket/host; quarantine unmanaged; stale-session GC; prerequisite for INBOX identity-addressing.
|
||
14. **P-WORKFLOW-001** accept — auto-sync ALLOWLIST not denylist; worktree/lease isolation for agent docs/source; DB-tracking obviates the flat-file-sweep criterion.
|
||
15. **P-CONFORMANCE-001** accept — fleet lifecycle harness on REAL runtime artifacts + fault injection; the 100-rotations-lossless bar is a test; target the DB substrate.
|
||
|
||
## Fleet operating model
|
||
|
||
- **Project orchestrator** `mos-remediation` (this seat) owns the mission; coordinates under Mos (lead).
|
||
- **Adversarial task decomposition:** `planner-opus` (robustness) + `planner-sol` (pragmatic) each decompose
|
||
the plan independently; orchestrator reconciles into `TASKS.md`/DB tasks. Oppositional by design.
|
||
- **Delivery gates (non-negotiable):** author≠reviewer, PRE-REGISTERED diff-blind acceptance checks committed
|
||
before reading the diff, CI terminal-green, completion = merged PR + closed issue. rev-974 = mosaicstack reviewer.
|
||
- **Compaction survival:** see `KICKSTART.md` in this dir — the resume procedure. Persist typed state, not transcript.
|