stack/docs/design/791-upgrade-config-protection.md

Design — #791: Framework upgrades must not destroy operator-owned config under ~/.config/mosaic

• Issue: Framework upgrades must not destroy operator-owned config under ~/.config/mosaic (mosaicstack/stack#791)
• Branch: feat/791-upgrade-config-protection (off origin/main 9745bc3f)
• Author: ms-791 worker lane
• Status: Phase 1 — DESIGN, awaiting MS-LEAD confirmation before implementation
• Ratified scope (Mos-approved, not re-litigated): deliver (b) strict ownership separation [PRIMARY] + (a) transactional pre-update snapshot [safety net] + (d) regeneration-from-SSOT [recovery]. (c) periodic backup timer is DEFERRED — noted as future work only.

---

1. Current updater behavior + exact wipe mechanism (evidence)

1.1 What runs on mosaic update

mosaic update re-seeds the framework by invoking the bash installer in sync-only, keep mode:

• packages/mosaic/src/runtime/update-checker.ts:509 buildReseedCommand() returns bash <frameworkRoot>/install.sh with env MOSAIC_SYNC_ONLY=1, MOSAIC_INSTALL_MODE=keep, MOSAIC_HOME=<mosaicHome>.
• The same install.sh is the direct/tools/install.sh upgrade path and the framework-vN migration path.

So the destructive surface is packages/mosaic/framework/install.sh.

1.2 The wipe

sync_framework() (install.sh:177) performs, in keep mode:

rsync -a --delete --exclude .git --exclude .framework-version --exclude '*.pre-constitution.bak' \
      [--exclude "/$path" for each PRESERVE_PATHS entry] SOURCE_DIR/ TARGET_DIR/

• install.sh:199 — rsync -a --delete. --delete prunes every path in ~/.config/mosaic that is NOT present in the shipped framework source, unless excluded.
• install.sh:47 — PRESERVE_PATHS is the only thing standing between --delete and operator data. It is a denylist of exclusions:

  PRESERVE_PATHS=("CONSTITUTION.md" "AGENTS.md" "SOUL.md" "USER.md" "TOOLS.md" "STANDARDS.md"
    "memory" "sources" "credentials" "fleet/roster.yaml" "fleet/roster.json" "fleet/agents"
    "fleet/run" "fleet/backlog" "fleet/roles.local")
  

• The cp-fallback (no rsync) is equally destructive: install.sh:223 find "$TARGET_DIR" -mindepth 1 -maxdepth 1 ... -exec rm -rf {} + then re-copies source, restoring only PRESERVE_PATHS globs.

Root-cause model: "Everything under ~/.config/mosaic is framework-owned and pruneable UNLESS explicitly preserved." Any operator path the list forgets is destroyed on the next upgrade.

1.3 The exact operator paths wiped

Cross-referencing the issue's operator-owned list against PRESERVE_PATHS:

Operator path (issue #791)	In PRESERVE_PATHS?	Fate on mosaic update
agents/*.conf (per-agent runtime)	NO	WIPED
policy/*.md (operator overlays)	NO	WIPED
*.local.md (SOUL/USER/STANDARDS)	NO	WIPED
harvester / SOP artifacts + timers	NO	WIPED
tools/_lib/credentials.json	NO (credentials/ dir ≠ this path)	WIPED
fleet/agents/*.env	yes (fleet/agents, added by #631)	survives
memory/, fleet/roster.*, fleet/backlog, fleet/roles.local	yes	survives

The fleet/agents, memory, fleet/backlog entries were retro-added after prior incidents (#631). This whack-a-mole is the structural signature of a denylist.

Stale-comment evidence: update-checker.ts:492 claims the reseed preserves "SOUL/USER/*.local/credentials" — but PRESERVE_PATHS contains no *.local entry. The code documents protection it does not deliver.

1.4 Second code path (TS) — already non-destructive, but drifted

FileConfigAdapter.syncFramework() (packages/mosaic/src/config/file-adapter.ts:157) → syncDirectory() (packages/mosaic/src/platform/file-ops.ts:66) is a copy-overlay: it copies source over target and skips preserved paths, but never deletes target paths absent from source (file-ops.ts:77-109). It is used by the wizard/init flow, not mosaic update.

Two problems remain:

1. Its preservePaths (file-adapter.ts:164-185) has already diverged from install.sh — it is missing fleet/backlog and fleet/roles.local. Two hand-maintained denylists, drifted. This is direct evidence for a single shared SSOT manifest.
2. Even non-destructive, it will happily overwrite an operator file that collides with a framework-shipped path unless that path is on its (incomplete) preserve list.

1.5 Existing snapshot is inadequate for rollback

make_snapshot()/restore_snapshot() (install.sh:76-87) copy TARGET_DIR to mktemp -d under /tmp, restore only on ERR/INT/TERM trap, and are deleted on success (cleanup_snapshot, install.sh:345). Consequences: ephemeral /tmp, no retention, no post-success rollback, and no mosaic restore. It is crash-safety only, not the transactional safety net #791 requires.

---

2. Fix (b) — Strict ownership separation [PRIMARY / root cause]

2.1 Ownership model (invert to allow-list)

Replace "framework-owned unless preserved" with "operator-owned unless framework-owned", resolved per target path with operator carve-outs winning inside shared framework subtrees.

Two declared lists, one SSOT data file shipped in the framework (framework/framework-manifest.json), consumed by both bash and TS:

• framework globs — paths the updater is entitled to create / overwrite / prune. Authored to match exactly what the framework ships in packages/mosaic/framework/ (e.g. CONSTITUTION.md, AGENTS.md, STANDARDS.md, TOOLS.md, guides/**, constitution/**, templates/**, tools/**, skills/**, mcp/**, defaults/**, fleet/examples/**, fleet/roles/**, fleet/profiles/**, fleet/roster.schema.json).
• operatorReserved globs — NEVER written or pruned, even nested inside a framework subtree; these win over framework (deny-wins / most-specific-wins). At minimum: agents/**, policy/**, memory/**, sources/**, credentials/**, *.local.md, tools/_lib/credentials.json, fleet/roster.yaml, fleet/roster.json, fleet/agents/**, fleet/run/**, fleet/backlog/**, fleet/roles.local/**, plus operator harvester/SOP artifacts.

2.2 Ownership resolution for a target path P

1. P matches operatorReserved → operator-owned: updater MUST NOT write, MUST NOT delete.
2. else P matches framework → framework-owned: may overwrite; may prune only if absent from the current SOURCE (a genuinely retired framework file).
3. else (matches neither) → UNKNOWN ⇒ operator-owned by default (fail-safe): never delete.

Rule 3 is the actual root-cause fix: an operator path the manifest authors forget is still protected, because unknown defaults to operator. A denylist can never provide this guarantee.

2.3 Sync mechanism change (the mechanically-critical part)

--delete cannot express "prune only framework-owned" without re-enumerating every operator path (the denylist trap). So:

1. Drop --delete from the bulk sync. Copy SOURCE → TARGET non-destructively (writes/overwrites all framework files; deletes nothing). rsync without --delete, or the existing overlay copy.
2. Explicit manifest-scoped prune pass. Iterate the framework manifest (not the whole tree); for each framework path present in TARGET but absent in SOURCE, delete it — after re-checking it does not match operatorReserved. Because the prune iterates only declared framework globs, operator/unknown paths are structurally unreachable by deletion.

This is implemented in both bash sync_framework() and TS syncFramework() from the shared manifest. A pure prune-planner function (TS) computes the delete-set from (manifest, sourceListing, targetListing) so the invariant is unit-testable in isolation. PRESERVE_PATHS becomes redundant (kept as a defense-in-depth alias mapping to operatorReserved, or removed) — either way the two lists stop drifting because they read one file.

2.4 HARD GATE test — "upgrade touches no path outside the manifest"

Filesystem-observation test in the existing test-install-migration.sh harness pattern (mktemp MOSAIC_HOME, MOSAIC_SYNC_ONLY=1), plus TS specs:

1. Seed a throwaway TARGET with a realistic operator mix — one sentinel per operator class: agents/x.conf, policy/p.md, SOUL.local.md, memory/m.md, tools/_lib/credentials.json (with a secret value), fleet/agents/a.env, fleet/roster.yaml, harvester/sop.md, and a deliberately-unanticipated unknown-operator-dir/x.
2. Record hash+mtime of every sentinel.
3. Run the upgrade from a SOURCE containing none of those operator paths.
4. Assert: every sentinel exists, byte-identical, mtime unchanged (not even rewritten). The unknown-operator-dir surviving proves the fail-safe default — a denylist could not pass this case.
5. Positive controls: framework files WERE updated; a retired framework file WAS pruned.
6. Property test (TS prune-planner): for fuzzed operator paths, deleteSet ⊆ {matches framework ∧ in target ∧ not in source} and deleteSet ∩ operatorReserved = ∅.

---

3. Fix (a) — Transactional pre-update snapshot [safety net]

• Destination: ${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-ts>/. Outside ~/.config/mosaic (so no future sync can sweep it) and outside any repo.
• Perms: dir 0700, files 0600 — enforced with umask 077 around the copy and explicit chmod. Never world-readable.
• Scope: the operator-owned surface (operatorReserved paths that exist) — bounded; does not copy the framework tree.
• Timing: taken before ANY mutation in the upgrade flow.
• Post-sync verify + selective restore: after sync, diff the operator surface against the snapshot; since (b) should never touch operator paths, any diff means a manifest bug — restore the affected paths from the snapshot and warn loudly. This is precisely (a) catching a miss in (b).
• Retention: keep N most-recent (default 5; MOSAIC_BACKUP_RETENTION override); prune older.
• mosaic restore: --list (default, dry-run) enumerates snapshots by timestamp; --from <ts> restores that snapshot over the operator surface, confirmation-gated. Reports counts/paths only.
• Secret-safety: snapshot copy and restore never emit file contents; only paths/counts. Tests assert 0700/0600 and that no secret value appears in stdout/stderr.

---

4. Fix (d) — Regeneration-from-SSOT [recovery]

The incident's live blast radius: fleet/agents/*.env (systemd EnvironmentFile sources) gone → mosaic-agent@<name> boots unit defaults on restart (because EnvironmentFile=-... is absent-tolerant) → silent identity/runtime/workdir downgrade.

The SSOT for those .env files is the roster. The reconciler already separates a regenerate-projections-from-roster projection phase from lifecycle (packages/mosaic/src/fleet/fleet-reconciler.ts:93,234; env rendering in generated-env-boundary.ts:149-264).

mosaic fleet regen is therefore a thin recovery-framed wrapper over the existing projection phase — it does NOT reimplement fleet logic and does NOT preempt in-flight FCM cards (M4/M5):

• Regenerates derivable config (per-agent *.env.generated, unit files) from roster SSOT.
• Preview-first: dry-run default; --write to apply. Idempotent.
• Never restarts agents (the recovery order forbids restart-before-verify).
• Prints the runbook's next step (verify EnvironmentFile resolves, THEN restart).

Alternatively documentable as install.sh --relink per the issue; mosaic fleet regen is preferred because it reuses the merged reconciler plumbing.

---

5. Secret-safety approach (secrev surface)

• Snapshots/backups: 0700/0600, outside any repo, never world-readable. (§3)
• No secret value ever emitted to logs/stdout/stderr by snapshot, restore, sync, or regen — paths/counts only. Adversarial test: a secret value placed in tools/_lib/credentials.json must never appear in installer or command output.
• tools/_lib/credentials.json is an explicit operatorReserved carve-out inside the framework-owned tools/** subtree — it is never overwritten or pruned.
• The HARD GATE test doubles as a secret-safety test (asserts the credentials sentinel is untouched).

---

6. Test plan (TDD, tests-first, ≥85% on new code, co-located *.spec.ts)

1. Manifest SSOT parity — bash and TS resolve identical framework/operator sets from the one file; a test fails if either path hard-codes a divergent list.
2. Manifest completeness — every path shipped in framework/ is covered by a framework glob (so a new shipped file cannot silently fall outside the manifest and become un-prunable/undeclared).
3. HARD GATE — upgrade touches nothing outside the manifest, incl. the unanticipated-path case (§2.4).
4. Prune-planner unit + property tests (§2.4.6).
5. Snapshot — perms 0700/0600, correct destination, retention prune, secret value absent from output.
6. Restore — --list / --from round-trip restores operator surface byte-exact; confirmation gate; no secret leakage.
7. Regen — roster→env projection deterministic + idempotent; dry-run makes no writes; --write restores *.env; never issues a lifecycle/restart call.
8. Cross-path regression — TS syncFramework and bash install.sh agree on a shared fixture (closes the current #631-style drift).

Gates before every push: pnpm typecheck && pnpm lint && pnpm format:check + mosaic package tests green. Never --no-verify.

---

7. web1 recovery runbook (operator-agnostic; web1 specifics live in the issue as evidence only)

For a currently-wiped fleet EnvironmentFile state — do NOT service-restart while fleet/agents/*.env is absent (a restart boots unit defaults and silently downgrades identity):

1. Regenerate: mosaic fleet regen --write — rebuild ~/.config/mosaic/fleet/agents/*.env from roster SSOT.
2. Verify each unit resolves to the intended runtime/workdir before any restart: systemctl --user show mosaic-agent@<name> -p EnvironmentFile and confirm the generated env exists and carries the intended MOSAIC_AGENT_* runtime/workdir values.
3. Only then systemctl --user restart mosaic-agent@<name>, one unit at a time.

If config (not just fleet env) was lost, mosaic restore --list → mosaic restore --from <ts> before step 1.

---

8. Proposed PR split (reviewable; DAG-ordered)

PR	Scope	Depends	Review focus
PR1	PRIMARY — shared framework-manifest.json + ownership resolver + non-deleting sync + scoped prune (bash + TS) + HARD GATE + prune-planner tests	—	correctness (root fix)
PR2	Safety net — pre-update snapshot (~/.local/state, 0700/0600, retention) + post-sync verify/restore + mosaic restore	PR1	secrev (backup/secret)
PR3	Recovery — mosaic fleet regen (projection-only, preview-first, no restart) + docs (upgrade-safety + recovery runbook)	PR1	correctness + docs

Rationale: PR1 closes the failure class on its own; if PR2/PR3 slip, the class stays fixed. Each PR is one reviewable unit with its own tests ≥85%. Independent review (author≠reviewer) on all; secrev on PR2 (and PR1's secret-sentinel assertions).

9. Deferred (noted per scope)

(c) periodic backup timer — a systemd user timer snapshotting operator dirs on a cadence (defense-in-depth for non-upgrade losses). Explicitly out of scope now; future phase.

10. Constraints honored

• Framework-PR firewall: manifest + logic are operator-agnostic; no SOUL/USER/operator specifics in framework code; web1 details are issue evidence only.
• Capacity-fill: must not preempt M5-001 or #790; fleet regen reuses merged FCM-M3 plumbing and does not overlap FCM-M4/M5 migration cards.
• Delivery gates: TDD tests-first, ≥85% new-code coverage, trunk-based squash PRs, independent review + secrev, completion = merged PR + descendant-main green + #791 closed.

---

Requesting MS-LEAD confirmation of: (1) the manifest allow-list + non-deleting-sync + scoped-prune approach as the (b) root-cause fix; (2) snapshot destination/retention + mosaic restore UX; (3) mosaic fleet regen as a projection-only wrapper; (4) the 3-PR split. Implementation begins only on your confirmation.