docs(791): Phase-1 design for upgrade config protection

Design-only planning artifact for #791. Traces the wipe mechanism
(install.sh:199 rsync --delete gated by PRESERVE_PATHS denylist) and
specifies the ratified (b)+(a)+(d) fix: framework-owned manifest allow-list
with fail-safe unknown=>operator default and manifest-scoped prune, a
transactional pre-update snapshot with mosaic restore, and a projection-only
mosaic fleet regen recovery path. No implementation changes.

Refs #791

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Hermes Agent
2026-07-16 15:14:50 -05:00
parent 9745bc3f29
commit 87e21fd933
2 changed files with 334 additions and 0 deletions

View File

@@ -0,0 +1,288 @@
# Design — #791: Framework upgrades must not destroy operator-owned config under `~/.config/mosaic`
- **Issue:** mosaicstack/stack#791
- **Branch:** `feat/791-upgrade-config-protection` (off `origin/main` `9745bc3f`)
- **Author:** ms-791 worker lane
- **Status:** Phase 1 — DESIGN, awaiting MS-LEAD confirmation before implementation
- **Ratified scope (Mos-approved, not re-litigated):** deliver **(b) strict ownership separation [PRIMARY]** + **(a) transactional pre-update snapshot [safety net]** + **(d) regeneration-from-SSOT [recovery]**. **(c) periodic backup timer is DEFERRED** — noted as future work only.
---
## 1. Current updater behavior + exact wipe mechanism (evidence)
### 1.1 What runs on `mosaic update`
`mosaic update` re-seeds the framework by invoking the **bash installer** in sync-only, keep mode:
- `packages/mosaic/src/runtime/update-checker.ts:509` `buildReseedCommand()` returns
`bash <frameworkRoot>/install.sh` with env `MOSAIC_SYNC_ONLY=1`, `MOSAIC_INSTALL_MODE=keep`,
`MOSAIC_HOME=<mosaicHome>`.
- The same `install.sh` is the direct/`tools/install.sh` upgrade path and the framework-vN migration path.
So the destructive surface is **`packages/mosaic/framework/install.sh`**.
### 1.2 The wipe
`sync_framework()` (`install.sh:177`) performs, in `keep` mode:
```
rsync -a --delete --exclude .git --exclude .framework-version --exclude '*.pre-constitution.bak' \
[--exclude "/$path" for each PRESERVE_PATHS entry] SOURCE_DIR/ TARGET_DIR/
```
- `install.sh:199``rsync -a --delete`. **`--delete` prunes every path in `~/.config/mosaic`
that is NOT present in the shipped framework source**, unless excluded.
- `install.sh:47``PRESERVE_PATHS` is the **only** thing standing between `--delete` and operator
data. It is a *denylist of exclusions*:
```
PRESERVE_PATHS=("CONSTITUTION.md" "AGENTS.md" "SOUL.md" "USER.md" "TOOLS.md" "STANDARDS.md"
"memory" "sources" "credentials" "fleet/roster.yaml" "fleet/roster.json" "fleet/agents"
"fleet/run" "fleet/backlog" "fleet/roles.local")
```
- The cp-fallback (no rsync) is equally destructive: `install.sh:223`
`find "$TARGET_DIR" -mindepth 1 -maxdepth 1 ... -exec rm -rf {} +` then re-copies source, restoring
only PRESERVE_PATHS globs.
**Root-cause model:** *"Everything under `~/.config/mosaic` is framework-owned and pruneable UNLESS
explicitly preserved."* Any operator path the list forgets is destroyed on the next upgrade.
### 1.3 The exact operator paths wiped
Cross-referencing the issue's operator-owned list against `PRESERVE_PATHS`:
| Operator path (issue #791) | In PRESERVE_PATHS? | Fate on `mosaic update` |
| ------------------------------------- | ------------------ | ----------------------- |
| `agents/*.conf` (per-agent runtime) | **NO** | **WIPED** |
| `policy/*.md` (operator overlays) | **NO** | **WIPED** |
| `*.local.md` (SOUL/USER/STANDARDS) | **NO** | **WIPED** |
| harvester / SOP artifacts + timers | **NO** | **WIPED** |
| `tools/_lib/credentials.json` | **NO** (`credentials/` dir ≠ this path) | **WIPED** |
| `fleet/agents/*.env` | yes (`fleet/agents`, added by #631) | survives |
| `memory/`, `fleet/roster.*`, `fleet/backlog`, `fleet/roles.local` | yes | survives |
The `fleet/agents`, `memory`, `fleet/backlog` entries were **retro-added after prior incidents**
(#631). This whack-a-mole is the structural signature of a denylist.
**Stale-comment evidence:** `update-checker.ts:492` claims the reseed preserves
"`SOUL/USER/*.local/credentials`" — but `PRESERVE_PATHS` contains **no `*.local` entry**. The code
documents protection it does not deliver.
### 1.4 Second code path (TS) — already non-destructive, but drifted
`FileConfigAdapter.syncFramework()` (`packages/mosaic/src/config/file-adapter.ts:157`) →
`syncDirectory()` (`packages/mosaic/src/platform/file-ops.ts:66`) is a **copy-overlay**: it copies
source over target and skips preserved paths, but **never deletes** target paths absent from source
(`file-ops.ts:77-109`). It is used by the wizard/init flow, not `mosaic update`.
Two problems remain:
1. Its `preservePaths` (`file-adapter.ts:164-185`) has **already diverged** from `install.sh` — it is
**missing `fleet/backlog` and `fleet/roles.local`**. Two hand-maintained denylists, drifted. This
is direct evidence for a single shared SSOT manifest.
2. Even non-destructive, it will happily *overwrite* an operator file that collides with a
framework-shipped path unless that path is on its (incomplete) preserve list.
### 1.5 Existing snapshot is inadequate for rollback
`make_snapshot()`/`restore_snapshot()` (`install.sh:76-87`) copy `TARGET_DIR` to `mktemp -d` under
`/tmp`, restore **only on `ERR/INT/TERM` trap**, and are **deleted on success** (`cleanup_snapshot`,
`install.sh:345`). Consequences: ephemeral `/tmp`, no retention, no post-success rollback, and **no
`mosaic restore`**. It is crash-safety only, not the transactional safety net #791 requires.
---
## 2. Fix (b) — Strict ownership separation [PRIMARY / root cause]
### 2.1 Ownership model (invert to allow-list)
Replace *"framework-owned unless preserved"* with *"operator-owned unless framework-owned"*, resolved
**per target path** with operator carve-outs winning inside shared framework subtrees.
Two declared lists, one SSOT data file shipped in the framework
(`framework/framework-manifest.json`), consumed by **both** bash and TS:
- **`framework` globs** — paths the updater is entitled to create / overwrite / prune. Authored to
match exactly what the framework ships in `packages/mosaic/framework/` (e.g. `CONSTITUTION.md`,
`AGENTS.md`, `STANDARDS.md`, `TOOLS.md`, `guides/**`, `constitution/**`, `templates/**`, `tools/**`,
`skills/**`, `mcp/**`, `defaults/**`, `fleet/examples/**`, `fleet/roles/**`, `fleet/profiles/**`,
`fleet/roster.schema.json`).
- **`operatorReserved` globs** — NEVER written or pruned, even nested inside a `framework` subtree;
these **win** over `framework` (deny-wins / most-specific-wins). At minimum:
`agents/**`, `policy/**`, `memory/**`, `sources/**`, `credentials/**`, `*.local.md`,
`tools/_lib/credentials.json`, `fleet/roster.yaml`, `fleet/roster.json`, `fleet/agents/**`,
`fleet/run/**`, `fleet/backlog/**`, `fleet/roles.local/**`, plus operator harvester/SOP artifacts.
### 2.2 Ownership resolution for a target path `P`
1. `P` matches `operatorReserved` → **operator-owned**: updater MUST NOT write, MUST NOT delete.
2. else `P` matches `framework` → **framework-owned**: may overwrite; may prune **only if absent from
the current SOURCE** (a genuinely retired framework file).
3. else (matches neither) → **UNKNOWN ⇒ operator-owned by default (fail-safe)**: never delete.
Rule 3 is the actual root-cause fix: an operator path the manifest authors forget is still protected,
because *unknown defaults to operator*. A denylist can never provide this guarantee.
### 2.3 Sync mechanism change (the mechanically-critical part)
`--delete` cannot express "prune only framework-owned" without re-enumerating every operator path
(the denylist trap). So:
1. **Drop `--delete` from the bulk sync.** Copy `SOURCE → TARGET` non-destructively (writes/overwrites
all framework files; deletes nothing). rsync without `--delete`, or the existing overlay copy.
2. **Explicit manifest-scoped prune pass.** Iterate the **`framework` manifest** (not the whole tree);
for each framework path present in `TARGET` but **absent in `SOURCE`**, delete it — after
re-checking it does not match `operatorReserved`. Because the prune iterates only declared
framework globs, operator/unknown paths are **structurally unreachable** by deletion.
This is implemented in both bash `sync_framework()` and TS `syncFramework()` from the shared manifest.
A pure **prune-planner** function (TS) computes the delete-set from
`(manifest, sourceListing, targetListing)` so the invariant is unit-testable in isolation.
`PRESERVE_PATHS` becomes redundant (kept as a defense-in-depth alias mapping to `operatorReserved`, or
removed) — either way the two lists stop drifting because they read one file.
### 2.4 HARD GATE test — "upgrade touches no path outside the manifest"
Filesystem-observation test in the existing `test-install-migration.sh` harness pattern (mktemp
`MOSAIC_HOME`, `MOSAIC_SYNC_ONLY=1`), plus TS specs:
1. Seed a throwaway `TARGET` with a realistic operator mix — one sentinel per operator class:
`agents/x.conf`, `policy/p.md`, `SOUL.local.md`, `memory/m.md`,
`tools/_lib/credentials.json` (with a secret value), `fleet/agents/a.env`, `fleet/roster.yaml`,
`harvester/sop.md`, **and a deliberately-unanticipated `unknown-operator-dir/x`**.
2. Record hash+mtime of every sentinel.
3. Run the upgrade from a `SOURCE` containing none of those operator paths.
4. **Assert:** every sentinel exists, byte-identical, **mtime unchanged** (not even rewritten). The
`unknown-operator-dir` surviving proves the fail-safe default — a denylist could not pass this case.
5. **Positive controls:** framework files WERE updated; a retired framework file WAS pruned.
6. **Property test** (TS prune-planner): for fuzzed operator paths, `deleteSet ⊆ {matches framework ∧
in target ∧ not in source}` and `deleteSet ∩ operatorReserved = ∅`.
---
## 3. Fix (a) — Transactional pre-update snapshot [safety net]
- **Destination:** `${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-ts>/`.
**Outside `~/.config/mosaic`** (so no future sync can sweep it) and outside any repo.
- **Perms:** dir `0700`, files `0600` — enforced with `umask 077` around the copy **and** explicit
`chmod`. Never world-readable.
- **Scope:** the operator-owned surface (`operatorReserved` paths that exist) — bounded; does not copy
the framework tree.
- **Timing:** taken before ANY mutation in the upgrade flow.
- **Post-sync verify + selective restore:** after sync, diff the operator surface against the snapshot;
since (b) should never touch operator paths, any diff means a manifest bug — restore the affected
paths from the snapshot and warn loudly. This is precisely (a) catching a miss in (b).
- **Retention:** keep N most-recent (default 5; `MOSAIC_BACKUP_RETENTION` override); prune older.
- **`mosaic restore`:** `--list` (default, dry-run) enumerates snapshots by timestamp;
`--from <ts>` restores that snapshot over the operator surface, confirmation-gated. Reports
counts/paths only.
- **Secret-safety:** snapshot copy and restore never emit file **contents**; only paths/counts.
Tests assert `0700/0600` and that no secret value appears in stdout/stderr.
---
## 4. Fix (d) — Regeneration-from-SSOT [recovery]
The incident's live blast radius: `fleet/agents/*.env` (systemd `EnvironmentFile` sources) gone →
`mosaic-agent@<name>` boots **unit defaults** on restart (because `EnvironmentFile=-...` is
absent-tolerant) → **silent identity/runtime/workdir downgrade**.
The SSOT for those `.env` files is the roster. The reconciler **already** separates a
`regenerate-projections-from-roster` projection phase from lifecycle
(`packages/mosaic/src/fleet/fleet-reconciler.ts:93,234`; env rendering in
`generated-env-boundary.ts:149-264`).
**`mosaic fleet regen`** is therefore a **thin recovery-framed wrapper over the existing projection
phase** — it does NOT reimplement fleet logic and does NOT preempt in-flight FCM cards (M4/M5):
- Regenerates derivable config (per-agent `*.env.generated`, unit files) from roster SSOT.
- **Preview-first:** dry-run default; `--write` to apply. Idempotent.
- **Never restarts agents** (the recovery order forbids restart-before-verify).
- Prints the runbook's next step (verify `EnvironmentFile` resolves, THEN restart).
Alternatively documentable as `install.sh --relink` per the issue; `mosaic fleet regen` is preferred
because it reuses the merged reconciler plumbing.
---
## 5. Secret-safety approach (secrev surface)
- Snapshots/backups: `0700`/`0600`, outside any repo, never world-readable. (§3)
- No secret **value** ever emitted to logs/stdout/stderr by snapshot, restore, sync, or regen —
paths/counts only. Adversarial test: a secret value placed in `tools/_lib/credentials.json` must
never appear in installer or command output.
- `tools/_lib/credentials.json` is an explicit `operatorReserved` carve-out inside the framework-owned
`tools/**` subtree — it is never overwritten or pruned.
- The HARD GATE test doubles as a secret-safety test (asserts the credentials sentinel is untouched).
---
## 6. Test plan (TDD, tests-first, ≥85% on new code, co-located `*.spec.ts`)
1. **Manifest SSOT parity** — bash and TS resolve identical framework/operator sets from the one file;
a test fails if either path hard-codes a divergent list.
2. **Manifest completeness** — every path shipped in `framework/` is covered by a `framework` glob (so
a new shipped file cannot silently fall outside the manifest and become un-prunable/undeclared).
3. **HARD GATE** — upgrade touches nothing outside the manifest, incl. the unanticipated-path case
(§2.4).
4. **Prune-planner** unit + property tests (§2.4.6).
5. **Snapshot** — perms `0700/0600`, correct destination, retention prune, secret value absent from
output.
6. **Restore** — `--list` / `--from` round-trip restores operator surface byte-exact; confirmation
gate; no secret leakage.
7. **Regen** — roster→env projection deterministic + idempotent; dry-run makes no writes; `--write`
restores `*.env`; **never** issues a lifecycle/restart call.
8. **Cross-path regression** — TS `syncFramework` and bash `install.sh` agree on a shared fixture
(closes the current #631-style drift).
Gates before every push: `pnpm typecheck && pnpm lint && pnpm format:check` + mosaic package tests
green. Never `--no-verify`.
---
## 7. web1 recovery runbook (operator-agnostic; web1 specifics live in the issue as evidence only)
For a currently-wiped fleet EnvironmentFile state — **do NOT service-restart while
`fleet/agents/*.env` is absent** (a restart boots unit defaults and silently downgrades identity):
1. **Regenerate:** `mosaic fleet regen --write` — rebuild `~/.config/mosaic/fleet/agents/*.env` from
roster SSOT.
2. **Verify each unit resolves to the intended runtime/workdir** *before* any restart:
`systemctl --user show mosaic-agent@<name> -p EnvironmentFile` and confirm the generated env exists
and carries the intended `MOSAIC_AGENT_*` runtime/workdir values.
3. **Only then** `systemctl --user restart mosaic-agent@<name>`, one unit at a time.
If config (not just fleet env) was lost, `mosaic restore --list` → `mosaic restore --from <ts>` before
step 1.
---
## 8. Proposed PR split (reviewable; DAG-ordered)
| PR | Scope | Depends | Review focus |
| --- | ------------------------------------------------------------------------------------------------------- | ------- | ----------------------- |
| PR1 | **PRIMARY** — shared `framework-manifest.json` + ownership resolver + non-deleting sync + scoped prune (bash + TS) + **HARD GATE** + prune-planner tests | — | correctness (root fix) |
| PR2 | **Safety net** — pre-update snapshot (`~/.local/state`, 0700/0600, retention) + post-sync verify/restore + `mosaic restore` | PR1 | **secrev** (backup/secret) |
| PR3 | **Recovery** — `mosaic fleet regen` (projection-only, preview-first, no restart) + docs (upgrade-safety + recovery runbook) | PR1 | correctness + docs |
Rationale: PR1 closes the failure class on its own; if PR2/PR3 slip, the class stays fixed. Each PR is
one reviewable unit with its own tests ≥85%. Independent review (author≠reviewer) on all; **secrev** on
PR2 (and PR1's secret-sentinel assertions).
## 9. Deferred (noted per scope)
**(c) periodic backup timer** — a systemd user timer snapshotting operator dirs on a cadence
(defense-in-depth for non-upgrade losses). Explicitly **out of scope now**; future phase.
## 10. Constraints honored
- **Framework-PR firewall:** manifest + logic are operator-agnostic; no SOUL/USER/operator specifics
in framework code; web1 details are issue evidence only.
- **Capacity-fill:** must not preempt M5-001 or #790; `fleet regen` reuses merged FCM-M3 plumbing and
does not overlap FCM-M4/M5 migration cards.
- **Delivery gates:** TDD tests-first, ≥85% new-code coverage, trunk-based squash PRs, independent
review + secrev, completion = merged PR + descendant-main green + #791 closed.
---
**Requesting MS-LEAD confirmation of:** (1) the manifest allow-list + non-deleting-sync + scoped-prune
approach as the (b) root-cause fix; (2) snapshot destination/retention + `mosaic restore` UX;
(3) `mosaic fleet regen` as a projection-only wrapper; (4) the 3-PR split. Implementation begins only
on your confirmation.

View File

@@ -0,0 +1,46 @@
# Scratchpad — #791 Upgrade config protection (ms-791 worker lane)
**Lane:** web1:ms-791 → reports to MS-LEAD (web1:mosaic-100). Do NOT contact Jason/Mos directly.
**Worktree:** `/home/hermes/agent-work/stack-agents-dir-791`, branch `feat/791-upgrade-config-protection`
off `origin/main` `9745bc3f` (verified exact head).
## Mission prompt (verbatim intent)
Protect operator-owned config under `~/.config/mosaic` from framework-upgrade wipes. Ratified
combination (Mos-approved, do NOT re-litigate): (b) strict ownership separation [PRIMARY] + (a)
transactional pre-update snapshot [safety net] + (d) regeneration-from-SSOT [recovery]. (c) periodic
timer DEFERRED. HARD GATE: unit test that an upgrade run touches NO path outside the manifest.
Design-first: write design doc, send to MS-LEAD, WAIT for confirmation before impl.
## Session 1 (2026-07-16) — Phase 1 design
### Evidence gathered (wipe mechanism, file/line)
- `mosaic update``update-checker.ts:509` `buildReseedCommand``bash install.sh`
(`MOSAIC_SYNC_ONLY=1`, `MOSAIC_INSTALL_MODE=keep`).
- Wipe = `packages/mosaic/framework/install.sh:199` `rsync -a --delete` + `PRESERVE_PATHS` denylist
(`install.sh:47`). cp-fallback `install.sh:223` `find ... -exec rm -rf`.
- Denylist gaps → WIPED: `agents/*.conf`, `policy/*.md`, `*.local.md`, harvester/SOP,
`tools/_lib/credentials.json`.
- Stale comment `update-checker.ts:492` claims `*.local` preserved — PRESERVE_PATHS has no such entry.
- TS path `file-adapter.ts:157``file-ops.ts:66` `syncDirectory` = non-destructive copy-overlay, BUT
its preserve list (`file-adapter.ts:164`) already DRIFTED from install.sh (missing `fleet/backlog`,
`fleet/roles.local`). Evidence for single shared manifest SSOT.
- Existing snapshot (`install.sh:76`) = /tmp, crash-trap only, deleted on success → inadequate; no
`mosaic restore`.
- `fleet-reconciler.ts:93,234` already has `regenerate-projections-from-roster` phase separate from
lifecycle → `mosaic fleet regen` = thin projection-only wrapper (no restart), no FCM-M4/M5 preemption.
### Design decisions
- **(b)** Invert to allow-list: shared `framework/framework-manifest.json` (framework globs +
operatorReserved carve-outs); resolve per-path, deny-wins; **UNKNOWN ⇒ operator (fail-safe)**.
Mechanism: drop `--delete`; non-deleting bulk copy + explicit manifest-scoped prune pass (iterate
framework globs only → operator/unknown structurally unreachable). Pure prune-planner fn for tests.
- **(a)** Snapshot to `~/.local/state/mosaic/backups/pre-update-<ts>/` 0700/0600, retention N=5,
post-sync verify+restore, `mosaic restore --list/--from`. No secret values in output.
- **(d)** `mosaic fleet regen` projection-only, preview-first, never restart.
- HARD GATE test includes a deliberately-unanticipated operator path to prove fail-safe default.
- **PR split:** PR1 manifest+guard (root fix, ships alone) → PR2 snapshot/restore (secrev) → PR3
regen+docs. PR2/PR3 depend on PR1.
### Status
Design doc written: `docs/design/791-upgrade-config-protection.md`. Sent to MS-LEAD. **WAITING for
confirmation before Phase 2 implementation.** No impl started.