stack/docs/scratchpads/771-kbn101-db-role-split.md

Scratchpad — KBN-101 DB runtime/migration role split (#771)

• Branch: docs/771-kbn101-db-role-split
• Base: main e9c4aa3
• Scope: planning/documentation only; authorized files are PRD, Native Kanban task/shared/index docs, sitemap, this scratchpad, and the new KBN-101 contract.
• Explicit exclusions: source/runtime/config/deployment/secret/migration/compose/CI/lock/package/KBN-100 branch edits; no production mutation.

Objective

Freeze an implementation-ready PostgreSQL role/connection split so the Gateway uses a non-owner runtime identity and only a dedicated migration phase uses an owner/migrator identity. Make real deployed-role certification—not synthetic role tests—a serial prerequisite of KBN-100 and KBN-105.

Intake and current-state evidence

• Mission MVP is active; W3 Native Kanban/SOT is planning-complete. The task state shows KBN-010 as the predecessor and KBN-100 as the current schema slice.
• Current branch started at e9c4aa3; .mosaic/orchestrator/{mission.json,session.lock} were already runtime-modified and remain untouched.
• packages/db/src/client.ts, migrate.ts, and drizzle.config.ts resolve one DATABASE_URL (with default fallback). packages/storage/src/adapters/postgres.ts calls runMigrations(this.url).
• apps/gateway/src/database/database.module.ts calls storageAdapter.migrate() at startup for PostgreSQL; this is the owner-runtime defect to remove in KBN-101 implementation.
• packages/config/src/mosaic-config.ts, installer wizard, local/federated compose, Portainer test stack, and .woodpecker/ci.yml currently expose one URL. PGlite has an existing explicit local migration path.
• Current KBN contract requires immutable events/checkpoints/artifacts/evidence, RESTRICT, and KBN-100 generated Drizzle consistency. It did not establish a deployable runtime identity split.

Frozen decisions

1. DATABASE_URL is the non-owner runtime URL; DATABASE_MIGRATION_URL is migration-only. Both are required in their respective PostgreSQL phases; PGlite is the explicit local exception; migration never falls back to runtime/default/config URL.
2. PostgreSQL Gateway runtime never auto-runs migration/DDL. Dedicated migrator uses pg_try_advisory_lock(hashtext('mosaic-schema-migration-v1')); replicas only check exact ordered Drizzle-ledger readiness and fail closed.
3. Roles are non-login mosaic_platform_database_owner, non-login mosaic_schema_owner, login/noinherit mosaic_migrator, non-login mosaic_runtime_capability, and login/inherit mosaic_runtime. Runtime inherits only its capability role with SET/ADMIN denied, has no owner/migrator membership, no unsafe attributes/ownership/DDL authority, and an explicit trusted search path.
4. Runtime gets mutable DML only as needed, but INSERT/SELECT only on task_events, artifacts, task_checkpoints, task_checkpoint_artifacts, and approval_decision_artifacts. KBN-100 still enforces RESTRICT/no-cascade.
5. Startup verifies effective role/ownership/attributes/inherited capability/TEMP/function-execute/ledger grants/search path/immutable denials/schema fingerprint without DSN exposure. It also requires authenticated CA/hostname-verified TLS. Stable sanitized errors and redaction rules are required.
6. N-1 retains single runtime URL only as a non-certified compatibility release; staged role provisioning/migration/runtime deployment then enforces the split. Rollback never injects migration URL into Gateway.
7. Vault target paths, rotation, deployment injection, CI, installer, compose, Portainer, and observability are separate one-card/one-PR handoffs. The migration-only file manifest includes packages/db/drizzle.config.ts; KBN-101 repairs the known PostgreSQL runner/journal ordering defect and proves a clean database applies every hash once before its foundation certificate. No application migration creates roles/passwords or hardcodes credentials.
8. KBN-101 foundation merges/certifies first. KBN-100 then rebases, restores Drizzle declaration/snapshot/journal consistency, and bounds procedural immutable-table grant/trigger/backfill work to its own slice. Because those immutable relations do not exist until KBN-100, KBN-101’s real deployed-role immutable-operation certificate follows KBN-100 and is the serial gate before KBN-105.

Assumptions

• standalone and federated are all current PostgreSQL production-like modes; a future PostgreSQL tier inherits this contract unless versioned otherwise.
• Deployment will support a dedicated migration Job/one-shot command. A target that cannot run it cannot receive production/federated KBN certification.
• Canonical Vault target paths require deployment-owner verification before provisioning; the planning document does not claim they already exist.

Documentation produced

• docs/PRD.md: bounded KBN-101 requirements and acceptance criteria.
• docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md: normative rc.5 implementation, threat, migration/rollback, evidence, and exact file DAG contract.
• docs/native-kanban-sot/SHARED-CONTRACT.md: rc.5 amendment preserving rc.4.
• docs/native-kanban-sot/TASKS.md: KBN-101 inserted before and blocks KBN-100; KBN-105 held.
• Native Kanban index and root sitemap links.

Validation plan

1. Prettier only for changed Markdown.
2. Markdown link target/check checks scoped to modified docs.
3. Strict contract check with pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json (the frozen TypeScript contracts remain unchanged).
4. Diff allowlist proves only authorized documentation files changed, apart from pre-existing Mosaic runtime state.
5. Independent documentation/security self-review: role escalation, fallback, startup DDL, schema readiness, grants/default privileges, immutable tables, secret leakage, deployment and KBN-100 boundaries.

Review corrections

Independent Codex review found two blockers and security review found two medium defects; all were remediated in the frozen contract:

1. Split the KBN-101 certificate into a foundation role/schema-boundary certificate (before KBN-100) and real immutable-operation certificate (after KBN-100, before KBN-105). This preserves the requested KBN-100 block without requiring evidence for tables not yet created.
2. Removed the legacy owner-runtime exception. N-1 compatibility preserves variable/config shape only; the KBN-101 runtime refuses owner/migrator identity and current single-URL installs remain on their previous release until role cutover.
3. Introduced mosaic_platform_database_owner as a separate non-login platform role. mosaic_schema_owner owns application/ledger schemas only, not the database and has no database CREATE/ALTER/extension authority.
4. Replaced blocking pg_advisory_lock with pg_try_advisory_lock and the deterministic DATABASE_MIGRATION_LOCKED failure.
5. Review also flagged active .mosaic/orchestrator state. It was pre-existing launcher state and remains unstaged/uncommitted.
6. Second review added packages/db/drizzle.config.ts to the migration-only slice, mandates DATABASE_MIGRATION_URL with a missing-variable negative, grants runtime only USAGE plus SELECT on drizzle.__drizzle_migrations, and verifies/revokes its ledger writes.
7. Security review added DATABASE_TLS_CA_CERT_PATH / DatabaseTlsConfigDto with authenticated TLS and hostname/CA verification in production-like modes, explicit database TEMPORARY revocation/catalog denial testing, and default-PUBLIC function EXECUTE revocation with SECURITY DEFINER prohibited by default.
8. Final review corrected the runtime login to inherit only its capability role with SET/ADMIN denied, and moved the known hash-complete migration-runner/journal repair into KBN-101-03 before the foundation certificate.
9. Final manifest review added all live runtime DDL paths (packages/storage/src/tier-detection.ts, Gateway startup, and fleet-backlog) to KBN-101-02, requiring read-only extension probes and no PostgreSQL runtime auto-migration. It also requires KBN-101-04 to stop persisting either DSN into generated .env/mosaic.config.json, using only Vault/deployment references and injected variables.
10. Provisioning review separated the external privileged platform bootstrap actor from the NOCREATEDB database-owner role and added KBN-101-00. That IaC/bootstrap card owns fresh/existing database role/ownership/grant/Vault transition evidence and is a foundation-certificate dependency.

Results

• pnpm exec prettier --check on every authorized Markdown file: PASS.
• Markdown link and whitespace checker on all seven authorized Markdown files: PASS.
• pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json: PASS (frozen strict contracts unchanged).
• Codex code review iterated through role inheritability, hash-complete migration ordering, all reachable runtime DDL entrypoints, installer DSN persistence, and platform-bootstrap ownership; each finding was incorporated into the final frozen contract/DAG. The last security review found no new KBN-101 vulnerability; its sole low finding is the pre-existing unstaged Mosaic session-lock metadata, which is excluded from this commit.
• Commit: 82ce3252df38a687c50485f8d048b53ca8db5989 (docs(#771): freeze database runtime role split).
• Pre-push queue guard: ci-queue-wait.sh --purpose push -B main returned state=unknown without failure. The push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check: PASS.
• Pushed branch docs/771-kbn101-db-role-split at the exact commit above; no PR was opened, merged, or closed. web1:mosaic-100 received the handoff with head, decisions, DAG, and validation.
• Awaiting independent security/Ultron review.

2026-07-15 — rc.6 exact-head remediation session

• Objective / correction: Replace the prior planning-author handoff and close every finding in the independent exact-head report against da742ca2da4a2ff466916c818fe275c4f7ffd384. The report is a verbatim durable copy of the task-supplied review artifact; scope remains documentation-only, .mosaic is excluded, and source/config/compose/CI/deployment/secrets/migrations remain untouched.
• Finding 1 — closed DDL control plane: rc.6 names mosaic-db-migrator as the sole application/CI/test PostgreSQL DDL runner, requires DATABASE_MIGRATION_URL before connection/DDL, inventories runMigrations, Drizzle config/scripts, db:push, storage CLI, adapter/Gateway startup, fleet-backlog, extension probes/bootstrap, direct federated integration DDL, CI, and future scripts, and specifies route/deny/test disposition for each. Tests use runner-prepared disposable PostgreSQL or invoke that runner; db:push is local-disposable-only and rejects production-like URLs.
• Finding 2 — deployable TLS: rc.6 freezes distinct runtime/migrator URL and CA/server leaf Vault/compose/Swarm secret identifiers, 0400 key and 0600 URL/cert/CA mount requirements, actual compose/Swarm service-DNS SANs, PostgreSQL TLS settings, legacy-client drain/termination plus hostssl enforcement, verified-TLS readiness ordering, fresh/existing transition, CA-overlap rotation/TLS-only rollback, and standalone plus federated/Swarm positive and missing/wrong CA/SAN/downgrade negatives. PGlite is explicitly non-PostgreSQL evidence.
• Finding 3 — manifest/0009: rc.6 defines manifest v1 canonical UTF-8 serialization and raw SQL-byte SHA-256, logical journal order, manifest ownership/grants, exact one-to-one observed hash tuple mapping, non-normative physical insertion order, safe original-0009 conditions, ambiguous-effect fail-closed recovery, and the full required reconciliation/backup test matrix. It preserves shipped 0009 bytes and forbids manual ledger adoption/insertion.
• Finding 4 — advisory lock: replaced hashtext with fixed signed-int4-safe (1297044289,1262636593) (MOSA,KBN1), one max:1 runner session, close-on-crash semantics, and contention/crash/readiness/unrelated-key evidence.
• Finding 5 — identifiers/search path: selects mosaic, exact pg_catalog,mosaic per pooled connection and SET LOCAL transactions, plans audited public-object/extension/Drizzle relocation, forbids config-derived identifiers, limits bootstrap quoting to server-side %I on fixed allowlist, and requires injection/pool-reset negatives.
• Finding 6 — safe DAG: cards 00–07 are inactive prepared capability while owner-runtime remains N-1; KBN-101-08 is the one atomic activation release after platform roles/TLS and compatible code. Mosaic control plane/Jason alone can activate/rollback; no force-on-red, runtime bypass, or temporary compatibility survives the gate. The approved role graph, post-KBN-100 immutable certification, and KBN-105 gate remain unchanged.
• Review remediation: Codex review found the legacy plaintext cutover gap, missing URL-secret bindings, historical public migration incompatibility, non-reproducible checkout-byte hashing, CONNECT allowlisting regression, and undocumented direct-DDL operator instructions. rc.6 now requires drain/scale-to-zero, residual non-TLS session termination, hostssl with no host rule, zero plaintext-session proof, TLS-only post-enforcement rollback, distinct named runtime/migrator secret consumers, canonical Git-blob/LF manifest bytes, a runner-only owner-controlled legacy-public bootstrap followed by mosaic relocation, explicit CONNECT/TEMP revocation, and KBN-101-07 replacement of direct-DDL documentation. It also required the durable exact-head report link above. Pre-existing .mosaic runtime state remains excluded.
• Validation: Prettier on all changed Markdown, repository Markdown link/whitespace check, and strict native-kanban contract TypeScript passed before final staging; the final staged diff excludes .mosaic. No source-code TDD applies because this is contract-only remediation.

2026-07-15 — rc.7 residual remediation session

• Objective / correction: Close every residual in the independent exact-head rc.6 re-review at /home/hermes/agent-work/reviews/771-kbn101-contract-rereview-45ba3d6.md for 45ba3d6ad4d5383f457a303c05bc816144cfa48a, without changing source, compose, CI, deployment, migration, or secret artifacts. Only the existing authorized planning/documentation paths are eligible; pre-existing .mosaic state remains excluded.
• Source-backed scope confirmed: the active federated-pgvector.integration.test.ts executes CREATE TEMP TABLE; tracked docker/init-db.sql and infra/pg-init/01-extensions.sql both create vector; migrate-tier.ts advertises raw CREATE EXTENSION; and tools/federation-harness/docker-compose.two-gateways.yml is current plaintext two-PostgreSQL/two-Gateway topology. Current schema.ts has 36 default-schema pgTable declarations, 6 default pgEnum declarations, and an unqualified vector custom type; historical migrations contain public references.
• Plan: (1) make the finite DDL/static-bypass inventory and DATABASE_URL-only denial matrix exact, including the runner-prepared persistent pgvector fixture and migrated two-gateway harness; (2) freeze executable public-to-mosaic and mosaic_extensions transition, Drizzle ownership, object-catalog classes/order, eligibility and rollback tests; (3) bind repository/control-plane ownership, UID/GID validation, exact artifact/mount rules, and both gateway TLS topology; (4) correct PRD acceptance mapping and cross-document rc.7 status; then run formatting, link/contract, source-path, diff, review, commit, queue guard, and push.
• Independent review closure: initial Codex review found Gateway-key consumer wording, CLAUDE.md omission, final schema-owner set, and placeholder SANs; all are now explicit. Re-review found the legacy 0001 vector-type resolution problem and docs/federation/SETUP.md raw-DDL instruction; the legacy runner now uses only its fixed non-writable pg_catalog,public,mosaic_extensions history path, while runtime remains pg_catalog,mosaic, and the federation setup path is assigned to KBN-101-07/static inventory. Security review final verdict: no confident vulnerability. The review also repeated the pre-existing tracked .mosaic session-state concern; it remains deliberately unstaged/excluded by this task.
• Completion evidence: changed Markdown is Prettier-formatted; local links and strict native-kanban TypeScript passed; source-path inventory confirmed all current referenced paths (the new apps/gateway/Dockerfile is explicitly a planned KBN-101-05 artifact); diff check and authorized-doc allowlist passed. No source-code TDD applies to this documentation-only remediation.

2026-07-15 — rc.8 exact residual remediation session

• Objective / correction: Close all three HIGH findings in the independent rc.7 exact-head re-review at /home/hermes/agent-work/reviews/771-kbn101-contract-rereview2-0778eba.md for 0778eba2db3c2dfbaca3af352b12ba0389d3552b. Scope remains documentation-only: no source, config, Compose, CI, deployment, secret, or migration artifact changed; pre-existing .mosaic/orchestrator state remains excluded.
• Finite authority closure: KBN-101-06 now classifies exact current source/scripts/package bins, operator docs, and deploy manifests. packages/db/src/index.ts has explicit removal/compile-import negative ownership; docs/fleet/backlog-conventions.md and docs/PERFORMANCE.md now remove first-use/direct-Drizzle/Gateway-startup migration instructions and point to sole runner/readiness. Byte-immutable historical SQL, PGlite-only routines, negative-test literals, vendored/generated artifacts, and labeled historical reports are exact-path/category reviewed allowlists; unknown hits fail. The contract explicitly rejects relying on a naive token scan alone.
• Executable and exclusive handoff closure: KBN-101-03 exclusively owns the published mosaic-db-migrator bin, packages/db/src/cli.ts, private migrator modules, docker/db-migrator.Dockerfile, exact --run|--verify|--help, environment/argv limits, sanitized exits, and command/order tests. KBN-101-00 exclusively owns infra/pg-bootstrap/roles.sql, infra/pg-bootstrap/extensions.sql, infra/pg-bootstrap/README.md, and bootstrap tests. KBN-101-05 exclusively owns tools/db/render-postgres-secrets.ts, its tests, and deployment declarations, consuming the versioned bootstrap interface without overlap.
• pgvector owner closure: mosaic_extension_owner is dedicated NOLOGIN, available only to the external bootstrap actor during bootstrap; fresh vector/member ownership remains there. The contract records PostgreSQL's unsupported extension-owner transfer and forbids catalog mutation, ownership adoption, and DROP CASCADE. Approved-owner existing extensions use verified ALTER EXTENSION ... SET SCHEMA; legacy runtime-owned extensions fail closed to a controlled backup/shadow/runner/copy-evidence/quiesce/final-delta/atomic-switch/read-only-rollback migration. It requires pg_extension.extowner, member/schema/version, and runtime/migrator/schema-owner ALTER/DROP/member-update denial tests across clean, approved-owner, legacy shadow, partial/resume/rollback, and N-1.
• Cross-document state: PRD, KBN contract, shared contract, task decomposition, index, sitemap, current operator docs, and this scratchpad are rc.8-consistent. The only intended next action is a fresh independent exact-head re-review after validation/push.
• Validation / review: Prettier passed for all nine changed Markdown documents; local links passed (9 documents); pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json passed; source-path inventory passed (20 paths: 8 current, 12 explicitly planned); finite-authority requirement checklist and git diff --check passed. Manual documentation/security review checked the three requested paths, private-only runner boundary/exit contract, non-overlapping 00/03/05 ownership, extension-owner denial and shadow path, and .mosaic exclusion. No source-code TDD applies because this is contract-only remediation.
• Delivery evidence: committed 1423c2ad02b5471eab006fb4c878808e5b29c387 as docs(#771): close role split rc.8 residuals. Push queue guard returned state=unknown without error; push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check, all PASS; branch push succeeded. This final evidence append is committed next, then the exact remote head is verified. The only intended next action is a fresh independent exact-head re-review.

2026-07-15 — rc.9 final residual remediation session

• Objective / correction: Close the three findings in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview3-9cf5d2f.md against exact head 9cf5d2f6641b14082dc3294e2a84d1fb4ccc019d: move mosaic_extensions schema ownership to mosaic_extension_owner; replace all broad/conflicting KBN-101 card ownership with a complete disjoint exact-path/test manifest; and classify the current architecture-plan db:migrate instruction with pinned scanner mechanics. Scope remains documentation-only; no source/config/Compose/CI/deployment/secret/migration artifact and no .mosaic path may be modified.
• Plan: inspect current tracked source topology to name only existing paths; update the normative contract first and synchronize PRD/shared/task/index/sitemap/version language; run Prettier, changed-doc links, strict contract TypeScript, source/path and manifest-overlap checks, diff allowlist, independent documentation/security review; then stage docs only, commit, queue-guard, push, and verify exact remote SHA. No source-code TDD applies because this is contract-only remediation.
• Closure implemented: rc.9 makes mosaic_extension_owner create and own mosaic_extensions, vector, and members; the external bootstrap actor alone temporarily SET ROLEs for fresh/approved-owner work, while schema owner has only USAGE for legacy type resolution and never temporary CREATE. Catalog/default-ACL plus direct DDL/member denials now cover runtime, migrator, and schema owner through fresh, relocation, shadow/resume, and rollback evidence.
• Delivery decomposition: Replaced broad ownership with complete disjoint 00–09 manifests, exact tests/evidence, producer-before-consumer edges, and an explicit no-intermediate-deploy N-1 activation statement. packages/storage/src/{cli,migrate-tier}.ts belongs only to -02; the current tracked init artifacts are retired by -02 as direct-DLL closure; -03 owns all runner/index/migrate/config/schema assets and exact compiled-bin/image mapping; -07 owns docs only; -08/-09 own evidence only.
• Classifier closure: -06 has exact scanner/inventory/matrix paths, canonical inventory fields, classes/dispositions, fixed token/rule set, exact allowlist categories/restrictions, and self-test requirements for unknown, duplicate-owner, ownerless, missing-path, and historical masking cases. The architecture plan now marks direct db:migrate superseded and uses mosaic-db-migrator --run.
• Validation: Prettier check, strict native-kanban contract TypeScript, changed-document local-link resolution, git diff --check, and an automated manifest-overlap/owner/current-source-path check passed. Targeted documentation/security review verified role ownership/default privileges/search path/preflight/legacy/shadow/rollback consistency, disjoint manifests/DAG/activation, scanner mechanics, exact bin/entrypoint, and no .mosaic staging intent. No source-code TDD applies because this is documentation-only remediation.
• Next: stage documentation only, commit, queue-guard, push, verify exact remote SHA, then wait for fresh exact-head review.
• Delivery evidence: committed 8cbad2bcd9bc7507052f74f35670ef7c8e39e44e as docs(#771): close role split rc.9 residuals; ci-queue-wait.sh --purpose push -B main returned state=unknown without error; push-hook pnpm typecheck, pnpm lint, and pnpm format:check all passed; push succeeded and origin/docs/771-kbn101-db-role-split resolved to that exact SHA. Pre-existing .mosaic/orchestrator/{mission.json,session.lock} remains modified but intentionally unstaged/excluded. The only next action is a fresh independent exact-head re-review.

2026-07-15 — rc.10 Ultron NO-GO remediation intake

• Objective / scope: Close only HIGH-1 and HIGH-2 in /home/hermes/agent-work/reviews/771-kbn101-ultron-11f09a1.md against exact head 11f09a15e43e72afda6a0374668996b4bda9e536. Documentation only: preserve approved content; no source/config/Compose/CI/deploy/secret/migration edits and no .mosaic staging.
• Plan: (1) replace the impossible NOLOGIN NOSUPERUSER extension owner with the exact non-login, zero-member NOLOGIN SUPERUSER external-control exception and document the non-delegable superuser residual; (2) require the audited external superuser session to SET ROLE/RESET ROLE for fresh, approved-owner, and shadow extension work, then prove catalog ownership and all service-role denial; (3) assign the active migrate-tier guide exclusively to KBN-101-07, add its active secure route to the -06 inventory/matrix/scanner schema, and freeze --target-url-file /run/secrets/mosaic_migrate_target_url plus pre-migrated target/dedicated non-DDL importer requirements; (4) synchronize PRD/shared/tasks/index/sitemap/guide and rc.10 status; (5) validate formatting, links, contracts, source paths, finite operator inventory, diff, review, commit, queue guard, push, and exact remote SHA.
• Target-image evidence before edits: local pgvector/pgvector:pg17 control file reports default_version = '0.8.2', relocatable = true, and no trusted/superuser override (untrusted PostgreSQL extension). An isolated PostgreSQL 17 container proved a NOLOGIN SUPERUSER mosaic_extension_owner can create mosaic_extensions and vector under external-superuser SET ROLE, returns to the external session after RESET ROLE, has rolcanlogin=false, rolsuper=true, zero role members, exact extension/schema and owner-bearing-member ownership, and denies SET ROLE, ALTER EXTENSION, DROP EXTENSION, and schema ownership changes to runtime, migrator, schema owner, and data importer. Ownerless PostgreSQL catalog member classes (pg_am, pg_cast) were intentionally not misrepresented as ownable members.
• TDD decision: skipped as not applicable: this is a documentation-only contract remediation. Future KBN-101-00/-02/-06 tests are specified as the situational evidence; no source/test artifact is permitted in this task.
• Final scope correction: Per control-plane direction, remediation remains bounded to the two Ultron findings. The active guide is explicitly a non-operative KBN-101 contract until its owned implementation/activation lands; no additional design, source, CI, deployment, secret, or test artifact was added.
• Validation evidence: target-image/container role proof PASS (pgvector 0.8.2, relocatable=true, trusted absent/untrusted; external-superuser SET ROLE/RESET ROLE; exact extension/schema/owner-bearing-member ownership; zero membership and service-role denials). Changed-doc Prettier, strict native-kanban contract TypeScript, local-link resolver (8 docs), finite operator-doc inventory (one active KBN-101-07 route with no credential argv in executable blocks), source-path check (6 current paths), and git diff --check PASS. Pre-existing .mosaic/orchestrator/{mission.json,session.lock} remains intentionally excluded.

2026-07-15 — rc.11 exact-head re-review remediation intake

• Objective / scope: Close only HIGH-1 and HIGH-2 in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview5-f60144e.md against f60144eb3eab6234ab01bda592052081c777897e: target-bind the non-DDL tier importer with a runner-produced signed attestation, and disposition every current non-normative documentation scanner hit, including the active user-guide route and federation historical status. Documentation only; no source/config/Compose/CI/deployment/secret/migration edits and no .mosaic staging.
• Frozen decision: mosaic-db-migrator --verify is trusted only after TLS/identity/manifest/schema verification and signs a credential-free JCS/Ed25519 v1 artifact from a runner-only fixed root-owned private-key file. The artifact binds secret version/exact URL-file digest, canonical TLS/CA/SPKI/server/database/importer/manifest/schema identity, issued/expiry/nonce, and producer build/correlation; importer gets pinned public key plus artifact only. It validates both files and all bindings before target connection, opens/digests/connects from one in-memory URL read, validates server identity before DML, and distinguishes zero connection from zero DML. Key overlap/revocation, secret-rotation invalidation, replay cache, atomic rename, and sanitized errors are mandatory.
• Ownership: -03 owns producer/signing DTO/tests; -02 importer interface/verification tests; -05 key/artifact mounts/render tests; -06 inventory/matrix and non-masking scanner tests; -07 operator guide. The exact manifests remain disjoint.
• Operator correction: storage migrate is schema-wrapper delegation only; legacy --from hot --to cold tier-copy guidance is unavailable. Secure tier copy is migrate-tier with --target-url-file plus --target-attestation-file. Federation M1 task language is status-only and adjacent KBN-101 text says it authorizes no current DDL.
• TDD decision: skipped as not applicable: this bounded task changes documentation only. Future -02/-03/-05/-06 tests are specified as the required implementation evidence.
• Validation / review: Prettier PASS on all 18 changed Markdown/root-doc files; pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json PASS; changed-doc local-link resolver PASS (71 links); full current docs scanner/disposition PASS (10 non-normative paths, 4 normative-contract paths; no unknown active command); legacy storage migrate --from and raw target-URL bypass scan PASS; attestation field/interface assertion PASS; exact source ownership/manifest-overlap assertion PASS; git diff --check, docs-only scope, and secret-leak scan PASS. Manual documentation/security review verified key isolation, JCS/detached signature, secret-file hash as non-secret evidence, verification ordering/TOCTOU/replay/rotation, no connection vs zero DML, non-masking scanner class, status-only federation history, and no regression to pgvector closure. No source-code TDD applies.
• Delivery evidence: committed 6227f076c819bd124383851633b16d4ef9c88a98 as docs(#771): bind tier importer to verified target; staged scope was 18 documentation files only and excluded .mosaic. Pre-push queue guard returned state=unknown without failure. Push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check: PASS. Branch push succeeded. Verify the exact remote SHA after this final delivery-evidence append, then idle for independent exact-head re-review and Ultron reverify. .mosaic/orchestrator/{mission.json,session.lock} remains pre-existing and excluded.

2026-07-15 — rc.12 bounded deployable-importer/SETUP remediation plan

• Objective / scope: Close the two HIGH findings in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview6-65663d4.md against exact head 65663d4f72f2ace5148bce9aeba04b5a8d5beee9. Documentation/tracking only; preserve every earlier closure; do not modify source, deployment, Compose, CI, migration, Vault, or .mosaic artifacts.
• Plan: (1) make KBN-101-05 own canonical KV-v2 importer URL/version provenance, separate immutable generation-pinned renderer consumers, importer CA/public-key/attestation mounts, fixed importer/migrator identities, safe-fd lifecycle, isolation/rotation/TOCTOU/error evidence, and -02/-03/-06 handoffs; (2) convert docs/federation/SETUP.md to a non-operative N-1 reference with only the required external-bootstrap → TLS/roles → runner --run → --verify → Gateway-readiness sequence; (3) broaden scanner grammar plus path-specific semantic negatives so indirect first-boot/startup/init/Compose authority cannot be masked by a named, normative, or status record; (4) synchronize PRD/shared/tasks/index/sitemap/federation task state and this scratchpad; (5) run formatting, links, strict contracts, complete-doc scanner/semantic assertions, diff/operator inventory, material/manifest overlap, review, docs-only stage, commit, queue guard, push, and exact remote-SHA verification.
• TDD decision: skipped because this bounded change is documentation-only; the affected -02/-03/-05/-06 implementation tests and scanner semantic fixtures are specified as mandatory future evidence.
• Review correction: independent Codex review found the initial 10003 producer → immutable 10002 importer artifact handoff impossible. rc.12 now specifies the required privileged deployment handoff controller: after runner success it safe-opens/verifies producer artifact plus generation, exact-byte copies/fsyncs/atomically renames to a distinct 10002:10002 0400 importer mount, seals it read-only, and starts no importer on partial/wrong-generation/owner/mode failure. It receives only a root-owned non-secret expected-version/URL-digest/generation descriptor plus public verifier key, never URL bytes/private key; producer/importer share no writable file or mount. Dry-run nonce consumption now requires fresh --verify and artifact before --yes; the active-route schema requires targetCredentialVersionFile. Pre-existing .mosaic state is confirmed excluded from staging.
• Validation result: changed-doc Prettier and git diff --check PASS; strict pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json PASS; contract/SETUP material and non-operative semantic assertions PASS. Pending final docs-only stage, commit, queue guard, push, and remote-head verification.

2026-07-15 — rc.13 MILESTONES semantic-scan remediation intake

• Objective / scope: Close only HIGH-1 from /home/hermes/agent-work/reviews/771-kbn101-contract-rereview7-7365dcf.md against 7365dcf15c09262a46132b9c011769ad98243641. Documentation/tracking only: assign docs/federation/MILESTONES.md exclusively to KBN-101-07 and its exact former startup-extension wording to the KBN-101-06 semantic fixture/inventory; replace the wording with a non-operative historical/status disposition. No source, deployment, Compose, CI, Vault, migration, provider, or .mosaic artifact is authorized.
• Frozen remediation: runtime/startup extension provisioning is superseded and forbidden. The sole eligible sequence is external bootstrap → TLS/roles → mosaic-db-migrator --run → mosaic-db-migrator --verify → Gateway readiness. The MILESTONES record authorizes no current DDL, Compose/init, or startup path. The scanner must prove the exact former wording fails before any inventory/status-only mask and rerun its full current-doc operator/deploy-manifest scan outside reports/scratchpads.
• Plan: update only MILESTONES plus the exact KBN-101 contract/inventory/manifests and necessary PRD/shared/task/index/sitemap/version/status references; run full lexical+semantic scan, Prettier, links, strict contract TypeScript, diff and manifest-overlap checks; stage docs only (excluding pre-existing .mosaic), commit, queue-guard, push, and verify the exact remote SHA. No source-code TDD applies because this bounded task changes documentation only.
• Remediation result (pre-commit): MILESTONES.md now makes runtime/startup extension provisioning superseded and forbidden, with only external bootstrap → TLS/roles → mosaic-db-migrator --run → --verify → Gateway readiness; it authorizes no current DDL/Compose/init/startup path. The KBN-101-07 manifest/inventory is exclusive and KBN-101-06 documents the exact former wording as a semantic negative that fails before inventory masking. Full current-doc scan outside reports/scratchpads: 103 Markdown files, 10 lexical-hit paths classified, 2 owned Compose-before-runner references, and zero ownerless indirect/literal routes. Prettier, strict native-kanban TypeScript, local links (64/0), diff check, and 95-path manifest-overlap reconstruction (0 overlaps; MILESTONES only -07) passed. Pre-existing .mosaic/orchestrator/{mission.json,session.lock} remains excluded.
• Delivery checkpoint: committed remediation as 237bac81c93dc4305470cea23a67e4ced730bd61 (docs(#771): close MILESTONES startup authority). Push queue guard returned state=unknown without failure; the push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check, all PASS; the remote branch resolved to that exact SHA. This final evidence append is committed next, then the exact remote head is verified and the branch waits for independent exact-head rereview/Ultron reverify. .mosaic remains unstaged.

2026-07-15 — rc.13 current-document safety remediation intake

• Objective / scope: Close only HIGH-1 and HIGH-2 in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview8-aeacc70.md against exact head aeacc702353740aad0f2f086974cc0670e360d1d. This is documentation/tracking only. Preserve all prior gates; do not edit source, Compose, deployment artifacts, CI, migrations, Vault data, reports, or .mosaic.
• Source-backed decision: Current docker-compose.yml mounts infra/pg-init into PostgreSQL init and that SQL creates vector; it cannot be used as a current PostgreSQL start route before KBN-101 bootstrap/runner artifacts exist. The checked-in configuration declares a supported local PGlite tier (DEFAULT_LOCAL_CONFIG and tier-detection both establish in-process PGlite with no external service probe), so docs may retain a local/PGlite route and start only non-PostgreSQL Compose services such as valkey.
• Plan: (1) replace the README and dev-guide Compose-first PostgreSQL instructions with a PGlite/no-PostgreSQL developer path and an explicit held PostgreSQL/federated future activation sequence; (2) replace the deployment quick-start and bare-metal production procedure with non-operative status, no production .env/automatic dotenv/EnvironmentFile/credential export-or-argv/restart guidance, and only a non-executable future renderer/Vault generation-pinned process-exec or LoadCredential schematic; (3) expand KBN-101-06/-07 semantic fixture/disposition language to fail the exact former README/dev/deployment Compose-first sequences and production credential routes before ownership/status masking; (4) synchronize PRD/shared/tasks/index/sitemap/status/version and this scratchpad; (5) run formatting, links, strict contract TypeScript, full current-doc lexical+semantic scan outside reports/scratchpads, manifest-overlap, review, docs-only stage, commit, queue guard, push, and exact remote-SHA verification.
• TDD decision: no source or fixture implementation may be changed in this documentation-only remediation. The -06 future fixture requirements are frozen as acceptance evidence; validation here is static semantic inventory plus documentation quality gates.
• Correction from independent review: The initial local-Gateway PGlite wording was unsafe. apps/gateway/src/main.ts loads daemon/root/app-local environment files before tier selection; an inherited daemon DATABASE_URL can select PostgreSQL, whose current startup reaches extension creation and migrations. No source is authorized in this docs-only task. The remediation therefore holds Gateway/Web local startup, preserves only PGlite data-layer plus selected non-PostgreSQL Compose work, and assigns KBN-101-02 the fail-closed daemon/inherited/root/app-local DSN and non-local-tier rejection before connection/DDL, with a regression proof. The future renderer boundary remains KBN-101-05.
• Remediation evidence: Removed active PostgreSQL Compose-first and production credential guidance from README, CLAUDE, dev/deployment, and the residual historical TUI/MCP routes; local documentation now permits only PGlite data-layer/non-PostgreSQL Valkey work while Gateway/Web startup is explicitly held. KBN-101-06/-07 now freeze exact former README/dev/deployment Compose sequences plus production credential patterns as pre-classification semantic negatives. Independent review surfaced the current daemon/root/app dotenv loader as an unsafe source boundary; no source is authorized here, so the docs hold that startup and assign fail-closed removal/regression proof to KBN-101-02.
• Validation: Prettier PASS (12 changed docs); local-link resolver PASS (71 links); pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json PASS; full README/CLAUDE/docs inventory PASS (104 documents, 0 active non-normative Compose/init/production-credential violations); manifest reconstruction PASS (10 cards, 95 declared path tokens, 0 overlaps); git diff --check PASS. Pre-existing .mosaic/orchestrator/{mission.json,session.lock} remains intentionally unstaged.
• Final route correction: Held the residual MCP environment/restart and historical TUI smoke-test routes after review showed they could bypass the Gateway local-start hold; no bearer-token-over-HTTP or Gateway restart route remains in this remediation scope.
• Delivery: committed 7cc156b777189ee89448e4d569a8b3f69560a240 (docs(#771): hold unsafe database startup routes) and d8f935c20ade835aa3ec03fe5d6961885d8b5f0b (docs(#771): record final route correction). Push queue guard returned state=unknown without failure; both pushes completed and the remote matched d8f935c before this final delivery-evidence append. .mosaic/orchestrator/{mission.json,session.lock} remains pre-existing and unstaged. Await a fresh independent exact-head re-review/Ultron verification.

2026-07-15 — rc.15 exact one-finding runner/legacy-CI remediation intake

• Objective / scope: Close only HIGH-1 in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview9-be0ebfd.md against be0ebfdc6a2b32a0ab6989117ebbb12f43854d71. Documentation/tracking only: no source, Compose, CI, deployment, migration, Vault, reports, or .mosaic edit.
• Plan: Replace imperative current runner routes in the architecture plan and PERFORMANCE with one explicit non-operative future procedure; make fleet backlog current behavior PGlite-only and PostgreSQL held; classify README's checked-in direct CI db:migrate as legacy N-1/uncertified/non-authorizing pending KBN-101-06 removal; then extend the future KBN-101-06 lexical/semantic inventory contract so unqualified runner/current-CI authority fails before masking while only the complete named held procedure passes. Synchronize required contract/PRD/shared/task/index/sitemap state, run full docs scan and document gates, stage docs only, commit, queue-guard, push, and verify remote SHA.
• TDD decision: not applicable: the user authorizes documentation only and the named -06 fixture/inventory files do not yet exist; the contract records their required future implementation evidence.
• Remediation / review closure: Architecture, PERFORMANCE, federation SETUP/MILESTONES, deployment/dev/migrate-tier, and README now use one Markdown-bounded Held future procedure form where needed: non-operative/no-current-command-authority; KBN-101-00/-03/-05; external bootstrap → TLS/roles → mosaic-db-migrator --run → mosaic-db-migrator --verify → Gateway/Compose readiness. Any runner hit outside that section is a -06 semantic failure. Fleet backlog current behavior is PGlite-only. README accurately records the checked-in direct CI migration as an active, isolated-disposable-database, uncertified legacy N-1 DDL exception that is non-authorizing as an operator route and pending -06 removal; it no longer falsely claims the current CI role lacks DDL capability. Independent Codex review found and this pass closed the readiness-endpoint, standalone-verify, CI-factuality, and scanner-boundary findings. Its only residual finding concerns pre-existing tracked .mosaic/orchestrator runtime state, which is explicitly excluded and unstaged by task scope; security review found no vulnerability.
• Validation: changed-doc Prettier PASS; pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json PASS; changed-doc local links 72/0; git diff --check PASS; full README/CLAUDE/docs lexical+semantic scan outside reports/scratchpads PASS (106 Markdown documents; 7 operator documents with runner tokens; zero unqualified future-runner/current-CI authority routes); KBN-101 manifest check PASS (10 dependency-ordered cards; -07 docs/-06 fixtures disjoint); docs-only allowlist PASS (16 docs, pre-existing .mosaic excluded).
• Delivery evidence: committed d857463a8a4658e34a77177737860cf82cc26ac6 (docs(#771): hold unimplemented runner routes). Pre-push queue guard returned state=unknown without failure; the push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check, all PASS. Push succeeded and origin/docs/771-kbn101-db-role-split matched d857463a8a4658e34a77177737860cf82cc26ac6. This evidence append is committed and pushed next; .mosaic/orchestrator/{mission.json,session.lock} stays pre-existing, unstaged, and excluded. Await exact-head independent re-review/Ultron reverify.

2026-07-15 — rc.16 exact one-finding generic-wrapper remediation intake

• Objective / scope: Close only HIGH-1 in /home/hermes/agent-work/reviews/771-kbn101-contract-rereview10-18e253c.md against exact head 18e253c8790bdbb5bc30a06c116472213b83b22f. Documentation/tracking only: no source, Compose, CI, deployment, migration, Vault, report, or .mosaic change.
• Source-backed correction: packages/storage/src/cli.ts currently labels storage migrate a thin wrapper for pnpm --filter @mosaicstack/db db:migrate and executes that direct Drizzle command with execSync; no mosaic-db-migrator executable exists. Therefore the README commented form and user-guide executable form must not describe runner delegation or provide current command authority.
• Plan: Remove the current wrapper command from README/user-guide command guidance; record it as legacy N-1, uncertified, non-operative, and forbidden pending KBN-101-02/-03/-06/-08 activation. Retain only the held future ordered bootstrap → TLS/roles → runner --run → --verify → readiness sequence and the separately held secure migrate-tier route. Extend KBN-101-06's future semantic fixture/matrix with both exact former forms (including the README commented code-fence form), requiring their failure before inventory/status masking and a source-consistency assertion that direct Drizzle wrapper source cannot be described as runner delegation. Synchronize status/version references only where required, then run document gates, stage docs only, commit, queue-guard, push, and verify the remote SHA.
• TDD decision: not applicable: this bounded task changes documentation only; the required future -06 semantic/source-consistency fixtures are specified as implementation acceptance evidence.
• Remediation / validation: README and user-guide remove the generic wrapper from command guidance and state the direct-Drizzle current-source truth, legacy-N-1/uncertified/non-operative MUST-NOT-INVOKE boundary, named -02/-03/-06/-08 activation cards, future external-bootstrap → TLS/roles → runner --run → --verify → readiness sequence, and separately held secure migrate-tier route. The KBN-101 rc.16 contract records both exact former forms (README commented code fence and user-guide executable code fence), requires failure before inventory/ownership/status masking, and requires the direct-Drizzle/no-runner-bin source-consistency proof. Prettier passed on all nine changed Markdown documents; changed-doc local links passed (72/0); strict native-kanban contract TypeScript and git diff --check passed; full README/CLAUDE/docs scan outside reports/scratchpads passed (106 documents, zero non-normative executable generic-wrapper or false runner-delegation route); and manifest validation passed (10 cards, 90 exact tokens, zero overlaps). Pre-existing .mosaic/orchestrator/{mission.json,session.lock} remains excluded.