Scratchpad — KBN-101 DB runtime/migration role split (#771)

Branch: docs/771-kbn101-db-role-split
Base: main e9c4aa3
Scope: planning/documentation only; authorized files are PRD, Native Kanban task/shared/index docs, sitemap, this scratchpad, and the new KBN-101 contract.
Explicit exclusions: source/runtime/config/deployment/secret/migration/compose/CI/lock/package/KBN-100 branch edits; no production mutation.

Objective

Freeze an implementation-ready PostgreSQL role/connection split so the Gateway uses a non-owner runtime identity and only a dedicated migration phase uses an owner/migrator identity. Make real deployed-role certification—not synthetic role tests—a serial prerequisite of KBN-100 and KBN-105.

Intake and current-state evidence

Mission MVP is active; W3 Native Kanban/SOT is planning-complete. The task state shows KBN-010 as the predecessor and KBN-100 as the current schema slice.
Current branch started at e9c4aa3; .mosaic/orchestrator/{mission.json,session.lock} were already runtime-modified and remain untouched.
packages/db/src/client.ts, migrate.ts, and drizzle.config.ts resolve one DATABASE_URL (with default fallback). packages/storage/src/adapters/postgres.ts calls runMigrations(this.url).
apps/gateway/src/database/database.module.ts calls storageAdapter.migrate() at startup for PostgreSQL; this is the owner-runtime defect to remove in KBN-101 implementation.
packages/config/src/mosaic-config.ts, installer wizard, local/federated compose, Portainer test stack, and .woodpecker/ci.yml currently expose one URL. PGlite has an existing explicit local migration path.
Current KBN contract requires immutable events/checkpoints/artifacts/evidence, RESTRICT, and KBN-100 generated Drizzle consistency. It did not establish a deployable runtime identity split.

Frozen decisions

DATABASE_URL is the non-owner runtime URL; DATABASE_MIGRATION_URL is migration-only. Both are required in their respective PostgreSQL phases; PGlite is the explicit local exception; migration never falls back to runtime/default/config URL.
PostgreSQL Gateway runtime never auto-runs migration/DDL. Dedicated migrator uses pg_try_advisory_lock(hashtext('mosaic-schema-migration-v1')); replicas only check exact ordered Drizzle-ledger readiness and fail closed.
Roles are non-login mosaic_platform_database_owner, non-login mosaic_schema_owner, login/noinherit mosaic_migrator, non-login mosaic_runtime_capability, and login/inherit mosaic_runtime. Runtime inherits only its capability role with SET/ADMIN denied, has no owner/migrator membership, no unsafe attributes/ownership/DDL authority, and an explicit trusted search path.
Runtime gets mutable DML only as needed, but INSERT/SELECT only on task_events, artifacts, task_checkpoints, task_checkpoint_artifacts, and approval_decision_artifacts. KBN-100 still enforces RESTRICT/no-cascade.
Startup verifies effective role/ownership/attributes/inherited capability/TEMP/function-execute/ledger grants/search path/immutable denials/schema fingerprint without DSN exposure. It also requires authenticated CA/hostname-verified TLS. Stable sanitized errors and redaction rules are required.
N-1 retains single runtime URL only as a non-certified compatibility release; staged role provisioning/migration/runtime deployment then enforces the split. Rollback never injects migration URL into Gateway.
Vault target paths, rotation, deployment injection, CI, installer, compose, Portainer, and observability are separate one-card/one-PR handoffs. The migration-only file manifest includes packages/db/drizzle.config.ts; KBN-101 repairs the known PostgreSQL runner/journal ordering defect and proves a clean database applies every hash once before its foundation certificate. No application migration creates roles/passwords or hardcodes credentials.
KBN-101 foundation merges/certifies first. KBN-100 then rebases, restores Drizzle declaration/snapshot/journal consistency, and bounds procedural immutable-table grant/trigger/backfill work to its own slice. Because those immutable relations do not exist until KBN-100, KBN-101’s real deployed-role immutable-operation certificate follows KBN-100 and is the serial gate before KBN-105.

Assumptions

standalone and federated are all current PostgreSQL production-like modes; a future PostgreSQL tier inherits this contract unless versioned otherwise.
Deployment will support a dedicated migration Job/one-shot command. A target that cannot run it cannot receive production/federated KBN certification.
Canonical Vault target paths require deployment-owner verification before provisioning; the planning document does not claim they already exist.

Documentation produced

docs/PRD.md: bounded KBN-101 requirements and acceptance criteria.
docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md: normative rc.5 implementation, threat, migration/rollback, evidence, and exact file DAG contract.
docs/native-kanban-sot/SHARED-CONTRACT.md: rc.5 amendment preserving rc.4.
docs/native-kanban-sot/TASKS.md: KBN-101 inserted before and blocks KBN-100; KBN-105 held.
Native Kanban index and root sitemap links.

Validation plan

Prettier only for changed Markdown.
Markdown link target/check checks scoped to modified docs.
Strict contract check with pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json (the frozen TypeScript contracts remain unchanged).
Diff allowlist proves only authorized documentation files changed, apart from pre-existing Mosaic runtime state.
Independent documentation/security self-review: role escalation, fallback, startup DDL, schema readiness, grants/default privileges, immutable tables, secret leakage, deployment and KBN-100 boundaries.

Review corrections

Independent Codex review found two blockers and security review found two medium defects; all were remediated in the frozen contract:

Split the KBN-101 certificate into a foundation role/schema-boundary certificate (before KBN-100) and real immutable-operation certificate (after KBN-100, before KBN-105). This preserves the requested KBN-100 block without requiring evidence for tables not yet created.
Removed the legacy owner-runtime exception. N-1 compatibility preserves variable/config shape only; the KBN-101 runtime refuses owner/migrator identity and current single-URL installs remain on their previous release until role cutover.
Introduced mosaic_platform_database_owner as a separate non-login platform role. mosaic_schema_owner owns application/ledger schemas only, not the database and has no database CREATE/ALTER/extension authority.
Replaced blocking pg_advisory_lock with pg_try_advisory_lock and the deterministic DATABASE_MIGRATION_LOCKED failure.
Review also flagged active .mosaic/orchestrator state. It was pre-existing launcher state and remains unstaged/uncommitted.
Second review added packages/db/drizzle.config.ts to the migration-only slice, mandates DATABASE_MIGRATION_URL with a missing-variable negative, grants runtime only USAGE plus SELECT on drizzle.__drizzle_migrations, and verifies/revokes its ledger writes.
Security review added DATABASE_TLS_CA_CERT_PATH / DatabaseTlsConfigDto with authenticated TLS and hostname/CA verification in production-like modes, explicit database TEMPORARY revocation/catalog denial testing, and default-PUBLIC function EXECUTE revocation with SECURITY DEFINER prohibited by default.
Final review corrected the runtime login to inherit only its capability role with SET/ADMIN denied, and moved the known hash-complete migration-runner/journal repair into KBN-101-03 before the foundation certificate.
Final manifest review added all live runtime DDL paths (packages/storage/src/tier-detection.ts, Gateway startup, and fleet-backlog) to KBN-101-02, requiring read-only extension probes and no PostgreSQL runtime auto-migration. It also requires KBN-101-04 to stop persisting either DSN into generated .env/mosaic.config.json, using only Vault/deployment references and injected variables.
Provisioning review separated the external privileged platform bootstrap actor from the NOCREATEDB database-owner role and added KBN-101-00. That IaC/bootstrap card owns fresh/existing database role/ownership/grant/Vault transition evidence and is a foundation-certificate dependency.

Results

pnpm exec prettier --check on every authorized Markdown file: PASS.
Markdown link and whitespace checker on all seven authorized Markdown files: PASS.
pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json: PASS (frozen strict contracts unchanged).
Codex code review iterated through role inheritability, hash-complete migration ordering, all reachable runtime DDL entrypoints, installer DSN persistence, and platform-bootstrap ownership; each finding was incorporated into the final frozen contract/DAG. The last security review found no new KBN-101 vulnerability; its sole low finding is the pre-existing unstaged Mosaic session-lock metadata, which is excluded from this commit.
Commit: 82ce3252df38a687c50485f8d048b53ca8db5989 (docs(#771): freeze database runtime role split).
Pre-push queue guard: ci-queue-wait.sh --purpose push -B main returned state=unknown without failure. The push hook ran repository pnpm typecheck, pnpm lint, and pnpm format:check: PASS.
Pushed branch docs/771-kbn101-db-role-split at the exact commit above; no PR was opened, merged, or closed. web1:mosaic-100 received the handoff with head, decisions, DAG, and validation.
Awaiting independent security/Ultron review.

2026-07-15 — rc.6 exact-head remediation session

Objective / correction: Replace the prior planning-author handoff and close every finding in the independent exact-head report against da742ca2da4a2ff466916c818fe275c4f7ffd384. The report is a verbatim durable copy of the task-supplied review artifact; scope remains documentation-only, .mosaic is excluded, and source/config/compose/CI/deployment/secrets/migrations remain untouched.
Finding 1 — closed DDL control plane: rc.6 names mosaic-db-migrator as the sole application/CI/test PostgreSQL DDL runner, requires DATABASE_MIGRATION_URL before connection/DDL, inventories runMigrations, Drizzle config/scripts, db:push, storage CLI, adapter/Gateway startup, fleet-backlog, extension probes/bootstrap, direct federated integration DDL, CI, and future scripts, and specifies route/deny/test disposition for each. Tests use runner-prepared disposable PostgreSQL or invoke that runner; db:push is local-disposable-only and rejects production-like URLs.
Finding 2 — deployable TLS: rc.6 freezes distinct runtime/migrator URL and CA/server leaf Vault/compose/Swarm secret identifiers, 0400 key and 0600 URL/cert/CA mount requirements, actual compose/Swarm service-DNS SANs, PostgreSQL TLS settings, legacy-client drain/termination plus hostssl enforcement, verified-TLS readiness ordering, fresh/existing transition, CA-overlap rotation/TLS-only rollback, and standalone plus federated/Swarm positive and missing/wrong CA/SAN/downgrade negatives. PGlite is explicitly non-PostgreSQL evidence.
Finding 3 — manifest/0009: rc.6 defines manifest v1 canonical UTF-8 serialization and raw SQL-byte SHA-256, logical journal order, manifest ownership/grants, exact one-to-one observed hash tuple mapping, non-normative physical insertion order, safe original-0009 conditions, ambiguous-effect fail-closed recovery, and the full required reconciliation/backup test matrix. It preserves shipped 0009 bytes and forbids manual ledger adoption/insertion.
Finding 4 — advisory lock: replaced hashtext with fixed signed-int4-safe (1297044289,1262636593) (MOSA,KBN1), one max:1 runner session, close-on-crash semantics, and contention/crash/readiness/unrelated-key evidence.
Finding 5 — identifiers/search path: selects mosaic, exact pg_catalog,mosaic per pooled connection and SET LOCAL transactions, plans audited public-object/extension/Drizzle relocation, forbids config-derived identifiers, limits bootstrap quoting to server-side %I on fixed allowlist, and requires injection/pool-reset negatives.
Finding 6 — safe DAG: cards 00–07 are inactive prepared capability while owner-runtime remains N-1; KBN-101-08 is the one atomic activation release after platform roles/TLS and compatible code. Mosaic control plane/Jason alone can activate/rollback; no force-on-red, runtime bypass, or temporary compatibility survives the gate. The approved role graph, post-KBN-100 immutable certification, and KBN-105 gate remain unchanged.
Review remediation: Codex review found the legacy plaintext cutover gap, missing URL-secret bindings, historical public migration incompatibility, non-reproducible checkout-byte hashing, CONNECT allowlisting regression, and undocumented direct-DDL operator instructions. rc.6 now requires drain/scale-to-zero, residual non-TLS session termination, hostssl with no host rule, zero plaintext-session proof, TLS-only post-enforcement rollback, distinct named runtime/migrator secret consumers, canonical Git-blob/LF manifest bytes, a runner-only owner-controlled legacy-public bootstrap followed by mosaic relocation, explicit CONNECT/TEMP revocation, and KBN-101-07 replacement of direct-DDL documentation. It also required the durable exact-head report link above. Pre-existing .mosaic runtime state remains excluded.
Validation: Prettier on all changed Markdown, repository Markdown link/whitespace check, and strict native-kanban contract TypeScript passed before final staging; the final staged diff excludes .mosaic. No source-code TDD applies because this is contract-only remediation.

2026-07-15 — rc.7 residual remediation session

Objective / correction: Close every residual in the independent exact-head rc.6 re-review at /home/hermes/agent-work/reviews/771-kbn101-contract-rereview-45ba3d6.md for 45ba3d6ad4d5383f457a303c05bc816144cfa48a, without changing source, compose, CI, deployment, migration, or secret artifacts. Only the existing authorized planning/documentation paths are eligible; pre-existing .mosaic state remains excluded.
Source-backed scope confirmed: the active federated-pgvector.integration.test.ts executes CREATE TEMP TABLE; tracked docker/init-db.sql and infra/pg-init/01-extensions.sql both create vector; migrate-tier.ts advertises raw CREATE EXTENSION; and tools/federation-harness/docker-compose.two-gateways.yml is current plaintext two-PostgreSQL/two-Gateway topology. Current schema.ts has 36 default-schema pgTable declarations, 6 default pgEnum declarations, and an unqualified vector custom type; historical migrations contain public references.
Plan: (1) make the finite DDL/static-bypass inventory and DATABASE_URL-only denial matrix exact, including the runner-prepared persistent pgvector fixture and migrated two-gateway harness; (2) freeze executable public-to-mosaic and mosaic_extensions transition, Drizzle ownership, object-catalog classes/order, eligibility and rollback tests; (3) bind repository/control-plane ownership, UID/GID validation, exact artifact/mount rules, and both gateway TLS topology; (4) correct PRD acceptance mapping and cross-document rc.7 status; then run formatting, link/contract, source-path, diff, review, commit, queue guard, and push.
Independent review closure: initial Codex review found Gateway-key consumer wording, CLAUDE.md omission, final schema-owner set, and placeholder SANs; all are now explicit. Re-review found the legacy 0001 vector-type resolution problem and docs/federation/SETUP.md raw-DDL instruction; the legacy runner now uses only its fixed non-writable pg_catalog,public,mosaic_extensions history path, while runtime remains pg_catalog,mosaic, and the federation setup path is assigned to KBN-101-07/static inventory. Security review final verdict: no confident vulnerability. The review also repeated the pre-existing tracked .mosaic session-state concern; it remains deliberately unstaged/excluded by this task.
Completion evidence: changed Markdown is Prettier-formatted; local links and strict native-kanban TypeScript passed; source-path inventory confirmed all current referenced paths (the new apps/gateway/Dockerfile is explicitly a planned KBN-101-05 artifact); diff check and authorized-doc allowlist passed. No source-code TDD applies to this documentation-only remediation.

16 KiB Raw Blame History Unescape Escape