| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.6 frozen DDL-runner, manifest/ledger, TLS/bootstrap, role/search-path, activation, and one-card/one-PR contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
| [`contracts/health-state.v1.ts`](./contracts/health-state.v1.ts) | Discriminated public health, separate branded transaction-local write proof, and non-overlapping denial/transport/version-conflict mappings |
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | `da742ca` REQUEST CHANGES findings that rc.6 remediates and requires to be independently re-reviewed |
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
| [`contracts/health-state.v1.ts`](./contracts/health-state.v1.ts) | Discriminated public health, separate branded transaction-local write proof, and non-overlapping denial/transport/version-conflict mappings |
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | Historical`da742ca` REQUEST CHANGES report retained as prior closure evidence; rc.7 awaits independent exact-head re-review for the later residual report |
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
| `packages/db/src/migrate.ts:runMigrations()` | Remove its optional URL, `DATABASE_URL`, and default fallback API from public/runtime exports. Its PostgreSQL behavior moves behind `mosaic-db-migrator`; callers cannot invoke it with an arbitrary URL. |
| `packages/db/drizzle.config.ts` and direct `drizzle-kit migrate` | A database-connecting Drizzle configuration reads only `DATABASE_MIGRATION_URL` through the migration DTO and rejects its absence before connection. Replace direct `drizzle-kit migrate` exposure with `mosaic-db-migrator`. `db:generate` is an offline schema artifact command and must neither resolve nor connect to a URL. |
| `pnpm --filter @mosaicstack/db db:migrate` and CI invocation | Make it a thin `mosaic-db-migrator` wrapper; no direct Drizzle migrator invocation remains. CI supplies an isolated disposable migration URL only to that job. |
| `db:push` / direct `drizzle-kit push` | Forbidden for standalone, federated, CI production-like, Portainer, and any URL outside a disposable developer database. If retained for local experimentation, a wrapper requires `MOSAIC_DISPOSABLE_DEVELOPER_DB=1`, a locally allowlisted disposable target, `DATABASE_MIGRATION_URL`, verified TLS when PostgreSQL is used, and rejects `DATABASE_URL`, any production-like tier, and every non-allowlisted host/database before connection. It is never a release, repair, or migration procedure. |
| `mosaic storage migrate --run` | Delegates only to `mosaic-db-migrator`; it rejects `DATABASE_URL`-only execution before spawning or connecting. Its PGlite form remains explicitly local-only. |
| `PostgresAdapter.migrate()`, Gateway `DatabaseModule`/startup, and PostgreSQL adapter factories | Runtime PostgreSQL migration is removed: no `runMigrations`, DDL, `CREATE EXTENSION`, or migration-compatible handle is reachable from startup. Gateway performs read-only identity, TLS, `search_path`, and manifest-ledger readiness checks only. |
| `packages/mosaic/src/commands/fleet-backlog.ts` | PostgreSQL fleet backlog never migrates or creates tables. It consumes a ready runtime connection; PGlite may use only its explicit local migration routine. |
| `packages/storage/src/{adapters/postgres,tier-detection}.ts` extension work and `infra/pg-init/01-extensions.sql` | Runtime probes become read-only catalog/extension-presence checks. Extension provisioning is a fixed external bootstrap prerequisite or a reviewed runner migration where the migrator has the required scoped authority; it is never a probe side effect or a reason to grant runtime database CREATE. |
| `packages/db/src/federation.integration.test.ts` direct type/table/index DDL | Replace with a pre-migrated disposable database created by `mosaic-db-migrator`, or make the test invoke that runner. The test itself has runtime credentials and no direct DDL. |
| `README.md` and `docs/guides/{dev-guide,deployment}.md` operatorinstructions | `KBN-101-07` replaces direct `db:migrate`, `db:push`, and `CREATE EXTENSION` production-like instructions with the runner/bootstrap procedure, and labels any remaining disposable-local command as non-production with the §2 guard. Documentation is a DDL entrypoint and cannot advertise a bypass. |
| New package scripts, test helpers, setup hooks, CLI commands, installers, CI steps, adapters, or operator docs | A repository check rejects any new PostgreSQL DDL-capable entrypoint or instruction unless it is the dedicated runner or the named external bootstrap artifact. No future script may accept a URL parameter or `DATABASE_URL` as a DDL escape hatch. |
| Current entrypoint | Required rc.7 disposition |
| `packages/db/src/migrate.ts:runMigrations()`| Remove its optional URL, `DATABASE_URL`, and default fallback API from public/runtime exports. Its PostgreSQL behavior moves behind `mosaic-db-migrator`; callers cannot invoke it with an arbitrary URL. |
| `packages/db/drizzle.config.ts` and direct `drizzle-kit migrate`| A database-connecting Drizzle configuration reads only `DATABASE_MIGRATION_URL` through the migration DTO and rejects its absence before connection. Replace direct `drizzle-kit migrate` exposure with `mosaic-db-migrator`. `db:generate` is an offline schema artifact command and must neither resolve nor connect to a URL. |
| `pnpm --filter @mosaicstack/db db:migrate` and CI invocation | Make it a thin `mosaic-db-migrator` wrapper; no direct Drizzle migrator invocation remains. CI supplies an isolated disposable migration URL only to that job. |
| `db:push` / direct `drizzle-kit push`| Forbidden for standalone, federated, CI production-like, Portainer, and any URL outside a disposable developer database. If retained for local experimentation, a wrapper requires `MOSAIC_DISPOSABLE_DEVELOPER_DB=1`, a locally allowlisted disposable target, `DATABASE_MIGRATION_URL`, verified TLS when PostgreSQL is used, and rejects `DATABASE_URL`, any production-like tier, and every non-allowlisted host/database before connection. It is never a release, repair, or migration procedure. |
| `mosaic storage migrate --run`| Delegates only to `mosaic-db-migrator`; it rejects `DATABASE_URL`-only execution before spawning or connecting. Its PGlite form remains explicitly local-only. |
| `PostgresAdapter.migrate()`, Gateway `DatabaseModule`/startup, and PostgreSQL adapter factories | Runtime PostgreSQL migration is removed: no `runMigrations`, DDL, `CREATE EXTENSION`, or migration-compatible handle is reachable from startup. Gateway performs read-only identity, TLS, `search_path`, and manifest-ledger readiness checks only. |
| `packages/mosaic/src/commands/fleet-backlog.ts`| PostgreSQL fleet backlog never migrates or creates tables. It consumes a ready runtime connection; PGlite may use only its explicit local migration routine. |
| `packages/storage/src/{adapters/postgres,tier-detection}.ts` extension work and `infra/pg-init/01-extensions.sql` | Runtime probes become read-only catalog/extension-presence checks. Extension provisioning is a fixed external bootstrap prerequisite or a reviewed runner migration where the migrator has the required scoped authority; it is never a probe side effect or a reason to grant runtime database CREATE. |
| `packages/db/src/federation.integration.test.ts` direct type/table/index DDL | Replace with a pre-migrated disposable database created by `mosaic-db-migrator`, or make the test invoke that runner. The test itself has runtime credentials and no direct DDL. |
| `apps/gateway/src/__tests__/integration/federated-pgvector.integration.test.ts``CREATE TEMP TABLE` | `KBN-101-06` replaces the temporary runtime DDL with a runner-prepared disposable **persistent**`mosaic.federated_pgvector_fixture` database/table. The test receives only its runtime URL/CA and performs read/query-only qualified-vector assertions (including the selected vector operator); it has no setup hook, temporary privilege, or DDL. Runtime `TEMPORARY` remains denied. |
| `docker/init-db.sql` | `KBN-101-00` deletes this duplicate tracked init artifact. `infra/pg-init/01-extensions.sql` is also retired from image-init ownership; neither may remain as a hidden extension authority. The sole extension action is the fixed external bootstrap artifact described in §5/§7, or the runner only when its reviewed implementation card explicitly grants that authority. |
| `packages/storage/src/migrate-tier.ts` operator remediation | `KBN-101-07` replaces the raw SQL recommendation with the sanitized instruction `mosaic-db-migrator --run` (secret injection and target selection are deployment-owned; no URL, SQL, or credential is accepted on the command line), or directs the operator to the approved external-bootstrap runbook when extension eligibility has not been met. It never recommends `CREATE EXTENSION`. |
| `tools/federation-harness/docker-compose.two-gateways.yml` | Active topology; `KBN-101-05` migrates it rather than retires it. It must contain `postgres-a`, `postgres-b`, `mosaic-db-migrator-a`, `mosaic-db-migrator-b`, `gateway-a`, and `gateway-b`; each database has separate runtime/migrator URL and CA consumers, TLS server material, verified-TLS readiness, and runner-before-Gateway ordering. Its existing plaintext URLs/init mount are forbidden. |
| `README.md`, `CLAUDE.md`, `docs/guides/{dev-guide,deployment}.md`, and `docs/federation/SETUP.md` operator instructions | `KBN-101-07` replaces direct `db:migrate`, `db:push`, and `CREATE EXTENSION` production-like instructions with the runner/bootstrap procedure, and labels any remaining disposable-local command as non-production with the §2 guard. Documentation is a DDL entrypoint and cannot advertise a bypass. |
| New package scripts, test helpers, setup hooks, CLI commands, installers, CI steps, adapters, or operator docs | A repository check rejects any new PostgreSQL DDL-capable entrypoint or instruction unless it is the dedicated runner or the named external bootstrap artifact. No future script may accept a URL parameter or `DATABASE_URL` as a DDL escape hatch. |
The runner must run the original migration bytes, including shipped `0009`; no migration command may repair a ledger by manual insertion, adoption, `db:push`, or schema diff. Tests requiring PostgreSQL schema consume a pre-migrated disposable database or invoke this exact runner. Every row above has a negative test proving `DATABASE_URL`-only execution fails before connection/DDL; the `db:push` negatives also cover production-like tier and production-like URL rejection.
The runner must run the original migration bytes, including shipped `0009`; no migration command may repair a ledger by manual insertion, adoption, `db:push`, or schema diff. Tests requiring PostgreSQL schema consume a pre-migrated disposable database or invoke this exact runner. `KBN-101-06` owns a finite static inventory containing every row above by exact path plus the existing `infra/pg-init/01-extensions.sql`, `.woodpecker/ci.yml`, `packages/db/{src/migrate.ts,drizzle.config.ts,package.json}`, `packages/db/src/migrate.test.ts`, `packages/storage/src/cli.ts`, `CLAUDE.md`, and `docs/federation/SETUP.md`; it fails on an unclassified current DDL token, URL-consuming setup hook, init mount, or raw operator DDL guidance. The same card owns a `DATABASE_URL`-only denial matrix for **each inventory path** and both `gateway-a`/`gateway-b`: every negative must fail before connection/DDL, including missing migration URL, runtime-only fixture/test, direct Drizzle, `db:push`, init artifact, tier migration guidance, and each harness migration job/Gateway. The `db:push` negatives also cover production-like tier and production-like URL rejection.
## 3. Exact migration manifest, ledger, and lock
@@ -93,18 +97,26 @@ Before any preflight that can decide migration state, `mosaic-db-migrator` acqui
Role creation, passwords, membership, database ownership, certificates, and Vault values are platform/IaC/operator work—not Drizzle/application migrations. Application SQL must not issue credential/role management statements or embed credentials.
| Role | Attributes and ownership | Membership / session use |
| `mosaic_platform_database_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; platform-only database owner after bootstrap. | Never granted to application roles. |
| external platform bootstrap actor | Provider/operator/IaC-controlled privileged identity outside the Mosaic role graph and Vault/application configuration. | Creates/transitions the database and roles, then retires from application use. |
| `mosaic_schema_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; owns only `mosaic` and`drizzle` schemas/objects. | Never an application login; no database/extension authority except reviewed bootstrap scope. |
| `mosaic_migrator` | `LOGIN NOINHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`. | Only `mosaic_schema_owner`; runner verifies `session_user=mosaic_migrator`, `SET ROLE mosaic_schema_owner`, then `current_user=mosaic_schema_owner`. |
| `mosaic_runtime_capability` | `NOLOGIN` and no ownership/administrative attributes. | Holds only named runtime grants. |
| `mosaic_runtime` | `LOGIN INHERIT` with no ownership/administrative attributes. | Only `mosaic_runtime_capability WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`; never owner/migrator member. |
| Role | Attributes and ownership | Membership / session use |
| `mosaic_platform_database_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; platform-only database owner after bootstrap. | Never granted to application roles. |
| external platform bootstrap actor | Provider/operator/IaC-controlled privileged identity outside the Mosaic role graph and Vault/application configuration. | Creates/transitions the database and roles, then retires from application use. |
| `mosaic_schema_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; final ownership is only `mosaic`,`drizzle`, and `mosaic_extensions` schemas/objects. | Never an application login; no database/extension authority except reviewed bootstrap scope. |
| `mosaic_migrator` | `LOGIN NOINHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`. | Only `mosaic_schema_owner`; runner verifies `session_user=mosaic_migrator`, `SET ROLE mosaic_schema_owner`, then `current_user=mosaic_schema_owner`. |
| `mosaic_runtime_capability` | `NOLOGIN` and no ownership/administrative attributes. | Holds only named runtime grants. |
| `mosaic_runtime` | `LOGIN INHERIT` with no ownership/administrative attributes. | Only `mosaic_runtime_capability WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`; never owner/migrator member. |
The fixed application/runtime schema is **`mosaic`**. Every application/runtime pooled connection executes and verifies exactly `SET search_path TO pg_catalog, mosaic` before its first application query; connection checkout repeats this after reset/reconnect. Transactional application operations use `SET LOCAL search_path TO pg_catalog, mosaic` and verify it before query execution. `public` and `$user` are forbidden in all runtime paths.
The sole runner has one narrowly bounded **legacy-history bootstrap subphase** for the immutable current journal: before it runs, external bootstrap revokes `CREATE` on `public` from `PUBLIC`, assigns its ownership to `mosaic_schema_owner`, and denies all runtime connections. In its locked `max:1` migration session only, after `SET ROLE mosaic_schema_owner`, it uses `SET LOCAL search_path TO pg_catalog, public` solely to execute the shipped historical bytes that contain unqualified objects and explicit `public.*` references. It then, in that same runner/session before manifest certification or any runtime readiness, inventories and relocates every application object to `mosaic`, reconciles extension/type dependencies, regenerates/verifies Drizzle declarations/snapshots, and restores the exact `pg_catalog,mosaic` runner path. This compatibility subphase is not a runtime/operator option, accepts no configuration identifier, has no fallback, and is removed/disabled before KBN-101-08. A clean-PostgreSQL test must prove original historical bytes succeed in this subphase and runtime reaches readiness only with `pg_catalog,mosaic`; existing-database transition follows the same inventory/relocation or fails closed on an unknown dependency. `vector` moves only if relocatable; otherwise every type/function reference is explicitly qualified and its non-writable extension schema is documented. The pre-activation rollback restores the approved backup or reverses only the reviewed relocation artifact before activation; no runtime `search_path` bypass exists.
The sole runner has one narrowly bounded **legacy-history bootstrap subphase** for the immutable current journal: before it runs, external bootstrap revokes `CREATE` on `public` from `PUBLIC`, temporarily assigns its ownership to `mosaic_schema_owner`, and denies all runtime connections. In its locked `max:1` migration session only, after `SET ROLE mosaic_schema_owner`, it uses the fixed legacy-only `SET LOCAL search_path TO pg_catalog, public, mosaic_extensions` solely to execute byte-immutable historical migrations `0000` through the current head. `mosaic_extensions` is non-writable in that session and is included only so immutable `0001` resolves its unqualified existing `vector` type; it is never in a runtime session path. After relocation/catalog verification and before manifest certification or any runtime readiness, the runner transfers `public` ownership to `mosaic_platform_database_owner`, revokes application `USAGE`/`CREATE`, and restores `pg_catalog,mosaic`. This compatibility subphase is not a runtime/operator option, accepts no configuration identifier, has no fallback, and is removed/disabled before KBN-101-08.
`KBN-101-03` exclusively owns `packages/db/src/schema.ts`, `packages/db/drizzle/meta/*`, `packages/db/drizzle/meta/_journal.json`, the generated relocation migration, and exact database/Drizzle tests. It freezes one exported `export const mosaic = pgSchema('mosaic')`; every application `pgTable` and every application `pgEnum` must be declared through that export. The generated snapshot/journal and future `db:generate` output must target `mosaic` only; a static declaration/SQL test rejects a default-schema application `pgTable`/`pgEnum`, `public` application declaration, or future generated application DDL outside `mosaic`. Historical `0000` through current SQL and the shipped journal provenance are byte-immutable and execute only in the trusted legacy-`public` subphase; they are never rewritten to claim they originally targeted `mosaic`.
The selected extension policy is fixed: external bootstrap creates `vector` fresh as `CREATE EXTENSION vector WITH SCHEMA mosaic_extensions`; `mosaic_extensions` is owned by `mosaic_schema_owner`, non-writable by runtime, and grants runtime only `USAGE`/read-only function access proven by catalog. Existing databases may execute `ALTER EXTENSION vector SET SCHEMA mosaic_extensions` only after a catalog eligibility check proves the exact supported `extversion`, `extrelocatable = true`, a complete expected extension-member set, and no dependent object that would change the selected qualification policy; any other version, false relocatability, extra/missing member, or partial prior move fails eligibility closed. `schema.ts` must emit `mosaic_extensions.vector(...)`, and all vector casts/operators/functions in runner, application queries, fixtures, and generated SQL must explicitly qualify `mosaic_extensions` (for example `OPERATOR(mosaic_extensions.<->)`); `mosaic_extensions` is deliberately **not** added to runtime `search_path`.
Before relocation, the runner records a parameterized catalog inventory and dependency graph. It must classify, in dependency order, application enum/domain/base types; owned and identity sequences; application tables; table columns/defaults; functions/procedures; views/materialized views; extension-owned objects; then triggers, constraints/FKs, indexes, and dependent rules/policies. It moves base types, sequences, tables, and functions as required; OID-bound constraints/indexes follow their owning objects and are catalog-verified rather than recreated blindly. Every object must be expected exactly once and be in either the immutable legacy/bootstrap allowlist, the `mosaic` application allowlist, or the selected `mosaic_extensions` extension membership; unknown, extra, duplicate, cross-schema, dependency-cycle, or partial-resume state fails closed. No raw client-side identifier interpolation is allowed.
`KBN-101-03` tests clean bootstrap, current-public relocation, interrupted/partial relocation resume, and pre-activation reverse rollback-before-activation; reverse rollback is allowed only before KBN-101-08 and restores the approved backup or the reviewed inverse relocation, never a runtime `search_path` bypass. Its N-1 order is: current legacy release → inactive bootstrap/runner and `mosaic` declarations → catalog relocation and generated-artifact verification → verified non-owner runtime activation. Tests include byte-immutable historical execution, no public application objects/declarations/future SQL, vector type/cast/operator query success under fixed `pg_catalog,mosaic`, and all extension eligibility negatives.
No SQL identifier may come from URL/config/environment/operator input. Catalog comparisons use parameter values. The fixed identifiers above are constants; the external bootstrap artifact alone may use server-side `format('%I', fixed_allowlisted_identifier)` after allowlist validation. Raw client-side interpolation for identifiers, `SET search_path`, database, schema, role, table, or extension names is forbidden. Tests include injection-shaped values, a poisoned pooled-session reset, and transaction `SET LOCAL` restoration negatives.
@@ -114,33 +126,38 @@ Immutable KBN relations, after KBN-100 creates them, grant runtime only `SELECT,
## 5. Deployable verified-TLS bootstrap
The **operator/IaC owner** provisions the CA and PostgreSQL leaf key/certificate. Mosaic applications never generate, self-sign, copy, or persist production certificates. The external bootstrap actor receives those secrets only through the deployment secret mechanism; no plaintext development exception exists for production-like modes.
`mosaicstack/stack` is the named repository/control plane for KBN-101-00 and KBN-101-05. Those cards exclusively own the current `docker-compose.yml`, `docker-compose.federated.yml`, `deploy/portainer/federated-test.stack.yml`, and `tools/federation-harness/docker-compose.two-gateways.yml`, plus the new `apps/gateway/Dockerfile`, bootstrap renderer/templates, and rendered-config validation tests. The named **Mosaic deployment control plane / Jason** is activation authority; the environment-specific IaC/Vault owner supplies only the approved input secret versions and may not substitute an unreviewed current-repository artifact. The renderer is the only handoff: it reads secret-provider references, validates owners/modes/digests/SANs, writes each output atomically (`mkstemp` on the target tmpfs, `fsync`, `chmod`/`chown`, atomic rename), and records only secret-version identifiers and hashes.
| Material | Vault target / deployment secret | Mount, injection, and authorized consumer |
| Runtime URL | `secret-{env}/mosaic-stack/database/runtime` (`url`) → `mosaic-db-runtime-url-v1` | `/run/secrets/mosaic-db-runtime-url`, `0600`, Gateway service UID only; entrypoint maps it to `DATABASE_URL` only at process exec. It is denied to migrator, storage runtime, fleet, ordinary CLI, and every test except an explicit runtime-negative fixture. |
| Migration URL | `secret-{env}/mosaic-stack/database/migrator` (`url`) → `mosaic-db-migrator-url-v1` | `/run/secrets/mosaic-db-migrator-url`, `0600`, one-shot `mosaic-db-migrator` service UID only; entrypoint maps it to `DATABASE_MIGRATION_URL` only at process exec. It is denied to Gateway, storage runtime, fleet, and ordinary CLI. |
| CA bundle | `secret-{env}/mosaic-stack/database/tls-ca` (`certificate`) → `mosaic-db-ca-v1` | Runtime and migrator: `/run/secrets/mosaic-db-ca.crt`, `0600`, service UID; PostgreSQL gets a read-only CA copy only when client-cert validation is enabled. |
| PostgreSQL private key | same Vault record (`private_key`) → `mosaic-postgres-server-key-v1` | PostgreSQL: `/run/secrets/mosaic-postgres-server.key`, `0400`, postgres UID, never mounted to Gateway or migrator. |
`KBN-101-05` changes the Gateway image to fixed non-root `USER 10001:10001`. Gateway CA and Gateway leaf-certificate mounts, and its own Gateway private key only when it terminates its HTTPS listener, must be readable by `10001:10001`; PostgreSQL private keys and migration-only material are never mounted there, and no secret is world-readable. PostgreSQL is not assigned a guessed UID/GID: its image must first be pinned by digest, and an image-inspection plus rendered Compose/Swarm test freezes the image's effective PostgreSQL UID:GID before the renderer selects mount owner/group. A digest, service UID/GID, rendered secret `uid`/`gid`/`mode`, or container `USER` mismatch is a KBN-101-05 failure. Mosaic applications never generate, self-sign, copy, or persist production certificates; the external bootstrap actor receives them only through the deployment secret mechanism and no plaintext development exception exists for production-like modes.
Compose uses identically named local **secret references**, rendered by an operator-controlled secret provider into a non-repository tmpfs; Swarm declares the named secrets with the same target paths, UID/GID, and modes. KBN-101-05 rejects bind-mounted committed cert/key files, environment-encoded PEM, and a missing secret. The existing target Vault names are planned canonical paths and must be verified/provisioned by the deployment owner; the planning card does not claim they exist.
| Material | Vault target / deployment secret | Mount, injection, and authorized consumer |
| Runtime URL | `secret-{env}/mosaic-stack/database/runtime` (`url`) → `mosaic-db-runtime-url-v1` | Gateway only: `/run/secrets/mosaic-db-runtime-url`, `0600`, `10001:10001`; entrypoint maps it to `DATABASE_URL` only at process exec. It is denied to every migrator, storage runtime, fleet, ordinary CLI, and test except an explicit runtime-negative fixture. |
| Migration URL | `secret-{env}/mosaic-stack/database/migrator` (`url`) → `mosaic-db-migrator-url-v1` | Each one-shot migrator only: `/run/secrets/mosaic-db-migrator-url`, `0600`, fixed migrator UID:GID asserted by the image/render test; entrypoint maps it only to `DATABASE_MIGRATION_URL`. It is denied to Gateway, storage runtime, fleet, and ordinary CLI. |
| CA bundle | `secret-{env}/mosaic-stack/database/tls-ca` (`certificate`) → `mosaic-db-ca-v1` | Gateway/migrator: `/run/secrets/mosaic-db-ca.crt`, `0444`, owned by the consuming UID:GID; CA is public trust material. PostgreSQL receives a distinct read-only CA copy only when client-cert validation is enabled. |
| Gateway leaf certificate | `secret-{env}/mosaic-stack/federation/gateway-server-tls` (`certificate`) → `mosaic-gateway-server-cert-v1` | Gateway only: `/run/secrets/mosaic-gateway-server.crt`, `0444`, `10001:10001`; renderer emits this exact Compose and Swarm target and validates it before start. |
| Gateway private key | same Vault record (`private_key`) → `mosaic-gateway-server-key-v1` | Gateway only: `/run/secrets/mosaic-gateway-server.key`, `0400`, `10001:10001`; not mounted to migrator or PostgreSQL and never world-readable. |
| PostgreSQL private key | same Vault record (`private_key`) → `mosaic-postgres-server-key-v1` | PostgreSQL only: `/run/secrets/mosaic-postgres-server.key`, `0400`, frozen verified postgres UID:GID; never mounted to Gateway or migrator. |
Compose uses identically named local **secret references** rendered by the named renderer into non-repository tmpfs paths; Swarm declares the same secrets and target paths with the tested `uid`, `gid`, and mode. KBN-101-05 rejects bind-mounted committed cert/key files, environment-encoded PEM, missing secrets, non-atomic renderer output, private-key access outside its named consumer (Gateway for Gateway key; PostgreSQL for PostgreSQL key), and any world-readable URL/key. The existing target Vault names are planned canonical paths and must be verified/provisioned by the deployment-owner input; the planning card does not claim they exist.
The server leaf SANs are frozen to actual connection DNS names, not a configurable alias:
| topology | PostgreSQL service DNS names that must be SANs |
| standalone compose | `DNS:postgres`, `DNS:localhost` (only for the documented host-port disposable test path) |`DNS:gateway`; its Gateway process consumes runtime URL + CA + Gateway leaf/key only |
| federated compose | `DNS:postgres-federated`, `DNS:localhost` (only for the documented host-port disposable test path) |`DNS:gateway-federated`; runtime/migrator consume only their database-specific material |
| Portainer/Swarm federated test stack | `DNS:postgres` (the in-stack service endpoint used by Gateway and migrator) |`DNS:gateway`; same consumer isolation |
| two-gateway harness | `DNS:postgres-a`, `DNS:postgres-b` | `DNS:gateway-a`, `DNS:gateway-b`; each Gateway gets only its own runtime URL, CA, and Gateway leaf/key; each `mosaic-db-migrator-{a,b}` gets only the matching migration URL and CA |
A new topology requires a versioned amendment before issuance. The PostgreSQL container activation artifact sets `ssl=on`, `ssl_cert_file`, and `ssl_key_file` to those paths, uses a locked-down `postgresql.conf` include, and verifies file ownership/mode before start. The server health/readiness gate makes a `verify-full` CA/SAN-validated connection as the approved runtime/migrator identity; `pg_isready` alone is insufficient. Migration Job starts only after server TLS readiness. Gateway replicas start only after a successful runner result and independently pass verified-TLS, identity, search-path, and ledger readiness.
A new topology requires a versioned amendment before issuance. The PostgreSQL container activation artifact sets `ssl=on`, `ssl_cert_file`, and `ssl_key_file` to those paths, uses a locked-down `postgresql.conf` include, and verifies file ownership/mode before start. The server health/readiness gate makes a `verify-full` CA/SAN-validated connection as the approved runtime/migrator identity; `pg_isready` alone is insufficient. Migration Job starts only after server TLS readiness. Gateway replicas start only after a successful runner result and independently pass verified-TLS, identity, search-path, and ledger readiness. In the two-gateway harness this ordering occurs independently as `postgres-a → mosaic-db-migrator-a → gateway-a` and `postgres-b → mosaic-db-migrator-b → gateway-b`; no Gateway starts against its database before its own runner certificate succeeds.
**Fresh DB:** provision CA/leaf/secrets and server TLS configuration before initial database bootstrap; bootstrap/extension prerequisites run, then the runner migrates over verified TLS, then runtime deploys. **Existing DB:** take the approved backup, provision/mount TLS material, and **drain/scale to zero every N-1 runtime, worker, CLI maintenance process, and replica before TLS enforcement**. Enable server TLS, terminate any residual non-TLS PostgreSQL backend sessions, set `pg_hba.conf` to `hostssl` for all application/migrator CIDRs with no matching `host` rule, reload/restart as required, and prove the non-TLS session count is zero. Only then prove `verify-full` through the existing endpoint, run reconciliation/migration, and roll the non-owner TLS runtime. There is no plaintext transition interval.
**Rotation/rollback:** stage a CA bundle containing old+new trust roots to runtime/migrator, validate a new server leaf with exact SANs, restart PostgreSQL and validate it, roll consumers, then remove the old root only after evidence. Credential rotation remains independent and never mounts migration material into Gateway. Before expiry, rollback restores the prior known-valid leaf/key and overlapping CA bundle, restarts PostgreSQL, enforces `hostssl`, terminates residual non-TLS sessions, and verifies `verify-full`; it never downgrades `sslmode`, restores a plaintext-only N-1 runtime after enforcement, or accepts plaintext. A pre-enforcement abort may restore the backed-up N-1 state only before `hostssl` is enabled and is recorded as an aborted—not activated—release. The runbook records expiry windows, secret versions, backup ID, drained-service/session evidence, activation actor, and validation result—not secret values.
Required disposable tests cover standalone compose and federated/Swarm positives using verified TLS, plus missing CA, wrong CA, wrong SAN, `sslmode` downgrade, missing/mispermissioned server key, runtime-with-migrator-secret, rendered secret-consumer separation, legacy plaintext drain/termination/`hostssl` enforcement, and readiness-before-migration negatives. PGlite local tests are explicitly classified as non-PostgreSQL and do not satisfy a PostgreSQL TLS test.
Required disposable tests cover standalone compose, federated/Swarm, and the two-gateway harness positives using verified TLS, plus for **both** gateway/database pairs: missing CA, wrong CA, wrong PostgreSQL SAN, wrong Gateway SAN, `sslmode` downgrade, missing/mispermissioned server or Gateway key, runtime-with-migrator-secret, cross-pair secret leakage, rendered secret-consumer/UID/GID/mode isolation, legacy plaintext drain/termination/`hostssl` enforcement, and readiness-before-migration negatives. PGlite local tests are explicitly classified as non-PostgreSQL and do not satisfy a PostgreSQL TLS test.
## 6. Runtime verification and sanitized failures
@@ -156,14 +173,14 @@ Every KBN-101 card remains one PR with exclusive ownership. Cards `00`–`07` ma
| Card | Depends on | Exact ownership and required result |
| `KBN-101-00` platform bootstrap / IaC | contract | Fixed external bootstrap artifact creates/transitions roles, `mosaic`/`drizzle` ownership, extension prerequisite, Vaultsecret bindings, server TLS config, fresh/existing inventory, and reversible evidence. It is not an app migration or a runtime command. |
| `KBN-101-00` platform bootstrap / IaC | contract | `mosaicstack/stack` owns the fixed external bootstrap artifact, its renderer/templates, and rendered validation; environment IaC/Vault is named deployment-owner input only. It creates/transitions roles, `mosaic`/`drizzle`/`mosaic_extensions` ownership, eligible vector bootstrap, Vault-to-secret bindings, service UID/GID declarations, server TLS config, fresh/existing inventory, and reversible evidence. It is not an app migration or runtime command. |
| `KBN-101-04` installer/wizard/config | 01 | Production-like config persists no DSN and has no fallback; only non-secret references/injected variables. |
| `KBN-101-05` compose/Portainer deployment | 00,01,03 | Owns secret references, TLS server/client mounts, server configuration, Job ordering, readiness, standalone/federated/Swarm TLS tests, and no plaintext exception. |
| `KBN-101-06` CI/test topology | 02,03,05 | Disposable identities and pre-migrated/runner test topology; repository checks reject DDL bypasses and insecure URLs. |
| `KBN-101-05` compose/Portainer deployment | 00,01,03 | `mosaicstack/stack` exclusively owns current local/federated Compose, Portainer test stack, the two-gateway harness, bootstrap renderer/templates, gateway `10001:10001`, verified PostgreSQL image UID/GID, secret references/targets, TLS server/client mounts, server configuration, one Job per database, readiness, standalone/federated/Swarm/two-gateway TLS tests, and no plaintext exception. |
| `KBN-101-06` CI/test topology | 02,03,05 | Disposable identities and pre-migrated/runner test topology; owns the finite static DDL inventory, every-path `DATABASE_URL`-only denial matrix, runner-prepared persistent pgvector fixture, and repository checks rejecting DDL bypasses/insecure URLs. |
| `KBN-101-07` Vault/runbook/observability | 00,04,05,06 | Named control-plane/Vault-bootstrap handoff, activation/rotation/incident/backup/rollback runbook, sanitized observability evidence, and replacement of direct-DDL operator documentation (`README.md`,`CLAUDE.md`,`docs/guides/dev-guide.md`, `docs/guides/deployment.md`, `docs/federation/SETUP.md`, `packages/storage/src/{cli,migrate-tier}.ts`). |
| `KBN-101-08` foundation certification and **atomic activation release** | 00…07 | Independent review verifies all prepared cards and terminal-green CI; Mosaic control plane/Jason authorizes one ordered activation: backup → drain/scale zero all N-1 runtime clients → TLS server start → terminate non-TLS sessions and enforce `hostssl`/no `host` rule → verify no plaintext session → roles → runner → verified readiness → rolling non-owner runtime. Any red test/readiness/CI result aborts; no force-on-red or bypass. Remove/disable all temporary compatibility support before this gate. |
| `KBN-101-09` post-KBN-100 certification | KBN-100,08 | Real deployed-role immutable INSERT/SELECT and UPDATE/DELETE-denial evidence plus independent security/Ultron approval. |
@@ -173,14 +190,16 @@ Every KBN-101 card remains one PR with exclusive ownership. Cards `00`–`07` ma
| K101-REQ-04 / AC-K101-04 | After KBN-100, real deployed runtime INSERT/SELECT success and UPDATE/DELETE denial for each frozen relation, with RESTRICT retention evidence. |
| K101-REQ-05 / AC-K101-05 | Prepared-card/no-intermediate-deploy evidence; one final activation authority record; N-1 drain/zero-plaintext-session/`hostssl`, backup/restore, CA overlap rotation, TLS-only rollback, Vault/redaction, and no-force-on-red evidence. |
| K101-REQ-06 / delivery integrity | One-card/one-PR DAG, exact file ownership, docs/link/contract checks, independent author≠reviewer re-review on the pushed exact head, and terminal-green CI for implementation cards. |
| K101-REQ-01 / AC-K101-01 | DTO and command matrix covers local/PGlite, standalone, federated, and both harness pairs; every §2 inventory entry rejects `DATABASE_URL`-only before connection/DDL; no runtime fallback/default. |
| K101-REQ-05 / AC-K101-05 | KBN-101-09 after KBN-100: real deployed runtime INSERT/SELECT success and UPDATE/DELETE denial for each frozen relation, with RESTRICT retention evidence. |
| K101-REQ-06 / AC-K101-06 | KBN-101-00…08 prepared-card/no-intermediate-deploy evidence; one final activation authority record; N-1 drain/zero-plaintext-session/`hostssl`, backup/restore, CA overlap rotation, TLS-only rollback, Vault/redaction, and no-force-on-red evidence. |
| K101-REQ-07 / KBN sequence | KBN-101-08 foundation certificate before KBN-100, KBN-101-09 real immutable-role certificate plus Ultron approval before KBN-105; KBN-100 rebases/restores Drizzle consistency and never bypasses the serial gates. |
| Delivery integrity | One-card/one-PR DAG, exact file ownership, docs/link/contract checks, independent author≠reviewer re-review on the pushed exact head, and terminal-green CI for implementation cards. |
**Status:** CONTROL-PLANE rc.6 DB role/connection remediation pending independent exact-head re-review; prior KCR-001–016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
**Version:** 1.0.0-rc.6
**Status:** CONTROL-PLANE rc.7 DB role/connection residual remediation pending independent exact-head re-review; prior KCR-001–016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
**Version:** 1.0.0-rc.7
**Date:** 2026-07-15
**Change authority:** Mosaic control plane/Jason only
**SI-001 amendment authority:**`web1:mosaic-100` control-plane decision under issue #753
## Amendment record
### 1.0.0-rc.7 — KBN-101 complete current-path, relocation, and two-gateway remediation
- **Finite current-path closure:** static inventory and the `DATABASE_URL`-only-before-connect/DDL denial matrix now explicitly include Gateway's former temporary-table pgvector test (runner-prepared persistent read/query-only fixture), `docker/init-db.sql` retirement, `migrate-tier.ts` runner/bootstrap-only guidance, and the active two-gateway harness. The harness is migrated, not retired: `postgres-a/b → mosaic-db-migrator-a/b → gateway-a/b`, each with isolated URL/CA material, verified readiness, SANs, and positive/negative TLS evidence.
- **Executable relocation:** KBN-101-03 exclusively owns `schema.ts`, Drizzle snapshots/journal/generated relocation and exact tests. All future application declarations use exported `pgSchema('mosaic')`; immutable historical SQL runs only in trusted legacy `public`. `vector` is fixed in non-writable `mosaic_extensions`, with exact catalog relocatability/version eligibility, explicit type/operator qualification, catalog-class ordering, unknown-object fail-closed behavior, clean/current-public/partial/reverse rollback tests, and an N-1 release order.
- **Bound deployment ownership:** `mosaicstack/stack` KBN-101-00/05 owns current Compose, Portainer, two-gateway, bootstrap renderer/templates, UID/GID declarations, and rendered validation. Gateway is fixed to `10001:10001`; PostgreSQL UID/GID is image-inspected and frozen only after digest pinning. Exact secret paths, atomic renderer behavior, Compose/Swarm targets/modes, Gateway/PostgreSQL leaf separation, and two-pair TLS failure evidence are required. Mosaic deployment control plane/Jason is the named activation authority; environment IaC/Vault supplies versioned input only.
- **Correct traceability:** REQ-03 maps to role/schema/search-path, REQ-04 to TLS, REQ-05 to post-KBN-100 immutability, REQ-06 to rollout/rollback, and REQ-07 to the KBN-101 → KBN-100 → KBN-101 → KBN-105 sequence. No prior manifest/lock/role/DAG/activation decision is weakened.
- **Choice:** `mosaic-db-migrator` is the sole application/CI/test PostgreSQL DDL control plane. Every legacy entrypoint is routed or denied, rejects `DATABASE_URL`-only before connection/DDL, and `db:push` is unavailable outside an allowlisted disposable developer target. The runner holds one `max:1` session with fixed `pg_try_advisory_lock(1297044289,1262636593)` across preflight through release.
@@ -35,7 +42,7 @@
## 1. Authority
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.6 DDL/ledger/TLS/role separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.7 DDL/ledger/TLS/role separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
@@ -95,14 +95,14 @@ No consumer implementation begins before KBN-105. No schema work begins before K
### KBN-101 — PostgreSQL runtime/migration role split and deployed-role certification
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.6 remediates the exact-head `da742ca` REQUEST CHANGES report and awaits independent exact-head re-review. Implementation remains held.
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.7 closes the rc.6 exact-head residuals for complete current DDL paths, executable relocation, and active two-gateway TLS/bootstrap. It awaits independent exact-head re-review; implementation remains held.
- **Owner:** Mos integration control plane; independently reviewed by security/Ultron.
- **Mode:** SERIAL foundation certificate blocks KBN-100; its post-KBN-100 real immutable-operation certificate blocks KBN-105.
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; sole `mosaic-db-migrator` PostgreSQL DDL path and all legacy/future entrypoint closure; `DATABASE_TLS_CA_CERT_PATH` plus operator/IaC CA/server-key/cert lifecycle, exact service-DNS SANs, Vault/compose/Swarm mount modes, TLS server/bootstrap/rotation/rollback; PGlite exception; fixed two-int advisory lock; manifest-v1 logical-index/tag/exact-byte-SHA-256 ledger reconciliation including safe `0009`; fixed `mosaic` schema and exact `pg_catalog,mosaic` pooled session path; platform/schema-owner/migrator/runtime roles; ownership, membership, TEMP/ledger-read/default privilege and immutable grant proof; N-1 inactive prepared cards then atomic activation/rollback authority; Vault/redaction/observability/operator runbooks; one-card/one-PR implementation DAG.
- **OUT:** Production mutation in this planning card; KBN-100 tables/data backfill; application API behavior; KBN-105 route/DTO freeze.
- **Evidence:** foundation: exact command/DTO entrypoint negatives for every finite current DDL/static-bypass path (including `DATABASE_URL`-only, runner fixture, retired init, sanitized operator guidance, both harness pairs, and `db:push` refusal); clean/pre-0009/skipped/applied-late/duplicate/unknown/missing/corrupt/stale/backup plus public-to-`mosaic`/partial/reverse runner proof; fixed-lock contention/crash/readiness/unrelated-key tests; runtime cannot invoke migrations/DDL/TEMP; disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus both-pair CA/SAN/downgrade/key mode/UID-GID/URL-secret consumer-isolation and legacy-drain/`hostssl` zero-plaintext negatives; catalog relocation/vector-query/operator/Drizzle-only-`mosaic`, role/grant/search-path/pool-reset/identifier checks; N-1/atomic TLS-only rollback/no-force-on-red rehearsal; named Vault/bootstrap-control-plane/CA-overlap/redaction/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
### KBN-100 — Unified Drizzle schema and concrete N-1 migration
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Hermes Agent