docs(#771): remediate database role split review

This commit is contained in:
Hermes Agent
2026-07-15 00:43:44 -05:00
parent da742ca2da
commit 45ba3d6ad4
8 changed files with 311 additions and 136 deletions

View File

@@ -6,22 +6,23 @@
## Artifacts
| Artifact | Purpose |
| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.5 frozen PostgreSQL runtime/migration connection, role, readiness, rollout, evidence, and one-card/one-PR contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
| [`contracts/health-state.v1.ts`](./contracts/health-state.v1.ts) | Discriminated public health, separate branded transaction-local write proof, and non-overlapping denial/transport/version-conflict mappings |
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001016 findings that blocked the first draft |
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
| Artifact | Purpose |
| -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.6 frozen DDL-runner, manifest/ledger, TLS/bootstrap, role/search-path, activation, and one-card/one-PR contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
| [`contracts/health-state.v1.ts`](./contracts/health-state.v1.ts) | Discriminated public health, separate branded transaction-local write proof, and non-overlapping denial/transport/version-conflict mappings |
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | `da742ca` REQUEST CHANGES findings that rc.6 remediates and requires to be independently re-reviewed |
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001016 findings that blocked the first draft |
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
## Recommended USC lane partition

View File

@@ -1,145 +1,187 @@
# KBN-101 — Database Runtime/Migration Role Split
**Status:** frozen implementation contract; documentation-only planning for [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771)
**Status:** frozen implementation contract; remediation pending independent exact-head re-review for [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771)
**Version:** 1.0.0-rc.5
**Version:** 1.0.0-rc.6
**Dependency:** KBN-010 → **KBN-101 foundation** → KBN-100 → **KBN-101 deployed-role certification** → KBN-105
**Scope:** PostgreSQL standalone/federated runtime identity, migration identity, readiness, deployment handoff, and evidence. No database, secret, deployment, CI, or runtime mutation is authorized by this card.
**Scope:** PostgreSQL standalone/federated runtime identity, the sole application DDL runner, TLS bootstrap, readiness, deployment handoff, and evidence. This documentation card changes no database, secret, deployment, CI, runtime, migration, or compose artifact.
## 1. Decision and current-state evidence
## 1. Decision, modes, and non-negotiable boundaries
Current `main` inspection found one `DATABASE_URL` used by `packages/db/src/client.ts`, `packages/db/src/migrate.ts`, Drizzle Kit, storage `PostgresAdapter.migrate()`, Gateway startup, installer output, Portainer federated stack, and CI. `DatabaseModule.onModuleInit()` invokes `storageAdapter.migrate()` for PostgreSQL, so each Gateway starts migrations under its application identity. The current config path also supplies PostgreSQL defaults. This prevents a production proof that the deployed runtime identity cannot update/delete KBN immutable tables.
Current `main` has one `DATABASE_URL` path in `packages/db/src/client.ts`, `migrate.ts`, `drizzle.config.ts`, storage adapters/CLI, Gateway startup, fleet-backlog, CI, installer output, compose, and Portainer. It also has PostgreSQL DDL outside a controlled migration phase: direct Drizzle scripts, `CREATE EXTENSION` probes, a direct-DDL federated integration test, and platform init SQL. None of those current paths is certified by this contract; every implementation card below must close its listed path.
This contract separates those duties. It does not claim that existing synthetic roles, current defaults, or current single-URL deployments satisfy the target model.
| Item | Exact name / shape | Rule |
| ------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Runtime URL | `DATABASE_URL` | PostgreSQL **non-owner runtime** connection. Required only by runtime services in `standalone`/`federated`; those services never receive `DATABASE_MIGRATION_URL`. |
| Migration URL | `DATABASE_MIGRATION_URL` | Required only by `mosaic-db-migrator`, the dedicated one-shot runner/Job. It is forbidden in Gateway, storage runtime, fleet, and ordinary CLI environments. |
| Runtime DTO | `DatabaseRuntimeConnectionConfigDto` | `tier`, `databaseUrl`, `runtimeMode`; maps only `DATABASE_URL`. It is not constructible from migration configuration. |
| Migration DTO | `DatabaseMigrationConnectionConfigDto` | `tier`, `migrationDatabaseUrl`, `migrationMode`; maps only `DATABASE_MIGRATION_URL`, is accepted only by `mosaic-db-migrator`, and has no runtime-bootstrap import path. |
| TLS DTO | `DatabaseTlsConfigDto` | `caCertificatePath`, `rejectUnauthorized: true`, `serverName`; validates the mounted CA without serializing its bytes. `serverName` is derived only from the validated connection host. |
| Readiness DTO | `DatabaseSchemaReadinessDto` | `state`, `expectedSchemaVersion`, `observedSchemaVersion`, `migrationRequired`, `roleCheck`, `checkedAt`; no URL, user, host, database name, or secret. |
| Modes | `local`, `standalone`, `federated` | `local` is the explicit PGlite-only exception. `standalone` and `federated` are production-like PostgreSQL modes. |
### Frozen names and modes
**No fallback or plaintext:** in production-like modes, missing `DATABASE_URL`, `DATABASE_MIGRATION_URL`, or `DATABASE_TLS_CA_CERT_PATH` is a phase-specific error. No command may substitute a runtime URL, config-file URL, default URL, inferred URL, or hard-coded URL. PostgreSQL URLs must use `sslmode=verify-full` with `rejectUnauthorized: true`; `disable`, `allow`, `prefer`, `require`, `no-verify`, an absent CA, a wrong CA, an unverified certificate, or host/SAN mismatch fails before readiness. PGlite uses only its configured local data directory and explicit PGlite routine; it neither reads nor interprets PostgreSQL URL/TLS variables.
| Item | Exact name / shape | Rule |
| ---------------------- | --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Runtime URL variable | `DATABASE_URL` | PostgreSQL **runtime non-owner** connection. Required for `standalone` and `federated`; runtime-only containers/services receive this variable and never `DATABASE_MIGRATION_URL`. |
| Migration URL variable | `DATABASE_MIGRATION_URL` | PostgreSQL migrator connection. Required only by an explicit migration Job/one-shot operator phase in `standalone` and `federated`; it is forbidden in Gateway runtime service environments. |
| Runtime config DTO | `DatabaseRuntimeConnectionConfigDto` | Internal configuration DTO: `tier`, `databaseUrl`, `runtimeMode`. It maps only `DATABASE_URL`; URL values are never returned, logged, traced, or placed in public DTOs. |
| Migration config DTO | `DatabaseMigrationConnectionConfigDto` | Internal migration-command DTO: `tier`, `migrationDatabaseUrl`, `migrationMode`. It maps only `DATABASE_MIGRATION_URL`; it cannot be constructed by the Gateway runtime bootstrap. |
| Readiness DTO | `DatabaseSchemaReadinessDto` | Sanitized internal/admin health shape: `state`, `expectedSchemaVersion`, `observedSchemaVersion`, `migrationRequired`, `roleCheck`, `checkedAt`. It contains neither URL nor password nor host. |
| TLS variable / DTO | `DATABASE_TLS_CA_CERT_PATH`; `DatabaseTlsConfigDto` | Required in PostgreSQL production-like modes. Internal DTO is `caCertificatePath`, `rejectUnauthorized: true`, `serverName`; it derives server name from the URL host and validates the mounted CA file without serializing its content. |
| Modes | `local`, `standalone`, `federated` | `local` is the PGlite exception. `standalone` and `federated` are PostgreSQL production-like modes and must enforce this contract. |
**ASSUMPTION K101-A1:** `standalone` and `federated` are the complete current PostgreSQL production-like modes. A future PostgreSQL tier inherits this contract until a versioned amendment names its DNS, secrets, bootstrap, and evidence.
**No fallback:** in `standalone` or `federated`, a missing `DATABASE_MIGRATION_URL` is a migration-phase configuration error. The migration command must not fall back to `DATABASE_URL`, a config-file URL, a default URL, or an inferred URL. A missing `DATABASE_URL` or `DATABASE_TLS_CA_CERT_PATH` is a runtime startup error. Both PostgreSQL URLs must be parsed with authenticated TLS configured from `DatabaseTlsConfigDto`; reject `sslmode=disable`, `allow`, `prefer`, `require`, `no-verify`, certificate-unverified configuration, and URL/host mismatch. PGlite uses its configured data directory and its explicit local migration routine; it neither requires nor interprets PostgreSQL connection/TLS variables.
## 2. The sole PostgreSQL DDL control plane
**ASSUMPTION K101-A1:** existing production-like tiers are `standalone` and `federated`, as defined by `MosaicConfig`. If a new PostgreSQL tier is introduced, it inherits the production-like contract until a versioned amendment says otherwise.
`mosaic-db-migrator` is the **only application/CI/test command that may connect with DDL authority to PostgreSQL**. It owns, in one `max: 1` PostgreSQL session: migration-DTO parse, TLS/identity/search-path preflight, advisory-lock acquisition, manifest/ledger reconciliation, migration execution, postflight/readiness verification, lock release, and session close. It rejects `DATABASE_URL`-only invocation **before opening a connection or emitting DDL**. The only exception is an external privileged platform/IaC bootstrap actor (§6), which is not an application command, never receives either application URL, and has its separately audited fixed bootstrap artifact.
## 2. Explicit migration phase and fail-closed readiness
The implementation must inventory and close every present and future entrypoint as follows:
1. Deployment supplies `DATABASE_MIGRATION_URL` only to a dedicated migration command/Job. That command opens one migration session, uses `pg_try_advisory_lock(hashtext('mosaic-schema-migration-v1'))`, returns `DATABASE_MIGRATION_LOCKED` immediately if unavailable, runs the ordered migration set as the migrator role, verifies the final schema version, releases the lock on that same session, and exits.
2. The application Gateway never calls `runMigrations()`, `drizzle-kit migrate`, adapter migration, extension creation, or any schema DDL on PostgreSQL startup. Replica startup can only perform runtime identity and schema readiness checks.
3. KBN-101-03 repairs the current PostgreSQL migration-runner/journal ordering defect before foundation certification: it must use a hash-complete ordered runner, repair the `_journal.json` order, and prove on a clean PostgreSQL database that every shipped migration hash (including `0009`) is applied exactly once. `expectedSchemaVersion` is the deterministic SHA-256 of the resulting ordered shipped Drizzle migration journal entries and migration hashes. `observedSchemaVersion` is calculated from `drizzle.__drizzle_migrations`; equality requires the same ordered complete set—no missing, duplicate, unknown, or out-of-order ledger entry. This is the readiness contract, not a best-effort count.
4. The Gateway starts/advertises readiness only when `DatabaseSchemaReadinessDto.state === 'ready'`, observed equals expected, and the role/grant checks in §4 pass. Any pending, partial, unknown, corrupt, or mismatched schema is `migration_required` or `schema_mismatch`, emits a sanitized operator code, and fails closed before serving requests.
5. Replicas do not wait for or race to migrate. They remain unready until the deployment controller observes a successful migration phase and each replica independently verifies the exact version/role checks. Migration concurrency is prevented by the non-blocking advisory lock; deployment sequencing permits no application replicas before its successful exit.
6. PostgreSQL extension provisioning requiring elevated database privilege (including any provider-specific extension prerequisite) is a separately audited platform/bootstrap step. It is not performed by Gateway runtime or accepted as a reason to grant the migration role superuser.
| Current entrypoint | Required rc.6 disposition |
| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `packages/db/src/migrate.ts:runMigrations()` | Remove its optional URL, `DATABASE_URL`, and default fallback API from public/runtime exports. Its PostgreSQL behavior moves behind `mosaic-db-migrator`; callers cannot invoke it with an arbitrary URL. |
| `packages/db/drizzle.config.ts` and direct `drizzle-kit migrate` | A database-connecting Drizzle configuration reads only `DATABASE_MIGRATION_URL` through the migration DTO and rejects its absence before connection. Replace direct `drizzle-kit migrate` exposure with `mosaic-db-migrator`. `db:generate` is an offline schema artifact command and must neither resolve nor connect to a URL. |
| `pnpm --filter @mosaicstack/db db:migrate` and CI invocation | Make it a thin `mosaic-db-migrator` wrapper; no direct Drizzle migrator invocation remains. CI supplies an isolated disposable migration URL only to that job. |
| `db:push` / direct `drizzle-kit push` | Forbidden for standalone, federated, CI production-like, Portainer, and any URL outside a disposable developer database. If retained for local experimentation, a wrapper requires `MOSAIC_DISPOSABLE_DEVELOPER_DB=1`, a locally allowlisted disposable target, `DATABASE_MIGRATION_URL`, verified TLS when PostgreSQL is used, and rejects `DATABASE_URL`, any production-like tier, and every non-allowlisted host/database before connection. It is never a release, repair, or migration procedure. |
| `mosaic storage migrate --run` | Delegates only to `mosaic-db-migrator`; it rejects `DATABASE_URL`-only execution before spawning or connecting. Its PGlite form remains explicitly local-only. |
| `PostgresAdapter.migrate()`, Gateway `DatabaseModule`/startup, and PostgreSQL adapter factories | Runtime PostgreSQL migration is removed: no `runMigrations`, DDL, `CREATE EXTENSION`, or migration-compatible handle is reachable from startup. Gateway performs read-only identity, TLS, `search_path`, and manifest-ledger readiness checks only. |
| `packages/mosaic/src/commands/fleet-backlog.ts` | PostgreSQL fleet backlog never migrates or creates tables. It consumes a ready runtime connection; PGlite may use only its explicit local migration routine. |
| `packages/storage/src/{adapters/postgres,tier-detection}.ts` extension work and `infra/pg-init/01-extensions.sql` | Runtime probes become read-only catalog/extension-presence checks. Extension provisioning is a fixed external bootstrap prerequisite or a reviewed runner migration where the migrator has the required scoped authority; it is never a probe side effect or a reason to grant runtime database CREATE. |
| `packages/db/src/federation.integration.test.ts` direct type/table/index DDL | Replace with a pre-migrated disposable database created by `mosaic-db-migrator`, or make the test invoke that runner. The test itself has runtime credentials and no direct DDL. |
| `README.md` and `docs/guides/{dev-guide,deployment}.md` operator instructions | `KBN-101-07` replaces direct `db:migrate`, `db:push`, and `CREATE EXTENSION` production-like instructions with the runner/bootstrap procedure, and labels any remaining disposable-local command as non-production with the §2 guard. Documentation is a DDL entrypoint and cannot advertise a bypass. |
| New package scripts, test helpers, setup hooks, CLI commands, installers, CI steps, adapters, or operator docs | A repository check rejects any new PostgreSQL DDL-capable entrypoint or instruction unless it is the dedicated runner or the named external bootstrap artifact. No future script may accept a URL parameter or `DATABASE_URL` as a DDL escape hatch. |
## 3. PostgreSQL role and privilege model
The runner must run the original migration bytes, including shipped `0009`; no migration command may repair a ledger by manual insertion, adoption, `db:push`, or schema diff. Tests requiring PostgreSQL schema consume a pre-migrated disposable database or invoke this exact runner. Every row above has a negative test proving `DATABASE_URL`-only execution fails before connection/DDL; the `db:push` negatives also cover production-like tier and production-like URL rejection.
Role creation, passwords, membership, database ownership, and Vault values are platform/IaC/operator work—not Drizzle/application migrations. Application SQL must not issue `CREATE ROLE`, `ALTER ROLE ... PASSWORD`, `GRANT ... TO LOGIN` credential creation, or embed credentials.
## 3. Exact migration manifest, ledger, and lock
| Role | Attributes and ownership | Membership / session use |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mosaic_platform_database_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; platform-only database owner after bootstrap. It owns the database but is never granted to application roles. | Outside application runtime/migration paths and separately audited. |
| external platform bootstrap actor | Provider/operator/IaC-controlled privileged identity, outside the Mosaic role graph and never placed in Vault/app configuration. | Creates a fresh database and roles, then transfers database ownership to `mosaic_platform_database_owner`; for an existing database it performs the audited ownership/grant transition. It is not used after bootstrap. |
| `mosaic_schema_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; owns only the Mosaic application schema and `drizzle` migration-ledger schema plus their objects. It does **not** own the database and lacks database `CREATE`, `ALTER DATABASE`, and extension-install authority. | Never used as an application login. |
| `mosaic_migrator` | `LOGIN NOINHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; has a separately stored migration credential. | Is granted only `mosaic_schema_owner`. The migration runner verifies `session_user = mosaic_migrator`, executes `SET ROLE mosaic_schema_owner`, and verifies `current_user = mosaic_schema_owner` before DDL. |
| `mosaic_runtime_capability` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; owns no schema/object. | Holds the application table/sequence/schema privileges below. |
| `mosaic_runtime` | `LOGIN INHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; owns no schema/object. | Is granted only `mosaic_runtime_capability WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`; it is never member of either owner/migrator role and cannot `SET ROLE` to them. Its inherited capability grants are effective without changing `current_user`. |
### 3.1 Manifest v1
The external platform bootstrap actor creates the database and role graph, transfers database ownership to `mosaic_platform_database_owner`, creates the application schema and `drizzle` ledger schema, makes `mosaic_schema_owner` their owner, revokes `CREATE` and function `EXECUTE` on both from `PUBLIC`, revokes database-level create/connect/TEMPORARY privileges not explicitly needed (including `REVOKE TEMPORARY ON DATABASE <database> FROM PUBLIC`), and grants `CONNECT` only to the named login roles. The runtime receives `USAGE` on the application schema, `USAGE, SELECT` on required sequences, and table rights only through `mosaic_runtime_capability`. It receives only `USAGE` on `drizzle` plus `SELECT` on `drizzle.__drizzle_migrations`; `INSERT`, `UPDATE`, `DELETE`, and `TRUNCATE` on that ledger are explicitly revoked. The migration role/owner retains schema DDL authority; ordinary runtime has no `CREATE`, `TEMP`, schema ownership, role-administration, extension, or database-administration privilege. Production `search_path` is explicitly `pg_catalog, <mosaic_application_schema>`; current-main `public` is the N-1 schema name until a separately versioned schema relocation is complete. Runtime startup verifies it and rejects untrusted writable schemas before application queries.
The runner generates and verifies a source-controlled **migration manifest v1** from the shipped Drizzle journal and SQL files. A record has exactly:
For ordinary mutable application tables, grant `SELECT, INSERT, UPDATE, DELETE` through `mosaic_runtime_capability`. `ALTER DEFAULT PRIVILEGES FOR ROLE mosaic_schema_owner IN SCHEMA <mosaic_application_schema>` grants that baseline to future ordinary tables and sequence rights to future sequences; it also revokes `EXECUTE ON FUNCTIONS FROM PUBLIC`. The KBN-101 N-1 implementation uses current-main `public` as that schema name until the separately versioned relocation. `SECURITY DEFINER` functions are forbidden by default; a separately versioned reviewed exception must pin search path, revoke PUBLIC execute, grant only the named capability role, and add a negative privilege test. Each migration creating an immutable table must then apply the immutable override in the same migration and verify it:
```text
logicalIndex: non-negative integer from journal array position
journalTag: exact journal `tag` string
migrationSha256: lowercase SHA-256 of the exact migration `.sql` file bytes
```
| KBN immutable relation | Runtime rights | Explicitly absent |
| ----------------------------- | ---------------- | ----------------------------------------------- |
| `task_events` | `SELECT, INSERT` | `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` |
| `artifacts` | `SELECT, INSERT` | `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` |
| `task_checkpoints` | `SELECT, INSERT` | `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` |
| `task_checkpoint_artifacts` | `SELECT, INSERT` | `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` |
| `approval_decision_artifacts` | `SELECT, INSERT` | `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` |
The canonical SQL bytes are the raw Git blob bytes at the signed source-release commit, not workstation checkout bytes. KBN-101-03 adds/validates an LF-pinning `.gitattributes` rule for `packages/db/drizzle/**/*.sql`, generates the source-controlled manifest from those canonical blobs in CI, and makes the runner verify deployed file bytes against the manifest before DDL. No newline, Unicode, whitespace, SQL, or line-ending normalization is applied. Manifest canonical serialization is UTF-8 bytes of:
The named list matches the KBN v1 contracts event/checkpoint/artifact/evidence relations. KBN-100 must retain `RESTRICT` parent deletion semantics; grants alone do not prevent a privileged owner or a cascading FK from erasing evidence. Any future immutable relation must be added to this table in a versioned amendment before code ships.
```text
mosaic-drizzle-manifest-v1\n
<logicalIndex>\t<JSON.stringify(journalTag)>\t<migrationSha256>\n
... in ascending logicalIndex with no omitted index
```
## 4. Effective-identity verification and operator errors
`manifestSha256` is SHA-256 of those canonical bytes. The manifest has no timestamp-derived ordering; `folderMillis`, legacy ledger `id`, and `created_at` are diagnostic only. KBN-101-03 corrects journal **logical** order metadata (including the current `0008`/`0009` anomaly) without changing shipped `0009` bytes.
Before accepting traffic, the runtime uses its already-open `DATABASE_URL` connection to query sanitized identity/privilege metadata (`session_user`, `current_user`, `current_database`, `current_schema`, `pg_has_role`, `has_schema_privilege`, `has_table_privilege`, `rolsuper`, `rolcreaterole`, `rolcreatedb`, `rolbypassrls`) and the migration ledger. It must fail closed when any condition holds:
The runner stores certification in `drizzle.__mosaic_migration_manifest` with exactly one active v1 row (`manifest_version=1`, `manifest_sha256`, `certified_at`). It is owned by `mosaic_schema_owner`; only the migrator after `SET ROLE mosaic_schema_owner` may insert/update it; runtime gets `SELECT` only, and `PUBLIC` gets no schema/table privilege. Existing databases receive this table through the runner after backup/preflight; it is not created by Gateway, a test, or manual SQL.
- `session_user` and `current_user` resolve to the same effective identity as `mosaic_platform_database_owner`, `mosaic_schema_owner`, or `mosaic_migrator`, or runtime can assume any of them;
- the runtime role is a superuser, `CREATEROLE`, `CREATEDB`, `BYPASSRLS`, database/table/schema owner, has database `TEMPORARY`, or has schema/DDL/extension/role-management authority;
- required runtime inherited-capability membership, grants, sequence rights, read-only Drizzle-ledger access, no unexpected PUBLIC/runtime function `EXECUTE`, `search_path`, authenticated-TLS configuration, database/schema identity, or schema readiness are missing/mismatched;
- **after KBN-100 creates the immutable relations**, immutable INSERT/SELECT rights or immutable UPDATE/DELETE denials are missing/mismatched, or an expected immutable relation is absent. The KBN-101 foundation verifies only the role/schema boundary; its post-KBN-100 certification verifies the immutable relation matrix.
### 3.2 Reconciliation and 0009 transition
The migration command performs the reciprocal preflight: it requires the dedicated migration URL, the named migrator session user, schema owner after `SET ROLE`, no superuser/CREATEROLE/database-owner/database-CREATE requirement, the approved database/schema target, and an acquired advisory lock. It fails before DDL otherwise.
The expected ledger is the ordered list of v1 manifest tuples. The runner reads every observed `drizzle.__drizzle_migrations.hash`, maps **each observed hash to exactly one** manifest tuple, and rejects a missing, unknown, duplicate, ambiguous, corrupt, or stale-replica mapping. Equality is tuple-complete: every expected tuple occurs once, no additional tuple occurs, and the manifest digest matches. A count comparison or hash-set comparison is forbidden. Physical legacy insertion `id`, insertion timestamp, and order are expressly non-normative.
Errors have stable sanitized codes: `DATABASE_RUNTIME_URL_REQUIRED`, `DATABASE_MIGRATION_URL_REQUIRED`, `DATABASE_ROLE_UNSAFE`, `DATABASE_ROLE_GRANT_MISMATCH`, `DATABASE_SEARCH_PATH_UNSAFE`, `DATABASE_SCHEMA_MISMATCH`, `DATABASE_MIGRATION_LOCKED`, and `DATABASE_MIGRATION_IDENTITY_UNSAFE`. Operator logs/metrics may contain code, tier, schema-version fingerprints, role **class** (`runtime`/`migrator`/`owner`), and correlation ID—not a DSN, username, host, database name, SQL parameter, secret, or raw catalog result. External health returns unavailable/not-ready without role detail.
For an existing database:
## 5. N-1 rollout, compatibility, and rollback
1. take the KBN-101-07 approved backup and capture a read-only inventory before changing anything;
2. if `0009` is missing **and** its effects are absent, run the original shipped `0009` through `mosaic-db-migrator` and record it normally;
3. if the `0009` hash is present but physically late, accept it when its one-to-one tuple mapping is exact;
4. if an expected hash is missing while its effects are partial or complete, or catalog/ledger evidence conflicts, fail closed as `DATABASE_MIGRATION_RECONCILIATION_AMBIGUOUS`. Recovery is backup restoration or an explicit separately reviewed repair artifact with its own owner, tests, backup, rollback, and approval—not manual ledger insertion/adoption; and
5. write/update the v1 certification row only after exact reconciliation and final catalog/readiness verification.
1. **Prepare (N-1, no production mutation in KBN-101):** publish this contract, enumerate all consumers of `DATABASE_URL`, identify installer/compose/Portainer/CI/Vault handoff owners, and add implementation tests. Existing deployments remain one-URL only until the implementation cards compatibility release.
2. **Compatibility release:** N-1 compatibility preserves the `DATABASE_URL` variable/config shape, not an owner-runtime exception. A current single-URL deployment remains on its prior release until roles are provisioned. The KBN-101 runtime release starts only after `DATABASE_URL` has been changed to the non-owner runtime role; it refuses owner/migrator identity, does not auto-migrate, and never treats that URL as `DATABASE_MIGRATION_URL`.
3. **Provision and migrate:** platform operators create the roles and Vault records, issue distinct URLs, run the explicit migration phase using `DATABASE_MIGRATION_URL`, verify exact schema and grants, then deploy runtime with only `DATABASE_URL`.
4. **Enforce:** after deployed-role certification is green, require non-owner `DATABASE_URL` for production-like runtime. Remove the N-1 compatibility path only in a separately versioned contract/release after fleet inventory proves no single-URL deployment remains.
5. **Rollback:** before any incompatible KBN-100 schema switch, stop/scale runtime replicas, restore the previous runtime release and its known schema-compatible backup/manifest, and re-run only the approved prior migration/restore procedure under the migration identity. Never put the migration URL into the application to make rollback work. After KBN-100 canonical mutation/switch, its existing freeze/reconciliation authority governs data rollback; role separation remains in force.
Required runner tests cover clean, pre-0009, skipped-0009/effects-absent, applied-late, duplicate, unknown, missing, corrupt tuple-pair, partial/full-effect ambiguity, stale replica, backup/restore, and rerun idempotence. The tests prove the `0009` SQL bytes are unchanged and logical journal ordering—not physical ledger order—controls reconciliation.
**ASSUMPTION K101-A2:** a controlled migration Job/one-shot command exists or will be added by the deployment slice. If a target cannot schedule one, its deployment is not eligible for federated/production-like KBN certification.
### 3.3 Fixed advisory-lock namespace
## 6. Secrets, observability, and deployment handoff
Before any preflight that can decide migration state, `mosaic-db-migrator` acquires `pg_try_advisory_lock(1297044289, 1262636593)`. The constants are signed-int4-safe fixed namespace values: class `1297044289` (`MOSA`) and object `1262636593` (`KBN1`). The same `max: 1` session retains it for preflight, reconcile, migrate, verify, release, and close. Failure returns `DATABASE_MIGRATION_LOCKED` immediately; a process crash releases it when the PostgreSQL connection closes. Runtime readiness remains unready while a holder is active and never waits by running migrations. Tests prove concurrent contention, crash/connection-loss release, readiness while held, and non-interference from an unrelated advisory key.
Vault owns three distinct values, with least-privilege policies and no plaintext copies: `secret-{env}/mosaic-stack/database/runtime` (`url`) for the runtime role, `secret-{env}/mosaic-stack/database/migrator` (`url`) for the migration role, and `secret-{env}/mosaic-stack/database/tls-ca` (`certificate`) mounted at `DATABASE_TLS_CA_CERT_PATH` for both authorized PostgreSQL phases. Exact existing mount/service naming must be verified by the deployment owner before creation; this naming is a planned canonical target, not a claim that those paths already exist. CI uses isolated disposable non-production roles/URLs and must model both variables without promoting test credentials.
## 4. PostgreSQL roles, identifiers, and trusted sessions
Rotation is independent: rotate the runtime credential first with overlapping database validity and rolling runtime restart; rotate the migration credential only while no migration is active; rotate the CA only with verified trust overlap and rolling consumers; revoke old material after connection/pipeline evidence. Rotation never downgrades a runtime role or causes a Gateway to receive the migration secret. Secret injection maps runtime secret only to Gateway and migration secret only to migration Job; both are redacted by config/error/logging/OTEL processors. Secret names may be logged; values, DSNs, URI components, and decoded credentials may not.
Role creation, passwords, membership, database ownership, certificates, and Vault values are platform/IaC/operator work—not Drizzle/application migrations. Application SQL must not issue credential/role management statements or embed credentials.
Readiness/metrics distinguish database connectivity, exact schema readiness, identity safety, immutable-grant verification, and migration-phase outcome. They expose codes and versions only. Runbooks must tell operators to inspect secret binding/role grants/schema phase through privileged deployment tooling—not to paste URLs into shell history, issues, logs, or chat.
| Role | Attributes and ownership | Membership / session use |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mosaic_platform_database_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; platform-only database owner after bootstrap. | Never granted to application roles. |
| external platform bootstrap actor | Provider/operator/IaC-controlled privileged identity outside the Mosaic role graph and Vault/application configuration. | Creates/transitions the database and roles, then retires from application use. |
| `mosaic_schema_owner` | `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`; owns only `mosaic` and `drizzle` schemas/objects. | Never an application login; no database/extension authority except reviewed bootstrap scope. |
| `mosaic_migrator` | `LOGIN NOINHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`. | Only `mosaic_schema_owner`; runner verifies `session_user=mosaic_migrator`, `SET ROLE mosaic_schema_owner`, then `current_user=mosaic_schema_owner`. |
| `mosaic_runtime_capability` | `NOLOGIN` and no ownership/administrative attributes. | Holds only named runtime grants. |
| `mosaic_runtime` | `LOGIN INHERIT` with no ownership/administrative attributes. | Only `mosaic_runtime_capability WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`; never owner/migrator member. |
Deployment-specific ownership is deliberately handed off: installer/wizard, local compose, federated compose, Portainer stack, CI, Vault policy/provisioning, and runtime deployment manifests each require a separate one-card/one-PR implementation slice below. KBN-101 performs none of those mutations.
The fixed application/runtime schema is **`mosaic`**. Every application/runtime pooled connection executes and verifies exactly `SET search_path TO pg_catalog, mosaic` before its first application query; connection checkout repeats this after reset/reconnect. Transactional application operations use `SET LOCAL search_path TO pg_catalog, mosaic` and verify it before query execution. `public` and `$user` are forbidden in all runtime paths.
## 7. Implementation DAG and file manifest
The sole runner has one narrowly bounded **legacy-history bootstrap subphase** for the immutable current journal: before it runs, external bootstrap revokes `CREATE` on `public` from `PUBLIC`, assigns its ownership to `mosaic_schema_owner`, and denies all runtime connections. In its locked `max:1` migration session only, after `SET ROLE mosaic_schema_owner`, it uses `SET LOCAL search_path TO pg_catalog, public` solely to execute the shipped historical bytes that contain unqualified objects and explicit `public.*` references. It then, in that same runner/session before manifest certification or any runtime readiness, inventories and relocates every application object to `mosaic`, reconciles extension/type dependencies, regenerates/verifies Drizzle declarations/snapshots, and restores the exact `pg_catalog,mosaic` runner path. This compatibility subphase is not a runtime/operator option, accepts no configuration identifier, has no fallback, and is removed/disabled before KBN-101-08. A clean-PostgreSQL test must prove original historical bytes succeed in this subphase and runtime reaches readiness only with `pg_catalog,mosaic`; existing-database transition follows the same inventory/relocation or fails closed on an unknown dependency. `vector` moves only if relocatable; otherwise every type/function reference is explicitly qualified and its non-writable extension schema is documented. The pre-activation rollback restores the approved backup or reverses only the reviewed relocation artifact before activation; no runtime `search_path` bypass exists.
Each card is one PR and has exclusive ownership. No card absorbs another cards files.
No SQL identifier may come from URL/config/environment/operator input. Catalog comparisons use parameter values. The fixed identifiers above are constants; the external bootstrap artifact alone may use server-side `format('%I', fixed_allowlisted_identifier)` after allowlist validation. Raw client-side interpolation for identifiers, `SET search_path`, database, schema, role, table, or extension names is forbidden. Tests include injection-shaped values, a poisoned pooled-session reset, and transaction `SET LOCAL` restoration negatives.
| Card | Depends on | Exact implementation ownership | Required result |
| --------------------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `KBN-101-00` platform bootstrap / IaC | KBN-101 contract | `infra/pg-init/01-extensions.sql`, new `infra/pg-init/02-kbn101-role-bootstrap.sql`, platform deployment/IaC change set, and KBN-101-07 transition runbook/evidence | External bootstrap actor creates fresh DB/roles then transfers database ownership; existing DB transition inventories owners, creates/rotates roles, transfers schema/table/ledger ownership, applies grants/default privileges, provisions Vault paths, and records reversible evidence. It is not an application migration and no login credential is hardcoded. |
| `KBN-101-01` runtime config/DB boundary | KBN-101-00 | `packages/config/src/mosaic-config.ts`, `.spec.ts`; `packages/db/src/client.ts`, `defaults.ts`, relevant DB tests; `apps/gateway/src/database/database.module.ts` and tests | Typed split DTO/config, authenticated TLS/CA validation, no production defaults/fallback, no PostgreSQL startup migration, readiness/identity verifier seam. |
| `KBN-101-02` storage migration separation | KBN-101-01 | `packages/db/drizzle.config.ts`; `packages/storage/src/adapters/postgres.ts`, `.spec.ts`; `packages/storage/src/{factory,types,cli,migrate-tier,tier-detection}.ts` and focused tests; `apps/gateway/src/main.ts` and startup tests; `packages/mosaic/src/commands/fleet-backlog.ts` and tests | Storage adapter/runtime cannot migrate on Gateway startup; Drizzle Kit/migration-capable commands use only migration DTO/URL and fail if absent; no runtime URL fallback. Replace all runtime `CREATE EXTENSION` probes with read-only extension/catalog checks; prohibit PostgreSQL fleet-backlog auto-migration or route it to the explicit migration command. |
| `KBN-101-03` role foundation + declarative snapshot | KBN-101-00, KBN-101-01 | `packages/db/src/{migrate,index}.ts`, `schema.ts`, `packages/db/drizzle/**`, migration tests, generated Drizzle snapshot/journal | Platform/schema ownership boundary, baseline grants/default privileges/function-execute revocation, ordered hash-complete PostgreSQL migration runner/journal repair, lock/version behavior; clean-PG proof every shipped hash applies exactly once; generated declarations and snapshot exact consistency. It creates no future KBN immutable relation. |
| `KBN-101-04` installer/wizard/config migration | KBN-101-01 | `packages/mosaic/src/stages/gateway-config.ts`, `.spec.ts`, relevant wizard/integration tests; `tools/install.sh` only if it owns emitted variables | Production-like installer/config output stores no PostgreSQL DSN in generated `.env` or `mosaic.config.json`; it records only non-secret deployment/Vault references and accepts injected runtime variables. Tests prove neither runtime nor migration URL is written, echoed, or logged; production-like config loading rejects persisted/config-file DSN fallback. |
| `KBN-101-05` compose/Portainer deployment handoff | KBN-101-00, KBN-101-01, KBN-101-03 | `docker-compose.yml`, `docker-compose.federated.yml`, `deploy/portainer/federated-test.stack.yml`, corresponding deployment docs/runbooks | Dedicated migration phase and secret binding boundary; runtime receives non-owner only. |
| `KBN-101-06` CI/test topology | KBN-101-02, KBN-101-03 | `.woodpecker/ci.yml`, CI-only test helpers, targeted integration tests | Ephemeral owner/migrator/runtime identities; migration then runtime grant/denial evidence; no production credential. |
| `KBN-101-07` Vault/runbook/observability | KBN-101-00, KBN-101-04, KBN-101-05, KBN-101-06 | deployment/Vault/runbook/operations docs and approved observability configuration only | Provision/rotation/incident/rollback instructions, redaction and handoff evidence. |
| `KBN-101-08` foundation certification gate | KBN-101-00…07 | integration evidence only; no opportunistic source scope | Independent security review and deployed runtime role/schema-boundary certificate, including platform-bootstrap fresh/existing transition evidence; no immutable KBN table claim before KBN-100. |
| `KBN-101-09` post-KBN-100 certification gate | KBN-100, KBN-101-08 | integration evidence only; no opportunistic source scope | Real deployed-role INSERT/SELECT and UPDATE/DELETE-denial evidence for every KBN-100 immutable relation; independent security/Ultron approval. |
External bootstrap executes `REVOKE CONNECT, TEMPORARY ON DATABASE <fixed_database> FROM PUBLIC`, then grants `CONNECT` only to `mosaic_runtime`, `mosaic_migrator`, and the time-bounded external bootstrap actor while it is required. Certification fails if an unrelated login retains `CONNECT` or either application login retains `TEMPORARY`. Runtime receives `USAGE` on `mosaic`, named table/sequence grants through `mosaic_runtime_capability`, and `USAGE` on `drizzle` plus `SELECT` only on `drizzle.__drizzle_migrations` and `drizzle.__mosaic_migration_manifest`. `INSERT`, `UPDATE`, `DELETE`, `TRUNCATE`, and DDL rights on ledger/manifest are revoked. Revoke public CREATE and function EXECUTE; `SECURITY DEFINER` is forbidden unless a separately reviewed exception pins trusted path and grants only the capability role. Database TEMPORARY, role management, extension, schema, and object ownership are denied.
`KBN-100` starts only after `KBN-101-08` passes. It must rebase on the resulting main head, restore Drizzle generated declaration/snapshot/journal consistency, and keep its procedural immutable-table grant/trigger/backfill additions bounded to KBN-100s schema migration ownership. It may not create roles, passwords, Vault entries, deployment secrets, or mutate environment wiring. `KBN-101-09` performs real deployed-role certification after those relations exist and must pass before KBN-105.
Immutable KBN relations, after KBN-100 creates them, grant runtime only `SELECT, INSERT` and explicitly deny `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER`: `task_events`, `artifacts`, `task_checkpoints`, `task_checkpoint_artifacts`, and `approval_decision_artifacts`. KBN-100 retains RESTRICT/no-cascade semantics. Foundation certification verifies the role/schema boundary only; post-KBN-100 certification verifies this real deployed-role matrix.
## 8. Threat, tests, and acceptance traceability
## 5. Deployable verified-TLS bootstrap
| Threat / failure | Frozen control | Required evidence |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Gateway role owns tables or runs DDL | Separate runtime/migration URLs; no runtime DDL path, including extension probes and fleet-backlog; owner/migrator checks | Runtime startup tests prove migration/extension calls impossible; deployed runtime catalog certificate. |
| Missing migration secret falls back to runtime URL | Explicit migration DTO requires `DATABASE_MIGRATION_URL` | Unit/integration negative proving absent migration URL fails before connection/DDL. |
| Replica migration race | Dedicated phase plus advisory lock; replicas readiness-only | Concurrent migration invocation: one lock holder, other `DATABASE_MIGRATION_LOCKED`; replicas do no DDL. |
| Role escalation / unsafe grant | Inherited runtime capability with SET/ADMIN denied; no owner membership; catalog privilege preflight fails closed | Superuser/CREATEROLE/owner/SET ROLE/TEMP/ledger/function-EXECUTE mismatch negatives. |
| Evidence rewrite/deletion | KBN-100 immutable grants plus `RESTRICT` FKs | After KBN-100, real `mosaic_runtime` can INSERT/SELECT and receives permission denial for UPDATE/DELETE on every listed relation; owner-only administrative path separately audited. |
| Schema drift / partial migration | Hash-complete ordered ledger fingerprint readiness | Clean-PG every-hash-once, missing/unknown/out-of-order/partial ledger, and stale replica tests fail unready. |
| Dsn/credential leakage / on-path attack | DTO boundary, redaction rules, secret separation, authenticated TLS | Error/log/OTEL snapshots contain no URL/user/host/password; runtime/migrator reject missing/unverified CA, insecure sslmode, and host mismatch; rotation test preserves redaction. |
| Rollback reintroduces owner runtime | Ordered rollback and secret binding controls | Rollback rehearsal verifies restored runtime gets only runtime URL and remains non-owner. |
The **operator/IaC owner** provisions the CA and PostgreSQL leaf key/certificate. Mosaic applications never generate, self-sign, copy, or persist production certificates. The external bootstrap actor receives those secrets only through the deployment secret mechanism; no plaintext development exception exists for production-like modes.
| Acceptance criterion | Evidence needed before KBN-101 closes |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| AC-K101-01 Split connection contract | Exact variable/DTO tests for local, standalone, federated; no production fallback/default. |
| AC-K101-02 Explicit migration/readiness | Clean PostgreSQL every-hash-once runner/journal test; dedicated migration command/lock/version tests; Gateway/replica non-migration test. |
| AC-K101-03 Least-privilege PostgreSQL roles | Catalog assertions for ownership, inherited-only capability with SET/ADMIN denial, search path, ledger read-only grant, TEMP/function-EXECUTE denial, grants/default privileges, unsafe attributes. |
| AC-K101-04 Immutable KBN privilege proof | `KBN-101-09`, after KBN-100, proves deployed `mosaic_runtime` INSERT/SELECT allowed and UPDATE/DELETE denied for every immutable relation. |
| AC-K101-05 N-1 and recovery | Single-URL compatibility diagnostic; staged rollout and rollback rehearsal with no production mutation in this planning card. |
| AC-K101-06 Secrets/operations | Vault/injection/rotation/redaction/runbook review, health/metrics codes, operator failure paths. |
| AC-K101-07 Delivery integrity | One-card/one-PR DAG; strict contract/docs/link checks; independent author≠reviewer security review; terminal-green CI. |
| Material | Vault target / deployment secret | Mount, injection, and authorized consumer |
| --------------------------- | ----------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Runtime URL | `secret-{env}/mosaic-stack/database/runtime` (`url`) → `mosaic-db-runtime-url-v1` | `/run/secrets/mosaic-db-runtime-url`, `0600`, Gateway service UID only; entrypoint maps it to `DATABASE_URL` only at process exec. It is denied to migrator, storage runtime, fleet, ordinary CLI, and every test except an explicit runtime-negative fixture. |
| Migration URL | `secret-{env}/mosaic-stack/database/migrator` (`url`) → `mosaic-db-migrator-url-v1` | `/run/secrets/mosaic-db-migrator-url`, `0600`, one-shot `mosaic-db-migrator` service UID only; entrypoint maps it to `DATABASE_MIGRATION_URL` only at process exec. It is denied to Gateway, storage runtime, fleet, and ordinary CLI. |
| CA bundle | `secret-{env}/mosaic-stack/database/tls-ca` (`certificate`) → `mosaic-db-ca-v1` | Runtime and migrator: `/run/secrets/mosaic-db-ca.crt`, `0600`, service UID; PostgreSQL gets a read-only CA copy only when client-cert validation is enabled. |
| PostgreSQL leaf certificate | `secret-{env}/mosaic-stack/database/postgres-server-tls` (`certificate`) → `mosaic-postgres-server-cert-v1` | PostgreSQL: `/run/secrets/mosaic-postgres-server.crt`, `0600`, postgres UID. |
| PostgreSQL private key | same Vault record (`private_key`) → `mosaic-postgres-server-key-v1` | PostgreSQL: `/run/secrets/mosaic-postgres-server.key`, `0400`, postgres UID, never mounted to Gateway or migrator. |
Compose uses identically named local **secret references**, rendered by an operator-controlled secret provider into a non-repository tmpfs; Swarm declares the named secrets with the same target paths, UID/GID, and modes. KBN-101-05 rejects bind-mounted committed cert/key files, environment-encoded PEM, and a missing secret. The existing target Vault names are planned canonical paths and must be verified/provisioned by the deployment owner; the planning card does not claim they exist.
The server leaf SANs are frozen to actual connection DNS names, not a configurable alias:
| topology | PostgreSQL service DNS names that must be SANs |
| ------------------------------------ | -------------------------------------------------------------------------------------------------- |
| standalone compose | `DNS:postgres`, `DNS:localhost` (only for the documented host-port disposable test path) |
| federated compose | `DNS:postgres-federated`, `DNS:localhost` (only for the documented host-port disposable test path) |
| Portainer/Swarm federated test stack | `DNS:postgres` (the in-stack service endpoint used by Gateway and migrator) |
A new topology requires a versioned amendment before issuance. The PostgreSQL container activation artifact sets `ssl=on`, `ssl_cert_file`, and `ssl_key_file` to those paths, uses a locked-down `postgresql.conf` include, and verifies file ownership/mode before start. The server health/readiness gate makes a `verify-full` CA/SAN-validated connection as the approved runtime/migrator identity; `pg_isready` alone is insufficient. Migration Job starts only after server TLS readiness. Gateway replicas start only after a successful runner result and independently pass verified-TLS, identity, search-path, and ledger readiness.
**Fresh DB:** provision CA/leaf/secrets and server TLS configuration before initial database bootstrap; bootstrap/extension prerequisites run, then the runner migrates over verified TLS, then runtime deploys. **Existing DB:** take the approved backup, provision/mount TLS material, and **drain/scale to zero every N-1 runtime, worker, CLI maintenance process, and replica before TLS enforcement**. Enable server TLS, terminate any residual non-TLS PostgreSQL backend sessions, set `pg_hba.conf` to `hostssl` for all application/migrator CIDRs with no matching `host` rule, reload/restart as required, and prove the non-TLS session count is zero. Only then prove `verify-full` through the existing endpoint, run reconciliation/migration, and roll the non-owner TLS runtime. There is no plaintext transition interval.
**Rotation/rollback:** stage a CA bundle containing old+new trust roots to runtime/migrator, validate a new server leaf with exact SANs, restart PostgreSQL and validate it, roll consumers, then remove the old root only after evidence. Credential rotation remains independent and never mounts migration material into Gateway. Before expiry, rollback restores the prior known-valid leaf/key and overlapping CA bundle, restarts PostgreSQL, enforces `hostssl`, terminates residual non-TLS sessions, and verifies `verify-full`; it never downgrades `sslmode`, restores a plaintext-only N-1 runtime after enforcement, or accepts plaintext. A pre-enforcement abort may restore the backed-up N-1 state only before `hostssl` is enabled and is recorded as an aborted—not activated—release. The runbook records expiry windows, secret versions, backup ID, drained-service/session evidence, activation actor, and validation result—not secret values.
Required disposable tests cover standalone compose and federated/Swarm positives using verified TLS, plus missing CA, wrong CA, wrong SAN, `sslmode` downgrade, missing/mispermissioned server key, runtime-with-migrator-secret, rendered secret-consumer separation, legacy plaintext drain/termination/`hostssl` enforcement, and readiness-before-migration negatives. PGlite local tests are explicitly classified as non-PostgreSQL and do not satisfy a PostgreSQL TLS test.
## 6. Runtime verification and sanitized failures
Before accepting traffic, runtime queries only parameterized/sanitized identity and privilege metadata on its already-open verified-TLS connection. It fails closed for owner/migrator identity or assumability; superuser/CREATEROLE/CREATEDB/BYPASSRLS; object/schema/database ownership; TEMPORARY; unexpected function execute; missing inherited capability/sequence/ledger grants; any non-exact search path; wrong schema/database target; bad TLS; or manifest/ledger mismatch. After KBN-100 it also checks every immutable grant/denial and relation presence.
The runner performs reciprocal preflight under the same session/lock: dedicated migration DTO only, verified TLS, exact allowlisted target, migrator `session_user`, schema-owner `current_user`, exact trusted search path, and no unsafe attributes. It fails before DDL otherwise.
Stable sanitized codes are `DATABASE_RUNTIME_URL_REQUIRED`, `DATABASE_MIGRATION_URL_REQUIRED`, `DATABASE_TLS_REQUIRED`, `DATABASE_TLS_VERIFICATION_FAILED`, `DATABASE_ROLE_UNSAFE`, `DATABASE_ROLE_GRANT_MISMATCH`, `DATABASE_SEARCH_PATH_UNSAFE`, `DATABASE_SCHEMA_MISMATCH`, `DATABASE_MIGRATION_RECONCILIATION_AMBIGUOUS`, `DATABASE_MIGRATION_LOCKED`, and `DATABASE_MIGRATION_IDENTITY_UNSAFE`. Logs/metrics may contain code, tier, manifest fingerprint, role class, and correlation ID only; never DSN, username, host, database name, SQL parameter, secret, or raw catalog result. External health exposes only unavailable/not-ready.
## 7. Safe DAG, activation, and rollback authority
Every KBN-101 card remains one PR with exclusive ownership. Cards `00``07` may merge only as **prepared, inactive capability**: no current owner-runtime deployment consumes their image/config, and no compatibility switch is exposed to a runtime operator. They must not retain `ALLOW_LEGACY_*`, runtime DDL, `DATABASE_URL` migration fallback, plaintext TLS, direct Drizzle, or test-only bypass flags. Current owner-runtime deployments remain on their known N-1 release until final activation.
| Card | Depends on | Exact ownership and required result |
| ----------------------------------------------------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `KBN-101-00` platform bootstrap / IaC | contract | Fixed external bootstrap artifact creates/transitions roles, `mosaic`/`drizzle` ownership, extension prerequisite, Vault secret bindings, server TLS config, fresh/existing inventory, and reversible evidence. It is not an app migration or a runtime command. |
| `KBN-101-01` runtime config/DB boundary | 00 | Typed split DTOs, verified TLS config, fixed schema/session verifier, no PostgreSQL Gateway startup migration. |
| `KBN-101-02` entrypoint closure | 01 | Owns all paths in §2 except runner/journal: storage, CLI, adapters, Gateway startup, tier probes, fleet backlog, and direct-DDL test closure. |
| `KBN-101-03` runner/manifest/schema foundation | 00,01 | Owns `mosaic-db-migrator`, canonical Git-blob/LF manifest v1, fixed lock, journal logical-order repair, runner-only legacy-public bootstrap then `mosaic` relocation, ledger/manifest grants, and PostgreSQL runner tests. Shipped `0009` bytes remain unchanged. |
| `KBN-101-04` installer/wizard/config | 01 | Production-like config persists no DSN and has no fallback; only non-secret references/injected variables. |
| `KBN-101-05` compose/Portainer deployment | 00,01,03 | Owns secret references, TLS server/client mounts, server configuration, Job ordering, readiness, standalone/federated/Swarm TLS tests, and no plaintext exception. |
| `KBN-101-06` CI/test topology | 02,03,05 | Disposable identities and pre-migrated/runner test topology; repository checks reject DDL bypasses and insecure URLs. |
| `KBN-101-07` Vault/runbook/observability | 00,04,05,06 | Activation/rotation/incident/backup/rollback runbook, sanitized observability evidence, and replacement of direct-DDL operator documentation (`README.md`, `docs/guides/dev-guide.md`, `docs/guides/deployment.md`). |
| `KBN-101-08` foundation certification and **atomic activation release** | 00…07 | Independent review verifies all prepared cards and terminal-green CI; Mosaic control plane/Jason authorizes one ordered activation: backup → drain/scale zero all N-1 runtime clients → TLS server start → terminate non-TLS sessions and enforce `hostssl`/no `host` rule → verify no plaintext session → roles → runner → verified readiness → rolling non-owner runtime. Any red test/readiness/CI result aborts; no force-on-red or bypass. Remove/disable all temporary compatibility support before this gate. |
| `KBN-101-09` post-KBN-100 certification | KBN-100,08 | Real deployed-role immutable INSERT/SELECT and UPDATE/DELETE-denial evidence plus independent security/Ultron approval. |
**Authority:** Mosaic control plane/Jason is the sole activation and rollback authority. CI, Gateway, migrator, Coordinator, and Certifier cannot activate, waive a red result, or force release. Before an incompatible KBN-100 switch, the authority stops/scales runtime, uses the approved backup/restore or separately reviewed runner artifact, restores only a known TLS-compatible runtime with its runtime secret after `hostssl` enforcement, and verifies no plaintext sessions plus TLS/readiness. Migration URL is never injected into Gateway to enable rollback. KBN-100 starts only after KBN-101-08; KBN-105 starts only after KBN-101-09.
**ASSUMPTION K101-A2:** every eligible production-like deployment can schedule a dedicated migration Job/one-shot command and an operator/IaC-controlled TLS bootstrap. A target that cannot do both is ineligible for KBN certification.
## 8. Acceptance traceability
| Requirement / acceptance criterion | Required implementation evidence |
| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| K101-REQ-01 / AC-K101-01 | DTO and command matrix covers local/PGlite, standalone, federated; every §2 entrypoint rejects `DATABASE_URL`-only before connection/DDL; no runtime fallback/default. |
| K101-REQ-02 / AC-K101-02 | One-session fixed two-int lock, contention/crash/readiness/unrelated-key tests; manifest v1 canonical bytes/digest and all reconciliation states; Gateway/replica DDL impossibility. |
| K101-REQ-03 / AC-K101-03 | Fresh/existing TLS bootstrap and compose/Swarm verified-TLS positives; CA/SAN/downgrade/key-permission negatives; exact runtime/migrator secret-consumer rendering/CI negatives; catalog role/grant/search-path/identifier/pool-reset tests. |
| K101-REQ-04 / AC-K101-04 | After KBN-100, real deployed runtime INSERT/SELECT success and UPDATE/DELETE denial for each frozen relation, with RESTRICT retention evidence. |
| K101-REQ-05 / AC-K101-05 | Prepared-card/no-intermediate-deploy evidence; one final activation authority record; N-1 drain/zero-plaintext-session/`hostssl`, backup/restore, CA overlap rotation, TLS-only rollback, Vault/redaction, and no-force-on-red evidence. |
| K101-REQ-06 / delivery integrity | One-card/one-PR DAG, exact file ownership, docs/link/contract checks, independent author≠reviewer re-review on the pushed exact head, and terminal-green CI for implementation cards. |
## 9. Non-goals and residual authority
KBN-101 planning does not execute a production role change, create Vault paths, change compose/CI/installer files, alter database objects, or certify any deployed identity. It does not replace KBN-100s data migration, immutable retention design, tenant constraints, or KBN-105 endpoint freeze. A database superuser or break-glass operator remains outside application containment and requires separately audited platform controls, backup/audit evidence, and drills.
KBN-101 planning does not create roles, certificates, Vault paths, migrations, deployment artifacts, or a deployed certificate. It does not replace KBN-100s data migration, immutable retention, tenant constraints, or KBN-105 endpoint freeze. A PostgreSQL superuser/break-glass operator remains outside application containment and requires separate audited platform controls, backup evidence, and drills.

View File

@@ -1,13 +1,21 @@
# Native Kanban/SOT — Remediated Shared Contract v1
**Status:** CONTROL-PLANE rc.5 DB role/connection amendment authorized; prior KCR-001016 and rc.4 SI-001 independent-review decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
**Version:** 1.0.0-rc.5
**Status:** CONTROL-PLANE rc.6 DB role/connection remediation pending independent exact-head re-review; prior KCR-001016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
**Version:** 1.0.0-rc.6
**Date:** 2026-07-15
**Change authority:** Mosaic control plane/Jason only
**SI-001 amendment authority:** `web1:mosaic-100` control-plane decision under issue #753
## Amendment record
### 1.0.0-rc.6 — KBN-101 closed DDL/TLS/ledger activation remediation
- **Choice:** `mosaic-db-migrator` is the sole application/CI/test PostgreSQL DDL control plane. Every legacy entrypoint is routed or denied, rejects `DATABASE_URL`-only before connection/DDL, and `db:push` is unavailable outside an allowlisted disposable developer target. The runner holds one `max:1` session with fixed `pg_try_advisory_lock(1297044289,1262636593)` across preflight through release.
- **Exact ledger:** manifest v1 canonically serializes journal logical index/tag and SHA-256 of exact shipped migration bytes. It maps each observed ledger hash to one tuple; physical insertion order is non-normative, while missing/unknown/duplicate/ambiguous/corrupt/stale states fail closed. Shipped `0009` bytes remain unchanged; a missing/effects-absent `0009` runs normally, an applied-late hash maps normally, and partial/full effects with missing hash require backup restoration or separately reviewed repair—not manual adoption.
- **TLS/search path:** operator/IaC owns CA and server leaf lifecycle, exact compose/Swarm secret mounts, server TLS activation, service-DNS SANs, verified-TLS readiness, transition, CA overlap rotation, and rollback. Runtime/migrator use `verify-full`; PGlite is not PostgreSQL TLS evidence. Application sessions use only `pg_catalog,mosaic`; no URL/config-derived identifier reaches SQL.
- **Safe release:** cards 0007 land prepared but inactive; owner-runtime deployments remain N-1. Mosaic control plane/Jason alone authorizes one atomic TLS/roles → runner → readiness → runtime activation or rollback. No runtime-operator compatibility switch, bypass, plaintext interval, or force-on-red exists; all temporary support is removed before KBN-101-08.
- **Non-effect:** role graph, immutable certification after KBN-100, KBN-105 gate, rc.5s preserved rc.4 SI-001 invariants, and all KCR-001016 decisions remain unchanged. Exact detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
### 1.0.0-rc.5 — KBN-101 role/connection split
- **Choice:** PostgreSQL `standalone` and `federated` runtime uses `DATABASE_URL` only as a non-owner `mosaic_runtime` login; an explicit migration phase uses `DATABASE_MIGRATION_URL` only as `mosaic_migrator`, which `SET ROLE`s to non-login `mosaic_schema_owner` for DDL. Local PGlite remains an explicit embedded exception.
@@ -27,7 +35,7 @@
## 1. Authority
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.5 role/connection separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.6 DDL/ledger/TLS/role separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
## 2. Health proof and exact failures

View File

@@ -95,14 +95,14 @@ No consumer implementation begins before KBN-105. No schema work begins before K
### KBN-101 — PostgreSQL runtime/migration role split and deployed-role certification
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); this documentation contract is frozen, implementation remains held.
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.6 remediates the exact-head `da742ca` REQUEST CHANGES report and awaits independent exact-head re-review. Implementation remains held.
- **Owner:** Mos integration control plane; independently reviewed by security/Ultron.
- **Mode:** SERIAL foundation certificate blocks KBN-100; its post-KBN-100 real immutable-operation certificate blocks KBN-105.
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; `DATABASE_TLS_CA_CERT_PATH` authenticated-TLS contract; PGlite exception; no production/federated migration fallback; explicit migration phase/advisory lock/exact ledger readiness; platform/schema-owner/migrator/runtime roles; ownership, membership, schema/search-path/TEMP/ledger-read/default privilege and immutable grant proof; N-1 rollout/rollback; Vault/rotation/redaction/observability/operator runbooks; deployment-specific handoff; one-card/one-PR implementation DAG.
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; sole `mosaic-db-migrator` PostgreSQL DDL path and all legacy/future entrypoint closure; `DATABASE_TLS_CA_CERT_PATH` plus operator/IaC CA/server-key/cert lifecycle, exact service-DNS SANs, Vault/compose/Swarm mount modes, TLS server/bootstrap/rotation/rollback; PGlite exception; fixed two-int advisory lock; manifest-v1 logical-index/tag/exact-byte-SHA-256 ledger reconciliation including safe `0009`; fixed `mosaic` schema and exact `pg_catalog,mosaic` pooled session path; platform/schema-owner/migrator/runtime roles; ownership, membership, TEMP/ledger-read/default privilege and immutable grant proof; N-1 inactive prepared cards then atomic activation/rollback authority; Vault/redaction/observability/operator runbooks; one-card/one-PR implementation DAG.
- **OUT:** Production mutation in this planning card; KBN-100 tables/data backfill; application API behavior; KBN-105 route/DTO freeze.
- **Depends on:** KBN-010 completed.
- **Contract surfaces:** [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md); `SHARED-CONTRACT.md` rc.5 amendment.
- **Evidence:** foundation: exact variable/DTO/TLS and no-fallback tests; clean-PostgreSQL every-hash-once migration-runner/journal proof; runtime cannot invoke migrations; single non-blocking migration lock/replica readiness; catalog ownership/inherited-capability-with-SET/ADMIN-denied/unsafe-attribute/TEMP/ledger-read/function-EXECUTE/default-grant/search-path checks; N-1 rollout/rollback rehearsal; Vault/redaction/rotation/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
- **Contract surfaces:** [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md); `SHARED-CONTRACT.md` rc.6 amendment.
- **Evidence:** foundation: exact command/DTO entrypoint negatives (including `DATABASE_URL`-only and `db:push` refusal); clean/pre-0009/skipped/applied-late/duplicate/unknown/missing/corrupt/stale/backup runner proof; fixed-lock contention/crash/readiness/unrelated-key tests; runtime cannot invoke migrations/DDL; disposable standalone and federated/Swarm verified-TLS positives plus CA/SAN/downgrade, URL-secret consumer-isolation, and legacy-drain/`hostssl` zero-plaintext negatives; catalog ownership/inherited-capability-with-SET/ADMIN-denied/unsafe-attribute/TEMP/ledger-read/function-EXECUTE/default-grant/fixed-search-path/pool-reset/identifier checks; N-1/atomic TLS-only rollback/no-force-on-red rehearsal; Vault/CA-overlap/redaction/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
### KBN-100 — Unified Drizzle schema and concrete N-1 migration