docs(kbn): freeze KBN-101 database role split contract (#774)
This commit was merged in pull request #774.
This commit is contained in:
13
CLAUDE.md
13
CLAUDE.md
@@ -26,13 +26,14 @@ pnpm test # Vitest (all packages)
|
|||||||
pnpm build # Build all packages
|
pnpm build # Build all packages
|
||||||
|
|
||||||
# Database
|
# Database
|
||||||
pnpm --filter @mosaicstack/db db:push # Push schema to PG (dev)
|
pnpm --filter @mosaicstack/db db:generate # Offline migration artifact generation only
|
||||||
pnpm --filter @mosaicstack/db db:generate # Generate migrations
|
# PostgreSQL execution is held until KBN-101-00/-03/-05 land. Do not invoke a runner,
|
||||||
pnpm --filter @mosaicstack/db db:migrate # Run migrations
|
# init SQL, or Compose PostgreSQL service from this checkout.
|
||||||
|
|
||||||
# Dev
|
# Dev: local PGlite data-layer work needs no PostgreSQL. Optional local queue service only:
|
||||||
docker compose up -d # Start PG, Valkey, OTEL, Jaeger
|
docker compose up -d valkey
|
||||||
pnpm --filter @mosaicstack/gateway exec tsx src/main.ts # Start gateway
|
# Do not start Gateway/Web or root pnpm dev as a local PGlite route: the current unguarded dotenv
|
||||||
|
# loader can inherit a daemon PostgreSQL DSN. KBN-101-02 must make that state fail closed first.
|
||||||
```
|
```
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|||||||
44
README.md
44
README.md
@@ -157,7 +157,12 @@ mosaic storage status
|
|||||||
mosaic storage tier
|
mosaic storage tier
|
||||||
mosaic storage export
|
mosaic storage export
|
||||||
mosaic storage import
|
mosaic storage import
|
||||||
mosaic storage migrate
|
# Schema migration is unavailable in this release. The current storage wrapper shells
|
||||||
|
# directly to `pnpm --filter @mosaicstack/db db:migrate`; it is legacy N-1,
|
||||||
|
# uncertified, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||||
|
# Future schema migration is non-operative: external bootstrap → TLS/roles → runner
|
||||||
|
# --run → runner --verify → readiness. Tier copy uses only the separately held secure
|
||||||
|
# migrate-tier route.
|
||||||
```
|
```
|
||||||
|
|
||||||
### Telemetry
|
### Telemetry
|
||||||
@@ -192,29 +197,32 @@ Consent state is persisted in config. Remote upload is a no-op until you run `mo
|
|||||||
git clone git@git.mosaicstack.dev:mosaicstack/stack.git
|
git clone git@git.mosaicstack.dev:mosaicstack/stack.git
|
||||||
cd stack
|
cd stack
|
||||||
|
|
||||||
# Start infrastructure (Postgres, Valkey, Jaeger)
|
# Install dependencies. The local tier uses in-process PGlite; leave DATABASE_URL unset.
|
||||||
docker compose up -d
|
|
||||||
|
|
||||||
# Install dependencies
|
|
||||||
pnpm install
|
pnpm install
|
||||||
|
|
||||||
# Run migrations
|
# Optional local queue service only. This does not start PostgreSQL.
|
||||||
pnpm --filter @mosaicstack/db run db:migrate
|
docker compose up -d valkey
|
||||||
|
|
||||||
# Start all services in dev mode
|
# The current Gateway/Web local process is held; see docs/guides/dev-guide.md.
|
||||||
pnpm dev
|
# Do not start it until KBN-101-02 makes inherited dotenv/DSN state fail closed.
|
||||||
```
|
```
|
||||||
|
|
||||||
### Infrastructure
|
### Held future procedure
|
||||||
|
|
||||||
Docker Compose provides:
|
The checked-in Compose PostgreSQL service mounts legacy initialization SQL and is **not** a
|
||||||
|
current PostgreSQL, standalone, or federated developer route. Do not start it with Compose,
|
||||||
|
invoke initialization SQL, or treat the planned migrator as currently executable.
|
||||||
|
|
||||||
| Service | Port | Purpose |
|
**Held future activation procedure — non-operative and no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05
|
||||||
| --------------------- | --------- | ---------------------- |
|
land:** external bootstrap → TLS/roles → `mosaic-db-migrator --run` →
|
||||||
| PostgreSQL (pgvector) | 5433 | Primary database |
|
`mosaic-db-migrator --verify` → Gateway/Compose readiness. The future deployment artifacts—not
|
||||||
| Valkey | 6380 | Task queue + caching |
|
this README—will provide the reviewed commands and secret-consumer interface.
|
||||||
| Jaeger | 16686 | Distributed tracing UI |
|
|
||||||
| OTEL Collector | 4317/4318 | Telemetry ingestion |
|
For local data-layer work, PGlite needs no PostgreSQL service. The optional Compose command above
|
||||||
|
starts only Valkey; OTEL Collector and Jaeger may likewise be started individually if needed,
|
||||||
|
without starting PostgreSQL. A Gateway/Web local process is not currently a safe PGlite route:
|
||||||
|
its unguarded dotenv loader may inherit a daemon PostgreSQL DSN. Do not use root `pnpm dev` or a
|
||||||
|
Gateway start command until KBN-101-02 makes that state fail closed.
|
||||||
|
|
||||||
### Quality Gates
|
### Quality Gates
|
||||||
|
|
||||||
@@ -231,7 +239,7 @@ pnpm format # Prettier auto-fix
|
|||||||
Woodpecker CI runs on every push:
|
Woodpecker CI runs on every push:
|
||||||
|
|
||||||
- `pnpm install --frozen-lockfile`
|
- `pnpm install --frozen-lockfile`
|
||||||
- Database migration against a fresh Postgres
|
- **Legacy N-1 CI status only — active, uncertified, and non-authorizing as an operator route:** the checked-in job currently invokes `pnpm --filter @mosaicstack/db run db:migrate` with `DATABASE_URL` against an isolated disposable PostgreSQL CI database. It performs direct DDL in that CI database, is not approved ordinary behavior or an operator route, and remains a known exception pending KBN-101-06 removal/replacement by the certified runner-backed CI path.
|
||||||
- `pnpm test` (Turbo-orchestrated across all packages)
|
- `pnpm test` (Turbo-orchestrated across all packages)
|
||||||
|
|
||||||
npm packages are published to the Gitea package registry on main merges.
|
npm packages are published to the Gitea package registry on main merges.
|
||||||
|
|||||||
@@ -149,15 +149,9 @@ for any `<Image>` components added in the future.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## How to Apply
|
## Held future procedure
|
||||||
|
|
||||||
```bash
|
This report is non-operative evidence, not a current runbook. Until **KBN-101-00, KBN-101-03, and KBN-101-05** land, do not execute a PostgreSQL runner from this checkout. The approved future procedure is exactly: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Deployment will supply the reviewed runner, migration-only credentials, and TLS material; Gateway startup only verifies readiness.
|
||||||
# Run the DB migration (requires a live DB)
|
|
||||||
pnpm --filter @mosaicstack/db exec drizzle-kit migrate
|
|
||||||
|
|
||||||
# Or, in Docker/Swarm — migrations run automatically on gateway startup
|
|
||||||
# via runMigrations() in packages/db/src/migrate.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
37
docs/PRD.md
37
docs/PRD.md
@@ -125,6 +125,35 @@ are defined in [docs/TASKS.md](./TASKS.md) and must remain one card/one PR.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## KBN-101 Database Runtime/Migration Role Split (#771)
|
||||||
|
|
||||||
|
### Problem and objective
|
||||||
|
|
||||||
|
PostgreSQL Gateway/storage currently uses one `DATABASE_URL` for runtime queries and migrations. That makes the deployed application identity an owner and prevents certification that KBN immutable event, artifact, checkpoint, and evidence relations reject runtime `UPDATE`/`DELETE`. KBN-101 freezes a least-privilege runtime/migration split before KBN-100 schema work.
|
||||||
|
|
||||||
|
### Normative requirements
|
||||||
|
|
||||||
|
1. `K101-REQ-01`: `DATABASE_URL` SHALL be the non-owner PostgreSQL runtime connection and `DATABASE_MIGRATION_URL` SHALL be the migration-only owner/migrator connection. They are required respectively for runtime and the dedicated `mosaic-db-migrator --run|--verify` phase in `standalone`/`federated`; local PGlite is the explicit exception. The published `@mosaicstack/db` bin maps exactly `mosaic-db-migrator` to `./dist/cli.js`, its image entrypoint is exactly `mosaic-db-migrator`, accepts no URL/SQL/schema/role argv, and returns stable sanitized exits. Every current/future PostgreSQL DDL entrypoint SHALL route to that runner or be denied, and SHALL reject `DATABASE_URL`-only execution before connection/DDL. Data migration may connect only after the runner prepares and verifies the PostgreSQL target, through dedicated non-DDL `mosaic_data_importer` and exactly `--target-url-file /run/secrets/mosaic-migrate-target-url`, its fixed paired authenticated provider-version file `/run/secrets/mosaic-migrate-target-version`, plus `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`. KBN-101-05 obtains URL key `url` and version only from the same successful Vault KV-v2 response at `secret-{env}/mosaic-stack/database/importer` (`data.metadata.version`), renders them as one immutable generation into separate consumer copies, and never infers a provider version from DSN bytes. The trusted runner verifies TLS/identity/manifest, reads its fixed importer URL/version copies only for binding through safe no-follow fd checks, and signs a credential-free JCS/Ed25519 attestation using its runner-only fixed root-owned private-key file; no signing key reaches importer/runtime. The artifact binds secret version and SHA-256 of exact high-entropy credential-file bytes, canonical TLS host/port/database, CA/SPKI, PostgreSQL system identifier/database OID, importer role, manifest/schema fingerprints, producer invocation/build/image digest, issued/expires/nonce, and correlation. Before target connection the importer validates URL/version/attestation/public-key files, signature/key/expiry/replay/authenticated provider version/digest/generation/bindings and the importer-only CA at exact `DATABASE_TLS_CA_CERT_PATH`; after verified TLS and before DML it validates server/database/role/CA/schema identity, with same-fd/in-memory-byte TOCTOU protection, rotation/revocation, a privileged producer-only-to-importer-only artifact handoff controller that verifies/copies/fsyncs/atomically renames/seals before importer start, consumer isolation/no logging-oracle, and sanitized errors. Raw `--target-url`, `DATABASE_URL` fallback, runtime-owner use, missing/unsafe/substituted files, stale/replayed/tampered/wrong-key attestation, wrong binding, and DDL attempt fail before target connection/DDL; post-connect mismatch closes with zero DML/DDL. A reviewed finite classifier inventories executable current source/scripts/package bins, operator docs, deploy manifests, and exact normative contracts by path; active secure records pin both options/files, producer/key/bindings/tests, while normative contracts cannot mask instructions. Unknown active commands, duplicate-owner, ownerless, missing-path, and historical/status-only masking hits fail. `db:push` is forbidden outside an explicitly disposable local developer database and cannot accept a production-like URL.
|
||||||
|
2. `K101-REQ-02`: Gateway runtime/replicas SHALL not execute migrations or DDL. The runner SHALL hold one `max:1` session and fixed two-int advisory namespace `1297044289` (`MOSA`), `1262636593` (`KBN1`) across preflight, reconciliation, migration, verification, and release. It SHALL compare the versioned canonical manifest v1 tuple (journal logical index/tag plus exact SQL-byte SHA-256) to the complete observed ledger mapping; count/set-only, timestamps, and physical insertion order are non-normative and insufficient.
|
||||||
|
3. `K101-REQ-03`: PostgreSQL SHALL separate non-login platform database owner, non-login schema owner, dedicated `NOLOGIN SUPERUSER` `mosaic_extension_owner`, login migrator, dedicated login non-DDL data importer, non-login runtime capability, and login runtime roles. For PostgreSQL 17 + pgvector 0.8.2, `vector` is untrusted (`trusted` is absent and `relocatable=true`): only an externally controlled audited platform-bootstrap superuser session may `SET ROLE mosaic_extension_owner` for CREATE/UPDATE/SET SCHEMA, then `RESET ROLE`; the role has `rolcanlogin=false`, `rolsuper=true`, zero members, no runtime credential/Vault secret, and is never provided to app containers. It owns `mosaic_extensions`, fresh `vector`, and owner-bearing extension members, while `mosaic_schema_owner` receives only `USAGE` for type resolution and never ownership/`CREATE`/`ALTER`/`DROP`/member-change/default-privilege authority there. Superuser cannot be constrained by `GRANT`/`REVOKE`; this is identity/non-login/no-membership/external-control/audit isolation, not a false least-privilege claim. Extension operations require control-plane change, independent review, backup/rollback, maintenance window, and audit evidence. Managed targets that cannot establish this exact role are ineligible until an independently approved versioned provider-owned extension-owner profile exists; app/migrator ownership is never silently retained. Existing approved-owner extension relocation validates exact `pg_namespace.nspowner`, `pg_extension.extowner`, member ownership/schema/version, while legacy runtime-owned extension fails closed to a controlled shadow-database migration—never unsupported ownership alteration, catalog mutation, ownership adoption, or `DROP CASCADE`. Runtime, migrator, schema owner, importer, and all service roles must fail `SET ROLE`, catalog/direct `ALTER`/`UPDATE`/`DROP`/membership-change denial, role ownership, superuser/role-creation/schema-creation/TEMPORARY, unsafe membership, untrusted search path, missing grants, unauthenticated TLS, and immutable privilege drift checks. Application schema is fixed `mosaic` with exact `pg_catalog,mosaic` session path; historical public migrations remain byte-immutable legacy bootstrap only, every future Drizzle application declaration targets `mosaic`, and `vector` is explicitly qualified from non-writable `mosaic_extensions`. No config-derived SQL identifier is permitted.
|
||||||
|
4. `K101-REQ-04`: `mosaicstack/stack` KBN-101-00 SHALL exclusively own `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, and bootstrap tests; KBN-101-05 SHALL exclusively own `tools/db/render-postgres-secrets.ts`, its tests, and current Compose/Portainer/two-gateway deployment declarations, consuming the versioned bootstrap interface without overlap. Environment IaC/Vault is named input and Mosaic deployment control plane/Jason is activation authority. Distinct runtime/migrator/importer URL, importer authenticated provider-version, DB-client CA, Gateway leaf, and PostgreSQL server key/certificate materials are provisioned before a production-like database starts. Importer and migrator have separate immutable URL/version copies at fixed `10002:10002`/`10003:10003` identities; runtime/unrelated containers receive neither importer material, attestation private key, or importer artifact. Runtime, migrator, and importer require their mounted CA plus `sslmode=verify-full`. Exact UID/GID/mode/rendering, service-DNS SANs, Vault/compose/Swarm consumer isolation, two-gateway pair ordering, server activation, pre-enforcement legacy-client drain and `hostssl` zero-plaintext-session proof, fresh/existing transition, CA-overlap rotation, TLS-only rollback, and standalone/federated/Swarm/two-gateway positive/negative TLS evidence are required. No application-generated production certificate or plaintext bootstrap exception is permitted.
|
||||||
|
5. `K101-REQ-05`: KBN immutable relations SHALL permit the real runtime role INSERT/SELECT only and deny UPDATE/DELETE; parent retention remains RESTRICT/no-cascade. Role/password/Vault creation is external platform control, never application migration/source.
|
||||||
|
6. `K101-REQ-06`: N-1 single-URL compatibility, rollout/rollback, Vault ownership/rotation/redaction, CI, installer, compose/Portainer, observability, and deployment handoffs SHALL be separately bounded one-card/one-PR work. Prepared slices remain inactive while current owner-runtime deployments stay N-1; Mosaic control plane/Jason alone authorizes one final atomic activation or rollback, with no force-on-red/bypass. KBN-101 planning itself SHALL not mutate production.
|
||||||
|
7. `K101-REQ-07`: KBN-100 SHALL begin only after the KBN-101 foundation role/schema-boundary certificate; it SHALL rebase on that main head, restore generated Drizzle declaration/snapshot/journal consistency, and bound procedural immutable-table grant/trigger/backfill additions to its schema slice. KBN-101 real deployed-role immutable-operation certification SHALL complete after KBN-100 creates those relations and before KBN-105.
|
||||||
|
|
||||||
|
### Acceptance criteria
|
||||||
|
|
||||||
|
1. `AC-K101-01`: DTO/command-matrix tests prove required modes, PGlite exception, `mosaic-db-migrator --help|--run|--verify`/stable exits/argv refusal, public-import negative, every finite classified DDL/static-bypass inventory path and both harness pairs reject `DATABASE_URL`-only before connection/DDL, no migration-to-runtime fallback, and `db:push` refusal outside an allowlisted disposable DB. Before inventory, ownership, or status masking, the semantic fixture fails README's exact former commented code-fence generic-wrapper form and the user guide's exact former executable generic-wrapper form; source-consistency proves current `packages/storage/src/cli.ts` directly `execSync`s `pnpm --filter @mosaicstack/db db:migrate` and no `mosaic-db-migrator` bin exists, so runner-delegation documentation fails. The active `docs/guides/migrate-tier.md` route is inventoried to KBN-101-07 and proves runner-produced `--target-url-file /run/secrets/mosaic-migrate-target-url`, fixed paired provider-version file, and `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`; runner-only signing/private-key isolation; Vault KV-v2 same-response version provenance, separate immutable generation mounts, importer CA, JCS/Ed25519 signature/key rotation/revocation, atomic artifact, expiry/replay, safe-fd secret-version/digest, canonical TLS/CA/server/database/role/manifest/schema bindings, dedicated non-DDL importer, consumer isolation/no log-oracle, and exact no-connection versus zero-DML rejection for missing/wrong/stale/replayed/tampered/wrong-key/substituted/generation-mismatched inputs. The full current non-normative docs inventory—including user guide, federation historical task/MILESTONES status, and non-operative SETUP—has an exact safe disposition. Scanner semantic checks reject automatic first-boot/startup extension/schema/migration wording, Compose-up-before-runner, init-script authority, production `.env`/monorepo auto-load/`EnvironmentFile=`/credential-export-or-argv/restart-as-secret-activation routes, and every unqualified operator-document `mosaic-db-migrator --run|--verify` hit regardless of named/normative/status classification. The exact former README/dev/deployment Compose-first sequences, former SETUP wording, exact former MILESTONES wording `pgvector extension installed + verified on startup`, former architecture-plan/PERFORMANCE/backlog runner routes, and any unqualified runner fixture fail before inventory masking. Only one `Held future procedure` Markdown section—bounded through the next equal-or-higher heading—may contain the explicit non-operative/no-current-command-authority form that names KBN-101-00/-03/-05 and preserves external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness; every runner hit outside that section fails. The README assertion for the checked-in direct CI `pnpm --filter @mosaicstack/db run db:migrate` with `DATABASE_URL` passes only as active legacy N-1, uncertified, non-authorizing-as-an-operator-route status against an isolated disposable CI database pending KBN-101-06 removal—not as an ordinary operator or approved DDL-authority route. Only local PGlite data-layer work or non-PostgreSQL Compose is current (Gateway/Web local startup is held pending daemon/inherited/project-DSN rejection).
|
||||||
|
2. `AC-K101-02`: Fixed namespace lock contention/crash/readiness/non-interference and exact manifest-v1 reconciliation tests prove no replica race/runtime auto-migration and fail closed on every missing/unknown/duplicate/ambiguous/corrupt/stale ledger state.
|
||||||
|
3. `AC-K101-03`: Actual PostgreSQL 17 + pgvector 0.8.2 control-file, catalog, Drizzle-generation, vector-query/operator, fresh/approved-owner/legacy-shadow/partial/resume/rollback/N-1, and real deployed-role tests prove `trusted` absent/untrusted plus relocatability, external-superuser `SET ROLE` create/update/`RESET ROLE` audit, exact `rolcanlogin=false`/`rolsuper=true`/zero-membership/no-runtime-secret state, platform/schema/extension-owner/migrator/importer/runtime separation, `pg_extension.extowner` plus owner-bearing extension-member/schema/version assertions, and runtime/migrator/schema-owner/importer/all-service-role `SET ROLE`/ALTER/DROP/member-update denial. They also prove `pg_catalog,mosaic` per-session pool safety, `mosaic_extensions` qualification, identifier injection denial, ownership/membership/ledger-read/TEMP/default grants, and unsafe privilege denial.
|
||||||
|
4. `AC-K101-04`: Disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus for both pairs missing CA/wrong CA/wrong SAN/sslmode downgrade, server/Gateway key mode, UID/GID, secret-consumer isolation, and legacy-drain/`hostssl` negatives prove server bootstrap, ordering, and readiness; PGlite is expressly excluded from this PostgreSQL evidence.
|
||||||
|
5. `AC-K101-05`: Real runtime-role evidence proves INSERT/SELECT succeeds and UPDATE/DELETE fails for every frozen immutable KBN relation.
|
||||||
|
6. `AC-K101-06`: N-1/atomic activation/rollback, Vault/CA-overlap rotation/redaction, health/operator behavior, CI/deployment handoff, independent exact-head security review, and terminal-green CI evidence the foundation before KBN-100; after KBN-100, the real deployed-role immutable-operation certificate and Ultron approval release KBN-105.
|
||||||
|
|
||||||
|
**Normative implementation contract:** [`docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md`](./native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md). `ASSUMPTION:` existing `standalone` and `federated` are all PostgreSQL production-like modes; any new PostgreSQL tier inherits these requirements until an explicit versioned amendment.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Tess Interaction Agent Workstream (TESS)
|
## Tess Interaction Agent Workstream (TESS)
|
||||||
|
|
||||||
### Problem and Objective
|
### Problem and Objective
|
||||||
@@ -1082,10 +1111,10 @@ Telegram remote control channel.
|
|||||||
|
|
||||||
### AC-10: Deployment
|
### AC-10: Deployment
|
||||||
|
|
||||||
- [ ] `docker compose up` starts full stack from clean state
|
- [ ] PGlite data-layer work uses no PostgreSQL; optional Compose services are selected individually and do not start PostgreSQL; Gateway/Web local start remains held until KBN-101-02 rejects daemon/inherited/project DSNs before connection or DDL
|
||||||
- [ ] `mosaic` CLI installable and functional on bare metal
|
- [ ] PostgreSQL/federated activation is unavailable until KBN-101-00/-03/-05 deliver external bootstrap, TLS/roles, runner `--run`, runner `--verify`, and Gateway/Compose readiness in that order
|
||||||
- [ ] Database migrations run automatically on first start
|
- [ ] `mosaic` CLI installable and functional on bare metal after the reviewed KBN-101-05 secret-renderer/process-exec or `LoadCredential` interface exists
|
||||||
- [ ] `.env.example` documents all required configuration
|
- [ ] Local-only configuration documentation is distinct from production generation-pinned Vault-rendered consumer material
|
||||||
|
|
||||||
### AC-11: @mosaicstack/\* Packages
|
### AC-11: @mosaicstack/\* Packages
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,10 @@
|
|||||||
- [Workstream index](native-kanban-sot/INDEX.md) — artifact map, lane partition, and delivery order.
|
- [Workstream index](native-kanban-sot/INDEX.md) — artifact map, lane partition, and delivery order.
|
||||||
- [Mission manifest](native-kanban-sot/MISSION-MANIFEST.md) — scope, authority, invariants, and gate model.
|
- [Mission manifest](native-kanban-sot/MISSION-MANIFEST.md) — scope, authority, invariants, and gate model.
|
||||||
- [Task decomposition](native-kanban-sot/TASKS.md) — dependency-ordered implementation slices and ownership boundaries.
|
- [Task decomposition](native-kanban-sot/TASKS.md) — dependency-ordered implementation slices and ownership boundaries.
|
||||||
|
- [KBN-101 database role split](native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md) — rc.16 direct-Drizzle storage-wrapper hold: legacy N-1/uncertified/non-operative pending -02/-03/-06/-08; exact README/user-guide wrapper forms fail before masking and source-consistency rejects runner-delegation copy; held bootstrap → TLS/roles → run → verify → readiness; plus prior attestation, pgvector owner, classifier, TLS, activation, and certification prerequisite.
|
||||||
|
- [Federated tier data migration](guides/migrate-tier.md) — active KBN-101-07 operator route: runner-produced target attestation, dedicated non-DDL importer, and paired credential-/attestation-file references only.
|
||||||
- [Frozen shared contract](native-kanban-sot/SHARED-CONTRACT.md) — schema, API, Coordinator, health, recovery, and migration contracts.
|
- [Frozen shared contract](native-kanban-sot/SHARED-CONTRACT.md) — schema, API, Coordinator, health, recovery, and migration contracts.
|
||||||
|
- [KBN-101 exact-head security review](reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) — retained prior REQUEST CHANGES evidence for `da742ca`; rc.16 awaits independent exact-head re-review after closing the current generic storage-wrapper authority HIGH finding.
|
||||||
- [Initial independent review](reports/native-kanban-sot/canon-initial-review-no-go.md) — KCR-001–016 findings that blocked the first draft.
|
- [Initial independent review](reports/native-kanban-sot/canon-initial-review-no-go.md) — KCR-001–016 findings that blocked the first draft.
|
||||||
- [Final independent re-review](reports/native-kanban-sot/canon-final-rereview-go.md) — closure evidence and GO verdict.
|
- [Final independent re-review](reports/native-kanban-sot/canon-final-rereview-go.md) — closure evidence and GO verdict.
|
||||||
- [Ultron final gate](reports/native-kanban-sot/ultron-final-go.md) — final requirements, authority, schema, migration, recovery, and evidence review.
|
- [Ultron final gate](reports/native-kanban-sot/ultron-final-go.md) — final requirements, authority, schema, migration, recovery, and evidence review.
|
||||||
|
|||||||
@@ -70,6 +70,10 @@ export function createQueue(config?: QueueConfig): QueueHandle {
|
|||||||
|
|
||||||
### `@mosaicstack/db` (packages/db/src/client.ts)
|
### `@mosaicstack/db` (packages/db/src/client.ts)
|
||||||
|
|
||||||
|
> **Historical design specimen — status-only, not an operator instruction.** KBN-101 supersedes
|
||||||
|
> this pre-split `DATABASE_URL` fallback shape; it cannot authorize runtime migration, DDL, or a
|
||||||
|
> connection-string fallback. See the KBN-101 runner/role contract for the produced interface.
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';
|
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';
|
||||||
import postgres from 'postgres';
|
import postgres from 'postgres';
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ Every milestone adds tests to these layers. A milestone cannot be claimed comple
|
|||||||
- Add `"tier": "federated"` to `mosaic.config.json` schema and validators
|
- Add `"tier": "federated"` to `mosaic.config.json` schema and validators
|
||||||
- Docker Compose `federated` profile (`docker-compose.federated.yml`) adds: Postgres+pgvector (5433), Valkey (6380), dedicated volumes
|
- Docker Compose `federated` profile (`docker-compose.federated.yml`) adds: Postgres+pgvector (5433), Valkey (6380), dedicated volumes
|
||||||
- Tier detector in gateway bootstrap: reads config, asserts required services reachable, refuses to start otherwise
|
- Tier detector in gateway bootstrap: reads config, asserts required services reachable, refuses to start otherwise
|
||||||
- `pgvector` extension installed + verified on startup
|
- **Historical/status only:** the prior startup-provisioning statement is superseded. Runtime/startup extension provisioning is forbidden. PostgreSQL activation remains non-operative with no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land; this record authorizes no current DDL, Compose/init, or startup path.
|
||||||
- Migration logic: safe upgrade path from `local`/`standalone` → `federated` (data export/import script, one-way)
|
- Migration logic: safe upgrade path from `local`/`standalone` → `federated` (data export/import script, one-way)
|
||||||
- `mosaic doctor` reports tier + service health
|
- `mosaic doctor` reports tier + service health
|
||||||
- Gateway continues to serve as a normal standalone instance (no federation yet)
|
- Gateway continues to serve as a normal standalone instance (no federation yet)
|
||||||
|
|||||||
@@ -1,280 +1,74 @@
|
|||||||
# Federated Tier Setup Guide
|
# Federated Tier Setup Guide
|
||||||
|
|
||||||
## What is the federated tier?
|
> **KBN-101 N-1 hold:** This page is **non-operative** and grants no current command
|
||||||
|
> authority until KBN-101-00, KBN-101-03, and KBN-101-05 land and KBN-101-08 activates a
|
||||||
|
> reviewed release. It does not authorize a deployment operation, initialization artifacts,
|
||||||
|
> implicit extension/schema/migration creation, raw `CREATE`, direct database initialization, or
|
||||||
|
> a Gateway against an unverified database. The prior direct-start wording is retired; its
|
||||||
|
> regression fixture is owned by KBN-101-06.
|
||||||
|
|
||||||
The federated tier is designed for multi-user and multi-host deployments. It consists of PostgreSQL 17 with pgvector extension (for embeddings and RAG), Valkey for distributed task queueing and caching, and a shared configuration across multiple Mosaic gateway instances. Use this tier when running Mosaic in production or when scaling beyond a single-host deployment.
|
## Held future procedure
|
||||||
|
|
||||||
## Prerequisites
|
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||||
|
|
||||||
- Docker and Docker Compose installed
|
The deployment control plane—not an operator shell or deployment lifecycle hook—performs this
|
||||||
- Ports 5433 (PostgreSQL) and 6380 (Valkey) available on your host (or adjust environment variables)
|
exact held future sequence after activation authorization: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||||
- At least 2 GB free disk space for data volumes
|
|
||||||
|
|
||||||
## Start the federated stack
|
1. External bootstrap provisions the approved database/extension prerequisites.
|
||||||
|
2. TLS/roles are installed through the generation-pinned renderer.
|
||||||
|
3. The dedicated one-shot runner executes `mosaic-db-migrator --run`.
|
||||||
|
4. The same runner executes `mosaic-db-migrator --verify`, including readiness and the
|
||||||
|
importer-target attestation where that route is enabled.
|
||||||
|
5. Only after successful verification may Gateway reach its independent verified-TLS Gateway
|
||||||
|
readiness gate.
|
||||||
|
|
||||||
Run the federated overlay:
|
No step may be reordered, skipped, replaced by a raw SQL command, or delegated to an initialization
|
||||||
|
hook.
|
||||||
|
A missing extension, schema, migration, role, secret generation, or readiness proof is a failed
|
||||||
|
control-plane precondition; it is not an instruction to start Compose, retry startup, or create
|
||||||
|
anything directly.
|
||||||
|
|
||||||
```bash
|
## N-1 status and required disposition
|
||||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
This starts PostgreSQL 17 with pgvector and Valkey 8. The pgvector extension is created automatically on first boot.
|
The current branch retains historical federation artifacts, but they are not a deployable
|
||||||
|
procedure. `docs/federation/TASKS.md` records their shipped status only. KBN-101-02 retires
|
||||||
|
runtime/init DDL; KBN-101-05 owns the renderer/deployment handoff; KBN-101-06 verifies the
|
||||||
|
finite scanner and command matrix; and KBN-101-07 owns this operator route. A path named in an
|
||||||
|
inventory, a historical-status label, or a normative requirement cannot suppress the semantic
|
||||||
|
checks above.
|
||||||
|
|
||||||
Verify the services are running:
|
Until the activation certificate names an exact release, use no database startup or recovery
|
||||||
|
command from this document. For the produced importer interface, see
|
||||||
|
[the federated tier migration contract](../guides/migrate-tier.md); it is likewise non-operative
|
||||||
|
until activation.
|
||||||
|
|
||||||
```bash
|
## Federation and Step-CA reference
|
||||||
docker compose -f docker-compose.federated.yml ps
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output shows `postgres-federated` and `valkey-federated` both healthy.
|
Federation uses PostgreSQL 17 with pgvector, Valkey, and a shared configuration across multiple
|
||||||
|
Gateway instances. Step-CA issues federation peer X.509 certificates whose custom OIDs carry a
|
||||||
## Configure mosaic for federated tier
|
grant and subject identity. The following facts are reference material only; provisioning and
|
||||||
|
secret delivery remain deployment-control-plane work under the activation sequence.
|
||||||
Create or update your `mosaic.config.json`:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"tier": "federated",
|
|
||||||
"database": "postgresql://mosaic:mosaic@localhost:5433/mosaic",
|
|
||||||
"queue": "redis://localhost:6380"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
If you're using environment variables instead:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export DATABASE_URL="postgresql://mosaic:mosaic@localhost:5433/mosaic"
|
|
||||||
export REDIS_URL="redis://localhost:6380"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verify health
|
|
||||||
|
|
||||||
Run the health check:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mosaic gateway doctor
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output (green):
|
|
||||||
|
|
||||||
```
|
|
||||||
Tier: federated Config: mosaic.config.json
|
|
||||||
✓ postgres localhost:5433 (42ms)
|
|
||||||
✓ valkey localhost:6380 (8ms)
|
|
||||||
✓ pgvector (embedded) (15ms)
|
|
||||||
```
|
|
||||||
|
|
||||||
For JSON output (useful in CI/automation):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mosaic gateway doctor --json
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 2: Step-CA Bootstrap
|
|
||||||
|
|
||||||
Step-CA is a certificate authority that issues X.509 certificates for federation peers. In Mosaic federation, it signs peer certificates with custom OIDs that embed grant and user identities, enforcing authorization at the certificate level.
|
|
||||||
|
|
||||||
### Prerequisites for Step-CA
|
|
||||||
|
|
||||||
Before starting the CA, you must set up the dev password:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cp infra/step-ca/dev-password.example infra/step-ca/dev-password
|
|
||||||
# Edit dev-password and set your CA password (minimum 16 characters)
|
|
||||||
```
|
|
||||||
|
|
||||||
The password is required for the CA to boot and derive the provisioner key used by the gateway.
|
|
||||||
|
|
||||||
### Start the Step-CA service
|
|
||||||
|
|
||||||
Add the step-ca service to your federated stack:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f docker-compose.federated.yml --profile federated up -d step-ca
|
|
||||||
```
|
|
||||||
|
|
||||||
On first boot, the init script (`infra/step-ca/init.sh`) runs automatically. It:
|
|
||||||
|
|
||||||
- Generates the CA root key and certificate in the Docker volume
|
|
||||||
- Creates the `mosaic-fed` JWK provisioner
|
|
||||||
- Applies the X.509 template from `infra/step-ca/templates/federation.tpl`
|
|
||||||
|
|
||||||
The volume is persistent, so subsequent boots reuse the existing CA keys.
|
|
||||||
|
|
||||||
Verify the CA is healthy:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl https://localhost:9000/health --cacert /tmp/step-ca-root.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
(If the root cert file doesn't exist yet, see the extraction steps below.)
|
|
||||||
|
|
||||||
### Extract credentials for the gateway
|
|
||||||
|
|
||||||
The gateway requires two credentials from the running CA:
|
|
||||||
|
|
||||||
**1. Provisioner key (for `STEP_CA_PROVISIONER_KEY_JSON`)**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec $(docker ps -qf name=step-ca) cat /home/step/secrets/mosaic-fed.json > /tmp/step-ca-provisioner.json
|
|
||||||
```
|
|
||||||
|
|
||||||
This JSON file contains the JWK public and private keys for the `mosaic-fed` provisioner. Store it securely and pass its contents to the gateway via the `STEP_CA_PROVISIONER_KEY_JSON` environment variable.
|
|
||||||
|
|
||||||
**2. Root certificate (for `STEP_CA_ROOT_CERT_PATH`)**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker cp $(docker ps -qf name=step-ca):/home/step/certs/root_ca.crt /tmp/step-ca-root.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
This PEM file is the CA's root certificate, used to verify peer certificates issued by step-ca. Pass its path to the gateway via `STEP_CA_ROOT_CERT_PATH`.
|
|
||||||
|
|
||||||
### Custom OID Registry
|
|
||||||
|
|
||||||
Federation certificates include custom OIDs in the certificate extension. These encode authorization metadata:
|
|
||||||
|
|
||||||
| OID | Name | Description |
|
| OID | Name | Description |
|
||||||
| ------------------- | ---------------------- | --------------------- |
|
| ------------------- | ------------------------ | --------------------- |
|
||||||
| 1.3.6.1.4.1.99999.1 | mosaic_grant_id | Federation grant UUID |
|
| 1.3.6.1.4.1.99999.1 | `mosaic_grant_id` | Federation grant UUID |
|
||||||
| 1.3.6.1.4.1.99999.2 | mosaic_subject_user_id | Subject user UUID |
|
| 1.3.6.1.4.1.99999.2 | `mosaic_subject_user_id` | Subject user UUID |
|
||||||
|
|
||||||
These OIDs are verified by the gateway after the CSR is signed, ensuring the certificate was issued with the correct grant and user context.
|
The internal arc `1.3.6.1.4.1.99999` is development-only. Before an externally reachable
|
||||||
|
production deployment, register an IANA Private Enterprise Number and version the assignments.
|
||||||
|
Each value is DER-encoded as an ASN.1 UTF8String containing the UUID.
|
||||||
|
|
||||||
### Environment Variables
|
The future activated Gateway requires `STEP_CA_URL`, `STEP_CA_PROVISIONER_PASSWORD`,
|
||||||
|
`STEP_CA_PROVISIONER_KEY_JSON`, `STEP_CA_ROOT_CERT_PATH`, and `BETTER_AUTH_SECRET` through the
|
||||||
|
reviewed secret mechanism. These names do not authorize shell exports, copied credential files,
|
||||||
|
or an ad hoc service start.
|
||||||
|
|
||||||
Configure the gateway with the following environment variables before startup:
|
## Failure disposition
|
||||||
|
|
||||||
| Variable | Required | Description |
|
- A TLS, CA, SAN, role, runner, or readiness failure is a control-plane incident. Preserve only
|
||||||
| ------------------------------ | -------- | --------------------------------------------------------------------------------------------------------- |
|
sanitized evidence and follow the approved rollback/repair record.
|
||||||
| `STEP_CA_URL` | Yes | Base URL of the step-ca instance, e.g. `https://step-ca:9000` (use `https://localhost:9000` in local dev) |
|
- A pgvector/extension failure is a failed external-bootstrap or runner precondition. Do not use
|
||||||
| `STEP_CA_PROVISIONER_KEY_JSON` | Yes | JSON-encoded JWK from `/home/step/secrets/mosaic-fed.json` |
|
direct extension SQL, init artifacts, or a startup retry as remediation.
|
||||||
| `STEP_CA_ROOT_CERT_PATH` | Yes | Absolute path to the root CA certificate (e.g. `/tmp/step-ca-root.crt`) |
|
- A port, container, or Valkey problem does not permit bypassing the activation sequence.
|
||||||
| `BETTER_AUTH_SECRET` | Yes | Secret used to seal peer private keys at rest; already required for M1 |
|
- Federation peer-key rotation remains deferred until its separately approved migration plan;
|
||||||
|
do not rotate `BETTER_AUTH_SECRET` without that plan.
|
||||||
Example environment setup:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export STEP_CA_URL="https://localhost:9000"
|
|
||||||
export STEP_CA_PROVISIONER_KEY_JSON="$(cat /tmp/step-ca-provisioner.json)"
|
|
||||||
export STEP_CA_ROOT_CERT_PATH="/tmp/step-ca-root.crt"
|
|
||||||
export BETTER_AUTH_SECRET="<your-secret>"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Port conflicts
|
|
||||||
|
|
||||||
**Symptom:** `bind: address already in use`
|
|
||||||
|
|
||||||
**Fix:** Stop the base dev stack first:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose down
|
|
||||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Or change the host port with an environment variable:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
PG_FEDERATED_HOST_PORT=5434 VALKEY_FEDERATED_HOST_PORT=6381 \
|
|
||||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### pgvector extension error
|
|
||||||
|
|
||||||
**Symptom:** `ERROR: could not open extension control file`
|
|
||||||
|
|
||||||
**Fix:** pgvector is created at first boot. Check logs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f docker-compose.federated.yml logs postgres-federated | grep -i vector
|
|
||||||
```
|
|
||||||
|
|
||||||
If missing, exec into the container and create it manually:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec <postgres-federated-id> psql -U mosaic -d mosaic -c "CREATE EXTENSION vector;"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Valkey connection refused
|
|
||||||
|
|
||||||
**Symptom:** `Error: connect ECONNREFUSED 127.0.0.1:6380`
|
|
||||||
|
|
||||||
**Fix:** Check service health:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f docker-compose.federated.yml logs valkey-federated
|
|
||||||
```
|
|
||||||
|
|
||||||
If Valkey is running, verify your firewall allows 6380. On macOS, Docker Desktop may require binding to `host.docker.internal` instead of `localhost`.
|
|
||||||
|
|
||||||
## Key rotation (deferred)
|
|
||||||
|
|
||||||
Federation peer private keys (`federation_peers.client_key_pem`) are sealed at rest using AES-256-GCM with a key derived from `BETTER_AUTH_SECRET` via SHA-256. If `BETTER_AUTH_SECRET` is rotated, all sealed `client_key_pem` values in the database become unreadable and must be re-sealed with the new key before rotation completes.
|
|
||||||
|
|
||||||
The full key rotation procedure (decrypt all rows with old key, re-encrypt with new key, atomically swap the secret) is out of scope for M2. Operators must not rotate `BETTER_AUTH_SECRET` without a migration plan for all sealed federation peer keys.
|
|
||||||
|
|
||||||
## OID Assignments — Mosaic Internal OID Arc
|
|
||||||
|
|
||||||
Mosaic uses the private enterprise arc `1.3.6.1.4.1.99999` for custom X.509
|
|
||||||
certificate extensions in federation grant certificates.
|
|
||||||
|
|
||||||
**IMPORTANT:** This is a development/internal OID arc. Before deploying to a
|
|
||||||
production environment accessible by external parties, register a proper IANA
|
|
||||||
Private Enterprise Number (PEN) at <https://pen.iana.org/pen/PenApplication.page>
|
|
||||||
and update these assignments accordingly.
|
|
||||||
|
|
||||||
### Assigned OIDs
|
|
||||||
|
|
||||||
| OID | Symbolic name | Description |
|
|
||||||
| --------------------- | --------------------------------- | --------------------------------------------------------- |
|
|
||||||
| `1.3.6.1.4.1.99999.1` | `mosaic.federation.grantId` | UUID of the `federation_grants` row authorising this cert |
|
|
||||||
| `1.3.6.1.4.1.99999.2` | `mosaic.federation.subjectUserId` | UUID of the local user on whose behalf the cert is issued |
|
|
||||||
|
|
||||||
### Encoding
|
|
||||||
|
|
||||||
Each extension value is DER-encoded as an ASN.1 **UTF8String**:
|
|
||||||
|
|
||||||
```
|
|
||||||
Tag 0x0C (UTF8String)
|
|
||||||
Length 0x24 (36 decimal — fixed length of a UUID string)
|
|
||||||
Value <36 ASCII bytes of the UUID>
|
|
||||||
```
|
|
||||||
|
|
||||||
The step-ca X.509 template at `infra/step-ca/templates/federation.tpl`
|
|
||||||
produces this encoding via the Go template expression:
|
|
||||||
|
|
||||||
```
|
|
||||||
{{ printf "\x0c\x24%s" .Token.mosaic_grant_id | b64enc }}
|
|
||||||
```
|
|
||||||
|
|
||||||
The resulting base64 value is passed as the `value` field of the extension
|
|
||||||
object in the template JSON.
|
|
||||||
|
|
||||||
### CA Environment Variables
|
|
||||||
|
|
||||||
The `CaService` (`apps/gateway/src/federation/ca.service.ts`) requires the
|
|
||||||
following environment variables at gateway startup:
|
|
||||||
|
|
||||||
| Variable | Required | Description |
|
|
||||||
| ------------------------------ | -------- | -------------------------------------------------------------------- |
|
|
||||||
| `STEP_CA_URL` | Yes | Base URL of the step-ca instance, e.g. `https://step-ca:9000` |
|
|
||||||
| `STEP_CA_PROVISIONER_PASSWORD` | Yes | JWK provisioner password for the `mosaic-fed` provisioner |
|
|
||||||
| `STEP_CA_PROVISIONER_KEY_JSON` | Yes | JSON-encoded JWK (public + private) for the `mosaic-fed` provisioner |
|
|
||||||
| `STEP_CA_ROOT_CERT_PATH` | Yes | Absolute path to the step-ca root CA certificate PEM file |
|
|
||||||
|
|
||||||
Set these variables in your environment or secret manager before starting
|
|
||||||
the gateway. In the federated Docker Compose stack they are expected to be
|
|
||||||
injected via Docker secrets and environment variable overrides.
|
|
||||||
|
|
||||||
### Fail-loud contract
|
|
||||||
|
|
||||||
The CA service (and the X.509 template) are designed to fail loudly if the
|
|
||||||
custom OIDs cannot be embedded:
|
|
||||||
|
|
||||||
- The template produces a malformed extension value (zero-length UTF8String
|
|
||||||
body) when the JWT claims `mosaic_grant_id` or `mosaic_subject_user_id` are
|
|
||||||
absent. step-ca rejects the CSR rather than issuing a cert without the OIDs.
|
|
||||||
- `CaService.issueCert()` throws a `CaServiceError` on every error path with
|
|
||||||
a human-readable `remediation` string. It never silently returns a cert that
|
|
||||||
may be missing the required extensions.
|
|
||||||
|
|||||||
@@ -16,12 +16,12 @@
|
|||||||
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
||||||
|
|
||||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
||||||
| FED-M1-02 | done | Author `docker-compose.federated.yml` as an overlay profile: Postgres 17 + pgvector extension (port 5433), Valkey (6380), named volumes, healthchecks. Compose-up should boot cleanly on a clean machine. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471. Overlay defines `postgres-federated`/`valkey-federated`, profile-gated, with pg-init for pgvector extension. |
|
| FED-M1-02 | done | Historical shipped-status record: authored a federated Compose overlay with PostgreSQL/pgvector, Valkey, volumes, and healthchecks. It is not a current startup, init, extension, schema, or migration procedure. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471 status only. KBN-101-02 retires its init authority; KBN-101-05 replaces deployment rendering; KBN-101-07 SETUP is non-operative until activation. |
|
||||||
| FED-M1-03 | done | Add pgvector support to `packages/storage/src/adapters/postgres.ts`: create extension on init (idempotent), expose vector column type in schema helpers. No adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472. `enableVector` flag on postgres StorageConfig; idempotent CREATE EXTENSION before migrations. |
|
| FED-M1-03 | done | Historical shipped-status record: add pgvector support to `packages/storage/src/adapters/postgres.ts`; no adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472 status only. **KBN-101 supersedes this behavior:** it cannot authorize current runtime extension creation or any DDL; only the runner/external bootstrap contract may do so. |
|
||||||
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
||||||
| FED-M1-05 | done | Write `scripts/migrate-to-federated.ts`: one-way migration from `local` (PGlite) / `standalone` (PG without pgvector) → `federated`. Dumps, transforms, loads; dry-run + confirm UX. Idempotent on re-run. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474. `mosaic storage migrate-tier`; DrizzleMigrationSource (corrects P0 found in review); 32 tests; idempotent. |
|
| FED-M1-05 | done | Historical shipped-status record: prior tier migration implementation. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474 status only. **KBN-101 supersedes this route:** it cannot authorize current credentials, target connection, or DDL. The future active route requires runner verification plus target URL-file and signed attestation-file binding. |
|
||||||
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
||||||
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
||||||
| FED-M1-08 | done | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Shipped in PR #477. Caught P0 in M1-05 (camelCase→snake_case) missed by mocked unit tests; fix in same PR. |
|
| FED-M1-08 | done | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Shipped in PR #477. Caught P0 in M1-05 (camelCase→snake_case) missed by mocked unit tests; fix in same PR. |
|
||||||
|
|||||||
@@ -15,8 +15,8 @@ The backlog uses the existing Mosaic storage layer; there is **no** new database
|
|||||||
engine (no sqlite, no raw client).
|
engine (no sqlite, no raw client).
|
||||||
|
|
||||||
| Condition | Tier | Data location |
|
| Condition | Tier | Data location |
|
||||||
| ------------------------------ | -------------------- | -------------------------------- |
|
| ---------------------------------- | -------------------- | ---------------------------------------------------------------- |
|
||||||
| `DATABASE_URL` set | Full server Postgres | the configured database |
|
| `DATABASE_URL` injected at runtime | Full server Postgres | the verified runtime database; it never authorizes migration/DDL |
|
||||||
| `PGLITE_DATA_DIR` set (no URL) | Embedded PGlite | that directory |
|
| `PGLITE_DATA_DIR` set (no URL) | Embedded PGlite | that directory |
|
||||||
| neither (default) | Embedded PGlite | `~/.config/mosaic/fleet/backlog` |
|
| neither (default) | Embedded PGlite | `~/.config/mosaic/fleet/backlog` |
|
||||||
|
|
||||||
@@ -24,8 +24,7 @@ PGlite is real Postgres semantics in-process — including the row locks the ato
|
|||||||
claim relies on — so the **same code** runs on a laptop (embedded, single-host
|
claim relies on — so the **same code** runs on a laptop (embedded, single-host
|
||||||
default) and on a full Postgres deployment. Switching tiers is config-only.
|
default) and on a full Postgres deployment. Switching tiers is config-only.
|
||||||
|
|
||||||
The schema (`backlog` table) is created automatically on first CLI use:
|
For embedded PGlite only, the local backlog routine may prepare its local schema on first use. **Current operator behavior is PGlite-only.** The PostgreSQL path is held until KBN-101 activation; no current PostgreSQL CLI route, runner, or first-use migration is available or authorized. A future activated PostgreSQL runtime may connect only after its separately certified readiness gate.
|
||||||
`runMigrations()` for Postgres, `runPgliteMigrations()` for embedded PGlite.
|
|
||||||
|
|
||||||
### Update safety
|
### Update safety
|
||||||
|
|
||||||
|
|||||||
@@ -224,9 +224,9 @@ external clients. Authentication requires a valid BetterAuth session (cookie or
|
|||||||
### Required
|
### Required
|
||||||
|
|
||||||
| Variable | Description |
|
| Variable | Description |
|
||||||
| -------------------- | ----------------------------------------------------------------------------------------- |
|
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||||
| `BETTER_AUTH_SECRET` | Secret key for BetterAuth session signing. Must be set or gateway will not start. |
|
| `BETTER_AUTH_SECRET` | Secret key for BetterAuth session signing. Must be set or gateway will not start. |
|
||||||
| `DATABASE_URL` | PostgreSQL connection string. Default: `postgresql://mosaic:mosaic@localhost:5433/mosaic` |
|
| `DATABASE_URL` | Runtime-only PostgreSQL connection injected from the dedicated deployment secret; no default or inline DSN. |
|
||||||
|
|
||||||
### Gateway
|
### Gateway
|
||||||
|
|
||||||
|
|||||||
@@ -1,384 +1,67 @@
|
|||||||
# Deployment Guide
|
# Deployment Guide
|
||||||
|
|
||||||
This guide covers deploying Mosaic in two modes: **Docker Compose** (recommended for quick setup) and **bare-metal** (production, full control).
|
> **Status: non-operative for PostgreSQL, federated, and bare-metal production.** The checked-in
|
||||||
|
> Compose PostgreSQL service mounts legacy initialization SQL and the KBN-101 bootstrap, runner,
|
||||||
|
> secret-renderer, and process-exec interfaces do not exist yet. This page does not authorize a
|
||||||
|
> production deployment, database initialization, manual DDL, secret provisioning, or service
|
||||||
|
> activation.
|
||||||
|
|
||||||
---
|
## Current safe local route
|
||||||
|
|
||||||
## Prerequisites
|
Use PGlite only for current in-process data-layer work; it requires no PostgreSQL. A Gateway/Web
|
||||||
|
local process is held because its unguarded dotenv loader can inherit a daemon PostgreSQL DSN and
|
||||||
| Dependency | Minimum version | Notes |
|
reach runtime DDL. If a local queue service is useful, start only Valkey:
|
||||||
| ---------------- | --------------- | ---------------------------------------------- |
|
|
||||||
| Node.js | 22 LTS | Required for ESM + `--experimental-vm-modules` |
|
|
||||||
| pnpm | 9 | `npm install -g pnpm` |
|
|
||||||
| PostgreSQL | 17 | Must have the `pgvector` extension |
|
|
||||||
| Valkey | 8 | Redis-compatible; Redis 7+ also works |
|
|
||||||
| Docker + Compose | v2 | For the Docker Compose path only |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Docker Compose Deployment (Quick Start)
|
|
||||||
|
|
||||||
The `docker-compose.yml` at the repository root starts PostgreSQL 17 (with pgvector), Valkey 8, an OpenTelemetry Collector, and Jaeger.
|
|
||||||
|
|
||||||
### 1. Clone and configure
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone <repo-url> mosaic
|
docker compose up -d valkey
|
||||||
cd mosaic
|
|
||||||
cp .env.example .env
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Edit `.env`. The minimum required change is:
|
This command intentionally does not start PostgreSQL. Do not run a broad Compose start, use its
|
||||||
|
PostgreSQL initialization mount, infer that current Compose is a production/federated route, or
|
||||||
|
start Gateway/Web until KBN-101-02 supplies fail-closed local-tier/DSN isolation.
|
||||||
|
|
||||||
```dotenv
|
## Held future procedure
|
||||||
BETTER_AUTH_SECRET=<output of: openssl rand -base64 32>
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Start infrastructure services
|
PostgreSQL local, federated, Compose, and bare-metal production activation are held until these
|
||||||
|
artifacts land and pass their independent gates:
|
||||||
|
|
||||||
```bash
|
1. **KBN-101-00** external privileged bootstrap artifact;
|
||||||
docker compose up -d
|
2. **KBN-101-03** sole `mosaic-db-migrator` runner and verified-readiness artifact; and
|
||||||
```
|
3. **KBN-101-05** Vault/secret-renderer-backed deployment and consumer-isolation artifact.
|
||||||
|
|
||||||
Services and their ports:
|
The required future order is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||||
|
|
||||||
| Service | Default port |
|
This is a held, non-operative future activation specification with no current command authority. Do not invoke the named
|
||||||
| --------------------- | ------------------------ |
|
runner, start PostgreSQL, or substitute a Compose/init/manual-SQL route until the owned artifacts
|
||||||
| PostgreSQL | `localhost:5433` |
|
are implemented and reviewed.
|
||||||
| Valkey | `localhost:6380` |
|
|
||||||
| OTEL Collector (HTTP) | `localhost:4318` |
|
|
||||||
| OTEL Collector (gRPC) | `localhost:4317` |
|
|
||||||
| Jaeger UI | `http://localhost:16686` |
|
|
||||||
|
|
||||||
Override host ports via `PG_HOST_PORT` and `VALKEY_HOST_PORT` in `.env` if the defaults conflict.
|
## Future production secret and unit boundary (schematic only)
|
||||||
|
|
||||||
### 3. Install dependencies
|
No current bare-metal production unit or command is published. KBN-101-05 must supply a reviewed,
|
||||||
|
generation-pinned Vault renderer and a process-exec or systemd `LoadCredential` interface before
|
||||||
|
production units can exist. The interface must preserve these exact consumer boundaries:
|
||||||
|
|
||||||
```bash
|
| Consumer | May receive | Must never receive |
|
||||||
pnpm install
|
| ----------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||||
```
|
| Gateway/runtime | Its own runtime URL and DB client CA at process exec | Migrator URL, importer URL/version, attestation material, signing key, PostgreSQL private key |
|
||||||
|
| One-shot migrator | Its own migration URL, DB client CA, and runner-only signing capability | Runtime URL, importer consumer copy, Gateway/private PostgreSQL keys |
|
||||||
|
| Data importer | Its own immutable URL/version copies, importer CA, pinned public key, and sealed attestation | Runtime/migrator URLs, signing key, shared writable mount |
|
||||||
|
| PostgreSQL | Its own server certificate/key and only its approved server material | Application, migrator, importer, or Gateway secrets |
|
||||||
|
|
||||||
### 4. Initialize the database
|
A future unit specification is non-executable until KBN-101-05 supplies it. It must obtain
|
||||||
|
credentials through the renderer’s Vault generation and process-exec/`LoadCredential` boundary;
|
||||||
|
it must not place credentials in a production environment file, a monorepo auto-load path, a shell
|
||||||
|
export, command arguments, logs, or a manual secret-activation lifecycle instruction. Rotation and
|
||||||
|
process replacement semantics must be delivered by the reviewed renderer/interface with generation,
|
||||||
|
consumer-isolation, mode/owner, and no-mixed-generation evidence—not improvised in this guide.
|
||||||
|
|
||||||
```bash
|
## Readiness and troubleshooting status
|
||||||
pnpm --filter @mosaicstack/db db:migrate
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Build all packages
|
Until the future procedure is implemented, do not diagnose PostgreSQL with ad hoc SQL, connection
|
||||||
|
strings, or initialization scripts. The future sanitized runner-verification readiness artifact is
|
||||||
|
the required PostgreSQL readiness authority after its bootstrap/TLS prerequisites pass.
|
||||||
|
For local PGlite development, diagnose application behavior without introducing a PostgreSQL
|
||||||
|
connection.
|
||||||
|
|
||||||
```bash
|
Non-database local services may be inspected with their ordinary local health/log tools. Those
|
||||||
pnpm build
|
checks do not certify PostgreSQL, federated deployment, or production readiness.
|
||||||
```
|
|
||||||
|
|
||||||
### 6. Start the gateway
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/gateway dev
|
|
||||||
```
|
|
||||||
|
|
||||||
Or for production (after build):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node apps/gateway/dist/main.js
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7. Start the web app
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Development
|
|
||||||
pnpm --filter @mosaicstack/web dev
|
|
||||||
|
|
||||||
# Production (after build)
|
|
||||||
pnpm --filter @mosaicstack/web start
|
|
||||||
```
|
|
||||||
|
|
||||||
The web app runs on port `3000` by default.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Bare-Metal Deployment
|
|
||||||
|
|
||||||
Use this path when you want to manage PostgreSQL and Valkey yourself (e.g., existing infrastructure, managed cloud databases).
|
|
||||||
|
|
||||||
### Step 1 — Install system dependencies
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Node.js 22 via nvm
|
|
||||||
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
|
|
||||||
nvm install 22
|
|
||||||
nvm use 22
|
|
||||||
|
|
||||||
# pnpm
|
|
||||||
npm install -g pnpm
|
|
||||||
|
|
||||||
# PostgreSQL 17 with pgvector (Debian/Ubuntu example)
|
|
||||||
sudo apt-get install -y postgresql-17 postgresql-17-pgvector
|
|
||||||
|
|
||||||
# Valkey
|
|
||||||
# Follow https://valkey.io/download/ for your distribution
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2 — Create the database
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- Run as the postgres superuser
|
|
||||||
CREATE USER mosaic WITH PASSWORD 'change-me';
|
|
||||||
CREATE DATABASE mosaic OWNER mosaic;
|
|
||||||
\c mosaic
|
|
||||||
CREATE EXTENSION IF NOT EXISTS vector;
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3 — Clone and configure
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone <repo-url> /opt/mosaic
|
|
||||||
cd /opt/mosaic
|
|
||||||
cp .env.example .env
|
|
||||||
```
|
|
||||||
|
|
||||||
Edit `/opt/mosaic/.env`. Required fields:
|
|
||||||
|
|
||||||
```dotenv
|
|
||||||
DATABASE_URL=postgresql://mosaic:<password>@localhost:5432/mosaic
|
|
||||||
VALKEY_URL=redis://localhost:6379
|
|
||||||
BETTER_AUTH_SECRET=<openssl rand -base64 32>
|
|
||||||
BETTER_AUTH_URL=https://your-domain.example.com
|
|
||||||
GATEWAY_CORS_ORIGIN=https://your-domain.example.com
|
|
||||||
NEXT_PUBLIC_GATEWAY_URL=https://your-domain.example.com
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4 — Install dependencies and build
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm install
|
|
||||||
pnpm build
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 5 — Run database migrations
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/db db:migrate
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 6 — Start the gateway
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node apps/gateway/dist/main.js
|
|
||||||
```
|
|
||||||
|
|
||||||
The gateway reads `.env` from the monorepo root automatically (via `dotenv` in `main.ts`).
|
|
||||||
|
|
||||||
### Step 7 — Start the web app
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Next.js standalone output
|
|
||||||
node apps/web/.next/standalone/server.js
|
|
||||||
```
|
|
||||||
|
|
||||||
The standalone build is self-contained; it does not require `node_modules` to be present at runtime.
|
|
||||||
|
|
||||||
### Step 8 — Configure a reverse proxy
|
|
||||||
|
|
||||||
#### Nginx example
|
|
||||||
|
|
||||||
```nginx
|
|
||||||
# /etc/nginx/sites-available/mosaic
|
|
||||||
|
|
||||||
# Gateway API
|
|
||||||
server {
|
|
||||||
listen 443 ssl;
|
|
||||||
server_name your-domain.example.com;
|
|
||||||
|
|
||||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
|
||||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
|
||||||
|
|
||||||
# WebSocket support (for chat.gateway.ts / Socket.IO)
|
|
||||||
location /socket.io/ {
|
|
||||||
proxy_pass http://127.0.0.1:14242;
|
|
||||||
proxy_http_version 1.1;
|
|
||||||
proxy_set_header Upgrade $http_upgrade;
|
|
||||||
proxy_set_header Connection "upgrade";
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
}
|
|
||||||
|
|
||||||
# REST + auth
|
|
||||||
location / {
|
|
||||||
proxy_pass http://127.0.0.1:14242;
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# Web app (optional — serve on a subdomain or a separate server block)
|
|
||||||
server {
|
|
||||||
listen 443 ssl;
|
|
||||||
server_name app.your-domain.example.com;
|
|
||||||
|
|
||||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
|
||||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
|
||||||
|
|
||||||
location / {
|
|
||||||
proxy_pass http://127.0.0.1:3000;
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Caddy example
|
|
||||||
|
|
||||||
```caddyfile
|
|
||||||
# /etc/caddy/Caddyfile
|
|
||||||
|
|
||||||
your-domain.example.com {
|
|
||||||
reverse_proxy /socket.io/* localhost:14242 {
|
|
||||||
header_up Upgrade {http.upgrade}
|
|
||||||
header_up Connection {http.connection}
|
|
||||||
}
|
|
||||||
reverse_proxy localhost:14242
|
|
||||||
}
|
|
||||||
|
|
||||||
app.your-domain.example.com {
|
|
||||||
reverse_proxy localhost:3000
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Production Considerations
|
|
||||||
|
|
||||||
### systemd Services
|
|
||||||
|
|
||||||
Create a service unit for each process.
|
|
||||||
|
|
||||||
**Gateway** — `/etc/systemd/system/mosaic-gateway.service`:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
[Unit]
|
|
||||||
Description=Mosaic Gateway
|
|
||||||
After=network.target postgresql.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
User=mosaic
|
|
||||||
WorkingDirectory=/opt/mosaic
|
|
||||||
EnvironmentFile=/opt/mosaic/.env
|
|
||||||
ExecStart=/usr/bin/node apps/gateway/dist/main.js
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=5
|
|
||||||
StandardOutput=journal
|
|
||||||
StandardError=journal
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
|
|
||||||
**Web app** — `/etc/systemd/system/mosaic-web.service`:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
[Unit]
|
|
||||||
Description=Mosaic Web App
|
|
||||||
After=network.target mosaic-gateway.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
User=mosaic
|
|
||||||
WorkingDirectory=/opt/mosaic/apps/web
|
|
||||||
EnvironmentFile=/opt/mosaic/.env
|
|
||||||
ExecStart=/usr/bin/node .next/standalone/server.js
|
|
||||||
Environment=PORT=3000
|
|
||||||
Environment=HOSTNAME=127.0.0.1
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=5
|
|
||||||
StandardOutput=journal
|
|
||||||
StandardError=journal
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
|
|
||||||
Enable and start:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
sudo systemctl enable --now mosaic-gateway mosaic-web
|
|
||||||
```
|
|
||||||
|
|
||||||
### Log Management
|
|
||||||
|
|
||||||
Gateway and web app logs go to systemd journal by default. View with:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
journalctl -u mosaic-gateway -f
|
|
||||||
journalctl -u mosaic-web -f
|
|
||||||
```
|
|
||||||
|
|
||||||
Rotate logs by configuring `journald` in `/etc/systemd/journald.conf`:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
SystemMaxUse=500M
|
|
||||||
MaxRetentionSec=30day
|
|
||||||
```
|
|
||||||
|
|
||||||
### Security Checklist
|
|
||||||
|
|
||||||
- Set `BETTER_AUTH_SECRET` to a cryptographically random value (`openssl rand -base64 32`).
|
|
||||||
- Restrict `GATEWAY_CORS_ORIGIN` to your exact frontend origin — do not use `*`.
|
|
||||||
- Run services as a dedicated non-root system user (e.g., `mosaic`).
|
|
||||||
- Firewall: only expose ports 80/443 externally; keep 14242 and 3000 bound to `127.0.0.1`.
|
|
||||||
- Set `AGENT_FILE_SANDBOX_DIR` to a directory outside the application root to prevent agent tools from accessing source code.
|
|
||||||
- If using `AGENT_USER_TOOLS`, enumerate only the tools non-admin users need.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Gateway fails to start — "BETTER_AUTH_SECRET is required"
|
|
||||||
|
|
||||||
`BETTER_AUTH_SECRET` is missing or empty. Set it in `.env` and restart.
|
|
||||||
|
|
||||||
### `DATABASE_URL` connection refused
|
|
||||||
|
|
||||||
Verify PostgreSQL is running and the port matches. The Docker Compose default is `5433`; bare-metal typically uses `5432`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
psql "$DATABASE_URL" -c '\conninfo'
|
|
||||||
```
|
|
||||||
|
|
||||||
### pgvector extension missing
|
|
||||||
|
|
||||||
```sql
|
|
||||||
\c mosaic
|
|
||||||
CREATE EXTENSION IF NOT EXISTS vector;
|
|
||||||
```
|
|
||||||
|
|
||||||
### Valkey / Redis connection refused
|
|
||||||
|
|
||||||
Check the URL in `VALKEY_URL`. The Docker Compose default is port `6380`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
redis-cli -u "$VALKEY_URL" ping
|
|
||||||
```
|
|
||||||
|
|
||||||
### WebSocket connections fail in production
|
|
||||||
|
|
||||||
Ensure your reverse proxy forwards the `Upgrade` and `Connection` headers. See the Nginx/Caddy examples above.
|
|
||||||
|
|
||||||
### Ollama models not appearing
|
|
||||||
|
|
||||||
Set `OLLAMA_BASE_URL` to the URL where Ollama is running (e.g., `http://localhost:11434`) and set `OLLAMA_MODELS` to a comma-separated list of model IDs you have pulled.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ollama pull llama3.2
|
|
||||||
```
|
|
||||||
|
|
||||||
### OTEL traces not appearing in Jaeger
|
|
||||||
|
|
||||||
Verify the collector is reachable at `OTEL_EXPORTER_OTLP_ENDPOINT`. With Docker Compose the default is `http://localhost:4318`. Check `docker compose ps` and `docker compose logs otel-collector`.
|
|
||||||
|
|
||||||
### Summarization / embedding features not working
|
|
||||||
|
|
||||||
These features require `OPENAI_API_KEY` to be set, or you must point `SUMMARIZATION_API_URL` / `EMBEDDING_API_URL` to an OpenAI-compatible endpoint (e.g., a local Ollama instance with an embeddings model).
|
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ mosaic-mono-v1/
|
|||||||
│ ├── queue/ # Valkey-backed task queue
|
│ ├── queue/ # Valkey-backed task queue
|
||||||
│ └── types/ # Shared TypeScript types
|
│ └── types/ # Shared TypeScript types
|
||||||
├── docker/ # Dockerfile(s) for containerized deployment
|
├── docker/ # Dockerfile(s) for containerized deployment
|
||||||
├── infra/ # Infra config (OTEL collector, pg-init scripts)
|
├── infra/ # Infrastructure configuration (for example, OTEL collector)
|
||||||
├── docker-compose.yml # Local services (Postgres, Valkey, OTEL, Jaeger)
|
├── docker-compose.yml # Local services (Postgres, Valkey, OTEL, Jaeger)
|
||||||
└── CLAUDE.md # Project conventions for AI coding agents
|
└── CLAUDE.md # Project conventions for AI coding agents
|
||||||
```
|
```
|
||||||
@@ -86,71 +86,54 @@ cd mosaic-mono-v1
|
|||||||
pnpm install
|
pnpm install
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Start Infrastructure Services
|
### 2. Use the local PGlite tier
|
||||||
|
|
||||||
|
The supported local tier is in-process PGlite and requires no PostgreSQL service. Leave
|
||||||
|
`DATABASE_URL` unset for this route. Its default local configuration uses PGlite and performs no
|
||||||
|
external database probe.
|
||||||
|
|
||||||
|
If a local queue service is useful, start only that non-PostgreSQL service:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose up -d
|
docker compose up -d valkey
|
||||||
```
|
```
|
||||||
|
|
||||||
This starts:
|
Do not use the current Compose PostgreSQL service: it mounts legacy `infra/pg-init` SQL and is
|
||||||
|
not qualified for KBN-101. Start OTEL Collector or Jaeger individually only when needed and
|
||||||
|
without starting PostgreSQL.
|
||||||
|
|
||||||
| Service | Port | Description |
|
### 3. Gateway/Web local process (held)
|
||||||
| ------------------------ | -------------- | -------------------- |
|
|
||||||
| PostgreSQL 17 + pgvector | `5433` (host) | Primary database |
|
|
||||||
| Valkey 8 | `6380` (host) | Queue and cache |
|
|
||||||
| OpenTelemetry Collector | `4317`, `4318` | OTEL gRPC and HTTP |
|
|
||||||
| Jaeger | `16686` | Distributed trace UI |
|
|
||||||
|
|
||||||
### 3. Configure Environment
|
Do not start the current Gateway or web process as a local PGlite route. Gateway first loads the
|
||||||
|
daemon configuration and then project environment files without a tier guard; a pre-existing
|
||||||
|
`DATABASE_URL` can select PostgreSQL, where current startup still reaches runtime DDL/migrations.
|
||||||
|
Creating a root `.env` that omits `DATABASE_URL` does not make this safe, so neither a local
|
||||||
|
credential file nor a web environment file is a current developer procedure.
|
||||||
|
|
||||||
Create a `.env` file in the monorepo root:
|
PGlite remains the supported in-process data-layer implementation, and the optional Valkey command
|
||||||
|
above remains safe because it does not start PostgreSQL. A safe Gateway/Web local procedure is held
|
||||||
|
until KBN-101-02 rejects a daemon, inherited, root, or app-local PostgreSQL DSN and any non-local
|
||||||
|
tier before connection or DDL; KBN-101-05 then supplies the production renderer/Vault process-exec
|
||||||
|
or `LoadCredential` boundary.
|
||||||
|
|
||||||
```env
|
### Held future procedure
|
||||||
# Database (matches docker-compose defaults)
|
|
||||||
DATABASE_URL=postgresql://mosaic:mosaic@localhost:5433/mosaic
|
|
||||||
|
|
||||||
# Auth (required — generate a random 32+ char string)
|
PostgreSQL local and federated deployment are held until KBN-101-00 (external bootstrap),
|
||||||
BETTER_AUTH_SECRET=change-me-to-a-random-secret
|
KBN-101-03 (runner), and KBN-101-05 (renderer-backed deployment) land. The following is the
|
||||||
|
**held, non-operative future activation order with no current command authority**:
|
||||||
|
|
||||||
# Gateway
|
external bootstrap → TLS/roles → `mosaic-db-migrator --run` →
|
||||||
GATEWAY_PORT=14242
|
`mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||||
GATEWAY_CORS_ORIGIN=http://localhost:3000
|
|
||||||
|
|
||||||
# Web
|
Neither current Compose nor this development guide authorizes PostgreSQL initialization SQL,
|
||||||
NEXT_PUBLIC_GATEWAY_URL=http://localhost:14242
|
manual DDL, or a pre-runner start.
|
||||||
|
|
||||||
# Optional: Ollama
|
### 5. Gateway/Web start (held)
|
||||||
OLLAMA_BASE_URL=http://localhost:11434
|
|
||||||
OLLAMA_MODELS=llama3.2
|
|
||||||
```
|
|
||||||
|
|
||||||
The gateway loads `.env` from the monorepo root via `dotenv` at startup
|
No Gateway/Web start command is currently authorized for the local PGlite route. Do not use root
|
||||||
(`apps/gateway/src/main.ts`).
|
`pnpm dev` as a workaround: it additionally starts configured integrations and cannot establish the
|
||||||
|
required local-tier/DSN isolation. Resume this section only after KBN-101-02 provides its
|
||||||
### 4. Push the Database Schema
|
fail-closed local-startup evidence.
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/db db:push
|
|
||||||
```
|
|
||||||
|
|
||||||
This applies the Drizzle schema directly to the database (development only; use
|
|
||||||
migrations in production).
|
|
||||||
|
|
||||||
### 5. Start the Gateway
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/gateway exec tsx src/main.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
The gateway starts on port `14242` by default.
|
|
||||||
|
|
||||||
### 6. Start the Web App
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/web dev
|
|
||||||
```
|
|
||||||
|
|
||||||
The web app starts on port `3000` by default.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -300,26 +283,13 @@ Implement a standard MCP server that exposes tools via the streamable HTTP
|
|||||||
transport or SSE transport. The server must accept connections at a `/mcp`
|
transport or SSE transport. The server must accept connections at a `/mcp`
|
||||||
endpoint.
|
endpoint.
|
||||||
|
|
||||||
### 2. Configure `MCP_SERVERS`
|
### 2. Gateway MCP configuration (held)
|
||||||
|
|
||||||
In your `.env`:
|
Do not configure MCP endpoint credentials, write them to a local environment file, or restart the
|
||||||
|
Gateway from this guide. Gateway/Web startup is held until KBN-101-02 supplies fail-closed
|
||||||
```env
|
local-tier/DSN isolation and KBN-101-05 supplies the renderer/Vault process-exec or
|
||||||
MCP_SERVERS='[{"name":"my-server","url":"http://localhost:3001/mcp"}]'
|
`LoadCredential` secret-consumer interface. The future authenticated MCP route requires verified
|
||||||
```
|
HTTPS and certificate validation; plaintext bearer-token examples are forbidden.
|
||||||
|
|
||||||
With authentication:
|
|
||||||
|
|
||||||
```env
|
|
||||||
MCP_SERVERS='[{"name":"secure-server","url":"http://my-server/mcp","headers":{"Authorization":"Bearer token"}}]'
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Restart the Gateway
|
|
||||||
|
|
||||||
On startup, `McpClientService` (`apps/gateway/src/mcp-client/mcp-client.service.ts`)
|
|
||||||
connects to each configured server, calls `tools/list`, and bridges the results
|
|
||||||
to Pi SDK `ToolDefinition` format. These tools become available in all new agent
|
|
||||||
sessions.
|
|
||||||
|
|
||||||
### Tool Naming
|
### Tool Naming
|
||||||
|
|
||||||
@@ -355,42 +325,31 @@ The schema lives in a single file:
|
|||||||
|
|
||||||
The `insights` table uses a `vector(1536)` column (pgvector) for semantic search.
|
The `insights` table uses a `vector(1536)` column (pgvector) for semantic search.
|
||||||
|
|
||||||
### Development: Push Schema
|
### PostgreSQL schema work (held)
|
||||||
|
|
||||||
Apply schema changes directly to the dev database (no migration files created):
|
Do not prepare or run a PostgreSQL target from this branch. The sole runner, bootstrap, and
|
||||||
|
renderer are future KBN-101 artifacts, not current commands. When KBN-101-00/-03/-05 land, the
|
||||||
|
owned activation documentation will require external bootstrap → TLS/roles → runner `--run` →
|
||||||
|
runner `--verify` → Gateway/Compose readiness.
|
||||||
|
|
||||||
```bash
|
### Generating migration artifacts
|
||||||
pnpm --filter @mosaicstack/db db:push
|
|
||||||
```
|
|
||||||
|
|
||||||
### Generating Migrations
|
`pnpm --filter @mosaicstack/db db:generate` is an offline artifact-generation command. It does
|
||||||
|
not authorize connecting to or initializing PostgreSQL. A future reviewed PostgreSQL procedure
|
||||||
For production-safe, versioned changes:
|
will determine when its output is applied.
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/db db:generate
|
|
||||||
```
|
|
||||||
|
|
||||||
This creates a new SQL migration file in `packages/db/drizzle/`.
|
|
||||||
|
|
||||||
### Running Migrations
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm --filter @mosaicstack/db db:migrate
|
|
||||||
```
|
|
||||||
|
|
||||||
### Drizzle Config
|
### Drizzle Config
|
||||||
|
|
||||||
Config is at `packages/db/drizzle.config.ts`. The schema file path and output
|
Config is at `packages/db/drizzle.config.ts`. The schema file path and output directory are
|
||||||
directory are defined there.
|
defined there.
|
||||||
|
|
||||||
### Adding a New Table
|
### Adding a New Table
|
||||||
|
|
||||||
1. Add the table definition to `packages/db/src/schema.ts`.
|
1. Add the table definition to `packages/db/src/schema.ts`.
|
||||||
2. Export it from `packages/db/src/index.ts`.
|
2. Export it from `packages/db/src/index.ts`.
|
||||||
3. Run `pnpm --filter @mosaicstack/db db:push` (dev) or
|
3. Generate the offline artifact with `pnpm --filter @mosaicstack/db db:generate`.
|
||||||
`pnpm --filter @mosaicstack/db db:generate && pnpm --filter @mosaicstack/db db:migrate`
|
4. Do not apply it to PostgreSQL until the future KBN-101 activation artifacts and their owned
|
||||||
(production).
|
procedure are available. Direct schema push is not a production-like workflow.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,147 +1,98 @@
|
|||||||
# Migrating to the Federated Tier
|
# Migrating to the Federated Tier
|
||||||
|
|
||||||
Step-by-step guide to migrate from `local` (PGlite) or `standalone` (PostgreSQL without pgvector) to `federated` (PostgreSQL 17 + pgvector + Valkey).
|
> **KBN-101-07 ownership:** This active documentation is a **non-operative KBN-101
|
||||||
|
> contract** with no current command authority until KBN-101-00, KBN-101-02, KBN-101-03, KBN-101-05, and KBN-101-06 land and
|
||||||
|
> KBN-101-08 activates an exact reviewed release. The commands below describe the produced interface only. Do not run them on the
|
||||||
|
> current branch or replace them with direct PostgreSQL, raw SQL, legacy storage migration, or
|
||||||
|
> credential-on-argv procedures.
|
||||||
|
|
||||||
## When to migrate
|
## Held future procedure
|
||||||
|
|
||||||
Migrate to federated tier when:
|
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||||
|
|
||||||
- Scaling from single-user to multi-user deployments
|
The deployment control plane executes the complete held future procedure, in order: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. The
|
||||||
- Adding vector embeddings or RAG features
|
runner is the only attestation producer after its verified TLS, identity, manifest, and schema
|
||||||
- Running Mosaic across multiple hosts
|
checks. A data importer is never a schema bootstrap, extension installer, repair command, or DDL
|
||||||
- Requires distributed task queueing and caching
|
consumer.
|
||||||
- Moving to production with high availability
|
|
||||||
|
|
||||||
## Prerequisites
|
## Target material contract
|
||||||
|
|
||||||
- Federated stack running and healthy (see [Federated Tier Setup](../federation/SETUP.md))
|
KBN-101-05 obtains the target URL from Vault KV-v2
|
||||||
- Source database accessible and empty target database at the federated URL
|
`secret-{env}/mosaic-stack/database/importer`, key `url`, and reads its authenticated version from
|
||||||
- Backup of source database (recommended before any migration)
|
the same successful response `data.metadata.version`. A hash or DSN byte sequence is not a
|
||||||
|
provider version. The renderer treats URL bytes and provider version as one generation, writes a
|
||||||
|
temporary generation directory with fsync plus atomic rename, and creates separate immutable
|
||||||
|
consumer mounts. Swarm uses distinct versioned secret/config references. A deployment cannot mix
|
||||||
|
generations.
|
||||||
|
|
||||||
## Dry-run first
|
| Consumer | Permitted material |
|
||||||
|
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| Migrator-attestation producer (`10003:10003`) | Its own migration URL/CA; read-only `/run/secrets/mosaic-migrate-target-url` and `/run/secrets/mosaic-migrate-target-version`, each `0400`, solely to bind; producer-only attestation output at `/run/mosaic-attestations-producer/migrate-target.v1.json`; root-wrapper-only signing key. It never connects with, uses, exports, logs, or forwards the importer URL/version. |
|
||||||
|
| Privileged deployment handoff controller | After runner success and before importer creation, it receives only root-owned non-secret expected provider-version/URL-SHA-256/generation descriptor and pinned public verifier key—not URL bytes or private key. It safe-opens/verifies descriptor and producer artifact, copies exact bytes to a new importer-only mount with fsync/atomic rename, sets `10002:10002` `0400`, seals it read-only, and refuses importer start on any partial/wrong-generation/wrong-owner/mode result. |
|
||||||
|
| Importer (`10002:10002`) | Its own immutable `0400` copies at the same URL/version paths; CA at exact `DATABASE_TLS_CA_CERT_PATH=/run/secrets/mosaic-db-ca.crt`; pinned Ed25519 public key; read-only `/run/mosaic-attestations/migrate-target.v1.json` supplied only by the sealed handoff. |
|
||||||
|
| Gateway/runtime/unrelated container | No importer URL/version, importer artifact, attestation private key, or unrelated CA mount. |
|
||||||
|
|
||||||
Always run a dry-run to validate the migration:
|
The migrator and importer safe-open URL, provider-version, attestation, and public-key files only
|
||||||
|
with `O_RDONLY|O_CLOEXEC|O_NOFOLLOW`; they validate from the opened fd that the file is regular,
|
||||||
|
has its expected owner/mode and link count one. The migrator digests only that URL fd for binding,
|
||||||
|
then zeroizes/closes it. The importer reads URL bytes once into protected memory, validates the
|
||||||
|
signed binding and exact CA before connecting from those same bytes, then zeroizes/closes every
|
||||||
|
fd. It neither logs nor exposes a URL/version/attestation/key oracle.
|
||||||
|
|
||||||
|
## Produced command interface
|
||||||
|
|
||||||
|
After activation and only after approved target preparation, the future interface is:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Deployment control plane has already completed the held runner procedure above.
|
||||||
mosaic storage migrate-tier --to federated \
|
mosaic storage migrate-tier --to federated \
|
||||||
--target-url postgresql://mosaic:mosaic@localhost:5433/mosaic \
|
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||||
|
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||||
--dry-run
|
--dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Expected output (partial example):
|
The provider-version file is fixed deployment material, not argv. This connecting dry-run consumes its nonce; before an actual copy, the deployment control plane must provide fresh runner verification and a new sealed handoff. The runner uses its migration
|
||||||
|
identity; the importer connects only as non-DDL `mosaic_data_importer` and only after all
|
||||||
|
pre-connect validation. After verified TLS and before DML it compares PostgreSQL system ID,
|
||||||
|
database OID, `current_user`, CA/SPKI, and manifest/schema fingerprints to the artifact.
|
||||||
|
|
||||||
```
|
## Required refusals and evidence
|
||||||
[migrate-tier] Analyzing source tier: pglite
|
|
||||||
[migrate-tier] Analyzing target tier: federated
|
|
||||||
[migrate-tier] Precondition: target is empty ✓
|
|
||||||
users: 5 rows
|
|
||||||
teams: 2 rows
|
|
||||||
conversations: 12 rows
|
|
||||||
messages: 187 rows
|
|
||||||
... (all tables listed)
|
|
||||||
[migrate-tier] NOTE: Source tier has no pgvector support. insights.embedding will be NULL on all migrated rows.
|
|
||||||
[migrate-tier] DRY-RUN COMPLETE (no data written). 206 total rows would be migrated.
|
|
||||||
```
|
|
||||||
|
|
||||||
Review the output. If it shows an error (e.g., target not empty), address it before proceeding.
|
KBN-101-02/-03/-05/-06 must prove, with stable sanitized errors, that no target connection occurs
|
||||||
|
for missing/unsafe URL/version/attestation/public-key files; symlink, hardlink, owner, mode, or
|
||||||
|
TOCTOU violations; mixed URL/version generations; missing/wrong CA mount; stale/replayed/tampered
|
||||||
|
or revoked-key artifacts; provider rotation/revocation; wrong TLS/server/database/role/manifest
|
||||||
|
binding; raw `--target-url`; `DATABASE_URL` fallback; runtime/owner identity; consumer leakage;
|
||||||
|
or any DDL attempt. Post-connect identity mismatch closes with zero DML/DDL. Tests also prove no
|
||||||
|
forwarding, child environment, logging, or error oracle leaks URL/version/key/artifact contents.
|
||||||
|
|
||||||
## Run the migration
|
The attestation is credential-free JCS with detached Ed25519 signature and binds issued/expiry,
|
||||||
|
nonce, authenticated provider version, exact URL-fd SHA-256, TLS host/port/database, CA/SPKI,
|
||||||
|
PostgreSQL system ID/database OID, importer role, manifest/schema, and producer identity. Provider
|
||||||
|
version rotation invalidates an old artifact and requires a fresh rendered generation plus runner
|
||||||
|
verification.
|
||||||
|
|
||||||
When ready, run without `--dry-run`:
|
## Actual copy after dry-run
|
||||||
|
|
||||||
|
After reviewed dry-run, obtain the required fresh verification/attestation generation, then use:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Deployment control plane has supplied fresh runner verification and attestation.
|
||||||
mosaic storage migrate-tier --to federated \
|
mosaic storage migrate-tier --to federated \
|
||||||
--target-url postgresql://mosaic:mosaic@localhost:5433/mosaic \
|
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||||
|
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||||
--yes
|
--yes
|
||||||
```
|
```
|
||||||
|
|
||||||
The `--yes` flag skips the confirmation prompt (required in non-TTY environments like CI).
|
The dry-run artifact is terminally replayed and must be rejected; `--yes` bypasses no file,
|
||||||
|
generation, signature, TLS, identity, or DDL control.
|
||||||
|
|
||||||
The command will:
|
## Data boundary and recovery
|
||||||
|
|
||||||
1. Acquire an advisory lock (blocks concurrent invocations)
|
The importer has only an allowlisted mutable-table DML registry. It has no grant for immutable KBN
|
||||||
2. Copy data from source to target in dependency order
|
relations, schemas, roles, memberships, extensions, catalogs, or the Drizzle ledger. Source PGlite
|
||||||
3. Report rows migrated per table
|
uses its explicit local directory and does not make a PostgreSQL URL fallback valid.
|
||||||
4. Display any warnings (e.g., null vector embeddings)
|
|
||||||
|
|
||||||
## What gets migrated
|
A failed or ambiguous migration is a control-plane incident: preserve sanitized evidence, retain
|
||||||
|
the approved backup/rollback state, and retry only after independent review. Never inspect,
|
||||||
All persistent, user-bound data is migrated in dependency order:
|
unlock, repair, or initialize the target with ad hoc SQL or copied credentials.
|
||||||
|
|
||||||
- **users, teams, team_members** — user and team ownership
|
|
||||||
- **accounts** — OAuth provider tokens (durable credentials)
|
|
||||||
- **projects, agents, missions, tasks** — all project and agent definitions
|
|
||||||
- **conversations, messages** — all chat history
|
|
||||||
- **preferences, insights, agent_logs** — preferences and observability
|
|
||||||
- **provider_credentials** — stored API keys and secrets
|
|
||||||
- **tickets, events, skills, routing_rules, appreciations** — auxiliary records
|
|
||||||
|
|
||||||
Full order is defined in code (`MIGRATION_ORDER` in `packages/storage/src/migrate-tier.ts`).
|
|
||||||
|
|
||||||
## What gets skipped and why
|
|
||||||
|
|
||||||
Three tables are intentionally not migrated:
|
|
||||||
|
|
||||||
| Table | Reason |
|
|
||||||
| ----------------- | ----------------------------------------------------------------------------------------------- |
|
|
||||||
| **sessions** | TTL'd auth sessions from the old environment; they will fail JWT verification on the new target |
|
|
||||||
| **verifications** | One-time tokens (email verify, password reset) that have either expired or been consumed |
|
|
||||||
| **admin_tokens** | Hashed tokens bound to the old environment's secret keys; must be re-issued |
|
|
||||||
|
|
||||||
**Note on accounts and provider_credentials:** These durable credentials ARE migrated because they are user-bound and required for resuming agent work on the target environment. After migration to a multi-tenant federated deployment, operators may want to audit or wipe these if users are untrusted or credentials should not be shared.
|
|
||||||
|
|
||||||
## Idempotency and concurrency
|
|
||||||
|
|
||||||
The migration is **idempotent**:
|
|
||||||
|
|
||||||
- Re-running is safe (uses `ON CONFLICT DO UPDATE` internally)
|
|
||||||
- Ideal for retries on transient failures
|
|
||||||
- Concurrent invocations are blocked by a Postgres advisory lock; the second caller will wait
|
|
||||||
|
|
||||||
If a previous run is stuck, check for advisory locks:
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SELECT * FROM pg_locks WHERE locktype='advisory';
|
|
||||||
```
|
|
||||||
|
|
||||||
If you need to force-unlock (dangerous):
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SELECT pg_advisory_unlock(<lock_id>);
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verify the migration
|
|
||||||
|
|
||||||
After migration completes, spot-check the target:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Count rows on a few critical tables
|
|
||||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
|
||||||
"SELECT 'users' as table, COUNT(*) FROM users UNION ALL
|
|
||||||
SELECT 'conversations' as table, COUNT(*) FROM conversations UNION ALL
|
|
||||||
SELECT 'messages' as table, COUNT(*) FROM messages;"
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify a known user or project exists by ID:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
|
||||||
"SELECT id, email FROM users WHERE email='<your-email>';"
|
|
||||||
```
|
|
||||||
|
|
||||||
Ensure vector embeddings are NULL (if source was PGlite) or populated (if source was postgres + pgvector):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
|
||||||
"SELECT embedding IS NOT NULL as has_vector FROM insights LIMIT 5;"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
|
|
||||||
There is no in-place rollback. If the migration fails:
|
|
||||||
|
|
||||||
1. Restore the target database from a pre-migration backup
|
|
||||||
2. Investigate the failure logs
|
|
||||||
3. Rerun the migration
|
|
||||||
|
|
||||||
Always test migrations in a staging environment first.
|
|
||||||
|
|||||||
@@ -522,8 +522,14 @@ mosaic storage export --bucket agent-artifacts --output ./artifacts.tar.gz
|
|||||||
# Import data into storage
|
# Import data into storage
|
||||||
mosaic storage import --bucket agent-artifacts --input ./artifacts.tar.gz
|
mosaic storage import --bucket agent-artifacts --input ./artifacts.tar.gz
|
||||||
|
|
||||||
# Migrate data between tiers
|
# Schema migration is unavailable in this release. The current storage wrapper shells
|
||||||
mosaic storage migrate --from hot --to cold --older-than 30d
|
# directly to `pnpm --filter @mosaicstack/db db:migrate`; it is legacy N-1,
|
||||||
|
# uncertified, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||||
|
# Future schema migration is non-operative: external bootstrap → TLS/roles → runner
|
||||||
|
# --run → runner --verify → readiness.
|
||||||
|
|
||||||
|
# Tier copy uses only the separately held secure migrate-tier route. Never use a legacy
|
||||||
|
# --from/--to storage-migrate command or pass a credential on argv.
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,16 +1,17 @@
|
|||||||
# Native Kanban/SOT Canon
|
# Native Kanban/SOT Canon
|
||||||
|
|
||||||
**Status:** KCR-001–016 independently cleared; canonical publication is in progress under issue [#751](https://git.mosaicstack.dev/mosaicstack/stack/issues/751)
|
**Status:** KCR-001–016 independently cleared; KBN-101 rc.16 current generic storage-wrapper authority remediation awaits independent exact-head re-review under issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771)
|
||||||
**Date:** 2026-07-14
|
**Date:** 2026-07-14
|
||||||
**Implementation hold:** no feature implementation starts until this canon is squash-merged to `main` with terminal-green CI; after merge, every slice remains held until its KBN prerequisite graph is satisfied.
|
**Implementation hold:** no feature implementation starts until this canon is squash-merged to `main` with terminal-green CI; after merge, every slice remains held until its KBN prerequisite graph is satisfied.
|
||||||
|
|
||||||
## Artifacts
|
## Artifacts
|
||||||
|
|
||||||
| Artifact | Purpose |
|
| Artifact | Purpose |
|
||||||
| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0–P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
|
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0–P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
|
||||||
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
|
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
|
||||||
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0–P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
|
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0–P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
|
||||||
|
| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.16 direct-Drizzle current storage-wrapper hold: legacy N-1/uncertified/non-operative pending -02/-03/-06/-08; exact README commented/user-guide executable forms fail before masking and source-consistency rejects runner-delegation copy; held future bootstrap → TLS/roles → run → verify → readiness; plus prior production boundary, pgvector owner, attestation, inventory, manifests, DDL classifier, TLS/bootstrap, activation, and certification contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
|
||||||
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
|
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
|
||||||
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
|
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
|
||||||
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
|
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
|
||||||
@@ -18,6 +19,7 @@
|
|||||||
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
|
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
|
||||||
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
|
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
|
||||||
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
|
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
|
||||||
|
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | Historical `da742ca` REQUEST CHANGES report retained as prior closure evidence; rc.16 awaits independent exact-head re-review after closing the current generic storage-wrapper authority HIGH finding |
|
||||||
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
|
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
|
||||||
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
|
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
|
||||||
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
|
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
|
||||||
@@ -32,7 +34,7 @@
|
|||||||
| **coder5** | Web | Tasks/Projects Kanban/List/detail and later Coordinator/migration-review UI |
|
| **coder5** | Web | Tasks/Projects Kanban/List/detail and later Coordinator/migration-review UI |
|
||||||
| **Mos** | Serialized integration | Canon publication, frozen-contract changes, shared-root/exports, integration gates, merge authority |
|
| **Mos** | Serialized integration | Canon publication, frozen-contract changes, shared-root/exports, integration gates, merge authority |
|
||||||
|
|
||||||
The safe order is KBN-010 → KBN-100 → KBN-105, then coder3 Gateway/MCP server, coder4 CLI/projection, coder5 web, and coder2 recovery can proceed on disjoint files. coder4 then runs pure Coordinator → importer → cutover tooling serially. No two active slices edit the same files.
|
The safe order is KBN-010 → KBN-101 foundation → KBN-100 → KBN-101 deployed-role immutable-operation certificate → KBN-105, then coder3 Gateway/MCP server, coder4 CLI/projection, coder5 web, and coder2 recovery can proceed on disjoint files. KBN-100 is blocked on the KBN-101 foundation; real deployed-role certification—not synthetic test roles—is required before KBN-105. coder4 then runs pure Coordinator → importer → cutover tooling serially. No two active slices edit the same files.
|
||||||
|
|
||||||
## Recovery defaults
|
## Recovery defaults
|
||||||
|
|
||||||
|
|||||||
266
docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md
Normal file
266
docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md
Normal file
File diff suppressed because one or more lines are too long
@@ -1,14 +1,95 @@
|
|||||||
# Native Kanban/SOT — Remediated Shared Contract v1
|
# Native Kanban/SOT — Remediated Shared Contract v1
|
||||||
|
|
||||||
**Status:** CONTROL-PLANE SI-001 AMENDMENT AUTHORIZED; prior KCR-001–016 independent-review GO retained; rc.4 requires independent schema/SecReview before KBN-100
|
**Status:** CONTROL-PLANE rc.16 KBN-101 current generic storage-wrapper authority remediation complete; awaiting independent exact-head re-review. Prior KCR-001–016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
|
||||||
**Version:** 1.0.0-rc.4
|
**Version:** 1.0.0-rc.16
|
||||||
**Date:** 2026-07-14
|
**Date:** 2026-07-15
|
||||||
**Change authority:** Mosaic control plane/Jason only
|
**Change authority:** Mosaic control plane/Jason only
|
||||||
**SI-001 amendment authority:** `web1:mosaic-100` control-plane decision under issue #753
|
**SI-001 amendment authority:** `web1:mosaic-100` control-plane decision under issue #753
|
||||||
|
|
||||||
## Amendment record
|
## Amendment record
|
||||||
|
|
||||||
### 1.0.0-rc.4 — KBN010-SI-001
|
### 1.0.0-rc.16 — Current generic storage-wrapper authority closure
|
||||||
|
|
||||||
|
- **Current-source truth:** `packages/storage/src/cli.ts` currently shells `storage migrate --run` directly to `pnpm --filter @mosaicstack/db db:migrate` through `execSync`; no `mosaic-db-migrator` executable exists. README and user-guide command guidance therefore remove that command and any runner-delegation claim. The current wrapper is legacy N-1, uncertified, non-operative, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||||
|
- **Future-only boundary:** future schema migration remains non-operative and follows external bootstrap → TLS/roles → runner `--run` → runner `--verify` → readiness; tier copy uses only the separately held secure migrate-tier route.
|
||||||
|
- **Unmaskable semantic/source-consistency evidence:** before inventory, ownership, or status masking, -06 fails the exact former README commented code-fence generic-wrapper form and exact user-guide executable generic-wrapper form. Its source-consistency test proves the direct-Drizzle `execSync` target and absent runner bin, so any documentation describing current wrapper delegation to the runner fails.
|
||||||
|
- **Non-effect:** prior runner, legacy-CI, Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||||
|
|
||||||
|
### 1.0.0-rc.15 — Held runner and legacy-CI authority closure
|
||||||
|
|
||||||
|
- **Held runner only:** Current operator documents cannot advertise `mosaic-db-migrator --run|--verify` as executable. The sole passing future form is one `Held future procedure` Markdown section, bounded through its next equal-or-higher heading, that explicitly says non-operative/no-current-command-authority, names KBN-101-00/-03/-05, and preserves external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Any runner hit outside that section fails before inventory/ownership/status masking.
|
||||||
|
- **PGlite/current-CI boundary:** Fleet backlog current behavior is PGlite-only; PostgreSQL CLI/runner authority remains held until activation. README classifies the checked-in direct `db:migrate` CI job as active legacy N-1, uncertified, non-authorizing as an operator route, and pending KBN-101-06 removal; it is a known direct-DDL exception against an isolated disposable CI database, not approved ordinary behavior. The -06 fixture asserts every required status term and rejects ordinary-authority presentation.
|
||||||
|
- **Non-effect:** prior Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||||
|
|
||||||
|
### 1.0.0-rc.14 — Current Compose and production-secret route closure
|
||||||
|
|
||||||
|
- **Current developer boundary:** `README.md` and `docs/guides/dev-guide.md` permit only in-process PGlite data-layer work and explicitly selected non-PostgreSQL Compose services. Gateway/Web local start is held because the current unguarded loader can inherit a daemon/project PostgreSQL DSN and reach runtime DDL; KBN-101-02 must reject it before connection. The current PostgreSQL Compose mount is legacy/unqualified; PostgreSQL and federated activation are held until KBN-101-00/-03/-05 and then follow external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway/Compose readiness.
|
||||||
|
- **Production boundary:** `docs/guides/deployment.md` is non-operative until the KBN-101-05 renderer-backed process-exec or `LoadCredential` interface exists. It contains no active production environment-file, monorepo auto-load, credential export/argv, or secret-activation lifecycle route; future units must preserve generation-pinned Vault consumer isolation.
|
||||||
|
- **Unmaskable semantic negatives:** -06 fails the exact former README/dev/deployment Compose-first sequences and every production `.env`, `EnvironmentFile=`, credential export/argv, or restart-as-secret-activation fixture before owned/status/normative classification. The held PGlite/non-PostgreSQL route and future ordered activation are the only passing fixtures.
|
||||||
|
|
||||||
|
### 1.0.0-rc.13 — Federation-MILESTONES indirect-startup closure
|
||||||
|
|
||||||
|
- **Complete operator inventory:** `docs/federation/MILESTONES.md` is exclusively KBN-101-07 and an exact KBN-101-06 `operator-document` `status-only` record. Its former `pgvector extension installed + verified on startup` wording is superseded and forbidden; it authorizes no current DDL, Compose/init, or runtime/startup path.
|
||||||
|
- **Unmaskable semantic negative:** before inventory disposition, the scanner fixture proves that exact former wording fails. The only passing status-only sequence is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway readiness.
|
||||||
|
|
||||||
|
### 1.0.0-rc.12 — Deployable importer generation and indirect-DDL-route closure
|
||||||
|
|
||||||
|
- **Authenticated generation:** KBN-101-05 owns one canonical Vault KV-v2 importer record, `secret-{env}/mosaic-stack/database/importer` key `url`, with its version taken only from the same successful `data.metadata.version` response. Value plus provider version are one generation, never inferred from DSN bytes. The renderer creates separate immutable `0400` URL/version copies for migrator `10003:10003` binding-only access and importer `10002:10002` access; it uses fsync/atomic generation replacement for Compose and distinct versioned secret/config references for Swarm, so deployment cannot mix generations.
|
||||||
|
- **Bounded consumers:** importer alone receives its URL/version, CA at `DATABASE_TLS_CA_CERT_PATH`, pinned public key, and read-only attestation; migrator receives its own migration URL/CA, the URL/version only for no-connect/no-export binding, attestation output, and the root-wrapper-only private key. Safe fd open/fstat/digest/zeroize/close semantics, a privileged producer-only-to-importer-only attestation handoff controller (verify, exact-byte copy, fsync/atomic rename, `10002:10002` `0400` seal, then importer start), no shared writable file, no logging/oracle, provider rotation/revocation, CA/mount, consumer-isolation, and symlink/hardlink/owner/mode/TOCTOU negatives are mandatory.
|
||||||
|
- **Indirect-DDL closure:** `docs/federation/SETUP.md` is non-operative until KBN-101 activation and documents only external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway readiness. The -06 scanner performs unsuppressible semantic checks for automatic first-boot/startup extension/schema/migration language, Compose-up-before-runner, and init-script authority; the former SETUP wording fails and the remediated sequence passes.
|
||||||
|
|
||||||
|
### 1.0.0-rc.11 — Target-bound importer attestation and exhaustive operator-route closure
|
||||||
|
|
||||||
|
- **Target-bound proof:** trusted `mosaic-db-migrator --verify` now produces the atomic, credential-free `migrate-target.v1.json` JCS/Ed25519 artifact from a runner-only root-owned signing-key reference; the importer receives only pinned public verification keys and the artifact. Its signed v1 fields bind issued/expiry/nonce, exact secret version and SHA-256 of high-entropy target-file bytes, canonical TLS host/port/database, CA/SPKI, PostgreSQL system identifier/database OID, expected importer role, manifest/schema fingerprints, and producer invocation/build/image/correlation. No DSN, username, password, credential bytes, or signing key enters the artifact, importer, runtime, logs, or output.
|
||||||
|
- **Fail-closed importer:** `mosaic storage migrate-tier` requires both `--target-url-file /run/secrets/mosaic-migrate-target-url` and `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`. Before target connection it validates files, signature/key/expiry/replay, secret version/digest, TLS/CA/role/manifest bindings and opens/digests/connects from the same in-memory URL bytes. After verified TLS but before transaction/DML it matches server ID, database OID, `current_user`, CA/SPKI, and manifest/schema; failure distinguishes zero connection from connection/zero-DML and DDL remains impossible. Rotation overlap/revocation, atomic rename, replay cache, secret rotation invalidation, and wrong/substituted/stale/tampered/file-change tests are mandatory.
|
||||||
|
- **Closed documentation surface:** KBN-101-06 inventories every current non-normative scanner hit, including `docs/guides/user-guide.md` and status-only `docs/federation/TASKS.md`; the latter is historical and cannot authorize DDL. The legacy `storage migrate` tier-copy syntax is unavailable. `storage migrate` is schema-wrapper delegation only; secure tier data copy is `migrate-tier`. Exact KBN PRD/contract/shared/task paths may be `normative-contract` scan class but are still scanned and cannot mask executable instructions. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
- **Non-effect:** pgvector closure, manifest, lock, role graph, TLS, activation, and KBN-100/KBN-105 serial gates are unchanged.
|
||||||
|
|
||||||
|
### 1.0.0-rc.10 — PostgreSQL-valid untrusted pgvector owner and active migrate-tier closure
|
||||||
|
|
||||||
|
- **Valid extension authority:** PostgreSQL 17 + pgvector 0.8.2 `vector` is untrusted (`trusted` absent; `relocatable=true`), so `mosaic_extension_owner` is exactly `NOLOGIN SUPERUSER`, not `NOSUPERUSER`. It is dedicated solely to `mosaic_extensions`, `vector`, and owner-bearing extension members; `rolcanlogin=false`, `rolsuper=true`, zero members, no runtime credential/Vault secret, and no app-container delivery are catalog and deployment proof. An externally controlled audited bootstrap-superuser session alone `SET ROLE`s for extension CREATE/UPDATE/SET SCHEMA, then `RESET ROLE`; fresh and shadow paths do so, while in-place existing work requires exact pre-existing `extowner`.
|
||||||
|
- **Explicit superuser exception:** `GRANT`/`REVOKE` cannot privilege-limit a superuser. The containment is dedicated identity, no login, no membership, external control plane, audit, independent review, backup/rollback, and maintenance window—not a false least-privilege claim. Runtime, migrator, schema owner, importer, and every service role cannot assume the role or alter/update/drop/change extension membership. Managed targets without this exact role are ineligible unless a versioned provider-owned extension-owner profile is independently approved.
|
||||||
|
- **Active secure data-migration route:** `docs/guides/migrate-tier.md` is exclusively KBN-101-07, is active rather than historical, and specifies runner-prepared/verified PostgreSQL destination plus a dedicated non-DDL importer. KBN-101-02 freezes `--target-url-file /run/secrets/mosaic-migrate-target-url`, never credential argv; raw `--target-url`, `DATABASE_URL` fallback, runtime owner, missing/unsafe file, wrong mode, and DDL all fail before target connection/DDL. KBN-101-06 inventory/matrix records the route and exact secure fields, then tests its finite operator-document closure.
|
||||||
|
- **Non-effect:** manifest, lock, `mosaic` application schema, TLS, activation, and KBN-100/KBN-105 serial gates remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
|
||||||
|
### 1.0.0-rc.9 — KBN-101 extension-schema boundary, disjoint manifests, and scanner mechanics
|
||||||
|
|
||||||
|
- **Extension schema owner:** `mosaic_extension_owner`, not `mosaic_schema_owner`, creates and owns `mosaic_extensions`, `vector`, and extension-member objects. The external bootstrap actor `SET ROLE`s for fresh creation or approved-owner relocation, then `RESET ROLE`s; rc.10 replaces the earlier membership wording with the PostgreSQL-valid zero-member superuser exception. Schema owner has only `USAGE` for legacy type resolution—never ownership, `CREATE`, `ALTER`, `DROP`, member change, or default-privilege authority. Runtime, migrator, and schema owner must fail catalog and direct DDL denials; shadow/resume/rollback repeat the owner/default-privilege proof.
|
||||||
|
- **Exclusive delivery DAG:** KBN-101-00…09 now has a complete, nonoverlapping exact file/glob manifest with named tests/evidence. The runner mapping is exactly `"mosaic-db-migrator": "./dist/cli.js"` and image `ENTRYPOINT ["mosaic-db-migrator"]`; `packages/storage/src/{cli,migrate-tier}.ts` belongs only to -02, and -07 is documentation only. -08/-09 own evidence paths only. -00…07 are prepared artifacts; the immutable N-1 image remains live until -08 atomic activation, so no independently deployed intermediate can bypass runtime controls.
|
||||||
|
- **Mechanical classifier:** -06 owns the exact scanner, inventory fixture, command-matrix harness, and CI wiring. Inventory records pin path/class/owner/disposition/allowed tokens/rationale/expiry/review revision; unknown, duplicate-owner, ownerless, missing-path, invalid allowlist, and historical-category masking fail. The architecture plan's operative direct `db:migrate` is replaced by sole-runner guidance rather than hidden under a historical category.
|
||||||
|
- **Non-effect:** manifest v1, lock, `mosaic` application-schema ownership, TLS, activation, KBN-100/KBN-105 serial gates, and all earlier canon decisions remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
|
||||||
|
### 1.0.0-rc.8 — KBN-101 finite authority, executable runner, and pgvector-owner remediation
|
||||||
|
|
||||||
|
- **Finite authority closure:** KBN-101-06 classifies every current executable source/script/package bin, operator document, and deploy manifest by exact path; unclassified current hits fail. Byte-immutable historical SQL, PGlite-only routines, negative-test literals, vendored/generated artifacts, and clearly labeled historical reports are exact-path/category reviewed allowlists only. `packages/db/src/index.ts` loses its public `runMigrations` export with a direct-import/compile negative; `docs/fleet/backlog-conventions.md` and `docs/PERFORMANCE.md` lose first-use/direct-Drizzle/Gateway-startup migration instructions and carry runner/readiness route negatives. A token scan is only input to the classifier, never proof of authority.
|
||||||
|
- **Executable exclusive cards:** KBN-101-03 alone publishes `mosaic-db-migrator` from `packages/db/package.json`/`src/cli.ts`, owns `docker/db-migrator.Dockerfile`, and keeps `{runner,config.dto,manifest,identity,tls}` private, with exact `--run|--verify|--help`, env-only input, stable exits, and command tests. KBN-101-00 alone owns `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, plus bootstrap tests. KBN-101-05 alone owns `tools/db/render-postgres-secrets.ts`, renderer tests, and Compose/Portainer/Swarm/two-gateway declarations, consuming the versioned bootstrap interface. No card overlaps renderer/bootstrap/deployment ownership.
|
||||||
|
- **Extension-owner transition:** `mosaic_extension_owner` is a dedicated NOLOGIN role whose membership/credentials never reach services; the external bootstrap actor alone may `SET ROLE` during bootstrap. Fresh vector and member objects retain that owner. PostgreSQL has no supported extension-owner alteration: approved-owner existing extension relocation validates `pg_extension.extowner`, members/schema/version and uses tested `ALTER EXTENSION ... SET SCHEMA`; legacy runtime-owned extension fails closed to a controlled shadow database migration with backup, evidence, quiesce/final delta, atomic switch, and read-only rollback window. No catalog mutation, ownership adoption, or `DROP CASCADE` is permitted. Runtime/migrator/schema-owner extension ALTER/DROP/member-update denial is mandatory.
|
||||||
|
- **Non-effect:** manifest v1, lock namespace, role/search-path, relocation/TLS/activation, KBN-100/KBN-105 serial gates, and all retained canon decisions are strengthened, not weakened. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
|
||||||
|
### 1.0.0-rc.7 — KBN-101 complete current-path, relocation, and two-gateway remediation
|
||||||
|
|
||||||
|
- **Finite current-path closure:** static inventory and the `DATABASE_URL`-only-before-connect/DDL denial matrix now explicitly include Gateway's former temporary-table pgvector test (runner-prepared persistent read/query-only fixture), `docker/init-db.sql` retirement, `migrate-tier.ts` runner/bootstrap-only guidance, and the active two-gateway harness. The harness is migrated, not retired: `postgres-a/b → mosaic-db-migrator-a/b → gateway-a/b`, each with isolated URL/CA material, verified readiness, SANs, and positive/negative TLS evidence.
|
||||||
|
- **Executable relocation:** KBN-101-03 exclusively owns `schema.ts`, Drizzle snapshots/journal/generated relocation and exact tests. All future application declarations use exported `pgSchema('mosaic')`; immutable historical SQL runs only in trusted legacy `public`. `vector` is fixed in non-writable `mosaic_extensions`, with exact catalog relocatability/version eligibility, explicit type/operator qualification, catalog-class ordering, unknown-object fail-closed behavior, clean/current-public/partial/reverse rollback tests, and an N-1 release order.
|
||||||
|
- **Bound deployment ownership:** `mosaicstack/stack` KBN-101-00/05 owns current Compose, Portainer, two-gateway, bootstrap renderer/templates, UID/GID declarations, and rendered validation. Gateway is fixed to `10001:10001`; PostgreSQL UID/GID is image-inspected and frozen only after digest pinning. Exact secret paths, atomic renderer behavior, Compose/Swarm targets/modes, Gateway/PostgreSQL leaf separation, and two-pair TLS failure evidence are required. Mosaic deployment control plane/Jason is the named activation authority; environment IaC/Vault supplies versioned input only.
|
||||||
|
- **Correct traceability:** REQ-03 maps to role/schema/search-path, REQ-04 to TLS, REQ-05 to post-KBN-100 immutability, REQ-06 to rollout/rollback, and REQ-07 to the KBN-101 → KBN-100 → KBN-101 → KBN-105 sequence. No prior manifest/lock/role/DAG/activation decision is weakened.
|
||||||
|
|
||||||
|
### 1.0.0-rc.6 — KBN-101 closed DDL/TLS/ledger activation remediation
|
||||||
|
|
||||||
|
- **Choice:** `mosaic-db-migrator` is the sole application/CI/test PostgreSQL DDL control plane. Every legacy entrypoint is routed or denied, rejects `DATABASE_URL`-only before connection/DDL, and `db:push` is unavailable outside an allowlisted disposable developer target. The runner holds one `max:1` session with fixed `pg_try_advisory_lock(1297044289,1262636593)` across preflight through release.
|
||||||
|
- **Exact ledger:** manifest v1 canonically serializes journal logical index/tag and SHA-256 of exact shipped migration bytes. It maps each observed ledger hash to one tuple; physical insertion order is non-normative, while missing/unknown/duplicate/ambiguous/corrupt/stale states fail closed. Shipped `0009` bytes remain unchanged; a missing/effects-absent `0009` runs normally, an applied-late hash maps normally, and partial/full effects with missing hash require backup restoration or separately reviewed repair—not manual adoption.
|
||||||
|
- **TLS/search path:** operator/IaC owns CA and server leaf lifecycle, exact compose/Swarm secret mounts, server TLS activation, service-DNS SANs, verified-TLS readiness, transition, CA overlap rotation, and rollback. Runtime/migrator use `verify-full`; PGlite is not PostgreSQL TLS evidence. Application sessions use only `pg_catalog,mosaic`; no URL/config-derived identifier reaches SQL.
|
||||||
|
- **Safe release:** cards 00–07 land prepared but inactive; owner-runtime deployments remain N-1. Mosaic control plane/Jason alone authorizes one atomic TLS/roles → runner → readiness → runtime activation or rollback. No runtime-operator compatibility switch, bypass, plaintext interval, or force-on-red exists; all temporary support is removed before KBN-101-08.
|
||||||
|
- **Non-effect:** role graph, immutable certification after KBN-100, KBN-105 gate, rc.5’s preserved rc.4 SI-001 invariants, and all KCR-001–016 decisions remain unchanged. Exact detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
|
||||||
|
### 1.0.0-rc.5 — KBN-101 role/connection split
|
||||||
|
|
||||||
|
- **Choice:** PostgreSQL `standalone` and `federated` runtime uses `DATABASE_URL` only as a non-owner `mosaic_runtime` login; an explicit migration phase uses `DATABASE_MIGRATION_URL` only as `mosaic_migrator`, which `SET ROLE`s to non-login `mosaic_schema_owner` for DDL. Local PGlite remains an explicit embedded exception.
|
||||||
|
- **No fallback / no startup DDL:** missing migration URL fails the migration phase; it never falls back to runtime URL/default/config. Gateway replicas do not run migrations. An advisory-locked migration phase verifies the exact ordered Drizzle ledger fingerprint before replicas may become ready.
|
||||||
|
- **Privilege model:** non-login `mosaic_platform_database_owner` is outside application paths; `mosaic_schema_owner` owns only application/ledger schemas. `mosaic_runtime` has only `mosaic_runtime_capability`, owns no object/schema, cannot assume owner/migrator, has no TEMPORARY privilege, has only read access to the Drizzle ledger, and must fail startup if effective identity, unsafe attributes, authenticated TLS, search path, schema version, grants, or immutable relation privileges differ from the frozen contract. `task_events`, `artifacts`, `task_checkpoints`, `task_checkpoint_artifacts`, and `approval_decision_artifacts` grant runtime only INSERT/SELECT; KBN-100 retains RESTRICT/no-cascade semantics.
|
||||||
|
- **Non-effect:** rc.4 SI-001 candidate-key/FK order and all KCR-001–016 tenancy, SOT, proposal-audit, approval, fence, recovery, no-cascade, endpoint, and wire invariants are unchanged. This amendment neither creates roles/secrets nor changes production deployment.
|
||||||
|
- **Gate:** KBN-101’s role/schema-boundary foundation certificate, Vault/redaction/rotation, N-1/rollback, and independent security GO are mandatory before KBN-100. After KBN-100 creates the immutable relations, KBN-101 real deployed-role immutable-operation certification plus Ultron GO is mandatory before KBN-105; synthetic test-role success alone is insufficient. Exact implementation detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||||
|
|
||||||
|
### 1.0.0-rc.4 — KBN010-SI-001 (preserved)
|
||||||
|
|
||||||
- **Choice:** add the explicitly named, non-partial unique candidate key `missions_workspace_id_uidx` on `missions(workspace_id, id)` and retain `missions_workspace_project_id_uidx` on `(workspace_id, project_id, id)`.
|
- **Choice:** add the explicitly named, non-partial unique candidate key `missions_workspace_id_uidx` on `missions(workspace_id, id)` and retain `missions_workspace_project_id_uidx` on `(workspace_id, project_id, id)`.
|
||||||
- **Rationale:** mission `id` remains globally unique, while the composite candidate key makes the frozen tenant-safe generic mission relations valid. `artifacts` and `approval_decisions` are polymorphic exactly-one-target records and do not consistently carry `project_id`; widening both children would unnecessarily broaden v1 and its target semantics.
|
- **Rationale:** mission `id` remains globally unique, while the composite candidate key makes the frozen tenant-safe generic mission relations valid. `artifacts` and `approval_decisions` are polymorphic exactly-one-target records and do not consistently carry `project_id`; widening both children would unnecessarily broaden v1 and its target semantics.
|
||||||
@@ -19,7 +100,7 @@
|
|||||||
|
|
||||||
## 1. Authority
|
## 1. Authority
|
||||||
|
|
||||||
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
|
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.13 DDL/ledger/TLS/role/attestation/generation separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
|
||||||
|
|
||||||
## 2. Health proof and exact failures
|
## 2. Health proof and exact failures
|
||||||
|
|
||||||
@@ -104,7 +185,7 @@ The candidate key is intentionally redundant with globally unique `missions.id`,
|
|||||||
|
|
||||||
### 5.3 New audit/proposal DDL order
|
### 5.3 New audit/proposal DDL order
|
||||||
|
|
||||||
KBN-100 migration DDL must execute in this order:
|
KBN-100 migration DDL may begin only after KBN-101 foundation role/schema-boundary certification. It runs in the explicit migrator/owner phase—not Gateway startup—and its generated Drizzle declaration/snapshot/journal must be mutually consistent. It must execute in this order:
|
||||||
|
|
||||||
1. create `task_events` and its unique `(workspace_id, id)` key;
|
1. create `task_events` and its unique `(workspace_id, id)` key;
|
||||||
2. create `change_proposals` with nullable acceptance-event ID and required submission-event ID;
|
2. create `change_proposals` with nullable acceptance-event ID and required submission-event ID;
|
||||||
@@ -243,7 +324,7 @@ KBN-115/coder2 owns `packages/config/src/recovery-posture.ts`, tests, and recove
|
|||||||
|
|
||||||
## 9. Integration, security, and hold
|
## 9. Integration, security, and hold
|
||||||
|
|
||||||
Required release evidence includes empty/prod/partial/rollback/N-1 migration tests; cross-workspace and same-workspace wrong-project negatives; active-membership owners/principals; proposal inertness/normal acceptance; exact failure mapping; concurrent monotonic bigint fences; relational lease/checkpoint/evidence mismatch; immutability privileges/RESTRICT; recovery validation/mechanism evidence; endpoint registry alignment; accessible web journeys; author≠reviewer; mandatory SecReview; final Certifier pass/no merge authority.
|
Required release evidence includes KBN-101 foundation role/schema-boundary and post-KBN-100 real immutable-operation deployed-role certificates (not synthetic roles), empty/prod/partial/rollback/N-1 migration tests; cross-workspace and same-workspace wrong-project negatives; active-membership owners/principals; proposal inertness/normal acceptance; exact failure mapping; concurrent monotonic bigint fences; relational lease/checkpoint/evidence mismatch; immutability privileges/RESTRICT; recovery validation/mechanism evidence; endpoint registry alignment; accessible web journeys; author≠reviewer; mandatory SecReview; final Certifier pass/no merge authority.
|
||||||
|
|
||||||
### 9.1 SI-001 amendment gate and #757 boundary
|
### 9.1 SI-001 amendment gate and #757 boundary
|
||||||
|
|
||||||
|
|||||||
@@ -43,8 +43,10 @@ Shared roots, package exports/manifests, lockfiles, and generated artifacts are
|
|||||||
```text
|
```text
|
||||||
KBN-000 canon remediation
|
KBN-000 canon remediation
|
||||||
-> KBN-010 threat/auth/constraint-impact gate (MUST COMPLETE)
|
-> KBN-010 threat/auth/constraint-impact gate (MUST COMPLETE)
|
||||||
|
-> KBN-101 foundation role/schema-boundary certificate (SERIAL)
|
||||||
-> KBN-100 schema + concrete N-1 migration implementation
|
-> KBN-100 schema + concrete N-1 migration implementation
|
||||||
├─ KBN-105 exact endpoint/DTO/error/registry freeze (SERIAL)
|
├─ KBN-101 post-KBN-100 deployed-role immutable-operation certificate (SERIAL)
|
||||||
|
│ -> KBN-105 exact endpoint/DTO/error/registry freeze (SERIAL)
|
||||||
│ ├─ KBN-110 domain + Gateway + MCP server implementation
|
│ ├─ KBN-110 domain + Gateway + MCP server implementation
|
||||||
│ ├─ KBN-120 CLI/projection implementation [coder4 first]
|
│ ├─ KBN-120 CLI/projection implementation [coder4 first]
|
||||||
│ └─ KBN-130 web MVP implementation
|
│ └─ KBN-130 web MVP implementation
|
||||||
@@ -64,7 +66,7 @@ KBN-310 + KBN-320
|
|||||||
-> KBN-340 owner-gated cutover/stabilization
|
-> KBN-340 owner-gated cutover/stabilization
|
||||||
```
|
```
|
||||||
|
|
||||||
No consumer implementation begins before KBN-105. No schema work begins before KBN-010 completes. The coder4 order is always KBN-120 → KBN-200 → KBN-300 → KBN-320.
|
No consumer implementation begins before KBN-105. No schema work begins before KBN-010 completes and the KBN-101 foundation role/schema-boundary certificate passes; the real immutable-operation certificate follows KBN-100 and blocks KBN-105. The coder4 order is always KBN-120 → KBN-200 → KBN-300 → KBN-320.
|
||||||
|
|
||||||
## 4. P0 — Canon, threat gate, schema, and exact API freeze
|
## 4. P0 — Canon, threat gate, schema, and exact API freeze
|
||||||
|
|
||||||
@@ -91,6 +93,17 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
|||||||
- **Contract surfaces:** schema constraints, health proof, exact errors, command-family authorization.
|
- **Contract surfaces:** schema constraints, health proof, exact errors, command-family authorization.
|
||||||
- **Evidence:** signed constraint-impact matrix; no unresolved schema-impact finding; SecReview pass.
|
- **Evidence:** signed constraint-impact matrix; no unresolved schema-impact finding; SecReview pass.
|
||||||
|
|
||||||
|
### KBN-101 — PostgreSQL runtime/migration role split and deployed-role certification
|
||||||
|
|
||||||
|
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.16 closes HIGH-1 current generic storage-wrapper authority: README/user-guide remove `storage migrate --run` guidance and false runner delegation; current source is direct-Drizzle, legacy N-1, uncertified, non-operative, and forbidden pending -02/-03/-06/-08 activation. The -06 fixture fails both exact former forms before inventory/status masking and source-consistency rejects current direct-Drizzle wrapper as runner delegation. It awaits independent exact-head re-review; implementation remains held.
|
||||||
|
- **Owner:** Mos integration control plane; independently reviewed by security/Ultron.
|
||||||
|
- **Mode:** SERIAL foundation certificate blocks KBN-100; its post-KBN-100 real immutable-operation certificate blocks KBN-105.
|
||||||
|
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; sole published `mosaic-db-migrator --run|--verify` PostgreSQL DDL path and all legacy/future entrypoint closure; active migrate-tier destination only after runner prepare/verify through exact `--target-url-file /run/secrets/mosaic-migrate-target-url`, paired authenticated provider-version file, and signed `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`; runner-only signing key/public-key isolation; canonical Vault KV-v2 target URL/version, generation-pinned renderer, importer CA/public-key/attestation plus privileged sealed producer-to-importer handoff, safe-fd/consumer-isolation/no-log-oracle, TLS/server/database/role/manifest/schema binding, expiry/replay/provider-rotation/TOCTOU/no-DML controls, and dedicated non-DDL importer; finite exact-path scanner/allowlist/active-route review plus unsuppressible automatic-startup/init/Compose-before-runner semantic negatives and every-path before-connect denial matrix; `DATABASE_TLS_CA_CERT_PATH` plus operator/IaC CA/server-key/cert lifecycle, exact service-DNS SANs, Vault/compose/Swarm mount modes, TLS server/bootstrap/rotation/rollback; PGlite exception; fixed two-int advisory lock; manifest-v1 logical-index/tag/exact-byte-SHA-256 ledger reconciliation including safe `0009`; fixed `mosaic` schema and exact `pg_catalog,mosaic` pooled session path; platform/schema/`NOLOGIN SUPERUSER` extension-owner/migrator/importer/runtime roles; approved-owner versus legacy-owner shadow pgvector transition; ownership, zero membership/no runtime secret, TEMP/ledger-read/default privilege and immutable grant proof; N-1 inactive prepared cards then atomic activation/rollback authority; Vault/redaction/observability/operator runbooks; one-card/one-PR implementation DAG.
|
||||||
|
- **OUT:** Production mutation in this planning card; KBN-100 tables/data backfill; application API behavior; KBN-105 route/DTO freeze.
|
||||||
|
- **Depends on:** KBN-010 completed.
|
||||||
|
- **Contract surfaces:** [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md); `SHARED-CONTRACT.md` rc.15 amendment.
|
||||||
|
- **Evidence:** foundation: exact `--help|--run|--verify`/exit/argv/import-negative plus DTO entrypoint negatives for every finite classified current DDL/static-bypass path (including `DATABASE_URL`-only, runner fixture, retired init, sanitized current operator guidance, both harness pairs, and `db:push` refusal); active migrate-tier paired URL/version/attestation files, signing/public-key isolation, canonical Vault KV-v2 authenticated version, generation-pinned renderer, importer CA, safe fd/TOCTOU/consumer-isolation/no-log-oracle, atomic JCS/Ed25519, digest/TLS/server/database/role/manifest/schema binding, expiry/replay/provider rotation/revocation, zero-connection versus zero-DML, prepared-target/importer/no-DDL negatives; clean/pre-0009/skipped/applied-late/duplicate/unknown/missing/corrupt/stale/backup plus public-to-`mosaic`/partial/reverse runner proof; fixed-lock contention/crash/readiness/unrelated-key tests; runtime cannot invoke migrations/DDL/TEMP; actual pgvector 0.8.2 control metadata, fresh/approved-owner existing/legacy-owner shadow/partial-resume-rollback/N-1 pgvector evidence with `rolcanlogin=false`, `rolsuper=true`, zero members, external-superuser `SET ROLE`/`RESET ROLE` audit, `pg_extension.extowner`, owner-bearing member/schema/version and runtime/migrator/schema-owner/importer/all-service-role `SET ROLE`/ALTER/DROP/member-update denial; disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus both-pair CA/SAN/downgrade/key mode/UID-GID/URL-secret consumer-isolation and legacy-drain/`hostssl` zero-plaintext negatives; exclusive bootstrap/renderer/manifest ownership test; catalog relocation/vector-query/operator/Drizzle-only-`mosaic`, role/grant/search-path/pool-reset/identifier checks; N-1/atomic TLS-only rollback/no-force-on-red rehearsal; named Vault/bootstrap-control-plane/CA-overlap/redaction/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
|
||||||
|
|
||||||
### KBN-100 — Unified Drizzle schema and concrete N-1 migration
|
### KBN-100 — Unified Drizzle schema and concrete N-1 migration
|
||||||
|
|
||||||
- **Owner:** **coder2**.
|
- **Owner:** **coder2**.
|
||||||
@@ -98,7 +111,7 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
|||||||
- **Exclusive files:** `packages/db/src/schema.ts`, `packages/db/drizzle/**`, DB tests.
|
- **Exclusive files:** `packages/db/src/schema.ts`, `packages/db/drizzle/**`, DB tests.
|
||||||
- **IN:** All frozen tables/joins/enums; workspace/project-congruent constraints; owners/principals; tags/archive; change proposals with both workspace-aware task-event composite FKs and frozen event-before-proposal DDL order; assignment approvals; durable execution/quarantine; monotonic bigint fence; exact checkpoint/evidence joins; RESTRICT/immutability; concrete current-main expand/backfill/switch/contract map.
|
- **IN:** All frozen tables/joins/enums; workspace/project-congruent constraints; owners/principals; tags/archive; change proposals with both workspace-aware task-event composite FKs and frozen event-before-proposal DDL order; assignment approvals; durable execution/quarantine; monotonic bigint fence; exact checkpoint/evidence joins; RESTRICT/immutability; concrete current-main expand/backfill/switch/contract map.
|
||||||
- **OUT:** Repositories, Gateway, Coordinator behavior, UI, importer.
|
- **OUT:** Repositories, Gateway, Coordinator behavior, UI, importer.
|
||||||
- **Depends on:** **KBN-010 completed**.
|
- **Depends on:** **KBN-010 completed and KBN-101 foundation role/schema-boundary certificate PASS**. KBN-100 is blocked until both are terminal; it rebases on KBN-101 main, restores generated Drizzle declaration/snapshot/journal consistency, and confines procedural immutable-table grant/trigger/backfill work to its schema ownership. Its new relations are then subject to KBN-101 post-KBN-100 deployed-role certification.
|
||||||
- **Contract surfaces:** `kanban-schema.v1.ts`; SHARED-CONTRACT current-main delta map.
|
- **Contract surfaces:** `kanban-schema.v1.ts`; SHARED-CONTRACT current-main delta map.
|
||||||
- **Evidence:** reviewed SQL; empty/prod-shape/partial-resume/rollback tests; N-1 app safety; legacy columns remain declared; workspace/project mismatch negatives; proposal event-FK missing/foreign-workspace tests; one active lease; monotonic fence; parent-delete RESTRICT; immutability privileges; SecReview.
|
- **Evidence:** reviewed SQL; empty/prod-shape/partial-resume/rollback tests; N-1 app safety; legacy columns remain declared; workspace/project mismatch negatives; proposal event-FK missing/foreign-workspace tests; one active lease; monotonic fence; parent-delete RESTRICT; immutability privileges; SecReview.
|
||||||
|
|
||||||
@@ -109,7 +122,7 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
|||||||
- **Exclusive files:** canonical endpoint-registry/DTO contract docs; no implementation.
|
- **Exclusive files:** canonical endpoint-registry/DTO contract docs; no implementation.
|
||||||
- **IN:** Exact routes and methods from SHARED-CONTRACT §8; request/success/error fields; status codes; pagination/filter/revision envelopes; idempotency/expected-version headers/fields; proposal commands; health proof exclusion from public DTOs; MCP tool-to-route map.
|
- **IN:** Exact routes and methods from SHARED-CONTRACT §8; request/success/error fields; status codes; pagination/filter/revision envelopes; idempotency/expected-version headers/fields; proposal commands; health proof exclusion from public DTOs; MCP tool-to-route map.
|
||||||
- **OUT:** Controller/service/client implementation.
|
- **OUT:** Controller/service/client implementation.
|
||||||
- **Depends on:** KBN-100.
|
- **Depends on:** KBN-100 and KBN-101 post-KBN-100 deployed-role immutable-operation certification PASS.
|
||||||
- **Contract surfaces:** health/error unions; schema IDs/statuses; Gateway DTO freeze.
|
- **Contract surfaces:** health/error unions; schema IDs/statuses; Gateway DTO freeze.
|
||||||
- **Evidence:** every FE/CLI/MCP call maps 1:1 to a route; 503/502-504/409 non-cross-map fixtures; contract digest published.
|
- **Evidence:** every FE/CLI/MCP call maps 1:1 to a route; 503/502-504/409 non-cross-map fixtures; contract digest published.
|
||||||
|
|
||||||
@@ -252,9 +265,11 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
|||||||
## 8. Consistent USC wave schedule
|
## 8. Consistent USC wave schedule
|
||||||
|
|
||||||
| Wave | coder2 | coder3 | coder4 | coder5 |
|
| Wave | coder2 | coder3 | coder4 | coder5 |
|
||||||
| ---- | ------------------------- | -------------------------------------- | ------------------------------ | ------------------------------ |
|
| ---- | ----------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------ | ------------------------------ |
|
||||||
| 0 | Wait | **KBN-010** | Wait | Wait |
|
| 0 | Wait | **KBN-010** | Wait | Wait |
|
||||||
| 1 | **KBN-100** | Review constraint implementation | Wait | Wait |
|
| 0.5 | Wait | **KBN-101 foundation** Mos-controlled role/connection contract and certificate | Wait | Wait |
|
||||||
|
| 1 | **KBN-100** after KBN-101 foundation PASS | Review bounded schema/grant implementation | Wait | Wait |
|
||||||
|
| 1.5 | Certification support | **KBN-101 post-KBN-100 deployed-role immutable-operation certificate**, then KBN-105 | Wait | Wait |
|
||||||
| 2 | **KBN-115** after KBN-100 | **KBN-105** exact freeze, then KBN-110 | **KBN-120** only after KBN-105 | **KBN-130** only after KBN-105 |
|
| 2 | **KBN-115** after KBN-100 | **KBN-105** exact freeze, then KBN-110 | **KBN-120** only after KBN-105 | **KBN-130** only after KBN-105 |
|
||||||
| 3 | Review support | Finish KBN-110 | **KBN-200 after KBN-120** | Finish KBN-130 |
|
| 3 | Review support | Finish KBN-110 | **KBN-200 after KBN-120** | Finish KBN-130 |
|
||||||
| 4 | — | **KBN-210 after KBN-200** | Review/support | **KBN-220 after KBN-210 DTOs** |
|
| 4 | — | **KBN-210 after KBN-200** | Review/support | **KBN-220 after KBN-210 DTOs** |
|
||||||
|
|||||||
@@ -1460,12 +1460,11 @@ Add to `packages/db/src/schema.ts` in the `preferences` table definition:
|
|||||||
mutable: boolean('mutable').notNull().default(true),
|
mutable: boolean('mutable').notNull().default(true),
|
||||||
```
|
```
|
||||||
|
|
||||||
Generate and apply:
|
### Held future procedure
|
||||||
|
|
||||||
```bash
|
This historical architecture plan grants **no current command authority**. PostgreSQL execution is non-operative until **KBN-101-00, KBN-101-03, and KBN-101-05** land; do not invoke a PostgreSQL runner from this checkout. After those cards land, the approved future procedure is exactly: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Offline migration artifact generation belongs to its owning implementation card and does not activate PostgreSQL execution.
|
||||||
pnpm --filter @mosaicstack/db db:generate # generates migration SQL
|
|
||||||
pnpm --filter @mosaicstack/db db:migrate # applies to PG
|
> **KBN-101 supersession:** `pnpm --filter @mosaicstack/db db:migrate` is superseded and MUST NOT be used. The future runner receives only deployment-injected migration credentials; it accepts no URL, SQL, schema, or role argv.
|
||||||
```
|
|
||||||
|
|
||||||
Platform enforcement keys (seeded with `mutable = false` by gateway `PreferencesService.onModuleInit()`):
|
Platform enforcement keys (seeded with `mutable = false` by gateway `PreferencesService.onModuleInit()`):
|
||||||
|
|
||||||
|
|||||||
@@ -946,13 +946,11 @@ pnpm --filter @mosaicstack/types typecheck
|
|||||||
|
|
||||||
Expected: All PASS
|
Expected: All PASS
|
||||||
|
|
||||||
**Step 2: Manual smoke test**
|
**Step 2: Manual smoke test (held)**
|
||||||
|
|
||||||
```bash
|
This historical TUI smoke test is unavailable until KBN-101-02 supplies a fail-closed Gateway local
|
||||||
cd /home/jwoltje/src/mosaic-mono-v1-worktrees/tui-improvements
|
startup route. Do not start current Compose PostgreSQL or infer a local Gateway from PGlite support.
|
||||||
docker compose up -d
|
A future reviewed test must use the correct Mosaic CLI package and an independently verified Gateway.
|
||||||
pnpm --filter @mosaicstack/cli exec tsx src/cli.ts tui
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
Verify:
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
# KBN-101 contract independent security/architecture review
|
||||||
|
|
||||||
|
**Verdict: REQUEST CHANGES**
|
||||||
|
|
||||||
|
## Review identity and scope
|
||||||
|
|
||||||
|
- **Exact reviewed head:** `da742ca2da4a2ff466916c818fe275c4f7ffd384` (`docs(#771): record role-split review evidence`)
|
||||||
|
- **Required comparison:** `origin/main...da742ca2da4a2ff466916c818fe275c4f7ffd384`
|
||||||
|
- **Range:** `82ce3252df38a687c50485f8d048b53ca8db5989` is an ancestor of the reviewed head; the final head adds the scratchpad evidence commit and was reviewed.
|
||||||
|
- **Changed docs:** `docs/PRD.md`, `docs/SITEMAP.md`, `docs/native-kanban-sot/{INDEX.md,KBN-101-DB-ROLE-SPLIT.md,SHARED-CONTRACT.md,TASKS.md}`, and `docs/scratchpads/771-kbn101-db-role-split.md` (300 additions / 25 deletions).
|
||||||
|
- **Reviewed inputs:** issue #771; current DB/Gateway/storage/config/wizard/installer/compose/Portainer/CI sources; all current migration/DDL references; KBN-010, rc.4/rc.5 shared contract, requirements/canon, KBN-100 #769 branch context, and the final scratchpad.
|
||||||
|
- **Repository/provider state:** not modified. The pre-existing `.mosaic/orchestrator/*` dirt was not touched.
|
||||||
|
|
||||||
|
The role graph itself is sound in principle: a NOLOGIN platform database owner, separate NOLOGIN schema owner, NOINHERIT migrator which explicitly `SET ROLE`s, and runtime membership only in a capability role with `SET FALSE` does not create circular privilege or application-created login roles. The split of foundation certification before KBN-100 and immutable-operation certification after KBN-100 is also correctly ordered.
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
### HIGH — DDL/migration control plane is not closed at every current entrypoint
|
||||||
|
|
||||||
|
The contract requires an explicit, locked migration phase and forbids Gateway/runtime DDL (`KBN-101-DB-ROLE-SPLIT.md:34-39`), but its KBN-101-02 result merely says migration-capable commands use the migration DTO (`:109`). It does not prohibit or route every existing bypass through that one command.
|
||||||
|
|
||||||
|
Current bypasses include:
|
||||||
|
|
||||||
|
- `runMigrations()` falls back from an argument to `DATABASE_URL` and a hard-coded URL (`packages/db/src/migrate.ts:24-35`), while `drizzle.config.ts` likewise uses `DATABASE_URL` plus a default (`packages/db/drizzle.config.ts:3-9`).
|
||||||
|
- Package scripts expose direct `drizzle-kit migrate` **and** `drizzle-kit push` (`packages/db/package.json:23-26`); `db:push` bypasses the planned journal/fingerprint/lock entirely.
|
||||||
|
- `mosaic storage migrate --run` shells out to the direct `db:migrate` script (`packages/storage/src/cli.ts:413-452`).
|
||||||
|
- The federated integration test can create types, tables, and indexes directly against `DATABASE_URL` and intentionally operates without a Drizzle ledger (`packages/db/src/federation.integration.test.ts:28-30,46-134`).
|
||||||
|
|
||||||
|
**Failure mode:** a runtime or CI environment with only `DATABASE_URL`, or an operator invoking an existing command, can apply unverified DDL outside the lock, `SET ROLE` preflight, exact-ledger gate, and deployment sequencing. This breaks the requested fail-closed split even if Gateway startup is repaired.
|
||||||
|
|
||||||
|
**Required remediation:** amend KBN-101-02/03/06 to enumerate these entrypoints and make the dedicated migrator runner the only PostgreSQL DDL path. Production-like `db:push` must be removed/blocked; `db:migrate`, `storage migrate --run`, and migration tests must invoke the same migration runner with `DATABASE_MIGRATION_URL`, lock, identity preflight, and ledger verification. Tests needing schema must consume a pre-migrated disposable database, or be explicitly run only by that migration phase. Add negative tests showing each command refuses `DATABASE_URL`-only execution and cannot reach DDL.
|
||||||
|
|
||||||
|
### HIGH — TLS requirement has no deployable server/bootstrap contract
|
||||||
|
|
||||||
|
The contract correctly requires a mounted CA and hostname-verified TLS (`KBN-101-DB-ROLE-SPLIT.md:25,28,93-95`). However KBN-101-05 promises only a “migration phase and secret binding boundary” (`:112`), not PostgreSQL server TLS, certificate issuance/SANs, CA distribution, startup ordering, or the fresh/existing-database bootstrap trust path.
|
||||||
|
|
||||||
|
Current standalone and federated compose expose plain PostgreSQL with no server TLS configuration or CA mount (`docker-compose.yml:2-14`; `docker-compose.federated.yml:27-44`). The Portainer test stack passes a single plaintext in-network URL and uses the same database login for Gateway and database bootstrap (`deploy/portainer/federated-test.stack.yml:51-60,110-117`).
|
||||||
|
|
||||||
|
**Failure mode:** enforcing the mandatory CA makes current local standalone/federated topologies unable to start; relaxing it to make bootstrap work silently violates K101-REQ-03. A first database cannot be safely migrated until the server certificate, its SAN for the actual service/DNS name, and trusted CA are provisioned, but this lifecycle is not owned or tested.
|
||||||
|
|
||||||
|
**Required remediation:** add a concrete KBN-101-00/05 TLS bootstrap sub-contract: issuer/CA owner; server key/cert and SAN inputs; secure storage/mount permissions; `postgresql.conf`/container TLS enablement; migration and runtime CA mounts; hostname used by each compose/Swarm service; readiness only after TLS authentication; CA overlap rotation; and an existing-database transition. Require a disposable standalone and federated/Swarm test to prove verified TLS succeeds and missing CA, wrong CA, wrong SAN, and `sslmode` downgrade fail before readiness. Do not merge KBN-101-05 with an implicit plaintext exception.
|
||||||
|
|
||||||
|
### HIGH — exact ledger fingerprint and historical 0009 repair are underspecified for existing databases
|
||||||
|
|
||||||
|
The contract requires an “ordered complete set” and rejection of out-of-order rows (`KBN-101-DB-ROLE-SPLIT.md:36-38`), but does not define the canonical serialized tuple, ledger ordering source, or safe upgrade rule for a historical ledger. The current ledger stores only `id`, `hash`, and `created_at` (`packages/db/src/migrate.ts:70-82,105-107`). Its journal is demonstrably non-monotonic: `0008` has `when=1776822435828`, followed by `0009` at `1745280000000` (`packages/db/drizzle/meta/_journal.json:62-79`); the existing PostgreSQL runner documents that this causes skipping (`packages/db/src/migrate.ts:29-35`).
|
||||||
|
|
||||||
|
**Failure mode:** an implementation can either reject a legitimate historical database after correcting 0009, or accept a reordered/duplicated ledger because no precise comparison rule exists. A count/hash-set implementation would fail to detect the condition that this contract explicitly calls unsafe; physical `id` order is not an adequate substitute after historical repair.
|
||||||
|
|
||||||
|
**Required remediation:** freeze a versioned manifest algorithm before implementation: canonical record fields (at least journal index/tag, corrected logical order, migration content hash, and an explicit migration-manifest version), canonical byte serialization, SHA-256 input, and exact observed-ledger mapping. State whether physical ledger insertion order is normative; if not, compare hash-to-manifest tuples rather than timestamps. Add an idempotent migrator-only 0009 existing-database remediation/reconciliation procedure with backup/rollback evidence. Require clean, pre-0009, 0009-skipped, 0009-applied-late, duplicate, unknown, missing, corrupt-pair, and stale-replica cases. No manual ledger insertion is an acceptable production recovery path.
|
||||||
|
|
||||||
|
### MEDIUM — advisory-lock namespace is collision-prone and lacks a fixed identifier contract
|
||||||
|
|
||||||
|
The specified lock is `pg_try_advisory_lock(hashtext('mosaic-schema-migration-v1'))` (`KBN-101-DB-ROLE-SPLIT.md:34`). `hashtext` produces a 32-bit key. Session ownership/crash behavior is otherwise correctly stated (one session, same-session release, connection-close release), but an unrelated database user can accidentally collide or deliberately hold the key and force `DATABASE_MIGRATION_LOCKED`.
|
||||||
|
|
||||||
|
**Failure mode:** avoidable migration denial of service in a shared PostgreSQL database. The current repository already uses separate `hashtext` advisory-lock names for migrate-tier, demonstrating the need for a documented namespace rather than a collision-prone implicit one.
|
||||||
|
|
||||||
|
**Required remediation:** freeze a two-int advisory-lock namespace (fixed documented class/object values) or a documented 64-bit `hashtextextended` key with fixed seed; keep acquisition, migration, verification, and release on the single `max:1` migrator session. Add tests for concurrent migration, connection loss/crash release, readiness while the lock holder is active, and an unrelated lock-key non-interference case.
|
||||||
|
|
||||||
|
### MEDIUM — identifier safety and `search_path` verification need executable constraints
|
||||||
|
|
||||||
|
The contract rightly requires `pg_catalog, <mosaic_application_schema>` and rejects writable paths (`KBN-101-DB-ROLE-SPLIT.md:54,70-77`), but uses dynamic placeholders for database/schema and does not state how migration/bootstrap SQL will avoid identifier interpolation. Existing code has raw-SQL facilities (`packages/storage/src/migrate-tier.ts` uses `.unsafe`), so this is not merely theoretical.
|
||||||
|
|
||||||
|
**Failure mode:** a future operator-configured database/schema value that reaches bootstrap or `SET search_path` through raw string construction can inject DDL, or a pooled connection can retain a mutable search path.
|
||||||
|
|
||||||
|
**Required remediation:** require fixed allowlisted identifiers or server-side identifier quoting (`format('%I', ...)`) only; never interpolate URL/config values into SQL. Set and verify the trusted path per connection/session before any query (`SET LOCAL` inside transactions where applicable), forbid `public`/`$user` additions, and add injection-shaped identifier and pooled-connection reset negatives. Include this in KBN-101-00/01 tests.
|
||||||
|
|
||||||
|
## Acceptance and threat traceability
|
||||||
|
|
||||||
|
| Requirement / threat | Review result | Evidence or blocking finding |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| K101-REQ-01 / AC-K101-01 split runtime/migration URLs | Partial | Role/DTO boundary is coherent; HIGH DDL-path finding requires all current commands to be closed. |
|
||||||
|
| K101-REQ-02 / AC-K101-02 explicit migration/readiness | Blocked | HIGH ledger definition and HIGH DDL-bypass findings. |
|
||||||
|
| K101-REQ-03 / AC-K101-03 least privilege, TLS, grants | Partial | Role model, default privileges, ledger read-only, TEMP/function checks are well specified (`KBN-101...:47-56,70-79`); HIGH TLS bootstrap and MEDIUM identifier constraints remain. |
|
||||||
|
| K101-REQ-04 / AC-K101-04 immutable relations | Correctly deferred | KBN-101-09 after KBN-100 is the correct serial gate (`KBN-101...:58-66,115-118`); no synthetic-only certification claim found. |
|
||||||
|
| K101-REQ-05 / AC-K101-05 N-1, secrets, rollback | Partial | No owner-runtime exception and rollback keeps migration URL out of Gateway (`:83-95`); deployable TLS and full command inventory are missing. |
|
||||||
|
| K101-REQ-06 / AC-K101-07 KBN gates and DAG | Structurally sound | DAG is acyclic: 00→01/{03}; 02→06; 00/01/03→05; 00/04/05/06→07→08→KBN-100→09→KBN-105. KBN-100’s current branch contains docs-only baseline tracking, not schema implementation. |
|
||||||
|
| T: runtime DDL / migration fallback | Blocked | HIGH finding 1. Current Gateway/storage, CLI, direct Drizzle scripts, and integration DDL require explicit closure. |
|
||||||
|
| T: race/crash/readiness | Partial | Same-session nonblocking lock and replica-unready rules are present (`:34-38`); lock namespace remediation required. |
|
||||||
|
| T: immutable evidence rewrite | Correctly staged | Explicit INSERT/SELECT-only matrix and RESTRICT retention are retained; proof is properly after table creation. |
|
||||||
|
| T: secret leakage / TLS downgrade | Partial | Redaction and distinct Vault paths are specified (`:93-97`), but no server TLS/bootstrap implementation contract exists. |
|
||||||
|
|
||||||
|
## Unresolved assumptions
|
||||||
|
|
||||||
|
1. `standalone` and `federated` are the complete PostgreSQL production-like set (K101-A1).
|
||||||
|
2. Each eligible deployment can execute a dedicated migration Job/one-shot phase (K101-A2).
|
||||||
|
3. Vault path names are targets, not verified existing paths; deployment ownership remains to be established.
|
||||||
|
4. PostgreSQL 17 is available for the selected membership and advisory-lock implementation.
|
||||||
|
5. The required server-side TLS issuer/certificate lifecycle and Swarm/compose secret transport have not been decided; this is blocking, not a permissible implicit plaintext bootstrap.
|
||||||
|
6. Historical databases containing the 0009 journal/ledger anomaly have no frozen reconciliation procedure.
|
||||||
|
|
||||||
|
## Independent test and consistency evidence
|
||||||
|
|
||||||
|
Read-only checks run in this review:
|
||||||
|
|
||||||
|
| Check | Result |
|
||||||
|
| --- | --- |
|
||||||
|
| `git diff --check origin/main...da742ca2...` | PASS |
|
||||||
|
| `pnpm exec prettier --check` on all seven changed docs | PASS |
|
||||||
|
| `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` | PASS |
|
||||||
|
| `docker compose -f docker-compose.yml config --quiet` (isolated test ports) | PASS |
|
||||||
|
| `docker compose -f docker-compose.federated.yml --profile federated config --quiet` (isolated test ports) | PASS |
|
||||||
|
| Static journal inspection | FAILS the required monotonic ordering premise: 0008 → 0009 `when` decreases; current runner documents skipping behavior. |
|
||||||
|
| Static DDL-entrypoint inventory | Found direct Drizzle scripts, storage CLI shell-out, runtime extension/migration calls, fleet backlog migration, tier probe extension creation, and a direct-DLL federated integration test. |
|
||||||
|
|
||||||
|
No live database, Vault, CI, deployment, issue, PR, or repository mutation was performed. The pass results validate documentation syntax/contract compilation and compose syntax only; they do **not** certify the proposed security behavior.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
Do not merge this frozen contract as implementation-ready until the HIGH findings are corrected and independently re-reviewed. The central role ownership/default-privilege design, immutable-table staging, and KBN-100/KBN-105 serial gating should be retained; they are not the reason for this REQUEST CHANGES verdict.
|
||||||
183
docs/scratchpads/771-kbn101-db-role-split.md
Normal file
183
docs/scratchpads/771-kbn101-db-role-split.md
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
# Scratchpad — KBN-101 DB runtime/migration role split (#771)
|
||||||
|
|
||||||
|
- **Branch:** `docs/771-kbn101-db-role-split`
|
||||||
|
- **Base:** `main` `e9c4aa3`
|
||||||
|
- **Scope:** planning/documentation only; authorized files are PRD, Native Kanban task/shared/index docs, sitemap, this scratchpad, and the new KBN-101 contract.
|
||||||
|
- **Explicit exclusions:** source/runtime/config/deployment/secret/migration/compose/CI/lock/package/KBN-100 branch edits; no production mutation.
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Freeze an implementation-ready PostgreSQL role/connection split so the Gateway uses a non-owner runtime identity and only a dedicated migration phase uses an owner/migrator identity. Make real deployed-role certification—not synthetic role tests—a serial prerequisite of KBN-100 and KBN-105.
|
||||||
|
|
||||||
|
## Intake and current-state evidence
|
||||||
|
|
||||||
|
- Mission MVP is active; W3 Native Kanban/SOT is planning-complete. The task state shows KBN-010 as the predecessor and KBN-100 as the current schema slice.
|
||||||
|
- Current branch started at `e9c4aa3`; `.mosaic/orchestrator/{mission.json,session.lock}` were already runtime-modified and remain untouched.
|
||||||
|
- `packages/db/src/client.ts`, `migrate.ts`, and `drizzle.config.ts` resolve one `DATABASE_URL` (with default fallback). `packages/storage/src/adapters/postgres.ts` calls `runMigrations(this.url)`.
|
||||||
|
- `apps/gateway/src/database/database.module.ts` calls `storageAdapter.migrate()` at startup for PostgreSQL; this is the owner-runtime defect to remove in KBN-101 implementation.
|
||||||
|
- `packages/config/src/mosaic-config.ts`, installer wizard, local/federated compose, Portainer test stack, and `.woodpecker/ci.yml` currently expose one URL. PGlite has an existing explicit local migration path.
|
||||||
|
- Current KBN contract requires immutable events/checkpoints/artifacts/evidence, `RESTRICT`, and KBN-100 generated Drizzle consistency. It did not establish a deployable runtime identity split.
|
||||||
|
|
||||||
|
## Frozen decisions
|
||||||
|
|
||||||
|
1. `DATABASE_URL` is the non-owner runtime URL; `DATABASE_MIGRATION_URL` is migration-only. Both are required in their respective PostgreSQL phases; PGlite is the explicit local exception; migration never falls back to runtime/default/config URL.
|
||||||
|
2. PostgreSQL Gateway runtime never auto-runs migration/DDL. Dedicated migrator uses `pg_try_advisory_lock(hashtext('mosaic-schema-migration-v1'))`; replicas only check exact ordered Drizzle-ledger readiness and fail closed.
|
||||||
|
3. Roles are non-login `mosaic_platform_database_owner`, non-login `mosaic_schema_owner`, login/noinherit `mosaic_migrator`, non-login `mosaic_runtime_capability`, and login/inherit `mosaic_runtime`. Runtime inherits only its capability role with SET/ADMIN denied, has no owner/migrator membership, no unsafe attributes/ownership/DDL authority, and an explicit trusted search path.
|
||||||
|
4. Runtime gets mutable DML only as needed, but INSERT/SELECT only on `task_events`, `artifacts`, `task_checkpoints`, `task_checkpoint_artifacts`, and `approval_decision_artifacts`. KBN-100 still enforces RESTRICT/no-cascade.
|
||||||
|
5. Startup verifies effective role/ownership/attributes/inherited capability/TEMP/function-execute/ledger grants/search path/immutable denials/schema fingerprint without DSN exposure. It also requires authenticated CA/hostname-verified TLS. Stable sanitized errors and redaction rules are required.
|
||||||
|
6. N-1 retains single runtime URL only as a non-certified compatibility release; staged role provisioning/migration/runtime deployment then enforces the split. Rollback never injects migration URL into Gateway.
|
||||||
|
7. Vault target paths, rotation, deployment injection, CI, installer, compose, Portainer, and observability are separate one-card/one-PR handoffs. The migration-only file manifest includes `packages/db/drizzle.config.ts`; KBN-101 repairs the known PostgreSQL runner/journal ordering defect and proves a clean database applies every hash once before its foundation certificate. No application migration creates roles/passwords or hardcodes credentials.
|
||||||
|
8. KBN-101 foundation merges/certifies first. KBN-100 then rebases, restores Drizzle declaration/snapshot/journal consistency, and bounds procedural immutable-table grant/trigger/backfill work to its own slice. Because those immutable relations do not exist until KBN-100, KBN-101’s real deployed-role immutable-operation certificate follows KBN-100 and is the serial gate before KBN-105.
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- `standalone` and `federated` are all current PostgreSQL production-like modes; a future PostgreSQL tier inherits this contract unless versioned otherwise.
|
||||||
|
- Deployment will support a dedicated migration Job/one-shot command. A target that cannot run it cannot receive production/federated KBN certification.
|
||||||
|
- Canonical Vault target paths require deployment-owner verification before provisioning; the planning document does not claim they already exist.
|
||||||
|
|
||||||
|
## Documentation produced
|
||||||
|
|
||||||
|
- `docs/PRD.md`: bounded KBN-101 requirements and acceptance criteria.
|
||||||
|
- `docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md`: normative rc.5 implementation, threat, migration/rollback, evidence, and exact file DAG contract.
|
||||||
|
- `docs/native-kanban-sot/SHARED-CONTRACT.md`: rc.5 amendment preserving rc.4.
|
||||||
|
- `docs/native-kanban-sot/TASKS.md`: KBN-101 inserted before and blocks KBN-100; KBN-105 held.
|
||||||
|
- Native Kanban index and root sitemap links.
|
||||||
|
|
||||||
|
## Validation plan
|
||||||
|
|
||||||
|
1. Prettier only for changed Markdown.
|
||||||
|
2. Markdown link target/check checks scoped to modified docs.
|
||||||
|
3. Strict contract check with `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` (the frozen TypeScript contracts remain unchanged).
|
||||||
|
4. Diff allowlist proves only authorized documentation files changed, apart from pre-existing Mosaic runtime state.
|
||||||
|
5. Independent documentation/security self-review: role escalation, fallback, startup DDL, schema readiness, grants/default privileges, immutable tables, secret leakage, deployment and KBN-100 boundaries.
|
||||||
|
|
||||||
|
## Review corrections
|
||||||
|
|
||||||
|
Independent Codex review found two blockers and security review found two medium defects; all were remediated in the frozen contract:
|
||||||
|
|
||||||
|
1. Split the KBN-101 certificate into a foundation role/schema-boundary certificate (before KBN-100) and real immutable-operation certificate (after KBN-100, before KBN-105). This preserves the requested KBN-100 block without requiring evidence for tables not yet created.
|
||||||
|
2. Removed the legacy owner-runtime exception. N-1 compatibility preserves variable/config shape only; the KBN-101 runtime refuses owner/migrator identity and current single-URL installs remain on their previous release until role cutover.
|
||||||
|
3. Introduced `mosaic_platform_database_owner` as a separate non-login platform role. `mosaic_schema_owner` owns application/ledger schemas only, not the database and has no database CREATE/ALTER/extension authority.
|
||||||
|
4. Replaced blocking `pg_advisory_lock` with `pg_try_advisory_lock` and the deterministic `DATABASE_MIGRATION_LOCKED` failure.
|
||||||
|
5. Review also flagged active `.mosaic/orchestrator` state. It was pre-existing launcher state and remains unstaged/uncommitted.
|
||||||
|
6. Second review added `packages/db/drizzle.config.ts` to the migration-only slice, mandates `DATABASE_MIGRATION_URL` with a missing-variable negative, grants runtime only `USAGE` plus `SELECT` on `drizzle.__drizzle_migrations`, and verifies/revokes its ledger writes.
|
||||||
|
7. Security review added `DATABASE_TLS_CA_CERT_PATH` / `DatabaseTlsConfigDto` with authenticated TLS and hostname/CA verification in production-like modes, explicit database `TEMPORARY` revocation/catalog denial testing, and default-PUBLIC function EXECUTE revocation with SECURITY DEFINER prohibited by default.
|
||||||
|
8. Final review corrected the runtime login to inherit only its capability role with SET/ADMIN denied, and moved the known hash-complete migration-runner/journal repair into KBN-101-03 before the foundation certificate.
|
||||||
|
9. Final manifest review added all live runtime DDL paths (`packages/storage/src/tier-detection.ts`, Gateway startup, and `fleet-backlog`) to KBN-101-02, requiring read-only extension probes and no PostgreSQL runtime auto-migration. It also requires KBN-101-04 to stop persisting either DSN into generated `.env`/`mosaic.config.json`, using only Vault/deployment references and injected variables.
|
||||||
|
10. Provisioning review separated the external privileged platform bootstrap actor from the NOCREATEDB database-owner role and added KBN-101-00. That IaC/bootstrap card owns fresh/existing database role/ownership/grant/Vault transition evidence and is a foundation-certificate dependency.
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
- `pnpm exec prettier --check` on every authorized Markdown file: PASS.
|
||||||
|
- Markdown link and whitespace checker on all seven authorized Markdown files: PASS.
|
||||||
|
- `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json`: PASS (frozen strict contracts unchanged).
|
||||||
|
- Codex code review iterated through role inheritability, hash-complete migration ordering, all reachable runtime DDL entrypoints, installer DSN persistence, and platform-bootstrap ownership; each finding was incorporated into the final frozen contract/DAG. The last security review found no new KBN-101 vulnerability; its sole low finding is the pre-existing unstaged Mosaic session-lock metadata, which is excluded from this commit.
|
||||||
|
- Commit: `82ce3252df38a687c50485f8d048b53ca8db5989` (`docs(#771): freeze database runtime role split`).
|
||||||
|
- Pre-push queue guard: `ci-queue-wait.sh --purpose push -B main` returned `state=unknown` without failure. The push hook ran repository `pnpm typecheck`, `pnpm lint`, and `pnpm format:check`: PASS.
|
||||||
|
- Pushed branch `docs/771-kbn101-db-role-split` at the exact commit above; no PR was opened, merged, or closed. `web1:mosaic-100` received the handoff with head, decisions, DAG, and validation.
|
||||||
|
- Awaiting independent security/Ultron review.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.6 exact-head remediation session
|
||||||
|
|
||||||
|
- **Objective / correction:** Replace the prior planning-author handoff and close every finding in the [independent exact-head report](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) against `da742ca2da4a2ff466916c818fe275c4f7ffd384`. The report is a verbatim durable copy of the task-supplied review artifact; scope remains documentation-only, `.mosaic` is excluded, and source/config/compose/CI/deployment/secrets/migrations remain untouched.
|
||||||
|
- **Finding 1 — closed DDL control plane:** rc.6 names `mosaic-db-migrator` as the sole application/CI/test PostgreSQL DDL runner, requires `DATABASE_MIGRATION_URL` before connection/DDL, inventories `runMigrations`, Drizzle config/scripts, `db:push`, storage CLI, adapter/Gateway startup, fleet-backlog, extension probes/bootstrap, direct federated integration DDL, CI, and future scripts, and specifies route/deny/test disposition for each. Tests use runner-prepared disposable PostgreSQL or invoke that runner; `db:push` is local-disposable-only and rejects production-like URLs.
|
||||||
|
- **Finding 2 — deployable TLS:** rc.6 freezes distinct runtime/migrator URL and CA/server leaf Vault/compose/Swarm secret identifiers, `0400` key and `0600` URL/cert/CA mount requirements, actual compose/Swarm service-DNS SANs, PostgreSQL TLS settings, legacy-client drain/termination plus `hostssl` enforcement, verified-TLS readiness ordering, fresh/existing transition, CA-overlap rotation/TLS-only rollback, and standalone plus federated/Swarm positive and missing/wrong CA/SAN/downgrade negatives. PGlite is explicitly non-PostgreSQL evidence.
|
||||||
|
- **Finding 3 — manifest/0009:** rc.6 defines manifest v1 canonical UTF-8 serialization and raw SQL-byte SHA-256, logical journal order, manifest ownership/grants, exact one-to-one observed hash tuple mapping, non-normative physical insertion order, safe original-0009 conditions, ambiguous-effect fail-closed recovery, and the full required reconciliation/backup test matrix. It preserves shipped 0009 bytes and forbids manual ledger adoption/insertion.
|
||||||
|
- **Finding 4 — advisory lock:** replaced `hashtext` with fixed signed-int4-safe `(1297044289,1262636593)` (`MOSA`,`KBN1`), one `max:1` runner session, close-on-crash semantics, and contention/crash/readiness/unrelated-key evidence.
|
||||||
|
- **Finding 5 — identifiers/search path:** selects `mosaic`, exact `pg_catalog,mosaic` per pooled connection and `SET LOCAL` transactions, plans audited public-object/extension/Drizzle relocation, forbids config-derived identifiers, limits bootstrap quoting to server-side `%I` on fixed allowlist, and requires injection/pool-reset negatives.
|
||||||
|
- **Finding 6 — safe DAG:** cards 00–07 are inactive prepared capability while owner-runtime remains N-1; KBN-101-08 is the one atomic activation release after platform roles/TLS and compatible code. Mosaic control plane/Jason alone can activate/rollback; no force-on-red, runtime bypass, or temporary compatibility survives the gate. The approved role graph, post-KBN-100 immutable certification, and KBN-105 gate remain unchanged.
|
||||||
|
- **Review remediation:** Codex review found the legacy plaintext cutover gap, missing URL-secret bindings, historical `public` migration incompatibility, non-reproducible checkout-byte hashing, CONNECT allowlisting regression, and undocumented direct-DDL operator instructions. rc.6 now requires drain/scale-to-zero, residual non-TLS session termination, `hostssl` with no `host` rule, zero plaintext-session proof, TLS-only post-enforcement rollback, distinct named runtime/migrator secret consumers, canonical Git-blob/LF manifest bytes, a runner-only owner-controlled legacy-public bootstrap followed by `mosaic` relocation, explicit CONNECT/TEMP revocation, and KBN-101-07 replacement of direct-DDL documentation. It also required the durable exact-head report link above. Pre-existing `.mosaic` runtime state remains excluded.
|
||||||
|
- **Validation:** Prettier on all changed Markdown, repository Markdown link/whitespace check, and strict native-kanban contract TypeScript passed before final staging; the final staged diff excludes `.mosaic`. No source-code TDD applies because this is contract-only remediation.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.7 residual remediation session
|
||||||
|
|
||||||
|
- **Objective / correction:** Close every residual in the independent exact-head rc.6 re-review at `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview-45ba3d6.md` for `45ba3d6ad4d5383f457a303c05bc816144cfa48a`, without changing source, compose, CI, deployment, migration, or secret artifacts. Only the existing authorized planning/documentation paths are eligible; pre-existing `.mosaic` state remains excluded.
|
||||||
|
- **Source-backed scope confirmed:** the active `federated-pgvector.integration.test.ts` executes `CREATE TEMP TABLE`; tracked `docker/init-db.sql` and `infra/pg-init/01-extensions.sql` both create `vector`; `migrate-tier.ts` advertises raw `CREATE EXTENSION`; and `tools/federation-harness/docker-compose.two-gateways.yml` is current plaintext two-PostgreSQL/two-Gateway topology. Current `schema.ts` has 36 default-schema `pgTable` declarations, 6 default `pgEnum` declarations, and an unqualified `vector` custom type; historical migrations contain `public` references.
|
||||||
|
- **Plan:** (1) make the finite DDL/static-bypass inventory and `DATABASE_URL`-only denial matrix exact, including the runner-prepared persistent pgvector fixture and migrated two-gateway harness; (2) freeze executable `public`-to-`mosaic` and `mosaic_extensions` transition, Drizzle ownership, object-catalog classes/order, eligibility and rollback tests; (3) bind repository/control-plane ownership, UID/GID validation, exact artifact/mount rules, and both gateway TLS topology; (4) correct PRD acceptance mapping and cross-document rc.7 status; then run formatting, link/contract, source-path, diff, review, commit, queue guard, and push.
|
||||||
|
- **Independent review closure:** initial Codex review found Gateway-key consumer wording, `CLAUDE.md` omission, final schema-owner set, and placeholder SANs; all are now explicit. Re-review found the legacy `0001` vector-type resolution problem and `docs/federation/SETUP.md` raw-DDL instruction; the legacy runner now uses only its fixed non-writable `pg_catalog,public,mosaic_extensions` history path, while runtime remains `pg_catalog,mosaic`, and the federation setup path is assigned to KBN-101-07/static inventory. Security review final verdict: no confident vulnerability. The review also repeated the pre-existing tracked `.mosaic` session-state concern; it remains deliberately unstaged/excluded by this task.
|
||||||
|
- **Completion evidence:** changed Markdown is Prettier-formatted; local links and strict native-kanban TypeScript passed; source-path inventory confirmed all current referenced paths (the new `apps/gateway/Dockerfile` is explicitly a planned KBN-101-05 artifact); diff check and authorized-doc allowlist passed. No source-code TDD applies to this documentation-only remediation.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.8 exact residual remediation session
|
||||||
|
|
||||||
|
- **Objective / correction:** Close all three HIGH findings in the independent rc.7 exact-head re-review at `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview2-0778eba.md` for `0778eba2db3c2dfbaca3af352b12ba0389d3552b`. Scope remains documentation-only: no source, config, Compose, CI, deployment, secret, or migration artifact changed; pre-existing `.mosaic/orchestrator` state remains excluded.
|
||||||
|
- **Finite authority closure:** KBN-101-06 now classifies exact current source/scripts/package bins, operator docs, and deploy manifests. `packages/db/src/index.ts` has explicit removal/compile-import negative ownership; `docs/fleet/backlog-conventions.md` and `docs/PERFORMANCE.md` now remove first-use/direct-Drizzle/Gateway-startup migration instructions and point to sole runner/readiness. Byte-immutable historical SQL, PGlite-only routines, negative-test literals, vendored/generated artifacts, and labeled historical reports are exact-path/category reviewed allowlists; unknown hits fail. The contract explicitly rejects relying on a naive token scan alone.
|
||||||
|
- **Executable and exclusive handoff closure:** KBN-101-03 exclusively owns the published `mosaic-db-migrator` bin, `packages/db/src/cli.ts`, private migrator modules, `docker/db-migrator.Dockerfile`, exact `--run|--verify|--help`, environment/argv limits, sanitized exits, and command/order tests. KBN-101-00 exclusively owns `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, and bootstrap tests. KBN-101-05 exclusively owns `tools/db/render-postgres-secrets.ts`, its tests, and deployment declarations, consuming the versioned bootstrap interface without overlap.
|
||||||
|
- **pgvector owner closure:** `mosaic_extension_owner` is dedicated NOLOGIN, available only to the external bootstrap actor during bootstrap; fresh vector/member ownership remains there. The contract records PostgreSQL's unsupported extension-owner transfer and forbids catalog mutation, ownership adoption, and `DROP CASCADE`. Approved-owner existing extensions use verified `ALTER EXTENSION ... SET SCHEMA`; legacy runtime-owned extensions fail closed to a controlled backup/shadow/runner/copy-evidence/quiesce/final-delta/atomic-switch/read-only-rollback migration. It requires `pg_extension.extowner`, member/schema/version, and runtime/migrator/schema-owner ALTER/DROP/member-update denial tests across clean, approved-owner, legacy shadow, partial/resume/rollback, and N-1.
|
||||||
|
- **Cross-document state:** PRD, KBN contract, shared contract, task decomposition, index, sitemap, current operator docs, and this scratchpad are rc.8-consistent. The only intended next action is a fresh independent exact-head re-review after validation/push.
|
||||||
|
- **Validation / review:** Prettier passed for all nine changed Markdown documents; local links passed (9 documents); `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` passed; source-path inventory passed (20 paths: 8 current, 12 explicitly planned); finite-authority requirement checklist and `git diff --check` passed. Manual documentation/security review checked the three requested paths, private-only runner boundary/exit contract, non-overlapping 00/03/05 ownership, extension-owner denial and shadow path, and `.mosaic` exclusion. No source-code TDD applies because this is contract-only remediation.
|
||||||
|
- **Delivery evidence:** committed `1423c2ad02b5471eab006fb4c878808e5b29c387` as `docs(#771): close role split rc.8 residuals`. Push queue guard returned `state=unknown` without error; push hook ran repository `pnpm typecheck`, `pnpm lint`, and `pnpm format:check`, all PASS; branch push succeeded. This final evidence append is committed next, then the exact remote head is verified. The only intended next action is a fresh independent exact-head re-review.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.9 final residual remediation session
|
||||||
|
|
||||||
|
- **Objective / correction:** Close the three findings in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview3-9cf5d2f.md` against exact head `9cf5d2f6641b14082dc3294e2a84d1fb4ccc019d`: move `mosaic_extensions` schema ownership to `mosaic_extension_owner`; replace all broad/conflicting KBN-101 card ownership with a complete disjoint exact-path/test manifest; and classify the current architecture-plan `db:migrate` instruction with pinned scanner mechanics. Scope remains documentation-only; no source/config/Compose/CI/deployment/secret/migration artifact and no `.mosaic` path may be modified.
|
||||||
|
- **Plan:** inspect current tracked source topology to name only existing paths; update the normative contract first and synchronize PRD/shared/task/index/sitemap/version language; run Prettier, changed-doc links, strict contract TypeScript, source/path and manifest-overlap checks, diff allowlist, independent documentation/security review; then stage docs only, commit, queue-guard, push, and verify exact remote SHA. No source-code TDD applies because this is contract-only remediation.
|
||||||
|
- **Closure implemented:** rc.9 makes `mosaic_extension_owner` create and own `mosaic_extensions`, `vector`, and members; the external bootstrap actor alone temporarily `SET ROLE`s for fresh/approved-owner work, while schema owner has only `USAGE` for legacy type resolution and never temporary `CREATE`. Catalog/default-ACL plus direct DDL/member denials now cover runtime, migrator, and schema owner through fresh, relocation, shadow/resume, and rollback evidence.
|
||||||
|
- **Delivery decomposition:** Replaced broad ownership with complete disjoint 00–09 manifests, exact tests/evidence, producer-before-consumer edges, and an explicit no-intermediate-deploy N-1 activation statement. `packages/storage/src/{cli,migrate-tier}.ts` belongs only to -02; the current tracked init artifacts are retired by -02 as direct-DLL closure; -03 owns all runner/index/migrate/config/schema assets and exact compiled-bin/image mapping; -07 owns docs only; -08/-09 own evidence only.
|
||||||
|
- **Classifier closure:** -06 has exact scanner/inventory/matrix paths, canonical inventory fields, classes/dispositions, fixed token/rule set, exact allowlist categories/restrictions, and self-test requirements for unknown, duplicate-owner, ownerless, missing-path, and historical masking cases. The architecture plan now marks direct `db:migrate` superseded and uses `mosaic-db-migrator --run`.
|
||||||
|
- **Validation:** Prettier check, strict native-kanban contract TypeScript, changed-document local-link resolution, `git diff --check`, and an automated manifest-overlap/owner/current-source-path check passed. Targeted documentation/security review verified role ownership/default privileges/search path/preflight/legacy/shadow/rollback consistency, disjoint manifests/DAG/activation, scanner mechanics, exact bin/entrypoint, and no `.mosaic` staging intent. No source-code TDD applies because this is documentation-only remediation.
|
||||||
|
- **Next:** stage documentation only, commit, queue-guard, push, verify exact remote SHA, then wait for fresh exact-head review.
|
||||||
|
- **Delivery evidence:** committed `8cbad2bcd9bc7507052f74f35670ef7c8e39e44e` as `docs(#771): close role split rc.9 residuals`; `ci-queue-wait.sh --purpose push -B main` returned `state=unknown` without error; push-hook `pnpm typecheck`, `pnpm lint`, and `pnpm format:check` all passed; push succeeded and `origin/docs/771-kbn101-db-role-split` resolved to that exact SHA. Pre-existing `.mosaic/orchestrator/{mission.json,session.lock}` remains modified but intentionally unstaged/excluded. The only next action is a fresh independent exact-head re-review.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.10 Ultron NO-GO remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 and HIGH-2 in `/home/hermes/agent-work/reviews/771-kbn101-ultron-11f09a1.md` against exact head `11f09a15e43e72afda6a0374668996b4bda9e536`. Documentation only: preserve approved content; no source/config/Compose/CI/deploy/secret/migration edits and no `.mosaic` staging.
|
||||||
|
- **Plan:** (1) replace the impossible `NOLOGIN NOSUPERUSER` extension owner with the exact non-login, zero-member `NOLOGIN SUPERUSER` external-control exception and document the non-delegable superuser residual; (2) require the audited external superuser session to `SET ROLE`/`RESET ROLE` for fresh, approved-owner, and shadow extension work, then prove catalog ownership and all service-role denial; (3) assign the active migrate-tier guide exclusively to KBN-101-07, add its active secure route to the -06 inventory/matrix/scanner schema, and freeze `--target-url-file /run/secrets/mosaic_migrate_target_url` plus pre-migrated target/dedicated non-DDL importer requirements; (4) synchronize PRD/shared/tasks/index/sitemap/guide and rc.10 status; (5) validate formatting, links, contracts, source paths, finite operator inventory, diff, review, commit, queue guard, push, and exact remote SHA.
|
||||||
|
- **Target-image evidence before edits:** local `pgvector/pgvector:pg17` control file reports `default_version = '0.8.2'`, `relocatable = true`, and no `trusted`/`superuser` override (untrusted PostgreSQL extension). An isolated PostgreSQL 17 container proved a `NOLOGIN SUPERUSER` `mosaic_extension_owner` can create `mosaic_extensions` and `vector` under external-superuser `SET ROLE`, returns to the external session after `RESET ROLE`, has `rolcanlogin=false`, `rolsuper=true`, zero role members, exact extension/schema and owner-bearing-member ownership, and denies `SET ROLE`, `ALTER EXTENSION`, `DROP EXTENSION`, and schema ownership changes to runtime, migrator, schema owner, and data importer. Ownerless PostgreSQL catalog member classes (`pg_am`, `pg_cast`) were intentionally not misrepresented as ownable members.
|
||||||
|
- **TDD decision:** skipped as not applicable: this is a documentation-only contract remediation. Future KBN-101-00/-02/-06 tests are specified as the situational evidence; no source/test artifact is permitted in this task.
|
||||||
|
- **Final scope correction:** Per control-plane direction, remediation remains bounded to the two Ultron findings. The active guide is explicitly a non-operative KBN-101 contract until its owned implementation/activation lands; no additional design, source, CI, deployment, secret, or test artifact was added.
|
||||||
|
- **Validation evidence:** target-image/container role proof PASS (pgvector `0.8.2`, `relocatable=true`, trusted absent/untrusted; external-superuser `SET ROLE`/`RESET ROLE`; exact extension/schema/owner-bearing-member ownership; zero membership and service-role denials). Changed-doc Prettier, strict native-kanban contract TypeScript, local-link resolver (8 docs), finite operator-doc inventory (one active KBN-101-07 route with no credential argv in executable blocks), source-path check (6 current paths), and `git diff --check` PASS. Pre-existing `.mosaic/orchestrator/{mission.json,session.lock}` remains intentionally excluded.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.11 exact-head re-review remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 and HIGH-2 in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview5-f60144e.md` against `f60144eb3eab6234ab01bda592052081c777897e`: target-bind the non-DDL tier importer with a runner-produced signed attestation, and disposition every current non-normative documentation scanner hit, including the active user-guide route and federation historical status. Documentation only; no source/config/Compose/CI/deployment/secret/migration edits and no `.mosaic` staging.
|
||||||
|
- **Frozen decision:** `mosaic-db-migrator --verify` is trusted only after TLS/identity/manifest/schema verification and signs a credential-free JCS/Ed25519 v1 artifact from a runner-only fixed root-owned private-key file. The artifact binds secret version/exact URL-file digest, canonical TLS/CA/SPKI/server/database/importer/manifest/schema identity, issued/expiry/nonce, and producer build/correlation; importer gets pinned public key plus artifact only. It validates both files and all bindings before target connection, opens/digests/connects from one in-memory URL read, validates server identity before DML, and distinguishes zero connection from zero DML. Key overlap/revocation, secret-rotation invalidation, replay cache, atomic rename, and sanitized errors are mandatory.
|
||||||
|
- **Ownership:** -03 owns producer/signing DTO/tests; -02 importer interface/verification tests; -05 key/artifact mounts/render tests; -06 inventory/matrix and non-masking scanner tests; -07 operator guide. The exact manifests remain disjoint.
|
||||||
|
- **Operator correction:** `storage migrate` is schema-wrapper delegation only; legacy `--from hot --to cold` tier-copy guidance is unavailable. Secure tier copy is `migrate-tier` with `--target-url-file` plus `--target-attestation-file`. Federation M1 task language is status-only and adjacent KBN-101 text says it authorizes no current DDL.
|
||||||
|
- **TDD decision:** skipped as not applicable: this bounded task changes documentation only. Future -02/-03/-05/-06 tests are specified as the required implementation evidence.
|
||||||
|
- **Validation / review:** Prettier PASS on all 18 changed Markdown/root-doc files; `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` PASS; changed-doc local-link resolver PASS (71 links); full current docs scanner/disposition PASS (10 non-normative paths, 4 normative-contract paths; no unknown active command); legacy `storage migrate --from` and raw target-URL bypass scan PASS; attestation field/interface assertion PASS; exact source ownership/manifest-overlap assertion PASS; `git diff --check`, docs-only scope, and secret-leak scan PASS. Manual documentation/security review verified key isolation, JCS/detached signature, secret-file hash as non-secret evidence, verification ordering/TOCTOU/replay/rotation, no connection vs zero DML, non-masking scanner class, status-only federation history, and no regression to pgvector closure. No source-code TDD applies.
|
||||||
|
- **Delivery evidence:** committed `6227f076c819bd124383851633b16d4ef9c88a98` as `docs(#771): bind tier importer to verified target`; staged scope was 18 documentation files only and excluded `.mosaic`. Pre-push queue guard returned `state=unknown` without failure. Push hook ran repository `pnpm typecheck`, `pnpm lint`, and `pnpm format:check`: PASS. Branch push succeeded. Verify the exact remote SHA after this final delivery-evidence append, then idle for independent exact-head re-review and Ultron reverify. `.mosaic/orchestrator/{mission.json,session.lock}` remains pre-existing and excluded.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.12 bounded deployable-importer/SETUP remediation plan
|
||||||
|
|
||||||
|
- **Objective / scope:** Close the two HIGH findings in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview6-65663d4.md` against exact head `65663d4f72f2ace5148bce9aeba04b5a8d5beee9`. Documentation/tracking only; preserve every earlier closure; do not modify source, deployment, Compose, CI, migration, Vault, or `.mosaic` artifacts.
|
||||||
|
- **Plan:** (1) make KBN-101-05 own canonical KV-v2 importer URL/version provenance, separate immutable generation-pinned renderer consumers, importer CA/public-key/attestation mounts, fixed importer/migrator identities, safe-fd lifecycle, isolation/rotation/TOCTOU/error evidence, and -02/-03/-06 handoffs; (2) convert `docs/federation/SETUP.md` to a non-operative N-1 reference with only the required external-bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway-readiness sequence; (3) broaden scanner grammar plus path-specific semantic negatives so indirect first-boot/startup/init/Compose authority cannot be masked by a named, normative, or status record; (4) synchronize PRD/shared/tasks/index/sitemap/federation task state and this scratchpad; (5) run formatting, links, strict contracts, complete-doc scanner/semantic assertions, diff/operator inventory, material/manifest overlap, review, docs-only stage, commit, queue guard, push, and exact remote-SHA verification.
|
||||||
|
- **TDD decision:** skipped because this bounded change is documentation-only; the affected -02/-03/-05/-06 implementation tests and scanner semantic fixtures are specified as mandatory future evidence.
|
||||||
|
- **Review correction:** independent Codex review found the initial `10003` producer → immutable `10002` importer artifact handoff impossible. rc.12 now specifies the required privileged deployment handoff controller: after runner success it safe-opens/verifies producer artifact plus generation, exact-byte copies/fsyncs/atomically renames to a distinct `10002:10002` `0400` importer mount, seals it read-only, and starts no importer on partial/wrong-generation/owner/mode failure. It receives only a root-owned non-secret expected-version/URL-digest/generation descriptor plus public verifier key, never URL bytes/private key; producer/importer share no writable file or mount. Dry-run nonce consumption now requires fresh `--verify` and artifact before `--yes`; the active-route schema requires `targetCredentialVersionFile`. Pre-existing `.mosaic` state is confirmed excluded from staging.
|
||||||
|
- **Validation result:** changed-doc Prettier and `git diff --check` PASS; strict `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` PASS; contract/SETUP material and non-operative semantic assertions PASS. Pending final docs-only stage, commit, queue guard, push, and remote-head verification.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.13 MILESTONES semantic-scan remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 from `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview7-7365dcf.md` against `7365dcf15c09262a46132b9c011769ad98243641`. Documentation/tracking only: assign `docs/federation/MILESTONES.md` exclusively to KBN-101-07 and its exact former startup-extension wording to the KBN-101-06 semantic fixture/inventory; replace the wording with a non-operative historical/status disposition. No source, deployment, Compose, CI, Vault, migration, provider, or `.mosaic` artifact is authorized.
|
||||||
|
- **Frozen remediation:** runtime/startup extension provisioning is superseded and forbidden. The sole eligible sequence is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway readiness. The MILESTONES record authorizes no current DDL, Compose/init, or startup path. The scanner must prove the exact former wording fails before any inventory/status-only mask and rerun its full current-doc operator/deploy-manifest scan outside reports/scratchpads.
|
||||||
|
- **Plan:** update only MILESTONES plus the exact KBN-101 contract/inventory/manifests and necessary PRD/shared/task/index/sitemap/version/status references; run full lexical+semantic scan, Prettier, links, strict contract TypeScript, diff and manifest-overlap checks; stage docs only (excluding pre-existing `.mosaic`), commit, queue-guard, push, and verify the exact remote SHA. No source-code TDD applies because this bounded task changes documentation only.
|
||||||
|
- **Remediation result (pre-commit):** `MILESTONES.md` now makes runtime/startup extension provisioning superseded and forbidden, with only external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `--verify` → Gateway readiness; it authorizes no current DDL/Compose/init/startup path. The KBN-101-07 manifest/inventory is exclusive and KBN-101-06 documents the exact former wording as a semantic negative that fails before inventory masking. Full current-doc scan outside reports/scratchpads: 103 Markdown files, 10 lexical-hit paths classified, 2 owned Compose-before-runner references, and zero ownerless indirect/literal routes. Prettier, strict native-kanban TypeScript, local links (64/0), diff check, and 95-path manifest-overlap reconstruction (0 overlaps; MILESTONES only -07) passed. Pre-existing `.mosaic/orchestrator/{mission.json,session.lock}` remains excluded.
|
||||||
|
- **Delivery checkpoint:** committed remediation as `237bac81c93dc4305470cea23a67e4ced730bd61` (`docs(#771): close MILESTONES startup authority`). Push queue guard returned `state=unknown` without failure; the push hook ran repository `pnpm typecheck`, `pnpm lint`, and `pnpm format:check`, all PASS; the remote branch resolved to that exact SHA. This final evidence append is committed next, then the exact remote head is verified and the branch waits for independent exact-head rereview/Ultron reverify. `.mosaic` remains unstaged.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.13 current-document safety remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 and HIGH-2 in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview8-aeacc70.md` against exact head `aeacc702353740aad0f2f086974cc0670e360d1d`. This is documentation/tracking only. Preserve all prior gates; do not edit source, Compose, deployment artifacts, CI, migrations, Vault data, reports, or `.mosaic`.
|
||||||
|
- **Source-backed decision:** Current `docker-compose.yml` mounts `infra/pg-init` into PostgreSQL init and that SQL creates `vector`; it cannot be used as a current PostgreSQL start route before KBN-101 bootstrap/runner artifacts exist. The checked-in configuration declares a supported `local` PGlite tier (`DEFAULT_LOCAL_CONFIG` and `tier-detection` both establish in-process PGlite with no external service probe), so docs may retain a local/PGlite route and start only non-PostgreSQL Compose services such as `valkey`.
|
||||||
|
- **Plan:** (1) replace the README and dev-guide Compose-first PostgreSQL instructions with a PGlite/no-PostgreSQL developer path and an explicit held PostgreSQL/federated future activation sequence; (2) replace the deployment quick-start and bare-metal production procedure with non-operative status, no production `.env`/automatic dotenv/`EnvironmentFile`/credential export-or-argv/restart guidance, and only a non-executable future renderer/Vault generation-pinned process-exec or `LoadCredential` schematic; (3) expand KBN-101-06/-07 semantic fixture/disposition language to fail the exact former README/dev/deployment Compose-first sequences and production credential routes before ownership/status masking; (4) synchronize PRD/shared/tasks/index/sitemap/status/version and this scratchpad; (5) run formatting, links, strict contract TypeScript, full current-doc lexical+semantic scan outside reports/scratchpads, manifest-overlap, review, docs-only stage, commit, queue guard, push, and exact remote-SHA verification.
|
||||||
|
- **TDD decision:** no source or fixture implementation may be changed in this documentation-only remediation. The -06 future fixture requirements are frozen as acceptance evidence; validation here is static semantic inventory plus documentation quality gates.
|
||||||
|
- **Correction from independent review:** The initial local-Gateway PGlite wording was unsafe. `apps/gateway/src/main.ts` loads daemon/root/app-local environment files before tier selection; an inherited daemon `DATABASE_URL` can select PostgreSQL, whose current startup reaches extension creation and migrations. No source is authorized in this docs-only task. The remediation therefore holds Gateway/Web local startup, preserves only PGlite data-layer plus selected non-PostgreSQL Compose work, and assigns KBN-101-02 the fail-closed daemon/inherited/root/app-local DSN and non-local-tier rejection before connection/DDL, with a regression proof. The future renderer boundary remains KBN-101-05.
|
||||||
|
- **Remediation evidence:** Removed active PostgreSQL Compose-first and production credential guidance from README, CLAUDE, dev/deployment, and the residual historical TUI/MCP routes; local documentation now permits only PGlite data-layer/non-PostgreSQL Valkey work while Gateway/Web startup is explicitly held. KBN-101-06/-07 now freeze exact former README/dev/deployment Compose sequences plus production credential patterns as pre-classification semantic negatives. Independent review surfaced the current daemon/root/app dotenv loader as an unsafe source boundary; no source is authorized here, so the docs hold that startup and assign fail-closed removal/regression proof to KBN-101-02.
|
||||||
|
- **Validation:** Prettier PASS (12 changed docs); local-link resolver PASS (71 links); `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` PASS; full README/CLAUDE/docs inventory PASS (104 documents, 0 active non-normative Compose/init/production-credential violations); manifest reconstruction PASS (10 cards, 95 declared path tokens, 0 overlaps); `git diff --check` PASS. Pre-existing `.mosaic/orchestrator/{mission.json,session.lock}` remains intentionally unstaged.
|
||||||
|
- **Final route correction:** Held the residual MCP environment/restart and historical TUI smoke-test routes after review showed they could bypass the Gateway local-start hold; no bearer-token-over-HTTP or Gateway restart route remains in this remediation scope.
|
||||||
|
- **Delivery:** committed `7cc156b777189ee89448e4d569a8b3f69560a240` (`docs(#771): hold unsafe database startup routes`) and `d8f935c20ade835aa3ec03fe5d6961885d8b5f0b` (`docs(#771): record final route correction`). Push queue guard returned `state=unknown` without failure; both pushes completed and the remote matched `d8f935c` before this final delivery-evidence append. `.mosaic/orchestrator/{mission.json,session.lock}` remains pre-existing and unstaged. Await a fresh independent exact-head re-review/Ultron verification.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.15 exact one-finding runner/legacy-CI remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview9-be0ebfd.md` against `be0ebfdc6a2b32a0ab6989117ebbb12f43854d71`. Documentation/tracking only: no source, Compose, CI, deployment, migration, Vault, reports, or `.mosaic` edit.
|
||||||
|
- **Plan:** Replace imperative current runner routes in the architecture plan and PERFORMANCE with one explicit non-operative future procedure; make fleet backlog current behavior PGlite-only and PostgreSQL held; classify README's checked-in direct CI `db:migrate` as legacy N-1/uncertified/non-authorizing pending KBN-101-06 removal; then extend the future KBN-101-06 lexical/semantic inventory contract so unqualified runner/current-CI authority fails before masking while only the complete named held procedure passes. Synchronize required contract/PRD/shared/task/index/sitemap state, run full docs scan and document gates, stage docs only, commit, queue-guard, push, and verify remote SHA.
|
||||||
|
- **TDD decision:** not applicable: the user authorizes documentation only and the named -06 fixture/inventory files do not yet exist; the contract records their required future implementation evidence.
|
||||||
|
- **Remediation / review closure:** Architecture, PERFORMANCE, federation SETUP/MILESTONES, deployment/dev/migrate-tier, and README now use one Markdown-bounded `Held future procedure` form where needed: non-operative/no-current-command-authority; KBN-101-00/-03/-05; external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Any runner hit outside that section is a -06 semantic failure. Fleet backlog current behavior is PGlite-only. README accurately records the checked-in direct CI migration as an active, isolated-disposable-database, uncertified legacy N-1 DDL exception that is non-authorizing as an operator route and pending -06 removal; it no longer falsely claims the current CI role lacks DDL capability. Independent Codex review found and this pass closed the readiness-endpoint, standalone-verify, CI-factuality, and scanner-boundary findings. Its only residual finding concerns pre-existing tracked `.mosaic/orchestrator` runtime state, which is explicitly excluded and unstaged by task scope; security review found no vulnerability.
|
||||||
|
- **Validation:** changed-doc Prettier PASS; `pnpm exec tsc --noEmit -p docs/native-kanban-sot/tsconfig.json` PASS; changed-doc local links 72/0; `git diff --check` PASS; full README/CLAUDE/docs lexical+semantic scan outside reports/scratchpads PASS (106 Markdown documents; 7 operator documents with runner tokens; zero unqualified future-runner/current-CI authority routes); KBN-101 manifest check PASS (10 dependency-ordered cards; -07 docs/-06 fixtures disjoint); docs-only allowlist PASS (16 docs, pre-existing `.mosaic` excluded).
|
||||||
|
- **Delivery evidence:** committed `d857463a8a4658e34a77177737860cf82cc26ac6` (`docs(#771): hold unimplemented runner routes`). Pre-push queue guard returned `state=unknown` without failure; the push hook ran repository `pnpm typecheck`, `pnpm lint`, and `pnpm format:check`, all PASS. Push succeeded and `origin/docs/771-kbn101-db-role-split` matched `d857463a8a4658e34a77177737860cf82cc26ac6`. This evidence append is committed and pushed next; `.mosaic/orchestrator/{mission.json,session.lock}` stays pre-existing, unstaged, and excluded. Await exact-head independent re-review/Ultron reverify.
|
||||||
|
|
||||||
|
## 2026-07-15 — rc.16 exact one-finding generic-wrapper remediation intake
|
||||||
|
|
||||||
|
- **Objective / scope:** Close only HIGH-1 in `/home/hermes/agent-work/reviews/771-kbn101-contract-rereview10-18e253c.md` against exact head `18e253c8790bdbb5bc30a06c116472213b83b22f`. Documentation/tracking only: no source, Compose, CI, deployment, migration, Vault, report, or `.mosaic` change.
|
||||||
|
- **Source-backed correction:** `packages/storage/src/cli.ts` currently labels `storage migrate` a thin wrapper for `pnpm --filter @mosaicstack/db db:migrate` and executes that direct Drizzle command with `execSync`; no `mosaic-db-migrator` executable exists. Therefore the README commented form and user-guide executable form must not describe runner delegation or provide current command authority.
|
||||||
|
- **Plan:** Remove the current wrapper command from README/user-guide command guidance; record it as legacy N-1, uncertified, non-operative, and forbidden pending KBN-101-02/-03/-06/-08 activation. Retain only the held future ordered bootstrap → TLS/roles → runner `--run` → `--verify` → readiness sequence and the separately held secure migrate-tier route. Extend KBN-101-06's future semantic fixture/matrix with both exact former forms (including the README commented code-fence form), requiring their failure before inventory/status masking and a source-consistency assertion that direct Drizzle wrapper source cannot be described as runner delegation. Synchronize status/version references only where required, then run document gates, stage docs only, commit, queue-guard, push, and verify the remote SHA.
|
||||||
|
- **TDD decision:** not applicable: this bounded task changes documentation only; the required future -06 semantic/source-consistency fixtures are specified as implementation acceptance evidence.
|
||||||
|
- **Remediation / validation:** README and user-guide remove the generic wrapper from command guidance and state the direct-Drizzle current-source truth, legacy-N-1/uncertified/non-operative MUST-NOT-INVOKE boundary, named -02/-03/-06/-08 activation cards, future external-bootstrap → TLS/roles → runner `--run` → `--verify` → readiness sequence, and separately held secure migrate-tier route. The KBN-101 rc.16 contract records both exact former forms (README commented code fence and user-guide executable code fence), requires failure before inventory/ownership/status masking, and requires the direct-Drizzle/no-runner-bin source-consistency proof. Prettier passed on all nine changed Markdown documents; changed-doc local links passed (72/0); strict native-kanban contract TypeScript and `git diff --check` passed; full README/CLAUDE/docs scan outside reports/scratchpads passed (106 documents, zero non-normative executable generic-wrapper or false runner-delegation route); and manifest validation passed (10 cards, 90 exact tokens, zero overlaps). Pre-existing `.mosaic/orchestrator/{mission.json,session.lock}` remains excluded.
|
||||||
Reference in New Issue
Block a user