efdbb13ab4
Seeds a temp PGlite with representative cross-table data (users, teams, team_members, conversations, messages), runs mosaic storage migrate-tier against a live federated PG+pgvector profile, asserts exact row counts and spot-checks key columns. Gated by FEDERATED_INTEGRATION=1. Caught and fixed P0 bug in migrate-tier: PostgresMigrationTarget passed Drizzle-selected camelCase keys (emailVerified, userId, ...) verbatim as SQL identifiers, failing on Postgres which expects snake_case. The 32 unit tests missed this because both source and target were mocked. normaliseSourceRow now applies toSnakeCase conversion that is idempotent on already-snake_case keys. Test infrastructure: - packages/storage/src/test-utils/pglite-with-vector.ts: PGlite + @electric-sql/pglite/vector (JS-native pgvector) and migration runner. Co-located with storage tests rather than exposed on @mosaicstack/db public surface (would have polluted prod consumers with WASM bundle). - packages/storage now declared "type": "module" (codebase convention, required for import.meta.url in test-utils). afterAll cleanup deletes prefix-namespaced rows even on test panic. Refs #460
77 lines
8.9 KiB
Markdown
77 lines
8.9 KiB
Markdown
# Tasks — Federation v1
|
||
|
||
> Single-writer: orchestrator only. Workers read but never modify.
|
||
>
|
||
> **Mission:** federation-v1-20260419
|
||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||
> **Agent values:** `codex` | `glm-5.1` | `haiku` | `sonnet` | `opus` | `—` (auto)
|
||
>
|
||
> **Scope of this file:** M1 is fully decomposed below. M2–M7 are placeholders pending each milestone's entry into active planning — the orchestrator expands them one milestone at a time to avoid speculative decomposition of work whose shape will depend on what M1 surfaces.
|
||
|
||
---
|
||
|
||
## Milestone 1 — Federated tier infrastructure (FED-M1)
|
||
|
||
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
||
|
||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||
| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ------------------------------- | ---------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- |
|
||
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
||
| FED-M1-02 | done | Author `docker-compose.federated.yml` as an overlay profile: Postgres 17 + pgvector extension (port 5433), Valkey (6380), named volumes, healthchecks. Compose-up should boot cleanly on a clean machine. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471. Overlay defines `postgres-federated`/`valkey-federated`, profile-gated, with pg-init for pgvector extension. |
|
||
| FED-M1-03 | done | Add pgvector support to `packages/storage/src/adapters/postgres.ts`: create extension on init (idempotent), expose vector column type in schema helpers. No adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472. `enableVector` flag on postgres StorageConfig; idempotent CREATE EXTENSION before migrations. |
|
||
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
||
| FED-M1-05 | done | Write `scripts/migrate-to-federated.ts`: one-way migration from `local` (PGlite) / `standalone` (PG without pgvector) → `federated`. Dumps, transforms, loads; dry-run + confirm UX. Idempotent on re-run. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474. `mosaic storage migrate-tier`; DrizzleMigrationSource (corrects P0 found in review); 32 tests; idempotent. |
|
||
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
||
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
||
| FED-M1-08 | in-progress | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Caught real P0 bug in M1-05 (missed by mocked unit tests): camelCase→snake_case column name conversion. Fix in same PR. |
|
||
| FED-M1-09 | not-started | Standalone regression: full agent-session E2E on existing `standalone` tier with a gateway built from this branch. Must pass without referencing any federation module. | #460 | haiku | feat/federation-m1-regression | FED-M1-07 | 4K | Reuse existing e2e harness; just re-point at the federation branch build. Canary that we didn't break it. |
|
||
| FED-M1-10 | not-started | Code review pass: security-focused on the migration script (data-at-rest during migration) + tier detector (error-message sensitivity leakage). Independent reviewer, not authors of tasks 01-09. | #460 | sonnet | — | FED-M1-09 | 8K | Use `feature-dev:code-reviewer` agent. Specifically: no secrets in error messages; no partial-migration footguns. |
|
||
| FED-M1-11 | not-started | Docs update: `docs/federation/` operator notes for tier setup; README blurb on federated tier; `docs/guides/` entry for migration. Do NOT touch runbook yet (deferred to FED-M7). | #460 | haiku | feat/federation-m1-docs | FED-M1-10 | 4K | Short, actionable. Link from MISSION-MANIFEST. No decisions captured here — those belong in PRD. |
|
||
| FED-M1-12 | not-started | PR, CI green, merge to main, close #460. | #460 | — | (aggregate) | FED-M1-11 | 3K | Queue-guard before push; wait for green; merge squashed; tea `issue-close` #460. |
|
||
|
||
**M1 total estimate:** ~74K tokens (over-budget vs 20K PRD estimate — explanation below)
|
||
|
||
**Why over-budget:** PRD's 20K estimate reflected implementation complexity only. The per-task breakdown includes tests, review, and docs as separate tasks per the delivery cycle, which catches the real cost. The final per-milestone budgets in MISSION-MANIFEST will be updated after M1 completes with actuals.
|
||
|
||
---
|
||
|
||
## Milestone 2 — Step-CA + grant schema + admin CLI (FED-M2)
|
||
|
||
_Deferred to mission planning when M1 is complete. Issue #461 tracks scope._
|
||
|
||
## Milestone 3 — mTLS handshake + list/get + scope enforcement (FED-M3)
|
||
|
||
_Deferred. Issue #462._
|
||
|
||
## Milestone 4 — search + audit + rate limit (FED-M4)
|
||
|
||
_Deferred. Issue #463._
|
||
|
||
## Milestone 5 — cache + offline + OTEL (FED-M5)
|
||
|
||
_Deferred. Issue #464._
|
||
|
||
## Milestone 6 — revocation + auto-renewal + CRL (FED-M6)
|
||
|
||
_Deferred. Issue #465._
|
||
|
||
## Milestone 7 — multi-user hardening + acceptance suite (FED-M7)
|
||
|
||
_Deferred. Issue #466._
|
||
|
||
---
|
||
|
||
## Execution Notes
|
||
|
||
**Agent assignment rationale:**
|
||
|
||
- `codex` for most implementation tasks (OpenAI credit pool preferred for feature code)
|
||
- `sonnet` for tests (pattern-based, moderate complexity), `doctor` work (cross-cutting), and independent code review
|
||
- `haiku` for docs and the standalone regression canary (cheapest tier for mechanical/verification work)
|
||
- No `opus` in M1 — save for cross-cutting architecture decisions if they surface later
|
||
|
||
**Branch strategy:** Each task gets its own feature branch off `main`. Tasks within a milestone merge in dependency order. Final aggregate PR (FED-M1-12) isn't a branch of its own — it's the merge of the last upstream task that closes the issue.
|
||
|
||
**Queue guard:** Every push and every merge in this mission must run `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge` per Mosaic hard gate #6.
|