This commit is contained in:
Jason Woltje
@@ -1,108 +0,0 @@
|
||||
# Mission Manifest — Federation v1
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** federation-v1-20260419
|
||||
**Statement:** Jarvis operates across 3–4 workstations in two physical locations (home, USC). The user currently reaches back to a single jarvis-brain checkout from every session; a prior OpenBrain attempt caused cache, latency, and opacity pain. This mission builds asymmetric federation between Mosaic Stack gateways so that a session on a user's home gateway can query their work gateway in real time without data ever persisting across the boundary, with full multi-tenant isolation and standard-PKI (X.509 / Step-CA) trust management.
|
||||
**Phase:** M3 active — mTLS handshake + list/get/capabilities verbs + scope enforcement
|
||||
**Current Milestone:** FED-M3
|
||||
**Progress:** 2 / 7 milestones
|
||||
**Status:** active
|
||||
**Last Updated:** 2026-04-21 (M2 closed via PR #503, tag `fed-v0.2.0-m2`, issue #461 closed; M3 decomposed into 14 tasks)
|
||||
**Parent Mission:** None — new mission
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Host | Role | Image | Tier |
|
||||
| ----------------------- | ----------------------------------- | ------------------------------------- | --------- |
|
||||
| `mos-test-1.woltje.com` | Federation Server A (querying side) | `gateway:fed-v0.1.0-m1` (M1 baseline) | federated |
|
||||
| `mos-test-2.woltje.com` | Federation Server B (serving side) | `gateway:fed-v0.1.0-m1` (M1 baseline) | federated |
|
||||
|
||||
These are TEST hosts for federation E2E (M3+). Distinct from PRD AC-12 production targets (`woltje.com` ↔ `uscllc.com`). Deployment workstream tracked in `docs/federation/TASKS.md` under FED-M2-DEPLOY-\*.
|
||||
|
||||
## Context
|
||||
|
||||
Federation is the solution to what originally drove OpenBrain. The prior attempt coupled every agent session to a remote service, introduced cache/latency/opacity pain, and created a hard dependency that punished offline use. This redesign:
|
||||
|
||||
1. Makes federation **gateway-to-gateway**, not agent-to-service
|
||||
2. Keeps each user's home instance as source of truth for their data
|
||||
3. Exposes scoped, read-only data on demand without persisting across the boundary
|
||||
4. Uses X.509 mTLS via Step-CA so rotation/revocation/CRL/OCSP are standard
|
||||
5. Supports multi-tenant serving sides (employees on uscllc.com each federating back to their own home gateway) with no cross-user leakage
|
||||
6. Requires federation-tier instances on both sides (PG + pgvector + Valkey) — local/standalone tiers cannot federate
|
||||
7. Works over public HTTPS (no VPN required); Tailscale is an optional overlay
|
||||
|
||||
Key design references:
|
||||
|
||||
- `docs/federation/PRD.md` — 16-section product requirements
|
||||
- `docs/federation/MILESTONES.md` — 7-milestone decomposition with per-milestone acceptance tests
|
||||
- `docs/federation/TASKS.md` — per-task breakdown (M1 populated; M2-M7 deferred to mission planning)
|
||||
- `docs/research/mempalace-evaluation/` (in jarvis-brain) — why we didn't adopt MemPalace
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] AC-1: Two Mosaic Stack gateways on different hosts can establish a federation grant via CLI-driven onboarding
|
||||
- [ ] AC-2: Server A can query Server B for `tasks`, `notes`, `memory` respecting scope filters
|
||||
- [ ] AC-3: User on B with no grant cannot be queried by A, even if A has a valid grant for another user (cross-user isolation)
|
||||
- [ ] AC-4: Revoking a grant on B causes A's next request to fail with a clear error within one request cycle
|
||||
- [ ] AC-5: Cert rotation happens automatically at T-7 days; in-progress session survives rotation without user action
|
||||
- [ ] AC-6: Rate-limit enforcement returns 429 with `Retry-After`; client backs off
|
||||
- [ ] AC-7: With B unreachable, a session on A completes using local data and surfaces "federation offline for `<peer>`" once per session
|
||||
- [ ] AC-8: Every federated request appears in B's `federation_audit_log` within 1 second
|
||||
- [ ] AC-9: Scope excluding `credentials` means credentials are never returned — even via `search` with matching keywords
|
||||
- [ ] AC-10: `mosaic federation status` shows cert expiry, grant status, last success/failure per peer
|
||||
- [ ] AC-11: Full 3-employee multi-tenant scenario passes with no cross-user leakage
|
||||
- [ ] AC-12: Two-gateway production deployment (woltje.com ↔ uscllc.com) operational ≥7 days without incident
|
||||
- [ ] AC-13: All 7 milestones ship as merged PRs with green CI and closed issues
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------ | --------------------------------------------- | ----------- | ------------------ | ----- | ---------- | ---------- |
|
||||
| 1 | FED-M1 | Federated tier infrastructure | done | (12 PRs #470-#481) | #460 | 2026-04-19 | 2026-04-19 |
|
||||
| 2 | FED-M2 | Step-CA + grant schema + admin CLI | done | (PRs #483-#503) | #461 | 2026-04-21 | 2026-04-21 |
|
||||
| 3 | FED-M3 | mTLS handshake + list/get + scope enforcement | in-progress | (decomposition) | #462 | 2026-04-21 | — |
|
||||
| 4 | FED-M4 | search verb + audit log + rate limit | not-started | — | #463 | — | — |
|
||||
| 5 | FED-M5 | Cache + offline degradation + OTEL | not-started | — | #464 | — | — |
|
||||
| 6 | FED-M6 | Revocation + auto-renewal + CRL | not-started | — | #465 | — | — |
|
||||
| 7 | FED-M7 | Multi-user RBAC hardening + acceptance suite | not-started | — | #466 | — | — |
|
||||
|
||||
## Budget
|
||||
|
||||
| Milestone | Est. tokens | Parallelizable? |
|
||||
| --------- | ----------- | ---------------------- |
|
||||
| FED-M1 | 20K | No (foundation) |
|
||||
| FED-M2 | 30K | No (needs M1) |
|
||||
| FED-M3 | 40K | No (needs M2) |
|
||||
| FED-M4 | 20K | No (needs M3) |
|
||||
| FED-M5 | 20K | Yes (with M6 after M4) |
|
||||
| FED-M6 | 20K | Yes (with M5 after M3) |
|
||||
| FED-M7 | 25K | No (needs all) |
|
||||
| **Total** | **~175K** | |
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Date | Runtime | Outcome |
|
||||
| ------- | ----------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| S1 | 2026-04-19 | claude | PRD authored, MILESTONES decomposed, 7 issues filed |
|
||||
| S2-S4 | 2026-04-19 | claude | FED-M1 complete: 12 tasks (PRs #470-#481) merged; tag `fed-v0.1.0-m1` |
|
||||
| S5-S22 | 2026-04-19 → 2026-04-21 | claude | FED-M2 complete: 13 tasks (PRs #483-#503) merged; tag `fed-v0.2.0-m2`; issue #461 closed. Step-CA + grant schema + admin CLI shipped. |
|
||||
| S23 | 2026-04-21 | claude | M3 decomposed into 14 tasks in `docs/federation/TASKS.md`. Manifest M3 row → in-progress. Next: kickoff M3-01. |
|
||||
|
||||
## Next Step
|
||||
|
||||
FED-M3 active. Decomposition landed in `docs/federation/TASKS.md` (M3-01..M3-14, ~100K estimate). Tracking issue #462.
|
||||
|
||||
Execution plan (parallel where possible):
|
||||
|
||||
- **Foundation**: M3-01 (DTOs in `packages/types/src/federation/`) starts immediately — sonnet subagent on `feat/federation-m3-types`. Blocks all server + client work.
|
||||
- **Server stream** (after M3-01): M3-03 (AuthGuard) + M3-04 (ScopeService) in series, then M3-05 / M3-06 / M3-07 (verbs) in parallel.
|
||||
- **Client stream** (after M3-01, parallel with server): M3-08 (FederationClient) → M3-09 (QuerySourceService).
|
||||
- **Harness** (parallel with everything): M3-02 (`tools/federation-harness/`) — needed for M3-11.
|
||||
- **Test gates**: M3-10 (Integration) → M3-11 (E2E with harness) → M3-12 (Independent security review, two rounds budgeted).
|
||||
- **Close**: M3-13 (Docs) → M3-14 (release tag `fed-v0.3.0-m3`, close #462).
|
||||
|
||||
**Test-bed fallback:** `mos-test-1/-2` deploy is still blocked on `FED-M2-DEPLOY-IMG-FIX`. The harness in M3-02 ships a local two-gateway docker-compose so M3-11 is not blocked. Production-host validation is M7's responsibility (PRD AC-12).
|
||||
@@ -1,154 +0,0 @@
|
||||
# Tasks — Federation v1
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** federation-v1-20260419
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `glm-5.1` | `haiku` | `sonnet` | `opus` | `—` (auto)
|
||||
>
|
||||
> **Scope of this file:** M1 is fully decomposed below. M2–M7 are placeholders pending each milestone's entry into active planning — the orchestrator expands them one milestone at a time to avoid speculative decomposition of work whose shape will depend on what M1 surfaces.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1 — Federated tier infrastructure (FED-M1)
|
||||
|
||||
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
||||
| FED-M1-02 | done | Historical shipped-status record: authored a federated Compose overlay with PostgreSQL/pgvector, Valkey, volumes, and healthchecks. It is not a current startup, init, extension, schema, or migration procedure. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471 status only. KBN-101-02 retires its init authority; KBN-101-05 replaces deployment rendering; KBN-101-07 SETUP is non-operative until activation. |
|
||||
| FED-M1-03 | done | Historical shipped-status record: add pgvector support to `packages/storage/src/adapters/postgres.ts`; no adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472 status only. **KBN-101 supersedes this behavior:** it cannot authorize current runtime extension creation or any DDL; only the runner/external bootstrap contract may do so. |
|
||||
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
||||
| FED-M1-05 | done | Historical shipped-status record: prior tier migration implementation. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474 status only. **KBN-101 supersedes this route:** it cannot authorize current credentials, target connection, or DDL. The future active route requires runner verification plus target URL-file and signed attestation-file binding. |
|
||||
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
||||
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
||||
| FED-M1-08 | done | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Shipped in PR #477. Caught P0 in M1-05 (camelCase→snake_case) missed by mocked unit tests; fix in same PR. |
|
||||
| FED-M1-09 | done | Standalone regression: full agent-session E2E on existing `standalone` tier with a gateway built from this branch. Must pass without referencing any federation module. | #460 | sonnet | feat/federation-m1-regression | FED-M1-07 | 4K | Clean canary. 351 gateway tests + 85 storage unit tests + full pnpm test all green; only FEDERATED_INTEGRATION-gated tests skip. |
|
||||
| FED-M1-10 | done | Code review pass: security-focused on the migration script (data-at-rest during migration) + tier detector (error-message sensitivity leakage). Independent reviewer, not authors of tasks 01-09. | #460 | sonnet | feat/federation-m1-security-review | FED-M1-09 | 8K | 2 review rounds caught 7 issues: credential leak in pg/valkey/pgvector errors + redact-error util; missing advisory lock; SKIP_TABLES rationale. |
|
||||
| FED-M1-11 | done | Docs update: `docs/federation/` operator notes for tier setup; README blurb on federated tier; `docs/guides/` entry for migration. Do NOT touch runbook yet (deferred to FED-M7). | #460 | haiku | feat/federation-m1-docs | FED-M1-10 | 4K | Shipped: `docs/federation/SETUP.md` (119 lines), `docs/guides/migrate-tier.md` (147 lines), README Configuration blurb. |
|
||||
| FED-M1-12 | done | PR, CI green, merge to main, close #460. | #460 | sonnet | feat/federation-m1-close | FED-M1-11 | 3K | M1 closed. PRs #470-#480 merged across 11 tasks. Issue #460 closed; release tag `fed-v0.1.0-m1` published. |
|
||||
|
||||
**M1 total estimate:** ~74K tokens (over-budget vs 20K PRD estimate — explanation below)
|
||||
|
||||
**Why over-budget:** PRD's 20K estimate reflected implementation complexity only. The per-task breakdown includes tests, review, and docs as separate tasks per the delivery cycle, which catches the real cost. The final per-milestone budgets in MISSION-MANIFEST will be updated after M1 completes with actuals.
|
||||
|
||||
---
|
||||
|
||||
## Pre-M2 — Test deployment infrastructure (FED-M2-DEPLOY)
|
||||
|
||||
Goal: Two federated-tier gateways stood up on Portainer at `mos-test-1.woltje.com` and `mos-test-2.woltje.com` running the M1 release (`gateway:fed-v0.1.0-m1`). This is the test bed for M2 enrollment work and the M3 federation E2E harness. No federation logic exercised yet — pure infrastructure validation.
|
||||
|
||||
> **Why now:** M2 enrollment requires a real second gateway to test peer-add flows; standing the test hosts up before M2 code lands gives both code and deployment streams a fast feedback loop.
|
||||
|
||||
> **Parallelizable:** This workstream runs in parallel with the M2 code workstream (M2-01 → M2-13). They re-converge at M2-10 (E2E test).
|
||||
|
||||
> **Tracking issue:** #482.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ------------------------------------- | ------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M2-DEPLOY-01 | done | Verify `gateway:fed-v0.1.0-m1` image was published by `.woodpecker/publish.yml` on tag push; if not, investigate and remediate. Document image URI in deployment artifact. | #482 | sonnet | (verified inline, no PR) | — | 2K | Tag exists; digest `sha256:9b72e202a9eecc27d31920b87b475b9e96e483c0323acc57856be4b1355db1ec` captured for digest-pinned deploys. |
|
||||
| FED-M2-DEPLOY-02 | done | Author Portainer git-stack compose file `deploy/portainer/federated-test.stack.yml` (gateway + PG-pgvector + Valkey, env-driven). Use immutable tag, not `latest`. | #482 | sonnet | feat/federation-deploy-stack-template | DEPLOY-01 | 5K | Shipped in PR #485. Digest-pinned. Env: STACK_NAME, HOST_FQDN, POSTGRES_PASSWORD, BETTER_AUTH_SECRET, BETTER_AUTH_URL. |
|
||||
| FED-M2-DEPLOY-IMG-FIX | in-progress | Gateway image runtime broken (ERR_MODULE_NOT_FOUND for `dotenv`); Dockerfile copies `.pnpm/` store but not `apps/gateway/node_modules` symlinks. Switch to `pnpm deploy` for self-contained runtime. | #482 | sonnet | (subagent in flight) | DEPLOY-02 | 4K | Subagent `a78a9ab0ddae91fbc` in flight. Triggers Kaniko rebuild on merge; capture new digest; bump stack template in follow-up PR before redeploy. |
|
||||
| FED-M2-DEPLOY-03 | blocked | Deploy stack to mos-test-1.woltje.com via `~/.config/mosaic/tools/portainer/`. Verify M1 acceptance: federated-tier boot succeeds; `mosaic gateway doctor --json` returns green; pgvector `vector(3)` round-trip works. | #482 | sonnet | feat/federation-deploy-test-1 | IMG-FIX | 3K | Stack created on Portainer endpoint 3 (Swarm `local`), but blocked on image fix. Container fails on boot until IMG-FIX merges + redeploy. |
|
||||
| FED-M2-DEPLOY-04 | blocked | Deploy stack to mos-test-2.woltje.com via Portainer wrapper. Same M1 acceptance probes as DEPLOY-03. | #482 | sonnet | feat/federation-deploy-test-2 | IMG-FIX | 3K | Same status as DEPLOY-03. Stack created; blocked on image fix. |
|
||||
| FED-M2-DEPLOY-05 | not-started | Document deployment in `docs/federation/TEST-INFRA.md`: hosts, image tags, secrets sourcing, redeploy procedure, teardown. Update MISSION-MANIFEST with deployment status. | #482 | haiku | feat/federation-deploy-docs | DEPLOY-03,04 | 3K | Operator-facing doc; mentions but does not duplicate `tools/portainer/README.md`. |
|
||||
|
||||
**Deploy workstream estimate:** ~16K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2 — Step-CA + grant schema + admin CLI (FED-M2)
|
||||
|
||||
Goal: An admin can create a federation grant; counterparty enrolls; cert is signed by Step-CA with SAN OIDs for `grantId` + `subjectUserId`. No runtime federation traffic flows yet (that's M3).
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| FED-M2-01 | done | DB migration: `federation_grants`, `federation_peers`, `federation_audit_log` tables + enum types (`grant_status`, `peer_state`). Drizzle schema + migration generation; migration tests. | #461 | sonnet | feat/federation-m2-schema | — | 5K | Shipped in PR #486. DESC indexes + reserved cols added after first review; migration tests green. |
|
||||
| FED-M2-02 | done | Add Step-CA sidecar to `docker-compose.federated.yml`: official `smallstep/step-ca` image, persistent CA volume, JWK provisioner config baked into init script. | #461 | sonnet | feat/federation-m2-stepca | DEPLOY-02 | 4K | Shipped in PR #494. Profile-gated under `federated`; CA password from secret; dev compose uses dev-only password file. |
|
||||
| FED-M2-03 | done | Scope JSON schema + validator: `resources` allowlist, `excluded_resources`, `include_teams`, `include_personal`, `max_rows_per_query`. Vitest unit tests for valid + invalid scopes. | #461 | sonnet | feat/federation-m2-scope-schema | — | 4K | Shipped in PR #496 (bundled with grants service). Validator independent of CA; reusable from grant CRUD + M3 scope enforcement. |
|
||||
| FED-M2-04 | done | `apps/gateway/src/federation/ca.service.ts`: Step-CA client (CSR submission, OID-bearing cert retrieval). Mocked + integration tests against real Step-CA container. | #461 | sonnet | feat/federation-m2-ca-service | M2-02 | 6K | Shipped in PR #494. SAN OIDs 1.3.6.1.4.1.99999.1 (grantId) + 1.3.6.1.4.1.99999.2 (subjectUserId); integration test asserts both OIDs present in issued cert. |
|
||||
| FED-M2-05 | done | Sealed storage for `client_key_pem` reusing existing `provider_credentials` sealing key. Tests prove DB-at-rest is ciphertext, not PEM. Key rotation path documented (deferred impl). | #461 | sonnet | feat/federation-m2-key-sealing | M2-01 | 5K | Shipped in PR #495. Crypto seam isolated; tests confirm ciphertext-at-rest; key rotation deferred to M6. |
|
||||
| FED-M2-06 | done | `grants.service.ts`: CRUD + status transitions (`pending` → `active` → `revoked`); integrates M2-03 (scope) + M2-05 (sealing). Unit tests cover all transitions including invalid ones. | #461 | sonnet | feat/federation-m2-grants-service | M2-03, M2-05 | 6K | Shipped in PR #496. All status transitions covered; invalid transition tests green; revocation handler deferred to M6. |
|
||||
| FED-M2-07 | done | `enrollment.controller.ts`: short-lived single-use token endpoint; CSR signing; updates grant `pending` → `active`; emits enrollment audit (table-only write, M4 tightens). | #461 | sonnet | feat/federation-m2-enrollment | M2-04, M2-06 | 6K | Shipped in PR #497. Tokens single-use with 410 on replay; TTL 15min; rate-limited at request layer. |
|
||||
| FED-M2-08 | done | Admin CLI: `mosaic federation grant create/list/show` + `peer add/list`. Integration with grants.service (no API duplication). Help output + machine-readable JSON option. | #461 | sonnet | feat/federation-m2-cli | M2-06, M2-07 | 7K | Shipped in PR #498. `peer add <enrollment-url>` client-side flow; JSON output flag; admin REST controller co-shipped. |
|
||||
| FED-M2-09 | done | Integration tests covering MILESTONES.md M2 acceptance tests #1, #2, #3, #5, #7, #8 (single-gateway suite). Real Step-CA container; vitest profile gated by `FEDERATED_INTEGRATION=1`. | #461 | sonnet | feat/federation-m2-integration | M2-08 | 8K | Shipped in PR #499. All 6 acceptance tests green; gated by FEDERATED_INTEGRATION=1. |
|
||||
| FED-M2-10 | done | E2E test against deployed mos-test-1 + mos-test-2 (or local two-gateway docker-compose if Portainer not ready): MILESTONES test #6 `peer add` yields `active` peer record with valid cert + key. | #461 | sonnet | feat/federation-m2-e2e | M2-08, DEPLOY-04 | 6K | Shipped in PR #500. Local two-gateway docker-compose path used; `peer add` yields active peer with valid cert + sealed key. |
|
||||
| FED-M2-11 | done | Independent security review (sonnet, not author of M2-04/05/06/07): focus on single-use token replay, sealing leak surfaces, OID match enforcement, scope schema bypass paths. | #461 | sonnet | feat/federation-m2-security-review | M2-10 | 8K | Shipped in PR #501. Two-round review; enrollment-token replay, OID-spoofing CSR, and key leak in error messages all verified and hardened. |
|
||||
| FED-M2-12 | done | Docs update: `docs/federation/SETUP.md` Step-CA section; new `docs/federation/ADMIN-CLI.md` with grant/peer commands; scope schema reference; OID registration note. Runbook still M7-deferred. | #461 | haiku | feat/federation-m2-docs | M2-11 | 4K | Shipped in PR #502. SETUP.md CA bootstrap section added; ADMIN-CLI.md created; scope schema reference and OID note included. |
|
||||
| FED-M2-13 | done | PR aggregate close, CI green, merge to main, close #461. Release tag `fed-v0.2.0-m2`. Mark deploy stream complete. Update mission manifest M2 row. | #461 | sonnet | chore/federation-m2-close | M2-12 | 3K | Release tag `fed-v0.2.0-m2` created; issue #461 closed; all M2 PRs #494–#502 merged to main. |
|
||||
|
||||
**M2 code workstream estimate:** ~72K tokens (vs MILESTONES.md 30K — same over-budget pattern as M1, where per-task breakdown including tests/review/docs catches the real cost).
|
||||
|
||||
**Deploy + code combined:** ~88K tokens.
|
||||
|
||||
## Milestone 3 — mTLS handshake + list/get + scope enforcement (FED-M3)
|
||||
|
||||
Goal: Two federated gateways exchange real data over mTLS. Inbound requests pass through cert validation → grant lookup → scope enforcement → native RBAC → response. `list`, `get`, and `capabilities` verbs land. The federation E2E harness (`tools/federation-harness/`) is the new permanent test bed for M3+ and is gated on every milestone going forward.
|
||||
|
||||
> **Critical trust boundary.** Every 401/403 path needs a test. Code review is non-negotiable; M3-12 budgets two review rounds.
|
||||
>
|
||||
> **Tracking issue:** #462.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ------------------------------------ | --------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M3-01 | done | `packages/types/src/federation/` — request/response DTOs for `list`, `get`, `capabilities` verbs. Wire-format zod schemas + inferred TS types. Includes `FederationRequest`, `FederationListResponse<T>`, `FederationGetResponse<T>`, `FederationCapabilitiesResponse`, error envelope, `_source` tag. | #462 | sonnet | feat/federation-m3-types | — | 4K | Reusable from gateway server + client + harness. Pure types — no I/O, no NestJS. |
|
||||
| FED-M3-02 | done | `tools/federation-harness/` scaffold: `docker-compose.two-gateways.yml` (Server A + Server B + step-CA), `seed.ts` (provisions grants, peers, sample tasks/notes/credentials per scope variant), `harness.ts` helper (boots stack, returns typed clients). README documents harness use. | #462 | sonnet | feat/federation-m3-harness | DEPLOY-04 (soft) | 8K | Falls back to local docker-compose if `mos-test-1/-2` not yet redeployed (DEPLOY chain blocked on IMG-FIX). Permanent test infra used by M3+. |
|
||||
| FED-M3-03 | done | `apps/gateway/src/federation/server/federation-auth.guard.ts` (NestJS guard). Validates inbound client cert from Fastify TLS context, extracts `grantId` + `subjectUserId` from custom OIDs, loads grant from DB, asserts `status='active'`, attaches `FederationContext` to request. | #462 | sonnet | feat/federation-m3-auth-guard | M3-01 | 8K | Reuses OID parsing logic mirrored from `ca.service.ts` post-issuance verification. 401 on malformed/missing OIDs; 403 on revoked/expired/missing grant. |
|
||||
| FED-M3-04 | in-progress | `apps/gateway/src/federation/server/scope.service.ts`. Pipeline: (1) resource allowlist + excluded check, (2) native RBAC eval as `subjectUserId`, (3) scope filter intersection (`include_teams`, `include_personal`), (4) `max_rows_per_query` cap. Pure service — DB calls injected. | #462 | sonnet | feat/federation-m3-scope-service | M3-01 | 10K | Hardest correctness target in M3. Reuses `parseFederationScope` (M2-03). Returns either `{ allowed: true, filter }` or structured deny reason for audit. |
|
||||
| FED-M3-05 | in-progress | `apps/gateway/src/federation/server/verbs/list.controller.ts`. Wires AuthGuard → ScopeService → tasks/notes/memory query layer; applies row cap; tags rows with `_source`. Resource selector via path param. | #462 | sonnet | feat/federation-m3-verb-list | M3-03, M3-04 | 6K | Routes: `POST /api/federation/v1/list/:resource`. No body persistence. Audit write deferred to M4. |
|
||||
| FED-M3-06 | not-started | `apps/gateway/src/federation/server/verbs/get.controller.ts`. Single-resource fetch by id; same pipeline as list. 404 on not-found, 403 on RBAC/scope deny — both audited the same way. | #462 | sonnet | feat/federation-m3-verb-get | M3-03, M3-04 | 6K | `POST /api/federation/v1/get/:resource/:id`. Mirrors list controller patterns. |
|
||||
| FED-M3-07 | done | `apps/gateway/src/federation/server/verbs/capabilities.controller.ts`. Read-only enumeration: returns `{ resources, excluded_resources, max_rows_per_query, supported_verbs }` derived from grant scope. Always allowed for an active grant — no RBAC eval. | #462 | sonnet | feat/federation-m3-verb-capabilities | M3-03 | 4K | `GET /api/federation/v1/capabilities`. Smallest verb; useful sanity check that mTLS + auth guard work end-to-end. |
|
||||
| FED-M3-08 | done | `apps/gateway/src/federation/client/federation-client.service.ts`. Outbound mTLS dialer: picks `(certPem, sealed clientKey)` from `federation_peers`, unwraps key, builds undici Agent with mTLS, calls peer verb, parses typed response, wraps non-2xx into `FederationClientError`. | #462 | sonnet | feat/federation-m3-client | M3-01 | 8K | Independent of server stream — can land in parallel with M3-03/04. Cert/key cached per-peer; flushed by future M5/M6 logic. |
|
||||
| FED-M3-09 | done | `apps/gateway/src/federation/client/query-source.service.ts`. Accepts `source: "local" \| "federated:<host>" \| "all"` from gateway query layer; for `"all"` fans out to local + each peer in parallel; merges results; tags every row with `_source`. | #462 | sonnet | feat/federation-m3-query-source | M3-08 | 8K | Per-peer failure surfaces as `_partial: true` in response, not hard failure (sets up M5 offline UX). M5 adds caching + circuit breaker on top. |
|
||||
| FED-M3-10 | not-started | Integration tests for MILESTONES.md M3 acceptance #6 (malformed OIDs → 401; valid cert + revoked grant → 403) and #7 (`max_rows_per_query` cap). Real PG, mocked TLS context (Fastify req shim). | #462 | sonnet | feat/federation-m3-integration | M3-05, M3-06 | 8K | Vitest profile gated by `FEDERATED_INTEGRATION=1`. Single-gateway suite; no harness required. |
|
||||
| FED-M3-11 | not-started | E2E tests for MILESTONES.md M3 acceptance #1, #2, #3, #4, #5, #8, #9, #10 (8 cases). Uses harness from M3-02; two real gateways, real Step-CA, real mTLS. Each test asserts both happy-path response and audit/no-persist invariants. | #462 | sonnet | feat/federation-m3-e2e | M3-02, M3-04, M3-05, M3-06, M3-09 | 12K | Largest single task. Each acceptance gets its own `it(...)` for clear failure attribution. |
|
||||
| FED-M3-12 | not-started | Independent security review (sonnet, not author of M3-03/04/05/06/07/08/09): focus on cert-SAN spoofing, OID extraction edge cases, scope-bypass via filter manipulation, RBAC-bypass via subjectUser swap, response leakage when scope deny. | #462 | sonnet | feat/federation-m3-security-review | M3-11 | 10K | Two review rounds budgeted. PRD requires explicit test for every 401/403 path — review verifies coverage. |
|
||||
| FED-M3-13 | not-started | Docs update: `docs/federation/SETUP.md` mTLS handshake section, new `docs/federation/HARNESS.md` for federation-harness usage, OID reference table in SETUP.md, scope enforcement pipeline diagram. Runbook still M7-deferred. | #462 | haiku | feat/federation-m3-docs | M3-12 | 5K | One ASCII diagram for the auth-guard → scope → RBAC pipeline; helps future reviewers reason about denial paths. |
|
||||
| FED-M3-14 | not-started | PR aggregate close, CI green, merge to main, close #462. Release tag `fed-v0.3.0-m3`. Update mission manifest M3 row → done; M4 row → in-progress when work begins. | #462 | sonnet | chore/federation-m3-close | M3-13 | 3K | Same close pattern as M1-12 / M2-13. |
|
||||
|
||||
**M3 estimate:** ~100K tokens (vs MILESTONES.md 40K — same per-task breakdown pattern as M1/M2: tests, review, and docs split out from implementation cost). Largest milestone in the federation mission.
|
||||
|
||||
**Parallelization opportunities:**
|
||||
|
||||
- M3-08 (client) can land in parallel with M3-03/M3-04 (server pipeline) — they only share DTOs from M3-01.
|
||||
- M3-02 (harness) can land in parallel with everything except M3-11.
|
||||
- M3-05/M3-06/M3-07 (verbs) are independent of each other once M3-03/M3-04 land.
|
||||
|
||||
**Test bed fallback:** If `mos-test-1.woltje.com` / `mos-test-2.woltje.com` are still blocked on `FED-M2-DEPLOY-IMG-FIX` when M3-11 is ready to run, the harness's local `docker-compose.two-gateways.yml` is a sufficient stand-in. Production-host validation moves to M7 acceptance suite (PRD AC-12).
|
||||
|
||||
**Backlog sync — 2026-06-24 (orchestrator):** Status reconciled against `origin/main` (release 0.0.48). Landed on main: **FED-M3-01** (DTOs, PR #506), **FED-M3-02** (harness scaffold, PR #505), **FED-M3-03** (mTLS auth-guard, PR #509 — CRIT-1/2 + HIGH-1..4 remediated in-PR), **FED-M3-08** (outbound mTLS client, PR #508). With M3-01/03/08 merged, three cards became dependency-clear and were dispatched to the idle coder lane: **FED-M3-04** scope.service → coder0 (`feat/federation-m3-scope-service`); **FED-M3-09** query-source + **FED-M3-07** capabilities verb → coder1 (`feat/federation-m3-query-source` first). Reviewer warmed for the M3 trust-boundary PRs. Remaining blocked-by-DAG: M3-05/06 (await M3-04), M3-10 (await M3-05/06), M3-11 (await M3-09), M3-12→14 (tail). Deploy chain (DEPLOY-IMG-FIX → 03/04) still independent of M3 code — harness local docker-compose fallback covers M3-11.
|
||||
|
||||
**Backlog sync #2 — 2026-06-24 (orchestrator):** **FED-M3-09** (query-source) merged via PR #673 and **FED-M3-07** (capabilities) merged via PR #674 — both squash-merged on independent agent review-of-record + green CI (formal Gitea approve unavailable under the shared service account; merge is not gated by the self-approve guard). **FED-M3-05** (list verb) dispatched to coder1 (based on the M3-04 branch, rebase onto main once #672 lands). **FED-M3-04** (scope.service, PR #672) is in review-changes (one include_personal no-leak test outstanding). **DAG fix:** corrected `FED-M3-11` depends_on from `M3-02, M3-09` → `M3-02, M3-04, M3-05, M3-06, M3-09` — the E2E acceptance cases (#1–#5, #8–#10) exercise list/get over mTLS, so the server verbs + scope service are hard prerequisites; the original edge set omitted them and caused a premature M3-11 dispatch. Note: M3 read-path invariant for M3-11 is **no-persist + existing enrollment audit only** — read-verb audit-log writes are deferred to M4 (see M3-05/06 notes), so M3-11 must not assert read-audit-log entries.
|
||||
|
||||
## Milestone 4 — search + audit + rate limit (FED-M4)
|
||||
|
||||
_Deferred. Issue #463._
|
||||
|
||||
## Milestone 5 — cache + offline + OTEL (FED-M5)
|
||||
|
||||
_Deferred. Issue #464._
|
||||
|
||||
## Milestone 6 — revocation + auto-renewal + CRL (FED-M6)
|
||||
|
||||
_Deferred. Issue #465._
|
||||
|
||||
## Milestone 7 — multi-user hardening + acceptance suite (FED-M7)
|
||||
|
||||
_Deferred. Issue #466._
|
||||
|
||||
---
|
||||
|
||||
## Execution Notes
|
||||
|
||||
**Agent assignment rationale:**
|
||||
|
||||
- `codex` for most implementation tasks (OpenAI credit pool preferred for feature code)
|
||||
- `sonnet` for tests (pattern-based, moderate complexity), `doctor` work (cross-cutting), and independent code review
|
||||
- `haiku` for docs and the standalone regression canary (cheapest tier for mechanical/verification work)
|
||||
- No `opus` in M1 — save for cross-cutting architecture decisions if they surface later
|
||||
|
||||
**Branch strategy:** Each task gets its own feature branch off `main`. Tasks within a milestone merge in dependency order. Final aggregate PR (FED-M1-12) isn't a branch of its own — it's the merge of the last upstream task that closes the issue.
|
||||
|
||||
**Queue guard:** Every push and every merge in this mission must run `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge` per Mosaic hard gate #6.
|
||||
@@ -1,655 +0,0 @@
|
||||
# Mission Scratchpad — MVP
|
||||
|
||||
> Append-only log. NEVER delete entries. NEVER overwrite sections.
|
||||
> This is the orchestrator's working memory across sessions.
|
||||
|
||||
## Original Mission Prompt
|
||||
|
||||
```
|
||||
Active mission detected: MVP. Read the mission state files and report status.
|
||||
User confirmed: start the planning gate.
|
||||
```
|
||||
|
||||
## Planning Decisions
|
||||
|
||||
### 2026-03-13 — Milestone and task breakdown
|
||||
|
||||
- PRD defines 8 phases (Phase 0–7), mapped 1:1 to Gitea milestones
|
||||
- 59 issues created on git.mosaicstack.dev/mosaic/mosaic-stack (#1–#59)
|
||||
- Each phase has a verification task as the final issue
|
||||
- Task IDs use P{phase}-{seq} format (P0-001 through P7-008)
|
||||
- Repo created as `mosaic/mosaic-stack` (private) on Gitea
|
||||
- Milestones: ms-157 (Phase 0) through ms-164 (Phase 7)
|
||||
- Total: 59 tasks across 8 milestones
|
||||
|
||||
### Phase structure
|
||||
|
||||
| Phase | Version | Tasks | Focus |
|
||||
| ----- | ------- | ----- | ---------------------------------------------------------- |
|
||||
| 0 | v0.0.1 | 9 | Foundation — monorepo, types, db, auth, OTEL, Docker, CI |
|
||||
| 1 | v0.0.2 | 9 | Core API — gateway, brain, queue, routes, WebSocket |
|
||||
| 2 | v0.0.3 | 7 | Agent Layer — Pi SDK, multi-provider, routing, coord |
|
||||
| 3 | v0.0.4 | 8 | Web Dashboard — Next.js, chat, tasks, projects, admin |
|
||||
| 4 | v0.0.5 | 7 | Memory & Intelligence — memory, log, summarization, skills |
|
||||
| 5 | v0.0.6 | 5 | Remote Control — Discord, Telegram, SSO |
|
||||
| 6 | v0.0.7 | 6 | CLI & Tools — CLI, prdy, quality-rails, installer, TUI |
|
||||
| 7 | v0.1.0 | 8 | Polish & Beta — MCP, providers, E2E, docs, release |
|
||||
|
||||
## Session Log
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | -------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| 1 | 2026-03-13 | Planning | Planning gate | Milestones created, 59 issues created, TASKS.md populated, manifest updated |
|
||||
| 2 | 2026-03-13 | Vertical slice | P1-001, P1-007, P1-008, P2-001, P5-002, P6-005 | Communication spine built and merged (PR #61). Gateway + TUI + Discord. 3-agent gatekeeper review, 10/16 issues remediated, 4 deferred. |
|
||||
| 3 | 2026-03-13 | Foundation | P0-002, P0-005, P0-006 | Foundation layer merged (PR #65). Docker Compose (PG+pgvector, Valkey, OTEL Collector, Jaeger), OTEL auto-instrumentation in gateway, @mosaicstack/types with DTOs + Socket.IO typed event maps. |
|
||||
|
||||
### Session 4 — Docker Compose fix
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | ---------- | ---------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| 4 | 2026-03-12 | Foundation | (fix) | Fixed Jaeger tag (2→2.6.0), remapped PG/Valkey ports (5433/6380) to avoid host conflicts. PR #66 merged to main. |
|
||||
|
||||
**Verification evidence:**
|
||||
|
||||
- All 4 containers healthy (PG, Valkey, OTEL Collector, Jaeger)
|
||||
- OTEL pipeline proven: `mosaic-gateway` service visible in Jaeger UI
|
||||
- Gateway traces flow through Collector → Jaeger
|
||||
|
||||
### Session 5 — Phase 0-1 completion
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | ---------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| 5 | 2026-03-12 | Phase 0, Phase 1 | P0-003, P0-004, P0-007, P0-008, P0-009, P1-002–P1-006, P1-009 | Foundation + Core API complete. DB, auth, CI, brain, queue, CRUD routes all merged and green. |
|
||||
|
||||
### Session 6 — Phase 2 agent layer
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| 6 | 2026-03-12 | Phase 2 | P2-002, P2-003, P2-004, P2-005, P2-006, FIX-01 | Multi-provider routing, tool registration, coord migration, session management, dispose() fix. PRs #74–#78. |
|
||||
|
||||
### Session 7-8 — Phase 2 verification + completion
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------- | ----------------------------------------------------------------------------------- |
|
||||
| 7-8 | 2026-03-12 | Phase 2 | P2-007 | 19 unit tests (routing + coord). PR #79 merged, issue #25 closed. Phase 2 complete. |
|
||||
|
||||
### Session 11 — Phase 5 completion
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 11 | 2026-03-14 | Phase 5 | P5-005 | Wired Telegram plugin into gateway (was stubbed). Updated .env.example with all P5 env vars. PR #99 merged, issue #45 closed. Phase 5 complete. |
|
||||
|
||||
**Findings during verification:**
|
||||
|
||||
- Telegram plugin was built but not wired into gateway (stub warning in plugin.module.ts)
|
||||
- Discord plugin was fully wired
|
||||
- SSO/Authentik OIDC adapter was fully wired
|
||||
- All three quality gates passing
|
||||
|
||||
### Session 11 (continued) — Phase 6 completion
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| 11 | 2026-03-14 | Phase 6 | P6-002, P6-003, P6-004, P6-001, P6-006 | Full CLI & Tools migration. PRs #100-#104 merged. Also fixed 2 gateway startup bugs (PR #102). Phase 6 complete. |
|
||||
|
||||
**Phase 6 details:**
|
||||
|
||||
- P6-002: @mosaicstack/prdy migrated from v0 (~400 LOC). PR #101.
|
||||
- P6-003: @mosaicstack/quality-rails migrated from v0 (~500 LOC). PR #100.
|
||||
- P6-004: @mosaicstack/mosaic wizard migrated from v0 (2272 LOC, 28 files). PR #103.
|
||||
- P6-001: CLI subcommands wired — tui, prdy, quality-rails, wizard all working. PR #104.
|
||||
- BUG-1: PLUGIN_REGISTRY circular import fixed via plugin.tokens.ts. PR #102.
|
||||
- BUG-2: AuthStorage.create() → .inMemory() to prevent silent exit. PR #102.
|
||||
|
||||
### Session 11 (continued) — E2E testing + bug fixes + Phase 7 rescope
|
||||
|
||||
**Bug fixes merged during E2E testing (PRs #107-#117):**
|
||||
|
||||
- CI: from_secret syntax for Woodpecker v2 (#107)
|
||||
- Gateway: dotenv loading from monorepo root (#108)
|
||||
- Gateway: missing @Inject() decorators causing silent hang (#109)
|
||||
- Gateway: CORS + memory userId + pgvector auto-init (#110)
|
||||
- Auth: BetterAuth trustedOrigins for web dashboard (#111)
|
||||
- Auth: CORS headers on raw BetterAuth HTTP handler (#112)
|
||||
- Husky: removed deprecated v9 shim lines (#113)
|
||||
- CLI: login command + authenticated TUI sessions (#114)
|
||||
- CLI: Origin header on auth requests (#115)
|
||||
- Agent: Ollama provider registration with openai-completions API (#116, #117)
|
||||
|
||||
**E2E testing results:**
|
||||
|
||||
- Web UI: login works, projects list, chats list (but chat doesn't function)
|
||||
- TUI: authenticated connection works, agent responds via Ollama llama3.2
|
||||
- Agent tools: brain, coord, memory tools confirmed working
|
||||
- Gateway: all routes mapped, providers register correctly
|
||||
|
||||
**Phase 7 rescoped (Jason directed):**
|
||||
|
||||
- Phase 7 renamed from "Polish & Beta" to "Feature Completion (v0.0.8)"
|
||||
- Added 13 new tasks (P7-009 through P7-021): web UI, agent tools, CLI, coord architecture
|
||||
- P7-002 (extra SSO), P7-003 (extra LLM), P7-005 (perf), P7-008 (v0.1.0 tag) moved to Phase 8
|
||||
- Phase 8 added as "Polish & Beta (v0.1.0)"
|
||||
- Reason: platform isn't feature-complete enough for beta — web UI is scaffolded but non-functional for real use, agent tooling is minimal, CLI needs model switching
|
||||
|
||||
## Open Questions
|
||||
|
||||
(none at this time)
|
||||
|
||||
## Corrections
|
||||
|
||||
### 2026-03-13 — Vertical slice reorder (Jason directed)
|
||||
|
||||
**Original plan:** Linear Phase 0 → 1 → 2 → ... execution.
|
||||
|
||||
**Correction:** Vertical slice first. Scaffold monorepo, then build the Pi TUI → Gateway → Discord communication spine end-to-end before backfilling auth, brain, memory, CRUD, etc.
|
||||
|
||||
**Why:** Validate the architecture's core message flow before investing in horizontal layers. If the communication channels don't work, nothing else matters.
|
||||
|
||||
**Revised execution sequence:**
|
||||
|
||||
| Step | Tasks (cross-phase) | What it proves |
|
||||
| ---- | -------------------------------------------------------------------- | ------------------------ |
|
||||
| 1 | P0-001: Scaffold monorepo | Build system works |
|
||||
| 2 | P0-005: Docker Compose (PG + Valkey) | Infrastructure runs |
|
||||
| 3 | P0-002: @mosaicstack/types (minimal — gateway, agent, chat types) | Shared contracts |
|
||||
| 4 | P1-001: Gateway scaffold (minimal NestJS + Fastify) | API surface boots |
|
||||
| 5 | P1-007: WebSocket server (chat streaming) | Real-time channel works |
|
||||
| 6 | P1-008: Basic agent dispatch (single provider) | LLM responds |
|
||||
| 7 | P2-001: @mosaicstack/agent — Pi SDK integration (minimal) | Pi sessions work |
|
||||
| 8 | P6-005: Pi TUI integration (mosaic tui → gateway) | TUI ↔ Gateway proven |
|
||||
| 9 | P5-001: Plugin host (channel plugin interface) | Plugin arch works |
|
||||
| 10 | P5-002: Discord plugin (bot + channel) | Discord ↔ Gateway proven |
|
||||
| — | Then backfill: auth, brain, db, queue, OTEL, CI, web dashboard, etc. |
|
||||
|
||||
### Session 9 — Phase 3 Web Dashboard (P3-001 through P3-007)
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 9 | 2026-03-12 | Phase 3 | P3-001 through P3-007 | Full web dashboard: Next.js 16 scaffold, auth pages, chat UI, tasks (list+kanban), projects, settings, admin. PRs #82-#89 merged. |
|
||||
|
||||
### Session 10 — Phase 3 verification (P3-008)
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 10 | 2026-03-13 | Phase 3 | P3-008 | Phase 3 verification: typecheck 18/18, lint 18/18, format clean, build green (10 routes), 10 tests pass. Phase 3 complete. |
|
||||
|
||||
### Session 10 (continued) — Phase 4 Memory & Intelligence
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 10 | 2026-03-13 | Phase 4 | P4-001 through P4-007 | Full memory + log system: DB schema (preferences, insights w/ pgvector, agent_logs, skills, summarization_jobs), @mosaicstack/memory + @mosaicstack/log packages, embedding service, summarization pipeline w/ cron, memory tools in agent sessions, skill management CRUD. All gates green. |
|
||||
|
||||
### Session 12 — Phase 7 planning + execution start
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 12 | 2026-03-15 | Phase 7 | Planning | Merged rescope PR #119. Created 15 Gitea issues (#120-#134) for P7-009 through P7-021 + FIX-02/FIX-03. Planned 10-wave execution order with 2-worker parallelism. |
|
||||
|
||||
**Phase 7 execution plan (10 waves, max 2 parallel workers):**
|
||||
|
||||
| Wave | Task A | Task B |
|
||||
| ---- | ------------------------------- | -------------------------------------------- |
|
||||
| 1 | P7-009 Web chat WS (#120) | P7-001 MCP hardening (#52) |
|
||||
| 2 | P7-010 Conversation mgmt (#121) | P7-015 Agent tools (#126) |
|
||||
| 3 | P7-011 Project views (#122) | P7-016 MCP client (#127) |
|
||||
| 4 | P7-012 Provider UI (#123) | P7-017 Skill invocation (#128) |
|
||||
| 5 | P7-013 Settings persist (#124) | P7-018 CLI model switch (#129) |
|
||||
| 6 | P7-014 Admin panel (#125) | P7-019 CLI sessions (#130) |
|
||||
| 7 | P7-020 Coord DB (#131) | — |
|
||||
| 8 | FIX-02 TUI state (#133) | FIX-03 Agent sandbox (#134) |
|
||||
| 9 | P7-004 E2E Playwright (#55) | P7-006 Docs (#57) + P7-007 Deploy docs (#58) |
|
||||
| 10 | P7-021 Verify Phase 7 (#132) | — |
|
||||
|
||||
### Session 12 — Phase 7 completion summary
|
||||
|
||||
**All 17 Phase 7 tasks + 2 backlog fixes completed in a single session.**
|
||||
|
||||
PRs merged: #136, #137, #138, #139, #140, #141, #142, #143, #144, #145, #146, #147, #148, #149, #150, #151, #152, #153
|
||||
Issues closed: #52, #55, #57, #58, #120-#134
|
||||
|
||||
**Verification evidence:**
|
||||
|
||||
- Typecheck: 32/32 tasks green
|
||||
- Lint: 18/18 packages green
|
||||
- Format: All files clean
|
||||
- 19 PRs squash-merged to main, all quality gates passed
|
||||
|
||||
**Phase 7 delivered:**
|
||||
|
||||
- Web: functional chat (WS streaming), conversation management, project detail views, provider UI, settings persistence, admin panel
|
||||
- Agent: 7 new tools (file/git/shell/web), MCP server (14 tools), MCP client (external server bridge), skill invocation
|
||||
- CLI: model/provider switching, session management
|
||||
- Infrastructure: coord DB migration, agent sandbox hardening
|
||||
- Quality: E2E Playwright suite (~35 tests), comprehensive docs (user/admin/dev/deployment)
|
||||
- Fixes: TUI state updater, agent session sandboxing
|
||||
|
||||
### Session 13 — CLI Command Architecture (P8-005, P8-006)
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | -------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| 13 | 2026-03-15 | Phase 8 | P8-005, P8-006 | CLI command architecture implemented. DB schema, brain repo, gateway endpoints, CLI commands. PR #158 merged. |
|
||||
|
||||
**Changes delivered:**
|
||||
|
||||
- DB: Extended agents table (projectId, ownerId, systemPrompt, allowedTools, skills, isSystem). Added agentId to conversations.
|
||||
- Brain: New agents repository with findAccessible (owner's + system agents).
|
||||
- Gateway: /api/agents CRUD, consolidated /api/missions with user-scoped CRUD + /tasks sub-routes, coord slimmed to file-based only, agentConfigId wired into session creation.
|
||||
- CLI: `mosaic agent` (--list, --new, --show, --update, --delete), `mosaic mission` (--list, --init, --plan, --update, task subcommand), `mosaic prdy` (gateway-aware), shared with-auth + select-dialog utilities.
|
||||
- TUI: --agent and --project flags, agent name display in top bar, agentId in socket payload.
|
||||
- Types: agentId added to ChatMessagePayload.
|
||||
- Tests: 23/23 gateway tests pass (updated ownership test for user-scoped missions).
|
||||
|
||||
### Session 14 — Platform Architecture Plan Augmentation + Task Breakdown
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | --------- | ---------- | ------------------------------------------------------------- |
|
||||
| 14 | 2026-03-15 | Phase 8 | P8-018 | Augmented plan, created 13 issues, created Phase 8 milestone. |
|
||||
|
||||
**Decisions made:**
|
||||
|
||||
- This plan is Phase 7 feature extension work, not Phase 8 beta scope. P8-001–P8-004 (SSO, LLM, perf, release gate) are deferred to far future.
|
||||
- `/provider` OAuth in TUI: URL-to-clipboard + Valkey poll token pattern (same as Pi agent)
|
||||
- Add `mutable` column to preferences now (P8-007 DB migration)
|
||||
- Teams architecture: `teams` + `team_members` tables, `teamId`/`ownerType` on projects. Workspace path branches on owner type: `users/<uid>/` vs `teams/<tid>/`.
|
||||
- Phase dependency chain decided: Wave 1 (DB+Types) → Wave 2 (TUI+toolhardening) → Wave 3 (gateway registry, gating) → Wave 4 (prefs+commands) → Wave 5 (reload+GC) → Wave 6 (workspaces) → Wave 7 (autocomplete) → Wave 8 (verify).
|
||||
|
||||
**Plan augmentations added:**
|
||||
|
||||
- Teams Architecture section (DB schema, workspace paths, RBAC)
|
||||
- REST Route Specifications table
|
||||
- `/provider` OAuth flow (URL+clipboard+polling)
|
||||
- Preferences `mutable` migration spec
|
||||
- Test Strategy (per-task test files + key test cases)
|
||||
- Phase Execution Order (dependency graph + wave plan)
|
||||
|
||||
**Issues created:** #160–#172 (Gitea milestone ms-165)
|
||||
**P8-018 closed:** Spin-off stubs created (gatekeeper-service.md, task-queue-unification.md, chroot-sandboxing.md)
|
||||
|
||||
**Next:** Begin execution at Wave 1 — P8-007 (DB migrations) + P8-008 (Types) in parallel.
|
||||
|
||||
---
|
||||
|
||||
### Session 15 — 2026-04-19 — MVP Rollup Manifest Authored
|
||||
|
||||
| Session | Date | Milestone | Tasks Done | Outcome |
|
||||
| ------- | ---------- | -------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 15 | 2026-04-19 | (rollup-level) | MVP-T01 (manifest), MVP-T02 (archive iuv-v2), MVP-T03 (land FED planning) | Authored MVP rollup manifest at `docs/MISSION-MANIFEST.md`. Federation v1 planning merged to `main` (PR #468 / commit `66512550`). Install-ux-v2 archived as complete. |
|
||||
|
||||
**Gap context:** The MVP scratchpad was last updated at Session 14 (2026-03-15). In the intervening month, two sub-missions ran outside the MVP framework: `install-ux-hardening` (complete, `mosaic-v0.0.25`) and `install-ux-v2` (complete on 2026-04-19, `0.0.27` → `0.0.29`). Both archived under `docs/archive/missions/`. The phase-based execution from Sessions 1–14 (Phases 0–8, issues #1–#172) substantially shipped during this window via those sub-missions and standalone PRs — the MVP mission was nominally active but had no rollup manifest tracking it.
|
||||
|
||||
**User reframe (this session):**
|
||||
|
||||
> There will be more in the MVP. This will inevitably become scope creep. I need a solution that works via webUI, TUI, CLI, and just works for MVP. Federation is required because I need it to work NOW, so my disparate jarvis-brain usage can be consolidated properly.
|
||||
|
||||
**Decisions:**
|
||||
|
||||
1. **MVP is the rollup mission**, not a single-purpose mission. Federation v1 is one workstream of MVP, not MVP itself. Phase 0–8 work is preserved as historical context but is no longer the primary control plane.
|
||||
2. **Three-surface parity (webUI / TUI / CLI) is a cross-cutting MVP requirement** (MVP-X1), not a workstream. Encoded explicitly so it can't be silently dropped.
|
||||
3. **Scope creep is named and accommodated.** Manifest has explicit "Likely Additional Workstreams" section listing PRD-derived candidates without committing execution capacity to them.
|
||||
4. **Workstream isolation** — each workstream gets its own manifest under `docs/{workstream}/MISSION-MANIFEST.md`. MVP manifest is rollup only.
|
||||
5. **Archive-don't-delete** — install-ux-v2 manifest moved to `docs/archive/missions/install-ux-v2-20260405/` with status corrected to `complete` (IUV-M03 closeout note added pointing at PR #446 + releases 0.0.27 → 0.0.29).
|
||||
6. **Federation planning landed first** — PR #468 merged before MVP manifest authored, so the manifest references real on-`main` artifacts.
|
||||
|
||||
**Open items:**
|
||||
|
||||
- `.mosaic/orchestrator/mission.json` MVP slot remains empty (zero milestones). Tracked as MVP-T04. Defer until next session — does not block W1 kickoff. Open question: hand-edit vs. `mosaic coord init` reinit.
|
||||
- Additional workstreams (web dashboard parity, TUI/CLI completion, remote control, multi-user/SSO, LLM provider expansion, MCP, brain) anticipated per PRD but not declared. Pre-staged in manifest's "Likely Additional Workstreams" list.
|
||||
|
||||
**Artifacts this session:**
|
||||
|
||||
| Artifact | Status |
|
||||
| -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| PR #468 (`docs(federation): PRD, milestones, mission manifest, and M1 task breakdown`) | merged 2026-04-19 → `main` (commit `66512550`) |
|
||||
| `docs/MISSION-MANIFEST.md` (MVP rollup, replaces install-ux-v2 manifest) | authored on `docs/mvp-mission-manifest` branch |
|
||||
| `docs/TASKS.md` (MVP rollup, points at workstream task files) | authored |
|
||||
| Install-ux-v2 manifest + tasks + scratchpad + iuv-m03-design | moved to `docs/archive/missions/install-ux-v2-20260405/` with status corrected to complete |
|
||||
|
||||
**Next:** PR `docs/mvp-mission-manifest` → merge to `main` → next session begins W1 / FED-M1 from clean state.
|
||||
|
||||
---
|
||||
|
||||
## Session 16 — 2026-04-19 — claude
|
||||
|
||||
**Mode:** Delivery (W1 / FED-M1 execution)
|
||||
**Branch:** `feat/federation-m1-tier-config`
|
||||
**Context budget:** 200K, currently ~45% used (compaction-aware)
|
||||
|
||||
**Goal:** FED-M1-01 — extend `mosaic.config.json` schema: add `"federated"` to tier enum.
|
||||
|
||||
**Critical reconciliation surfaced during pre-flight:**
|
||||
|
||||
The federation PRD (`docs/federation/PRD.md` line 247) defines three tiers: `local | standalone | federated`.
|
||||
The existing code (`packages/config/src/mosaic-config.ts`, `packages/mosaic/src/types.ts`, `packages/mosaic/src/stages/gateway-config.ts`) uses `local | team`.
|
||||
|
||||
`team` is the same conceptual tier as PRD `standalone` (Postgres + Valkey, no pgvector). Rather than carrying a confusing alias forever, FED-M1-01 will rename `team` → `standalone` and add `federated` as a third value, so all downstream federation work has a coherent vocabulary.
|
||||
|
||||
Affected files (storage-tier semantics only — Team/workspace usages unaffected):
|
||||
|
||||
- `packages/config/src/mosaic-config.ts` (StorageTier type, validator enum, defaults)
|
||||
- `packages/mosaic/src/types.ts` (GatewayStorageTier)
|
||||
- `packages/mosaic/src/stages/gateway-config.ts` (~10 references)
|
||||
- `packages/mosaic/src/stages/gateway-config.spec.ts` (test references)
|
||||
- Possibly `tools/e2e-install-test.sh` (referenced grep) and headless env hint string
|
||||
|
||||
**Worker plan:**
|
||||
|
||||
1. Spawn sonnet subagent with explicit task spec + the reconciliation context above.
|
||||
2. Worker delivers diff; orchestrator runs `pnpm typecheck && pnpm lint && pnpm format:check`.
|
||||
3. Independent `feature-dev:code-reviewer` subagent reviews diff.
|
||||
4. Second independent verification subagent (general-purpose, sonnet) verifies reviewer's claims and confirms all `'team'` storage-tier references migrated, no `Team`/workspace bleed.
|
||||
5. Open PR via tea CLI; wait for CI; queue-guard; squash merge; record actuals.
|
||||
|
||||
**Open items:**
|
||||
|
||||
- `MVP-T04` (sync `.mosaic/orchestrator/mission.json`) still deferred.
|
||||
- `team` tier rename touches install wizard headless env vars (`MOSAIC_STORAGE_TIER=team`); will need 0.0.x deprecation note in scratchpad if release notes are written this milestone.
|
||||
|
||||
---
|
||||
|
||||
## Session 17 — 2026-04-19 — claude
|
||||
|
||||
**Mode:** Delivery (W1 / FED-M1 execution; resumed after compaction)
|
||||
**Branches landed this run:** `feat/federation-m1-tier-config` (PR #470), `feat/federation-m1-compose` (PR #471), `feat/federation-m1-pgvector` (PR #472)
|
||||
**Branch active at end:** `feat/federation-m1-detector` (FED-M1-04, ready to push)
|
||||
|
||||
**Tasks closed:** FED-M1-01, FED-M1-02, FED-M1-03 (all merged to `main` via squash, CI green, issue #460 still open as milestone).
|
||||
|
||||
**FED-M1-04 — tier-detector:** Worker delivered `apps/gateway/src/bootstrap/tier-detector.ts` (~210 lines) + `tier-detector.spec.ts` (12 tests). Independent code review (sonnet) returned `changes-required` with 3 issues:
|
||||
|
||||
1. CRITICAL: `probeValkey` missing `connectTimeout: 5000` on the ioredis Redis client (defaulted to 10s, violated fail-fast spec).
|
||||
2. IMPORTANT: `probePgvector` catch block did not discriminate "library not installed" (use `pgvector/pgvector:pg17`) from permission errors.
|
||||
3. IMPORTANT: Federated tier silently skipped Valkey probe when `queue.type !== 'bullmq'` (computed Valkey URL conditionally).
|
||||
|
||||
Worker fix-up round addressed all three:
|
||||
|
||||
- L147: `connectTimeout: 5000` added to Redis options
|
||||
- L113-117: catch block branches on `extension "vector" is not available` substring → distinct remediation per failure mode
|
||||
- L206-215: federated branch fails fast with `service: 'config'` if `queue.type !== 'bullmq'`, then probes Valkey unconditionally
|
||||
- 4 new tests (8 → 12 total) cover each fix specifically
|
||||
|
||||
Independent verifier (haiku) confirmed all 6 verification claims (line numbers, test presence, suite green: 12/12 PASS).
|
||||
|
||||
**Process note — review pipeline working as designed:**
|
||||
|
||||
Initial verifier (haiku) on the first delivery returned "OK to ship" but missed the 3 deeper issues that the sonnet code-reviewer caught. This validates the user's "always verify subagent claims independently with another subagent" rule — but specifically with the **right tier** for the task: code review needs sonnet-level reasoning, while haiku is fine for verifying surface claims (line counts, file existence) once review issues are known. Going forward: code review uses sonnet (`feature-dev:code-reviewer`), claim verification uses haiku.
|
||||
|
||||
**Followup tasks tracked but deferred:**
|
||||
|
||||
- #7: `tier=local` hardcoded in gateway-config resume branches (~262, ~317) — pre-existing bug, fix during M1-06 (doctor) or M1-09 (regression).
|
||||
- #8: confirm `packages/config/dist` not git-tracked.
|
||||
|
||||
**Next:** PR for FED-M1-04 → CI wait → merge. Then FED-M1-05 (migration script, codex/sonnet, 10K).
|
||||
|
||||
---
|
||||
|
||||
## Session 18 — 2026-04-19 — FED-M1-07 + FED-M1-08
|
||||
|
||||
**Branches landed this run:** `feat/federation-m1-integration` (PR #476, FED-M1-07), `feat/federation-m1-migrate-test` (PR #477, FED-M1-08)
|
||||
**Branch active at end:** none — both PRs merged to main, branches deleted
|
||||
|
||||
**M1 progress:** 8 of 12 tasks done. Remaining: M1-09 (regression e2e, haiku), M1-10 (security review, sonnet), M1-11 (docs, haiku), M1-12 (close + release, orchestrator).
|
||||
|
||||
### FED-M1-07 — Integration tests for federated tier gateway boot
|
||||
|
||||
Three test files under `apps/gateway/src/__tests__/integration/` gated by `FEDERATED_INTEGRATION=1`:
|
||||
|
||||
- `federated-boot.success.integration.test.ts` — `detectAndAssertTier` resolves; `pg_extension` row for `vector` exists
|
||||
- `federated-boot.pg-unreachable.integration.test.ts` — throws `TierDetectionError` with `service: 'postgres'` when PG port is closed
|
||||
- `federated-pgvector.integration.test.ts` — TEMP table with `vector(3)` column round-trips data
|
||||
|
||||
Independent code review (sonnet) returned VERDICT: B with two IMPORTANT items, both fixed in the same PR:
|
||||
|
||||
- Port 5499 collision risk → replaced with `net.createServer().listen(0)` reserved-port helper
|
||||
- `afterAll` and `sql` scoped outside `describe` → moved both inside `describe.skipIf` block
|
||||
|
||||
Independent surface verifier (haiku) confirmed all claims. 4/4 tests pass live; 4/4 skip cleanly without env var.
|
||||
|
||||
### FED-M1-08 — Migration integration test (caught real P0 bug)
|
||||
|
||||
`packages/storage/src/migrate-tier.integration.test.ts` seeds temp PGlite with cross-table data (users, teams, team_members, conversations, messages), runs `runMigrateTier`, asserts row counts + spot-checks. Gated by `FEDERATED_INTEGRATION=1`.
|
||||
|
||||
**P0 bug surfaced and fixed in same PR:** `DrizzleMigrationSource.readTable()` returns Drizzle's camelCase keys (`emailVerified`, `userId`); `PostgresMigrationTarget.upsertBatch()` was using them verbatim as SQL identifiers, producing `column "emailVerified" does not exist` against real federated PG. The 32 unit tests in M1-05 missed this because both source and target were mocked. Fix: `normaliseSourceRow` now applies `toSnakeCase` (`/[A-Z]/g` → `_<lowercase>`), idempotent on already-snake_case keys.
|
||||
|
||||
Code review (sonnet) returned VERDICT: B with one IMPORTANT and one MINOR, both fixed:
|
||||
|
||||
- `createPgliteDbWithVector` and `runPgliteMigrations` were initially added to `@mosaicstack/db` public exports → moved to `packages/storage/src/test-utils/pglite-with-vector.ts` (avoids polluting prod consumers with WASM bundle)
|
||||
- `afterAll` did not call `cleanTarget` → added before connection close, ensuring orphan rows cleaned even on test panic
|
||||
|
||||
Side change: `packages/storage/package.json` gained `"type": "module"` (codebase convention; required for `import.meta.url` in test-utils). All other workspace packages already declared this.
|
||||
|
||||
### Process notes for this session
|
||||
|
||||
- Review-then-verify pipeline now battle-tested: M1-08 reviewer caught the P0 bug + the public-API leak that the worker would have shipped. Without review, both would have gone to main.
|
||||
- Integration tests are paying for themselves immediately: M1-08 caught a real P0 in M1-05 that 32 mocked unit tests missed. Going forward, **at least one real-services integration test per code-mutating PR** should become a soft norm where feasible.
|
||||
- TASKS.md status updates continue to ride on the matching feature branch (avoids direct-to-main commits).
|
||||
|
||||
**Followup tasks tracked but still deferred (no change):**
|
||||
|
||||
- #7: `tier=local` hardcoded in gateway-config resume branches (~262, ~317)
|
||||
- #8: confirm `packages/config/dist` not git-tracked
|
||||
|
||||
**Next:** FED-M1-09 — standalone regression e2e (haiku canary, ~4K). Verifies that the existing `standalone` tier behavior still works end-to-end on the federation-touched build, since M1 changes touched shared paths (storage, config, gateway boot).
|
||||
|
||||
---
|
||||
|
||||
## Session 19 — 2026-04-19 — FED-M1-09 → FED-M1-12 (M1 close)
|
||||
|
||||
**Branches landed this run:** `feat/federation-m1-regression` (PR #478, M1-09), `feat/federation-m1-security-review` (PR #479, M1-10), `feat/federation-m1-docs` (PR #480, M1-11), `feat/federation-m1-close` (PR #481, M1-12)
|
||||
**Branch active at end:** none — M1 closed, all branches deleted, issue #460 closed, release tag `fed-v0.1.0-m1` published
|
||||
|
||||
**M1 progress:** 12 of 12 tasks done. **Milestone complete.**
|
||||
|
||||
### FED-M1-09 — Standalone regression canary
|
||||
|
||||
Verification-only milestone. Re-ran the existing standalone/local test suites against current `main` (with M1-01 → M1-08 merged):
|
||||
|
||||
- 4 target gateway test files: 148/148 pass (conversation-persistence, cross-user-isolation, resource-ownership, session-hardening)
|
||||
- Full gateway suite: 351 pass, 4 skipped (FEDERATED_INTEGRATION-gated only)
|
||||
- Storage unit tests: 85 pass, 1 skipped (integration-gated)
|
||||
- Top-level `pnpm test`: all green; only env-gated skips
|
||||
|
||||
No regression in standalone or local tier. Federation M1 changes are non-disruptive.
|
||||
|
||||
### FED-M1-10 — Security review (two rounds, 7 findings)
|
||||
|
||||
Independent security review surfaced three high-impact and four medium findings; all fixed in same PR.
|
||||
|
||||
**Round 1 (4 findings):**
|
||||
|
||||
- MEDIUM: Credential leak via `postgres`/`ioredis` driver error messages (DSN strings) re-thrown by `migrate-tier.ts` → caller; `cli.ts:402` outer catch
|
||||
- MEDIUM: Same leak in `tier-detection.ts` `probePostgresMeasured` / `probePgvectorMeasured` → emitted as JSON by `mosaic gateway doctor --json`
|
||||
- LOW-MEDIUM: No advisory lock on `migrate-tier`; two concurrent invocations could both pass `checkTargetPreconditions` (non-atomic) and race
|
||||
- ADVISORY: `SKIP_TABLES` lacked rationale comment
|
||||
|
||||
**Fixes:**
|
||||
|
||||
- New internal helper `packages/storage/src/redact-error.ts` — regex `(postgres(?:ql)?|rediss?):\/\/[^@\s]*@` → `<scheme>://***@`. NOT exported from package public surface. 10 unit tests covering all schemes, multi-URL, no-creds, case-insensitive.
|
||||
- `redactErrMsg` applied at all 5 leak sites
|
||||
- `PostgresMigrationTarget.tryAcquireAdvisoryLock()` / `releaseAdvisoryLock()` using session-scoped `pg_try_advisory_lock(hashtext('mosaic-migrate-tier'))`. Acquired before preflight, released in `finally`. Dry-run skips. Non-blocking.
|
||||
- `SKIP_TABLES` comment expanded with rationale for skipped tables (TTL'd / one-time / env-bound) AND why `accounts` (OAuth) and `provider_credentials` (AI keys) are intentionally migrated (durable user-bound, not deployment-bound).
|
||||
|
||||
**Round 2 (3 findings missed by first round):**
|
||||
|
||||
- HIGH: Round 1 regex only covered `postgres` scheme, not `redis`/`rediss` — extended to `(postgres(?:ql)?|rediss?)`
|
||||
- HIGH: `probeValkeyMeasured` was missed in Round 1 → applied `redactErrMsg`
|
||||
- MEDIUM: `cli.ts:402` migrate-tier outer catch was missed in Round 1 → applied `redactErrMsg`
|
||||
|
||||
**Process validation:** the two-round review pattern proved load-bearing for security work. A single review-then-fix cycle would have shipped the Valkey credential leak.
|
||||
|
||||
### FED-M1-11 — Docs (haiku)
|
||||
|
||||
- `docs/federation/SETUP.md` (119 lines): federated tier setup — what it is, prerequisites, docker compose start, mosaic.config.json snippet, doctor health check, troubleshooting
|
||||
- `docs/guides/migrate-tier.md` (147 lines): when to migrate, dry-run first, what migrates/skips with rationale, idempotency + advisory-lock semantics, no in-place rollback
|
||||
- `README.md` Configuration blurb linking to both
|
||||
- Runbook deferred to FED-M7 per TASKS.md scope rule
|
||||
|
||||
### FED-M1-12 — Aggregate close (this PR)
|
||||
|
||||
- Marked M1-12 done in TASKS.md
|
||||
- MISSION-MANIFEST.md: phase → "M1 complete", progress 1/7, M1 row done with PR range #470-#481, session log appended
|
||||
- This Session 19 entry added
|
||||
- Issue #460 closed via `~/.config/mosaic/tools/git/issue-close.sh -i 460`
|
||||
- Release tag `fed-v0.1.0-m1` created and pushed to gitea
|
||||
|
||||
### M1 PR ledger
|
||||
|
||||
| PR | Task | Branch |
|
||||
| ---- | ----------------------------------------- | ---------------------------------- |
|
||||
| #470 | M1-01 (tier config schema) | feat/federation-m1-tier-config |
|
||||
| #471 | M1-02 (compose overlay) | feat/federation-m1-compose |
|
||||
| #472 | M1-03 (pgvector adapter) | feat/federation-m1-pgvector |
|
||||
| #473 | M1-04 (tier-detector) | feat/federation-m1-detector |
|
||||
| #474 | M1-05 (migrate-tier script) | feat/federation-m1-migrate |
|
||||
| #475 | M1-06 (gateway doctor) | feat/federation-m1-doctor |
|
||||
| #476 | M1-07 (boot integration tests) | feat/federation-m1-integration |
|
||||
| #477 | M1-08 (migrate integration test + P0 fix) | feat/federation-m1-migrate-test |
|
||||
| #478 | M1-09 (standalone regression) | feat/federation-m1-regression |
|
||||
| #479 | M1-10 (security review fixes) | feat/federation-m1-security-review |
|
||||
| #480 | M1-11 (docs) | feat/federation-m1-docs |
|
||||
| #481 | M1-12 (aggregate close) | feat/federation-m1-close |
|
||||
|
||||
### Process learnings (M1 retrospective)
|
||||
|
||||
1. **Two-round security review is non-negotiable for security work.** First round caught postgres credential leaks; second round caught equivalent valkey leaks the worker missed when extending the regex. Single-round would have shipped HIGH severity issues.
|
||||
2. **Real-services integration tests catch what mocked unit tests cannot.** M1-08 caught a P0 in M1-05 (camelCase column names) that 32 mocked unit tests missed because both source and target were mocked. Going forward: at least one real-services test per code-mutating PR where feasible.
|
||||
3. **Test-utils for live services co-locate with consumer, not in shared library.** M1-08 reviewer caught `createPgliteDbWithVector` initially being added to `@mosaicstack/db` public exports — would have polluted prod consumers with WASM bundle. Moved to `packages/storage/src/test-utils/`.
|
||||
4. **Per-task budgets including tests/review/docs more accurate than PRD's implementation-only estimates.** M1 PRD estimated 20K; actual ~74K. Future milestones should budget the full delivery cycle.
|
||||
5. **TASKS.md status updates ride feature branches, never direct-to-main.** Caught one violation early in M1; pattern held for all 12 tasks.
|
||||
6. **Subagent tier matters.** Code review needs sonnet-level reasoning (haiku missed deep issues in M1-04); claim verification (line counts, file existence) is fine on haiku.
|
||||
|
||||
**Followup tasks still deferred (carry forward to M2):**
|
||||
|
||||
- #7: `tier=local` hardcoded in gateway-config resume branches (~262, ~317)
|
||||
- #8: confirm `packages/config/dist` not git-tracked
|
||||
|
||||
**Next mission step:** FED-M2 (Step-CA + grant schema + admin CLI). Per TASKS.md scope rule, M2 will be decomposed when it enters active planning. Issue #461 tracks scope.
|
||||
|
||||
## Session 20 — 2026-04-21 — FED-M2 kickoff
|
||||
|
||||
### Decisions
|
||||
|
||||
- **Workstream split**: parallel CODE (M2-01..M2-13, ~72K) + DEPLOY (DEPLOY-01..DEPLOY-05, ~16K) tracks; re-converge at M2-10 E2E.
|
||||
- **Test hosts**: `mos-test-1.woltje.com` (querying side / Server A), `mos-test-2.woltje.com` (serving side / Server B). Wildcard `*.woltje.com` A→174.137.97.162 already exists; Traefik wildcard cert covers both subdomains. No DNS or cert work needed pre-deploy.
|
||||
- **Portainer access**: requires `PORTAINER_INSECURE=1` flag added to mosaic wrappers (self-signed cert at `https://10.1.1.43:9443`). PR pending on `feat/mosaic-portainer-tls-flag`.
|
||||
- **Image policy**: deploy by digest (immutable) per Mosaic policy. `gateway:fed-v0.1.0-m1` digest = `sha256:9b72e202a9eecc27d31920b87b475b9e96e483c0323acc57856be4b1355db1ec`.
|
||||
|
||||
### DEPLOY-01 — image manifest verified
|
||||
|
||||
- Tag `fed-v0.1.0-m1` exists at `git.mosaicstack.dev/mosaicstack/stack/gateway`
|
||||
- Digest: `sha256:9b72e202a9eecc27d31920b87b475b9e96e483c0323acc57856be4b1355db1ec`
|
||||
- 9 layers, ~530MB total
|
||||
- Use this digest in DEPLOY-02 stack template (do NOT reference `:fed-v0.1.0-m1` tag in stack — pin to digest)
|
||||
|
||||
### Registry auth note
|
||||
|
||||
- Gitea container registry uses Bearer token flow (`/v2/token?service=container_registry&scope=repository:<repo>:pull`)
|
||||
- Username: `jarvis` (NOT `mosaicstack`); password: `gitea.mosaicstack.token` from credentials.json
|
||||
- Direct `Authorization: Bearer <pat>` does NOT work — must exchange PAT for registry token first
|
||||
|
||||
### Active PRs
|
||||
|
||||
- #483 — docs: M2 mission planning (TASKS decomposition + manifest update) — CI running
|
||||
- (pending) `feat/mosaic-portainer-tls-flag` — wrapper PORTAINER_INSECURE flag (sonnet subagent in progress)
|
||||
- (pending) `feat/federation-m2-schema` — FED-M2-01 DB schema migration (sonnet subagent in progress)
|
||||
|
||||
### MISSION-MANIFEST layout fix
|
||||
|
||||
- Initial M2 commit had Test Infrastructure block inserted by lint-staged prettier between "Last Updated" and "Parent Mission" — split mission frontmatter
|
||||
- Fixed in 3d001fdb: moved Parent Mission back to frontmatter, kept Test Infrastructure as standalone H2 between Mission and Context
|
||||
|
||||
## Session 21 — 2026-04-21/22 — DEPLOY-02 merged, gateway image bug discovered, M2-01 in remediation
|
||||
|
||||
### PRs merged
|
||||
|
||||
- **#483** — docs(federation): M2 mission planning (TASKS decomposition + manifest update)
|
||||
- **#484** — feat(mosaic-portainer): PORTAINER_INSECURE flag for self-signed TLS (wrapper sync to `~/.config/mosaic/tools/portainer/` done manually due to broken `mosaic upgrade` `set -o pipefail` on dash)
|
||||
- **#485** — feat(deploy): portainer stack template `deploy/portainer/federated-test.stack.yml` for federation test instances [DEPLOY-02]
|
||||
|
||||
### Stack deployed (mos-test-1, mos-test-2)
|
||||
|
||||
- Both stacks created on Portainer endpoint 3 (`local` Swarm @ 10.1.1.43, the only endpoint with traefik-public + woltje.com wildcard cert)
|
||||
- Swarm ID `l7z67tfpd4bvj4979ufpkyi50`
|
||||
- Image pinned to digest `sha256:9b72e202a9eecc27d31920b87b475b9e96e483c0323acc57856be4b1355db1ec`
|
||||
- Traefik labels target `${HOST_FQDN}` per env
|
||||
|
||||
### CRITICAL FINDING — gateway image runtime-broken
|
||||
|
||||
- `docker run` against `gateway:fed-v0.1.0-m1` fails immediately:
|
||||
`Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'dotenv' imported from /app/dist/main.js`
|
||||
- Root cause: `docker/gateway.Dockerfile` copies `/app/node_modules` from builder — but pnpm puts deps in the content-addressed `.pnpm/` store with symlinks at `apps/gateway/node_modules/*`. The runner stage misses the symlinks → Node can't resolve workspace deps.
|
||||
- M1 release was never runtime-tested as a stripped container; CI passed because tests run in dev tree where pnpm symlinks are intact.
|
||||
- **Fix in flight** (subagent `a78a9ab0ddae91fbc`): switch builder to `pnpm --filter @mosaic/gateway --prod deploy /deploy`, then runner copies `/deploy/node_modules` + `/deploy/dist` + `/deploy/package.json`.
|
||||
|
||||
### M2-01 schema review verdict — NEEDS CHANGES
|
||||
|
||||
- PR #486 (`feat/federation-m2-schema`) — independent reviewer (sonnet) found 2 real issues:
|
||||
1. `federation_audit_log` time-range indexes missing `.desc()` on `created_at` (3 places)
|
||||
2. Reserved columns missing per TASKS.md M2-01 spec: `query_hash`, `outcome`, `bytes_out` (M4 will write; spec said reserve now)
|
||||
- Also notes (advisory): subject_user_id correctly `text` (matches BetterAuth users.id; spec defect, not code defect); peer→grant cascade test not present (would be trivial to add)
|
||||
- **Remediation in flight** (subagent `a673dd9355dc26f82` in worktree `agent-a4404ac1`): apply DESC + reserved cols, regenerate migration in place (preferred) or stack 0009 (fallback), force-push, post PR comment.
|
||||
|
||||
### Process notes
|
||||
|
||||
- Branch race incident: schema subagent + wrapper subagent both ran in main checkout → schema files appeared on wrapper branch. Recovered by TaskStop, `git checkout --` to clean, respawned schema subagent with `isolation: "worktree"`. **Rule going forward:** any subagent doing code edits gets `isolation: "worktree"` unless work is single-file and the orchestrator confirms no other branch will touch overlapping files.
|
||||
- `pr-create.sh` shell-quotes backticks badly → use `tea pr create --repo mosaicstack/stack` directly (matches CLI-skill behavior). Will leave a followup to harden pr-create.sh.
|
||||
- Gitea registry auth: bearer-token exchange flow (`/v2/token?service=container_registry&scope=repository:<repo>:pull`) — direct `Authorization: Bearer <pat>` returns 401.
|
||||
- Portainer Swarm stack create endpoint: `POST /api/stacks/create/swarm/string?endpointId=<id>` (NOT `/api/stacks?type=1` — deprecated and rejected with 400).
|
||||
|
||||
### In-flight at compaction boundary
|
||||
|
||||
- Subagent `a78a9ab0ddae91fbc` — Dockerfile pnpm-deploy fix → PR (not yet opened at handoff)
|
||||
- Subagent `a673dd9355dc26f82` — M2-01 schema remediation (DESC + reserved cols) → force-push to PR #486
|
||||
- Both will trigger CI; orchestrator must independently re-review fixes (especially the security-adjacent schema work) per "always verify subagent claims" rule.
|
||||
|
||||
### Next after subagents return
|
||||
|
||||
1. Independent re-review of schema remediation (different subagent, fresh context)
|
||||
2. Merge #486 if green
|
||||
3. Merge Dockerfile fix PR if green → triggers Kaniko CI rebuild → capture new digest
|
||||
4. Update `deploy/portainer/federated-test.stack.yml` to new digest in a small PR
|
||||
5. Redeploy mos-test-1 + mos-test-2 (Portainer stack update via API)
|
||||
6. Verify HTTPS reachability + `/health` endpoint at both hosts
|
||||
7. DEPLOY-03/04 acceptance probes (`mosaic gateway doctor --json`, pgvector `vector(3)` round-trip)
|
||||
8. DEPLOY-05: author `docs/federation/TEST-INFRA.md`
|
||||
9. M2-02 (Step-CA sidecar) kicks off after image health is green
|
||||
|
||||
### Session 23 — 2026-04-21 — M2 close + M3 decomposition
|
||||
|
||||
**Closed at compaction boundary:** all 13 M2 tasks done, PRs #494–#503 merged to `main`, tag `fed-v0.2.0-m2` published, Gitea release notes posted, issue #461 closed. Main at `4ece6dc6`.
|
||||
|
||||
**M2 hardening landed in PR #501** (security review remediation):
|
||||
|
||||
- CRIT-1: post-issuance OID verification in `ca.service.ts` (rejects cert if `mosaic_grant_id` / `mosaic_subject_user_id` extensions missing or mismatched)
|
||||
- CRIT-2: atomic activation guard `WHERE status='pending'` on grant + `WHERE state='pending'` on peer; throws `ConflictException` if lost race
|
||||
- HIGH-2: removed try/catch fallback in `extractCertNotAfter` — parse failures propagate as 500 (no silent 90-day default)
|
||||
- HIGH-4: token slice for logging (`${token.slice(0, 8)}...`) — no full token in stdout
|
||||
- HIGH-5: `redeem()` wrapped in try/catch with best-effort failure audit; uses `null` (not `'unknown'`) for nullable UUID FK fallback
|
||||
- MED-3: `createToken` validates `grant.peerId === dto.peerId`; `BadRequestException` on mismatch
|
||||
|
||||
**Remaining M2 security findings deferred to M3+:**
|
||||
|
||||
- HIGH-1: peerId/subjectUserId tenancy validation on `createGrant` (M3 ScopeService work surfaces this)
|
||||
- HIGH-3: Step-CA cert SHA-256 fingerprint pinning (M5 cert handling)
|
||||
- MED-1: token entropy already 32 bytes — wontfix
|
||||
- MED-2: per-route rate limit on enrollment endpoint (M4 rate limit work)
|
||||
- MED-4: CSR CN binding to peer's commonName (M3 AuthGuard work)
|
||||
|
||||
**M3 decomposition landed in this session:**
|
||||
|
||||
- 14 tasks (M3-01..M3-14), ~100K estimate
|
||||
- Structure mirrors M1/M2 pattern: foundation → server stream + client stream + harness in parallel → integration → E2E → security review → docs → close
|
||||
- M3-02 ships local two-gateway docker-compose (`tools/federation-harness/`) so M3-11 E2E is not blocked on the Portainer test bed (which is still blocked on `FED-M2-DEPLOY-IMG-FIX`)
|
||||
|
||||
**Subagent doctrine retained from M2:**
|
||||
|
||||
- All worker subagents use `isolation: "worktree"` to prevent branch-race incidents
|
||||
- Code review is independent (different subagent, no overlap with author of work)
|
||||
- `tea pr create --repo mosaicstack/stack --login mosaicstack` is the working PR-create path; `pr-create.sh` has shell-quoting bugs (followup #45 if not already filed)
|
||||
- Cost tier: foundational implementation = sonnet, docs = haiku, complex multi-file architecture (security review, scope service) = sonnet with two review rounds
|
||||
|
||||
**Next concrete step:**
|
||||
|
||||
1. PR for the M3 planning artifact (this commit) — branch `docs/federation-m3-planning`
|
||||
2. After merge, kickoff M3-01 (DTOs) on `feat/federation-m3-types` with sonnet subagent in worktree
|
||||
3. Once M3-01 lands, fan out: M3-02 (harness) || M3-03 (AuthGuard) → M3-04 (ScopeService) || M3-08 (FederationClient)
|
||||
4. Re-converge at M3-10 (Integration) → M3-11 (E2E)
|
||||
@@ -1,46 +0,0 @@
|
||||
# Mission Manifest — Tess Interaction Agent
|
||||
|
||||
## Mission
|
||||
|
||||
- **ID:** tess-20260712
|
||||
- **Issue:** #706
|
||||
- **Branch:** `feat/tess-interaction-agent`
|
||||
- **Phase:** Execution
|
||||
- **Current Milestone:** TESS-M1 — Runtime contracts and security foundation
|
||||
- **Progress:** 0 / 5 delivery milestones complete
|
||||
- **Status:** active
|
||||
- **Owner:** Mosaic orchestrator; Mos is coordinating fleet authority
|
||||
- **Source PRD:** `docs/PRD.md` — `TESS-*` requirements
|
||||
- **Scratchpad:** `docs/scratchpads/tess-20260712.md`
|
||||
|
||||
## Mission Statement
|
||||
|
||||
Ship Tess as Jason's durable Pi-native GPT-5.6 Sol high-reasoning interaction agent for Discord and CLI, with safe visibility/control of Mosaic fleet and transitional Hermes capabilities, while Mos remains the coding/general orchestration authority.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. Mosaic is the enterprise AI hub; Hermes is a reference migration adapter.
|
||||
2. Gateway is the single API surface.
|
||||
3. Mos owns coding/general fleet orchestration; Tess owns human interaction, visibility, mediation, and migration access.
|
||||
4. Runtime, transport, channel, memory, and external-agent integrations are replaceable adapters.
|
||||
5. No source task completes before merged PR, terminal-green CI, independent review, and linked task/issue closure.
|
||||
|
||||
## Milestones
|
||||
|
||||
| ID | Issue | Name | Status | Exit gate |
|
||||
| ------- | ----- | ------------------------------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| TESS-M1 | #707 | Runtime contracts and security foundation | ready | AgentRuntimeProvider, normalized events/capabilities/errors, RBAC/audit contracts and contract tests merged |
|
||||
| TESS-M2 | #708 | Durable Pi Tess service and state | not-started | GPT-5.6 Sol high service starts, resumes, checkpoints, and passes restart/compaction tests |
|
||||
| TESS-M3 | #709 | Discord and CLI interaction surfaces | not-started | One durable session works through dedicated Discord binding and `mosaic tess`, including attach and approvals |
|
||||
| TESS-M4 | #710 | Fleet, Mos, Hermes, memory, state, and tool plugins | in-progress | Fleet/Mos boundary and transitional capability matrix demonstrated end-to-end |
|
||||
| TESS-M5 | #711 | Matrix/native migration, recovery, documentation, and qualification | not-started | Transport parity, migration/rollback matrix, security review, docs, greenfield and deployment validation complete |
|
||||
|
||||
## Success Criteria
|
||||
|
||||
All `AC-TESS-*` criteria in `docs/PRD.md` are mapped to reproducible evidence. The final operational test must prove Discord + CLI session continuity, fleet/Mos coordination, authorized Hermes transition capabilities, denial/audit paths, restart recovery, and rollback.
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Date | Runtime | Outcome |
|
||||
| ------- | ---------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| S1 | 2026-07-12 | Hermes / GPT-5.6 Sol | User commission captured; Mosaic/OpenViking/session/code archaeology completed; issue #706 created; PRD and task control plane initialized. |
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user