Files
stack/docs/scratchpads/mvp-20260312.md
Jarvis b9fb8aab57
All checks were successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
docs(federation): close FED-M1 milestone
- TASKS.md: mark FED-M1-12 done with PR/issue/tag references
- MISSION-MANIFEST.md: phase=M1 complete, progress 1/7, M1 row done with PR range #470-#481, session log appended
- scratchpad: Session 19 entry covering M1-09 → M1-12 with PR ledger and M1 retrospective learnings

Refs #460
2026-04-19 21:12:52 -05:00

39 KiB
Raw Blame History

Mission Scratchpad — MVP

Append-only log. NEVER delete entries. NEVER overwrite sections. This is the orchestrator's working memory across sessions.

Original Mission Prompt

Active mission detected: MVP. Read the mission state files and report status.
User confirmed: start the planning gate.

Planning Decisions

2026-03-13 — Milestone and task breakdown

  • PRD defines 8 phases (Phase 07), mapped 1:1 to Gitea milestones
  • 59 issues created on git.mosaicstack.dev/mosaic/mosaic-stack (#1#59)
  • Each phase has a verification task as the final issue
  • Task IDs use P{phase}-{seq} format (P0-001 through P7-008)
  • Repo created as mosaic/mosaic-stack (private) on Gitea
  • Milestones: ms-157 (Phase 0) through ms-164 (Phase 7)
  • Total: 59 tasks across 8 milestones

Phase structure

Phase Version Tasks Focus
0 v0.0.1 9 Foundation — monorepo, types, db, auth, OTEL, Docker, CI
1 v0.0.2 9 Core API — gateway, brain, queue, routes, WebSocket
2 v0.0.3 7 Agent Layer — Pi SDK, multi-provider, routing, coord
3 v0.0.4 8 Web Dashboard — Next.js, chat, tasks, projects, admin
4 v0.0.5 7 Memory & Intelligence — memory, log, summarization, skills
5 v0.0.6 5 Remote Control — Discord, Telegram, SSO
6 v0.0.7 6 CLI & Tools — CLI, prdy, quality-rails, installer, TUI
7 v0.1.0 8 Polish & Beta — MCP, providers, E2E, docs, release

Session Log

Session Date Milestone Tasks Done Outcome
1 2026-03-13 Planning Planning gate Milestones created, 59 issues created, TASKS.md populated, manifest updated
2 2026-03-13 Vertical slice P1-001, P1-007, P1-008, P2-001, P5-002, P6-005 Communication spine built and merged (PR #61). Gateway + TUI + Discord. 3-agent gatekeeper review, 10/16 issues remediated, 4 deferred.
3 2026-03-13 Foundation P0-002, P0-005, P0-006 Foundation layer merged (PR #65). Docker Compose (PG+pgvector, Valkey, OTEL Collector, Jaeger), OTEL auto-instrumentation in gateway, @mosaicstack/types with DTOs + Socket.IO typed event maps.

Session 4 — Docker Compose fix

Session Date Milestone Tasks Done Outcome
4 2026-03-12 Foundation (fix) Fixed Jaeger tag (2→2.6.0), remapped PG/Valkey ports (5433/6380) to avoid host conflicts. PR #66 merged to main.

Verification evidence:

  • All 4 containers healthy (PG, Valkey, OTEL Collector, Jaeger)
  • OTEL pipeline proven: mosaic-gateway service visible in Jaeger UI
  • Gateway traces flow through Collector → Jaeger

Session 5 — Phase 0-1 completion

Session Date Milestone Tasks Done Outcome
5 2026-03-12 Phase 0, Phase 1 P0-003, P0-004, P0-007, P0-008, P0-009, P1-002P1-006, P1-009 Foundation + Core API complete. DB, auth, CI, brain, queue, CRUD routes all merged and green.

Session 6 — Phase 2 agent layer

Session Date Milestone Tasks Done Outcome
6 2026-03-12 Phase 2 P2-002, P2-003, P2-004, P2-005, P2-006, FIX-01 Multi-provider routing, tool registration, coord migration, session management, dispose() fix. PRs #74#78.

Session 7-8 — Phase 2 verification + completion

Session Date Milestone Tasks Done Outcome
7-8 2026-03-12 Phase 2 P2-007 19 unit tests (routing + coord). PR #79 merged, issue #25 closed. Phase 2 complete.

Session 11 — Phase 5 completion

Session Date Milestone Tasks Done Outcome
11 2026-03-14 Phase 5 P5-005 Wired Telegram plugin into gateway (was stubbed). Updated .env.example with all P5 env vars. PR #99 merged, issue #45 closed. Phase 5 complete.

Findings during verification:

  • Telegram plugin was built but not wired into gateway (stub warning in plugin.module.ts)
  • Discord plugin was fully wired
  • SSO/Authentik OIDC adapter was fully wired
  • All three quality gates passing

Session 11 (continued) — Phase 6 completion

Session Date Milestone Tasks Done Outcome
11 2026-03-14 Phase 6 P6-002, P6-003, P6-004, P6-001, P6-006 Full CLI & Tools migration. PRs #100-#104 merged. Also fixed 2 gateway startup bugs (PR #102). Phase 6 complete.

Phase 6 details:

  • P6-002: @mosaicstack/prdy migrated from v0 (~400 LOC). PR #101.
  • P6-003: @mosaicstack/quality-rails migrated from v0 (~500 LOC). PR #100.
  • P6-004: @mosaicstack/mosaic wizard migrated from v0 (2272 LOC, 28 files). PR #103.
  • P6-001: CLI subcommands wired — tui, prdy, quality-rails, wizard all working. PR #104.
  • BUG-1: PLUGIN_REGISTRY circular import fixed via plugin.tokens.ts. PR #102.
  • BUG-2: AuthStorage.create() → .inMemory() to prevent silent exit. PR #102.

Session 11 (continued) — E2E testing + bug fixes + Phase 7 rescope

Bug fixes merged during E2E testing (PRs #107-#117):

  • CI: from_secret syntax for Woodpecker v2 (#107)
  • Gateway: dotenv loading from monorepo root (#108)
  • Gateway: missing @Inject() decorators causing silent hang (#109)
  • Gateway: CORS + memory userId + pgvector auto-init (#110)
  • Auth: BetterAuth trustedOrigins for web dashboard (#111)
  • Auth: CORS headers on raw BetterAuth HTTP handler (#112)
  • Husky: removed deprecated v9 shim lines (#113)
  • CLI: login command + authenticated TUI sessions (#114)
  • CLI: Origin header on auth requests (#115)
  • Agent: Ollama provider registration with openai-completions API (#116, #117)

E2E testing results:

  • Web UI: login works, projects list, chats list (but chat doesn't function)
  • TUI: authenticated connection works, agent responds via Ollama llama3.2
  • Agent tools: brain, coord, memory tools confirmed working
  • Gateway: all routes mapped, providers register correctly

Phase 7 rescoped (Jason directed):

  • Phase 7 renamed from "Polish & Beta" to "Feature Completion (v0.0.8)"
  • Added 13 new tasks (P7-009 through P7-021): web UI, agent tools, CLI, coord architecture
  • P7-002 (extra SSO), P7-003 (extra LLM), P7-005 (perf), P7-008 (v0.1.0 tag) moved to Phase 8
  • Phase 8 added as "Polish & Beta (v0.1.0)"
  • Reason: platform isn't feature-complete enough for beta — web UI is scaffolded but non-functional for real use, agent tooling is minimal, CLI needs model switching

Open Questions

(none at this time)

Corrections

2026-03-13 — Vertical slice reorder (Jason directed)

Original plan: Linear Phase 0 → 1 → 2 → ... execution.

Correction: Vertical slice first. Scaffold monorepo, then build the Pi TUI → Gateway → Discord communication spine end-to-end before backfilling auth, brain, memory, CRUD, etc.

Why: Validate the architecture's core message flow before investing in horizontal layers. If the communication channels don't work, nothing else matters.

Revised execution sequence:

Step Tasks (cross-phase) What it proves
1 P0-001: Scaffold monorepo Build system works
2 P0-005: Docker Compose (PG + Valkey) Infrastructure runs
3 P0-002: @mosaicstack/types (minimal — gateway, agent, chat types) Shared contracts
4 P1-001: Gateway scaffold (minimal NestJS + Fastify) API surface boots
5 P1-007: WebSocket server (chat streaming) Real-time channel works
6 P1-008: Basic agent dispatch (single provider) LLM responds
7 P2-001: @mosaicstack/agent — Pi SDK integration (minimal) Pi sessions work
8 P6-005: Pi TUI integration (mosaic tui → gateway) TUI ↔ Gateway proven
9 P5-001: Plugin host (channel plugin interface) Plugin arch works
10 P5-002: Discord plugin (bot + channel) Discord ↔ Gateway proven
Then backfill: auth, brain, db, queue, OTEL, CI, web dashboard, etc.

Session 9 — Phase 3 Web Dashboard (P3-001 through P3-007)

Session Date Milestone Tasks Done Outcome
9 2026-03-12 Phase 3 P3-001 through P3-007 Full web dashboard: Next.js 16 scaffold, auth pages, chat UI, tasks (list+kanban), projects, settings, admin. PRs #82-#89 merged.

Session 10 — Phase 3 verification (P3-008)

Session Date Milestone Tasks Done Outcome
10 2026-03-13 Phase 3 P3-008 Phase 3 verification: typecheck 18/18, lint 18/18, format clean, build green (10 routes), 10 tests pass. Phase 3 complete.

Session 10 (continued) — Phase 4 Memory & Intelligence

Session Date Milestone Tasks Done Outcome
10 2026-03-13 Phase 4 P4-001 through P4-007 Full memory + log system: DB schema (preferences, insights w/ pgvector, agent_logs, skills, summarization_jobs), @mosaicstack/memory + @mosaicstack/log packages, embedding service, summarization pipeline w/ cron, memory tools in agent sessions, skill management CRUD. All gates green.

Session 12 — Phase 7 planning + execution start

Session Date Milestone Tasks Done Outcome
12 2026-03-15 Phase 7 Planning Merged rescope PR #119. Created 15 Gitea issues (#120-#134) for P7-009 through P7-021 + FIX-02/FIX-03. Planned 10-wave execution order with 2-worker parallelism.

Phase 7 execution plan (10 waves, max 2 parallel workers):

Wave Task A Task B
1 P7-009 Web chat WS (#120) P7-001 MCP hardening (#52)
2 P7-010 Conversation mgmt (#121) P7-015 Agent tools (#126)
3 P7-011 Project views (#122) P7-016 MCP client (#127)
4 P7-012 Provider UI (#123) P7-017 Skill invocation (#128)
5 P7-013 Settings persist (#124) P7-018 CLI model switch (#129)
6 P7-014 Admin panel (#125) P7-019 CLI sessions (#130)
7 P7-020 Coord DB (#131)
8 FIX-02 TUI state (#133) FIX-03 Agent sandbox (#134)
9 P7-004 E2E Playwright (#55) P7-006 Docs (#57) + P7-007 Deploy docs (#58)
10 P7-021 Verify Phase 7 (#132)

Session 12 — Phase 7 completion summary

All 17 Phase 7 tasks + 2 backlog fixes completed in a single session.

PRs merged: #136, #137, #138, #139, #140, #141, #142, #143, #144, #145, #146, #147, #148, #149, #150, #151, #152, #153 Issues closed: #52, #55, #57, #58, #120-#134

Verification evidence:

  • Typecheck: 32/32 tasks green
  • Lint: 18/18 packages green
  • Format: All files clean
  • 19 PRs squash-merged to main, all quality gates passed

Phase 7 delivered:

  • Web: functional chat (WS streaming), conversation management, project detail views, provider UI, settings persistence, admin panel
  • Agent: 7 new tools (file/git/shell/web), MCP server (14 tools), MCP client (external server bridge), skill invocation
  • CLI: model/provider switching, session management
  • Infrastructure: coord DB migration, agent sandbox hardening
  • Quality: E2E Playwright suite (~35 tests), comprehensive docs (user/admin/dev/deployment)
  • Fixes: TUI state updater, agent session sandboxing

Session 13 — CLI Command Architecture (P8-005, P8-006)

Session Date Milestone Tasks Done Outcome
13 2026-03-15 Phase 8 P8-005, P8-006 CLI command architecture implemented. DB schema, brain repo, gateway endpoints, CLI commands. PR #158 merged.

Changes delivered:

  • DB: Extended agents table (projectId, ownerId, systemPrompt, allowedTools, skills, isSystem). Added agentId to conversations.
  • Brain: New agents repository with findAccessible (owner's + system agents).
  • Gateway: /api/agents CRUD, consolidated /api/missions with user-scoped CRUD + /tasks sub-routes, coord slimmed to file-based only, agentConfigId wired into session creation.
  • CLI: mosaic agent (--list, --new, --show, --update, --delete), mosaic mission (--list, --init, --plan, --update, task subcommand), mosaic prdy (gateway-aware), shared with-auth + select-dialog utilities.
  • TUI: --agent and --project flags, agent name display in top bar, agentId in socket payload.
  • Types: agentId added to ChatMessagePayload.
  • Tests: 23/23 gateway tests pass (updated ownership test for user-scoped missions).

Session 14 — Platform Architecture Plan Augmentation + Task Breakdown

Session Date Milestone Tasks Done Outcome
14 2026-03-15 Phase 8 P8-018 Augmented plan, created 13 issues, created Phase 8 milestone.

Decisions made:

  • This plan is Phase 7 feature extension work, not Phase 8 beta scope. P8-001P8-004 (SSO, LLM, perf, release gate) are deferred to far future.
  • /provider OAuth in TUI: URL-to-clipboard + Valkey poll token pattern (same as Pi agent)
  • Add mutable column to preferences now (P8-007 DB migration)
  • Teams architecture: teams + team_members tables, teamId/ownerType on projects. Workspace path branches on owner type: users/<uid>/ vs teams/<tid>/.
  • Phase dependency chain decided: Wave 1 (DB+Types) → Wave 2 (TUI+toolhardening) → Wave 3 (gateway registry, gating) → Wave 4 (prefs+commands) → Wave 5 (reload+GC) → Wave 6 (workspaces) → Wave 7 (autocomplete) → Wave 8 (verify).

Plan augmentations added:

  • Teams Architecture section (DB schema, workspace paths, RBAC)
  • REST Route Specifications table
  • /provider OAuth flow (URL+clipboard+polling)
  • Preferences mutable migration spec
  • Test Strategy (per-task test files + key test cases)
  • Phase Execution Order (dependency graph + wave plan)

Issues created: #160#172 (Gitea milestone ms-165) P8-018 closed: Spin-off stubs created (gatekeeper-service.md, task-queue-unification.md, chroot-sandboxing.md)

Next: Begin execution at Wave 1 — P8-007 (DB migrations) + P8-008 (Types) in parallel.


Session 15 — 2026-04-19 — MVP Rollup Manifest Authored

Session Date Milestone Tasks Done Outcome
15 2026-04-19 (rollup-level) MVP-T01 (manifest), MVP-T02 (archive iuv-v2), MVP-T03 (land FED planning) Authored MVP rollup manifest at docs/MISSION-MANIFEST.md. Federation v1 planning merged to main (PR #468 / commit 66512550). Install-ux-v2 archived as complete.

Gap context: The MVP scratchpad was last updated at Session 14 (2026-03-15). In the intervening month, two sub-missions ran outside the MVP framework: install-ux-hardening (complete, mosaic-v0.0.25) and install-ux-v2 (complete on 2026-04-19, 0.0.270.0.29). Both archived under docs/archive/missions/. The phase-based execution from Sessions 114 (Phases 08, issues #1#172) substantially shipped during this window via those sub-missions and standalone PRs — the MVP mission was nominally active but had no rollup manifest tracking it.

User reframe (this session):

There will be more in the MVP. This will inevitably become scope creep. I need a solution that works via webUI, TUI, CLI, and just works for MVP. Federation is required because I need it to work NOW, so my disparate jarvis-brain usage can be consolidated properly.

Decisions:

  1. MVP is the rollup mission, not a single-purpose mission. Federation v1 is one workstream of MVP, not MVP itself. Phase 08 work is preserved as historical context but is no longer the primary control plane.
  2. Three-surface parity (webUI / TUI / CLI) is a cross-cutting MVP requirement (MVP-X1), not a workstream. Encoded explicitly so it can't be silently dropped.
  3. Scope creep is named and accommodated. Manifest has explicit "Likely Additional Workstreams" section listing PRD-derived candidates without committing execution capacity to them.
  4. Workstream isolation — each workstream gets its own manifest under docs/{workstream}/MISSION-MANIFEST.md. MVP manifest is rollup only.
  5. Archive-don't-delete — install-ux-v2 manifest moved to docs/archive/missions/install-ux-v2-20260405/ with status corrected to complete (IUV-M03 closeout note added pointing at PR #446 + releases 0.0.27 → 0.0.29).
  6. Federation planning landed first — PR #468 merged before MVP manifest authored, so the manifest references real on-main artifacts.

Open items:

  • .mosaic/orchestrator/mission.json MVP slot remains empty (zero milestones). Tracked as MVP-T04. Defer until next session — does not block W1 kickoff. Open question: hand-edit vs. mosaic coord init reinit.
  • Additional workstreams (web dashboard parity, TUI/CLI completion, remote control, multi-user/SSO, LLM provider expansion, MCP, brain) anticipated per PRD but not declared. Pre-staged in manifest's "Likely Additional Workstreams" list.

Artifacts this session:

Artifact Status
PR #468 (docs(federation): PRD, milestones, mission manifest, and M1 task breakdown) merged 2026-04-19 → main (commit 66512550)
docs/MISSION-MANIFEST.md (MVP rollup, replaces install-ux-v2 manifest) authored on docs/mvp-mission-manifest branch
docs/TASKS.md (MVP rollup, points at workstream task files) authored
Install-ux-v2 manifest + tasks + scratchpad + iuv-m03-design moved to docs/archive/missions/install-ux-v2-20260405/ with status corrected to complete

Next: PR docs/mvp-mission-manifest → merge to main → next session begins W1 / FED-M1 from clean state.


Session 16 — 2026-04-19 — claude

Mode: Delivery (W1 / FED-M1 execution) Branch: feat/federation-m1-tier-config Context budget: 200K, currently ~45% used (compaction-aware)

Goal: FED-M1-01 — extend mosaic.config.json schema: add "federated" to tier enum.

Critical reconciliation surfaced during pre-flight:

The federation PRD (docs/federation/PRD.md line 247) defines three tiers: local | standalone | federated. The existing code (packages/config/src/mosaic-config.ts, packages/mosaic/src/types.ts, packages/mosaic/src/stages/gateway-config.ts) uses local | team.

team is the same conceptual tier as PRD standalone (Postgres + Valkey, no pgvector). Rather than carrying a confusing alias forever, FED-M1-01 will rename teamstandalone and add federated as a third value, so all downstream federation work has a coherent vocabulary.

Affected files (storage-tier semantics only — Team/workspace usages unaffected):

  • packages/config/src/mosaic-config.ts (StorageTier type, validator enum, defaults)
  • packages/mosaic/src/types.ts (GatewayStorageTier)
  • packages/mosaic/src/stages/gateway-config.ts (~10 references)
  • packages/mosaic/src/stages/gateway-config.spec.ts (test references)
  • Possibly tools/e2e-install-test.sh (referenced grep) and headless env hint string

Worker plan:

  1. Spawn sonnet subagent with explicit task spec + the reconciliation context above.
  2. Worker delivers diff; orchestrator runs pnpm typecheck && pnpm lint && pnpm format:check.
  3. Independent feature-dev:code-reviewer subagent reviews diff.
  4. Second independent verification subagent (general-purpose, sonnet) verifies reviewer's claims and confirms all 'team' storage-tier references migrated, no Team/workspace bleed.
  5. Open PR via tea CLI; wait for CI; queue-guard; squash merge; record actuals.

Open items:

  • MVP-T04 (sync .mosaic/orchestrator/mission.json) still deferred.
  • team tier rename touches install wizard headless env vars (MOSAIC_STORAGE_TIER=team); will need 0.0.x deprecation note in scratchpad if release notes are written this milestone.

Session 17 — 2026-04-19 — claude

Mode: Delivery (W1 / FED-M1 execution; resumed after compaction) Branches landed this run: feat/federation-m1-tier-config (PR #470), feat/federation-m1-compose (PR #471), feat/federation-m1-pgvector (PR #472) Branch active at end: feat/federation-m1-detector (FED-M1-04, ready to push)

Tasks closed: FED-M1-01, FED-M1-02, FED-M1-03 (all merged to main via squash, CI green, issue #460 still open as milestone).

FED-M1-04 — tier-detector: Worker delivered apps/gateway/src/bootstrap/tier-detector.ts (~210 lines) + tier-detector.spec.ts (12 tests). Independent code review (sonnet) returned changes-required with 3 issues:

  1. CRITICAL: probeValkey missing connectTimeout: 5000 on the ioredis Redis client (defaulted to 10s, violated fail-fast spec).
  2. IMPORTANT: probePgvector catch block did not discriminate "library not installed" (use pgvector/pgvector:pg17) from permission errors.
  3. IMPORTANT: Federated tier silently skipped Valkey probe when queue.type !== 'bullmq' (computed Valkey URL conditionally).

Worker fix-up round addressed all three:

  • L147: connectTimeout: 5000 added to Redis options
  • L113-117: catch block branches on extension "vector" is not available substring → distinct remediation per failure mode
  • L206-215: federated branch fails fast with service: 'config' if queue.type !== 'bullmq', then probes Valkey unconditionally
  • 4 new tests (8 → 12 total) cover each fix specifically

Independent verifier (haiku) confirmed all 6 verification claims (line numbers, test presence, suite green: 12/12 PASS).

Process note — review pipeline working as designed:

Initial verifier (haiku) on the first delivery returned "OK to ship" but missed the 3 deeper issues that the sonnet code-reviewer caught. This validates the user's "always verify subagent claims independently with another subagent" rule — but specifically with the right tier for the task: code review needs sonnet-level reasoning, while haiku is fine for verifying surface claims (line counts, file existence) once review issues are known. Going forward: code review uses sonnet (feature-dev:code-reviewer), claim verification uses haiku.

Followup tasks tracked but deferred:

  • #7: tier=local hardcoded in gateway-config resume branches (~262, ~317) — pre-existing bug, fix during M1-06 (doctor) or M1-09 (regression).
  • #8: confirm packages/config/dist not git-tracked.

Next: PR for FED-M1-04 → CI wait → merge. Then FED-M1-05 (migration script, codex/sonnet, 10K).


Session 18 — 2026-04-19 — FED-M1-07 + FED-M1-08

Branches landed this run: feat/federation-m1-integration (PR #476, FED-M1-07), feat/federation-m1-migrate-test (PR #477, FED-M1-08) Branch active at end: none — both PRs merged to main, branches deleted

M1 progress: 8 of 12 tasks done. Remaining: M1-09 (regression e2e, haiku), M1-10 (security review, sonnet), M1-11 (docs, haiku), M1-12 (close + release, orchestrator).

FED-M1-07 — Integration tests for federated tier gateway boot

Three test files under apps/gateway/src/__tests__/integration/ gated by FEDERATED_INTEGRATION=1:

  • federated-boot.success.integration.test.tsdetectAndAssertTier resolves; pg_extension row for vector exists
  • federated-boot.pg-unreachable.integration.test.ts — throws TierDetectionError with service: 'postgres' when PG port is closed
  • federated-pgvector.integration.test.ts — TEMP table with vector(3) column round-trips data

Independent code review (sonnet) returned VERDICT: B with two IMPORTANT items, both fixed in the same PR:

  • Port 5499 collision risk → replaced with net.createServer().listen(0) reserved-port helper
  • afterAll and sql scoped outside describe → moved both inside describe.skipIf block

Independent surface verifier (haiku) confirmed all claims. 4/4 tests pass live; 4/4 skip cleanly without env var.

FED-M1-08 — Migration integration test (caught real P0 bug)

packages/storage/src/migrate-tier.integration.test.ts seeds temp PGlite with cross-table data (users, teams, team_members, conversations, messages), runs runMigrateTier, asserts row counts + spot-checks. Gated by FEDERATED_INTEGRATION=1.

P0 bug surfaced and fixed in same PR: DrizzleMigrationSource.readTable() returns Drizzle's camelCase keys (emailVerified, userId); PostgresMigrationTarget.upsertBatch() was using them verbatim as SQL identifiers, producing column "emailVerified" does not exist against real federated PG. The 32 unit tests in M1-05 missed this because both source and target were mocked. Fix: normaliseSourceRow now applies toSnakeCase (/[A-Z]/g_<lowercase>), idempotent on already-snake_case keys.

Code review (sonnet) returned VERDICT: B with one IMPORTANT and one MINOR, both fixed:

  • createPgliteDbWithVector and runPgliteMigrations were initially added to @mosaicstack/db public exports → moved to packages/storage/src/test-utils/pglite-with-vector.ts (avoids polluting prod consumers with WASM bundle)
  • afterAll did not call cleanTarget → added before connection close, ensuring orphan rows cleaned even on test panic

Side change: packages/storage/package.json gained "type": "module" (codebase convention; required for import.meta.url in test-utils). All other workspace packages already declared this.

Process notes for this session

  • Review-then-verify pipeline now battle-tested: M1-08 reviewer caught the P0 bug + the public-API leak that the worker would have shipped. Without review, both would have gone to main.
  • Integration tests are paying for themselves immediately: M1-08 caught a real P0 in M1-05 that 32 mocked unit tests missed. Going forward, at least one real-services integration test per code-mutating PR should become a soft norm where feasible.
  • TASKS.md status updates continue to ride on the matching feature branch (avoids direct-to-main commits).

Followup tasks tracked but still deferred (no change):

  • #7: tier=local hardcoded in gateway-config resume branches (~262, ~317)
  • #8: confirm packages/config/dist not git-tracked

Next: FED-M1-09 — standalone regression e2e (haiku canary, ~4K). Verifies that the existing standalone tier behavior still works end-to-end on the federation-touched build, since M1 changes touched shared paths (storage, config, gateway boot).


Session 19 — 2026-04-19 — FED-M1-09 → FED-M1-12 (M1 close)

Branches landed this run: feat/federation-m1-regression (PR #478, M1-09), feat/federation-m1-security-review (PR #479, M1-10), feat/federation-m1-docs (PR #480, M1-11), feat/federation-m1-close (PR #481, M1-12) Branch active at end: none — M1 closed, all branches deleted, issue #460 closed, release tag fed-v0.1.0-m1 published

M1 progress: 12 of 12 tasks done. Milestone complete.

FED-M1-09 — Standalone regression canary

Verification-only milestone. Re-ran the existing standalone/local test suites against current main (with M1-01 → M1-08 merged):

  • 4 target gateway test files: 148/148 pass (conversation-persistence, cross-user-isolation, resource-ownership, session-hardening)
  • Full gateway suite: 351 pass, 4 skipped (FEDERATED_INTEGRATION-gated only)
  • Storage unit tests: 85 pass, 1 skipped (integration-gated)
  • Top-level pnpm test: all green; only env-gated skips

No regression in standalone or local tier. Federation M1 changes are non-disruptive.

FED-M1-10 — Security review (two rounds, 7 findings)

Independent security review surfaced three high-impact and four medium findings; all fixed in same PR.

Round 1 (4 findings):

  • MEDIUM: Credential leak via postgres/ioredis driver error messages (DSN strings) re-thrown by migrate-tier.ts → caller; cli.ts:402 outer catch
  • MEDIUM: Same leak in tier-detection.ts probePostgresMeasured / probePgvectorMeasured → emitted as JSON by mosaic gateway doctor --json
  • LOW-MEDIUM: No advisory lock on migrate-tier; two concurrent invocations could both pass checkTargetPreconditions (non-atomic) and race
  • ADVISORY: SKIP_TABLES lacked rationale comment

Fixes:

  • New internal helper packages/storage/src/redact-error.ts — regex (postgres(?:ql)?|rediss?):\/\/[^@\s]*@<scheme>://***@. NOT exported from package public surface. 10 unit tests covering all schemes, multi-URL, no-creds, case-insensitive.
  • redactErrMsg applied at all 5 leak sites
  • PostgresMigrationTarget.tryAcquireAdvisoryLock() / releaseAdvisoryLock() using session-scoped pg_try_advisory_lock(hashtext('mosaic-migrate-tier')). Acquired before preflight, released in finally. Dry-run skips. Non-blocking.
  • SKIP_TABLES comment expanded with rationale for skipped tables (TTL'd / one-time / env-bound) AND why accounts (OAuth) and provider_credentials (AI keys) are intentionally migrated (durable user-bound, not deployment-bound).

Round 2 (3 findings missed by first round):

  • HIGH: Round 1 regex only covered postgres scheme, not redis/rediss — extended to (postgres(?:ql)?|rediss?)
  • HIGH: probeValkeyMeasured was missed in Round 1 → applied redactErrMsg
  • MEDIUM: cli.ts:402 migrate-tier outer catch was missed in Round 1 → applied redactErrMsg

Process validation: the two-round review pattern proved load-bearing for security work. A single review-then-fix cycle would have shipped the Valkey credential leak.

FED-M1-11 — Docs (haiku)

  • docs/federation/SETUP.md (119 lines): federated tier setup — what it is, prerequisites, docker compose start, mosaic.config.json snippet, doctor health check, troubleshooting
  • docs/guides/migrate-tier.md (147 lines): when to migrate, dry-run first, what migrates/skips with rationale, idempotency + advisory-lock semantics, no in-place rollback
  • README.md Configuration blurb linking to both
  • Runbook deferred to FED-M7 per TASKS.md scope rule

FED-M1-12 — Aggregate close (this PR)

  • Marked M1-12 done in TASKS.md
  • MISSION-MANIFEST.md: phase → "M1 complete", progress 1/7, M1 row done with PR range #470-#481, session log appended
  • This Session 19 entry added
  • Issue #460 closed via ~/.config/mosaic/tools/git/issue-close.sh -i 460
  • Release tag fed-v0.1.0-m1 created and pushed to gitea

M1 PR ledger

PR Task Branch
#470 M1-01 (tier config schema) feat/federation-m1-tier-config
#471 M1-02 (compose overlay) feat/federation-m1-compose
#472 M1-03 (pgvector adapter) feat/federation-m1-pgvector
#473 M1-04 (tier-detector) feat/federation-m1-detector
#474 M1-05 (migrate-tier script) feat/federation-m1-migrate
#475 M1-06 (gateway doctor) feat/federation-m1-doctor
#476 M1-07 (boot integration tests) feat/federation-m1-integration
#477 M1-08 (migrate integration test + P0 fix) feat/federation-m1-migrate-test
#478 M1-09 (standalone regression) feat/federation-m1-regression
#479 M1-10 (security review fixes) feat/federation-m1-security-review
#480 M1-11 (docs) feat/federation-m1-docs
#481 M1-12 (aggregate close) feat/federation-m1-close

Process learnings (M1 retrospective)

  1. Two-round security review is non-negotiable for security work. First round caught postgres credential leaks; second round caught equivalent valkey leaks the worker missed when extending the regex. Single-round would have shipped HIGH severity issues.
  2. Real-services integration tests catch what mocked unit tests cannot. M1-08 caught a P0 in M1-05 (camelCase column names) that 32 mocked unit tests missed because both source and target were mocked. Going forward: at least one real-services test per code-mutating PR where feasible.
  3. Test-utils for live services co-locate with consumer, not in shared library. M1-08 reviewer caught createPgliteDbWithVector initially being added to @mosaicstack/db public exports — would have polluted prod consumers with WASM bundle. Moved to packages/storage/src/test-utils/.
  4. Per-task budgets including tests/review/docs more accurate than PRD's implementation-only estimates. M1 PRD estimated 20K; actual ~74K. Future milestones should budget the full delivery cycle.
  5. TASKS.md status updates ride feature branches, never direct-to-main. Caught one violation early in M1; pattern held for all 12 tasks.
  6. Subagent tier matters. Code review needs sonnet-level reasoning (haiku missed deep issues in M1-04); claim verification (line counts, file existence) is fine on haiku.

Followup tasks still deferred (carry forward to M2):

  • #7: tier=local hardcoded in gateway-config resume branches (~262, ~317)
  • #8: confirm packages/config/dist not git-tracked

Next mission step: FED-M2 (Step-CA + grant schema + admin CLI). Per TASKS.md scope rule, M2 will be decomposed when it enters active planning. Issue #461 tracks scope.