chore(orchestrator): Bootstrap storage abstraction retrofit
Mission: Decouple gateway from hardcoded Postgres/Valkey backends. 20 tasks across 5 phases. Estimated total: ~214K tokens. Phase 1: Interface extraction (4 tasks) Phase 2: Wrap existing backends as adapters (5 tasks) Phase 3: Local tier implementation (4 tasks) Phase 4: Config + CLI commands (4 tasks) Phase 5: Migration + docs (3 tasks)
This commit is contained in:
Jarvis
@@ -1,73 +1,30 @@
|
||||
# Tasks — Harness Foundation
|
||||
# Tasks — Storage Abstraction Retrofit
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** Decouple gateway from hardcoded Postgres/Valkey backends. Introduce interface-driven middleware so the gateway is backend-agnostic. Default to local tier (SQLite + JSON) for zero-dependency installs.
|
||||
>
|
||||
> **`agent` column values:** `codex` | `sonnet` | `haiku` | `glm-5` | `opus` | `—` (auto/default)
|
||||
|
||||
| id | status | agent | milestone | description | pr | notes |
|
||||
| ------ | ------ | ------ | ------------------ | ------------------------------------------------------------------ | ---- | ----------- |
|
||||
| M1-001 | done | sonnet | M1: Persistence | Wire ChatGateway → ConversationsRepo for user messages | #292 | #224 closed |
|
||||
| M1-002 | done | sonnet | M1: Persistence | Wire agent event relay → ConversationsRepo for assistant responses | #292 | #225 closed |
|
||||
| M1-003 | done | sonnet | M1: Persistence | Store message metadata: model, provider, tokens, tool calls | #292 | #226 closed |
|
||||
| M1-004 | done | sonnet | M1: Persistence | Load message history into Pi session on resume | #301 | #227 closed |
|
||||
| M1-005 | done | sonnet | M1: Persistence | Context window management: summarize when >80% | #301 | #228 closed |
|
||||
| M1-006 | done | sonnet | M1: Persistence | Conversation search endpoint | #299 | #229 closed |
|
||||
| M1-007 | done | sonnet | M1: Persistence | TUI /history command | #297 | #230 closed |
|
||||
| M1-008 | done | sonnet | M1: Persistence | Verify persistence — 20 tests | #304 | #231 closed |
|
||||
| M2-001 | done | sonnet | M2: Security | InsightsRepo userId on searchByEmbedding | #290 | #232 closed |
|
||||
| M2-002 | done | sonnet | M2: Security | InsightsRepo userId on findByUser/decay | #290 | #233 closed |
|
||||
| M2-003 | done | sonnet | M2: Security | PreferencesRepo userId verified | #294 | #234 closed |
|
||||
| M2-004 | done | sonnet | M2: Security | Memory tools userId injection fixed | #294 | #235 closed |
|
||||
| M2-005 | done | sonnet | M2: Security | ConversationsRepo ownership checks | #293 | #236 closed |
|
||||
| M2-006 | done | sonnet | M2: Security | AgentsRepo findAccessible scoped | #293 | #237 closed |
|
||||
| M2-007 | done | sonnet | M2: Security | Cross-user isolation — 28 tests | #305 | #238 closed |
|
||||
| M2-008 | done | sonnet | M2: Security | Valkey SCAN + /gc admin-only | #298 | #239 closed |
|
||||
| M3-001 | done | sonnet | M3: Providers | IProviderAdapter + OllamaAdapter | #306 | #240 closed |
|
||||
| M3-002 | done | sonnet | M3: Providers | AnthropicAdapter | #309 | #241 closed |
|
||||
| M3-003 | done | sonnet | M3: Providers | OpenAIAdapter | #310 | #242 closed |
|
||||
| M3-004 | done | sonnet | M3: Providers | OpenRouterAdapter | #311 | #243 closed |
|
||||
| M3-005 | done | sonnet | M3: Providers | ZaiAdapter (GLM-5) | #314 | #244 closed |
|
||||
| M3-006 | done | sonnet | M3: Providers | Ollama embedding support | #311 | #245 closed |
|
||||
| M3-007 | done | sonnet | M3: Providers | Provider health checks | #308 | #246 closed |
|
||||
| M3-008 | done | sonnet | M3: Providers | Model capability matrix | #303 | #247 closed |
|
||||
| M3-009 | done | sonnet | M3: Providers | EmbeddingService → Ollama default | #308 | #248 closed |
|
||||
| M3-010 | done | sonnet | M3: Providers | OAuth token storage (AES-256-GCM) | #317 | #249 closed |
|
||||
| M3-011 | done | sonnet | M3: Providers | Provider credentials CRUD | #317 | #250 closed |
|
||||
| M3-012 | done | sonnet | M3: Providers | Verify providers — 40 tests | #319 | #251 closed |
|
||||
| M4-001 | done | sonnet | M4: Routing | routing_rules DB schema | #315 | #252 closed |
|
||||
| M4-002 | done | sonnet | M4: Routing | Condition types | #315 | #253 closed |
|
||||
| M4-003 | done | sonnet | M4: Routing | Action types | #315 | #254 closed |
|
||||
| M4-004 | done | sonnet | M4: Routing | Default routing rules (11 seeds) | #316 | #255 closed |
|
||||
| M4-005 | done | sonnet | M4: Routing | Task classifier (60+ tests) | #316 | #256 closed |
|
||||
| M4-006 | done | sonnet | M4: Routing | Routing decision pipeline | #318 | #257 closed |
|
||||
| M4-007 | done | sonnet | M4: Routing | /model override | #323 | #258 closed |
|
||||
| M4-008 | done | sonnet | M4: Routing | Routing transparency in session:info | #323 | #259 closed |
|
||||
| M4-009 | done | sonnet | M4: Routing | Routing rules CRUD API | #320 | #260 closed |
|
||||
| M4-010 | done | sonnet | M4: Routing | Per-user routing overrides | #320 | #261 closed |
|
||||
| M4-011 | done | sonnet | M4: Routing | Agent specialization capabilities | #320 | #262 closed |
|
||||
| M4-012 | done | sonnet | M4: Routing | Routing wired into ChatGateway | #323 | #263 closed |
|
||||
| M4-013 | done | sonnet | M4: Routing | Verify routing — 9 E2E tests | #323 | #264 closed |
|
||||
| M5-001 | done | sonnet | M5: Sessions | Agent config loaded on session create | #323 | #265 closed |
|
||||
| M5-002 | done | sonnet | M5: Sessions | /model command end-to-end | #323 | #266 closed |
|
||||
| M5-003 | done | sonnet | M5: Sessions | /agent command mid-session | #323 | #267 closed |
|
||||
| M5-004 | done | sonnet | M5: Sessions | Session ↔ conversation binding | #321 | #268 closed |
|
||||
| M5-005 | done | sonnet | M5: Sessions | Session info broadcast | #321 | #269 closed |
|
||||
| M5-006 | done | sonnet | M5: Sessions | /agent new from TUI | #321 | #270 closed |
|
||||
| M5-007 | done | sonnet | M5: Sessions | Session metrics | #321 | #271 closed |
|
||||
| M5-008 | done | sonnet | M5: Sessions | Verify sessions — 28 tests | #324 | #272 closed |
|
||||
| M6-001 | done | sonnet | M6: Jobs | BullMQ + Valkey config | #324 | #273 closed |
|
||||
| M6-002 | done | sonnet | M6: Jobs | Queue service with typed jobs | #324 | #274 closed |
|
||||
| M6-003 | done | sonnet | M6: Jobs | Summarization → BullMQ | #324 | #275 closed |
|
||||
| M6-004 | done | sonnet | M6: Jobs | GC → BullMQ | #324 | #276 closed |
|
||||
| M6-005 | done | sonnet | M6: Jobs | Tier management → BullMQ | #324 | #277 closed |
|
||||
| M6-006 | done | sonnet | M6: Jobs | Admin jobs API | #325 | #278 closed |
|
||||
| M6-007 | done | sonnet | M6: Jobs | Job event logging | #325 | #279 closed |
|
||||
| M6-008 | done | sonnet | M6: Jobs | Verify jobs | #324 | #280 closed |
|
||||
| M7-001 | done | sonnet | M7: Channel Design | IChannelAdapter interface | #325 | #281 closed |
|
||||
| M7-002 | done | sonnet | M7: Channel Design | Channel message protocol | #325 | #282 closed |
|
||||
| M7-003 | done | sonnet | M7: Channel Design | Matrix integration design | #326 | #283 closed |
|
||||
| M7-004 | done | sonnet | M7: Channel Design | Conversation multiplexing | #326 | #284 closed |
|
||||
| M7-005 | done | sonnet | M7: Channel Design | Remote auth bridging | #326 | #285 closed |
|
||||
| M7-006 | done | sonnet | M7: Channel Design | Agent-to-agent via Matrix | #326 | #286 closed |
|
||||
| M7-007 | done | sonnet | M7: Channel Design | Multi-user isolation in Matrix | #326 | #287 closed |
|
||||
| M7-008 | done | sonnet | M7: Channel Design | channel-protocol.md published | #326 | #288 closed |
|
||||
| id | status | description | issue | repo | branch | depends_on | blocks | agent | started_at | completed_at | estimate | used | notes |
|
||||
| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------ | ----- | -------------------- | ------------------------ | ----------------------------- | --------- | ----- | ---------- | ------------ | -------- | ---- | ----- |
|
||||
| SA-P1-001 | not-started | Define QueueAdapter interface in packages/queue/src/types.ts — enqueue, dequeue, length, publish, subscribe, close | | queue | feat/storage-abstraction | | SA-P1-004 | codex | | | 5K | | |
|
||||
| SA-P1-002 | not-started | Define StorageAdapter interface in packages/storage/src/types.ts — CRUD, query, find, transaction, close | | storage | feat/storage-abstraction | | SA-P1-004 | codex | | | 8K | | |
|
||||
| SA-P1-003 | not-started | Define MemoryAdapter interface in packages/memory/src/types.ts — preferences, insights, search, embedder support | | memory | feat/storage-abstraction | | SA-P1-004 | codex | | | 5K | | |
|
||||
| SA-P1-004 | not-started | Create adapter factory pattern + config types: createQueue(config), createStorage(config), createMemory(config) | | queue,storage,memory | feat/storage-abstraction | SA-P1-001,SA-P1-002,SA-P1-003 | SA-P2-001 | codex | | | 8K | | |
|
||||
| SA-P2-001 | not-started | Refactor @mosaic/queue: move queue.ts → adapters/bullmq.ts, implement QueueAdapter interface, export factory | | queue | feat/storage-abstraction | SA-P1-004 | SA-P2-004 | codex | | | 12K | | |
|
||||
| SA-P2-002 | not-started | Create @mosaic/storage package: move @mosaic/db Drizzle logic → adapters/postgres.ts, implement StorageAdapter interface | | storage | feat/storage-abstraction | SA-P1-004 | SA-P2-004 | codex | | | 15K | | |
|
||||
| SA-P2-003 | not-started | Refactor @mosaic/memory: extract pgvector logic → adapters/pgvector.ts, implement MemoryAdapter interface | | memory | feat/storage-abstraction | SA-P1-004 | SA-P2-004 | codex | | | 12K | | |
|
||||
| SA-P2-004 | not-started | Update gateway database.module.ts, queue.module.ts, memory.module.ts to use factories + NestJS DI tokens | | gateway | feat/storage-abstraction | SA-P2-001,SA-P2-002,SA-P2-003 | SA-P2-005 | codex | | | 15K | | |
|
||||
| SA-P2-005 | not-started | Verify Phase 2: existing Postgres/Valkey behavior unchanged — all existing tests pass, typecheck clean | | gateway | feat/storage-abstraction | SA-P2-004 | SA-P3-001 | codex | | | 10K | | |
|
||||
| SA-P3-001 | not-started | Implement local queue adapter: in-process Map + JSON file persistence in adapters/local.ts | | queue | feat/storage-abstraction | SA-P2-005 | SA-P3-004 | codex | | | 10K | | |
|
||||
| SA-P3-002 | not-started | Implement SQLite storage adapter: better-sqlite3 with schema mirroring Drizzle tables, in adapters/sqlite.ts | | storage | feat/storage-abstraction | SA-P2-005 | SA-P3-004 | codex | | | 20K | | |
|
||||
| SA-P3-003 | not-started | Implement keyword memory adapter: TF-IDF or simple keyword matching, no vector dependency, in adapters/keyword.ts | | memory | feat/storage-abstraction | SA-P2-005 | SA-P3-004 | codex | | | 12K | | |
|
||||
| SA-P3-004 | not-started | Verify Phase 3: gateway starts with local config (no PG/Valkey), basic CRUD works, all adapter tests pass | | gateway | feat/storage-abstraction | SA-P3-001,SA-P3-002,SA-P3-003 | SA-P4-001 | codex | | | 15K | | |
|
||||
| SA-P4-001 | not-started | Add mosaic.config.ts schema + loader: storage tier selection (local/team), backend config, defaults to local | | config | feat/storage-abstraction | SA-P3-004 | SA-P4-002 | codex | | | 10K | | |
|
||||
| SA-P4-002 | not-started | CLI: mosaic gateway init — interactive wizard to generate mosaic.config.ts with tier selection | | cli | feat/storage-abstraction | SA-P4-001 | SA-P4-003 | codex | | | 12K | | |
|
||||
| SA-P4-003 | not-started | CLI: mosaic gateway start/stop/status — lifecycle management for the gateway daemon | | cli | feat/storage-abstraction | SA-P4-002 | SA-P4-004 | codex | | | 12K | | |
|
||||
| SA-P4-004 | not-started | Verify Phase 4: fresh install with `mosaic gateway init && mosaic gateway start` works end-to-end | | cli,gateway | feat/storage-abstraction | SA-P4-003 | SA-P5-001 | codex | | | 10K | | |
|
||||
| SA-P5-001 | not-started | Migration tooling: mosaic storage export/import for local↔postgres tier migration | | cli,storage | feat/storage-abstraction | SA-P4-004 | SA-P5-002 | codex | | | 15K | | |
|
||||
| SA-P5-002 | not-started | Docker Compose profiles: local (gateway only) vs team (gateway+pg+valkey), update docker-compose.yml | | infra | feat/storage-abstraction | SA-P5-001 | SA-P5-003 | codex | | | 8K | | |
|
||||
| SA-P5-003 | not-started | Final verification + docs: README update, architecture diagram, configuration guide | | docs | feat/storage-abstraction | SA-P5-002 | | codex | | | 10K | | |
|
||||
|
||||
Reference in New Issue
Block a user