stack/docs/plans/task-queue-unification.md

Task Queue Unification — @mosaic/queue as Unified Orchestration Layer

Status: Stub — deferred. Referenced from 2026-03-15-agent-platform-architecture.md (Task Queue & Orchestration section). Implement after Workspaces (P8-015) is complete. Requires workspace file structure to be in place.

Date: 2026-03-15 Packages: packages/queue, packages/coord, packages/db, apps/gateway

---

Problem Statement

Two disconnected task systems exist:

1. @mosaic/coord — file-based missions (mission.json, TASKS.md), file locks, subprocess spawning. Single-machine orchestrator pattern.
2. PG tables (tasks, mission_tasks, missions) — DB-backed CRUD, REST API, Brain repos.

An agent using coord_mission_status gets file data. The dashboard shows DB data. They are never in sync.

---

Vision

@mosaic/queue becomes the unified task orchestration service bridging PG, workspace files, and Valkey:

• DB is source of truth for structured state (status, assignees, timestamps)
• Workspace files (TASKS.md, PRDs) are working copies for agent interaction
• Valkey handles real-time assignment queues and agent claim locks
• Flatfile fallback for no-DB single-machine deployments (preserves @mosaic/coord pattern)

---

Scope (To Be Designed)

• @mosaic/queue refactor — elevate from ioredis primitive to task orchestration service
• DB ↔ file sync layer — writes to PG propagate to TASKS.md; file edits by agents sync back
• Task assignment queue — Valkey-backed RPUSH/BLPOP for agent task claiming
• Agent claim locks — mosaic:queue:project:{id}:lock:{taskId} with TTL
• @mosaic/coord consolidation — file-based ops ported into queue service; @mosaic/coord becomes thin adapter or deprecated
• Flatfile fallback — queue service writes JSON manifests when PG unavailable
• Status pub/sub — real-time task status updates via Valkey pub/sub
• Dependency resolution — block task assignment until dependencies are met
• Orchestrator monitor — gateway process watches task queue, assigns next based on dependency graph
• API surface — queue service exposes typed interface used by agents, gateway, and CLI

---

Dependencies

• Workspaces (P8-015) — file sync targets the workspace directory structure
• Teams architecture (P8-007) — project ownership determines queue namespacing
• DB schema stable — task/mission tables must not change mid-unification

---

References

• Original design context: docs/plans/2026-03-15-agent-platform-architecture.md → "Task Queue & Orchestration" section
• Current @mosaic/coord implementation: packages/coord/src/
• Current @mosaic/queue implementation: packages/queue/src/