Files
stack/docs/plans/task-queue-unification.md
Jarvis 774b76447d
Some checks failed
ci/woodpecker/pr/ci Pipeline failed
ci/woodpecker/push/ci Pipeline failed
fix: rename all packages from @mosaic/* to @mosaicstack/*
- Updated all package.json name fields and dependency references
- Updated all TypeScript/JavaScript imports
- Updated .woodpecker/publish.yml filters and registry paths
- Updated tools/install.sh scope default
- Updated .npmrc registry paths (worktree + host)
- Enhanced update-checker.ts with checkForAllUpdates() multi-package support
- Updated CLI update command to show table of all packages
- Added KNOWN_PACKAGES, formatAllPackagesTable, getInstallAllCommand
- Marked checkForUpdate() with @deprecated JSDoc

Closes #391
2026-04-04 21:43:23 -05:00

2.7 KiB

Task Queue Unification — @mosaicstack/queue as Unified Orchestration Layer

Status: Stub — deferred. Referenced from 2026-03-15-agent-platform-architecture.md (Task Queue & Orchestration section). Implement after Workspaces (P8-015) is complete. Requires workspace file structure to be in place.

Date: 2026-03-15 Packages: packages/queue, packages/coord, packages/db, apps/gateway


Problem Statement

Two disconnected task systems exist:

  1. @mosaicstack/coord — file-based missions (mission.json, TASKS.md), file locks, subprocess spawning. Single-machine orchestrator pattern.
  2. PG tables (tasks, mission_tasks, missions) — DB-backed CRUD, REST API, Brain repos.

An agent using coord_mission_status gets file data. The dashboard shows DB data. They are never in sync.


Vision

@mosaicstack/queue becomes the unified task orchestration service bridging PG, workspace files, and Valkey:

  • DB is source of truth for structured state (status, assignees, timestamps)
  • Workspace files (TASKS.md, PRDs) are working copies for agent interaction
  • Valkey handles real-time assignment queues and agent claim locks
  • Flatfile fallback for no-DB single-machine deployments (preserves @mosaicstack/coord pattern)

Scope (To Be Designed)

  • @mosaicstack/queue refactor — elevate from ioredis primitive to task orchestration service
  • DB ↔ file sync layer — writes to PG propagate to TASKS.md; file edits by agents sync back
  • Task assignment queue — Valkey-backed RPUSH/BLPOP for agent task claiming
  • Agent claim locks — mosaic:queue:project:{id}:lock:{taskId} with TTL
  • @mosaicstack/coord consolidation — file-based ops ported into queue service; @mosaicstack/coord becomes thin adapter or deprecated
  • Flatfile fallback — queue service writes JSON manifests when PG unavailable
  • Status pub/sub — real-time task status updates via Valkey pub/sub
  • Dependency resolution — block task assignment until dependencies are met
  • Orchestrator monitor — gateway process watches task queue, assigns next based on dependency graph
  • API surface — queue service exposes typed interface used by agents, gateway, and CLI

Dependencies

  • Workspaces (P8-015) — file sync targets the workspace directory structure
  • Teams architecture (P8-007) — project ownership determines queue namespacing
  • DB schema stable — task/mission tables must not change mid-unification

References

  • Original design context: docs/plans/2026-03-15-agent-platform-architecture.md → "Task Queue & Orchestration" section
  • Current @mosaicstack/coord implementation: packages/coord/src/
  • Current @mosaicstack/queue implementation: packages/queue/src/