# M6 New Orchestrator Issues - Ready to Create **Total:** 34 new issues for `apps/orchestrator/` implementation **Milestone:** M6-AgentOrchestration (0.0.6) **Labels:** `orchestrator` (create this label first) --- ## Label Creation Command ```bash cd /home/localadmin/src/mosaic-stack tea labels create orchestrator --color "#FF6B35" --description "Orchestrator service (apps/orchestrator/)" ``` --- ## Phase 1: Foundation (Days 1-2) ### ORCH-101: Set up apps/orchestrator structure **Labels:** task, setup, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Create the directory structure for the orchestrator service in the monorepo. ## Acceptance Criteria - [ ] Directory structure created: `apps/orchestrator/src/{api,spawner,queue,monitor,git,killswitch,coordinator,valkey}` - [ ] Test directories created: `apps/orchestrator/tests/{unit,integration}` - [ ] package.json created with dependencies (@mosaic/shared, @mosaic/config, ioredis, bullmq, @anthropic-ai/sdk, dockerode, simple-git, fastify, zod) - [ ] tsconfig.json extends root tsconfig.base.json - [ ] .eslintrc.js and .prettierrc configured - [ ] README.md with service overview ## Dependencies None (foundation work) ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 2 for complete structure. --- ### ORCH-102: Create Fastify server with health checks **Labels:** feature, api, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Basic HTTP server for orchestrator API with health check endpoint. ## Acceptance Criteria - [ ] Fastify server in `src/api/server.ts` - [ ] Health check endpoint: GET /health (returns 200 OK) - [ ] Configuration loaded from environment variables - [ ] Pino logger integrated - [ ] Server starts on port 3001 (configurable) - [ ] Graceful shutdown handler ## Dependencies - Blocked by: #ORCH-101 ## Technical Notes ```typescript GET /health Response 200 OK: { "status": "healthy", "uptime": 12345, "timestamp": "2026-02-02T10:00:00Z" } ``` --- ### ORCH-103: Docker Compose integration for orchestrator **Labels:** task, infrastructure, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Add orchestrator service to docker-compose.yml. ## Acceptance Criteria - [ ] orchestrator service added to docker-compose.yml - [ ] Depends on: valkey, coordinator - [ ] Environment variables configured (VALKEY_URL, COORDINATOR_URL, CLAUDE_API_KEY) - [ ] Volume mounts: /var/run/docker.sock (for Docker-in-Docker), /workspace (git operations) - [ ] Health check configured - [ ] Port 3001 exposed ## Dependencies - Blocked by: #ORCH-101 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 3.3 for docker-compose.yml template. --- ### ORCH-104: Monorepo build pipeline for orchestrator **Labels:** task, infrastructure, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Update TurboRepo configuration to include orchestrator in build pipeline. ## Acceptance Criteria - [ ] turbo.json updated with orchestrator tasks - [ ] Build order: packages/\* → coordinator → orchestrator → api → web - [ ] Root package.json scripts updated (dev:orchestrator, docker:logs) - [ ] `npm run build` builds orchestrator - [ ] `npm run dev` runs orchestrator in watch mode ## Dependencies - Blocked by: #ORCH-101 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 3.2 for turbo.json configuration. --- ## Phase 2: Agent Spawning (Days 3-4) ### ORCH-105: Implement agent spawner (Claude SDK) **Labels:** feature, core, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Spawn Claude agents using Anthropic SDK. ## Acceptance Criteria - [ ] `src/spawner/agent-spawner.ts` implemented - [ ] Spawn agent with task context (repo, branch, instructions) - [ ] Claude SDK integration (@anthropic-ai/sdk) - [ ] Agent session management - [ ] Return agentId on successful spawn ## Dependencies - Blocked by: #ORCH-102 ## Technical Notes ```typescript interface SpawnAgentRequest { taskId: string; agentType: "worker" | "reviewer" | "tester"; context: { repository: string; branch: string; workItems: string[]; skills?: string[]; }; options?: { sandbox?: boolean; timeout?: number; maxRetries?: number; }; } ``` --- ### ORCH-106: Docker sandbox isolation **Labels:** feature, security, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Isolate agents in Docker containers for security. ## Acceptance Criteria - [ ] `src/spawner/docker-sandbox.ts` implemented - [ ] dockerode integration for container management - [ ] Agent runs in isolated container - [ ] Resource limits enforced (CPU, memory) - [ ] Non-root user in container - [ ] Container cleanup on agent termination ## Dependencies - Blocked by: #ORCH-105 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 7 for Docker security hardening. --- ### ORCH-107: Valkey client and state management **Labels:** feature, core, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Valkey client for orchestrator state management. ## Acceptance Criteria - [ ] `src/valkey/client.ts` with ioredis connection - [ ] State schema implemented (tasks, agents, queue) - [ ] Pub/sub for events (agent spawned, completed, failed) - [ ] Task state: pending, assigned, executing, completed, failed - [ ] Agent state: spawning, running, completed, failed, killed ## Dependencies - Blocked by: #98 (Valkey Integration), #ORCH-102 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 5 for Valkey state schema. --- ### ORCH-108: BullMQ task queue **Labels:** feature, core, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Task queue with priority and retry logic using BullMQ. ## Acceptance Criteria - [ ] `src/queue/task-queue.ts` implemented - [ ] BullMQ queue on Valkey - [ ] Priority-based task ordering - [ ] Retry logic with exponential backoff - [ ] Queue worker processes tasks - [ ] Queue monitoring (pending, active, completed, failed counts) ## Dependencies - Blocked by: #ORCH-107 ## Technical Notes ```typescript interface QueuedTask { taskId: string; priority: number; // 1-10 retries: number; maxRetries: number; context: TaskContext; } ``` --- ### ORCH-109: Agent lifecycle management **Labels:** feature, core, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Manage agent states through lifecycle (spawning → running → completed/failed). ## Acceptance Criteria - [ ] `src/spawner/agent-lifecycle.ts` implemented - [ ] State transitions: spawning → running → completed/failed/killed - [ ] State persisted in Valkey - [ ] Events emitted on state changes (pub/sub) - [ ] Agent metadata tracked (startedAt, completedAt, error) ## Dependencies - Blocked by: #ORCH-105, #ORCH-108 ## Technical Notes State machine enforces valid transitions only. --- ## Phase 3: Git Integration (Days 5-6) ### ORCH-110: Git operations (clone, commit, push) **Labels:** feature, git, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Implement git operations using simple-git. ## Acceptance Criteria - [ ] `src/git/git-operations.ts` implemented - [ ] Clone repository - [ ] Create branch - [ ] Commit changes with message - [ ] Push to remote - [ ] Git config (user.name, user.email) ## Dependencies - Blocked by: #ORCH-105 ## Technical Notes Use simple-git library. Configure git user from environment variables. --- ### ORCH-111: Git worktree management **Labels:** feature, git, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Create and manage git worktrees for agent isolation. ## Acceptance Criteria - [ ] `src/git/worktree-manager.ts` implemented - [ ] Create worktree for each agent - [ ] Worktree naming: `agent-{agentId}-{taskId}` - [ ] Cleanup worktree on agent completion - [ ] Handle worktree conflicts ## Dependencies - Blocked by: #ORCH-110 ## Technical Notes Git worktrees allow multiple agents to work on same repo without conflicts. --- ### ORCH-112: Conflict detection **Labels:** feature, git, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Detect merge conflicts before pushing. ## Acceptance Criteria - [ ] `src/git/conflict-detection.ts` implemented - [ ] Fetch latest from remote before push - [ ] Detect merge conflicts - [ ] Return conflict details to agent - [ ] Agent retries with rebase/merge ## Dependencies - Blocked by: #ORCH-110 ## Technical Notes Check for conflicts before push. If conflicts, agent must resolve. --- ## Phase 4: Coordinator Integration (Days 7-8) ### ORCH-113: Coordinator API client **Labels:** feature, integration, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** HTTP client for calling coordinator quality gates. ## Acceptance Criteria - [ ] `src/coordinator/coordinator-client.ts` implemented - [ ] POST /api/quality/check endpoint - [ ] Quality check request serialization - [ ] Response parsing (approved/rejected) - [ ] Retry on coordinator unavailable ## Dependencies - Blocked by: #ORCH-102 ## Related - Coordinator exists at `apps/coordinator/` ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 6.1 for API contract. --- ### ORCH-114: Quality gate callbacks **Labels:** feature, quality, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Call coordinator quality gates before commit/push. ## Acceptance Criteria - [ ] `src/coordinator/quality-gates.ts` implemented - [ ] Pre-commit quality check (before git commit) - [ ] Post-commit quality check (before git push) - [ ] Parse quality gate response - [ ] Block commit/push if rejected - [ ] Return rejection details to agent ## Dependencies - Blocked by: #ORCH-113 ## Technical Notes Coordinator runs: typecheck, lint, tests, coverage. Orchestrator calls coordinator. --- ### ORCH-115: Task dispatch from coordinator **Labels:** feature, integration, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Coordinator dispatches validated tasks to orchestrator. ## Acceptance Criteria - [ ] Orchestrator API endpoint: POST /agents/spawn - [ ] Coordinator calls orchestrator after quality pre-check - [ ] Task queued in Valkey - [ ] Agent spawned - [ ] Return agentId to coordinator ## Dependencies - Blocked by: #99 (Task Dispatcher), #ORCH-113 ## Related - Extends #99 (Dispatcher in control plane) ## Technical Notes Flow: User → Mosaic Stack → Coordinator (pre-check) → Orchestrator (dispatch). --- ### ORCH-116: 50% rule enforcement **Labels:** feature, quality, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Enforce 50% rule: no more than 50% AI-generated code in PR. ## Acceptance Criteria - [ ] Mechanical gates: typecheck, lint, tests, coverage (coordinator) - [ ] AI confirmation: independent AI agent reviews (coordinator) - [ ] Orchestrator calls both mechanical and AI gates - [ ] Reject if either fails - [ ] Return detailed failure reasons ## Dependencies - Blocked by: #ORCH-114 ## Technical Notes Coordinator enforces 50% rule. Orchestrator calls coordinator. --- ## Phase 5: Killswitch + Security (Days 9-10) ### ORCH-117: Killswitch implementation **Labels:** feature, security, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Emergency stop: kill single agent or all agents. ## Acceptance Criteria - [ ] `src/killswitch/killswitch.ts` implemented - [ ] POST /agents/{agentId}/kill endpoint - [ ] POST /agents/kill-all endpoint - [ ] Immediate termination (SIGKILL) - [ ] Cleanup Docker containers - [ ] Cleanup git worktrees - [ ] Update agent state to 'killed' - [ ] Audit trail logged ## Dependencies - Blocked by: #ORCH-109 ## Related - #114 (Kill Authority in control plane) ## Technical Notes Killswitch bypasses all queues. Must respond within seconds. --- ### ORCH-118: Resource cleanup **Labels:** task, infrastructure, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Clean up resources when agent terminates. ## Acceptance Criteria - [ ] `src/killswitch/cleanup.ts` implemented - [ ] Stop Docker container - [ ] Remove Docker container - [ ] Remove git worktree - [ ] Clear Valkey state - [ ] Emit cleanup event ## Dependencies - Blocked by: #ORCH-117 ## Technical Notes Run cleanup on: agent completion, agent failure, killswitch. --- ### ORCH-119: Docker security hardening **Labels:** security, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Harden Docker container security for agents. ## Acceptance Criteria - [ ] Dockerfile with multi-stage build - [ ] Non-root user (nodejs:nodejs) - [ ] Minimal base image (node:20-alpine) - [ ] No unnecessary packages - [ ] Health check in Dockerfile - [ ] Security scan passes (docker scan) ## Dependencies - Blocked by: #ORCH-106 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 7 for Dockerfile template. --- ### ORCH-120: Secret scanning **Labels:** security, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Prevent secrets from being committed. ## Acceptance Criteria - [ ] git-secrets integrated - [ ] Pre-commit hook scans for secrets - [ ] Block commit if secrets detected - [ ] Scan for API keys, tokens, passwords - [ ] Custom patterns for Claude API keys ## Dependencies - Blocked by: #ORCH-110 ## Technical Notes ```bash git secrets --add 'sk-[a-zA-Z0-9]{48}' # Claude API keys ``` --- ## Phase 6: Quality Gates (Days 11-12) ### ORCH-121: Mechanical quality gates **Labels:** feature, quality, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Implement mechanical quality gates (non-AI). ## Acceptance Criteria - [ ] TypeScript type checking - [ ] ESLint linting - [ ] Test execution (vitest) - [ ] Coverage check (>= 85%) - [ ] Build check (tsup) ## Dependencies - Blocked by: #ORCH-114 ## Related - Coordinator has gate implementations ## Technical Notes Mechanical gates are deterministic (no AI). Run via coordinator. --- ### ORCH-122: AI agent confirmation **Labels:** feature, quality, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Independent AI agent reviews changes for quality. ## Acceptance Criteria - [ ] Spawn independent AI reviewer agent - [ ] Review code changes - [ ] Check for: logic errors, security issues, best practices - [ ] Return confidence score (0.0 - 1.0) - [ ] Approve if confidence >= 0.9 ## Dependencies - Blocked by: #ORCH-114 ## Related - Coordinator calls AI reviewer ## Technical Notes AI reviewer is INDEPENDENT of worker agent (no self-review). --- ### ORCH-123: YOLO mode (gate bypass) **Labels:** feature, configuration, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** User-configurable approval gates (YOLO mode bypasses gates). ## Acceptance Criteria - [ ] Configuration option: `YOLO_MODE=true` - [ ] If YOLO mode enabled, skip quality gates - [ ] Log YOLO mode usage (audit trail) - [ ] UI warning: "Quality gates disabled" ## Dependencies - Blocked by: #ORCH-114 ## Technical Notes YOLO mode is opt-in. Default: quality gates enabled. --- ### ORCH-124: Gate configuration per-task **Labels:** feature, configuration, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Different quality gates for different task types. ## Acceptance Criteria - [ ] Task metadata includes required gates - [ ] Gate profiles: strict (all gates), standard (tests + lint), minimal (tests only) - [ ] User selects profile on task creation - [ ] Orchestrator enforces selected gates ## Dependencies - Blocked by: #ORCH-114 ## Technical Notes Example: docs tasks need fewer gates than backend tasks. --- ## Phase 7: Testing (Days 13-14) ### ORCH-125: E2E test: Full agent lifecycle **Labels:** test, e2e, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** End-to-end test: spawn agent → git operations → quality gates → completion. ## Acceptance Criteria - [ ] E2E test spawns agent - [ ] Agent clones repo - [ ] Agent makes code change - [ ] Agent commits (quality gates pass) - [ ] Agent pushes - [ ] Agent completes - [ ] State transitions tracked - [ ] Test passes consistently ## Dependencies - Blocked by: All above ## Technical Notes Use test fixtures for repo, tasks, quality gates. --- ### ORCH-126: E2E test: Killswitch **Labels:** test, e2e, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** End-to-end test: killswitch terminates agents. ## Acceptance Criteria - [ ] E2E test spawns agent - [ ] Trigger killswitch - [ ] Agent terminated within 5 seconds - [ ] Docker container stopped - [ ] Git worktree cleaned up - [ ] State updated to 'killed' - [ ] Test passes consistently ## Dependencies - Blocked by: #ORCH-117 ## Technical Notes Test both single agent kill and kill-all. --- ### ORCH-127: E2E test: Concurrent agents **Labels:** test, e2e, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** End-to-end test: 10 concurrent agents. ## Acceptance Criteria - [ ] E2E test spawns 10 agents - [ ] All agents work on different tasks - [ ] No resource conflicts - [ ] All agents complete successfully - [ ] Test passes consistently ## Dependencies - Blocked by: #ORCH-109 ## Technical Notes Test resource limits, queue concurrency, Valkey performance. --- ### ORCH-128: Performance testing **Labels:** test, performance, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Load testing and resource monitoring. ## Acceptance Criteria - [ ] Load test: 10 concurrent agents - [ ] Monitor: CPU, memory, Valkey connections - [ ] Measure: agent spawn time, task completion time - [ ] Results documented - [ ] Performance within acceptable limits ## Dependencies - Blocked by: #ORCH-125 ## Technical Notes Acceptable limits: - Agent spawn: < 10 seconds - Task completion: < 1 hour (configurable) - CPU: < 80% - Memory: < 4GB --- ### ORCH-129: Documentation **Labels:** documentation, orchestrator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Complete orchestrator documentation. ## Acceptance Criteria - [ ] README.md with overview - [ ] API documentation (OpenAPI spec) - [ ] Architecture diagrams (spawning, lifecycle, killswitch) - [ ] Runbook (deployment, monitoring, troubleshooting) - [ ] Development guide (setup, testing, contributing) ## Dependencies - Blocked by: All above ## Technical Notes Documentation goes in `apps/orchestrator/` and root `docs/`. --- ## Phase 8: Integration (Existing Apps) ### ORCH-130: apps/api: Add orchestrator client **Labels:** feature, integration, api **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** HTTP client for orchestrator API in apps/api. ## Acceptance Criteria - [ ] `apps/api/src/orchestrator/orchestrator.client.ts` created - [ ] Methods: spawnAgent, getAgentStatus, killAgent, killAllAgents - [ ] WebSocket subscription for events - [ ] Error handling and retries ## Dependencies - Blocked by: #ORCH-102, #99 (uses this client) ## Related - Extends #99 (Dispatcher uses this client) ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 4.1 for client template. --- ### ORCH-131: apps/coordinator: Add orchestrator dispatcher **Labels:** feature, integration, coordinator **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Dispatch tasks to orchestrator after quality pre-check. ## Acceptance Criteria - [ ] `apps/coordinator/src/dispatcher/orchestrator.dispatcher.ts` created - [ ] Pre-check tasks before dispatch - [ ] Call orchestrator API to spawn agent - [ ] Handle dispatch errors - [ ] Update task state to 'dispatched' ## Dependencies - Blocked by: #ORCH-102, #99 ## Related - Coordinator already exists ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 4.2 for dispatcher template. --- ### ORCH-132: apps/web: Add agent dashboard **Labels:** feature, ui, web **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Real-time agent status dashboard in web UI. ## Acceptance Criteria - [ ] `apps/web/src/features/agents/AgentDashboard.tsx` created - [ ] Display: active agents, status, progress, uptime - [ ] Real-time updates via WebSocket - [ ] Kill button per agent - [ ] Kill All button (admin only) ## Dependencies - Blocked by: #101 (extends this), #ORCH-102 ## Related - Extends #101 (Task Progress UI) ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 4.3 for component template. --- ### ORCH-133: docker-compose: Add orchestrator service **Labels:** task, infrastructure **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Integrate orchestrator into docker-compose.yml. ## Acceptance Criteria - [ ] orchestrator service in docker-compose.yml - [ ] Depends on: valkey, coordinator - [ ] Environment variables set - [ ] Volume mounts configured - [ ] Health check configured - [ ] Port 3001 exposed ## Dependencies - Blocked by: #ORCH-103 ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 3.3 for docker-compose.yml template. --- ### ORCH-134: Update root documentation **Labels:** documentation **Milestone:** M6-AgentOrchestration (0.0.6) **Description:** Update root README and ARCHITECTURE.md with orchestrator. ## Acceptance Criteria - [ ] README.md updated with orchestrator overview - [ ] ARCHITECTURE.md updated with orchestrator layer - [ ] Architecture diagram includes orchestrator - [ ] Development guide includes orchestrator setup ## Dependencies - Blocked by: #ORCH-129 ## Technical Notes Documentation at root level explains entire monorepo architecture. --- ## Issue Creation Script Use this script to create all 34 issues at once: ```bash cd /home/localadmin/src/mosaic-stack # Create orchestrator label first tea labels create orchestrator --color "#FF6B35" --description "Orchestrator service (apps/orchestrator/)" # Then create issues (example for ORCH-101) tea issues create \ --title "[ORCH-101] Set up apps/orchestrator structure" \ --body "$(cat <<'EOF' Create the directory structure for the orchestrator service in the monorepo. ## Acceptance Criteria - [ ] Directory structure created: `apps/orchestrator/src/{api,spawner,queue,monitor,git,killswitch,coordinator,valkey}` - [ ] Test directories created: `apps/orchestrator/tests/{unit,integration}` - [ ] package.json created with dependencies - [ ] tsconfig.json extends root tsconfig.base.json - [ ] .eslintrc.js and .prettierrc configured - [ ] README.md with service overview ## Dependencies None (foundation work) ## Technical Notes See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 2 for complete structure. EOF )" \ --milestone "M6-AgentOrchestration (0.0.6)" \ --labels "task,setup,orchestrator" # Repeat for all 34 issues... ``` --- ## Summary - **34 new issues ready to create** - **All issues have templates above** - **Dependencies mapped** - **No conflicts with existing M6 issues** - **Ready for Jason's approval**