Implements FED-010: Agent Spawn via Federation feature that enables spawning and managing Claude agents on remote federated Mosaic Stack instances via COMMAND message type. Features: - Federation agent command types (spawn, status, kill) - FederationAgentService for handling agent operations - Integration with orchestrator's agent spawner/lifecycle services - API endpoints for spawning, querying status, and killing agents - Full command routing through federation COMMAND infrastructure - Comprehensive test coverage (12/12 tests passing) Architecture: - Hub → Spoke: Spawn agents on remote instances - Command flow: FederationController → FederationAgentService → CommandService → Remote Orchestrator - Response handling: Remote orchestrator returns agent status/results - Security: Connection validation, signature verification Files created: - apps/api/src/federation/types/federation-agent.types.ts - apps/api/src/federation/federation-agent.service.ts - apps/api/src/federation/federation-agent.service.spec.ts Files modified: - apps/api/src/federation/command.service.ts (agent command routing) - apps/api/src/federation/federation.controller.ts (agent endpoints) - apps/api/src/federation/federation.module.ts (service registration) - apps/orchestrator/src/api/agents/agents.controller.ts (status endpoint) - apps/orchestrator/src/api/agents/agents.module.ts (lifecycle integration) Testing: - 12/12 tests passing for FederationAgentService - All command service tests passing - TypeScript compilation successful - Linting passed Refs #93 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
23 KiB
M6 New Orchestrator Issues - Ready to Create
Total: 34 new issues for apps/orchestrator/ implementation
Milestone: M6-AgentOrchestration (0.0.6)
Labels: orchestrator (create this label first)
Label Creation Command
cd /home/localadmin/src/mosaic-stack
tea labels create orchestrator --color "#FF6B35" --description "Orchestrator service (apps/orchestrator/)"
Phase 1: Foundation (Days 1-2)
ORCH-101: Set up apps/orchestrator structure
Labels: task, setup, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Create the directory structure for the orchestrator service in the monorepo.
Acceptance Criteria
- Directory structure created:
apps/orchestrator/src/{api,spawner,queue,monitor,git,killswitch,coordinator,valkey} - Test directories created:
apps/orchestrator/tests/{unit,integration} - package.json created with dependencies (@mosaic/shared, @mosaic/config, ioredis, bullmq, @anthropic-ai/sdk, dockerode, simple-git, fastify, zod)
- tsconfig.json extends root tsconfig.base.json
- .eslintrc.js and .prettierrc configured
- README.md with service overview
Dependencies
None (foundation work)
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 2 for complete structure.
ORCH-102: Create Fastify server with health checks
Labels: feature, api, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Basic HTTP server for orchestrator API with health check endpoint.
Acceptance Criteria
- Fastify server in
src/api/server.ts - Health check endpoint: GET /health (returns 200 OK)
- Configuration loaded from environment variables
- Pino logger integrated
- Server starts on port 3001 (configurable)
- Graceful shutdown handler
Dependencies
- Blocked by: #ORCH-101
Technical Notes
GET /health
Response 200 OK:
{
"status": "healthy",
"uptime": 12345,
"timestamp": "2026-02-02T10:00:00Z"
}
ORCH-103: Docker Compose integration for orchestrator
Labels: task, infrastructure, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Add orchestrator service to docker-compose.yml.
Acceptance Criteria
- orchestrator service added to docker-compose.yml
- Depends on: valkey, coordinator
- Environment variables configured (VALKEY_URL, COORDINATOR_URL, CLAUDE_API_KEY)
- Volume mounts: /var/run/docker.sock (for Docker-in-Docker), /workspace (git operations)
- Health check configured
- Port 3001 exposed
Dependencies
- Blocked by: #ORCH-101
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 3.3 for docker-compose.yml template.
ORCH-104: Monorepo build pipeline for orchestrator
Labels: task, infrastructure, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Update TurboRepo configuration to include orchestrator in build pipeline.
Acceptance Criteria
- turbo.json updated with orchestrator tasks
- Build order: packages/* → coordinator → orchestrator → api → web
- Root package.json scripts updated (dev:orchestrator, docker:logs)
npm run buildbuilds orchestratornpm run devruns orchestrator in watch mode
Dependencies
- Blocked by: #ORCH-101
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 3.2 for turbo.json configuration.
Phase 2: Agent Spawning (Days 3-4)
ORCH-105: Implement agent spawner (Claude SDK)
Labels: feature, core, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Spawn Claude agents using Anthropic SDK.
Acceptance Criteria
src/spawner/agent-spawner.tsimplemented- Spawn agent with task context (repo, branch, instructions)
- Claude SDK integration (@anthropic-ai/sdk)
- Agent session management
- Return agentId on successful spawn
Dependencies
- Blocked by: #ORCH-102
Technical Notes
interface SpawnAgentRequest {
taskId: string;
agentType: "worker" | "reviewer" | "tester";
context: {
repository: string;
branch: string;
workItems: string[];
skills?: string[];
};
options?: {
sandbox?: boolean;
timeout?: number;
maxRetries?: number;
};
}
ORCH-106: Docker sandbox isolation
Labels: feature, security, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Isolate agents in Docker containers for security.
Acceptance Criteria
src/spawner/docker-sandbox.tsimplemented- dockerode integration for container management
- Agent runs in isolated container
- Resource limits enforced (CPU, memory)
- Non-root user in container
- Container cleanup on agent termination
Dependencies
- Blocked by: #ORCH-105
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 7 for Docker security hardening.
ORCH-107: Valkey client and state management
Labels: feature, core, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Valkey client for orchestrator state management.
Acceptance Criteria
src/valkey/client.tswith ioredis connection- State schema implemented (tasks, agents, queue)
- Pub/sub for events (agent spawned, completed, failed)
- Task state: pending, assigned, executing, completed, failed
- Agent state: spawning, running, completed, failed, killed
Dependencies
- Blocked by: #98 (Valkey Integration), #ORCH-102
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 5 for Valkey state schema.
ORCH-108: BullMQ task queue
Labels: feature, core, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Task queue with priority and retry logic using BullMQ.
Acceptance Criteria
src/queue/task-queue.tsimplemented- BullMQ queue on Valkey
- Priority-based task ordering
- Retry logic with exponential backoff
- Queue worker processes tasks
- Queue monitoring (pending, active, completed, failed counts)
Dependencies
- Blocked by: #ORCH-107
Technical Notes
interface QueuedTask {
taskId: string;
priority: number; // 1-10
retries: number;
maxRetries: number;
context: TaskContext;
}
ORCH-109: Agent lifecycle management
Labels: feature, core, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Manage agent states through lifecycle (spawning → running → completed/failed).
Acceptance Criteria
src/spawner/agent-lifecycle.tsimplemented- State transitions: spawning → running → completed/failed/killed
- State persisted in Valkey
- Events emitted on state changes (pub/sub)
- Agent metadata tracked (startedAt, completedAt, error)
Dependencies
- Blocked by: #ORCH-105, #ORCH-108
Technical Notes
State machine enforces valid transitions only.
Phase 3: Git Integration (Days 5-6)
ORCH-110: Git operations (clone, commit, push)
Labels: feature, git, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Implement git operations using simple-git.
Acceptance Criteria
src/git/git-operations.tsimplemented- Clone repository
- Create branch
- Commit changes with message
- Push to remote
- Git config (user.name, user.email)
Dependencies
- Blocked by: #ORCH-105
Technical Notes
Use simple-git library. Configure git user from environment variables.
ORCH-111: Git worktree management
Labels: feature, git, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Create and manage git worktrees for agent isolation.
Acceptance Criteria
src/git/worktree-manager.tsimplemented- Create worktree for each agent
- Worktree naming:
agent-{agentId}-{taskId} - Cleanup worktree on agent completion
- Handle worktree conflicts
Dependencies
- Blocked by: #ORCH-110
Technical Notes
Git worktrees allow multiple agents to work on same repo without conflicts.
ORCH-112: Conflict detection
Labels: feature, git, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Detect merge conflicts before pushing.
Acceptance Criteria
src/git/conflict-detection.tsimplemented- Fetch latest from remote before push
- Detect merge conflicts
- Return conflict details to agent
- Agent retries with rebase/merge
Dependencies
- Blocked by: #ORCH-110
Technical Notes
Check for conflicts before push. If conflicts, agent must resolve.
Phase 4: Coordinator Integration (Days 7-8)
ORCH-113: Coordinator API client
Labels: feature, integration, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
HTTP client for calling coordinator quality gates.
Acceptance Criteria
src/coordinator/coordinator-client.tsimplemented- POST /api/quality/check endpoint
- Quality check request serialization
- Response parsing (approved/rejected)
- Retry on coordinator unavailable
Dependencies
- Blocked by: #ORCH-102
Related
- Coordinator exists at
apps/coordinator/
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 6.1 for API contract.
ORCH-114: Quality gate callbacks
Labels: feature, quality, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Call coordinator quality gates before commit/push.
Acceptance Criteria
src/coordinator/quality-gates.tsimplemented- Pre-commit quality check (before git commit)
- Post-commit quality check (before git push)
- Parse quality gate response
- Block commit/push if rejected
- Return rejection details to agent
Dependencies
- Blocked by: #ORCH-113
Technical Notes
Coordinator runs: typecheck, lint, tests, coverage. Orchestrator calls coordinator.
ORCH-115: Task dispatch from coordinator
Labels: feature, integration, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Coordinator dispatches validated tasks to orchestrator.
Acceptance Criteria
- Orchestrator API endpoint: POST /agents/spawn
- Coordinator calls orchestrator after quality pre-check
- Task queued in Valkey
- Agent spawned
- Return agentId to coordinator
Dependencies
- Blocked by: #99 (Task Dispatcher), #ORCH-113
Related
- Extends #99 (Dispatcher in control plane)
Technical Notes
Flow: User → Mosaic Stack → Coordinator (pre-check) → Orchestrator (dispatch).
ORCH-116: 50% rule enforcement
Labels: feature, quality, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Enforce 50% rule: no more than 50% AI-generated code in PR.
Acceptance Criteria
- Mechanical gates: typecheck, lint, tests, coverage (coordinator)
- AI confirmation: independent AI agent reviews (coordinator)
- Orchestrator calls both mechanical and AI gates
- Reject if either fails
- Return detailed failure reasons
Dependencies
- Blocked by: #ORCH-114
Technical Notes
Coordinator enforces 50% rule. Orchestrator calls coordinator.
Phase 5: Killswitch + Security (Days 9-10)
ORCH-117: Killswitch implementation
Labels: feature, security, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Emergency stop: kill single agent or all agents.
Acceptance Criteria
src/killswitch/killswitch.tsimplemented- POST /agents/{agentId}/kill endpoint
- POST /agents/kill-all endpoint
- Immediate termination (SIGKILL)
- Cleanup Docker containers
- Cleanup git worktrees
- Update agent state to 'killed'
- Audit trail logged
Dependencies
- Blocked by: #ORCH-109
Related
- #114 (Kill Authority in control plane)
Technical Notes
Killswitch bypasses all queues. Must respond within seconds.
ORCH-118: Resource cleanup
Labels: task, infrastructure, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Clean up resources when agent terminates.
Acceptance Criteria
src/killswitch/cleanup.tsimplemented- Stop Docker container
- Remove Docker container
- Remove git worktree
- Clear Valkey state
- Emit cleanup event
Dependencies
- Blocked by: #ORCH-117
Technical Notes
Run cleanup on: agent completion, agent failure, killswitch.
ORCH-119: Docker security hardening
Labels: security, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Harden Docker container security for agents.
Acceptance Criteria
- Dockerfile with multi-stage build
- Non-root user (nodejs:nodejs)
- Minimal base image (node:20-alpine)
- No unnecessary packages
- Health check in Dockerfile
- Security scan passes (docker scan)
Dependencies
- Blocked by: #ORCH-106
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 7 for Dockerfile template.
ORCH-120: Secret scanning
Labels: security, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Prevent secrets from being committed.
Acceptance Criteria
- git-secrets integrated
- Pre-commit hook scans for secrets
- Block commit if secrets detected
- Scan for API keys, tokens, passwords
- Custom patterns for Claude API keys
Dependencies
- Blocked by: #ORCH-110
Technical Notes
git secrets --add 'sk-[a-zA-Z0-9]{48}' # Claude API keys
Phase 6: Quality Gates (Days 11-12)
ORCH-121: Mechanical quality gates
Labels: feature, quality, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Implement mechanical quality gates (non-AI).
Acceptance Criteria
- TypeScript type checking
- ESLint linting
- Test execution (vitest)
- Coverage check (>= 85%)
- Build check (tsup)
Dependencies
- Blocked by: #ORCH-114
Related
- Coordinator has gate implementations
Technical Notes
Mechanical gates are deterministic (no AI). Run via coordinator.
ORCH-122: AI agent confirmation
Labels: feature, quality, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Independent AI agent reviews changes for quality.
Acceptance Criteria
- Spawn independent AI reviewer agent
- Review code changes
- Check for: logic errors, security issues, best practices
- Return confidence score (0.0 - 1.0)
- Approve if confidence >= 0.9
Dependencies
- Blocked by: #ORCH-114
Related
- Coordinator calls AI reviewer
Technical Notes
AI reviewer is INDEPENDENT of worker agent (no self-review).
ORCH-123: YOLO mode (gate bypass)
Labels: feature, configuration, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
User-configurable approval gates (YOLO mode bypasses gates).
Acceptance Criteria
- Configuration option:
YOLO_MODE=true - If YOLO mode enabled, skip quality gates
- Log YOLO mode usage (audit trail)
- UI warning: "Quality gates disabled"
Dependencies
- Blocked by: #ORCH-114
Technical Notes
YOLO mode is opt-in. Default: quality gates enabled.
ORCH-124: Gate configuration per-task
Labels: feature, configuration, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Different quality gates for different task types.
Acceptance Criteria
- Task metadata includes required gates
- Gate profiles: strict (all gates), standard (tests + lint), minimal (tests only)
- User selects profile on task creation
- Orchestrator enforces selected gates
Dependencies
- Blocked by: #ORCH-114
Technical Notes
Example: docs tasks need fewer gates than backend tasks.
Phase 7: Testing (Days 13-14)
ORCH-125: E2E test: Full agent lifecycle
Labels: test, e2e, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
End-to-end test: spawn agent → git operations → quality gates → completion.
Acceptance Criteria
- E2E test spawns agent
- Agent clones repo
- Agent makes code change
- Agent commits (quality gates pass)
- Agent pushes
- Agent completes
- State transitions tracked
- Test passes consistently
Dependencies
- Blocked by: All above
Technical Notes
Use test fixtures for repo, tasks, quality gates.
ORCH-126: E2E test: Killswitch
Labels: test, e2e, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
End-to-end test: killswitch terminates agents.
Acceptance Criteria
- E2E test spawns agent
- Trigger killswitch
- Agent terminated within 5 seconds
- Docker container stopped
- Git worktree cleaned up
- State updated to 'killed'
- Test passes consistently
Dependencies
- Blocked by: #ORCH-117
Technical Notes
Test both single agent kill and kill-all.
ORCH-127: E2E test: Concurrent agents
Labels: test, e2e, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
End-to-end test: 10 concurrent agents.
Acceptance Criteria
- E2E test spawns 10 agents
- All agents work on different tasks
- No resource conflicts
- All agents complete successfully
- Test passes consistently
Dependencies
- Blocked by: #ORCH-109
Technical Notes
Test resource limits, queue concurrency, Valkey performance.
ORCH-128: Performance testing
Labels: test, performance, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Load testing and resource monitoring.
Acceptance Criteria
- Load test: 10 concurrent agents
- Monitor: CPU, memory, Valkey connections
- Measure: agent spawn time, task completion time
- Results documented
- Performance within acceptable limits
Dependencies
- Blocked by: #ORCH-125
Technical Notes
Acceptable limits:
- Agent spawn: < 10 seconds
- Task completion: < 1 hour (configurable)
- CPU: < 80%
- Memory: < 4GB
ORCH-129: Documentation
Labels: documentation, orchestrator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Complete orchestrator documentation.
Acceptance Criteria
- README.md with overview
- API documentation (OpenAPI spec)
- Architecture diagrams (spawning, lifecycle, killswitch)
- Runbook (deployment, monitoring, troubleshooting)
- Development guide (setup, testing, contributing)
Dependencies
- Blocked by: All above
Technical Notes
Documentation goes in apps/orchestrator/ and root docs/.
Phase 8: Integration (Existing Apps)
ORCH-130: apps/api: Add orchestrator client
Labels: feature, integration, api Milestone: M6-AgentOrchestration (0.0.6)
Description:
HTTP client for orchestrator API in apps/api.
Acceptance Criteria
apps/api/src/orchestrator/orchestrator.client.tscreated- Methods: spawnAgent, getAgentStatus, killAgent, killAllAgents
- WebSocket subscription for events
- Error handling and retries
Dependencies
- Blocked by: #ORCH-102, #99 (uses this client)
Related
- Extends #99 (Dispatcher uses this client)
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 4.1 for client template.
ORCH-131: apps/coordinator: Add orchestrator dispatcher
Labels: feature, integration, coordinator Milestone: M6-AgentOrchestration (0.0.6)
Description:
Dispatch tasks to orchestrator after quality pre-check.
Acceptance Criteria
apps/coordinator/src/dispatcher/orchestrator.dispatcher.tscreated- Pre-check tasks before dispatch
- Call orchestrator API to spawn agent
- Handle dispatch errors
- Update task state to 'dispatched'
Dependencies
- Blocked by: #ORCH-102, #99
Related
- Coordinator already exists
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 4.2 for dispatcher template.
ORCH-132: apps/web: Add agent dashboard
Labels: feature, ui, web Milestone: M6-AgentOrchestration (0.0.6)
Description:
Real-time agent status dashboard in web UI.
Acceptance Criteria
apps/web/src/features/agents/AgentDashboard.tsxcreated- Display: active agents, status, progress, uptime
- Real-time updates via WebSocket
- Kill button per agent
- Kill All button (admin only)
Dependencies
- Blocked by: #101 (extends this), #ORCH-102
Related
- Extends #101 (Task Progress UI)
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 4.3 for component template.
ORCH-133: docker-compose: Add orchestrator service
Labels: task, infrastructure Milestone: M6-AgentOrchestration (0.0.6)
Description:
Integrate orchestrator into docker-compose.yml.
Acceptance Criteria
- orchestrator service in docker-compose.yml
- Depends on: valkey, coordinator
- Environment variables set
- Volume mounts configured
- Health check configured
- Port 3001 exposed
Dependencies
- Blocked by: #ORCH-103
Technical Notes
See ORCHESTRATOR-MONOREPO-SETUP.md Section 3.3 for docker-compose.yml template.
ORCH-134: Update root documentation
Labels: documentation Milestone: M6-AgentOrchestration (0.0.6)
Description:
Update root README and ARCHITECTURE.md with orchestrator.
Acceptance Criteria
- README.md updated with orchestrator overview
- ARCHITECTURE.md updated with orchestrator layer
- Architecture diagram includes orchestrator
- Development guide includes orchestrator setup
Dependencies
- Blocked by: #ORCH-129
Technical Notes
Documentation at root level explains entire monorepo architecture.
Issue Creation Script
Use this script to create all 34 issues at once:
cd /home/localadmin/src/mosaic-stack
# Create orchestrator label first
tea labels create orchestrator --color "#FF6B35" --description "Orchestrator service (apps/orchestrator/)"
# Then create issues (example for ORCH-101)
tea issues create \
--title "[ORCH-101] Set up apps/orchestrator structure" \
--body "$(cat <<'EOF'
Create the directory structure for the orchestrator service in the monorepo.
## Acceptance Criteria
- [ ] Directory structure created: `apps/orchestrator/src/{api,spawner,queue,monitor,git,killswitch,coordinator,valkey}`
- [ ] Test directories created: `apps/orchestrator/tests/{unit,integration}`
- [ ] package.json created with dependencies
- [ ] tsconfig.json extends root tsconfig.base.json
- [ ] .eslintrc.js and .prettierrc configured
- [ ] README.md with service overview
## Dependencies
None (foundation work)
## Technical Notes
See `ORCHESTRATOR-MONOREPO-SETUP.md` Section 2 for complete structure.
EOF
)" \
--milestone "M6-AgentOrchestration (0.0.6)" \
--labels "task,setup,orchestrator"
# Repeat for all 34 issues...
Summary
- 34 new issues ready to create
- All issues have templates above
- Dependencies mapped
- No conflicts with existing M6 issues
- Ready for Jason's approval