stack/docs/scratchpads/orch-103-docker.md

Issue ORCH-103: Docker Compose integration for orchestrator

Objective

Add orchestrator service to docker-compose.yml files with proper dependencies, environment variables, volume mounts, health check, and port exposure.

Current State Analysis

Existing Docker Compose Files

1. Root docker-compose.yml - Main production compose file

   • Already has orchestrator service configured (lines 353-397)
   • Dependencies: valkey, api (NOT coordinator)
   • Port: 3002:3001 (external:internal)
   • Volumes: docker.sock, orchestrator_workspace
   • Health check: configured
   • Network: mosaic-internal

2. docker/docker-compose.yml - Development compose file

   • Has coordinator service (lines 42-69)
   • No orchestrator service yet
   • Uses mosaic-network

ORCH-103 Acceptance Criteria Review

From docs/M6-NEW-ISSUES-TEMPLATES.md:

• orchestrator service added to docker-compose.yml (EXISTS in root)
• Depends on: valkey, coordinator (root has valkey, api instead)
• Environment variables configured (VALKEY_URL, COORDINATOR_URL, CLAUDE_API_KEY)
  • Missing COORDINATOR_URL in root
• Volume mounts: /var/run/docker.sock (Docker-in-Docker), /workspace (git operations)
• Health check configured
• Port 3001 exposed (externally as 3002)

Issues Identified

1. Root docker-compose.yml

• Missing dependency: Should depend on coordinator, not api
• Missing env var: COORDINATOR_URL not set
• Wrong dependency: Currently depends on api, should be coordinator

2. docker/docker-compose.yml

• Missing service: No orchestrator service at all
• Needs to be added following the same pattern as root

Implementation Plan

Task 1: Fix Root docker-compose.yml

1. Change dependencies from api to coordinator
2. Add COORDINATOR_URL environment variable
3. Verify all other requirements match

Task 2: Add Orchestrator to docker/docker-compose.yml

1. Add orchestrator service configuration
2. Set dependencies: valkey, coordinator
3. Configure environment variables
4. Mount volumes (docker.sock, workspace)
5. Add health check
6. Expose port 3001

Notes

Coordinator Service Discovery

• Root compose: No coordinator service (coordinator runs separately)
• docker/ compose: Has coordinator service on port 8000
• Need to handle both scenarios

Port Mapping

• Root: 3002:3001 (avoid conflict with API on 3001)
• docker/: Can use 3001:3001 (isolated environment)

Network Isolation

• Root: Uses mosaic-internal (isolated from public)
• docker/: Uses mosaic-network (single network)

Testing Plan

1. Validate docker-compose.yml syntax
2. Check for port conflicts
3. Verify environment variables reference correct services
4. Ensure dependencies exist in same compose file

Implementation Complete

Changes Made

1. Root docker-compose.yml (/home/localadmin/src/mosaic-stack/docker-compose.yml)

• Added coordinator service before orchestrator (lines 353-387)
  • Build context: ./apps/coordinator
  • Port: 8000
  • Dependencies: valkey
  • Environment: GITEA integration, VALKEY_URL
  • Health check: Python urllib check on /health endpoint
  • Network: mosaic-internal
• Updated orchestrator service (lines 389-440)
  • Changed dependency from api to coordinator
  • Added COORDINATOR_URL environment variable: http://coordinator:8000
  • All other requirements already met

2. docker/docker-compose.yml (/home/localadmin/src/mosaic-stack/docker/docker-compose.yml)

• Updated coordinator service (lines 42-69)
  • Added VALKEY_URL environment variable
  • Added dependency on valkey service
• Added orchestrator service (lines 71-112)
  • Build context: .. (parent directory)
  • Dockerfile: ./apps/orchestrator/Dockerfile
  • Port: 3001:3001
  • Dependencies: valkey, coordinator
  • Environment variables:
    • ORCHESTRATOR_PORT: 3001
    • VALKEY_URL: redis://valkey:6379
    • COORDINATOR_URL: http://coordinator:8000
    • CLAUDE_API_KEY: ${CLAUDE_API_KEY}
    • DOCKER_SOCKET: /var/run/docker.sock
    • GIT_USER_NAME, GIT_USER_EMAIL
    • KILLSWITCH_ENABLED, SANDBOX_ENABLED
  • Volume mounts:
    • /var/run/docker.sock:/var/run/docker.sock (Docker-in-Docker)
    • orchestrator_workspace:/workspace (git operations)
  • Health check: wget check on http://localhost:3001/health
  • Network: mosaic-network
• Added orchestrator_workspace volume (line 78)

3. .env.example

• Added COORDINATOR_PORT=8000 configuration (lines 148-151)

Validation Results

• Root docker-compose.yml: PASSED (syntax valid)
• docker/docker-compose.yml: PASSED (syntax valid)
• Both files show expected warnings for unset environment variables (normal)

Acceptance Criteria Status

• orchestrator service added to docker-compose.yml (BOTH files)
• Depends on: valkey, coordinator (BOTH files)
• Environment variables configured (VALKEY_URL, COORDINATOR_URL, CLAUDE_API_KEY)
• Volume mounts: /var/run/docker.sock (Docker-in-Docker), /workspace (git operations)
• Health check configured
• Port 3001 exposed (3002:3001 in root, 3001:3001 in docker/)

Additional Improvements

1. Added coordinator service to root docker-compose.yml (was missing)
2. Documented coordinator in both compose files
3. Added COORDINATOR_PORT to .env.example for consistency
4. Ensured coordinator dependency on valkey in both files

Port Mappings Summary

• Root docker-compose.yml (production):
  • API: 3001 (internal)
  • Coordinator: 8000:8000
  • Orchestrator: 3002:3001 (avoids conflict with API)
• docker/docker-compose.yml (development):
  • Coordinator: 8000:8000
  • Orchestrator: 3001:3001 (isolated environment)

Network Configuration

• Root: mosaic-internal (isolated)
• Docker: mosaic-network (single network for dev)

All requirements for ORCH-103 have been successfully implemented.

Final Verification

Syntax Validation

Both docker-compose files pass syntax validation:

docker compose -f /home/localadmin/src/mosaic-stack/docker-compose.yml config --quiet
docker compose -f /home/localadmin/src/mosaic-stack/docker/docker-compose.yml config --quiet

Result: PASSED (warnings for unset env vars are expected)

Port Conflict Check

Root docker-compose.yml published ports:

• 3000: web
• 3001: api
• 3002: orchestrator (internal 3001)
• 5432: postgres
• 6379: valkey
• 8000: coordinator
• 9000/9443: authentik

Docker/docker-compose.yml published ports:

• 3001: orchestrator
• 5432: postgres
• 6379: valkey
• 8000: coordinator

Result: NO CONFLICTS

Service Dependency Graph

Root docker-compose.yml:
  orchestrator → coordinator → valkey
  orchestrator → valkey

Docker/docker-compose.yml:
  orchestrator → coordinator → valkey
  orchestrator → valkey

Environment Variables Documented

All orchestrator environment variables are documented in .env.example:

• COORDINATOR_PORT=8000 (NEW)
• ORCHESTRATOR_PORT=3001
• CLAUDE_API_KEY
• GIT_USER_NAME
• GIT_USER_EMAIL
• KILLSWITCH_ENABLED
• SANDBOX_ENABLED

Files Modified

1. /home/localadmin/src/mosaic-stack/docker-compose.yml

   • Added coordinator service (38 lines)
   • Updated orchestrator service (2 lines: dependency + env var)

2. /home/localadmin/src/mosaic-stack/docker/docker-compose.yml

   • Updated coordinator service (2 lines: dependency + env var)
   • Added orchestrator service (42 lines)
   • Added volume definition (3 lines)

3. /home/localadmin/src/mosaic-stack/.env.example

   • Added COORDINATOR_PORT section (5 lines)

Ready for Testing

The configuration is syntactically valid and ready for:

1. Building the orchestrator Docker image
2. Starting services with docker-compose up
3. Testing orchestrator health endpoint
4. Testing coordinator integration

Next steps (when ready):

# Build and start services
docker compose up -d coordinator orchestrator

# Check health
curl http://localhost:8000/health  # coordinator
curl http://localhost:3002/health  # orchestrator (root)
# or
curl http://localhost:3001/health  # orchestrator (docker/)

# View logs
docker compose logs -f orchestrator
docker compose logs -f coordinator