12abdfe81d
Implements FED-010: Agent Spawn via Federation feature that enables spawning and managing Claude agents on remote federated Mosaic Stack instances via COMMAND message type. Features: - Federation agent command types (spawn, status, kill) - FederationAgentService for handling agent operations - Integration with orchestrator's agent spawner/lifecycle services - API endpoints for spawning, querying status, and killing agents - Full command routing through federation COMMAND infrastructure - Comprehensive test coverage (12/12 tests passing) Architecture: - Hub → Spoke: Spawn agents on remote instances - Command flow: FederationController → FederationAgentService → CommandService → Remote Orchestrator - Response handling: Remote orchestrator returns agent status/results - Security: Connection validation, signature verification Files created: - apps/api/src/federation/types/federation-agent.types.ts - apps/api/src/federation/federation-agent.service.ts - apps/api/src/federation/federation-agent.service.spec.ts Files modified: - apps/api/src/federation/command.service.ts (agent command routing) - apps/api/src/federation/federation.controller.ts (agent endpoints) - apps/api/src/federation/federation.module.ts (service registration) - apps/orchestrator/src/api/agents/agents.controller.ts (status endpoint) - apps/orchestrator/src/api/agents/agents.module.ts (lifecycle integration) Testing: - 12/12 tests passing for FederationAgentService - All command service tests passing - TypeScript compilation successful - Linting passed Refs #93 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
219 lines
6.7 KiB
Markdown
219 lines
6.7 KiB
Markdown
# Issue #8: Docker Compose setup (turnkey)
|
|
|
|
## Objective
|
|
|
|
Create a complete turnkey Docker Compose setup that allows users to start the entire Mosaic Stack with a single command. The setup must include all necessary services with proper health checks, dependency ordering, and initialization.
|
|
|
|
## Approach
|
|
|
|
1. Create comprehensive docker-compose.yml with all services:
|
|
- PostgreSQL 17 + pgvector extension
|
|
- Valkey (Redis-compatible cache)
|
|
- Authentik (OIDC provider)
|
|
- Ollama (optional AI service)
|
|
- mosaic-api (NestJS backend)
|
|
- mosaic-web (Next.js frontend)
|
|
- Traefik reverse proxy (optional, for dev environments)
|
|
|
|
2. Implement proper service orchestration:
|
|
- Health checks for all services
|
|
- Dependency ordering (depends_on with conditions)
|
|
- Volume persistence
|
|
- Network isolation and connectivity
|
|
- Environment variable management
|
|
|
|
3. Create initialization scripts:
|
|
- PostgreSQL init scripts for pgvector extension
|
|
- Database seeding
|
|
- Authentik bootstrap configuration
|
|
|
|
4. Write integration tests:
|
|
- Test service startup order
|
|
- Verify health checks
|
|
- Test service connectivity
|
|
- Validate database initialization
|
|
|
|
5. Create comprehensive documentation:
|
|
- .env.example with all required variables
|
|
- SETUP.md - installation guide
|
|
- DOCKER.md - deployment details
|
|
- CONFIGURATION.md - configuration options
|
|
|
|
## Progress
|
|
|
|
- [x] Create scratchpad (this file)
|
|
- [x] Examine current project structure
|
|
- [x] Design docker-compose.yml structure
|
|
- [x] Create PostgreSQL init scripts (already existed)
|
|
- [x] Implement docker-compose.yml with all services
|
|
- [x] Create .env.example with comprehensive variables
|
|
- [x] Create Dockerfiles for apps (api and web)
|
|
- [x] Create .dockerignore files
|
|
- [x] Write integration tests for Docker stack
|
|
- [x] Update 3-docker-setup.md with new service profiles
|
|
- [x] Create comprehensive Docker deployment guide
|
|
- [x] Update CONFIGURATION.md with Docker variables
|
|
- [x] Create docker-compose.override.yml.example template
|
|
- [x] Update root README.md with Docker instructions
|
|
- [x] Add test scripts to package.json
|
|
- [x] Create smoke test script
|
|
- [x] Create Makefile for common operations
|
|
- [x] Create CHANGELOG.md entry
|
|
- [x] Complete implementation documentation
|
|
|
|
## COMPLETION STATUS: READY FOR TESTING
|
|
|
|
All implementation work is complete. The Docker Compose setup is:
|
|
|
|
- ✓ Fully documented
|
|
- ✓ Comprehensively configured
|
|
- ✓ Test scripts ready
|
|
- ✓ Production-ready with security considerations
|
|
|
|
Next steps for deployment testing:
|
|
|
|
1. Run smoke test: `./scripts/test-docker-deployment.sh`
|
|
2. Run integration tests: `pnpm test:docker`
|
|
3. Manual validation of all service profiles
|
|
4. Performance testing under load
|
|
5. Security audit of default configurations
|
|
|
|
## Testing
|
|
|
|
- Integration tests for Docker stack startup
|
|
- Health check validation
|
|
- Service connectivity tests
|
|
- Database initialization verification
|
|
- End-to-end deployment test
|
|
|
|
### Testing Commands
|
|
|
|
```bash
|
|
# Run integration tests
|
|
pnpm test:docker
|
|
|
|
# Run smoke test (manual deployment validation)
|
|
./scripts/test-docker-deployment.sh
|
|
|
|
# Quick deployment test
|
|
docker compose up -d
|
|
docker compose ps
|
|
curl http://localhost:3001/health
|
|
curl http://localhost:3000
|
|
docker compose down -v
|
|
```
|
|
|
|
### Manual Testing Checklist
|
|
|
|
- [x] docker-compose.yml syntax validation
|
|
- [x] All services defined with proper configuration
|
|
- [x] Health checks on all services
|
|
- [x] Network configuration (internal and public)
|
|
- [x] Volume configuration
|
|
- [x] Environment variable handling
|
|
- [ ] Actual deployment test (requires Docker runtime)
|
|
- [ ] Service startup order verification
|
|
- [ ] Health endpoint accessibility
|
|
- [ ] Inter-service connectivity
|
|
- [ ] Profile-based service activation
|
|
|
|
Note: Full deployment testing requires Docker environment.
|
|
The implementation is complete and ready for testing.
|
|
|
|
## Notes
|
|
|
|
- Must be truly turnkey - one command starts everything
|
|
- Support both bundled and external service configurations
|
|
- Follow project design principles (PDA-friendly)
|
|
- Ensure proper security defaults
|
|
- Include development and production configurations
|
|
|
|
## Implementation Summary
|
|
|
|
### Files Created
|
|
|
|
1. **Docker Compose Files:**
|
|
- `/docker-compose.yml` - Main compose file with all services
|
|
- `/docker-compose.override.yml.example` - Template for customization
|
|
- PostgreSQL Dockerfile and init scripts (already existed)
|
|
|
|
2. **Application Dockerfiles:**
|
|
- `/apps/api/Dockerfile` - Multi-stage build for NestJS API
|
|
- `/apps/api/.dockerignore` - Ignore unnecessary files
|
|
- `/apps/web/Dockerfile` - Multi-stage build for Next.js web
|
|
- `/apps/web/.dockerignore` - Ignore unnecessary files
|
|
|
|
3. **Configuration:**
|
|
- `/.env.example` - Updated with all Docker variables
|
|
- `/docs/1-getting-started/3-configuration/3-docker.md` - Docker configuration guide
|
|
- Updated `/docs/1-getting-started/3-configuration/1-environment.md`
|
|
|
|
4. **Documentation:**
|
|
- `/docs/1-getting-started/4-docker-deployment/README.md` - Comprehensive deployment guide
|
|
- Updated `/docs/1-getting-started/2-installation/3-docker-setup.md`
|
|
- Updated `/README.md` with Docker quick start
|
|
|
|
5. **Testing:**
|
|
- `/tests/integration/docker-stack.test.ts` - Integration tests
|
|
- `/tests/integration/vitest.config.ts` - Test configuration
|
|
- `/scripts/test-docker-deployment.sh` - Smoke test script
|
|
- Updated `/package.json` with Docker test scripts
|
|
|
|
### Services Implemented
|
|
|
|
**Core Services (Always Active):**
|
|
|
|
- PostgreSQL 17 with pgvector
|
|
- Valkey (Redis-compatible cache)
|
|
- Mosaic API (NestJS)
|
|
- Mosaic Web (Next.js)
|
|
|
|
**Optional Services (Profiles):**
|
|
|
|
- Authentik OIDC stack (profile: authentik)
|
|
- Authentik PostgreSQL
|
|
- Authentik Redis
|
|
- Authentik Server
|
|
- Authentik Worker
|
|
- Ollama AI (profile: ollama)
|
|
|
|
### Key Features
|
|
|
|
1. **Health Checks:** All services have proper health checks
|
|
2. **Dependency Ordering:** Services start in correct order
|
|
3. **Network Isolation:** Internal and public networks
|
|
4. **Volume Persistence:** Named volumes for all data
|
|
5. **Security:** Non-root users, proper secrets handling
|
|
6. **Multi-stage Builds:** Optimized Docker images
|
|
7. **Service Profiles:** Optional services via profiles
|
|
8. **Customization:** Override template for custom configs
|
|
|
|
### Environment Variables
|
|
|
|
Comprehensive `.env.example` includes:
|
|
|
|
- Application ports (API, Web)
|
|
- PostgreSQL configuration
|
|
- Valkey configuration
|
|
- Authentik OIDC settings
|
|
- Ollama AI settings
|
|
- JWT configuration
|
|
- Logging and debugging
|
|
|
|
### Testing Strategy
|
|
|
|
1. Integration tests for Docker stack
|
|
2. Health check validation
|
|
3. Service connectivity tests
|
|
4. Volume and network verification
|
|
5. Smoke test script for quick validation
|
|
|
|
### Documentation Coverage
|
|
|
|
- Quick start guide
|
|
- Complete deployment guide
|
|
- Configuration reference
|
|
- Troubleshooting guide
|
|
- Production considerations
|
|
- Backup and restore procedures
|