Jason Woltje
3cdcbf6774
- Create comprehensive E2E test suite for job orchestration - Add test fixtures for Discord, BullMQ, and Prisma mocks - Implement 9 end-to-end test scenarios covering: * Happy path: webhook → job → step execution → completion * Event emission throughout job lifecycle * Step failure and retry handling * Job failure after max retries * Discord command parsing and job creation * WebSocket status updates integration * Job cancellation workflow * Job retry mechanism * Progress percentage tracking - Add helper methods to services for simplified testing: * JobStepsService: start(), complete(), fail(), findByJob() * RunnerJobsService: updateStatus(), updateProgress() * JobEventsService: findByJob() - Configure vitest.e2e.config.ts for E2E test execution - All 9 E2E tests passing - All 1405 unit tests passing - Quality gates: typecheck, lint, build all passing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
111 lines
3.5 KiB
Markdown
111 lines
3.5 KiB
Markdown
# Issue #175: E2E Test Harness
|
|
|
|
## Objective
|
|
|
|
Create a comprehensive end-to-end test harness that validates the complete flow from webhook to job completion, including chat integration.
|
|
|
|
## Approach
|
|
|
|
1. Explore existing test patterns in the codebase
|
|
2. Set up E2E test directory structure
|
|
3. Create test fixtures (Mock Discord, BullMQ, Prisma)
|
|
4. Implement E2E test scenarios following TDD
|
|
5. Verify all quality gates pass
|
|
|
|
## Progress
|
|
|
|
- [x] Create scratchpad
|
|
- [x] Pull latest code (skipped - unstaged changes)
|
|
- [x] Explore existing test patterns
|
|
- [x] Create E2E directory structure
|
|
- [x] Create vitest.e2e.config.ts
|
|
- [x] Implement test fixtures
|
|
- [x] Mock Discord client fixture
|
|
- [x] Mock BullMQ queues fixture
|
|
- [x] Mock Prisma client fixture
|
|
- [x] Write E2E tests (TDD)
|
|
- [x] Happy path: webhook → job → completion
|
|
- [x] Error handling: step failure → retry
|
|
- [x] Chat integration: command → job → updates
|
|
- [x] Add helper methods to services
|
|
- [x] JobStepsService: start(), complete(), fail(), findByJob()
|
|
- [x] RunnerJobsService: updateStatus(), updateProgress()
|
|
- [x] JobEventsService: findByJob()
|
|
- [x] Run quality gates
|
|
- [x] All 9 E2E tests passing
|
|
- [x] All 1405 unit tests passing
|
|
- [x] Typecheck passing
|
|
- [x] Lint passing
|
|
- [x] Build passing
|
|
- [x] Commit changes
|
|
|
|
## Test Patterns Observed
|
|
|
|
- Use Vitest with NestJS Testing module
|
|
- Mock services with vi.fn()
|
|
- Use Test.createTestingModule for DI
|
|
- Follow existing integration test pattern from quality-orchestrator
|
|
- Mock child_process.exec for command execution
|
|
- Create helper functions for test data
|
|
|
|
## Testing
|
|
|
|
### Test Scenarios
|
|
|
|
1. **Happy Path**: webhook → job creation → step execution → completion
|
|
2. **Error Handling**: step failure → retry → final failure
|
|
3. **Chat Integration**: command → job → status updates
|
|
|
|
### Quality Gates
|
|
|
|
- pnpm typecheck
|
|
- pnpm lint
|
|
- pnpm test
|
|
- pnpm build
|
|
|
|
## Notes
|
|
|
|
- All dependencies (Phase 1-4) are complete
|
|
- Herald (#172) may complete during this task
|
|
- Follow TDD: RED → GREEN → REFACTOR
|
|
- Use existing test patterns as reference
|
|
|
|
## Implementation Summary
|
|
|
|
### Files Created
|
|
|
|
1. `apps/api/vitest.e2e.config.ts` - E2E test configuration
|
|
2. `apps/api/test/fixtures/` - Mock fixtures directory
|
|
- `mock-discord.fixture.ts` - Mock Discord client and messages
|
|
- `mock-bullmq.fixture.ts` - Mock BullMQ queues and jobs
|
|
- `mock-prisma.fixture.ts` - Mock Prisma service with CRUD operations
|
|
- `index.ts` - Fixture exports
|
|
3. `apps/api/test/e2e/job-orchestration.e2e-spec.ts` - 9 E2E tests
|
|
|
|
### Files Modified
|
|
|
|
1. `apps/api/src/job-steps/job-steps.service.ts`
|
|
- Added `start(id)` - simplified start without jobId parameter
|
|
- Added `complete(id, data)` - simplified complete with optional output/tokens
|
|
- Added `fail(id, data)` - simplified fail with optional error message
|
|
- Added `findByJob(jobId)` - alias for findAllByJob
|
|
|
|
2. `apps/api/src/runner-jobs/runner-jobs.service.ts`
|
|
- Added `updateStatus(id, workspaceId, status, data)` - update job status with timestamps
|
|
- Added `updateProgress(id, workspaceId, progressPercent)` - update job progress
|
|
|
|
3. `apps/api/src/job-events/job-events.service.ts`
|
|
- Added `findByJob(jobId)` - get all events for a job without pagination
|
|
|
|
### E2E Tests Coverage
|
|
|
|
1. Happy path: webhook → job creation → step execution → completion
|
|
2. Event emission throughout job lifecycle
|
|
3. Step failure and retry handling
|
|
4. Job failure after max retries
|
|
5. Discord command parsing and job creation
|
|
6. WebSocket status updates
|
|
7. Job cancellation
|
|
8. Job retry mechanism
|
|
9. Progress percentage tracking
|