Jason Woltje
a5416e4a66
Updated pnpm version from 10.19.0 to 10.27.0 to fix HIGH severity vulnerabilities (CVE-2025-69262, CVE-2025-69263, CVE-2025-6926). Changes: - apps/api/Dockerfile: line 8 - apps/web/Dockerfile: lines 8 and 81 Fixes #180
60 lines
2.8 KiB
Markdown
60 lines
2.8 KiB
Markdown
# Issue #149: [COORD-009] Test rejection loop
|
|
|
|
## Objective
|
|
|
|
Validate quality gates prevent premature completion through simulated rejection scenarios.
|
|
|
|
## Approach
|
|
|
|
1. Create comprehensive integration tests for rejection loop scenarios
|
|
2. Test each gate failure type triggers proper rejection
|
|
3. Verify forced continuation prompts are generated correctly
|
|
4. Ensure agents cannot bypass gates
|
|
5. Validate loop continues until all gates pass
|
|
|
|
## Test Scenarios
|
|
|
|
- [x] Agent claims done with failing tests → `test_rejection_on_failing_tests`
|
|
- [x] Agent claims done with linting errors → `test_rejection_on_linting_errors`
|
|
- [x] Agent claims done with low coverage → `test_rejection_on_low_coverage`
|
|
- [x] Agent claims done with build errors → `test_rejection_on_build_errors`
|
|
- [x] All gates passing allows completion → `test_acceptance_on_all_gates_passing`
|
|
- [x] Multiple simultaneous gate failures handled correctly → `test_rejection_on_multiple_gate_failures`
|
|
- [x] Forced continuation prompts are non-negotiable → `test_continuation_prompt_is_non_negotiable`
|
|
- [x] Remediation steps included in prompts → `test_continuation_prompt_includes_remediation_steps`
|
|
- [x] Agents cannot bypass gates → `test_agent_cannot_bypass_gates`
|
|
|
|
## Progress
|
|
|
|
- [x] Read existing QualityOrchestrator and ForcedContinuationService code
|
|
- [x] Write comprehensive integration tests (TDD)
|
|
- [x] Run tests - all 9 tests pass
|
|
- [x] Fix linting issues
|
|
- [x] Run type checking - passes
|
|
- [x] All quality gates pass
|
|
- [x] Commit changes
|
|
|
|
## Testing
|
|
|
|
Test file: `apps/coordinator/tests/test_rejection_loop.py`
|
|
|
|
## Notes
|
|
|
|
The services already exist from Issue 148, so this is primarily testing the rejection loop behavior through integration tests that simulate agent completion scenarios.
|
|
|
|
## Summary
|
|
|
|
Successfully implemented 9 comprehensive integration tests for rejection loop scenarios:
|
|
|
|
1. **test_rejection_on_failing_tests** - Validates test failures trigger rejection and continuation prompt
|
|
2. **test_rejection_on_linting_errors** - Validates lint errors trigger rejection and continuation prompt
|
|
3. **test_rejection_on_low_coverage** - Validates low coverage triggers rejection and continuation prompt
|
|
4. **test_rejection_on_build_errors** - Validates build errors trigger rejection and continuation prompt
|
|
5. **test_acceptance_on_all_gates_passing** - Validates completion allowed when all gates pass
|
|
6. **test_rejection_on_multiple_gate_failures** - Validates multiple failures handled correctly
|
|
7. **test_continuation_prompt_is_non_negotiable** - Validates prompts use directive language
|
|
8. **test_continuation_prompt_includes_remediation_steps** - Validates actionable remediation steps
|
|
9. **test_agent_cannot_bypass_gates** - Validates all gates run without short-circuiting
|
|
|
|
All tests pass, linting passes, type checking passes.
|