stack/docs/scratchpads/149-test-rejection-loop.md

Issue #149: [COORD-009] Test rejection loop

Objective

Validate quality gates prevent premature completion through simulated rejection scenarios.

Approach

1. Create comprehensive integration tests for rejection loop scenarios
2. Test each gate failure type triggers proper rejection
3. Verify forced continuation prompts are generated correctly
4. Ensure agents cannot bypass gates
5. Validate loop continues until all gates pass

Test Scenarios

• Agent claims done with failing tests → test_rejection_on_failing_tests
• Agent claims done with linting errors → test_rejection_on_linting_errors
• Agent claims done with low coverage → test_rejection_on_low_coverage
• Agent claims done with build errors → test_rejection_on_build_errors
• All gates passing allows completion → test_acceptance_on_all_gates_passing
• Multiple simultaneous gate failures handled correctly → test_rejection_on_multiple_gate_failures
• Forced continuation prompts are non-negotiable → test_continuation_prompt_is_non_negotiable
• Remediation steps included in prompts → test_continuation_prompt_includes_remediation_steps
• Agents cannot bypass gates → test_agent_cannot_bypass_gates

Progress

• Read existing QualityOrchestrator and ForcedContinuationService code
• Write comprehensive integration tests (TDD)
• Run tests - all 9 tests pass
• Fix linting issues
• Run type checking - passes
• All quality gates pass
• Commit changes

Testing

Test file: apps/coordinator/tests/test_rejection_loop.py

Notes

The services already exist from Issue 148, so this is primarily testing the rejection loop behavior through integration tests that simulate agent completion scenarios.

Summary

Successfully implemented 9 comprehensive integration tests for rejection loop scenarios:

1. test_rejection_on_failing_tests - Validates test failures trigger rejection and continuation prompt
2. test_rejection_on_linting_errors - Validates lint errors trigger rejection and continuation prompt
3. test_rejection_on_low_coverage - Validates low coverage triggers rejection and continuation prompt
4. test_rejection_on_build_errors - Validates build errors trigger rejection and continuation prompt
5. test_acceptance_on_all_gates_passing - Validates completion allowed when all gates pass
6. test_rejection_on_multiple_gate_failures - Validates multiple failures handled correctly
7. test_continuation_prompt_is_non_negotiable - Validates prompts use directive language
8. test_continuation_prompt_includes_remediation_steps - Validates actionable remediation steps
9. test_agent_cannot_bypass_gates - Validates all gates run without short-circuiting

All tests pass, linting passes, type checking passes.