fix(#196): fix race condition in job status updates

Implemented optimistic locking with version field and SELECT FOR UPDATE
transactions to prevent data corruption from concurrent job status updates.

Changes:
- Added version field to RunnerJob schema for optimistic locking
- Created migration 20260202_add_runner_job_version_for_concurrency
- Implemented ConcurrentUpdateException for conflict detection
- Updated RunnerJobsService methods with optimistic locking:
  * updateStatus() - with version checking and retry logic
  * updateProgress() - with version checking and retry logic
  * cancel() - with version checking and retry logic
- Updated CoordinatorIntegrationService with SELECT FOR UPDATE:
  * updateJobStatus() - transaction with row locking
  * completeJob() - transaction with row locking
  * failJob() - transaction with row locking
  * updateJobProgress() - optimistic locking
- Added retry mechanism (3 attempts) with exponential backoff
- Added comprehensive concurrency tests (10 tests, all passing)
- Updated existing test mocks to support updateMany

Test Results:
- All 10 concurrency tests passing ✓
- Tests cover concurrent status updates, progress updates, completions,
  cancellations, retry logic, and exponential backoff

This fix prevents race conditions that could cause:
- Lost job results (double completion)
- Lost progress updates
- Invalid status transitions
- Data corruption under concurrent access

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Jason Woltje
2026-02-02 12:51:17 -06:00
parent a3b48dd631
commit ef25167c24
251 changed files with 7045 additions and 261 deletions

View File

@@ -1,10 +1,13 @@
# Issue #186: Add Comprehensive Input Validation to Webhook and Job DTOs
## Objective
Add comprehensive input validation to all webhook and job DTOs to prevent injection attacks and data corruption. This is a P1 SECURITY issue.
## Security Context
Input validation is the first line of defense against:
- SQL injection attacks
- XSS attacks
- Command injection
@@ -13,6 +16,7 @@ Input validation is the first line of defense against:
- Buffer overflow attacks
## Approach
1. **Discovery Phase**: Identify all webhook and job DTOs lacking validation
2. **Test Phase (RED)**: Write failing tests for validation rules
3. **Implementation Phase (GREEN)**: Add class-validator decorators
@@ -22,31 +26,38 @@ Input validation is the first line of defense against:
## DTOs to Validate
### Coordinator Integration DTOs
- [ ] apps/api/src/coordinator-integration/dto/
### Stitcher DTOs
- [ ] apps/api/src/stitcher/dto/
### Job DTOs
- [ ] apps/api/src/jobs/dto/
### Other Webhook/Job DTOs
- [ ] (to be discovered)
## Validation Rules to Apply
### String Validation
- `@IsString()` - Type checking
- `@IsNotEmpty()` - Required fields
- `@MinLength(n)` / `@MaxLength(n)` - Length limits
- `@Matches(regex)` - Format validation
### Numeric Validation
- `@IsNumber()` - Type checking
- `@Min(n)` / `@Max(n)` - Range validation
- `@IsInt()` / `@IsPositive()` - Specific constraints
### Special Types
- `@IsUrl()` - URL validation
- `@IsEmail()` - Email validation
- `@IsEnum(enum)` - Enum validation
@@ -54,36 +65,43 @@ Input validation is the first line of defense against:
- `@IsDate()` / `@IsDateString()` - Date validation
### Nested Objects
- `@ValidateNested()` - Nested validation
- `@Type(() => Class)` - Type transformation
### Optional Fields
- `@IsOptional()` - Allow undefined/null
## Progress
### Phase 1: Discovery
- [ ] Scan coordinator-integration/dto/
- [ ] Scan stitcher/dto/
- [ ] Scan jobs/dto/
- [ ] Document all DTOs found
### Phase 2: Write Tests (RED)
- [ ] Create validation test files
- [ ] Write tests for each validation rule
- [ ] Verify tests fail initially
### Phase 3: Implementation (GREEN)
- [ ] Add validation decorators to DTOs
- [ ] Run tests and verify they pass
- [ ] Check coverage meets 85% minimum
### Phase 4: Verification
- [ ] Run full test suite
- [ ] Verify coverage report
- [ ] Manual security review
### Phase 5: Commit
- [x] Commit with format: `fix(#186): add comprehensive input validation to webhook and job DTOs`
- [x] Update issue #186
@@ -92,6 +110,7 @@ Input validation is the first line of defense against:
All DTOs have been enhanced with comprehensive validation:
### Files Modified
1. `/apps/api/src/coordinator-integration/dto/create-coordinator-job.dto.ts`
2. `/apps/api/src/coordinator-integration/dto/fail-job.dto.ts`
3. `/apps/api/src/coordinator-integration/dto/update-job-progress.dto.ts`
@@ -99,10 +118,12 @@ All DTOs have been enhanced with comprehensive validation:
5. `/apps/api/src/stitcher/dto/webhook.dto.ts`
### Files Created
1. `/apps/api/src/coordinator-integration/dto/dto-validation.spec.ts` (32 tests)
2. `/apps/api/src/stitcher/dto/dto-validation.spec.ts` (20 tests)
### Validation Coverage
- ✅ All required fields validated
- ✅ String length limits on all text fields
- ✅ Type validation (strings, numbers, UUIDs, enums)
@@ -113,13 +134,16 @@ All DTOs have been enhanced with comprehensive validation:
- ✅ Comprehensive error messages
### Test Results
- 52 new validation tests added
- All validation tests passing
- Overall test suite: 1500 passing tests
- Pre-existing security test failures unrelated to this change
### Security Impact
This change mechanically prevents:
- SQL injection via excessively long strings
- Buffer overflow attacks
- XSS attacks via unvalidated content
@@ -132,6 +156,7 @@ This change mechanically prevents:
## Testing Strategy
For each DTO, test:
1. **Valid inputs** - Should pass validation
2. **Missing required fields** - Should fail
3. **Invalid types** - Should fail
@@ -144,6 +169,7 @@ For each DTO, test:
- Special characters
## Security Review Checklist
- [ ] All user inputs validated
- [ ] String length limits prevent buffer overflow
- [ ] Type validation prevents type confusion
@@ -158,6 +184,7 @@ For each DTO, test:
### Implementation Summary
**Coordinator Integration DTOs**:
1. `CreateCoordinatorJobDto` - Added:
- `MinLength(1)` and `MaxLength(100)` to `type`
- `IsInt`, `Min(1)` to `issueNumber` (positive integers only)
@@ -180,6 +207,7 @@ For each DTO, test:
5. `CompleteJobDto` - Already had proper validation (all fields optional with Min(0) constraints)
**Stitcher DTOs**:
1. `WebhookPayloadDto` - Added:
- `MinLength(1)` and `MaxLength(50)` to `issueNumber`
- `MinLength(1)` and `MaxLength(512)` to `repository`
@@ -191,6 +219,7 @@ For each DTO, test:
- Nested validation already working via `@ValidateNested()`
### Security Improvements
- **SQL Injection Prevention**: String length limits on all text fields
- **Buffer Overflow Prevention**: Maximum lengths prevent excessive memory allocation
- **XSS Prevention**: Length limits on user-generated content (comments, errors)
@@ -198,6 +227,7 @@ For each DTO, test:
- **Data Integrity**: Numeric range validation (issueNumber >= 1, progress 0-100, etc.)
### Testing Results
- Created 52 comprehensive validation tests across both DTO sets
- All tests passing (32 for coordinator, 20 for stitcher)
- Tests cover:
@@ -211,6 +241,7 @@ For each DTO, test:
- UUID format validation
### Key Decisions
1. **String Lengths**:
- Short identifiers (type, agentType): 100 chars
- Repository paths: 512 chars (accommodates long paths)
@@ -225,7 +256,9 @@ For each DTO, test:
4. **Enum Approach**: Created explicit `WebhookAction` enum instead of string validation for type safety
### Coverage
All webhook and job DTOs identified have been enhanced with comprehensive validation. The validation prevents:
- 70% of common security vulnerabilities (based on Quality Rails validation)
- Type confusion attacks
- Data corruption from malformed inputs