fix(#196): fix race condition in job status updates
Implemented optimistic locking with version field and SELECT FOR UPDATE transactions to prevent data corruption from concurrent job status updates. Changes: - Added version field to RunnerJob schema for optimistic locking - Created migration 20260202_add_runner_job_version_for_concurrency - Implemented ConcurrentUpdateException for conflict detection - Updated RunnerJobsService methods with optimistic locking: * updateStatus() - with version checking and retry logic * updateProgress() - with version checking and retry logic * cancel() - with version checking and retry logic - Updated CoordinatorIntegrationService with SELECT FOR UPDATE: * updateJobStatus() - transaction with row locking * completeJob() - transaction with row locking * failJob() - transaction with row locking * updateJobProgress() - optimistic locking - Added retry mechanism (3 attempts) with exponential backoff - Added comprehensive concurrency tests (10 tests, all passing) - Updated existing test mocks to support updateMany Test Results: - All 10 concurrency tests passing ✓ - Tests cover concurrent status updates, progress updates, completions, cancellations, retry logic, and exponential backoff This fix prevents race conditions that could cause: - Lost job results (double completion) - Lost progress updates - Invalid status transitions - Data corruption under concurrent access Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,7 @@
|
||||
-- Add version field for optimistic locking to prevent race conditions
|
||||
-- This allows safe concurrent updates to runner job status
|
||||
|
||||
ALTER TABLE "runner_jobs" ADD COLUMN "version" INTEGER NOT NULL DEFAULT 1;
|
||||
|
||||
-- Create index for better performance on version checks
|
||||
CREATE INDEX "runner_jobs_version_idx" ON "runner_jobs"("version");
|
||||
@@ -1135,6 +1135,7 @@ model RunnerJob {
|
||||
status RunnerJobStatus @default(PENDING)
|
||||
priority Int
|
||||
progressPercent Int @default(0) @map("progress_percent")
|
||||
version Int @default(1) // Optimistic locking version
|
||||
|
||||
// Results
|
||||
result Json?
|
||||
|
||||
Reference in New Issue
Block a user