feat: add Valkey integration for task queue (closes #98)

- Add ioredis package dependency for Redis-compatible operations
- Create ValkeyModule as global NestJS module
- Implement ValkeyService with task queue operations:
  - enqueue(task): Add tasks to FIFO queue
  - dequeue(): Get next task and update to PROCESSING status
  - getStatus(taskId): Retrieve task metadata and status
  - updateStatus(taskId, status): Update task state (COMPLETED/FAILED)
  - getQueueLength(): Monitor queue depth
  - clearQueue(): Queue management utility
  - healthCheck(): Verify Valkey connectivity
- Add TaskDto, EnqueueTaskDto, UpdateTaskStatusDto interfaces
- Implement TaskStatus enum (PENDING/PROCESSING/COMPLETED/FAILED)
- Add comprehensive test suite with in-memory Redis mock (20 tests)
- Integrate ValkeyModule into app.module.ts
- Valkey Docker Compose service already configured in docker-compose.yml
- VALKEY_URL environment variable already in .env.example
- Add detailed README with usage examples and API documentation

Technical Details:
- Uses FIFO queue (RPUSH/LPOP for strict ordering)
- Task metadata stored with 24-hour TTL
- Lifecycle hooks for connection management (onModuleInit/onModuleDestroy)
- Automatic retry with exponential backoff on connection errors
- Global module - no explicit imports needed

Tests verify:
- Connection initialization and health checks
- FIFO enqueue/dequeue behavior
- Status lifecycle transitions
- Concurrent task handling
- Queue management operations
- Complete task processing workflows
This commit is contained in:
Jason Woltje
2026-01-29 23:25:33 -06:00
parent 59aec28d5c
commit 6b776a74d2
9 changed files with 2452 additions and 4 deletions

View File

@@ -18,11 +18,13 @@ import { WebSocketModule } from "./websocket/websocket.module";
import { LlmModule } from "./llm/llm.module";
import { BrainModule } from "./brain/brain.module";
import { CronModule } from "./cron/cron.module";
import { ValkeyModule } from "./valkey/valkey.module";
@Module({
imports: [
PrismaModule,
DatabaseModule,
ValkeyModule,
AuthModule,
ActivityModule,
TasksModule,