- Create LlmTelemetryTrackerService for non-blocking event emission
- Normalize token usage across Anthropic, OpenAI, Ollama providers
- Add cost table with per-token pricing in microdollars
- Instrument chat, chatStream, and embed methods
- Infer task type from calling context
- Aggregate streaming tokens after stream ends with fallback estimation
- Add 69 unit tests for tracker service, cost table, and LLM service
Refs #371
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements comprehensive LLM usage tracking with analytics endpoints.
Implementation:
- Added LlmUsageLog model to Prisma schema
- Created llm-usage module with service, controller, and DTOs
- Added tracking for token usage, costs, and durations
- Implemented analytics aggregation by provider, model, and task type
- Added filtering by workspace, provider, model, user, and date range
Testing:
- 20 unit tests with 90.8% coverage (exceeds 85% requirement)
- Tests for service and controller with full error handling
- Tests use Vitest following project conventions
API Endpoints:
- GET /api/llm-usage/analytics - Aggregated usage analytics
- GET /api/llm-usage/by-workspace/:workspaceId - Workspace usage logs
- GET /api/llm-usage/by-workspace/:workspaceId/provider/:provider - Provider logs
- GET /api/llm-usage/by-workspace/:workspaceId/model/:model - Model logs
Database:
- LlmUsageLog table with indexes for efficient queries
- Relations to User, Workspace, and LlmProviderInstance
- Ready for migration with: pnpm prisma migrate dev
Refs #309
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implement REST API endpoints for managing LLM provider instances.
Changes:
- Created DTOs for provider CRUD operations (CreateLlmProviderDto, UpdateLlmProviderDto, LlmProviderResponseDto)
- Implemented LlmProviderAdminController with full CRUD endpoints:
- GET /llm/admin/providers - List all providers
- GET /llm/admin/providers/:id - Get provider details
- POST /llm/admin/providers - Create new provider
- PATCH /llm/admin/providers/:id - Update provider
- DELETE /llm/admin/providers/:id - Delete provider
- POST /llm/admin/providers/:id/test - Test connection
- POST /llm/admin/reload - Reload from database
- Updated llm-manager.service.ts to support OpenAI and Claude providers
- Added comprehensive test suite with 97.95% coverage
- Proper validation, error handling, and type safety
All tests pass. Pre-commit hooks pass.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor LlmService to delegate to LlmManagerService instead of using
Ollama directly. This enables multiple provider support and user-specific
provider configuration.
Changes:
- Remove direct Ollama client from LlmService
- Delegate all LLM operations to provider via LlmManagerService
- Update health status to use provider-agnostic interface
- Add PrismaModule to LlmModule for manager service
- Maintain backward compatibility with existing API
- Achieve 89.74% test coverage
Fixes#127
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implemented centralized service for managing multiple LLM provider instances.
Architecture:
- LlmManagerService manages provider lifecycle and selection
- Loads provider instances from Prisma database on startup
- Maintains in-memory registry of active providers
- Factory pattern for provider instantiation
Core Features:
- Database integration via PrismaService
- Provider initialization on module startup (OnModuleInit)
- Get provider by ID
- Get all active providers
- Get system default provider
- Get user-specific provider with fallback to system default
- Health check all registered providers
- Dynamic registration/unregistration (hot reload)
- Reload from database without restart
Provider Selection Logic:
- User-level providers: userId matches, is enabled
- System-level providers: userId is NULL, is enabled
- Fallback: system default if no user provider found
- Graceful error handling with detailed logging
Integration:
- Added to LlmModule providers and exports
- Uses PrismaService for database queries
- Factory creates OllamaProvider from config
- Extensible for future providers (Claude, OpenAI)
Testing:
- 31 comprehensive unit tests
- 93.05% code coverage (exceeds 85% requirement)
- All error scenarios covered
- Proper mocking of dependencies
Quality Gates:
- ✅ All 31 tests passing
- ✅ 93.05% coverage
- ✅ Linting clean
- ✅ Type checking passed
- ✅ Code review approved
Fixes#126
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add Ollama client library (ollama npm package)
- Create LlmService for chat completion and embeddings
- Support streaming responses via Server-Sent Events
- Add configuration via env vars (OLLAMA_HOST, OLLAMA_TIMEOUT)
- Create endpoints: GET /llm/health, GET /llm/models, POST /llm/chat, POST /llm/embed
- Replace old OllamaModule with new LlmModule
- Add comprehensive tests with >85% coverage
Closes#21
