feat: M10-Telemetry — Mosaic Telemetry integration #407

Merged
jason.woltje merged 14 commits from feature/m10-telemetry into develop 2026-02-15 17:32:08 +00:00

14 Commits

Author SHA1 Message Date
a943ae139a fix(#375): resolve lint errors in usage dashboard
All checks were successful
ci/woodpecker/push/web Pipeline was successful
- Fix prettier formatting for Tooltip formatter props (single-line)
- Fix no-base-to-string by using typed props instead of Record<string, unknown>
- Fix restrict-template-expressions by wrapping number in String()

Refs #375

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:25:51 -06:00
8e27f73f8f fix(#375): resolve recharts TypeScript strict mode type errors
Some checks failed
ci/woodpecker/push/web Pipeline failed
- Fix Tooltip formatter/labelFormatter type overload conflicts
- Fix Pie label render props type mismatch
- Fix telemetry.ts date split array access type

Refs #375

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:21:54 -06:00
2eafa91e70 fix(#370): add mypy import-untyped ignore for mosaicstack_telemetry
All checks were successful
ci/woodpecker/push/coordinator Pipeline was successful
The mosaicstack-telemetry package lacks py.typed marker. Add type
ignore comment consistent with other import sites.

Refs #370

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:16:44 -06:00
248f711571 fix(#370): add Gitea PyPI registry to coordinator CI install step
Some checks failed
ci/woodpecker/push/coordinator Pipeline failed
The mosaicstack-telemetry package is hosted on the Gitea PyPI registry.
CI pip install needs --extra-index-url to find it.

Refs #370

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:14:11 -06:00
306c2e5bd8 fix(#371): resolve TypeScript strictness errors in telemetry tracking
Some checks failed
ci/woodpecker/push/coordinator Pipeline failed
ci/woodpecker/push/orchestrator Pipeline was successful
ci/woodpecker/push/api Pipeline was successful
ci/woodpecker/push/web Pipeline failed
- llm-cost-table.ts: Add undefined guard for MODEL_COSTS lookup
- llm-telemetry-tracker.service.ts: Allow undefined in callingContext
  for exactOptionalPropertyTypes compatibility

Refs #371

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
746ab20c38 chore: update tasks.md — all M10-Telemetry tasks complete
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
a5ee974765 feat(#375): frontend token usage and cost dashboard
- Install recharts for data visualization
- Add Usage nav item to sidebar navigation
- Create telemetry API service with data fetching functions
- Build dashboard page with summary cards, charts, and time range selector
- Token usage line chart, cost breakdown bar chart, task outcome pie chart
- Loading and empty states handled
- Responsive layout with PDA-friendly design
- Add unit tests (14 tests passing)

Refs #375

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
5958569cba docs(#376): telemetry integration guide
- Create comprehensive telemetry documentation at docs/telemetry.md
- Cover configuration, event schema, predictions, SDK reference
- Include development guide with dry-run mode and troubleshooting
- Link from main README.md

Refs #376

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
d6c6af10d9 feat(#372): track orchestrator agent task completions via telemetry
- Instrument Coordinator.process_queue() with timing and telemetry events
- Instrument OrchestrationLoop.process_next_issue() with quality gate tracking
- Add agent-to-telemetry mapping (model, provider, harness per agent name)
- Map difficulty levels to Complexity enum and gate names to QualityGate enum
- Track retry counts per issue (increment on failure, clear on success)
- Emit FAILURE outcome on agent spawn failure or quality gate rejection
- Non-blocking: telemetry errors are logged and swallowed, never delay tasks
- Pass telemetry client from FastAPI lifespan to Coordinator constructor
- Add 33 unit tests covering all telemetry scenarios

Refs #372

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
ed23293e1a feat(#373): prediction integration for cost estimation
- Create PredictionService for pre-task cost/token estimates
- Refresh common predictions on startup
- Integrate predictions into LLM telemetry tracker
- Add GET /api/telemetry/estimate endpoint
- Graceful degradation when no prediction data available
- Add unit tests for prediction service

Refs #373

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
fcecf3654b feat(#371): track LLM task completions via Mosaic Telemetry
- Create LlmTelemetryTrackerService for non-blocking event emission
- Normalize token usage across Anthropic, OpenAI, Ollama providers
- Add cost table with per-token pricing in microdollars
- Instrument chat, chatStream, and embed methods
- Infer task type from calling context
- Aggregate streaming tokens after stream ends with fallback estimation
- Add 69 unit tests for tracker service, cost table, and LLM service

Refs #371

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
24c21f45b3 feat(#374): add telemetry config to docker-compose and .env
- Add MOSAIC_TELEMETRY_* variables to .env.example with descriptions
- Pass telemetry env vars to api service in production compose
- Pass telemetry env vars to coordinator service in dev and swarm composes
- Swarm composes default to production URL (https://tel-api.mosaicstack.dev)
- Dev compose includes commented-out telemetry-api service placeholder
- All compose files default MOSAIC_TELEMETRY_ENABLED to false for safety

Refs #374

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
314dd24dce feat(#369): install @mosaicstack/telemetry-client in API
- Add .npmrc with scoped Gitea npm registry for @mosaicstack packages
- Create MosaicTelemetryModule (global, lifecycle-aware) at
  apps/api/src/mosaic-telemetry/
- Create MosaicTelemetryService wrapping TelemetryClient with
  convenience methods: trackTaskCompletion, getPrediction,
  refreshPredictions, eventBuilder
- Create mosaic-telemetry.config.ts for env var integration via
  NestJS ConfigService
- Register MosaicTelemetryModule in AppModule
- Add 32 unit tests covering module init, service methods, disabled
  mode, dry-run mode, and lifecycle management

Refs #369

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00
8d8d37dbf9 feat(#370): install mosaicstack-telemetry in Coordinator
- Add mosaicstack-telemetry>=0.1.0 to pyproject.toml dependencies
- Configure Gitea PyPI registry via pip.conf (extra-index-url)
- Integrate TelemetryClient in FastAPI lifespan (start_async/stop_async)
- Store client on app.state.mosaic_telemetry for downstream access
- Create mosaic_telemetry.py helper module with:
  - get_telemetry_client(): retrieve client from app state
  - build_task_event(): construct TaskCompletionEvent with coordinator defaults
  - create_telemetry_config(): create config from MOSAIC_TELEMETRY_* env vars
- Add 28 unit tests covering config, helpers, disabled mode, and lifespan
- New module has 100% test coverage

Refs #370

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 02:10:22 -06:00