mosaic/telemetry-client-py
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(#1): SDK integration guide, API reference, and CI pipeline
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Details

Browse Source 
- Rewrite README with quick start, FastAPI snippet, async/sync patterns,
  config reference with env vars, and API version targeting (v1, schema 1.0)
- Add docs/integration-guide.md with full FastAPI and generic Python
  integration examples, environment-specific config, prediction queries,
  error handling, and dry-run mode documentation
- Add docs/api-reference.md covering all exported classes, methods, Pydantic
  models, enums (TaskType, Complexity, Harness, Provider, QualityGate,
  Outcome, RepoSizeCategory), and internal components
- Add Woodpecker CI pipeline (.woodpecker.yml) with quality gates: lint,
  format check, typecheck, bandit security scan, pip-audit, and pytest
  with 85% coverage gate
- Add bandit and pip-audit to dev dependencies

Fixes #1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jason Woltje
2026-02-14 22:39:19 -06:00
parent f02207e33c
commit 883fd4d60f
7 changed files with 1807 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

205
README.md
View File

@@ -2,6 +2,8 @@
Python client SDK for [Mosaic Stack Telemetry](https://github.com/mosaicstack/telemetry). Report AI coding task-completion telemetry and query crowd-sourced predictions for token usage, cost, and quality outcomes.
**Targets:** Telemetry API **v1** | Event schema **1.0** | Python **3.10+**
## Installation
```bash
@@ -10,9 +12,21 @@ pip install mosaicstack-telemetry
uv add mosaicstack-telemetry
```
## Quick Start (Sync)
Runtime dependencies: `httpx` and `pydantic`.
Best for scripts, aider integrations, and non-async contexts:
## Quick Start
### 1. Configure
Set environment variables (or pass values to the constructor):
```bash
export MOSAIC_TELEMETRY_SERVER_URL=https://tel-api.mosaicstack.dev
export MOSAIC_TELEMETRY_API_KEY=your-64-char-hex-api-key
export MOSAIC_TELEMETRY_INSTANCE_ID=a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d
```
### 2. Track Events
```python
from mosaicstack_telemetry import (
@@ -27,97 +41,75 @@ from mosaicstack_telemetry import (
QualityGate,
)
config = TelemetryConfig(
server_url="https://telemetry.mosaicstack.dev",
api_key="your-64-char-hex-api-key-here...",
instance_id="your-uuid-instance-id",
)
config = TelemetryConfig() # Reads from MOSAIC_TELEMETRY_* env vars
client = TelemetryClient(config)
client.start() # Starts background submission thread
with TelemetryClient(config) as client:
event = (
EventBuilder(instance_id=config.instance_id)
.task_type(TaskType.IMPLEMENTATION)
.model("claude-sonnet-4-5-20250929")
.provider(Provider.ANTHROPIC)
.harness_type(Harness.CLAUDE_CODE)
.complexity_level(Complexity.MEDIUM)
.outcome_value(Outcome.SUCCESS)
.duration_ms(45000)
.tokens(estimated_in=105000, estimated_out=45000, actual_in=112340, actual_out=38760)
.cost(estimated=630000, actual=919200) # Microdollars (1 USD = 1,000,000)
.quality(passed=True, gates_run=[QualityGate.BUILD, QualityGate.LINT, QualityGate.TEST])
.context(compactions=2, rotations=0, utilization=0.72)
.language("typescript")
.build()
)
# Build and track an event
event = (
EventBuilder(instance_id=config.instance_id)
.task_type(TaskType.IMPLEMENTATION)
.model("claude-sonnet-4-20250514")
.provider(Provider.ANTHROPIC)
.harness_type(Harness.AIDER)
.complexity_level(Complexity.MEDIUM)
.outcome_value(Outcome.SUCCESS)
.duration_ms(45000)
.tokens(estimated_in=5000, estimated_out=2000, actual_in=5200, actual_out=1800)
.cost(estimated=50000, actual=48000)
.quality(passed=True, gates_run=[QualityGate.LINT, QualityGate.TEST])
.context(compactions=0, rotations=0, utilization=0.4)
.language("python")
.build()
)
client.track(event) # Non-blocking, thread-safe
# When done
client.stop() # Flushes remaining events
client.track(event) # Non-blocking, thread-safe, never throws
```
`track()` queues the event in memory. A background thread flushes batches to the server every 5 minutes (configurable). The context manager ensures all events are flushed on exit.
## Async Usage
For asyncio-based applications:
For asyncio applications (FastAPI, aiohttp, etc.):
```python
import asyncio
async with TelemetryClient(config) as client:
client.track(event) # track() is always synchronous
```
Or manually:
```python
client = TelemetryClient(config)
await client.start_async()
client.track(event)
await client.stop_async()
```
## FastAPI Integration
```python
from contextlib import asynccontextmanager
from fastapi import FastAPI
from mosaicstack_telemetry import TelemetryClient, TelemetryConfig
async def main():
config = TelemetryConfig(
server_url="https://telemetry.mosaicstack.dev",
api_key="your-64-char-hex-api-key-here...",
instance_id="your-uuid-instance-id",
)
config = TelemetryConfig() # From env vars
telemetry = TelemetryClient(config)
client = TelemetryClient(config)
await client.start_async() # Starts asyncio background task
@asynccontextmanager
async def lifespan(app: FastAPI):
await telemetry.start_async()
yield
await telemetry.stop_async()
# track() is always synchronous
client.track(event)
app = FastAPI(lifespan=lifespan)
await client.stop_async() # Flushes remaining events
asyncio.run(main())
@app.post("/tasks/complete")
async def complete_task():
# ... build event from request data ...
telemetry.track(event)
return {"status": "tracked"}
```
## Context Manager
Both sync and async context managers are supported:
```python
# Sync
with TelemetryClient(config) as client:
client.track(event)
# Async
async with TelemetryClient(config) as client:
client.track(event)
```
## Configuration via Environment Variables
All core settings can be set via environment variables:
```bash
export MOSAIC_TELEMETRY_ENABLED=true
export MOSAIC_TELEMETRY_SERVER_URL=https://telemetry.mosaicstack.dev
export MOSAIC_TELEMETRY_API_KEY=your-64-char-hex-api-key
export MOSAIC_TELEMETRY_INSTANCE_ID=your-uuid-instance-id
```
Then create a config with defaults:
```python
config = TelemetryConfig() # Picks up env vars automatically
```
Explicit constructor values take priority over environment variables.
See the [Integration Guide](docs/integration-guide.md) for the full FastAPI example.
## Querying Predictions
@@ -128,53 +120,66 @@ from mosaicstack_telemetry import PredictionQuery, TaskType, Provider, Complexit
query = PredictionQuery(
task_type=TaskType.IMPLEMENTATION,
model="claude-sonnet-4-20250514",
model="claude-sonnet-4-5-20250929",
provider=Provider.ANTHROPIC,
complexity=Complexity.MEDIUM,
)
# Async
await client.refresh_predictions([query])
# Sync
client.refresh_predictions_sync([query])
# Async
await client.refresh_predictions([query])
# Read from cache
prediction = client.get_prediction(query)
if prediction and prediction.prediction:
print(f"Expected input tokens (median): {prediction.prediction.input_tokens.median}")
print(f"Expected cost (median): ${prediction.prediction.cost_usd_micros['median'] / 1_000_000:.4f}")
print(f"Median input tokens: {prediction.prediction.input_tokens.median}")
print(f"Median cost: ${prediction.prediction.cost_usd_micros['median'] / 1_000_000:.4f}")
print(f"Quality gate pass rate: {prediction.prediction.quality.gate_pass_rate:.0%}")
```
## Configuration Reference
| Parameter | Default | Env Var | Description |
|-----------|---------|---------|-------------|
| `server_url` | (required) | `MOSAIC_TELEMETRY_SERVER_URL` | Telemetry API base URL |
| `api_key` | (required) | `MOSAIC_TELEMETRY_API_KEY` | 64-character hex API key |
| `instance_id` | (required) | `MOSAIC_TELEMETRY_INSTANCE_ID` | UUID identifying this instance |
| `enabled` | `True` | `MOSAIC_TELEMETRY_ENABLED` | Enable/disable telemetry |
| `submit_interval_seconds` | `300.0` | -- | Background flush interval (seconds) |
| `max_queue_size` | `1000` | -- | Max events in memory queue |
| `batch_size` | `100` | -- | Events per batch (server max: 100) |
| `request_timeout_seconds` | `10.0` | -- | HTTP request timeout |
| `prediction_cache_ttl_seconds` | `21600.0` | -- | Prediction cache TTL (6 hours) |
| `dry_run` | `False` | -- | Log batches but don't send to server |
| `max_retries` | `3` | -- | Retries on transient failures |
Constructor values take priority over environment variables.
## Error Handling
- `track()` **never throws** and **never blocks**. Errors are logged, not raised.
- When the queue is full, the oldest events are evicted.
- Failed submissions are retried with exponential backoff (honors `Retry-After` on 429).
- All logging uses the `mosaicstack_telemetry` logger.
## Dry-Run Mode
Test your integration without sending data to the server:
Test your integration without sending data:
```python
config = TelemetryConfig(
server_url="https://telemetry.mosaicstack.dev",
server_url="https://tel-api.mosaicstack.dev",
api_key="a" * 64,
instance_id="12345678-1234-1234-1234-123456789abc",
dry_run=True, # Logs batches but doesn't send
)
```
## Configuration Reference
## Documentation
| Parameter | Default | Description |
|-----------|---------|-------------|
| `server_url` | (required) | Telemetry server base URL |
| `api_key` | (required) | 64-character hex API key |
| `instance_id` | (required) | UUID identifying this instance |
| `enabled` | `True` | Enable/disable telemetry |
| `submit_interval_seconds` | `300.0` | Background flush interval |
| `max_queue_size` | `1000` | Max events in memory queue |
| `batch_size` | `100` | Events per batch (server max) |
| `request_timeout_seconds` | `10.0` | HTTP request timeout |
| `prediction_cache_ttl_seconds` | `21600.0` | Prediction cache TTL (6h) |
| `dry_run` | `False` | Log but don't send |
| `max_retries` | `3` | Retries on failure |
- **[Integration Guide](docs/integration-guide.md)** -- Installation, configuration, FastAPI and generic Python examples, async vs sync patterns, prediction queries, error handling
- **[API Reference](docs/api-reference.md)** -- All exported classes, methods, types, and enums
## License