From 934a6fc0f41d33fe5ab3100005f5f076f5a968a8 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Mon, 2 Mar 2026 21:34:33 -0600 Subject: [PATCH] docs: add README with REST API, MCP setup, and Claude Code gotchas Documents full CRUD REST API, MCP tool list, Claude Code registration procedure (claude mcp add --scope user --transport http), and critical gotchas: - MCP config belongs in ~/.claude.json not ~/.claude/settings.json - transport must be 'http' not 'sse' for FastMCP streamable HTTP - capture_async daemon thread issue in CLI scripts - JSONB metadata filter pattern - re-embedding behavior on PATCH Co-Authored-By: Claude Sonnet 4.6 --- README.md | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..e551dab --- /dev/null +++ b/README.md @@ -0,0 +1,197 @@ +# OpenBrain + +Self-hosted semantic brain — pgvector + REST API + MCP server for any AI agent. + +Stores and retrieves decisions, context, gotchas, and project state via semantic search. +Works across all agent harnesses (Claude Code, Codex, OpenCode) in the same session or across sessions. + +## What It Does + +- **Capture** thoughts with natural-language content, a source tag, and arbitrary metadata +- **Search** by meaning using pgvector cosine similarity (Ollama embeddings) +- **Retrieve** recent thoughts or filter by source/metadata +- **Full CRUD** — get, update, delete single or bulk by filter + +## Stack + +- FastAPI + asyncpg (PostgreSQL + pgvector) +- Ollama for embeddings (`bge-m3:latest`, 1024-dim) +- FastMCP for MCP server (streamable HTTP transport) +- Deployed via Docker Swarm + Portainer + +## REST API + +All endpoints require `Authorization: Bearer `. + +```bash +BASE=https://your-openbrain-host + +# Capture +curl -X POST $BASE/v1/thoughts \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"content": "...", "source": "agent-name", "metadata": {}}' + +# Search (semantic) +curl -X POST $BASE/v1/search \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"query": "your search", "limit": 5}' + +# Recent +curl $BASE/v1/thoughts/recent?limit=10 -H "Authorization: Bearer $TOKEN" + +# Get by ID +curl $BASE/v1/thoughts/{id} -H "Authorization: Bearer $TOKEN" + +# Update (re-embeds if content changes) +curl -X PATCH $BASE/v1/thoughts/{id} \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"content": "updated text", "metadata": {"key": "val"}}' + +# Delete single +curl -X DELETE $BASE/v1/thoughts/{id} -H "Authorization: Bearer $TOKEN" + +# Bulk delete by filter (source and/or metadata_id required) +curl -X DELETE "$BASE/v1/thoughts?source=agent-name&metadata_id=my-entity" \ + -H "Authorization: Bearer $TOKEN" + +# List with filters +curl "$BASE/v1/thoughts?source=agent-name&limit=20" -H "Authorization: Bearer $TOKEN" + +# Stats +curl $BASE/v1/stats -H "Authorization: Bearer $TOKEN" +``` + +## MCP Server (Claude Code) + +OpenBrain exposes an MCP server at `/mcp` using FastMCP's **streamable HTTP transport**. + +### Registering in Claude Code + +**MCPs must be registered in `~/.claude.json` via the CLI — not in `~/.claude/settings.json`.** +`settings.json` does not load MCP servers. Edits there are silently ignored. + +```bash +# Register once per machine (user scope = persists across all projects) +claude mcp add --scope user --transport http openbrain https://your-openbrain-host/mcp \ + --header "Authorization: Bearer YOUR_TOKEN" + +# Verify it registered +claude mcp list +``` + +**Transport must be `http`** — not `sse`. FastMCP uses streamable HTTP. `type: "sse"` is a +different (deprecated) protocol and will silently fail to connect. + +### Available MCP Tools + +Once registered, the following tools are available natively in Claude Code sessions: + +| Tool | Description | +|------|-------------| +| `capture` | Store a thought with content, source, and metadata | +| `search` | Semantic search by meaning | +| `recent` | List most recently added thoughts | +| `get` | Retrieve a thought by ID | +| `update` | Update content, source, or metadata (re-embeds if content changes) | +| `delete` | Delete a thought by ID | +| `delete_where` | Bulk delete by source and/or metadata ID | +| `list_thoughts` | List with source/metadata filters and pagination | +| `stats` | Total counts and per-source breakdown | + +### When to Use Each Tool + +| Trigger | Action | +|---------|--------| +| Session start | `search` or `recent` to load prior context | +| Discovery or gotcha | `capture` immediately | +| Significant decision | `capture` with rationale | +| Task or milestone complete | `capture` summary | +| Cross-agent handoff | `capture` current state before ending | +| Repeated migration | `delete_where` + recapture (upsert pattern) | + +## Environment Variables + +| Variable | Required | Description | +|----------|----------|-------------| +| `DATABASE_URL` | Yes | PostgreSQL connection string with pgvector | +| `API_KEY` | Yes | Bearer token for authentication | +| `OLLAMA_URL` | Yes | Ollama base URL (e.g. `http://10.1.1.42:11434`) | +| `OLLAMA_MODEL` | No | Embedding model (default: `bge-m3:latest`) | +| `MCP_ALLOWED_HOSTS` | Yes | Comma-separated allowed hostnames for DNS rebinding protection | + +## Deployment + +### Docker Compose (Portainer / Swarm) + +```yaml +services: + openbrain: + image: your-registry/openbrain:sha- + environment: + DATABASE_URL: postgresql://user:pass@db:5432/openbrain + API_KEY: your-api-key + OLLAMA_URL: http://ollama-host:11434 + MCP_ALLOWED_HOSTS: your-openbrain-host.com + ports: + - "8000:8000" +``` + +See `docker-compose.portainer.yml` for the full Swarm stack definition. + +### Database Setup + +Requires PostgreSQL with the `vector` extension: + +```sql +CREATE EXTENSION IF NOT EXISTS vector; +CREATE TABLE thoughts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + content TEXT NOT NULL, + embedding vector(1024), + source TEXT NOT NULL DEFAULT '', + metadata JSONB NOT NULL DEFAULT '{}', + created_at TIMESTAMPTZ NOT NULL DEFAULT now() +); +CREATE INDEX ON thoughts USING ivfflat (embedding vector_cosine_ops); +CREATE INDEX ON thoughts (source); +CREATE INDEX ON thoughts ((metadata->>'id')); +``` + +## Development + +```bash +# Install deps +pip install -e ".[dev]" + +# Run locally +uvicorn src.main:app --reload + +# Lint + format +python -m ruff check src/ +python -m ruff format src/ + +# Tests +pytest +``` + +## Known Gotchas + +- **`capture_async` drops data in CLI scripts** — uses daemon threads that die on process exit. + Use sync `capture` in any short-lived script context. `capture_async` is only safe in + long-running processes (servers, daemons). + +- **`type: "http"` required for Claude Code MCP** — not `type: "sse"`. They are different + protocols. SSE is deprecated. Using `sse` against this server silently fails to connect. + +- **MCP registration goes in `~/.claude.json`** — not `~/.claude/settings.json`. + Always use `claude mcp add --scope user` to register. Never hand-edit `mcpServers` in + `settings.json` — that key is ignored by Claude Code's MCP loader. + +- **JSONB filter** — `metadata->>'id'` is exposed as `?metadata_id=` in list/delete endpoints. + Only the top-level `id` field in metadata is indexed this way. + +- **Re-embedding on update** — `PATCH /v1/thoughts/{id}` calls Ollama again only when + `content` changes. Metadata-only updates skip re-embedding.