openbrain/README.md

# OpenBrain

Self-hosted semantic brain — pgvector + REST API + MCP server for any AI agent.

Stores and retrieves decisions, context, gotchas, and project state via semantic search.
Works across all agent harnesses (Claude Code, Codex, OpenCode) in the same session or across sessions.

## What It Does

- **Capture** thoughts with natural-language content, a source tag, and arbitrary metadata
- **Search** by meaning using pgvector cosine similarity (Ollama embeddings)
- **Retrieve** recent thoughts or filter by source/metadata
- **Full CRUD** — get, update, delete single or bulk by filter

## Stack

- FastAPI + asyncpg (PostgreSQL + pgvector)
- Ollama for embeddings (`bge-m3:latest`, 1024-dim)
- FastMCP for MCP server (streamable HTTP transport)
- Deployed via Docker Swarm + Portainer

## REST API

All endpoints require `Authorization: Bearer <token>`.

```bash
BASE=https://your-openbrain-host

# Capture
curl -X POST $BASE/v1/thoughts \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"content": "...", "source": "agent-name", "metadata": {}}'

# Search (semantic)
curl -X POST $BASE/v1/search \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "your search", "limit": 5}'

# Recent
curl $BASE/v1/thoughts/recent?limit=10 -H "Authorization: Bearer $TOKEN"

# Get by ID
curl $BASE/v1/thoughts/{id} -H "Authorization: Bearer $TOKEN"

# Update (re-embeds if content changes)
curl -X PATCH $BASE/v1/thoughts/{id} \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"content": "updated text", "metadata": {"key": "val"}}'

# Delete single
curl -X DELETE $BASE/v1/thoughts/{id} -H "Authorization: Bearer $TOKEN"

# Bulk delete by filter (source and/or metadata_id required)
curl -X DELETE "$BASE/v1/thoughts?source=agent-name&metadata_id=my-entity" \
  -H "Authorization: Bearer $TOKEN"

# List with filters
curl "$BASE/v1/thoughts?source=agent-name&limit=20" -H "Authorization: Bearer $TOKEN"

# Stats
curl $BASE/v1/stats -H "Authorization: Bearer $TOKEN"
```

## MCP Server (Claude Code)

OpenBrain exposes an MCP server at `/mcp` using FastMCP's **streamable HTTP transport**.

### Registering in Claude Code

**MCPs must be registered in `~/.claude.json` via the CLI — not in `~/.claude/settings.json`.**
`settings.json` does not load MCP servers. Edits there are silently ignored.

```bash
# Register once per machine (user scope = persists across all projects)
claude mcp add --scope user --transport http openbrain https://your-openbrain-host/mcp \
  --header "Authorization: Bearer YOUR_TOKEN"

# Verify it registered
claude mcp list
```

**Transport must be `http`** — not `sse`. FastMCP uses streamable HTTP. `type: "sse"` is a
different (deprecated) protocol and will silently fail to connect.

### Available MCP Tools

Once registered, the following tools are available natively in Claude Code sessions:

| Tool | Description |
|------|-------------|
| `capture` | Store a thought with content, source, and metadata |
| `search` | Semantic search by meaning |
| `recent` | List most recently added thoughts |
| `get` | Retrieve a thought by ID |
| `update` | Update content, source, or metadata (re-embeds if content changes) |
| `delete` | Delete a thought by ID |
| `delete_where` | Bulk delete by source and/or metadata ID |
| `list_thoughts` | List with source/metadata filters and pagination |
| `stats` | Total counts and per-source breakdown |

### When to Use Each Tool

| Trigger | Action |
|---------|--------|
| Session start | `search` or `recent` to load prior context |
| Discovery or gotcha | `capture` immediately |
| Significant decision | `capture` with rationale |
| Task or milestone complete | `capture` summary |
| Cross-agent handoff | `capture` current state before ending |
| Repeated migration | `delete_where` + recapture (upsert pattern) |

## Environment Variables

| Variable | Required | Description |
|----------|----------|-------------|
| `DATABASE_URL` | Yes | PostgreSQL connection string with pgvector |
| `API_KEY` | Yes | Bearer token for authentication |
| `OLLAMA_URL` | Yes | Ollama base URL (e.g. `http://10.1.1.42:11434`) |
| `OLLAMA_MODEL` | No | Embedding model (default: `bge-m3:latest`) |
| `MCP_ALLOWED_HOSTS` | Yes | Comma-separated allowed hostnames for DNS rebinding protection |

## Deployment

### Docker Compose (Portainer / Swarm)

```yaml
services:
  openbrain:
    image: your-registry/openbrain:sha-<digest>
    environment:
      DATABASE_URL: postgresql://user:pass@db:5432/openbrain
      API_KEY: your-api-key
      OLLAMA_URL: http://ollama-host:11434
      MCP_ALLOWED_HOSTS: your-openbrain-host.com
    ports:
      - "8000:8000"
```

See `docker-compose.portainer.yml` for the full Swarm stack definition.

### Database Setup

Requires PostgreSQL with the `vector` extension:

```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE thoughts (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    content TEXT NOT NULL,
    embedding vector(1024),
    source TEXT NOT NULL DEFAULT '',
    metadata JSONB NOT NULL DEFAULT '{}',
    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX ON thoughts USING ivfflat (embedding vector_cosine_ops);
CREATE INDEX ON thoughts (source);
CREATE INDEX ON thoughts ((metadata->>'id'));
```

## Development

```bash
# Install deps
pip install -e ".[dev]"

# Run locally
uvicorn src.main:app --reload

# Lint + format
python -m ruff check src/
python -m ruff format src/

# Tests
pytest
```

## Known Gotchas

- **`capture_async` drops data in CLI scripts** — uses daemon threads that die on process exit.
  Use sync `capture` in any short-lived script context. `capture_async` is only safe in
  long-running processes (servers, daemons).

- **`type: "http"` required for Claude Code MCP** — not `type: "sse"`. They are different
  protocols. SSE is deprecated. Using `sse` against this server silently fails to connect.

- **MCP registration goes in `~/.claude.json`** — not `~/.claude/settings.json`.
  Always use `claude mcp add --scope user` to register. Never hand-edit `mcpServers` in
  `settings.json` — that key is ignored by Claude Code's MCP loader.

- **JSONB filter** — `metadata->>'id'` is exposed as `?metadata_id=` in list/delete endpoints.
  Only the top-level `id` field in metadata is indexed this way.

- **Re-embedding on update** — `PATCH /v1/thoughts/{id}` calls Ollama again only when
  `content` changes. Metadata-only updates skip re-embedding.