feat: initial alpha scaffold — FastAPI + MCP + pgvector
Implements v0.0.1 of OpenBrain: - FastAPI REST API (capture, search, recent, stats) with Bearer auth - MCP server (streamable HTTP at /mcp) exposing all 4 tools - pgvector schema (vector(1024) for bge-m3) - asyncpg connection pool with lazy init + graceful close - Ollama embedding client with fallback (stores thought without vector if Ollama unreachable) - Woodpecker CI pipeline (lint + kaniko build + push to Gitea registry) - Portainer/Swarm deployment compose - Mosaic framework files: AGENTS.md, PRD.md, TASKS.md, scratchpad Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
12
.env.example
Normal file
12
.env.example
Normal file
@@ -0,0 +1,12 @@
|
||||
# Database — update host/credentials for your deployment
|
||||
DATABASE_URL=postgresql://openbrain:changeme@brain-db:5432/openbrain
|
||||
|
||||
# Auth — generate a strong random key: openssl rand -hex 32
|
||||
API_KEY=your-secret-key-here
|
||||
|
||||
# Ollama — point at your Ollama instance
|
||||
OLLAMA_URL=http://your-ollama-host:11434
|
||||
OLLAMA_EMBEDDING_MODEL=bge-m3:latest
|
||||
|
||||
# Service
|
||||
LOG_LEVEL=info
|
||||
11
.gitignore
vendored
Normal file
11
.gitignore
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
.env
|
||||
.env.local
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
dist/
|
||||
*.egg-info/
|
||||
.venv/
|
||||
venv/
|
||||
49
.woodpecker/build.yml
Normal file
49
.woodpecker/build.yml
Normal file
@@ -0,0 +1,49 @@
|
||||
when:
|
||||
- event: push
|
||||
branch: main
|
||||
- event: tag
|
||||
|
||||
variables:
|
||||
- ®istry git.mosaicstack.dev
|
||||
- &image git.mosaicstack.dev/mosaic/openbrain
|
||||
|
||||
steps:
|
||||
lint:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- pip install ruff --quiet
|
||||
- ruff check src/
|
||||
- ruff format --check src/
|
||||
|
||||
build:
|
||||
image: plugins/kaniko
|
||||
settings:
|
||||
registry: *registry
|
||||
repo: *image
|
||||
tags:
|
||||
- sha-${CI_COMMIT_SHA:0:8}
|
||||
- latest
|
||||
username:
|
||||
from_secret: GITEA_USERNAME
|
||||
password:
|
||||
from_secret: GITEA_TOKEN
|
||||
build_args:
|
||||
- BUILDKIT_INLINE_CACHE=1
|
||||
when:
|
||||
- event: push
|
||||
branch: main
|
||||
|
||||
build-tag:
|
||||
image: plugins/kaniko
|
||||
settings:
|
||||
registry: *registry
|
||||
repo: *image
|
||||
tags:
|
||||
- ${CI_COMMIT_TAG}
|
||||
- sha-${CI_COMMIT_SHA:0:8}
|
||||
username:
|
||||
from_secret: GITEA_USERNAME
|
||||
password:
|
||||
from_secret: GITEA_TOKEN
|
||||
when:
|
||||
- event: tag
|
||||
87
AGENTS.md
Normal file
87
AGENTS.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# OpenBrain — Agent Guidelines
|
||||
|
||||
> **Purpose**: Self-hosted semantic brain — pgvector + MCP server for any AI agent
|
||||
> **SSOT**: https://git.mosaicstack.dev/mosaic/openbrain
|
||||
> **Status**: Alpha (0.0.1)
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Tech |
|
||||
|-------|------|
|
||||
| Language | Python 3.12 |
|
||||
| API | FastAPI + uvicorn |
|
||||
| MCP | `mcp[cli]` Python SDK (streamable HTTP transport) |
|
||||
| Database | PostgreSQL 17 + pgvector |
|
||||
| Embeddings | Ollama (`bge-m3:latest`, 1024-dim) |
|
||||
| CI/CD | Woodpecker → Gitea registry |
|
||||
| Deployment | Docker Swarm via Portainer |
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
src/
|
||||
config.py — env-based settings (pydantic-settings)
|
||||
db.py — asyncpg connection pool
|
||||
embeddings.py — Ollama embedding client
|
||||
models.py — Pydantic request/response models
|
||||
brain.py — core operations (capture, search, recent, stats)
|
||||
main.py — FastAPI app + MCP server mount
|
||||
docker/
|
||||
postgres/init.sql — schema + pgvector setup
|
||||
.woodpecker/
|
||||
build.yml — lint → kaniko build → push
|
||||
```
|
||||
|
||||
## Key Rules
|
||||
|
||||
1. **Never hardcode secrets, IPs, or internal hostnames.** All config via env vars.
|
||||
2. **Public repo.** `.env` is gitignored. `.env.example` has placeholders only.
|
||||
3. **MCP transport is Streamable HTTP** mounted at `/mcp`. Not stdio.
|
||||
4. **REST + MCP live in one process** (`src/main.py`). No separate MCP container.
|
||||
5. **Schema is append-only** in alpha. Migrations via new SQL files in `docker/postgres/`.
|
||||
6. **Embeddings are best-effort**: if Ollama is unreachable, thought is stored without embedding.
|
||||
|
||||
## Auth
|
||||
|
||||
All REST endpoints require: `Authorization: Bearer <API_KEY>`
|
||||
|
||||
MCP server at `/mcp` uses the same key via MCP client config headers.
|
||||
|
||||
## Local Dev
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# Fill in DATABASE_URL, API_KEY, OLLAMA_URL
|
||||
|
||||
uv pip install -e ".[dev]"
|
||||
uvicorn src.main:app --reload
|
||||
```
|
||||
|
||||
## CI/CD
|
||||
|
||||
Push to `main` → Woodpecker lints + builds image → pushes `sha-<hash>` + `latest` tags.
|
||||
Tag a release → pushes `v0.0.x` + `sha-<hash>` tags.
|
||||
|
||||
## Deployment
|
||||
|
||||
Use `docker-compose.portainer.yml` as a Portainer stack.
|
||||
Required env vars: `POSTGRES_PASSWORD`, `API_KEY`, `OLLAMA_URL`, `IMAGE_TAG`.
|
||||
Init SQL must be copied to host at `/opt/openbrain/init.sql` before first deploy.
|
||||
|
||||
## MCP Client Config (Claude Code)
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"openbrain": {
|
||||
"type": "http",
|
||||
"url": "https://brain.woltje.com/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer <API_KEY>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
20
Dockerfile
Normal file
20
Dockerfile
Normal file
@@ -0,0 +1,20 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install uv for fast dependency installation
|
||||
RUN pip install uv --no-cache-dir
|
||||
|
||||
# Copy dependency spec first for layer caching
|
||||
COPY pyproject.toml .
|
||||
RUN uv pip install --system --no-cache .
|
||||
|
||||
# Copy source
|
||||
COPY src/ ./src/
|
||||
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
ENV PYTHONPATH=/app
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
65
docker-compose.portainer.yml
Normal file
65
docker-compose.portainer.yml
Normal file
@@ -0,0 +1,65 @@
|
||||
# OpenBrain — Portainer / Docker Swarm deployment
|
||||
#
|
||||
# Required environment variables (set in Portainer stack env):
|
||||
# POSTGRES_PASSWORD — postgres user password
|
||||
# API_KEY — secret key for API/MCP auth
|
||||
# OLLAMA_URL — Ollama endpoint (e.g. http://10.x.x.x:11434)
|
||||
# IMAGE_TAG — image tag to deploy (e.g. sha-abc1234 or 0.0.1)
|
||||
#
|
||||
# Optional:
|
||||
# OLLAMA_EMBEDDING_MODEL — default: bge-m3:latest
|
||||
# LOG_LEVEL — default: info
|
||||
|
||||
services:
|
||||
brain-db:
|
||||
image: pgvector/pgvector:pg17
|
||||
environment:
|
||||
POSTGRES_USER: openbrain
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: openbrain
|
||||
volumes:
|
||||
- brain_db_data:/var/lib/postgresql/data
|
||||
- /opt/openbrain/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U openbrain -d openbrain"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
networks:
|
||||
- brain-internal
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
|
||||
brain-api:
|
||||
image: git.mosaicstack.dev/mosaic/openbrain:${IMAGE_TAG:-latest}
|
||||
environment:
|
||||
DATABASE_URL: postgresql://openbrain:${POSTGRES_PASSWORD}@brain-db:5432/openbrain
|
||||
API_KEY: ${API_KEY}
|
||||
OLLAMA_URL: ${OLLAMA_URL}
|
||||
OLLAMA_EMBEDDING_MODEL: ${OLLAMA_EMBEDDING_MODEL:-bge-m3:latest}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-info}
|
||||
ports:
|
||||
- "8765:8000"
|
||||
depends_on:
|
||||
- brain-db
|
||||
networks:
|
||||
- brain-internal
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.openbrain.rule=Host(`brain.woltje.com`)"
|
||||
- "traefik.http.routers.openbrain.entrypoints=websecure"
|
||||
- "traefik.http.routers.openbrain.tls=true"
|
||||
- "traefik.http.services.openbrain.loadbalancer.server.port=8000"
|
||||
|
||||
volumes:
|
||||
brain_db_data:
|
||||
|
||||
networks:
|
||||
brain-internal:
|
||||
driver: overlay
|
||||
28
docker/postgres/init.sql
Normal file
28
docker/postgres/init.sql
Normal file
@@ -0,0 +1,28 @@
|
||||
-- OpenBrain — Database Initialization
|
||||
-- Runs once on first container start
|
||||
|
||||
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS thoughts (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
content TEXT NOT NULL,
|
||||
embedding vector(1024), -- bge-m3 native dimension
|
||||
source VARCHAR(100) NOT NULL DEFAULT 'unknown',
|
||||
metadata JSONB NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Vector similarity search index (cosine)
|
||||
CREATE INDEX IF NOT EXISTS thoughts_embedding_idx
|
||||
ON thoughts USING ivfflat (embedding vector_cosine_ops)
|
||||
WITH (lists = 100);
|
||||
|
||||
-- Recent queries
|
||||
CREATE INDEX IF NOT EXISTS thoughts_created_at_idx
|
||||
ON thoughts (created_at DESC);
|
||||
|
||||
-- Filter by source
|
||||
CREATE INDEX IF NOT EXISTS thoughts_source_idx
|
||||
ON thoughts (source);
|
||||
70
docs/PRD.md
Normal file
70
docs/PRD.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# OpenBrain — Product Requirements Document
|
||||
|
||||
**Version**: 0.0.1
|
||||
**Status**: Active
|
||||
**Owner**: Jason Woltje
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
AI agents and tools have no shared persistent memory. Every session starts from zero.
|
||||
Platform memory (Claude, ChatGPT, etc.) is siloed — each tool can't see what the others know.
|
||||
This forces constant context re-injection, burns tokens, and prevents compounding knowledge.
|
||||
|
||||
## Goal
|
||||
|
||||
A self-hosted, agent-readable semantic brain that any AI tool can plug into via MCP.
|
||||
One database. Standard protocol. Owned infrastructure. No SaaS middlemen.
|
||||
|
||||
## Users
|
||||
|
||||
1. **Jason** — primary human user, captures thoughts from any AI tool
|
||||
2. **AI agents** — Claude Code, Codex, Claude Desktop, any MCP-compatible client
|
||||
3. **Future**: Mosaic Stack integration as the knowledge layer for the agent fleet
|
||||
|
||||
## Requirements
|
||||
|
||||
### v0.0.1 (Alpha — Current)
|
||||
|
||||
| ID | Requirement | Priority |
|
||||
|----|-------------|----------|
|
||||
| R1 | Capture a thought with content, source, and metadata | Must |
|
||||
| R2 | Generate vector embedding via Ollama (bge-m3) | Must |
|
||||
| R3 | Semantic search by meaning (cosine similarity) | Must |
|
||||
| R4 | List recent thoughts | Must |
|
||||
| R5 | Usage stats (total, embedded, by source) | Must |
|
||||
| R6 | REST API with Bearer token auth | Must |
|
||||
| R7 | MCP server (streamable HTTP) exposing all 4 tools | Must |
|
||||
| R8 | Deployable as Portainer/Swarm stack | Must |
|
||||
| R9 | CI/CD via Woodpecker (lint + build + push) | Must |
|
||||
| R10 | Graceful embedding fallback (store without vector if Ollama down) | Must |
|
||||
| R11 | Public repo — zero secrets in code | Must |
|
||||
|
||||
### v0.1.0 (Future)
|
||||
|
||||
- Thought tagging and tag-based filtering
|
||||
- Batch import (ingest jarvis-brain data, Claude memory, etc.)
|
||||
- Scheduled re-embedding for thoughts stored without vectors
|
||||
- Webhook capture endpoint (ingest from any tool without MCP)
|
||||
- Usage dashboard (thoughts/day, source breakdown)
|
||||
- Mosaic Stack integration (knowledge module backend)
|
||||
|
||||
## Acceptance Criteria (v0.0.1)
|
||||
|
||||
1. `POST /v1/thoughts` stores a thought and returns it with embedded=true when Ollama is reachable
|
||||
2. `POST /v1/search` with a natural-language query returns semantically relevant results
|
||||
3. `GET /v1/thoughts/recent` returns the last N thoughts in reverse chronological order
|
||||
4. `GET /v1/stats` returns total count, embedded count, and source breakdown
|
||||
5. MCP server at `/mcp` exposes all 4 tools (capture, search, recent, stats)
|
||||
6. Claude Code can connect to the MCP server and execute all 4 tools
|
||||
7. Portainer stack deploys both brain-db and brain-api successfully
|
||||
8. CI pipeline runs on push to main and produces a tagged image
|
||||
|
||||
## Out of Scope (v0.0.1)
|
||||
|
||||
- User accounts / multi-user
|
||||
- Workspace isolation
|
||||
- Web UI
|
||||
- Rate limiting
|
||||
- Mosaic Stack integration
|
||||
28
docs/TASKS.md
Normal file
28
docs/TASKS.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# OpenBrain — Tasks
|
||||
|
||||
**Project**: openbrain
|
||||
**Provider**: https://git.mosaicstack.dev/mosaic/openbrain
|
||||
|
||||
---
|
||||
|
||||
## Active
|
||||
|
||||
| ID | Title | Status | Notes |
|
||||
|----|-------|--------|-------|
|
||||
| T1 | Scaffold repo + core service | in-progress | Building now |
|
||||
| T2 | CI/CD pipeline (Woodpecker) | in-progress | Building now |
|
||||
| T3 | Portainer deployment | pending | Follows T1, T2 |
|
||||
| T4 | Copy init.sql to host, deploy stack | pending | Requires server access |
|
||||
| T5 | Configure MCP in Claude Code settings | pending | Follows T3 |
|
||||
| T6 | Smoke test: capture + search via MCP | pending | Follows T5 |
|
||||
|
||||
## Backlog
|
||||
|
||||
| ID | Title | Notes |
|
||||
|----|-------|-------|
|
||||
| T10 | Woodpecker CI secrets setup (GITEA_USERNAME, GITEA_TOKEN) | Required for build pipeline |
|
||||
| T11 | DNS: brain.woltje.com → Swarm ingress | Required for HTTPS access |
|
||||
| T12 | Traefik TLS cert for brain.woltje.com | Required for HTTPS MCP |
|
||||
| T20 | Batch import: ingest jarvis-brain JSON data | v0.1.0 |
|
||||
| T21 | Scheduled re-embedding for non-embedded thoughts | v0.1.0 |
|
||||
| T22 | Mosaic Stack knowledge module integration | v0.1.0+ |
|
||||
41
docs/scratchpads/v001-build.md
Normal file
41
docs/scratchpads/v001-build.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Scratchpad: v0.0.1 Build
|
||||
|
||||
**Date**: 2026-03-02
|
||||
**Objective**: Build and deploy alpha OpenBrain service
|
||||
|
||||
## Plan
|
||||
|
||||
1. [x] Scaffold project structure
|
||||
2. [x] Core brain operations (capture, search, recent, stats)
|
||||
3. [x] FastAPI REST + MCP server (single process)
|
||||
4. [x] pgvector schema
|
||||
5. [x] Dockerfile
|
||||
6. [x] Portainer compose
|
||||
7. [x] Woodpecker CI pipeline
|
||||
8. [x] Mosaic framework files (AGENTS.md, PRD.md, TASKS.md)
|
||||
9. [ ] Initial commit + push
|
||||
10. [ ] Woodpecker secrets verified
|
||||
11. [ ] DNS + Traefik config for brain.woltje.com
|
||||
12. [ ] Host init.sql copy + Portainer stack deploy
|
||||
13. [ ] Smoke test via MCP
|
||||
|
||||
## Decisions
|
||||
|
||||
- Single Python process for REST + MCP (avoids 2-container overhead for alpha)
|
||||
- Streamable HTTP MCP transport (not stdio — deployed service, needs HTTP)
|
||||
- bge-m3 via existing Ollama at 10.1.1.42 (verified live)
|
||||
- vector(1024) — bge-m3 native, no padding
|
||||
- Graceful fallback: thoughts stored without embedding if Ollama unreachable
|
||||
- pgvector/pgvector:pg17 official image — no custom build needed
|
||||
|
||||
## Blockers / Notes
|
||||
|
||||
- Woodpecker CI secrets (GITEA_USERNAME, GITEA_TOKEN) must be set for build pipeline
|
||||
- DNS record for brain.woltje.com needs to be created
|
||||
- Init SQL must be on host at /opt/openbrain/init.sql before first Portainer deploy
|
||||
- MCP auth: headers passed via Claude Code settings — confirm MCP SDK accepts headers on streamable HTTP
|
||||
|
||||
## Risks
|
||||
|
||||
- MCP streamable HTTP transport is newer spec — need to verify Claude Code supports it
|
||||
- Fallback: switch to SSE transport (mcp.server.sse.SseServerTransport)
|
||||
32
pyproject.toml
Normal file
32
pyproject.toml
Normal file
@@ -0,0 +1,32 @@
|
||||
[project]
|
||||
name = "openbrain"
|
||||
version = "0.0.1"
|
||||
description = "Self-hosted semantic brain — pgvector + MCP server for any AI agent"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"fastapi>=0.115.0",
|
||||
"uvicorn[standard]>=0.32.0",
|
||||
"asyncpg>=0.30.0",
|
||||
"httpx>=0.28.0",
|
||||
"pydantic>=2.10.0",
|
||||
"pydantic-settings>=2.7.0",
|
||||
"mcp[cli]>=1.6.0",
|
||||
"python-multipart>=0.0.20",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0.0",
|
||||
"pytest-asyncio>=0.24.0",
|
||||
"ruff>=0.8.0",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
asyncio_mode = "auto"
|
||||
0
src/__init__.py
Normal file
0
src/__init__.py
Normal file
127
src/brain.py
Normal file
127
src/brain.py
Normal file
@@ -0,0 +1,127 @@
|
||||
"""Core brain operations — capture, search, recent, stats."""
|
||||
import json
|
||||
from src import db, embeddings
|
||||
from src.models import CaptureRequest, Thought, SearchRequest, SearchResult, Stats
|
||||
|
||||
|
||||
async def capture(req: CaptureRequest) -> Thought:
|
||||
pool = await db.get_pool()
|
||||
embedding = await embeddings.embed(req.content)
|
||||
|
||||
async with pool.acquire() as conn:
|
||||
if embedding is not None:
|
||||
vec = f"[{','.join(str(v) for v in embedding)}]"
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO thoughts (content, embedding, source, metadata)
|
||||
VALUES ($1, $2::vector, $3, $4::jsonb)
|
||||
RETURNING id::text, content, source, metadata, created_at, embedding IS NOT NULL AS embedded
|
||||
""",
|
||||
req.content, vec, req.source, json.dumps(req.metadata),
|
||||
)
|
||||
else:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO thoughts (content, source, metadata)
|
||||
VALUES ($1, $2, $3::jsonb)
|
||||
RETURNING id::text, content, source, metadata, created_at, embedding IS NOT NULL AS embedded
|
||||
""",
|
||||
req.content, req.source, json.dumps(req.metadata),
|
||||
)
|
||||
|
||||
return Thought(
|
||||
id=row["id"],
|
||||
content=row["content"],
|
||||
source=row["source"],
|
||||
metadata=json.loads(row["metadata"]) if isinstance(row["metadata"], str) else row["metadata"],
|
||||
created_at=row["created_at"],
|
||||
embedded=row["embedded"],
|
||||
)
|
||||
|
||||
|
||||
async def search(req: SearchRequest) -> list[SearchResult]:
|
||||
embedding = await embeddings.embed(req.query)
|
||||
if embedding is None:
|
||||
return []
|
||||
|
||||
pool = await db.get_pool()
|
||||
vec = f"[{','.join(str(v) for v in embedding)}]"
|
||||
|
||||
async with pool.acquire() as conn:
|
||||
if req.source:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id::text, content, source, metadata, created_at,
|
||||
1 - (embedding <=> $1::vector) AS similarity
|
||||
FROM thoughts
|
||||
WHERE embedding IS NOT NULL AND source = $2
|
||||
ORDER BY embedding <=> $1::vector
|
||||
LIMIT $3
|
||||
""",
|
||||
vec, req.source, req.limit,
|
||||
)
|
||||
else:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id::text, content, source, metadata, created_at,
|
||||
1 - (embedding <=> $1::vector) AS similarity
|
||||
FROM thoughts
|
||||
WHERE embedding IS NOT NULL
|
||||
ORDER BY embedding <=> $1::vector
|
||||
LIMIT $2
|
||||
""",
|
||||
vec, req.limit,
|
||||
)
|
||||
|
||||
return [
|
||||
SearchResult(
|
||||
id=r["id"],
|
||||
content=r["content"],
|
||||
source=r["source"],
|
||||
similarity=float(r["similarity"]),
|
||||
created_at=r["created_at"],
|
||||
metadata=json.loads(r["metadata"]) if isinstance(r["metadata"], str) else r["metadata"],
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
|
||||
async def recent(limit: int = 20) -> list[Thought]:
|
||||
pool = await db.get_pool()
|
||||
async with pool.acquire() as conn:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id::text, content, source, metadata, created_at,
|
||||
embedding IS NOT NULL AS embedded
|
||||
FROM thoughts
|
||||
ORDER BY created_at DESC
|
||||
LIMIT $1
|
||||
""",
|
||||
limit,
|
||||
)
|
||||
return [
|
||||
Thought(
|
||||
id=r["id"],
|
||||
content=r["content"],
|
||||
source=r["source"],
|
||||
metadata=json.loads(r["metadata"]) if isinstance(r["metadata"], str) else r["metadata"],
|
||||
created_at=r["created_at"],
|
||||
embedded=r["embedded"],
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
|
||||
async def stats() -> Stats:
|
||||
pool = await db.get_pool()
|
||||
async with pool.acquire() as conn:
|
||||
total = await conn.fetchval("SELECT COUNT(*) FROM thoughts")
|
||||
embedded = await conn.fetchval("SELECT COUNT(*) FROM thoughts WHERE embedding IS NOT NULL")
|
||||
sources = await conn.fetch(
|
||||
"SELECT source, COUNT(*) AS count FROM thoughts GROUP BY source ORDER BY count DESC"
|
||||
)
|
||||
return Stats(
|
||||
total_thoughts=total,
|
||||
embedded_count=embedded,
|
||||
sources=[{"source": r["source"], "count": r["count"]} for r in sources],
|
||||
)
|
||||
23
src/config.py
Normal file
23
src/config.py
Normal file
@@ -0,0 +1,23 @@
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8")
|
||||
|
||||
# Database
|
||||
database_url: str = "postgresql://openbrain:openbrain@localhost:5432/openbrain"
|
||||
|
||||
# Auth
|
||||
api_key: str # Required — no default, must be set
|
||||
|
||||
# Ollama
|
||||
ollama_url: str = "http://localhost:11434"
|
||||
ollama_embedding_model: str = "bge-m3:latest"
|
||||
|
||||
# Service
|
||||
host: str = "0.0.0.0"
|
||||
port: int = 8000
|
||||
log_level: str = "info"
|
||||
|
||||
|
||||
settings = Settings()
|
||||
18
src/db.py
Normal file
18
src/db.py
Normal file
@@ -0,0 +1,18 @@
|
||||
import asyncpg
|
||||
from src.config import settings
|
||||
|
||||
_pool: asyncpg.Pool | None = None
|
||||
|
||||
|
||||
async def get_pool() -> asyncpg.Pool:
|
||||
global _pool
|
||||
if _pool is None:
|
||||
_pool = await asyncpg.create_pool(settings.database_url, min_size=2, max_size=10)
|
||||
return _pool
|
||||
|
||||
|
||||
async def close_pool() -> None:
|
||||
global _pool
|
||||
if _pool:
|
||||
await _pool.close()
|
||||
_pool = None
|
||||
16
src/embeddings.py
Normal file
16
src/embeddings.py
Normal file
@@ -0,0 +1,16 @@
|
||||
import httpx
|
||||
from src.config import settings
|
||||
|
||||
|
||||
async def embed(text: str) -> list[float] | None:
|
||||
"""Generate embedding via Ollama. Returns None if Ollama is unreachable."""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.post(
|
||||
f"{settings.ollama_url}/api/embeddings",
|
||||
json={"model": settings.ollama_embedding_model, "prompt": text},
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()["embedding"]
|
||||
except Exception:
|
||||
return None
|
||||
126
src/main.py
Normal file
126
src/main.py
Normal file
@@ -0,0 +1,126 @@
|
||||
"""OpenBrain — FastAPI REST + MCP server (single process)."""
|
||||
import contextlib
|
||||
import logging
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException, Security
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
|
||||
from src import brain, db
|
||||
from src.config import settings
|
||||
from src.models import CaptureRequest, SearchRequest, SearchResult, Stats, Thought
|
||||
|
||||
logging.basicConfig(level=settings.log_level.upper())
|
||||
logger = logging.getLogger("openbrain")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth
|
||||
# ---------------------------------------------------------------------------
|
||||
bearer = HTTPBearer()
|
||||
|
||||
|
||||
def require_api_key(credentials: HTTPAuthorizationCredentials = Security(bearer)) -> str:
|
||||
if credentials.credentials != settings.api_key:
|
||||
raise HTTPException(status_code=401, detail="Invalid API key")
|
||||
return credentials.credentials
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP server
|
||||
# ---------------------------------------------------------------------------
|
||||
mcp = FastMCP("openbrain", stateless_http=True)
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def capture(content: str, source: str = "unknown", metadata: dict | None = None) -> dict:
|
||||
"""Store a thought or piece of information in your brain.
|
||||
|
||||
Args:
|
||||
content: The text to remember
|
||||
source: Which agent or tool is capturing this (e.g. 'claude-code', 'codex')
|
||||
metadata: Optional key/value pairs (tags, project, etc.)
|
||||
"""
|
||||
thought = await brain.capture(CaptureRequest(content=content, source=source, metadata=metadata or {}))
|
||||
return thought.model_dump(mode="json")
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def search(query: str, limit: int = 10, source: str | None = None) -> list[dict]:
|
||||
"""Search your brain by meaning (semantic search).
|
||||
|
||||
Args:
|
||||
query: What you're looking for — describe it naturally
|
||||
limit: Max results to return (default 10)
|
||||
source: Optional — filter to a specific agent/tool
|
||||
"""
|
||||
results = await brain.search(SearchRequest(query=query, limit=limit, source=source))
|
||||
return [r.model_dump(mode="json") for r in results]
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def recent(limit: int = 20) -> list[dict]:
|
||||
"""Get recently captured thoughts.
|
||||
|
||||
Args:
|
||||
limit: How many to return (default 20)
|
||||
"""
|
||||
thoughts = await brain.recent(limit=limit)
|
||||
return [t.model_dump(mode="json") for t in thoughts]
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def stats() -> dict:
|
||||
"""Get statistics about your brain — total thoughts, embedding coverage, sources."""
|
||||
s = await brain.stats()
|
||||
return s.model_dump(mode="json")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# FastAPI app
|
||||
# ---------------------------------------------------------------------------
|
||||
@contextlib.asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
logger.info("OpenBrain starting up")
|
||||
await db.get_pool() # Warm the connection pool
|
||||
yield
|
||||
await db.close_pool()
|
||||
logger.info("OpenBrain shut down")
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="OpenBrain",
|
||||
description="Self-hosted semantic brain — pgvector + MCP for any AI agent",
|
||||
version="0.0.1",
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
# Mount MCP server at /mcp (HTTP streamable transport)
|
||||
app.mount("/mcp", mcp.streamable_http_app())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# REST endpoints (for direct API access and health checks)
|
||||
# ---------------------------------------------------------------------------
|
||||
@app.get("/health")
|
||||
async def health() -> dict:
|
||||
return {"status": "ok", "version": "0.0.1"}
|
||||
|
||||
|
||||
@app.post("/v1/thoughts", response_model=Thought)
|
||||
async def api_capture(req: CaptureRequest, _: str = Depends(require_api_key)) -> Thought:
|
||||
return await brain.capture(req)
|
||||
|
||||
|
||||
@app.post("/v1/search", response_model=list[SearchResult])
|
||||
async def api_search(req: SearchRequest, _: str = Depends(require_api_key)) -> list[SearchResult]:
|
||||
return await brain.search(req)
|
||||
|
||||
|
||||
@app.get("/v1/thoughts/recent", response_model=list[Thought])
|
||||
async def api_recent(limit: int = 20, _: str = Depends(require_api_key)) -> list[Thought]:
|
||||
return await brain.recent(limit=limit)
|
||||
|
||||
|
||||
@app.get("/v1/stats", response_model=Stats)
|
||||
async def api_stats(_: str = Depends(require_api_key)) -> Stats:
|
||||
return await brain.stats()
|
||||
39
src/models.py
Normal file
39
src/models.py
Normal file
@@ -0,0 +1,39 @@
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class CaptureRequest(BaseModel):
|
||||
content: str
|
||||
source: str = "unknown"
|
||||
metadata: dict[str, Any] = {}
|
||||
|
||||
|
||||
class Thought(BaseModel):
|
||||
id: str
|
||||
content: str
|
||||
source: str
|
||||
metadata: dict[str, Any]
|
||||
created_at: datetime
|
||||
embedded: bool
|
||||
|
||||
|
||||
class SearchRequest(BaseModel):
|
||||
query: str
|
||||
limit: int = 10
|
||||
source: str | None = None
|
||||
|
||||
|
||||
class SearchResult(BaseModel):
|
||||
id: str
|
||||
content: str
|
||||
source: str
|
||||
similarity: float
|
||||
created_at: datetime
|
||||
metadata: dict[str, Any]
|
||||
|
||||
|
||||
class Stats(BaseModel):
|
||||
total_thoughts: int
|
||||
embedded_count: int
|
||||
sources: list[dict[str, Any]]
|
||||
Reference in New Issue
Block a user