stack/docs/guides/dev-guide.md

Mosaic Stack — Developer Guide

Table of Contents

1. Architecture Overview
2. Local Development Setup
3. Building and Testing
4. Adding New Agent Tools
5. Adding New MCP Tools
6. Database Schema and Migrations
7. Claude Code Skill Bridge
8. Pi Persistent Goal Extension
9. API Endpoint Reference
10. Local Fleet Canary

---

Architecture Overview

Mosaic Stack is a TypeScript monorepo managed with pnpm workspaces and Turborepo.

mosaic-mono-v1/
├── apps/
│   ├── gateway/          # NestJS + Fastify API server
│   └── web/              # Next.js 16 + React 19 web dashboard
├── packages/
│   ├── agent/            # Agent session types (shared)
│   ├── auth/             # BetterAuth configuration
│   ├── brain/            # Structured data layer (projects, tasks, missions)
│   ├── cli/              # mosaic CLI and TUI (Ink)
│   ├── coord/            # Mission coordination engine
│   ├── db/               # Drizzle ORM schema, migrations, client
│   ├── design-tokens/    # Shared design system tokens
│   ├── log/              # Agent log ingestion and tiering
│   ├── memory/           # Preference and insight storage
│   ├── mosaic/           # Install wizard and bootstrap utilities
│   ├── prdy/             # PRD wizard CLI
│   ├── quality-rails/    # Code quality scaffolder CLI
│   ├── queue/            # Valkey-backed task queue
│   └── types/            # Shared TypeScript types
├── docker/               # Dockerfile(s) for containerized deployment
├── infra/                # Infrastructure configuration (for example, OTEL collector)
├── docker-compose.yml    # Local services (Postgres, Valkey, OTEL, Jaeger)
└── CLAUDE.md             # Project conventions for AI coding agents

Key Technology Choices

Concern	Technology
API framework	NestJS with Fastify adapter
Web framework	Next.js 16 (App Router), React 19
ORM	Drizzle ORM
Database	PostgreSQL 17 + pgvector extension
Auth	BetterAuth
Agent harness	Pi SDK (@mariozechner/pi-coding-agent)
Queue	Valkey 8 (Redis-compatible)
Build	pnpm workspaces + Turborepo
CI	Woodpecker CI
Observability	OpenTelemetry → Jaeger
Module resolution	NodeNext (ESM everywhere)

Module System

All packages use "type": "module" and NodeNext resolution. Import paths must include the .js extension even when the source file is .ts.

NestJS @Inject() decorators must be used explicitly because tsx/esbuild does not support emitDecoratorMetadata.

---

Local Development Setup

Prerequisites

• Node.js 20+
• pnpm 9+
• Docker and Docker Compose

1. Clone and Install Dependencies

git clone <repo-url> mosaic-mono-v1
cd mosaic-mono-v1
pnpm install

2. Use the local PGlite tier

The supported local tier is in-process PGlite and requires no PostgreSQL service. Leave DATABASE_URL unset for this route. Its default local configuration uses PGlite and performs no external database probe.

If a local queue service is useful, start only that non-PostgreSQL service:

docker compose up -d valkey

Do not use the current Compose PostgreSQL service: it mounts legacy infra/pg-init SQL and is not qualified for KBN-101. Start OTEL Collector or Jaeger individually only when needed and without starting PostgreSQL.

3. Gateway/Web local process (held)

Do not start the current Gateway or web process as a local PGlite route. Gateway first loads the daemon configuration and then project environment files without a tier guard; a pre-existing DATABASE_URL can select PostgreSQL, where current startup still reaches runtime DDL/migrations. Creating a root .env that omits DATABASE_URL does not make this safe, so neither a local credential file nor a web environment file is a current developer procedure.

PGlite remains the supported in-process data-layer implementation, and the optional Valkey command above remains safe because it does not start PostgreSQL. A safe Gateway/Web local procedure is held until KBN-101-02 rejects a daemon, inherited, root, or app-local PostgreSQL DSN and any non-local tier before connection or DDL; KBN-101-05 then supplies the production renderer/Vault process-exec or LoadCredential boundary.

Held future procedure

PostgreSQL local and federated deployment are held until KBN-101-00 (external bootstrap), KBN-101-03 (runner), and KBN-101-05 (renderer-backed deployment) land. The following is the held, non-operative future activation order with no current command authority:

external bootstrap → TLS/roles → mosaic-db-migrator --run → mosaic-db-migrator --verify → Gateway/Compose readiness.

Neither current Compose nor this development guide authorizes PostgreSQL initialization SQL, manual DDL, or a pre-runner start.

5. Gateway/Web start (held)

No Gateway/Web start command is currently authorized for the local PGlite route. Do not use root pnpm dev as a workaround: it additionally starts configured integrations and cannot establish the required local-tier/DSN isolation. Resume this section only after KBN-101-02 provides its fail-closed local-startup evidence.

---

Building and Testing

TypeScript Typecheck

pnpm typecheck

Runs tsc --noEmit across all packages in dependency order via Turborepo.

Lint

pnpm lint

Runs ESLint across all packages. Config is in eslint.config.mjs at the root.

Format Check

pnpm format:check

Runs Prettier in check mode. To auto-fix:

pnpm format

Tests

pnpm test

Runs Vitest across all packages. The workspace config is at vitest.workspace.ts.

Build

pnpm build

Builds all packages and apps in dependency order.

Pre-Push Gates (MANDATORY)

All three must pass before any push:

pnpm format:check && pnpm typecheck && pnpm lint

A pre-push hook enforces this mechanically.

---

Adding New Agent Tools

Agent tools are Pi SDK ToolDefinition objects registered in apps/gateway/src/agent/agent.service.ts.

1. Create a Tool Factory File

Add a new file in apps/gateway/src/agent/tools/:

// apps/gateway/src/agent/tools/my-tools.ts
import { Type } from '@sinclair/typebox';
import type { ToolDefinition } from '@mariozechner/pi-coding-agent';

export function createMyTools(): ToolDefinition[] {
  const myTool: ToolDefinition = {
    name: 'my_tool_name',
    label: 'Human Readable Label',
    description: 'What this tool does.',
    parameters: Type.Object({
      input: Type.String({ description: 'The input parameter' }),
    }),
    async execute(_toolCallId, params) {
      const { input } = params as { input: string };
      const result = `Processed: ${input}`;
      return {
        content: [{ type: 'text' as const, text: result }],
        details: undefined,
      };
    },
  };

  return [myTool];
}

2. Register the Tools in AgentService

In apps/gateway/src/agent/agent.service.ts, import and call your factory alongside the existing tool registrations:

import { createMyTools } from './tools/my-tools.js';

// Inside the session creation logic where tools are assembled:
const tools: ToolDefinition[] = [
  ...createBrainTools(this.brain),
  ...createCoordTools(this.coordService),
  ...createMemoryTools(this.memory, this.embeddingService),
  ...createFileTools(sandboxDir),
  ...createGitTools(sandboxDir),
  ...createShellTools(sandboxDir),
  ...createWebTools(),
  ...createMyTools(), // Add this line
  ...mcpTools,
  ...skillTools,
];

3. Export from the Tools Index

Add an export to apps/gateway/src/agent/tools/index.ts:

export { createMyTools } from './my-tools.js';

4. Typecheck and Test

pnpm typecheck
pnpm test

---

Adding New MCP Tools

Mosaic connects to external MCP servers via McpClientService. To expose tools from a new MCP server:

1. Run an MCP Server

Implement a standard MCP server that exposes tools via the streamable HTTP transport or SSE transport. The server must accept connections at a /mcp endpoint.

2. Gateway MCP configuration (held)

Do not configure MCP endpoint credentials, write them to a local environment file, or restart the Gateway from this guide. Gateway/Web startup is held until KBN-101-02 supplies fail-closed local-tier/DSN isolation and KBN-101-05 supplies the renderer/Vault process-exec or LoadCredential secret-consumer interface. The future authenticated MCP route requires verified HTTPS and certificate validation; plaintext bearer-token examples are forbidden.

Tool Naming

Bridged MCP tool names are taken directly from the MCP server's tool manifest. Ensure names do not conflict with built-in tools (check apps/gateway/src/agent/tools/).

---

Database Schema and Migrations

The schema lives in a single file: packages/db/src/schema.ts

Schema Overview

Table	Purpose
users	User accounts (BetterAuth-compatible)
sessions	Auth sessions
accounts	OAuth accounts
verifications	Email verification tokens
projects	Project records
missions	Mission records (linked to projects)
tasks	Task records (linked to projects and/or missions)
conversations	Chat conversation metadata
messages	Individual chat messages
preferences	Per-user key-value preference store
insights	Vector-embedded memory insights
agent_logs	Agent interaction logs (hot/warm/cold tiers)
skills	Installed agent skills
summarization_jobs	Log summarization job tracking

The insights table uses a vector(1536) column (pgvector) for semantic search.

PostgreSQL schema work (held)

Do not prepare or run a PostgreSQL target from this branch. The sole runner, bootstrap, and renderer are future KBN-101 artifacts, not current commands. When KBN-101-00/-03/-05 land, the owned activation documentation will require external bootstrap → TLS/roles → runner --run → runner --verify → Gateway/Compose readiness.

Generating migration artifacts

pnpm --filter @mosaicstack/db db:generate is an offline artifact-generation command. It does not authorize connecting to or initializing PostgreSQL. A future reviewed PostgreSQL procedure will determine when its output is applied.

Drizzle Config

Config is at packages/db/drizzle.config.ts. The schema file path and output directory are defined there.

Adding a New Table

1. Add the table definition to packages/db/src/schema.ts.
2. Export it from packages/db/src/index.ts.
3. Generate the offline artifact with pnpm --filter @mosaicstack/db db:generate.
4. Do not apply it to PostgreSQL until the future KBN-101 activation artifacts and their owned procedure are available. Direct schema push is not a production-like workflow.

---

Claude Code Skill Bridge

The framework's canonical skill root is ~/.config/mosaic/skills/; Claude Code requires registrations under ~/.claude/skills/. The implementation in packages/mosaic/src/commands/skill.ts owns only direct-child symlinks whose resolved target remains inside the canonical root.

Security invariants:

1. Validate the user-supplied name before filesystem access against [A-Za-z0-9][A-Za-z0-9._-]*. Separators, control characters, whitespace, .., absolute paths, and leading - are invalid; filesystem-derived invalid names are escaped before terminal output.
2. Never replace a real file, directory, foreign symlink, or live misdirected symlink in the Claude skill directory.
3. Repair a dangling link only when its lexical target is inside the canonical Mosaic skills root.
4. Unregister only a symlink pointing inside that root.
5. Enumerate canonical directories at runtime; never hardcode framework skill names.

finalizeStage reconciles after wizard/framework synchronization, and runFrameworkReseed reconciles after the sync-only mosaic update path. A foreign conflict is reported but does not prevent unrelated canonical skills from registering. Filesystem tests use injected temporary roots in skill.spec.ts, finalize-skills.spec.ts, and update-checker.reseed.spec.ts.

M1 intentionally manages Claude Code only. Pi's Mosaic launcher can discover the canonical root directly. Codex still relies on the existing full skill-sync linker and needs separate parity analysis before this lifecycle API is extended.

Pi Persistent Goal Extension

The source of the Mosaic-owned Pi goal controller is:

packages/mosaic/framework/runtime/pi/goal-extension.ts

The framework manifest classifies runtime/** as framework-owned. Both the bash installer and the TypeScript file adapter therefore deploy the same reviewed source to:

$MOSAIC_HOME/runtime/pi/goal-extension.ts
# default: ~/.config/mosaic/runtime/pi/goal-extension.ts

Do not copy or link this extension into ~/.pi/agent/extensions/. The launcher function discoverPiExtensionArgs() emits the core mosaic-extension.ts first and the optional goal-extension.ts second, preserving compatibility with an older installed framework that does not have the goal file yet.

Lifecycle design

Pi API	Goal-controller responsibility
registerCommand('goal')	Set, inspect, pause, resume, or cancel one branch-specific goal
registerTool(...)	Record a terminating structured progress report with evidence
context	Inject the active goal contract before every provider request
turn_end	Record every turn, reject mixed final reports, and enforce the turn bound
agent_settled	Start one deduplicated continuation only after Pi has no retry/compact/queue work
session_compact	Record the compact check, reset provisional verification, and defer idle work
session_start/session_tree	Rebuild state from custom entries on the active branch
session_shutdown	Invalidate deferred callbacks and clear UI state

State is appended as mosaic-goal-state custom entries, which do not enter model context. The context hook creates a fresh hidden mosaic-goal-context message for each request instead of trusting compaction summaries. The mosaic_goal_report result uses terminate: true; when it is the sole final tool call, Pi avoids an unnecessary model response before the controller decides whether to verify, continue, or stop.

Before state is appended or displayed, the controller applies bounded credential-pattern redaction to the goal statement, report summary/evidence/next step, and stop reason. Fingerprints are computed over redacted report content. Pi session entries are append-only, so a credential-bearing legacy entry cannot honestly be erased by the extension: restoration fails closed, emits a warning, and requires removal of the affected session before setting a new goal. This is defense-in-depth rather than a secret-storage contract, and it does not rewrite Pi's separate model-message/tool-call history. Goal prompts tell the agent not to submit credentials or raw sensitive output, and tests use canaries to prove known forms do not reach new custom entries, status text, context, or tool details while ordinary typed fields such as token: string remain intact.

Completion remains evidence-gated but semantic: two consecutive achieved reports are required, and the second run is explicitly a verification pass. This avoids an extra judge-model request after every turn. Deterministic validator commands are intentionally not accepted as /goal input in this slice, so never describe this mechanism as proof of arbitrary natural-language completion.

Tests and local smoke workflow

pnpm --filter @mosaicstack/mosaic exec vitest run \
  src/runtime/pi-goal-extension.spec.ts \
  src/commands/launch.spec.ts \
  src/config/file-adapter.test.ts

bash packages/mosaic/framework/tools/quality/scripts/test-install-migration.sh

For an additive local smoke test without reseeding unrelated live framework files:

install -D -m 0644 \
  packages/mosaic/framework/runtime/pi/goal-extension.ts \
  ~/.config/mosaic/runtime/pi/goal-extension.ts

pi --extension ~/.config/mosaic/runtime/pi/goal-extension.ts

Use /goal help, /goal set ..., and /goal status in that test session. A released framework sync installs the file, and a released Mosaic CLI loads it automatically through mosaic pi.

API Endpoint Reference

All endpoints are served by the gateway at http://localhost:14242 by default.

Authentication

Authentication uses BetterAuth session cookies. The auth handler is mounted at /api/auth/* via a Fastify low-level hook in apps/gateway/src/auth/auth.controller.ts.

Endpoint	Method	Description
/api/auth/sign-in/email	POST	Sign in with email/password
/api/auth/sign-up/email	POST	Register a new account
/api/auth/sign-out	POST	Sign out (clears session cookie)
/api/auth/get-session	GET	Returns the current session

Chat

WebSocket namespace /chat (Socket.IO). Authentication via session cookie.

Events sent by the client:

Event	Payload	Description
message	{ content, conversationId?, provider?, modelId? }	Send a message

Events emitted by the server:

Event	Payload	Description
token	{ token, conversationId }	Streaming token
end	{ conversationId }	Stream complete
error	{ message }	Error during streaming

HTTP endpoints (apps/gateway/src/chat/chat.controller.ts):

Endpoint	Method	Auth	Description
/api/chat/conversations	GET	User	List conversations
/api/chat/conversations/:id/messages	GET	User	Get messages for a conversation

Admin

All admin endpoints require role = admin.

Endpoint	Method	Description
GET /api/admin/users	GET	List all users
GET /api/admin/users/:id	GET	Get a single user
POST /api/admin/users	POST	Create a user
PATCH /api/admin/users/:id/role	PATCH	Update user role
POST /api/admin/users/:id/ban	POST	Ban a user
POST /api/admin/users/:id/unban	POST	Unban a user
DELETE /api/admin/users/:id	DELETE	Delete a user
GET /api/admin/health	GET	System health status

Agent / Providers

Endpoint	Method	Auth	Description
GET /api/agent/providers	GET	User	List all providers and their models
GET /api/agent/providers/models	GET	User	List available models
POST /api/agent/providers/:id/test	POST	User	Test provider connectivity

Projects / Brain

Endpoint	Method	Auth	Description
GET /api/brain/projects	GET	User	List projects
POST /api/brain/projects	POST	User	Create a project
GET /api/brain/projects/:id	GET	User	Get a project
PATCH /api/brain/projects/:id	PATCH	User	Update a project
DELETE /api/brain/projects/:id	DELETE	User	Delete a project
GET /api/brain/tasks	GET	User	List tasks
POST /api/brain/tasks	POST	User	Create a task
GET /api/brain/tasks/:id	GET	User	Get a task
PATCH /api/brain/tasks/:id	PATCH	User	Update a task
DELETE /api/brain/tasks/:id	DELETE	User	Delete a task

Memory / Preferences

Endpoint	Method	Auth	Description
GET /api/memory/preferences	GET	User	Get user preferences
PUT /api/memory/preferences	PUT	User	Upsert a preference

MCP Server (Gateway-side)

Endpoint	Method	Auth	Description
POST /mcp	POST	User (session cookie or Authorization header)	MCP streamable HTTP transport
GET /mcp	GET	User	MCP SSE stream reconnect

Skills

Endpoint	Method	Auth	Description
GET /api/skills	GET	User	List installed skills
POST /api/skills	POST	Admin	Install a skill
PATCH /api/skills/:id	PATCH	Admin	Update a skill
DELETE /api/skills/:id	DELETE	Admin	Remove a skill

Coord (Mission Coordination)

Endpoint	Method	Auth	Description
GET /api/coord/missions	GET	User	List missions
POST /api/coord/missions	POST	User	Create a mission
GET /api/coord/missions/:id	GET	User	Get a mission
PATCH /api/coord/missions/:id	PATCH	User	Update a mission

Observability

OpenTelemetry traces are exported to the OTEL collector (OTEL_EXPORTER_OTLP_ENDPOINT). View traces in Jaeger at http://localhost:16686.

Tracing is initialized before NestJS bootstrap in apps/gateway/src/tracing.ts. The import order in apps/gateway/src/main.ts is intentional: import './tracing.js' must come before any NestJS imports.