diff --git a/docs/PRD.md b/docs/PRD.md index 3aad4016..016165f9 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -1,9 +1,250 @@ --- kind: spec status: active +source_of_truth: true --- -# PRD: Mosaic Stack v0.1.0 +# PRD: Mosaic Stack — North Star + +This document is the product source of truth for Mosaic Stack. + +- **Part I** defines the product north star. It is written from the ratified + decision set D1–D14 (operator decision session, 2026-08-25; decision owner + Jason Woltje). Each section cites the decisions it implements. +- **Part II** preserves the active workstream contracts unchanged. Open issues + bind to them; this rewrite does not alter a single normative word in them. +- The previous v0.1.0 beta PRD body is archived verbatim at + [docs/archive/PRD-v0.1.md](./archive/PRD-v0.1.md) and is no longer authority. +- The delivery roadmap lives in [docs/ROADMAP.md](./ROADMAP.md). Per D11, every + planned phase appears there from day one, even as a placeholder. + +## Metadata + +- **Owner / decision authority:** Jason Woltje +- **Status:** active (supersedes the v0.1.0 PRD as product authority) +- **Date:** 2026-08-26 +- **Decision registry:** D1–D14, recorded in Part I §12 +- **SSOT rule:** this repository's `docs/` tree is the product source of truth + (D5). Estate brains hold operational records, not product canon; only + product-relevant material migrates here (D6). + +--- + +## Part I — Product north star + +### 1. What Mosaic Stack is (D1) + +Mosaic Stack is an **open-source, AI-first platform for people who want a +self-hosted environment for agentic management and a life operating system.** +It serves personal, business, and employee needs from one deployment, and the +work is offered freely. + +"AI-first" means agents are first-class operators of the system, not a bolted-on +chat box: the platform exists to let humans direct fleets of agents over their +projects, tasks, communications, and infrastructure, with the same tools and +the same guarantees whether a human or an agent is acting. + +### 2. Who it is for (D1, D9) + +The operator of a deployment is its user. Mosaic Stack is **not a hosted +business**: running the system as a service for external customers is outside +the north star. Multi-tenancy exists WITHIN a deployment so that one operator +can separate their world — for example, several LLCs plus a personal domain — +while every deployment is self-hosted by its own operator. + +"Company" in the hierarchy is organizational separation for one operator's +world, not a customer account. + +### 3. Deployment modes (D3) + +Two modes, chosen at install time: + +| | Standalone / personal | Enterprise | +| ------------------- | -------------------------------------- | ----------------------------------------------------- | +| Brains | one mosaic-brain (system + user files) | system brain for config + one brain per user | +| User-data isolation | single user | no user-data leakage between users; sharing is opt-in | +| Secrets | OpenBao/Vault or flat files | OpenBao/Vault REQUIRED | +| Conversion | Standalone → Enterprise, **one-way** | terminal state | + +Brains are configurable as external git repositories (recommended, not +required); git tracking is always on locally. + +**Federation** (connecting deployments: system-level config, assigned users, +rights and data-access control, trusts with boundaries, exfiltration +monitoring) is intentionally not fully designed. It is deferred, appears on the +roadmap as a placeholder phase per D11, and nothing in v1 may foreclose it. + +### 4. Structure and tenancy (D2, D9, D13) + +The hierarchy: + +``` +company/organization (N per deployment) +└─ estate (each in exactly one company) + └─ project (each in exactly one estate) + └─ workspace (project-specific; carries the Kanban) +``` + +Rules: + +- Users can create N companies, N estates, N projects. +- Tasks bubble UP the hierarchy so whole-system status is visible at every + level. Bubble-up is **read-only aggregation**, never a cross-workspace write. +- Granular RBAC: admins restrict access per company, estate, and project; + grants are evaluated down the chain. Assets are transferable subject to the + structure. +- **`workspace_id` remains the hard mechanical isolation unit** exactly as + ratified in + [docs/requirements/native-kanban-sot.md](./requirements/native-kanban-sot.md) + (#751): PostgreSQL sole writable SOT, cross-workspace relationships rejected, + fail-closed mutations. The hierarchy is parent structure ABOVE workspaces, + used for RBAC evaluation and read-only roll-ups. The kanban SOT carries this + as Amendment A1, added by reviewed PR — an amendment, not a rewrite (D13). + +### 5. Identity (D10) + +Built-in auth (better-auth) is the **account system of record**. Authentik and +other external IdPs federate in via OIDC as login methods; they never become +the system of record. Perimeter shims (forward-auth in front of a web host) are +deployment workarounds, not the design. + +### 6. Onboarding (D4) + +Onboarding is a **wizard that differs by mode, is re-runnable (no lock-in), and +is extensible** — new wizards attach as tabs. + +Standalone flow captures: system and company name; component choices (Mosaic +Comms/Matrix vs external; Mosaic SSO/Authentik vs external; Mosaic +DB/PostgreSQL vs external; vector DB); the initial user +(email/password/name/SSO); comms setup (Matrix/Discord/Slack); agent enrollment +(harness choice and install, OAuth or API-key login, multi-account, model +choice with recommendation, agent name and persona, account assignment, +optional comms auto-enroll); a user onboarding profile (disabilities including +ADHD/autism/PDA/vision, professional background, education, desired agent +communication style, optional voice-matching interview, family/pets/friends/ +hobbies/likes-dislikes); email and drive connectors (Gmail/IMAP, Google +Drive/OneDrive/Dropbox) with granular agentic-access consent; SSO/OIDC +configuration; an initial estate, an initial project, and seeded example data. + +Enterprise uses the same skeleton with personal data optional; the focus moves +to business structure, org chart, RBAC, M365 and external systems, immediate +OIDC, SSO prominent. + +Profile answers feed `USER.md` and/or the user's data store subject to the +custody rule in §7. + +### 7. Data custody (D6, D14) + +- **Sensitive profile categories** (disabilities, family, communication style, + and similar) live in the **user's own brain ONLY**. PostgreSQL holds + structural data, consent records, and pointers — never the content. "User + data does not leak" is enforced by architecture, not policy (D14). +- Standalone (one user, one brain) **may** keep the same split — D14 makes it + optional in Standalone, not required. Keeping it is the recommended default + because it preserves forward-compatibility with the one-way Enterprise + conversion (D3). +- Estate brains hold operational records. Only product-relevant material + migrates into this repository's docs; operational records stay in their + brains and are linked (D6). + +### 8. Architecture gate — the webUI sits OVER official tooling (D8, D12) + +**HARD RULE:** every webUI operation goes through the Gateway API backed by the +same official framework tooling the CLI uses. The CLI remains the primary +execution method; the webUI uses the tools to operate and configure the +system. The webUI never bypasses tooling to reach the database or filesystem +directly. + +Consequence for planning: when a desired webUI operation has no backing tool, +the gap is scored **"blocked on tooling"** and the tool is built first. The +product baseline therefore always includes all three D8 inputs: the tool +inventory (what exists and what is missing), the webUI→tool mapping, and the +measured current state of the `next` branch. + +### 9. v1 slice (D11) + +v1 is deliberately small: + +1. **Standalone onboarding wizard** — system/company name, component choices, + initial user, initial estate + project, seeded examples, re-runnable. +2. **Hierarchy core** — company → estate → project → workspace → kanban, with + read-only task bubble-up. +3. **Basic RBAC** on the hierarchy. +4. **Minimal agent enrollment** — one harness, API key, name/persona. + +Deferred beyond v1: connectors, comms integrations, voice-matching, M365, +Enterprise conversion, federation. Every deferred item appears in +[docs/ROADMAP.md](./ROADMAP.md) per the D11 rule: nothing exists only in heads. + +### 10. Relationship to the fleet north star + +[docs/fleet/NORTH_STAR.md](./fleet/NORTH_STAR.md) (generated from +`docs/fleet/NORTH_STAR.yaml`) is the **delivery-fleet** north star: how the +agent fleet that builds and operates the system should run (NS-1..NS-10, +workstreams A–L). This PRD is the **product** north star. They are not +competitors: the fleet north star is subordinate product-wise — its workstream +J ("Web control plane") is one consumer of this PRD's D8/D12 gate — and this +PRD does not redefine fleet invariants. The subordination rule is ratified in +the frozen audit-input baseline (T2 operator freeze, 2026-08-25: "the PRD must +cite and subordinate it, never fork it"). A change that would put the two in +conflict must amend one of them explicitly, never fork a third document +(drafting addition — see §12.1). + +### 11. Explicit non-goals + +- Hosted/SaaS operation for external customers (D9). +- A webUI that writes to the database or filesystem around the tooling (D12). +- A second writable task store beside PostgreSQL (native-kanban-sot invariants). +- Fully-designed federation in v1 (D3 — roadmap placeholder only). + +### 12. Decision registry + +| ID | Decision (short form) | +| --- | ------------------------------------------------------------------------------------------------------------------ | +| D1 | Open-source, AI-first, self-hosted platform for agentic management + life OS | +| D2 | Hierarchy company→estate→project→workspace→kanban; bubble-up; granular RBAC | +| D3 | Standalone vs Enterprise; one-way conversion; per-user brains + Vault required in Enterprise; federation deferred | +| D4 | Re-runnable, extensible, per-mode onboarding wizards | +| D5 | North star = this rewrite of docs/PRD.md; stack docs/ = product SSOT | +| D6 | Only product-relevant material migrates from brains; operational records stay and link | +| D7 | Spec-inventory sweep launched immediately (executed; INPUTS baseline frozen by operator ruling T2, 2026-08-25) | +| D8 | webUI sits over official framework tooling; CLI primary | +| D9 | Not a hosted business; company = organizational separation for one operator | +| D10 | better-auth is the account system of record; external IdPs via OIDC | +| D11 | Small v1 slice; ALL phases on the documented roadmap from day one | +| D12 | HARD RULE: webUI never bypasses tooling; missing tool ⇒ build the tool first | +| D13 | workspace_id stays the hard isolation unit; hierarchy is parent structure above; kanban SOT amended, not rewritten | +| D14 | Sensitive profile data in the user's own brain only; postgres holds structure/consent/pointers | + +The full decision texts are recorded in the operator decision log (USC estate +brain, webui-audit lane, `GRILL.md`). + +### 12.1 Drafting additions beyond D1–D14 + +Independent review of this rewrite identified rules in this document that are +not present in the D1–D14 record or the frozen T2 baseline. They are listed +here so their ratification is explicit: approval of the PR that introduces +this document, by the decision owner, ratifies them. If any is rejected it is +removed, not silently kept. + +1. **Federation forward-compatibility gate:** "nothing in v1 may foreclose + federation" (§3), and scoping federation later requires its own PRD plus + threat model ([ROADMAP](./ROADMAP.md) P5). D3 defers federation; these + protective gates are additions. +2. **North-star amendment rule:** a product/fleet north-star conflict must be + resolved by amending one of the two documents explicitly, never by forking + a third (§10). The subordination itself is T2-ratified; this amendment + procedure is an addition. + +--- + +## Part II — Active workstream contracts (preserved unchanged) + +The sections below are normative, in-flight workstream contracts carried over +verbatim from the previous revision of this file. Open issues bind to them. +This rewrite moved no text and changed no requirement in them; they are +governed by their own issues and review gates, and they graduate out of this +file individually when their workstreams close. ## Current addendum: #1194 — Installed framework-tool drift detection @@ -14,85 +255,6 @@ status: active - Document identity/messaging/gate behavior changes in the current stale set, the reviewed quiet-window keep-mode refresh command, and post-refresh probes against the installed path. - Prove by construction that a stale and missing deployed tool are detected; that regression must fail before this checker exists. -## Metadata - -- **Owner:** Jason Woltje -- **Date:** 2026-03-12 -- **Status:** draft -- **Best-Guess Mode:** true -- Repo (target): `git.mosaicstack.dev/mosaic/mosaic-stack` -- Baseline: `~/src/jarvis-old` (jarvis v0.2.0) -- Package source: `~/src/mosaic-mono-v0` (@mosaicstack/\* packages) -- Agent harness: [pi](https://github.com/badlogic/pi-mono) (v0.57.1) -- Remote control reference: [OpenClaw](https://github.com/openclaw/openclaw) (upstream, canonical) - ---- - -## Problem Statement - -Jarvis (v0.2.0) is a self-hosted AI assistant with a Python FastAPI backend and Next.js frontend. It handles chat, projects, tasks, and LLM routing but lacks orchestration depth, agent coordination, shared memory, and remote access. The Mosaic framework (`~/.config/mosaic`) provides agent guides, shell-based orchestration tools, and quality rails — but these are loose scripts, not an integrated platform. The `@mosaicstack/*` packages in mosaic-mono-v0 began consolidating these into TypeScript packages (brain, queue, coord, cli, prdy, quality-rails) but have no UI, no auth, and no agent runtime integration. - -**The gap:** Three codebases with overlapping concerns, no unified runtime, no remote control surface (Discord/Telegram), no gateway orchestrator, and a Python backend that doesn't align with the target TypeScript-everywhere stack. - -**What Mosaic Stack solves:** A single monorepo that brings together many pieces to make one beautiful picture — a self-hosted, multi-user AI agent platform with web dashboard, TUI, remote control, shared memory, mission orchestration, and extensible skill/plugin architecture. All TypeScript. Pi as the agent harness. Brain as the knowledge layer. Queue as the coordination backbone. - ---- - -## Objectives - -1. **Unified TypeScript monorepo** — One repo, one language, one build pipeline for all Mosaic Stack components -2. **Pi-powered agent runtime** — Pi SDK embedded as the core agent loop; Pi TUI as the terminal interface -3. **Web + TUI + Remote** — Next.js dashboard for visual management, Pi TUI for terminal work, Discord/Telegram for remote control -4. **Gateway orchestrator** — Central routing layer that dispatches tasks to appropriate agents based on capability, cost, and context -5. **Shared memory** — PostgreSQL canonical store + vector DB for semantic search + tiered log summarization to prevent context creep -6. **Multi-user with SSO** — BetterAuth with Authentik/WorkOS/Keycloak SSO, RBAC for family/team/business use -7. **Full @mosaicstack/\* package integration** — brain, queue, coord, mosaic, prdy, quality-rails, cli all integrated -8. **Extensible** — MCP capability, skill import interface, plugin architecture for LLM providers and remote channels - ---- - -## Scope - -### In Scope (v0.1.0 Beta) - -1. Chat/conversation UI (web) — carry forward from jarvis-old, rewrite frontend to work with new backend -2. Pi TUI integration — terminal-based agent interaction using Pi SDK -3. Web dashboard — settings, task management, projects, PRDs, missions, agent status -4. Gateway orchestrator (`@mosaicstack/gateway`) — central dispatch for agent tasks with routing logic -5. Task management — CRUD, kanban, mission-scoped tasks, dependency tracking -6. Project management — projects, milestones, PRDs linked to missions -7. Shared memory system — learned preferences, behaviors, defaults; tiered storage with summarization -8. User management — RBAC (admin, member, viewer), multi-user capable -9. SSO — BetterAuth with Authentik/WorkOS/Keycloak adapter -10. Remote control — Discord plugin (high priority), Telegram plugin -11. LLM provider support — Anthropic subs, Codex subs, Z.ai subs, other API-based, Ollama, LM Studio, llama.cpp -12. Agent routing — task-based model/provider selection (cost/capability matrix) -13. MCP capability — server and client, tool registration -14. Skill import interface — browse, install, manage agent skills -15. `@mosaicstack/brain` — structured data layer (migrated to PG + vector DB backend) -16. `@mosaicstack/queue` — Valkey-backed task queue with MCP tools -17. `@mosaicstack/coord` — mission coordination engine -18. `@mosaicstack/mosaic` — install wizard / bootstrap -19. `@mosaicstack/prdy` — PRD wizard -20. `@mosaicstack/quality-rails` — code quality scaffolder -21. `@mosaicstack/cli` — unified `mosaic` CLI -22. Docker Compose deployment + bare-metal capability -23. Agent log service — ingest, parse, tier, summarize agent interaction logs -24. Local durable agent fleet canary — `mosaic fleet` / `mosaic agent` CLI for an isolated tmux-backed canary fleet using a named socket, with roster-driven local customization and rollback-safe verification - -### Out of Scope (v0.1.0) - -1. SaaS / multi-tenant revenue model — this is a personal/family/team tool -2. Mobile native apps — web responsive is sufficient -3. Public npm registry publishing — Gitea registry only -4. Video/voice agent interaction -5. Full OpenClaw feature parity — we take inspiration, not wholesale migration -6. Calendar integration (deferred — brain tracks events, but no gcal sync yet) -7. GLPI/helpdesk ticket sync (deferred) -8. Woodpecker CI integration tooling (deferred — focus on core platform first) - ---- - ## Compaction Refresh Trust Lifecycle (M1, #827–#830) ### Problem and objective @@ -635,914 +797,6 @@ Canonical checkpoint/handoff payloads, exactly-once connector receipts, concrete --- -## Architecture - -### High-Level System Diagram - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Mosaic Stack │ -│ │ -│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ ┌──────────────┐ │ -│ │ Next.js │ │ Pi TUI │ │ Discord │ │ Telegram │ │ -│ │ Web App │ │ Terminal │ │ Plugin │ │ Plugin │ │ -│ └────┬─────┘ └────┬─────┘ └──────┬──────┘ └──────┬───────┘ │ -│ │ │ │ │ │ -│ └──────────────┴───────┬───────┴────────────────┘ │ -│ │ │ -│ ┌─────────▼──────────┐ │ -│ │ @mosaicstack/gateway │ ← Central Orchestrator│ -│ │ (NestJS+Fastify) │ │ -│ └────┬────┬────┬─────┘ │ -│ │ │ │ │ -│ ┌──────────────┤ │ ├──────────────┐ │ -│ │ │ │ │ │ │ -│ ┌───────▼──────┐ ┌────▼────▼──┐ │ ┌───────────▼────────┐ │ -│ │ @mosaicstack/brain│ │ @mosaicstack/ │ │ │ Agent Pool │ │ -│ │ (Data Layer) │ │ queue │ │ │ (Pi SDK sessions) │ │ -│ └───────┬──────┘ └────────────┘ │ │ - Anthropic │ │ -│ │ │ │ - Codex │ │ -│ ┌───────▼──────────────────┐ │ │ - Z.ai │ │ -│ │ PostgreSQL │ VectorDB │ │ │ - Ollama │ │ -│ │ (canonical) │ (semantic)│ │ │ - LM Studio │ │ -│ └──────────────┴───────────┘ │ │ - llama.cpp │ │ -│ │ └────────────────────┘ │ -│ ┌─────────────▼──────┐ │ -│ │ @mosaicstack/coord │ │ -│ │ Mission lifecycle │ │ -│ └────────────────────┘ │ -│ │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │ -│ │ @mosaicstack/cli │ │ @mosaicstack/prdy │ │ @mosaicstack/ │ │ -│ │ │ │ │ │ quality-rails │ │ -│ └──────────────┘ └──────────────┘ └──────────────────┘ │ -│ │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ Valkey (queue backend) │ BetterAuth (SSO/RBAC) │ │ -│ └──────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────────┘ -``` - -### Technology Decisions - -| Layer | Technology | Rationale | -| ------------------ | ------------------------------------ | ----------------------------------------------------------------------------------------------------------- | -| **Web Frontend** | Next.js 16 + React 19 + Tailwind CSS | SSR, RSC; design tokens from @mosaicstack/design-tokens (mosaic-stack-website) | -| **API / Gateway** | NestJS + Fastify adapter | Module system, DI, guards/interceptors for complex gateway; Fastify performance underneath | -| **Agent Runtime** | Pi SDK (embedded) | Extensible harness with tools, skills, session management | -| **TUI** | Pi interactive mode | Native terminal agent interaction | -| **Auth** | BetterAuth + SSO adapters | Multi-user RBAC with Authentik/WorkOS/Keycloak | -| **Database** | PostgreSQL 17 + pgvector | Canonical store; pgvector for embedding search | -| **Vector DB** | pgvector + VectorStore interface | pgvector for v0.1.0; `VectorStore` abstraction in @mosaicstack/memory makes Qdrant a drop-in later | -| **Cache / Queue** | Valkey 8 | Redis-compatible; proven in @mosaicstack/queue | -| **ORM** | Drizzle ORM | TypeScript-native, lightweight, good migration story | -| **Validation** | Zod | Already used across @mosaicstack/\* packages | -| **Build** | pnpm workspaces + Turborepo | Proven in both jarvis-old and mosaic-mono-v0 | -| **Testing** | Vitest + Playwright | Unit/integration via Vitest, E2E via Playwright | -| **Remote Control** | Discord.js + Telegraf | Inspired by OpenClaw plugin architecture | -| **MCP** | @modelcontextprotocol/sdk | Already used in @mosaicstack/brain and @mosaicstack/queue | -| **Container** | Docker Compose | Self-hosted; bare-metal also supported | -| **CI** | Woodpecker CI | Existing infrastructure at git.mosaicstack.dev | -| **Observability** | OpenTelemetry + SigNoz | Wide-event logging from day one; OTEL auto-instrumentation for NestJS/PG/HTTP; SigNoz as all-in-one backend | -| **Log Processing** | Custom ingest service | Parse agent logs → tiered storage → summarization | - -### Key Architecture Decisions - -**AD-1: TypeScript everywhere (no Python backend)** -The jarvis-old FastAPI backend is not carried forward as code. Its domain logic (conversation management, LLM routing, task/project CRUD, auth) is reimplemented in TypeScript. The Python plugin system is replaced by Pi's extension/skill system and MCP tool registration. - -**AD-2: Pi SDK as the agent runtime** -Instead of a custom LLM provider abstraction (jarvis-old's `BaseLLMProvider`), Pi SDK manages agent sessions. Pi handles model selection, tool calling, context management, and compaction. The gateway dispatches work to Pi sessions configured with appropriate providers. - -**AD-3: Gateway as the central nervous system (NestJS + Fastify adapter)** -`@mosaicstack/gateway` is the single API surface. The web app, TUI, Discord, and Telegram all talk to the gateway. The gateway routes to brain (data), queue (coordination), agent pool (LLM work), and coord (mission lifecycle). This replaces the direct FastAPI-to-DB pattern from jarvis-old. - -NestJS was chosen over raw Fastify because the gateway is inherently complex — it hosts channel plugins, agent pool management, routing engine, WebSocket hub, MCP server, auth middleware, and integrates brain, queue, memory, and log services. NestJS provides the module system, dependency injection, guards, and interceptors needed to organize this cleanly. NestJS uses Fastify as its HTTP adapter, so Fastify's performance is preserved. This also aligns with the stated stack preference in USER.md ("NestJS API + Next.js web"). @mosaicstack/brain's existing Fastify code migrates naturally into a NestJS module with Fastify adapter. - -**AD-4: Brain migrates from JSON files to PostgreSQL** -`@mosaicstack/brain` currently uses a JSON file store. For Mosaic Stack, brain's data model (tasks, projects, events, agents, missions, tickets) moves to PostgreSQL via Drizzle ORM. Brain's REST + MCP interface is preserved — only the storage backend changes. - -**AD-5: Tiered memory with summarization** -Agent interaction logs are ingested into a log service. Raw logs are stored short-term. A summarization pipeline (using a cheap LLM) periodically compresses logs into structured insights stored in the vector DB. This prevents unbounded log growth while preserving searchable context. - -**AD-6: Remote control via plugin architecture** -Discord and Telegram plugins follow a channel plugin pattern inspired by OpenClaw (https://github.com/openclaw/openclaw). Each plugin registers as a channel with the gateway, receives messages, and dispatches them through the same routing pipeline as web/TUI messages. - -**AD-7: Gateway state persistence via Valkey (restart resilience)** -The gateway persists its orchestration state (active sessions, pending dispatches, routing context, agent assignments) to Valkey. On restart, the gateway reads Valkey state and resumes operations — active agent sessions are reconnected or gracefully recovered. `mosaic gateway restart --fresh` is the nuclear option: clears the Valkey queue and all in-flight state, starting with a clean slate. This prevents context/focus/direction loss that would otherwise occur on every restart. - -**AD-8: Multi-session agent architecture** -Each agent operates in a distinct session. Multiple authorized input channels (TUI, web UI, Discord) can connect to the same agent session simultaneously. This means a user can start a conversation in Discord, continue in the web UI, and monitor via TUI — all feeding into the same agent context. OpenClaw has this concept; Mosaic Stack evolves it with proper session authorization and channel multiplexing at the gateway level. - -**AD-9: Discord channel-to-agent binding** -Discord channels pair to specific agent/session combinations via channel ID binding. This provides data segregation — messages in #project-alpha route to the project-alpha agent session, messages in #general route to a general-purpose session. Prevents cross-contamination between contexts and provides clear boundaries for multi-channel use. - -**AD-10: Agent session barge-in via tmux** -Each agent session runs in a dedicated, named tmux session (e.g., `mosaic-agent-project-alpha`). This enables barge-in — a user can attach to any active agent's tmux session to observe, interrupt, or redirect. `mosaic agent attach ` connects to the tmux session. This provides direct low-level access when the normal channel interfaces are insufficient. - -**AD-11: Cron-based scheduled jobs** -The gateway includes a cron scheduler for recurring tasks: log summarization runs, stale task detection, memory decay, provider health checks, scheduled agent dispatches. Uses node-cron or similar; schedules are configurable via web dashboard and stored in PG. Each cron job is a gateway-dispatched task that goes through the normal routing pipeline. - -**AD-12: Web search tool (DuckDuckGo MCP)** -Agent sessions include a web search tool for information retrieval. DuckDuckGo via MCP server is the primary option (privacy-respecting, no API key required). Falls back to other search MCP providers if configured. Registered as a standard MCP tool available to all agent sessions. - -**AD-13: Design system from @mosaicstack/design-tokens** -The web dashboard uses the Mosaic Stack design system established in `mosaic-stack-website`. The `@mosaicstack/design-tokens` package provides CSS custom properties, Tailwind preset, and TS color/font/radius exports. Dark theme default with light theme support. Fonts: Outfit (sans), Fira Code (mono). Color palette: deep blue-grays with blue/purple/teal accents. - -**AD-14: Multi-tier deployment readiness** -Code is structured assuming eventual multi-node deployment with dedicated roles (gateway nodes, agent worker nodes, brain/DB nodes). Packages communicate via well-defined APIs (HTTP/WS/MCP), not in-process calls where avoidable. Service boundaries are clean: gateway is stateless (state in PG/Valkey), agent pool can scale independently, brain is a separate service. v0.1.0 runs single-node; the architecture doesn't fight horizontal scaling later. - ---- - -## Package Structure - -### Monorepo Layout - -``` -mosaic-mono-v1/ -├── apps/ -│ ├── web/ Next.js 16 web dashboard -│ └── gateway/ @mosaicstack/gateway — NestJS API + WebSocket -├── packages/ -│ ├── types/ @mosaicstack/types — shared type contracts -│ ├── brain/ @mosaicstack/brain — data layer (PG-backed) -│ ├── queue/ @mosaicstack/queue — Valkey task queue + MCP -│ ├── coord/ @mosaicstack/coord — mission coordination -│ ├── mosaic/ @mosaicstack/mosaic — install wizard -│ ├── prdy/ @mosaicstack/prdy — PRD wizard -│ ├── quality-rails/ @mosaicstack/quality-rails — code quality scaffolder -│ ├── cli/ @mosaicstack/cli — unified CLI -│ ├── auth/ @mosaicstack/auth — BetterAuth config + SSO adapters -│ ├── db/ @mosaicstack/db — Drizzle schema, migrations, connection -│ ├── agent/ @mosaicstack/agent — Pi SDK integration, agent pool manager -│ ├── memory/ @mosaicstack/memory — tiered memory + summarization service -│ ├── log/ @mosaicstack/log — agent log ingest + processing -│ └── design-tokens/ @mosaicstack/design-tokens — CSS vars, Tailwind preset, colors -├── plugins/ -│ ├── discord/ @mosaicstack/discord-plugin — Discord channel -│ └── telegram/ @mosaicstack/telegram-plugin — Telegram channel -├── docker/ -│ ├── gateway.Dockerfile -│ ├── web.Dockerfile -│ └── init-db.sql -├── docs/ -│ ├── PRD.md (this file) -│ ├── TASKS.md -│ └── scratchpads/ -├── docker-compose.yml -├── pnpm-workspace.yaml -├── turbo.json -├── tsconfig.base.json -├── vitest.workspace.ts -├── AGENTS.md -├── CLAUDE.md -└── README.md -``` - -### Package Responsibilities - -#### `apps/gateway` — @mosaicstack/gateway (NEW — critical path) - -The central nervous system. All clients connect here. Built with NestJS (Fastify adapter). - -- **NestJS modules** — Each concern (chat, brain, agent, auth, queue, memory, plugins) is a module with clear boundaries -- **Fastify adapter** — Fastify performance under NestJS's organizational structure -- **WebSocket gateway** — NestJS built-in WebSocket support for chat streaming, agent status, notifications -- **Agent routing engine** — Routes tasks to appropriate LLM provider/model based on task type, cost tier, capability requirements -- **Session management** — Tracks active conversations, agent sessions, user contexts -- **MCP server** — Exposes Mosaic capabilities as MCP tools -- **Plugin host** — Loads and manages channel plugins (Discord, Telegram) -- **Auth middleware** — BetterAuth session validation, RBAC enforcement - -Key routes: - -``` -POST /api/chat Send message, get streamed response -GET /api/conversations List conversations -POST /api/conversations Create conversation -GET /api/conversations/:id Get conversation with messages -DELETE /api/conversations/:id Delete conversation -POST /api/tasks Create task (brain-backed) -GET /api/tasks List/filter tasks -PATCH /api/tasks/:id Update task -GET /api/projects List projects -POST /api/projects Create project -GET /api/missions List missions -POST /api/missions Create mission -GET /api/missions/:id Mission summary with tasks -POST /api/agents/dispatch Dispatch work to agent pool -GET /api/agents/status Active agent sessions -GET /api/memory/search Semantic search across memory -POST /api/memory/preferences Store learned preference -GET /api/skills List available skills -POST /api/skills/install Install a skill -GET /api/providers List configured LLM providers -POST /api/providers Configure LLM provider -GET /api/admin/users User management (admin) -POST /api/admin/users Create user (admin) -WS /ws/chat/:conversationId Streaming chat via WebSocket -WS /ws/agents Agent status stream -GET /mcp MCP endpoint (streamable HTTP) -``` - -#### `apps/web` — Next.js Web Dashboard - -Carried forward from jarvis-old with significant refactoring. - -- Chat/conversation UI (primary interaction surface) -- Settings management (providers, integrations, profile) -- Task management (list, kanban, detail views) -- Project management (list, detail, linked missions) -- Mission dashboard (status, progress, task breakdown) -- PRD viewer/editor -- Agent status dashboard (active sessions, routing stats) -- Skill browser and installer -- User management (admin RBAC panel) -- Auth pages (login, SSO redirect, registration) - -#### `packages/types` — @mosaicstack/types - -Migrated from mosaic-mono-v0. Extended with: - -- Gateway types (routing, dispatch, agent pool) -- Auth types (user, role, permission) -- Conversation/message types (from jarvis-old domain) -- Memory types (preference, insight, summary) -- Plugin channel types (Discord, Telegram message mapping) - -#### `packages/brain` — @mosaicstack/brain - -Migrated from mosaic-mono-v0. **Storage backend changes from JSON to PostgreSQL.** - -- REST API preserved (mounted as gateway sub-router or standalone) -- MCP tools preserved -- Collections layer rewritten to use Drizzle ORM queries instead of JSON file I/O -- Same entity model: tasks, projects, events, agents, missions, mission-tasks, tickets -- New: computed endpoints (today, stale, stats, search, audit) run against PG -- New: appreciation collection preserved for family use - -#### `packages/queue` — @mosaicstack/queue - -Migrated from mosaic-mono-v0 with minimal changes. - -- Valkey-backed task queue with atomic WATCH/MULTI/EXEC -- MCP server with 8 tools -- Used by gateway for agent task dispatch and coordination - -#### `packages/coord` — @mosaicstack/coord - -Migrated from mosaic-mono-v0. - -- Mission lifecycle: init, run, resume, status, drain -- TASKS.md parsing and management -- Session lock management -- Continuation prompt generation -- Integration with gateway for mission-driven orchestration - -#### `packages/db` — @mosaicstack/db (NEW) - -Shared database package. - -- Drizzle ORM schema definitions (all tables) -- Migration management -- Connection pool configuration -- Shared by gateway, brain, auth, memory - -#### `packages/auth` — @mosaicstack/auth (NEW) - -Authentication and authorization. - -- BetterAuth configuration -- SSO adapters: Authentik, WorkOS, Keycloak -- RBAC: roles (admin, member, viewer), permissions -- API key generation for brain/MCP access -- Session management middleware - -#### `packages/agent` — @mosaicstack/agent (NEW — critical path) - -Pi SDK integration layer. - -- Agent pool manager — spawns and manages Pi agent sessions -- Provider configuration — Anthropic, Codex, Z.ai, Ollama, LM Studio, llama.cpp -- Agent routing logic — selects provider/model based on task characteristics -- Tool registration — registers Mosaic-specific tools (brain access, queue ops, memory search) -- Skill management — loads and configures Pi skills for agent sessions -- Session lifecycle — create, monitor, complete, fail, timeout - -#### `packages/memory` — @mosaicstack/memory (NEW) - -Tiered memory system. - -- Preference store — learned user preferences, behaviors, defaults (PG) -- Insight store — distilled knowledge from agent interactions (PG + vector) -- Semantic search — query across memory using pgvector embeddings -- Summarization pipeline — compress raw logs into structured insights -- Memory API — used by gateway and agent sessions - -#### `packages/log` — @mosaicstack/log (NEW) - -Agent log service. - -- Log ingest — receives structured logs from agent sessions -- Log parsing — extracts decisions, learnings, tool usage patterns -- Tiered storage — hot (recent, full detail), warm (summarized), cold (archived) -- Summarization trigger — invokes cheap LLM to compress aging logs -- Retention policy — configurable TTLs per tier - -#### `packages/mosaic` — @mosaicstack/mosaic - -Migrated from mosaic-mono-v0, updated for v1. - -- Install wizard for Mosaic Stack setup -- Detects existing installations, offers upgrade path -- Configures `~/.config/mosaic/` with guides, tools, runtime configs - -#### `packages/prdy` — @mosaicstack/prdy - -Migrated from mosaic-mono-v0. - -- PRD generation wizard -- Template-based PRD creation with Zod validation -- CLI integration via `mosaic prdy` - -#### `packages/quality-rails` — @mosaicstack/quality-rails - -Migrated from mosaic-mono-v0. - -- TypeScript scaffolder for project quality config -- Generates ESLint, tsconfig, Woodpecker, husky, lint-staged configs -- Supports project types: monorepo, typescript-node, nextjs - -#### `packages/cli` — @mosaicstack/cli - -Migrated from mosaic-mono-v0, extended. - -- Unified `mosaic` binary -- Subcommands: `mosaic coord`, `mosaic prdy`, `mosaic queue`, `mosaic quality`, `mosaic gateway`, `mosaic brain` -- Plugin discovery for installed @mosaicstack/\* packages - -#### `plugins/discord` — @mosaicstack/discord-plugin (NEW — high priority) - -Discord remote control channel. Architecture inspired by OpenClaw (https://github.com/openclaw/openclaw). - -- Channel plugin that registers with the gateway as a NestJS dynamic module -- Single-guild binding only (v0.1.0) — prevents data leaks between servers -- Receives Discord messages, dispatches through gateway routing -- Streams agent responses back to Discord (chunked for 2000-char limit) -- Routes authorized untagged messages in-channel; mentions create threads (or reuse the same message's attached thread) for multi-turn topics -- Uses stable logical-agent/channel conversation addresses independent of the active harness/provider -- Bot pairing and permission management (Discord user → Mosaic user mapping) -- DM support for private conversations - -#### `plugins/telegram` — @mosaicstack/telegram-plugin (NEW) - -Telegram remote control channel. - -- Same channel plugin pattern as Discord -- Telegraf-based bot -- Message routing through gateway -- Inline keyboard for interactive responses - ---- - -## User/Stakeholder Requirements - -### US-001 Multi-Channel Chat - -**As a user**, I can chat with an AI assistant via web browser, terminal (Pi TUI), Discord, or Telegram and get consistent responses regardless of channel. - -### US-002 Task & Project Dashboard - -**As a user**, I can manage my tasks, projects, and missions from the web dashboard with kanban and list views. - -### US-003 PRD Management - -**As a user**, I can view and edit PRDs for active missions from the web dashboard. - -### US-004 Agent Visibility - -**As a user**, I can see which agents are active, what they're working on, and their status in real-time. - -### US-005 Provider Configuration - -**As a user**, I can configure which LLM providers to use and set routing preferences (cost vs capability). - -### US-006 Skill Management - -**As a user**, I can install and manage agent skills through the web dashboard. - -### US-007 Persistent Memory - -**As a user**, the system remembers my preferences, learned behaviors, and past decisions across sessions. - -### US-008 Semantic Search - -**As a user**, I can search across my memory, conversations, and knowledge semantically. - -### US-009 User Management - -**As an admin**, I can manage users, assign roles, and control access. - -### US-010 SSO Configuration - -**As an admin**, I can configure SSO via Authentik, WorkOS, or Keycloak. - -### US-011 Self-Hosted Deployment - -**As a user**, I can run Mosaic Stack via Docker Compose or directly on bare metal. - -### US-012 Intelligent Routing - -**As an agent operator**, the gateway intelligently routes tasks to the cheapest capable model. - -### US-013 CLI Tooling - -**As a user**, I can use the `mosaic` CLI for PRD creation, quality rail setup, queue management, and mission coordination. - ---- - -## Functional Requirements - -- FR-1: Chat System -- FR-2: Gateway Orchestrator -- FR-3: Agent Pool -- FR-4: Task Management -- FR-5: Project Management -- FR-6: Mission System -- FR-7: Memory System -- FR-8: Authentication & Authorization -- FR-9: Remote Control — Discord -- FR-10: Remote Control — Telegram -- FR-11: LLM Provider Management -- FR-12: Agent Routing -- FR-13: MCP Capability -- FR-14: Skill Management -- FR-15: CLI Integration -- FR-16: Log Service -- FR-17: Gateway State Persistence -- FR-18: Multi-Session Agent Architecture -- FR-19: Cron Scheduler -- FR-20: Web Search Tool -- FR-21: Skill Import from skills.sh - -### FR-1: Chat System - -- Conversation CRUD (create, list, get with messages, delete) -- Real-time streaming responses via WebSocket -- Multi-provider support (route to configured LLM) -- Conversation history with search -- Project-scoped conversations -- System prompt per project/conversation -- Message rendering with markdown, code blocks, tool call display - -### FR-2: Gateway Orchestrator - -- Central API surface for all clients (web, TUI, Discord, Telegram) -- Agent dispatch — receive task, select provider/model, spawn Pi session, return result -- Routing engine — cost/capability matrix, user preference overrides, task-type heuristics -- Plugin host — load channel plugins at startup, manage lifecycle -- MCP server — expose Mosaic tools via MCP protocol -- WebSocket hub — real-time updates for chat, agent status, notifications -- Rate limiting and request validation - -### FR-3: Agent Pool (@mosaicstack/agent) - -- Manage concurrent Pi SDK sessions -- Provider configuration: API key management, endpoint URLs, model lists -- Support providers: Anthropic (subscription + API), OpenAI/Codex (subscription + API), Z.ai, Ollama (local), LM Studio (local), llama.cpp (local) -- Tool injection — all agent sessions get Mosaic tools (brain, queue, memory) -- Skill loading — configure skills per agent session based on task type -- Session monitoring — track active sessions, token usage, duration -- Graceful shutdown — drain active sessions on shutdown - -### FR-4: Task Management - -- Brain-backed task CRUD with full filter/sort -- Task statuses: backlog, scheduled, in-progress, blocked, done, cancelled -- Priority levels: critical, high, medium, low -- Domain categorization -- Dependency tracking (blocks/blocked_by) -- Project association -- Assignee tracking -- Kanban board view in web dashboard -- Due date tracking with stale detection - -### FR-5: Project Management - -- Project CRUD with domain, status, priority -- Link to repository, branch, current/next milestone -- Progress tracking -- Blocker tracking -- Owner assignment - -### FR-6: Mission System - -- Mission CRUD (linked to project and PRD) -- Mission tasks with phases, dependencies, ordering -- Mission summary with computed progress -- Mission coordination via @mosaicstack/coord -- Active mission dashboard in web UI - -### FR-7: Memory System - -- **Preferences**: Key-value store for learned user preferences (e.g., "prefers tables over paragraphs", "timezone: America/Chicago") -- **Insights**: Distilled knowledge from agent interactions, stored with embeddings -- **Semantic search**: Query across all memory using natural language -- **Auto-capture**: Agent sessions automatically log decisions and learnings -- **Summarization**: Periodic compression of raw logs into structured insights -- **Decay**: Old, unused insights decay in relevance score over time - -### FR-8: Authentication & Authorization - -- BetterAuth integration with Next.js -- Email/password registration and login -- SSO via OIDC/SAML: Authentik, WorkOS, Keycloak -- RBAC roles: admin (full access), member (own resources + shared), viewer (read-only) -- API key generation for programmatic/MCP access -- Session management (web + API) - -### FR-9: Remote Control — Discord - -- Discord bot that connects to the gateway through a transport-neutral channel adapter contract -- Authorized messages in configured agent-bound channels work without a mention and respond in-channel -- Mentions in parent channels create threads, or reuse a thread already attached to that same native message, for multi-turn conversations -- Messages already in a thread remain there without requiring repeated mentions -- Stable logical-agent/channel conversation identity survives underlying harness/provider changes -- DM support for private conversations -- Chunked message delivery (Discord 2000-char limit) -- Bot configuration via web dashboard -- Permission management (which Discord users/roles can interact) - -### FR-10: Remote Control — Telegram - -- Telegram bot via Telegraf -- Private and group chat support -- Command-based interaction (`/ask`, `/task`, `/status`) -- Inline keyboard for task management -- Message routing through gateway - -### FR-11: LLM Provider Management - -- Provider configuration UI in web dashboard -- Per-provider: API key/endpoint, enabled models, cost per token -- Subscription-based providers: detect available models from subscription -- Local providers: Ollama model list, LM Studio endpoint, llama.cpp binary path -- Provider health monitoring -- Usage tracking per provider/model - -### FR-12: Agent Routing - -- Task-type to model-tier mapping (from AGENTS.md cost matrix) -- User preference overrides (e.g., "always use Claude for code review") -- Fallback chains (if primary provider unavailable, try next) -- Cost tracking and budget enforcement -- Routing transparency — user can see why a particular model was chosen - -### FR-13: MCP Capability - -- Gateway exposes MCP server (streamable HTTP transport) -- Brain tools registered as MCP tools -- Queue tools registered as MCP tools -- Memory search registered as MCP tool -- Agent sessions can call MCP tools from other services -- External MCP server connectivity (agent can use third-party MCP servers) - -### FR-14: Skill Management - -- Skill catalog — list available skills from configured sources -- Skill install — install skill to `~/.config/mosaic/skills/` or project-local -- Skill configuration — per-skill settings -- Skill status — installed, available, update available -- Web UI for browsing and managing skills - -### FR-15: CLI Integration - -- `mosaic gateway start` — start the gateway server -- `mosaic brain` — brain data management -- `mosaic queue` — queue operations -- `mosaic coord` — mission coordination -- `mosaic prdy` — PRD wizard -- `mosaic quality` — quality rail management -- `mosaic tui` — launch Pi TUI connected to gateway - -### FR-16: Log Service - -- Structured log ingest from agent sessions -- Parse logs for: decisions made, tools used, errors encountered, learnings captured -- Tier management: hot (7 days, full detail), warm (30 days, summarized), cold (90 days, key facts only) -- Summarization pipeline: cheap LLM compresses aging logs on schedule -- Query interface for log search - -### FR-17: Gateway State Persistence - -- Orchestration state persisted to Valkey (active sessions, pending dispatches, routing context) -- On restart, gateway reads Valkey state and resumes — reconnects to active agent sessions -- `mosaic gateway restart --fresh` clears Valkey queue and all in-flight state (nuclear option) -- Session recovery: detect orphaned agent sessions, offer reconnect or cleanup - -### FR-18: Multi-Session Agent Architecture - -- Each agent has a distinct session with dedicated context -- Multiple input channels (TUI, web, Discord, Telegram) can connect to same agent session -- Channel multiplexing at gateway level with proper authorization -- Discord channel ID paired to specific agent/session (prevents cross-contamination) -- Agent session runs in named tmux session for barge-in capability -- `mosaic agent attach ` connects to agent's tmux session -- `mosaic agent list` shows active sessions with connected channels - -### FR-19: Cron Scheduler - -- Built-in cron scheduler in gateway for recurring tasks -- Default schedules: log summarization, stale task detection, memory decay, provider health checks -- Custom schedules: user-defined agent dispatches on cron expressions -- Schedule management via web dashboard and CLI -- Cron jobs dispatched through normal gateway routing pipeline -- Persistence: schedules stored in PG, survive gateway restart - -### FR-20: Web Search Tool - -- DuckDuckGo web search via MCP server (primary — privacy-respecting, no API key) -- Registered as standard MCP tool available to all agent sessions -- Configurable: can swap to other search providers (Brave, SearXNG, Tavily) -- Results formatted for agent consumption (title, snippet, URL) - -### FR-21: Skill Import from skills.sh - -- Browse skills from https://skills.sh directory via API -- Import skills into `~/.config/mosaic/skills/` or project-local `.mosaic/skills/` -- Vetting workflow: imported skills marked as "unvetted" until admin approves -- Skill review interface in web dashboard (view skill content before approval) -- Vetted skills auto-available to agent sessions; unvetted require explicit enable -- `mosaic skill import ` CLI command -- Track installed skills, versions, update availability - ---- - -## Non-Functional Requirements - -### Security - -- No hardcoded secrets — all secrets via environment variables or vault -- API key rotation capability -- RBAC enforcement at gateway level -- Input validation (Zod) on all API endpoints -- Rate limiting on public endpoints -- CORS configuration for web app -- Secure WebSocket connections -- SSO token validation -- Database connection encryption (SSL) - -### Performance - -- Chat response streaming latency < 200ms TTFB (gateway overhead, not LLM latency) -- Dashboard page loads < 2s -- Brain query responses < 100ms for filtered reads -- Semantic search < 500ms -- Support 10+ concurrent agent sessions -- WebSocket connection handling for 50+ concurrent users - -### Reliability - -- Graceful degradation when LLM provider is unavailable (fallback chain) -- Queue persistence — tasks survive gateway restart -- Database connection pooling with retry -- Health check endpoints for all services -- Structured error responses with correlation IDs - -### Observability (Wide-Event Logging — Required from Phase 0) - -- **OpenTelemetry instrumentation** across all services from day one - - `@opentelemetry/sdk-node` + `@opentelemetry/auto-instrumentations-node` for auto-instrumentation (HTTP, PG, Fastify/NestJS) - - NestJS interceptors for custom spans on agent dispatch, routing decisions, memory writes, summarization runs - - Every significant operation emits a structured event with rich context (wide events, not just request/response) -- **SigNoz** as OTEL backend (single Docker service: traces, metrics, logs, built-in UI) -- Request tracing with correlation IDs (trace-id propagated across gateway → agent → brain → queue) -- Agent session metrics (duration, tokens, cost, success/failure, model used, routing reason) -- Provider availability monitoring (health check spans) -- Queue depth monitoring (periodic gauge metrics) -- Memory usage metrics (embedding count, search latency, summarization runs) -- Migrate to Grafana stack (Tempo + Loki + Grafana) post-beta if more customization is needed - -### Scalability (Multi-Tier Readiness) - -- Single-node deployment is the MVP target for v0.1.0 -- Code structured with assumption that multi-tiered deployment will follow: dedicated gateway nodes, agent worker nodes, brain/DB nodes -- Service boundaries communicate via HTTP/WS/MCP APIs, not in-process calls where avoidable -- Gateway is stateless (all state in PG/Valkey) to enable horizontal scaling -- Agent pool designed as independently scalable service -- Database migrations support forward-only schema evolution -- Hierarchical deployment with dedicated roles/specialties is the post-beta target - ---- - -## Acceptance Criteria - -### AC-1: Core Chat Flow - -- [ ] User can log in via web UI, send a message, and receive a streamed response -- [ ] Conversation persists across page refreshes -- [ ] User can create, list, search, and delete conversations -- [ ] Conversations can be scoped to projects - -### AC-2: TUI Integration - -- [ ] `mosaic tui` launches Pi interactive mode connected to gateway -- [ ] User can chat with same conversation context as web UI -- [ ] Agent has access to brain, queue, and memory tools - -### AC-3: Discord Remote Control - -- [ ] Discord bot connects through the harness-neutral channel contract -- [ ] Authorized untagged channel messages route through the gateway and respond in-channel -- [ ] Mentioned parent-channel messages create a thread (or reuse their already-attached thread) and respond there -- [ ] Existing-thread follow-ups stay in the thread without repeated mentions -- [ ] Channel/session identity remains stable while the underlying harness/provider changes -- [ ] Responses stream back to Discord (chunked) -- [ ] Unauthorized guilds, channels, users, pairings, and roles create no thread and dispatch no message - -### AC-4: Gateway Orchestration - -- [ ] Gateway dispatches tasks to appropriate provider/model -- [ ] Routing decision logged and inspectable -- [ ] Fallback when primary provider unavailable -- [ ] Multiple concurrent agent sessions managed correctly - -### AC-5: Task & Project Management - -- [ ] CRUD operations for tasks, projects, missions via web dashboard -- [ ] Kanban board view for tasks -- [ ] Mission progress tracking with computed stats -- [ ] Brain MCP tools accessible from agent sessions - -### AC-6: Memory System - -- [ ] Agent sessions auto-capture decisions and learnings -- [ ] Semantic search returns relevant past context -- [ ] Learned preferences are applied in new sessions -- [ ] Log summarization runs on schedule, old logs compressed - -### AC-7: Authentication & RBAC - -- [ ] Email/password login works -- [ ] At least one SSO provider (Authentik) works end-to-end -- [ ] Admin can create users and assign roles -- [ ] RBAC enforced on API endpoints - -### AC-8: Multi-Provider LLM Support - -- [ ] At least 3 providers configured and routing correctly (e.g., Anthropic + Ollama + Z.ai) -- [ ] Agent routing selects appropriate model for task type -- [ ] Provider configuration manageable from web UI - -### AC-9: MCP - -- [ ] Gateway exposes MCP endpoint -- [ ] Brain and queue tools callable via MCP -- [ ] Agent sessions can connect to external MCP servers - -### AC-10: Deployment - -- [ ] PGlite data-layer work uses no PostgreSQL; optional Compose services are selected individually and do not start PostgreSQL; Gateway/Web local start remains held until KBN-101-02 rejects daemon/inherited/project DSNs before connection or DDL -- [ ] PostgreSQL/federated activation is unavailable until KBN-101-00/-03/-05 deliver external bootstrap, TLS/roles, runner `--run`, runner `--verify`, and Gateway/Compose readiness in that order -- [ ] `mosaic` CLI installable and functional on bare metal after the reviewed KBN-101-05 secret-renderer/process-exec or `LoadCredential` interface exists -- [ ] Local-only configuration documentation is distinct from production generation-pinned Vault-rendered consumer material - -### AC-11: @mosaicstack/\* Packages - -- [ ] All 7 migrated packages build, pass tests, and integrate with gateway -- [ ] `mosaic` CLI provides subcommands for each package -- [ ] Types package is the single source of shared interfaces - ---- - -## Constraints and Dependencies - -1. **Pi SDK** — Core dependency; any Pi breaking changes affect the agent layer. Pin to known-good version. -2. **BetterAuth** — Auth framework; must support SSO adapters. Verify Authentik/WorkOS/Keycloak support before committing. -3. **Drizzle ORM** — Database layer; must support PostgreSQL + pgvector extension. -4. **Discord API** — Rate limits, intent requirements, message size limits (2000 chars). -5. **Valkey** — Queue backend; must be available for queue and caching. -6. **Gitea registry** — Package publishing target; `.npmrc` must be configured. -7. **OpenClaw** — Reference architecture for Discord/Telegram plugin pattern (https://github.com/openclaw/openclaw). Inspiration only, not a dependency. - ---- - -## Risks and Open Questions - -### Risks - -| Risk | Likelihood | Impact | Mitigation | -| -------------------------------------------------- | ---------- | ------ | ---------------------------------------------------------------------------------------- | -| Pi SDK API instability (pre-1.0) | Medium | High | Pin version, abstract behind @mosaicstack/agent interface | -| Brain PG migration complexity | Medium | Medium | Preserve Brain REST/MCP API contract; only storage changes | -| Discord plugin complexity (OpenClaw has ~60 files) | Medium | Medium | Start minimal (DM + mention in channel), single-guild only; expand iteratively post-beta | -| LLM provider subscription auth varies by provider | Medium | Medium | Abstract behind provider interface; implement per-provider adapters | -| Drizzle + pgvector extension compatibility | Low | Medium | Validate in Phase 0 with spike | -| Agent log volume overwhelming storage | Medium | High | Tiered storage with aggressive summarization; configurable retention | -| Scope creep from jarvis-old feature surface | High | High | Strict v0.1.0 scope; features not listed above are post-beta | - -### Open Questions - -| # | Question | Priority | Status | -| --- | ------------------------------------------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 1 | Pi SDK version to pin for v0.1.0? | High | ✅ Resolved — Pin `@mariozechner/pi-coding-agent@~0.57.1` (current stable). Abstract behind `@mosaicstack/agent` interface to insulate from breaking changes. Bump deliberately after testing. | -| 2 | Authentik vs WorkOS vs Keycloak — which SSO provider to implement first? | Medium | ✅ Resolved — Authentik first (already in Jason's infrastructure) | -| 3 | Vector DB: pgvector sufficient or need Qdrant from the start? | Medium | ✅ Resolved — pgvector with VectorStore interface abstraction. Qdrant drops in later if needed. | -| 4 | Summarization LLM: which model for log compression? | Medium | ✅ Resolved — Haiku-tier default with structured output guardrails, configurable via routing engine. | -| 5 | LM Studio and llama.cpp — provider adapters exist in Pi or need custom? | Medium | ✅ Resolved — Pi handles both natively. LM Studio and llama.cpp (server mode) expose OpenAI-compatible APIs; configure via Pi's `models.json` with `openai-completions` API type. No custom adapters needed. | -| 6 | Discord bot — single guild or multi-guild from day one? | Medium | ✅ Resolved — Single-guild only for v0.1.0 to prevent data leaks. Bot binds to one guild. Multi-guild with tenant isolation is a post-beta feature requiring explicit data boundary design. | -| 7 | Bare-metal install — systemd units or just docs? | Low | ASSUMPTION: Docs + CLI launch commands; systemd units post-beta | - ---- - -## Testing and Verification Expectations - -1. **Baseline checks**: `pnpm typecheck && pnpm lint && pnpm test` must pass across all packages -2. **Unit tests**: Vitest for all packages; mocked dependencies for isolation -3. **Integration tests**: Gateway + Brain + Queue with test PG + Valkey (Docker services in CI) -4. **E2E tests**: Playwright for web dashboard critical paths (login, chat, task CRUD) -5. **Agent tests**: Pi SDK session tests with mock provider (verify tool registration, routing) -6. **Evidence format**: CI pipeline green + test count report per package - ---- - -## Milestone / Delivery Intent - -All work is **alpha** (< 0.1.0) until Jason approves 0.1.0 beta release. - -### Phase 0: Foundation (v0.0.1) - -- Scaffold monorepo (pnpm + turbo + tsconfig + eslint + vitest) -- `@mosaicstack/types` — migrate and extend from v0 -- `@mosaicstack/db` — Drizzle schema, PG connection, migrations -- `@mosaicstack/auth` — BetterAuth setup with email/password -- OTEL foundation — `@opentelemetry/sdk-node` setup, SigNoz in docker-compose, trace propagation wired -- Docker Compose (PG 17 + Valkey + SigNoz) -- CI pipeline (Woodpecker) -- AGENTS.md, CLAUDE.md, README.md - -### Phase 1: Core API (v0.0.2) - -- `apps/gateway` — NestJS server (Fastify adapter), auth middleware, health endpoints -- `@mosaicstack/brain` — migrate from v0, swap JSON store for PG via @mosaicstack/db -- `@mosaicstack/queue` — migrate from v0 (minimal changes) -- Gateway routes: conversations, tasks, projects, missions -- WebSocket server for chat streaming -- Basic agent dispatch (single provider, no routing) - -### Phase 2: Agent Layer (v0.0.3) - -- `@mosaicstack/agent` — Pi SDK integration, agent pool manager -- Multi-provider support (Anthropic + Ollama minimum) -- Agent routing engine (cost/capability matrix) -- Tool registration (brain, queue, memory tools injected into agent sessions) -- `@mosaicstack/coord` — migrate from v0, integrate with gateway - -### Phase 3: Web Dashboard (v0.0.4) - -- `apps/web` — Next.js app with BetterAuth -- Chat UI (conversation list, message display, streaming input) -- Task management (list + kanban) -- Project and mission views -- Settings (provider config, profile) -- Admin panel (user management, RBAC) - -### Phase 4: Memory & Intelligence (v0.0.5) - -- `@mosaicstack/memory` — preference store, insight store, semantic search -- `@mosaicstack/log` — log ingest, parsing, tiered storage -- Summarization pipeline -- Memory integration into agent sessions -- Skill management interface (web UI + CLI) - -### Phase 5: Remote Control (v0.0.6) - -- `@mosaicstack/discord-plugin` — Discord channel plugin -- `@mosaicstack/telegram-plugin` — Telegram channel plugin -- Plugin host in gateway -- SSO configuration (Authentik) - -### Phase 6: CLI & Tools (v0.0.7) - -- `@mosaicstack/cli` — unified CLI with all subcommands -- `@mosaicstack/prdy` — migrate from v0 -- `@mosaicstack/quality-rails` — migrate from v0 -- `@mosaicstack/mosaic` — install wizard updated for v1 -- Pi TUI integration (`mosaic tui`) - -### Phase 7: Polish & Beta (v0.0.8 → v0.1.0) - -- MCP endpoint hardening -- Additional SSO providers (WorkOS/Keycloak) -- Additional LLM providers (Codex, Z.ai, LM Studio, llama.cpp) -- Bare-metal deployment documentation -- E2E test suite -- Performance optimization -- Documentation: user guide, admin guide, developer guide -- **Jason approval gate → v0.1.0 beta release** - ---- - ## Workspace placement guard hardening (#1174) ### Problem and objective @@ -1596,32 +850,6 @@ will route around it. --- -## Assumptions - -1. RESOLVED: **pgvector is sufficient** for semantic search at v0.1.0 scale (personal/family/team = thousands to low hundreds-of-thousands of vectors). `@mosaicstack/memory` defines a `VectorStore` interface with pgvector as the default adapter. The interface boundary makes Qdrant a drop-in migration if PG resource contention or scale demands it later. Zero additional infrastructure for v0.1.0. Rationale: Reduces ops burden; pgvector HNSW indexes are fast at this scale; interface abstraction costs almost nothing now. - -2. RESOLVED: **Authentik is the first SSO provider** — confirmed, already running in Jason's infrastructure. WorkOS and Keycloak adapters follow in Phase 7. - -3. RESOLVED: **NestJS with Fastify adapter for the gateway.** The gateway's complexity (plugin host, agent pool, routing engine, WebSocket hub, MCP server, auth, brain/queue/memory/log integration) warrants NestJS's module system, DI, and guards. Fastify performance preserved via adapter. Aligns with USER.md stated stack ("NestJS API + Next.js web"). @mosaicstack/brain's Fastify code migrates into a NestJS module. - -4. RESOLVED: **OpenTelemetry from Phase 0.** Wide-event logging is required from the start. OTEL auto-instrumentation for NestJS/PG/HTTP via `@opentelemetry/sdk-node`. SigNoz as the all-in-one OTEL backend (single Docker service). Every significant operation emits structured events with rich context. Custom spans for agent dispatch, routing decisions, memory writes. Rationale: Retrofitting observability is painful; baking it in from day one means consistent instrumentation across all services. - -5. ASSUMPTION: **Single-node deployment for v0.1.0**, but code structured for multi-tier. No Kubernetes yet. Docker Compose + bare metal. Service boundaries use HTTP/WS/MCP APIs (not in-process) so gateway, agent pool, and brain can split to separate nodes later. Rationale: Ship single-node MVP; the architecture doesn't fight horizontal scaling when needed. - -6. ASSUMPTION: **Log summarization uses Haiku-tier LLM by default, configurable.** Haiku is well-suited for summarization (compression, not generation — source material is in context). Guardrails: structured output via Zod schema (force extraction of decisions/tools/outcomes/errors as discrete fields), chunked per-session processing (no bulk conflation), extraction-focused prompts. Raw logs stay in hot tier (7 days) as safety net. Users can override the summarization model via routing engine config if they want higher fidelity. Rationale: Haiku is 10-20x cheaper than Sonnet; log summarization runs on schedule against large volumes where cost matters. - -7. ASSUMPTION: **Discord plugin starts minimal and single-guild only** — explicitly configured agent-bound channels accept authorized untagged messages in-channel, while mentions create threads or reuse a thread already attached to that same native message; responses are chunked. Single guild binding prevents data leaks between servers. DM support, voice, components, slash commands, and multi-guild operation are post-beta. Rationale: Ship the requested core interaction model while preserving default-deny data isolation. - -8. ASSUMPTION: **Telegram plugin is lower priority than Discord** and may ship as v0.0.7 or later if Discord takes longer than expected. Rationale: Jason indicated Discord as the high-priority remote channel. - -9. ASSUMPTION: **Brain's REST API is preserved** as a gateway sub-router (mounted at `/api/brain/*` or similar). Existing MCP tools continue to work. Only the storage backend changes. Rationale: Minimize migration risk; brain's API contract is proven. - -10. ASSUMPTION: **Conversations and messages get their own PG tables** (not stored in brain's entity model). They follow a chat-specific schema with proper foreign keys to users and projects. Rationale: Chat has different access patterns (streaming, pagination, search) than brain entities. - -11. RESOLVED: **Pi handles all target LLM providers natively.** Anthropic, OpenAI/Codex, Z.ai, Ollama, LM Studio, and llama.cpp are all supported via Pi's built-in providers or `models.json` configuration with `openai-completions` API type. No custom provider adapters needed in @mosaicstack/agent — only configuration management. - ---- - ## Release Integrity Workstream (RI, #1275) ### Problem and objective diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 00000000..306f9f78 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,77 @@ +--- +kind: spec +status: active +--- + +# Mosaic Stack Roadmap + +Companion to [docs/PRD.md](./PRD.md). Governed by the D11 rule: **every planned +phase appears here from day one, even as a placeholder** — nothing exists only +in heads. A phase marked _placeholder_ is a commitment to design it, not a +design; scoping one requires its own PRD section or requirements doc plus +review. + +Phases are product phases. The in-flight platform workstreams (KBN-100/101 +kanban SOT implementation, FCM #758, FCOM #766, TESS, RI #1275, and the other +Part II contracts in the PRD) run as parallel tracks under their own issues +and are prerequisites where noted. + +| Phase | Scope | Status | +| ----- | ------------------------------------------------------------------------------ | ----------------------------------------- | +| P0 | Current state on `next`: read-only dashboard, chat, auth/SSO login, admin tabs | shipped, evolving | +| P1 | **v1 slice** (PRD Part I §9) | next up | +| P2 | Connectors + comms + wizard expansion | placeholder | +| P3 | Full onboarding profile + M365 | placeholder | +| P4 | Enterprise mode + one-way conversion | placeholder | +| P5 | Federation | placeholder (deliberately undesigned, D3) | + +## P0 — current state + +What exists on `next` today: web dashboard (login/register/SSO, chat, +read-only projects/tasks, settings, admin user/system-health tabs), the +Gateway, the CLI-first framework tooling, and the fleet control plane. The +webUI audit (USC estate, webui-audit lane) measures the gap between this and +P1. + +## P1 — v1 slice (D11) + +1. Standalone onboarding wizard: system/company name, component choices, + initial user, initial estate + project, seeded examples, re-runnable. +2. Hierarchy core: company → estate → project → workspace → kanban, read-only + task bubble-up (kanban SOT Amendment A1 is the schema contract). +3. Basic RBAC on the hierarchy. +4. Minimal agent enrollment: one harness, API key, name/persona. + +Prerequisites: KBN-100/101 schema foundation; the D8 tool inventory and +webUI→tool mapping (any missing tool is built first, D12). + +## P2 — connectors + comms + wizard expansion (placeholder) + +Email and drive connectors (Gmail/IMAP, Google Drive/OneDrive/Dropbox) with +granular agentic-access consent; comms integrations (Matrix/Discord/Slack) +including agent auto-enroll. Wizard gains the corresponding tabs (D4), plus +the D4 capabilities deferred out of P1's minimal slice: expanded agent +enrollment (OAuth login, multi-account, model choice with recommendation, +account assignment, comms auto-enroll) and the Standalone SSO/OIDC +configuration tab. + +## P3 — full onboarding profile + M365 (placeholder) + +Complete user onboarding profile (communication-style capture, optional +voice-matching interview) under the D14 custody rule; M365 connectors, +available to both deployment modes as ordinary connectors (same consent model +as the P2 connector class). The Enterprise install flow's M365 prominence +(D4) arrives with the Enterprise phase, P4. + +## P4 — Enterprise mode + conversion (placeholder) + +Enterprise install flow (org chart, RBAC focus, immediate OIDC, SSO +prominent); per-user brains with architectural isolation (D14); Vault +required; the one-way Standalone → Enterprise conversion (D3). + +## P5 — federation (placeholder) + +Connecting deployments: system-level config, assigned users, rights and +data-access control, trusts with boundaries, strict data access, exfiltration +monitoring. Explicitly not designed yet (D3); nothing in earlier phases may +foreclose it. Requires its own PRD + threat model before any scoping. diff --git a/docs/archive/PRD-v0.1.md b/docs/archive/PRD-v0.1.md new file mode 100644 index 00000000..dca91df2 --- /dev/null +++ b/docs/archive/PRD-v0.1.md @@ -0,0 +1,1028 @@ +--- +kind: archive +status: superseded +--- + +# PRD: Mosaic Stack v0.1.0 (ARCHIVED) + +Archived 2026-08-26 by the north-star PRD rewrite (`docs/PRD.md`). This is the +v0.1.0 beta PRD body (dated 2026-03-12) preserved verbatim for reference. It is +no longer product authority. The active workstream contracts that previously +shared the file (KBN-101, FCM #758, FCOM #766, TESS, #756, MOS-PORT, #1150, +#1174, #1194, RI #1275, M1 #827–#830) were NOT archived — they remain normative +in `docs/PRD.md` Part II. + +--- + +## Metadata + +- **Owner:** Jason Woltje +- **Date:** 2026-03-12 +- **Status:** draft +- **Best-Guess Mode:** true +- Repo (target): `git.mosaicstack.dev/mosaic/mosaic-stack` +- Baseline: `~/src/jarvis-old` (jarvis v0.2.0) +- Package source: `~/src/mosaic-mono-v0` (@mosaicstack/\* packages) +- Agent harness: [pi](https://github.com/badlogic/pi-mono) (v0.57.1) +- Remote control reference: [OpenClaw](https://github.com/openclaw/openclaw) (upstream, canonical) + +--- + +## Problem Statement + +Jarvis (v0.2.0) is a self-hosted AI assistant with a Python FastAPI backend and Next.js frontend. It handles chat, projects, tasks, and LLM routing but lacks orchestration depth, agent coordination, shared memory, and remote access. The Mosaic framework (`~/.config/mosaic`) provides agent guides, shell-based orchestration tools, and quality rails — but these are loose scripts, not an integrated platform. The `@mosaicstack/*` packages in mosaic-mono-v0 began consolidating these into TypeScript packages (brain, queue, coord, cli, prdy, quality-rails) but have no UI, no auth, and no agent runtime integration. + +**The gap:** Three codebases with overlapping concerns, no unified runtime, no remote control surface (Discord/Telegram), no gateway orchestrator, and a Python backend that doesn't align with the target TypeScript-everywhere stack. + +**What Mosaic Stack solves:** A single monorepo that brings together many pieces to make one beautiful picture — a self-hosted, multi-user AI agent platform with web dashboard, TUI, remote control, shared memory, mission orchestration, and extensible skill/plugin architecture. All TypeScript. Pi as the agent harness. Brain as the knowledge layer. Queue as the coordination backbone. + +--- + +## Objectives + +1. **Unified TypeScript monorepo** — One repo, one language, one build pipeline for all Mosaic Stack components +2. **Pi-powered agent runtime** — Pi SDK embedded as the core agent loop; Pi TUI as the terminal interface +3. **Web + TUI + Remote** — Next.js dashboard for visual management, Pi TUI for terminal work, Discord/Telegram for remote control +4. **Gateway orchestrator** — Central routing layer that dispatches tasks to appropriate agents based on capability, cost, and context +5. **Shared memory** — PostgreSQL canonical store + vector DB for semantic search + tiered log summarization to prevent context creep +6. **Multi-user with SSO** — BetterAuth with Authentik/WorkOS/Keycloak SSO, RBAC for family/team/business use +7. **Full @mosaicstack/\* package integration** — brain, queue, coord, mosaic, prdy, quality-rails, cli all integrated +8. **Extensible** — MCP capability, skill import interface, plugin architecture for LLM providers and remote channels + +--- + +## Scope + +### In Scope (v0.1.0 Beta) + +1. Chat/conversation UI (web) — carry forward from jarvis-old, rewrite frontend to work with new backend +2. Pi TUI integration — terminal-based agent interaction using Pi SDK +3. Web dashboard — settings, task management, projects, PRDs, missions, agent status +4. Gateway orchestrator (`@mosaicstack/gateway`) — central dispatch for agent tasks with routing logic +5. Task management — CRUD, kanban, mission-scoped tasks, dependency tracking +6. Project management — projects, milestones, PRDs linked to missions +7. Shared memory system — learned preferences, behaviors, defaults; tiered storage with summarization +8. User management — RBAC (admin, member, viewer), multi-user capable +9. SSO — BetterAuth with Authentik/WorkOS/Keycloak adapter +10. Remote control — Discord plugin (high priority), Telegram plugin +11. LLM provider support — Anthropic subs, Codex subs, Z.ai subs, other API-based, Ollama, LM Studio, llama.cpp +12. Agent routing — task-based model/provider selection (cost/capability matrix) +13. MCP capability — server and client, tool registration +14. Skill import interface — browse, install, manage agent skills +15. `@mosaicstack/brain` — structured data layer (migrated to PG + vector DB backend) +16. `@mosaicstack/queue` — Valkey-backed task queue with MCP tools +17. `@mosaicstack/coord` — mission coordination engine +18. `@mosaicstack/mosaic` — install wizard / bootstrap +19. `@mosaicstack/prdy` — PRD wizard +20. `@mosaicstack/quality-rails` — code quality scaffolder +21. `@mosaicstack/cli` — unified `mosaic` CLI +22. Docker Compose deployment + bare-metal capability +23. Agent log service — ingest, parse, tier, summarize agent interaction logs +24. Local durable agent fleet canary — `mosaic fleet` / `mosaic agent` CLI for an isolated tmux-backed canary fleet using a named socket, with roster-driven local customization and rollback-safe verification + +### Out of Scope (v0.1.0) + +1. SaaS / multi-tenant revenue model — this is a personal/family/team tool +2. Mobile native apps — web responsive is sufficient +3. Public npm registry publishing — Gitea registry only +4. Video/voice agent interaction +5. Full OpenClaw feature parity — we take inspiration, not wholesale migration +6. Calendar integration (deferred — brain tracks events, but no gcal sync yet) +7. GLPI/helpdesk ticket sync (deferred) +8. Woodpecker CI integration tooling (deferred — focus on core platform first) + +--- + +## Architecture + +### High-Level System Diagram + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Mosaic Stack │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ ┌──────────────┐ │ +│ │ Next.js │ │ Pi TUI │ │ Discord │ │ Telegram │ │ +│ │ Web App │ │ Terminal │ │ Plugin │ │ Plugin │ │ +│ └────┬─────┘ └────┬─────┘ └──────┬──────┘ └──────┬───────┘ │ +│ │ │ │ │ │ +│ └──────────────┴───────┬───────┴────────────────┘ │ +│ │ │ +│ ┌─────────▼──────────┐ │ +│ │ @mosaicstack/gateway │ ← Central Orchestrator│ +│ │ (NestJS+Fastify) │ │ +│ └────┬────┬────┬─────┘ │ +│ │ │ │ │ +│ ┌──────────────┤ │ ├──────────────┐ │ +│ │ │ │ │ │ │ +│ ┌───────▼──────┐ ┌────▼────▼──┐ │ ┌───────────▼────────┐ │ +│ │ @mosaicstack/brain│ │ @mosaicstack/ │ │ │ Agent Pool │ │ +│ │ (Data Layer) │ │ queue │ │ │ (Pi SDK sessions) │ │ +│ └───────┬──────┘ └────────────┘ │ │ - Anthropic │ │ +│ │ │ │ - Codex │ │ +│ ┌───────▼──────────────────┐ │ │ - Z.ai │ │ +│ │ PostgreSQL │ VectorDB │ │ │ - Ollama │ │ +│ │ (canonical) │ (semantic)│ │ │ - LM Studio │ │ +│ └──────────────┴───────────┘ │ │ - llama.cpp │ │ +│ │ └────────────────────┘ │ +│ ┌─────────────▼──────┐ │ +│ │ @mosaicstack/coord │ │ +│ │ Mission lifecycle │ │ +│ └────────────────────┘ │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │ +│ │ @mosaicstack/cli │ │ @mosaicstack/prdy │ │ @mosaicstack/ │ │ +│ │ │ │ │ │ quality-rails │ │ +│ └──────────────┘ └──────────────┘ └──────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────┐ │ +│ │ Valkey (queue backend) │ BetterAuth (SSO/RBAC) │ │ +│ └──────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### Technology Decisions + +| Layer | Technology | Rationale | +| ------------------ | ------------------------------------ | ----------------------------------------------------------------------------------------------------------- | +| **Web Frontend** | Next.js 16 + React 19 + Tailwind CSS | SSR, RSC; design tokens from @mosaicstack/design-tokens (mosaic-stack-website) | +| **API / Gateway** | NestJS + Fastify adapter | Module system, DI, guards/interceptors for complex gateway; Fastify performance underneath | +| **Agent Runtime** | Pi SDK (embedded) | Extensible harness with tools, skills, session management | +| **TUI** | Pi interactive mode | Native terminal agent interaction | +| **Auth** | BetterAuth + SSO adapters | Multi-user RBAC with Authentik/WorkOS/Keycloak | +| **Database** | PostgreSQL 17 + pgvector | Canonical store; pgvector for embedding search | +| **Vector DB** | pgvector + VectorStore interface | pgvector for v0.1.0; `VectorStore` abstraction in @mosaicstack/memory makes Qdrant a drop-in later | +| **Cache / Queue** | Valkey 8 | Redis-compatible; proven in @mosaicstack/queue | +| **ORM** | Drizzle ORM | TypeScript-native, lightweight, good migration story | +| **Validation** | Zod | Already used across @mosaicstack/\* packages | +| **Build** | pnpm workspaces + Turborepo | Proven in both jarvis-old and mosaic-mono-v0 | +| **Testing** | Vitest + Playwright | Unit/integration via Vitest, E2E via Playwright | +| **Remote Control** | Discord.js + Telegraf | Inspired by OpenClaw plugin architecture | +| **MCP** | @modelcontextprotocol/sdk | Already used in @mosaicstack/brain and @mosaicstack/queue | +| **Container** | Docker Compose | Self-hosted; bare-metal also supported | +| **CI** | Woodpecker CI | Existing infrastructure at git.mosaicstack.dev | +| **Observability** | OpenTelemetry + SigNoz | Wide-event logging from day one; OTEL auto-instrumentation for NestJS/PG/HTTP; SigNoz as all-in-one backend | +| **Log Processing** | Custom ingest service | Parse agent logs → tiered storage → summarization | + +### Key Architecture Decisions + +**AD-1: TypeScript everywhere (no Python backend)** +The jarvis-old FastAPI backend is not carried forward as code. Its domain logic (conversation management, LLM routing, task/project CRUD, auth) is reimplemented in TypeScript. The Python plugin system is replaced by Pi's extension/skill system and MCP tool registration. + +**AD-2: Pi SDK as the agent runtime** +Instead of a custom LLM provider abstraction (jarvis-old's `BaseLLMProvider`), Pi SDK manages agent sessions. Pi handles model selection, tool calling, context management, and compaction. The gateway dispatches work to Pi sessions configured with appropriate providers. + +**AD-3: Gateway as the central nervous system (NestJS + Fastify adapter)** +`@mosaicstack/gateway` is the single API surface. The web app, TUI, Discord, and Telegram all talk to the gateway. The gateway routes to brain (data), queue (coordination), agent pool (LLM work), and coord (mission lifecycle). This replaces the direct FastAPI-to-DB pattern from jarvis-old. + +NestJS was chosen over raw Fastify because the gateway is inherently complex — it hosts channel plugins, agent pool management, routing engine, WebSocket hub, MCP server, auth middleware, and integrates brain, queue, memory, and log services. NestJS provides the module system, dependency injection, guards, and interceptors needed to organize this cleanly. NestJS uses Fastify as its HTTP adapter, so Fastify's performance is preserved. This also aligns with the stated stack preference in USER.md ("NestJS API + Next.js web"). @mosaicstack/brain's existing Fastify code migrates naturally into a NestJS module with Fastify adapter. + +**AD-4: Brain migrates from JSON files to PostgreSQL** +`@mosaicstack/brain` currently uses a JSON file store. For Mosaic Stack, brain's data model (tasks, projects, events, agents, missions, tickets) moves to PostgreSQL via Drizzle ORM. Brain's REST + MCP interface is preserved — only the storage backend changes. + +**AD-5: Tiered memory with summarization** +Agent interaction logs are ingested into a log service. Raw logs are stored short-term. A summarization pipeline (using a cheap LLM) periodically compresses logs into structured insights stored in the vector DB. This prevents unbounded log growth while preserving searchable context. + +**AD-6: Remote control via plugin architecture** +Discord and Telegram plugins follow a channel plugin pattern inspired by OpenClaw (https://github.com/openclaw/openclaw). Each plugin registers as a channel with the gateway, receives messages, and dispatches them through the same routing pipeline as web/TUI messages. + +**AD-7: Gateway state persistence via Valkey (restart resilience)** +The gateway persists its orchestration state (active sessions, pending dispatches, routing context, agent assignments) to Valkey. On restart, the gateway reads Valkey state and resumes operations — active agent sessions are reconnected or gracefully recovered. `mosaic gateway restart --fresh` is the nuclear option: clears the Valkey queue and all in-flight state, starting with a clean slate. This prevents context/focus/direction loss that would otherwise occur on every restart. + +**AD-8: Multi-session agent architecture** +Each agent operates in a distinct session. Multiple authorized input channels (TUI, web UI, Discord) can connect to the same agent session simultaneously. This means a user can start a conversation in Discord, continue in the web UI, and monitor via TUI — all feeding into the same agent context. OpenClaw has this concept; Mosaic Stack evolves it with proper session authorization and channel multiplexing at the gateway level. + +**AD-9: Discord channel-to-agent binding** +Discord channels pair to specific agent/session combinations via channel ID binding. This provides data segregation — messages in #project-alpha route to the project-alpha agent session, messages in #general route to a general-purpose session. Prevents cross-contamination between contexts and provides clear boundaries for multi-channel use. + +**AD-10: Agent session barge-in via tmux** +Each agent session runs in a dedicated, named tmux session (e.g., `mosaic-agent-project-alpha`). This enables barge-in — a user can attach to any active agent's tmux session to observe, interrupt, or redirect. `mosaic agent attach ` connects to the tmux session. This provides direct low-level access when the normal channel interfaces are insufficient. + +**AD-11: Cron-based scheduled jobs** +The gateway includes a cron scheduler for recurring tasks: log summarization runs, stale task detection, memory decay, provider health checks, scheduled agent dispatches. Uses node-cron or similar; schedules are configurable via web dashboard and stored in PG. Each cron job is a gateway-dispatched task that goes through the normal routing pipeline. + +**AD-12: Web search tool (DuckDuckGo MCP)** +Agent sessions include a web search tool for information retrieval. DuckDuckGo via MCP server is the primary option (privacy-respecting, no API key required). Falls back to other search MCP providers if configured. Registered as a standard MCP tool available to all agent sessions. + +**AD-13: Design system from @mosaicstack/design-tokens** +The web dashboard uses the Mosaic Stack design system established in `mosaic-stack-website`. The `@mosaicstack/design-tokens` package provides CSS custom properties, Tailwind preset, and TS color/font/radius exports. Dark theme default with light theme support. Fonts: Outfit (sans), Fira Code (mono). Color palette: deep blue-grays with blue/purple/teal accents. + +**AD-14: Multi-tier deployment readiness** +Code is structured assuming eventual multi-node deployment with dedicated roles (gateway nodes, agent worker nodes, brain/DB nodes). Packages communicate via well-defined APIs (HTTP/WS/MCP), not in-process calls where avoidable. Service boundaries are clean: gateway is stateless (state in PG/Valkey), agent pool can scale independently, brain is a separate service. v0.1.0 runs single-node; the architecture doesn't fight horizontal scaling later. + +--- + +## Package Structure + +### Monorepo Layout + +``` +mosaic-mono-v1/ +├── apps/ +│ ├── web/ Next.js 16 web dashboard +│ └── gateway/ @mosaicstack/gateway — NestJS API + WebSocket +├── packages/ +│ ├── types/ @mosaicstack/types — shared type contracts +│ ├── brain/ @mosaicstack/brain — data layer (PG-backed) +│ ├── queue/ @mosaicstack/queue — Valkey task queue + MCP +│ ├── coord/ @mosaicstack/coord — mission coordination +│ ├── mosaic/ @mosaicstack/mosaic — install wizard +│ ├── prdy/ @mosaicstack/prdy — PRD wizard +│ ├── quality-rails/ @mosaicstack/quality-rails — code quality scaffolder +│ ├── cli/ @mosaicstack/cli — unified CLI +│ ├── auth/ @mosaicstack/auth — BetterAuth config + SSO adapters +│ ├── db/ @mosaicstack/db — Drizzle schema, migrations, connection +│ ├── agent/ @mosaicstack/agent — Pi SDK integration, agent pool manager +│ ├── memory/ @mosaicstack/memory — tiered memory + summarization service +│ ├── log/ @mosaicstack/log — agent log ingest + processing +│ └── design-tokens/ @mosaicstack/design-tokens — CSS vars, Tailwind preset, colors +├── plugins/ +│ ├── discord/ @mosaicstack/discord-plugin — Discord channel +│ └── telegram/ @mosaicstack/telegram-plugin — Telegram channel +├── docker/ +│ ├── gateway.Dockerfile +│ ├── web.Dockerfile +│ └── init-db.sql +├── docs/ +│ ├── PRD.md (this file) +│ ├── TASKS.md +│ └── scratchpads/ +├── docker-compose.yml +├── pnpm-workspace.yaml +├── turbo.json +├── tsconfig.base.json +├── vitest.workspace.ts +├── AGENTS.md +├── CLAUDE.md +└── README.md +``` + +### Package Responsibilities + +#### `apps/gateway` — @mosaicstack/gateway (NEW — critical path) + +The central nervous system. All clients connect here. Built with NestJS (Fastify adapter). + +- **NestJS modules** — Each concern (chat, brain, agent, auth, queue, memory, plugins) is a module with clear boundaries +- **Fastify adapter** — Fastify performance under NestJS's organizational structure +- **WebSocket gateway** — NestJS built-in WebSocket support for chat streaming, agent status, notifications +- **Agent routing engine** — Routes tasks to appropriate LLM provider/model based on task type, cost tier, capability requirements +- **Session management** — Tracks active conversations, agent sessions, user contexts +- **MCP server** — Exposes Mosaic capabilities as MCP tools +- **Plugin host** — Loads and manages channel plugins (Discord, Telegram) +- **Auth middleware** — BetterAuth session validation, RBAC enforcement + +Key routes: + +``` +POST /api/chat Send message, get streamed response +GET /api/conversations List conversations +POST /api/conversations Create conversation +GET /api/conversations/:id Get conversation with messages +DELETE /api/conversations/:id Delete conversation +POST /api/tasks Create task (brain-backed) +GET /api/tasks List/filter tasks +PATCH /api/tasks/:id Update task +GET /api/projects List projects +POST /api/projects Create project +GET /api/missions List missions +POST /api/missions Create mission +GET /api/missions/:id Mission summary with tasks +POST /api/agents/dispatch Dispatch work to agent pool +GET /api/agents/status Active agent sessions +GET /api/memory/search Semantic search across memory +POST /api/memory/preferences Store learned preference +GET /api/skills List available skills +POST /api/skills/install Install a skill +GET /api/providers List configured LLM providers +POST /api/providers Configure LLM provider +GET /api/admin/users User management (admin) +POST /api/admin/users Create user (admin) +WS /ws/chat/:conversationId Streaming chat via WebSocket +WS /ws/agents Agent status stream +GET /mcp MCP endpoint (streamable HTTP) +``` + +#### `apps/web` — Next.js Web Dashboard + +Carried forward from jarvis-old with significant refactoring. + +- Chat/conversation UI (primary interaction surface) +- Settings management (providers, integrations, profile) +- Task management (list, kanban, detail views) +- Project management (list, detail, linked missions) +- Mission dashboard (status, progress, task breakdown) +- PRD viewer/editor +- Agent status dashboard (active sessions, routing stats) +- Skill browser and installer +- User management (admin RBAC panel) +- Auth pages (login, SSO redirect, registration) + +#### `packages/types` — @mosaicstack/types + +Migrated from mosaic-mono-v0. Extended with: + +- Gateway types (routing, dispatch, agent pool) +- Auth types (user, role, permission) +- Conversation/message types (from jarvis-old domain) +- Memory types (preference, insight, summary) +- Plugin channel types (Discord, Telegram message mapping) + +#### `packages/brain` — @mosaicstack/brain + +Migrated from mosaic-mono-v0. **Storage backend changes from JSON to PostgreSQL.** + +- REST API preserved (mounted as gateway sub-router or standalone) +- MCP tools preserved +- Collections layer rewritten to use Drizzle ORM queries instead of JSON file I/O +- Same entity model: tasks, projects, events, agents, missions, mission-tasks, tickets +- New: computed endpoints (today, stale, stats, search, audit) run against PG +- New: appreciation collection preserved for family use + +#### `packages/queue` — @mosaicstack/queue + +Migrated from mosaic-mono-v0 with minimal changes. + +- Valkey-backed task queue with atomic WATCH/MULTI/EXEC +- MCP server with 8 tools +- Used by gateway for agent task dispatch and coordination + +#### `packages/coord` — @mosaicstack/coord + +Migrated from mosaic-mono-v0. + +- Mission lifecycle: init, run, resume, status, drain +- TASKS.md parsing and management +- Session lock management +- Continuation prompt generation +- Integration with gateway for mission-driven orchestration + +#### `packages/db` — @mosaicstack/db (NEW) + +Shared database package. + +- Drizzle ORM schema definitions (all tables) +- Migration management +- Connection pool configuration +- Shared by gateway, brain, auth, memory + +#### `packages/auth` — @mosaicstack/auth (NEW) + +Authentication and authorization. + +- BetterAuth configuration +- SSO adapters: Authentik, WorkOS, Keycloak +- RBAC: roles (admin, member, viewer), permissions +- API key generation for brain/MCP access +- Session management middleware + +#### `packages/agent` — @mosaicstack/agent (NEW — critical path) + +Pi SDK integration layer. + +- Agent pool manager — spawns and manages Pi agent sessions +- Provider configuration — Anthropic, Codex, Z.ai, Ollama, LM Studio, llama.cpp +- Agent routing logic — selects provider/model based on task characteristics +- Tool registration — registers Mosaic-specific tools (brain access, queue ops, memory search) +- Skill management — loads and configures Pi skills for agent sessions +- Session lifecycle — create, monitor, complete, fail, timeout + +#### `packages/memory` — @mosaicstack/memory (NEW) + +Tiered memory system. + +- Preference store — learned user preferences, behaviors, defaults (PG) +- Insight store — distilled knowledge from agent interactions (PG + vector) +- Semantic search — query across memory using pgvector embeddings +- Summarization pipeline — compress raw logs into structured insights +- Memory API — used by gateway and agent sessions + +#### `packages/log` — @mosaicstack/log (NEW) + +Agent log service. + +- Log ingest — receives structured logs from agent sessions +- Log parsing — extracts decisions, learnings, tool usage patterns +- Tiered storage — hot (recent, full detail), warm (summarized), cold (archived) +- Summarization trigger — invokes cheap LLM to compress aging logs +- Retention policy — configurable TTLs per tier + +#### `packages/mosaic` — @mosaicstack/mosaic + +Migrated from mosaic-mono-v0, updated for v1. + +- Install wizard for Mosaic Stack setup +- Detects existing installations, offers upgrade path +- Configures `~/.config/mosaic/` with guides, tools, runtime configs + +#### `packages/prdy` — @mosaicstack/prdy + +Migrated from mosaic-mono-v0. + +- PRD generation wizard +- Template-based PRD creation with Zod validation +- CLI integration via `mosaic prdy` + +#### `packages/quality-rails` — @mosaicstack/quality-rails + +Migrated from mosaic-mono-v0. + +- TypeScript scaffolder for project quality config +- Generates ESLint, tsconfig, Woodpecker, husky, lint-staged configs +- Supports project types: monorepo, typescript-node, nextjs + +#### `packages/cli` — @mosaicstack/cli + +Migrated from mosaic-mono-v0, extended. + +- Unified `mosaic` binary +- Subcommands: `mosaic coord`, `mosaic prdy`, `mosaic queue`, `mosaic quality`, `mosaic gateway`, `mosaic brain` +- Plugin discovery for installed @mosaicstack/\* packages + +#### `plugins/discord` — @mosaicstack/discord-plugin (NEW — high priority) + +Discord remote control channel. Architecture inspired by OpenClaw (https://github.com/openclaw/openclaw). + +- Channel plugin that registers with the gateway as a NestJS dynamic module +- Single-guild binding only (v0.1.0) — prevents data leaks between servers +- Receives Discord messages, dispatches through gateway routing +- Streams agent responses back to Discord (chunked for 2000-char limit) +- Routes authorized untagged messages in-channel; mentions create threads (or reuse the same message's attached thread) for multi-turn topics +- Uses stable logical-agent/channel conversation addresses independent of the active harness/provider +- Bot pairing and permission management (Discord user → Mosaic user mapping) +- DM support for private conversations + +#### `plugins/telegram` — @mosaicstack/telegram-plugin (NEW) + +Telegram remote control channel. + +- Same channel plugin pattern as Discord +- Telegraf-based bot +- Message routing through gateway +- Inline keyboard for interactive responses + +--- + +## User/Stakeholder Requirements + +### US-001 Multi-Channel Chat + +**As a user**, I can chat with an AI assistant via web browser, terminal (Pi TUI), Discord, or Telegram and get consistent responses regardless of channel. + +### US-002 Task & Project Dashboard + +**As a user**, I can manage my tasks, projects, and missions from the web dashboard with kanban and list views. + +### US-003 PRD Management + +**As a user**, I can view and edit PRDs for active missions from the web dashboard. + +### US-004 Agent Visibility + +**As a user**, I can see which agents are active, what they're working on, and their status in real-time. + +### US-005 Provider Configuration + +**As a user**, I can configure which LLM providers to use and set routing preferences (cost vs capability). + +### US-006 Skill Management + +**As a user**, I can install and manage agent skills through the web dashboard. + +### US-007 Persistent Memory + +**As a user**, the system remembers my preferences, learned behaviors, and past decisions across sessions. + +### US-008 Semantic Search + +**As a user**, I can search across my memory, conversations, and knowledge semantically. + +### US-009 User Management + +**As an admin**, I can manage users, assign roles, and control access. + +### US-010 SSO Configuration + +**As an admin**, I can configure SSO via Authentik, WorkOS, or Keycloak. + +### US-011 Self-Hosted Deployment + +**As a user**, I can run Mosaic Stack via Docker Compose or directly on bare metal. + +### US-012 Intelligent Routing + +**As an agent operator**, the gateway intelligently routes tasks to the cheapest capable model. + +### US-013 CLI Tooling + +**As a user**, I can use the `mosaic` CLI for PRD creation, quality rail setup, queue management, and mission coordination. + +--- + +## Functional Requirements + +- FR-1: Chat System +- FR-2: Gateway Orchestrator +- FR-3: Agent Pool +- FR-4: Task Management +- FR-5: Project Management +- FR-6: Mission System +- FR-7: Memory System +- FR-8: Authentication & Authorization +- FR-9: Remote Control — Discord +- FR-10: Remote Control — Telegram +- FR-11: LLM Provider Management +- FR-12: Agent Routing +- FR-13: MCP Capability +- FR-14: Skill Management +- FR-15: CLI Integration +- FR-16: Log Service +- FR-17: Gateway State Persistence +- FR-18: Multi-Session Agent Architecture +- FR-19: Cron Scheduler +- FR-20: Web Search Tool +- FR-21: Skill Import from skills.sh + +### FR-1: Chat System + +- Conversation CRUD (create, list, get with messages, delete) +- Real-time streaming responses via WebSocket +- Multi-provider support (route to configured LLM) +- Conversation history with search +- Project-scoped conversations +- System prompt per project/conversation +- Message rendering with markdown, code blocks, tool call display + +### FR-2: Gateway Orchestrator + +- Central API surface for all clients (web, TUI, Discord, Telegram) +- Agent dispatch — receive task, select provider/model, spawn Pi session, return result +- Routing engine — cost/capability matrix, user preference overrides, task-type heuristics +- Plugin host — load channel plugins at startup, manage lifecycle +- MCP server — expose Mosaic tools via MCP protocol +- WebSocket hub — real-time updates for chat, agent status, notifications +- Rate limiting and request validation + +### FR-3: Agent Pool (@mosaicstack/agent) + +- Manage concurrent Pi SDK sessions +- Provider configuration: API key management, endpoint URLs, model lists +- Support providers: Anthropic (subscription + API), OpenAI/Codex (subscription + API), Z.ai, Ollama (local), LM Studio (local), llama.cpp (local) +- Tool injection — all agent sessions get Mosaic tools (brain, queue, memory) +- Skill loading — configure skills per agent session based on task type +- Session monitoring — track active sessions, token usage, duration +- Graceful shutdown — drain active sessions on shutdown + +### FR-4: Task Management + +- Brain-backed task CRUD with full filter/sort +- Task statuses: backlog, scheduled, in-progress, blocked, done, cancelled +- Priority levels: critical, high, medium, low +- Domain categorization +- Dependency tracking (blocks/blocked_by) +- Project association +- Assignee tracking +- Kanban board view in web dashboard +- Due date tracking with stale detection + +### FR-5: Project Management + +- Project CRUD with domain, status, priority +- Link to repository, branch, current/next milestone +- Progress tracking +- Blocker tracking +- Owner assignment + +### FR-6: Mission System + +- Mission CRUD (linked to project and PRD) +- Mission tasks with phases, dependencies, ordering +- Mission summary with computed progress +- Mission coordination via @mosaicstack/coord +- Active mission dashboard in web UI + +### FR-7: Memory System + +- **Preferences**: Key-value store for learned user preferences (e.g., "prefers tables over paragraphs", "timezone: America/Chicago") +- **Insights**: Distilled knowledge from agent interactions, stored with embeddings +- **Semantic search**: Query across all memory using natural language +- **Auto-capture**: Agent sessions automatically log decisions and learnings +- **Summarization**: Periodic compression of raw logs into structured insights +- **Decay**: Old, unused insights decay in relevance score over time + +### FR-8: Authentication & Authorization + +- BetterAuth integration with Next.js +- Email/password registration and login +- SSO via OIDC/SAML: Authentik, WorkOS, Keycloak +- RBAC roles: admin (full access), member (own resources + shared), viewer (read-only) +- API key generation for programmatic/MCP access +- Session management (web + API) + +### FR-9: Remote Control — Discord + +- Discord bot that connects to the gateway through a transport-neutral channel adapter contract +- Authorized messages in configured agent-bound channels work without a mention and respond in-channel +- Mentions in parent channels create threads, or reuse a thread already attached to that same native message, for multi-turn conversations +- Messages already in a thread remain there without requiring repeated mentions +- Stable logical-agent/channel conversation identity survives underlying harness/provider changes +- DM support for private conversations +- Chunked message delivery (Discord 2000-char limit) +- Bot configuration via web dashboard +- Permission management (which Discord users/roles can interact) + +### FR-10: Remote Control — Telegram + +- Telegram bot via Telegraf +- Private and group chat support +- Command-based interaction (`/ask`, `/task`, `/status`) +- Inline keyboard for task management +- Message routing through gateway + +### FR-11: LLM Provider Management + +- Provider configuration UI in web dashboard +- Per-provider: API key/endpoint, enabled models, cost per token +- Subscription-based providers: detect available models from subscription +- Local providers: Ollama model list, LM Studio endpoint, llama.cpp binary path +- Provider health monitoring +- Usage tracking per provider/model + +### FR-12: Agent Routing + +- Task-type to model-tier mapping (from AGENTS.md cost matrix) +- User preference overrides (e.g., "always use Claude for code review") +- Fallback chains (if primary provider unavailable, try next) +- Cost tracking and budget enforcement +- Routing transparency — user can see why a particular model was chosen + +### FR-13: MCP Capability + +- Gateway exposes MCP server (streamable HTTP transport) +- Brain tools registered as MCP tools +- Queue tools registered as MCP tools +- Memory search registered as MCP tool +- Agent sessions can call MCP tools from other services +- External MCP server connectivity (agent can use third-party MCP servers) + +### FR-14: Skill Management + +- Skill catalog — list available skills from configured sources +- Skill install — install skill to `~/.config/mosaic/skills/` or project-local +- Skill configuration — per-skill settings +- Skill status — installed, available, update available +- Web UI for browsing and managing skills + +### FR-15: CLI Integration + +- `mosaic gateway start` — start the gateway server +- `mosaic brain` — brain data management +- `mosaic queue` — queue operations +- `mosaic coord` — mission coordination +- `mosaic prdy` — PRD wizard +- `mosaic quality` — quality rail management +- `mosaic tui` — launch Pi TUI connected to gateway + +### FR-16: Log Service + +- Structured log ingest from agent sessions +- Parse logs for: decisions made, tools used, errors encountered, learnings captured +- Tier management: hot (7 days, full detail), warm (30 days, summarized), cold (90 days, key facts only) +- Summarization pipeline: cheap LLM compresses aging logs on schedule +- Query interface for log search + +### FR-17: Gateway State Persistence + +- Orchestration state persisted to Valkey (active sessions, pending dispatches, routing context) +- On restart, gateway reads Valkey state and resumes — reconnects to active agent sessions +- `mosaic gateway restart --fresh` clears Valkey queue and all in-flight state (nuclear option) +- Session recovery: detect orphaned agent sessions, offer reconnect or cleanup + +### FR-18: Multi-Session Agent Architecture + +- Each agent has a distinct session with dedicated context +- Multiple input channels (TUI, web, Discord, Telegram) can connect to same agent session +- Channel multiplexing at gateway level with proper authorization +- Discord channel ID paired to specific agent/session (prevents cross-contamination) +- Agent session runs in named tmux session for barge-in capability +- `mosaic agent attach ` connects to agent's tmux session +- `mosaic agent list` shows active sessions with connected channels + +### FR-19: Cron Scheduler + +- Built-in cron scheduler in gateway for recurring tasks +- Default schedules: log summarization, stale task detection, memory decay, provider health checks +- Custom schedules: user-defined agent dispatches on cron expressions +- Schedule management via web dashboard and CLI +- Cron jobs dispatched through normal gateway routing pipeline +- Persistence: schedules stored in PG, survive gateway restart + +### FR-20: Web Search Tool + +- DuckDuckGo web search via MCP server (primary — privacy-respecting, no API key) +- Registered as standard MCP tool available to all agent sessions +- Configurable: can swap to other search providers (Brave, SearXNG, Tavily) +- Results formatted for agent consumption (title, snippet, URL) + +### FR-21: Skill Import from skills.sh + +- Browse skills from https://skills.sh directory via API +- Import skills into `~/.config/mosaic/skills/` or project-local `.mosaic/skills/` +- Vetting workflow: imported skills marked as "unvetted" until admin approves +- Skill review interface in web dashboard (view skill content before approval) +- Vetted skills auto-available to agent sessions; unvetted require explicit enable +- `mosaic skill import ` CLI command +- Track installed skills, versions, update availability + +--- + +## Non-Functional Requirements + +### Security + +- No hardcoded secrets — all secrets via environment variables or vault +- API key rotation capability +- RBAC enforcement at gateway level +- Input validation (Zod) on all API endpoints +- Rate limiting on public endpoints +- CORS configuration for web app +- Secure WebSocket connections +- SSO token validation +- Database connection encryption (SSL) + +### Performance + +- Chat response streaming latency < 200ms TTFB (gateway overhead, not LLM latency) +- Dashboard page loads < 2s +- Brain query responses < 100ms for filtered reads +- Semantic search < 500ms +- Support 10+ concurrent agent sessions +- WebSocket connection handling for 50+ concurrent users + +### Reliability + +- Graceful degradation when LLM provider is unavailable (fallback chain) +- Queue persistence — tasks survive gateway restart +- Database connection pooling with retry +- Health check endpoints for all services +- Structured error responses with correlation IDs + +### Observability (Wide-Event Logging — Required from Phase 0) + +- **OpenTelemetry instrumentation** across all services from day one + - `@opentelemetry/sdk-node` + `@opentelemetry/auto-instrumentations-node` for auto-instrumentation (HTTP, PG, Fastify/NestJS) + - NestJS interceptors for custom spans on agent dispatch, routing decisions, memory writes, summarization runs + - Every significant operation emits a structured event with rich context (wide events, not just request/response) +- **SigNoz** as OTEL backend (single Docker service: traces, metrics, logs, built-in UI) +- Request tracing with correlation IDs (trace-id propagated across gateway → agent → brain → queue) +- Agent session metrics (duration, tokens, cost, success/failure, model used, routing reason) +- Provider availability monitoring (health check spans) +- Queue depth monitoring (periodic gauge metrics) +- Memory usage metrics (embedding count, search latency, summarization runs) +- Migrate to Grafana stack (Tempo + Loki + Grafana) post-beta if more customization is needed + +### Scalability (Multi-Tier Readiness) + +- Single-node deployment is the MVP target for v0.1.0 +- Code structured with assumption that multi-tiered deployment will follow: dedicated gateway nodes, agent worker nodes, brain/DB nodes +- Service boundaries communicate via HTTP/WS/MCP APIs, not in-process calls where avoidable +- Gateway is stateless (all state in PG/Valkey) to enable horizontal scaling +- Agent pool designed as independently scalable service +- Database migrations support forward-only schema evolution +- Hierarchical deployment with dedicated roles/specialties is the post-beta target + +--- + +## Acceptance Criteria + +### AC-1: Core Chat Flow + +- [ ] User can log in via web UI, send a message, and receive a streamed response +- [ ] Conversation persists across page refreshes +- [ ] User can create, list, search, and delete conversations +- [ ] Conversations can be scoped to projects + +### AC-2: TUI Integration + +- [ ] `mosaic tui` launches Pi interactive mode connected to gateway +- [ ] User can chat with same conversation context as web UI +- [ ] Agent has access to brain, queue, and memory tools + +### AC-3: Discord Remote Control + +- [ ] Discord bot connects through the harness-neutral channel contract +- [ ] Authorized untagged channel messages route through the gateway and respond in-channel +- [ ] Mentioned parent-channel messages create a thread (or reuse their already-attached thread) and respond there +- [ ] Existing-thread follow-ups stay in the thread without repeated mentions +- [ ] Channel/session identity remains stable while the underlying harness/provider changes +- [ ] Responses stream back to Discord (chunked) +- [ ] Unauthorized guilds, channels, users, pairings, and roles create no thread and dispatch no message + +### AC-4: Gateway Orchestration + +- [ ] Gateway dispatches tasks to appropriate provider/model +- [ ] Routing decision logged and inspectable +- [ ] Fallback when primary provider unavailable +- [ ] Multiple concurrent agent sessions managed correctly + +### AC-5: Task & Project Management + +- [ ] CRUD operations for tasks, projects, missions via web dashboard +- [ ] Kanban board view for tasks +- [ ] Mission progress tracking with computed stats +- [ ] Brain MCP tools accessible from agent sessions + +### AC-6: Memory System + +- [ ] Agent sessions auto-capture decisions and learnings +- [ ] Semantic search returns relevant past context +- [ ] Learned preferences are applied in new sessions +- [ ] Log summarization runs on schedule, old logs compressed + +### AC-7: Authentication & RBAC + +- [ ] Email/password login works +- [ ] At least one SSO provider (Authentik) works end-to-end +- [ ] Admin can create users and assign roles +- [ ] RBAC enforced on API endpoints + +### AC-8: Multi-Provider LLM Support + +- [ ] At least 3 providers configured and routing correctly (e.g., Anthropic + Ollama + Z.ai) +- [ ] Agent routing selects appropriate model for task type +- [ ] Provider configuration manageable from web UI + +### AC-9: MCP + +- [ ] Gateway exposes MCP endpoint +- [ ] Brain and queue tools callable via MCP +- [ ] Agent sessions can connect to external MCP servers + +### AC-10: Deployment + +- [ ] PGlite data-layer work uses no PostgreSQL; optional Compose services are selected individually and do not start PostgreSQL; Gateway/Web local start remains held until KBN-101-02 rejects daemon/inherited/project DSNs before connection or DDL +- [ ] PostgreSQL/federated activation is unavailable until KBN-101-00/-03/-05 deliver external bootstrap, TLS/roles, runner `--run`, runner `--verify`, and Gateway/Compose readiness in that order +- [ ] `mosaic` CLI installable and functional on bare metal after the reviewed KBN-101-05 secret-renderer/process-exec or `LoadCredential` interface exists +- [ ] Local-only configuration documentation is distinct from production generation-pinned Vault-rendered consumer material + +### AC-11: @mosaicstack/\* Packages + +- [ ] All 7 migrated packages build, pass tests, and integrate with gateway +- [ ] `mosaic` CLI provides subcommands for each package +- [ ] Types package is the single source of shared interfaces + +--- + +## Constraints and Dependencies + +1. **Pi SDK** — Core dependency; any Pi breaking changes affect the agent layer. Pin to known-good version. +2. **BetterAuth** — Auth framework; must support SSO adapters. Verify Authentik/WorkOS/Keycloak support before committing. +3. **Drizzle ORM** — Database layer; must support PostgreSQL + pgvector extension. +4. **Discord API** — Rate limits, intent requirements, message size limits (2000 chars). +5. **Valkey** — Queue backend; must be available for queue and caching. +6. **Gitea registry** — Package publishing target; `.npmrc` must be configured. +7. **OpenClaw** — Reference architecture for Discord/Telegram plugin pattern (https://github.com/openclaw/openclaw). Inspiration only, not a dependency. + +--- + +## Risks and Open Questions + +### Risks + +| Risk | Likelihood | Impact | Mitigation | +| -------------------------------------------------- | ---------- | ------ | ---------------------------------------------------------------------------------------- | +| Pi SDK API instability (pre-1.0) | Medium | High | Pin version, abstract behind @mosaicstack/agent interface | +| Brain PG migration complexity | Medium | Medium | Preserve Brain REST/MCP API contract; only storage changes | +| Discord plugin complexity (OpenClaw has ~60 files) | Medium | Medium | Start minimal (DM + mention in channel), single-guild only; expand iteratively post-beta | +| LLM provider subscription auth varies by provider | Medium | Medium | Abstract behind provider interface; implement per-provider adapters | +| Drizzle + pgvector extension compatibility | Low | Medium | Validate in Phase 0 with spike | +| Agent log volume overwhelming storage | Medium | High | Tiered storage with aggressive summarization; configurable retention | +| Scope creep from jarvis-old feature surface | High | High | Strict v0.1.0 scope; features not listed above are post-beta | + +### Open Questions + +| # | Question | Priority | Status | +| --- | ------------------------------------------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | Pi SDK version to pin for v0.1.0? | High | ✅ Resolved — Pin `@mariozechner/pi-coding-agent@~0.57.1` (current stable). Abstract behind `@mosaicstack/agent` interface to insulate from breaking changes. Bump deliberately after testing. | +| 2 | Authentik vs WorkOS vs Keycloak — which SSO provider to implement first? | Medium | ✅ Resolved — Authentik first (already in Jason's infrastructure) | +| 3 | Vector DB: pgvector sufficient or need Qdrant from the start? | Medium | ✅ Resolved — pgvector with VectorStore interface abstraction. Qdrant drops in later if needed. | +| 4 | Summarization LLM: which model for log compression? | Medium | ✅ Resolved — Haiku-tier default with structured output guardrails, configurable via routing engine. | +| 5 | LM Studio and llama.cpp — provider adapters exist in Pi or need custom? | Medium | ✅ Resolved — Pi handles both natively. LM Studio and llama.cpp (server mode) expose OpenAI-compatible APIs; configure via Pi's `models.json` with `openai-completions` API type. No custom adapters needed. | +| 6 | Discord bot — single guild or multi-guild from day one? | Medium | ✅ Resolved — Single-guild only for v0.1.0 to prevent data leaks. Bot binds to one guild. Multi-guild with tenant isolation is a post-beta feature requiring explicit data boundary design. | +| 7 | Bare-metal install — systemd units or just docs? | Low | ASSUMPTION: Docs + CLI launch commands; systemd units post-beta | + +--- + +## Testing and Verification Expectations + +1. **Baseline checks**: `pnpm typecheck && pnpm lint && pnpm test` must pass across all packages +2. **Unit tests**: Vitest for all packages; mocked dependencies for isolation +3. **Integration tests**: Gateway + Brain + Queue with test PG + Valkey (Docker services in CI) +4. **E2E tests**: Playwright for web dashboard critical paths (login, chat, task CRUD) +5. **Agent tests**: Pi SDK session tests with mock provider (verify tool registration, routing) +6. **Evidence format**: CI pipeline green + test count report per package + +--- + +## Milestone / Delivery Intent + +All work is **alpha** (< 0.1.0) until Jason approves 0.1.0 beta release. + +### Phase 0: Foundation (v0.0.1) + +- Scaffold monorepo (pnpm + turbo + tsconfig + eslint + vitest) +- `@mosaicstack/types` — migrate and extend from v0 +- `@mosaicstack/db` — Drizzle schema, PG connection, migrations +- `@mosaicstack/auth` — BetterAuth setup with email/password +- OTEL foundation — `@opentelemetry/sdk-node` setup, SigNoz in docker-compose, trace propagation wired +- Docker Compose (PG 17 + Valkey + SigNoz) +- CI pipeline (Woodpecker) +- AGENTS.md, CLAUDE.md, README.md + +### Phase 1: Core API (v0.0.2) + +- `apps/gateway` — NestJS server (Fastify adapter), auth middleware, health endpoints +- `@mosaicstack/brain` — migrate from v0, swap JSON store for PG via @mosaicstack/db +- `@mosaicstack/queue` — migrate from v0 (minimal changes) +- Gateway routes: conversations, tasks, projects, missions +- WebSocket server for chat streaming +- Basic agent dispatch (single provider, no routing) + +### Phase 2: Agent Layer (v0.0.3) + +- `@mosaicstack/agent` — Pi SDK integration, agent pool manager +- Multi-provider support (Anthropic + Ollama minimum) +- Agent routing engine (cost/capability matrix) +- Tool registration (brain, queue, memory tools injected into agent sessions) +- `@mosaicstack/coord` — migrate from v0, integrate with gateway + +### Phase 3: Web Dashboard (v0.0.4) + +- `apps/web` — Next.js app with BetterAuth +- Chat UI (conversation list, message display, streaming input) +- Task management (list + kanban) +- Project and mission views +- Settings (provider config, profile) +- Admin panel (user management, RBAC) + +### Phase 4: Memory & Intelligence (v0.0.5) + +- `@mosaicstack/memory` — preference store, insight store, semantic search +- `@mosaicstack/log` — log ingest, parsing, tiered storage +- Summarization pipeline +- Memory integration into agent sessions +- Skill management interface (web UI + CLI) + +### Phase 5: Remote Control (v0.0.6) + +- `@mosaicstack/discord-plugin` — Discord channel plugin +- `@mosaicstack/telegram-plugin` — Telegram channel plugin +- Plugin host in gateway +- SSO configuration (Authentik) + +### Phase 6: CLI & Tools (v0.0.7) + +- `@mosaicstack/cli` — unified CLI with all subcommands +- `@mosaicstack/prdy` — migrate from v0 +- `@mosaicstack/quality-rails` — migrate from v0 +- `@mosaicstack/mosaic` — install wizard updated for v1 +- Pi TUI integration (`mosaic tui`) + +### Phase 7: Polish & Beta (v0.0.8 → v0.1.0) + +- MCP endpoint hardening +- Additional SSO providers (WorkOS/Keycloak) +- Additional LLM providers (Codex, Z.ai, LM Studio, llama.cpp) +- Bare-metal deployment documentation +- E2E test suite +- Performance optimization +- Documentation: user guide, admin guide, developer guide +- **Jason approval gate → v0.1.0 beta release** + +--- + +## Assumptions + +1. RESOLVED: **pgvector is sufficient** for semantic search at v0.1.0 scale (personal/family/team = thousands to low hundreds-of-thousands of vectors). `@mosaicstack/memory` defines a `VectorStore` interface with pgvector as the default adapter. The interface boundary makes Qdrant a drop-in migration if PG resource contention or scale demands it later. Zero additional infrastructure for v0.1.0. Rationale: Reduces ops burden; pgvector HNSW indexes are fast at this scale; interface abstraction costs almost nothing now. + +2. RESOLVED: **Authentik is the first SSO provider** — confirmed, already running in Jason's infrastructure. WorkOS and Keycloak adapters follow in Phase 7. + +3. RESOLVED: **NestJS with Fastify adapter for the gateway.** The gateway's complexity (plugin host, agent pool, routing engine, WebSocket hub, MCP server, auth, brain/queue/memory/log integration) warrants NestJS's module system, DI, and guards. Fastify performance preserved via adapter. Aligns with USER.md stated stack ("NestJS API + Next.js web"). @mosaicstack/brain's Fastify code migrates into a NestJS module. + +4. RESOLVED: **OpenTelemetry from Phase 0.** Wide-event logging is required from the start. OTEL auto-instrumentation for NestJS/PG/HTTP via `@opentelemetry/sdk-node`. SigNoz as the all-in-one OTEL backend (single Docker service). Every significant operation emits structured events with rich context. Custom spans for agent dispatch, routing decisions, memory writes. Rationale: Retrofitting observability is painful; baking it in from day one means consistent instrumentation across all services. + +5. ASSUMPTION: **Single-node deployment for v0.1.0**, but code structured for multi-tier. No Kubernetes yet. Docker Compose + bare metal. Service boundaries use HTTP/WS/MCP APIs (not in-process) so gateway, agent pool, and brain can split to separate nodes later. Rationale: Ship single-node MVP; the architecture doesn't fight horizontal scaling when needed. + +6. ASSUMPTION: **Log summarization uses Haiku-tier LLM by default, configurable.** Haiku is well-suited for summarization (compression, not generation — source material is in context). Guardrails: structured output via Zod schema (force extraction of decisions/tools/outcomes/errors as discrete fields), chunked per-session processing (no bulk conflation), extraction-focused prompts. Raw logs stay in hot tier (7 days) as safety net. Users can override the summarization model via routing engine config if they want higher fidelity. Rationale: Haiku is 10-20x cheaper than Sonnet; log summarization runs on schedule against large volumes where cost matters. + +7. ASSUMPTION: **Discord plugin starts minimal and single-guild only** — explicitly configured agent-bound channels accept authorized untagged messages in-channel, while mentions create threads or reuse a thread already attached to that same native message; responses are chunked. Single guild binding prevents data leaks between servers. DM support, voice, components, slash commands, and multi-guild operation are post-beta. Rationale: Ship the requested core interaction model while preserving default-deny data isolation. + +8. ASSUMPTION: **Telegram plugin is lower priority than Discord** and may ship as v0.0.7 or later if Discord takes longer than expected. Rationale: Jason indicated Discord as the high-priority remote channel. + +9. ASSUMPTION: **Brain's REST API is preserved** as a gateway sub-router (mounted at `/api/brain/*` or similar). Existing MCP tools continue to work. Only the storage backend changes. Rationale: Minimize migration risk; brain's API contract is proven. + +10. ASSUMPTION: **Conversations and messages get their own PG tables** (not stored in brain's entity model). They follow a chat-specific schema with proper foreign keys to users and projects. Rationale: Chat has different access patterns (streaming, pagination, search) than brain entities. + +11. RESOLVED: **Pi handles all target LLM providers natively.** Anthropic, OpenAI/Codex, Z.ai, Ollama, LM Studio, and llama.cpp are all supported via Pi's built-in providers or `models.json` configuration with `openai-completions` API type. No custom provider adapters needed in @mosaicstack/agent — only configuration management. + +--- diff --git a/docs/requirements/native-kanban-sot.md b/docs/requirements/native-kanban-sot.md index 535b22a5..9ba524fe 100644 --- a/docs/requirements/native-kanban-sot.md +++ b/docs/requirements/native-kanban-sot.md @@ -372,3 +372,87 @@ The P0–P3 canon does not authorize: ## 7. Global release evidence P0–P3 may close only when requirements traceability maps every requirement above to automated and situational evidence, including cross-workspace denials, DB/Valkey fault injection, concurrent leases, stale fencing, generated-file immutability, UI conflict/reconnect behavior, migration reconciliation, independent review, mandatory SecReview, and final Certifier evidence. + +## 8. Amendment A1 — hierarchy parentage and RBAC chain above workspaces + +**Status:** amendment to the ratified canon, added by reviewed PR under +decision D13 (operator ruling, 2026-08-25; decision owner Jason). It adds +parent structure ABOVE workspaces. Sections 1–7, every invariant in §3, and +every REQ above remain binding verbatim, with exactly one express modification: +the narrow portfolio-analytics carve-out stated in §8.2.4. Nothing else below +this line is weakened. + +### 8.1 What is added + +1. A platform hierarchy exists above workspaces: + **company/organization → estate → platform-project → workspace**. Each + workspace belongs to exactly one platform-project, each platform-project to + exactly one estate, each estate to exactly one company. +2. **Record class.** Hierarchy records (company, estate, platform-project, + their parentage edges, and hierarchy-level access grants) are a new, + explicitly named record class: **tenancy/authorization structure records**. + They are not business or orchestration records, so §3 invariant 10 and + REQ-TEN-001 do not apply to them and are not weakened by them — those two + requirements bind business/orchestration rows exactly as before. + Constraints on the new class: + - Hierarchy tables MUST NOT carry task, plan, or any other + business/orchestration payload — parentage, naming, and grant data only. + - A hierarchy record can never be the subject of work: it cannot be + claimed, ordered, gated, or referenced as a dependency by any + business/orchestration row. + - Hierarchy mutations flow through the same sole-writable-SOT, fail-closed, + audited mutation path as everything else (§8.2.3). +3. The hierarchy serves exactly two runtime functions, plus audited + maintenance of its own structure: + - **RBAC evaluation:** access grants are declared per company, estate, or + platform-project and evaluate down the chain to workspace-scoped + authorization. Tenant context continues to be derived from authenticated + authority (REQ-TEN-001); the chain adds where grants can be declared, + not a bypass of workspace authorization. + - **Read-only roll-ups:** task and status visualization bubbles up the + hierarchy as aggregation over workspaces the reader is authorized on. + - **Chain maintenance (not a third runtime function):** re-parenting an + asset — moving a workspace to another platform-project, a + platform-project to another estate, and so on ("assets are transferable + subject to the structure", PRD Part I §4) — is an audited edit of the + hierarchy records themselves under §8.3. It never modifies + business/orchestration rows and never crosses a workspace boundary for + them; the workspace's contents move with the workspace untouched. +4. Naming: this amendment says **platform-project** for the hierarchy level + above workspaces, because §5 REQ-PLAN-001 already defines `projects` as + planning entities INSIDE a workspace. The two are different objects. Final + terminology (rename of one or the other) is an implementation-PR decision + under this amendment's review; the schema MUST NOT merge them. + +### 8.2 What is explicitly unchanged + +1. `workspace_id` remains the hard mechanical isolation unit (§2 D2, + REQ-TEN-001). Hierarchy tables carry parentage; they do not create + cross-workspace relationships between business/orchestration rows, which + remain rejected (§3 invariant 10). +2. Roll-up is **never a write**: no aggregation path may mutate, claim, order, + or gate work in any workspace. Bubble-up views are generated projections in + the sense of §3 invariant 5 — non-authoritative and never import sources. +3. Fail-closed mutation health (§3 invariants 3–4), sole writable PostgreSQL + SOT, fencing, audit, and the Coordinator/Certifier authority rules are + untouched. +4. No §6 non-goal is authorized, with one express, narrow carve-out that this + amendment makes to the "portfolio analytics" non-goal: the read-only + roll-up of §8.1 — per-workspace task counts and statuses aggregated up the + parent chain, over workspaces the reader is authorized on — is in scope. + Everything beyond that boundary (metrics, trends, forecasting, scoring, + dashboards computed across workspaces, any derived analytic that is not a + direct count/status aggregation) remains a non-goal. This is an explicit + narrowing by amendment, not a claim that §6 is unchanged; every other §6 + non-goal is untouched. + +### 8.3 Acceptance (binding on the implementing PRs) + +- Schema tests prove each workspace resolves to exactly one + platform-project/estate/company chain and that chain edits are audited. +- Authorization tests prove a grant at each hierarchy level yields exactly the + workspace permissions the chain implies, and that revocation up the chain + propagates. +- Negative tests prove roll-up endpoints cannot mutate state and that a + reader sees aggregates only over workspaces they are authorized on + (no cross-tenant existence oracles).