# Mosaic Stack Multi-tenant personal assistant platform with PostgreSQL backend, Authentik SSO, and MoltBot integration. ## Overview Mosaic Stack is a modern, PDA-friendly platform designed to help users manage their personal and professional lives with: - **Multi-user workspaces** with team collaboration - **Knowledge management** with wiki-style linking and version history - **Task management** with flexible organization - **Event & calendar** integration - **Project tracking** with Gantt charts and Kanban boards - **MoltBot integration** for natural language interactions - **Authentik OIDC** for secure, enterprise-grade authentication **Version:** 0.0.1 (Pre-MVP) **Repository:** https://git.mosaicstack.dev/mosaic/stack ## Technology Stack | Layer | Technology | | -------------- | ---------------------------------------------- | | **Frontend** | Next.js 16 + React + TailwindCSS + Shadcn/ui | | **Backend** | NestJS + Prisma ORM | | **Database** | PostgreSQL 17 + pgvector | | **Cache** | Valkey (Redis-compatible) | | **Auth** | Authentik (OIDC) via BetterAuth | | **AI** | Ollama (local or remote) | | **Messaging** | MoltBot (stock + plugins) | | **Real-time** | WebSockets (Socket.io) | | **Speech** | Speaches (STT) + Kokoro/Chatterbox/Piper (TTS) | | **Monorepo** | pnpm workspaces + TurboRepo | | **Testing** | Vitest + Playwright | | **Deployment** | Docker + docker-compose | ## Quick Start ### One-Line Install (Recommended) The fastest way to get Mosaic Stack running on macOS or Linux: ```bash curl -fsSL https://get.mosaicstack.dev | bash ``` This installer: - ✅ Detects your platform (macOS, Debian/Ubuntu, Arch, Fedora) - ✅ Installs all required dependencies (Docker, Node.js, etc.) - ✅ Generates secure secrets automatically - ✅ Configures the environment for you - ✅ Starts all services with Docker Compose - ✅ Validates the installation with health checks **Installer Options:** ```bash # Non-interactive Docker deployment curl -fsSL https://get.mosaicstack.dev | bash -s -- --non-interactive --mode docker # Preview installation without making changes curl -fsSL https://get.mosaicstack.dev | bash -s -- --dry-run # With SSO and local Ollama curl -fsSL https://get.mosaicstack.dev | bash -s -- \ --mode docker \ --enable-sso --bundled-authentik \ --ollama-mode local # Skip dependency installation (if already installed) curl -fsSL https://get.mosaicstack.dev | bash -s -- --skip-deps ``` **After Installation:** ```bash # Check system health ./scripts/commands/doctor.sh # View service logs docker compose logs -f # Stop services docker compose down ``` ### Prerequisites If you prefer manual installation, you'll need: - **Docker mode:** Docker 24+ and Docker Compose - **Native mode:** Node.js 24+, pnpm 10+, PostgreSQL 17+ The installer handles these automatically. ### Manual Installation ```bash # Clone the repository git clone https://git.mosaicstack.dev/mosaic/stack mosaic-stack cd mosaic-stack # Install dependencies pnpm install # Copy environment file cp .env.example .env # Configure environment variables (see CONFIGURATION.md) # Edit .env with your database and auth settings # Generate Prisma client pnpm prisma:generate # Run database migrations pnpm prisma:migrate # Seed development data (optional) pnpm prisma:seed # Start development servers pnpm dev ``` ### Docker Deployment **Recommended for quick setup and production deployments.** #### Development (Turnkey - All Services Bundled) ```bash # Clone repository git clone https://git.mosaicstack.dev/mosaic/stack mosaic-stack cd mosaic-stack # Copy and configure environment cp .env.example .env # Set COMPOSE_PROFILES=full in .env # Start all services (PostgreSQL, Valkey, OpenBao, Authentik, Ollama, API, Web) docker compose up -d # View logs docker compose logs -f # Access services # Web: http://localhost:3000 # API: http://localhost:3001 # Auth: http://localhost:9000 ``` #### Production (External Managed Services) ```bash # Clone repository git clone https://git.mosaicstack.dev/mosaic/stack mosaic-stack cd mosaic-stack # Copy environment template and example cp .env.example .env cp docker/docker-compose.example.external.yml docker-compose.override.yml # Edit .env with external service URLs: # - DATABASE_URL=postgresql://... (RDS, Cloud SQL, etc.) # - VALKEY_URL=redis://... (ElastiCache, Memorystore, etc.) # - OPENBAO_ADDR=https://... (HashiCorp Vault, etc.) # - OIDC_ISSUER=https://... (Auth0, Okta, etc.) # - Set COMPOSE_PROFILES= (empty) # Start API and Web only docker compose up -d # View logs docker compose logs -f ``` #### Hybrid (Mix of Bundled and External) ```bash # Use bundled database/cache, external auth/secrets cp docker/docker-compose.example.hybrid.yml docker-compose.override.yml # Edit .env: # - COMPOSE_PROFILES=database,cache,ollama # - OPENBAO_ADDR=https://... (external vault) # - OIDC_ISSUER=https://... (external auth) # Start mixed deployment docker compose up -d ``` **Stop services:** ```bash docker compose down ``` **What's included:** - PostgreSQL 17 with pgvector extension - Valkey (Redis-compatible cache) - Mosaic API (NestJS) - Mosaic Web (Next.js) - Mosaic Orchestrator (Agent lifecycle management) - Mosaic Coordinator (Task assignment & monitoring) - Authentik OIDC (optional, use `--profile authentik`) - Ollama AI (optional, use `--profile ollama`) See [Docker Deployment Guide](docs/1-getting-started/4-docker-deployment/) for complete documentation. ### Docker Swarm Deployment (Production) **Recommended for production deployments with high availability and auto-scaling.** Deploy to a Docker Swarm cluster with integrated Traefik reverse proxy: ```bash # 1. Initialize swarm (if not already done) docker swarm init --advertise-addr # 2. Create Traefik network docker network create --driver=overlay traefik-public # 3. Configure environment for swarm cp .env.swarm.example .env nano .env # Configure domains, passwords, API keys # 4. CRITICAL: Deploy OpenBao standalone FIRST # OpenBao cannot run in swarm mode - deploy as standalone container docker compose -f docker-compose.openbao.yml up -d sleep 30 # Wait for auto-initialization # 5. Deploy swarm stack IMAGE_TAG=latest ./scripts/deploy-swarm.sh mosaic # 6. Check deployment status docker stack services mosaic docker stack ps mosaic # Access services via Traefik # Web: http://mosaic.mosaicstack.dev # API: http://api.mosaicstack.dev # Auth: http://auth.mosaicstack.dev (if using bundled Authentik) ``` **Key features:** - Automatic Traefik integration for routing - Overlay networking for multi-host deployments - Built-in health checks and rolling updates - Horizontal scaling for web and API services - Zero-downtime deployments - Service orchestration across multiple nodes **Important Notes:** - **OpenBao Requirement:** OpenBao MUST be deployed as standalone container (not in swarm). Use `docker-compose.openbao.yml` or external Vault. - Swarm does NOT support docker-compose profiles - To use external services (PostgreSQL, Authentik, etc.), manually comment them out in `docker-compose.swarm.yml` See [Docker Swarm Deployment Guide](docs/SWARM-DEPLOYMENT.md) and [Quick Reference](docs/SWARM-QUICKREF.md) for complete documentation. ### Portainer Deployment **Recommended for GUI-based stack management.** Portainer provides a web UI for managing Docker containers and stacks. Use the Portainer-optimized compose file: **File:** `docker-compose.portainer.yml` **Key differences from standard compose:** - No `env_file` directive (define variables in Portainer UI) - Port exposed on all interfaces (Portainer limitation) - Optimized for Portainer's stack parser **Quick Steps:** 1. Create `mosaic_internal` overlay network in Portainer 2. Deploy `mosaic-openbao` stack with `docker-compose.portainer.yml` 3. Deploy `mosaic` swarm stack with `docker-compose.swarm.yml` 4. Configure environment variables in Portainer UI See [Portainer Deployment Guide](docs/PORTAINER-DEPLOYMENT.md) for detailed instructions. ## Project Structure ``` mosaic-stack/ ├── apps/ │ ├── api/ # NestJS backend API │ │ ├── src/ │ │ │ ├── auth/ # BetterAuth + Authentik OIDC │ │ │ ├── prisma/ # Database service │ │ │ ├── coordinator-integration/ # Coordinator API client │ │ │ └── app.module.ts # Main application module │ │ ├── prisma/ │ │ │ └── schema.prisma # Database schema │ │ └── Dockerfile │ ├── web/ # Next.js 16 frontend │ │ ├── app/ │ │ ├── components/ │ │ │ └── widgets/ # HUD widgets (agent status, etc.) │ │ └── Dockerfile │ ├── orchestrator/ # Agent lifecycle & spawning (NestJS) │ │ ├── src/ │ │ │ ├── spawner/ # Agent spawning service │ │ │ ├── queue/ # Valkey-backed task queue │ │ │ ├── monitor/ # Health monitoring │ │ │ ├── git/ # Git worktree management │ │ │ └── killswitch/ # Emergency agent termination │ │ └── Dockerfile │ └── coordinator/ # Task assignment & monitoring (FastAPI) │ ├── src/ │ │ ├── webhook.py # Gitea webhook receiver │ │ ├── parser.py # Issue metadata parser │ │ └── security.py # HMAC signature verification │ └── Dockerfile ├── packages/ │ ├── shared/ # Shared types & utilities │ │ └── src/types/ │ │ ├── auth.types.ts # Auth types (AuthUser, Session, etc.) │ │ ├── database.types.ts # DB entity types │ │ └── enums.ts # Shared enums │ ├── ui/ # Shared UI components (planned) │ └── config/ # Shared configuration (planned) ├── plugins/ # MoltBot skills (planned) │ ├── mosaic-plugin-brain/ # API query skill │ ├── mosaic-plugin-calendar/ # Calendar skill │ └── mosaic-plugin-tasks/ # Task management skill ├── docker/ │ ├── docker-compose.yml # Production deployment │ └── init-scripts/ # PostgreSQL initialization ├── docs/ │ ├── SETUP.md # Installation guide │ ├── CONFIGURATION.md # Environment configuration │ ├── DESIGN-PRINCIPLES.md # PDA-friendly design patterns │ ├── API.md # API documentation │ ├── TYPE-SHARING.md # Type sharing strategy │ └── scratchpads/ # Development notes ├── .env.example # Environment template ├── turbo.json # TurboRepo configuration └── pnpm-workspace.yaml # Workspace configuration ``` ## Agent Orchestration Layer (v0.0.6) Mosaic Stack includes a sophisticated agent orchestration system for autonomous task execution: - **Orchestrator Service** (NestJS) - Manages agent lifecycle, spawning, and health monitoring - **Coordinator Service** (FastAPI) - Receives Gitea webhooks, assigns tasks to agents - **Task Queue** - Valkey-backed queue for distributed task management - **Git Worktrees** - Isolated workspaces for parallel agent execution - **Killswitch** - Emergency stop mechanism for runaway agents - **Agent Dashboard** - Real-time monitoring UI with status widgets See [Agent Orchestration Design](docs/design/agent-orchestration.md) for architecture details. ## Speech Services Mosaic Stack includes integrated speech-to-text (STT) and text-to-speech (TTS) capabilities through a modular provider architecture. Each component is optional and independently configurable. - **Speech-to-Text** - Transcribe audio files and real-time audio streams using Whisper (via Speaches) - **Text-to-Speech** - Synthesize speech with 54+ voices across 8 languages (via Kokoro, CPU-based) - **Premium Voice Cloning** - Clone voices from audio samples with emotion control (via Chatterbox, GPU) - **Fallback TTS** - Ultra-lightweight CPU fallback for low-resource environments (via Piper/OpenedAI Speech) - **WebSocket Streaming** - Real-time streaming transcription via Socket.IO `/speech` namespace - **Automatic Fallback** - TTS tier system with graceful degradation (premium -> default -> fallback) **Quick Start:** ```bash # Start speech services alongside core stack make speech-up # Or with Docker Compose directly docker compose -f docker-compose.yml -f docker-compose.speech.yml up -d ``` See [Speech Services Documentation](docs/SPEECH.md) for architecture details, API reference, provider configuration, and deployment options. ## Current Implementation Status ### ✅ Completed (v0.0.1-0.0.6) - **M1-Foundation:** Project scaffold, PostgreSQL 17 + pgvector, Prisma ORM - **M2-MultiTenant:** Workspace isolation with RLS, team management - **M3-Features:** Knowledge management, tasks, calendar, authentication - **M4-MoltBot:** Bot integration architecture (in progress) - **M6-AgentOrchestration:** Orchestrator service, coordinator, agent dashboard ✅ **Test Coverage:** 2168+ tests passing ### 🚧 In Progress (v0.0.x) - Agent orchestration E2E testing - Usage budget management - Performance optimization ### 📋 Planned Features (v0.1.0 MVP) - Event/calendar management - Project tracking with Gantt charts - MoltBot integration - WebSocket real-time updates - Data migration from jarvis-brain See the [issue tracker](https://git.mosaicstack.dev/mosaic/stack/issues) for complete roadmap. ## Knowledge Module The **Knowledge Module** is a powerful personal wiki and knowledge management system built into Mosaic Stack. Create interconnected notes, organize with tags, track changes over time, and visualize relationships. ### Features - **📝 Markdown-based entries** — Write using familiar Markdown syntax - **🔗 Wiki-style linking** — Connect entries using `[[wiki-links]]` - **🏷️ Tag organization** — Categorize and filter with flexible tagging - **📜 Full version history** — Every edit is tracked and recoverable - **🔍 Powerful search** — Full-text search across titles and content - **📊 Knowledge graph** — Visualize relationships between entries - **📤 Import/Export** — Bulk import/export for portability - **⚡ Valkey caching** — High-performance caching for fast access ### Quick Examples **Create an entry:** ```bash curl -X POST http://localhost:3001/api/knowledge/entries \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "x-workspace-id: WORKSPACE_ID" \ -d '{ "title": "React Hooks Guide", "content": "# React Hooks\n\nSee [[Component Patterns]] for more.", "tags": ["react", "frontend"], "status": "PUBLISHED" }' ``` **Search entries:** ```bash curl -X GET 'http://localhost:3001/api/knowledge/search?q=react+hooks' \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "x-workspace-id: WORKSPACE_ID" ``` **Export knowledge base:** ```bash curl -X GET 'http://localhost:3001/api/knowledge/export?format=markdown' \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "x-workspace-id: WORKSPACE_ID" \ -o knowledge-export.zip ``` ### Documentation - **[User Guide](KNOWLEDGE_USER_GUIDE.md)** — Getting started, features, and workflows - **[API Documentation](KNOWLEDGE_API.md)** — Complete REST API reference with examples - **[Developer Guide](KNOWLEDGE_DEV.md)** — Architecture, implementation, and contributing ### Key Concepts **Wiki-links** Connect entries using double-bracket syntax: ```markdown See [[Entry Title]] or [[entry-slug]] for details. Use [[Page|custom text]] for custom display text. ``` **Version History** Every edit creates a new version. View history, compare changes, and restore previous versions: ```bash # List versions GET /api/knowledge/entries/:slug/versions # Get specific version GET /api/knowledge/entries/:slug/versions/:version # Restore version POST /api/knowledge/entries/:slug/restore/:version ``` **Backlinks** Automatically discover entries that link to a given entry: ```bash GET /api/knowledge/entries/:slug/backlinks ``` **Tags** Organize entries with tags: ```bash # Create tag POST /api/knowledge/tags { "name": "React", "color": "#61dafb" } # Find entries with tags GET /api/knowledge/search/by-tags?tags=react,frontend ``` ### Performance With Valkey caching enabled: - **Entry retrieval:** ~2-5ms (vs ~50ms uncached) - **Search queries:** ~2-5ms (vs ~200ms uncached) - **Graph traversals:** ~2-5ms (vs ~400ms uncached) - **Cache hit rates:** 70-90% for active workspaces Configure caching via environment variables: ```bash VALKEY_URL=redis://localhost:6379 KNOWLEDGE_CACHE_ENABLED=true KNOWLEDGE_CACHE_TTL=300 # 5 minutes ``` ## Development Workflow ### Branch Strategy - `main` — Trunk branch (all development merges here) - `feature/*` — Feature branches from main - `fix/*` — Bug fix branches from main ### Running Locally ```bash # Start all services (requires Docker for PostgreSQL) pnpm dev # All apps # Or run individually pnpm dev:api # API only (http://localhost:3001) pnpm dev:web # Web only (http://localhost:3000) # Database tools pnpm prisma:studio # Open Prisma Studio pnpm prisma:migrate # Run migrations pnpm prisma:generate # Regenerate Prisma client ``` ### Testing ```bash pnpm test # All tests pnpm test:api # API tests only pnpm test:web # Web tests only pnpm test:e2e # E2E tests with Playwright pnpm test:coverage # Generate coverage report ``` ### Building ```bash pnpm build # Build all apps pnpm build:api # Build API only pnpm build:web # Build web only ``` ## Design Philosophy Mosaic Stack follows strict **PDA-friendly design principles**: ### Language Guidelines We **never** use demanding or stressful language: | ❌ NEVER | ✅ ALWAYS | | ----------- | -------------------- | | OVERDUE | Target passed | | URGENT | Approaching target | | MUST DO | Scheduled for | | CRITICAL | High priority | | YOU NEED TO | Consider / Option to | | REQUIRED | Recommended | ### Visual Principles - **10-second scannability** — Key info visible immediately - **Visual chunking** — Clear sections with headers - **Single-line items** — Compact, scannable lists - **Calm colors** — No aggressive reds for status indicators - **Progressive disclosure** — Details on click, not upfront See [Design Principles](docs/3-architecture/3-design-principles/1-pda-friendly.md) for complete guidelines. ## API Conventions ### Endpoints ``` GET /api/{resource} # List (with pagination, filters) GET /api/{resource}/:id # Get single item POST /api/{resource} # Create new item PATCH /api/{resource}/:id # Update item DELETE /api/{resource}/:id # Delete item ``` ### Authentication All authenticated endpoints require a Bearer token: ```http Authorization: Bearer {session_token} ``` See [API Reference](docs/4-api/) for complete API documentation. ## Configuration Key environment variables: ```bash # Database DATABASE_URL=postgresql://mosaic:password@localhost:5432/mosaic # Authentik OIDC OIDC_ISSUER=https://auth.example.com/application/o/mosaic-stack/ OIDC_CLIENT_ID=your-client-id OIDC_CLIENT_SECRET=your-client-secret # JWT Session JWT_SECRET=change-this-to-a-random-secret-in-production JWT_EXPIRATION=24h # Application NEXT_PUBLIC_APP_URL=http://localhost:3000 ``` See [Configuration](docs/1-getting-started/3-configuration/1-environment.md) for all configuration options. ## Caching Mosaic Stack uses **Valkey** (Redis-compatible) for high-performance caching, significantly improving response times for frequently accessed data. ### Knowledge Module Caching The Knowledge module implements intelligent caching for: - **Entry Details** - Individual knowledge entries (GET `/api/knowledge/entries/:slug`) - **Search Results** - Full-text search queries with filters - **Graph Queries** - Knowledge graph traversals with depth limits ### Cache Configuration Configure caching via environment variables: ```bash # Valkey connection VALKEY_URL=redis://localhost:6379 # Knowledge cache settings KNOWLEDGE_CACHE_ENABLED=true # Set to false to disable caching (dev mode) KNOWLEDGE_CACHE_TTL=300 # Time-to-live in seconds (default: 5 minutes) ``` ### Cache Invalidation Strategy Caches are automatically invalidated on data changes: - **Entry Updates** - Invalidates entry cache, search caches, and related graph caches - **Entry Creation** - Invalidates search caches and graph caches - **Entry Deletion** - Invalidates entry cache, search caches, and graph caches - **Link Changes** - Invalidates graph caches for affected entries ### Cache Statistics & Management Monitor and manage caches via REST endpoints: ```bash # Get cache statistics (hits, misses, hit rate) GET /api/knowledge/cache/stats # Clear all caches for a workspace (admin only) POST /api/knowledge/cache/clear # Reset cache statistics (admin only) POST /api/knowledge/cache/stats/reset ``` **Example response:** ```json { "enabled": true, "stats": { "hits": 1250, "misses": 180, "sets": 195, "deletes": 15, "hitRate": 0.874 } } ``` ### Performance Benefits - **Entry retrieval:** ~10-50ms → ~2-5ms (80-90% improvement) - **Search queries:** ~100-300ms → ~2-5ms (95-98% improvement) - **Graph traversals:** ~200-500ms → ~2-5ms (95-99% improvement) Cache hit rates typically stabilize at 70-90% for active workspaces. ## Type Sharing Types used by both frontend and backend live in `@mosaic/shared`: ```typescript import type { AuthUser, Task, Event } from "@mosaic/shared"; // Frontend function UserProfile({ user }: { user: AuthUser }) { return
{user.name}
; } // Backend @Get("profile") @UseGuards(AuthGuard) getProfile(@CurrentUser() user: AuthUser) { return user; } ``` See [Type Sharing Strategy](docs/2-development/3-type-sharing/1-strategy.md) for the complete type sharing strategy. ## Contributing 1. Check the [issue tracker](https://git.mosaicstack.dev/mosaic/stack/issues) for open issues 2. Create a feature branch: `git checkout -b feature/my-feature develop` 3. Make your changes with tests (minimum 85% coverage required) 4. Run tests: `pnpm test` 5. Build: `pnpm build` 6. Commit with conventional format: `feat(#issue): Description` 7. Push and create a pull request to `main` ### Commit Format ``` (#issue): Brief description Detailed explanation if needed. Fixes #123 ``` **Types:** `feat`, `fix`, `docs`, `test`, `refactor`, `chore` ## Testing Requirements - **Minimum 85% coverage** for new code - **TDD approach** — Write tests before implementation - **All tests pass** before PR merge - Use Vitest for unit/integration tests - Use Playwright for E2E tests ## Documentation Complete documentation is organized in a Bookstack-compatible structure in the `docs/` directory. ### 📚 Getting Started - **[Quick Start](docs/1-getting-started/1-quick-start/1-overview.md)** — Get running in 5 minutes - **[Installation](docs/1-getting-started/2-installation/)** — Prerequisites, local setup, Docker deployment - **[Configuration](docs/1-getting-started/3-configuration/)** — Environment variables and Authentik OIDC ### 💻 Development - **[Workflow](docs/2-development/1-workflow/)** — Branching strategy, testing requirements, commit guidelines - **[Database](docs/2-development/2-database/)** — Schema design, migrations, Prisma usage - **[Type Sharing](docs/2-development/3-type-sharing/1-strategy.md)** — Shared types across monorepo ### 🏗️ Architecture - **[Overview](docs/3-architecture/1-overview/)** — System design and components - **[Authentication](docs/3-architecture/2-authentication/)** — BetterAuth and OIDC integration - **[Design Principles](docs/3-architecture/3-design-principles/1-pda-friendly.md)** — PDA-friendly patterns (non-negotiable) - **[Telemetry](docs/telemetry.md)** — AI task completion tracking, predictions, and SDK reference ### 🔌 API Reference - **[Conventions](docs/4-api/1-conventions/1-endpoints.md)** — REST patterns, pagination, filtering - **[Authentication](docs/4-api/2-authentication/1-endpoints.md)** — Auth endpoints and flows **Browse all documentation:** [docs/](docs/) ## Related Projects - **jarvis-brain** — Original JSON-based personal assistant (migration source) - **MoltBot** — Stock messaging gateway for multi-platform integration ## Security - **Session-based authentication** with secure JWT tokens - **Row-level security** ready for multi-tenant isolation - **OIDC integration** with Authentik for enterprise SSO - **Secure error handling** — No sensitive data in logs or responses - **Type-safe validation** — TypeScript catches issues at compile time ## License [Add license information] ## Support - **Issues:** https://git.mosaicstack.dev/mosaic/stack/issues - **Documentation:** https://git.mosaicstack.dev/mosaic/stack/wiki --- **Mosaic Stack v0.0.1** — Building the future of personal assistants.