chore: scaffold mosaic monorepo (Phase 0)

- pnpm workspaces + Turborepo - tsconfig.base.json strict ESM - @mosaic/types stub package - Woodpecker CI pipeline with Valkey service - docs/PRD.md, docs/TASKS.md, CONTEXT.md
2026-03-06 13:15:47 -06:00
parent e9d4d8a9e2
commit 5103406c93
18 changed files with 1415 additions and 0 deletions
--- a/docs/PRD.md
+++ b/docs/PRD.md
@@ -0,0 +1,373 @@
+# PRD: Mosaic Monorepo & @mosaic/* Package Ecosystem
+
+**Status:** Draft  
+**Created:** 2026-03-06  
+**Updated:** 2026-03-06 (post-bootstrap audit)  
+**Owner:** Jason Woltje  
+**Repo (target):** `git.mosaicstack.dev/mosaic/mosaic`  
+
+---
+
+## 1. Problem Statement
+
+Mosaic tooling is currently spread across five polyrepos. Each repo duplicates TypeScript config, ESLint rules, Vitest setup, and core type definitions. Cross-repo changes require synchronized PRs with a window of inconsistency. Agent workers lose context when they must span multiple repos. There is no standardized installation path — getting the full Mosaic toolkit requires knowing which repos exist and cloning them individually.
+
+**Symptoms:**
+- `tsconfig.json`, `eslint.config.js`, `vitest.config.ts` duplicated across every repo
+- `Task`, `Mission`, `Agent`, and related types redefined in each package
+- A feature touching `coord` + `queue` requires 2 PRs, 2 CI runs, 2 merges
+- No `npm install` story — setup is tribal knowledge
+- Agent workers clone one repo with no visibility into sibling packages
+
+---
+
+## 2. Goals
+
+1. **Reduce duplication** — One `tsconfig`, one ESLint config, one set of shared types, inherited by all packages
+2. **Standardize naming** — All packages published under `@mosaic/` scope to the Gitea npm registry
+3. **Single install entry point** — `npm i -g @mosaic/mosaic` runs guided onboarding, installs selected packages
+4. **CI publishes packages** — Push to `main` triggers Turborepo affected-only build + publish to registry
+5. **Agent-friendly context** — One monorepo means one clone, one `TASKS.md`, one root `CONTEXT.md` for workers
+6. **Eventual full TypeScript** — `coord` and `prdy` start as shell distribution, graduate to TypeScript packages over time
+
+## 3. Non-Goals
+
+- **Not replacing OpenBrain** — Queue is for coordination, OpenBrain is for memory. No overlap.
+- **Not open-sourcing** — These are personal/internal tools. Public registry not in scope.
+- **Not containerizing packages** — Packages are CLI tools and libraries, not services.
+- **Not forced independent versioning** — All packages use fixed/synchronized versioning (simpler for solo developer).
+- **`agent-skills` not included** — Skills are files consumed by agents, not npm packages. Stays in its own repo.
+- **Not rewriting coord/prdy immediately** — Shell scripts are distributed first; TypeScript rewrite is a later phase.
+
+---
+
+## 4. Bootstrap Audit (Current State)
+
+`mosaic-bootstrap` contains four distinct categories of content. Understanding this is critical before migrating:
+
+### 4a. Installation Wizard (TypeScript) — becomes `@mosaic/mosaic`
+28 TypeScript source files. Handles fresh Mosaic install: creates `SOUL.md`, `USER.md`, `TOOLS.md`, configures MCP for Claude/Codex/opencode, selects and installs skills. Currently published as `mosaic-wizard@0.2.0`. This is the **`npm i -g @mosaic/mosaic`** entry point. Already substantially complete — needs rename, cleanup, monorepo integration.
+
+### 4b. Shell Tool Suite — distributed as files, not npm packages
+~150 shell scripts, ~2,500+ lines of bash across:
+- `tools/git/` — PR, issue, milestone management (Gitea API)
+- `tools/orchestrator/` — ~1,800 lines: `mission-init`, `session-run`, `session-resume`, `session-status`
+- `tools/prdy/` — ~470 lines: `prdy-init`, `prdy-update`, `prdy-validate`, `prdy-status`
+- `tools/orchestrator-matrix/` — Python Matrix transport for multi-agent coordination
+- `tools/qa/` — hook handlers, memory write prevention, QA monitors
+- `tools/woodpecker/`, `portainer/`, `cloudflare/`, `authentik/`, `coolify/`, `codex/`, `glpi/` — infra integrations
+
+**These are NOT npm packages today.** They are bash/Python scripts distributed by the wizard to `~/.config/mosaic/`. The wizard is the install vehicle.
+
+### 4c. Agent Guides (Markdown) — distributed as files
+17 markdown guides: `ORCHESTRATOR.md`, `E2E-DELIVERY.md`, `CODE-REVIEW.md`, etc. Loaded by agents at runtime from `~/.config/mosaic/guides/`. Not code, not packages. Distributed by the wizard.
+
+### 4d. Runtime Configs — distributed as files
+Per-agent config overlays for Claude, Codex, opencode. Distributed to `~/.config/mosaic/runtime/` on install.
+
+---
+
+## 5. Package Structure
+
+### Migration Philosophy: Staged
+
+The monorepo is built in waves. Existing TypeScript code migrates first. Shell-based tools (`coord`, `prdy`) are distributed as-is initially, then rewritten as TypeScript packages in a later wave when the monorepo foundation is proven.
+
+```
+Wave 1 — Foundation
+  @mosaic/types           Shared type definitions (new)
+  @mosaic/queue           Migrate from mosaic/queue (TypeScript, already mature)
+Wave 2 — Entry Point
+  @mosaic/mosaic          Migrate wizard from mosaic-bootstrap (TypeScript, already built)
+  @mosaic/openclaw-context  Migrate from mosaic/openclaw-openbrain-context (TypeScript)
+
+Wave 3 — TypeScript Rewrites (future)
+  @mosaic/coord           Rewrite of tools/orchestrator/ bash suite
+  @mosaic/prdy            Rewrite of tools/prdy/ bash suite
+  @mosaic/quality-rails   Rewrite of quality-rails template distributor (shell+templates → TS scaffolder)
+  @mosaic/cli             Unified CLI binary composing all @mosaic/* packages
+```
+
+### Repository Layout
+
+```
+mosaic/mosaic  (monorepo)
+├── packages/
+│   ├── types/              @mosaic/types            Shared interfaces — zero deps
+│   ├── queue/              @mosaic/queue            Task queue + MCP server
+│   ├── quality-rails/      @mosaic/quality-rails    Quality gate enforcement
+│   ├── mosaic/             @mosaic/mosaic           Install wizard + meta package
+│   ├── coord/              @mosaic/coord            [Wave 3] Mission coordination
+│   ├── prdy/               @mosaic/prdy             [Wave 3] PRD generation
+│   └── cli/                @mosaic/cli              [Wave 3] Unified CLI binary
+├── plugins/
+│   └── openclaw-context/   @mosaic/openclaw-context  OpenClaw → OpenBrain plugin
+└── docs/
+    ├── PRD.md              (this file, symlinked from jarvis-brain)
+    └── TASKS.md            (orchestrator-owned)
+```
+
+### Package Responsibilities
+
+#### `@mosaic/types` _(Wave 1)_
+- Shared interfaces: `Task`, `TaskStatus`, `TaskPriority`, `Mission`, `Agent`, `QueueConfig`
+- No runtime code — types only, zero dependencies
+- **Every other `@mosaic/*` package imports from here**; never defines its own copies
+- Published first; version bump propagates to all consumers
+
+#### `@mosaic/queue` _(Wave 1)_
+- Task queue backed by Valkey/Redis with atomic WATCH/MULTI/EXEC ownership
+- MCP server with 8 tools: `queue_list/get/claim/heartbeat/release/complete/fail/status`
+- Migrated from `mosaic/queue` — code mature (Phase 1 + atomicity fixes complete)
+- Updates imports to use `@mosaic/types`
+
+#### `@mosaic/quality-rails` _(Wave 3 — TypeScript rewrite)_
+- **Audit result:** `mosaic/quality-rails` is shell scripts + template files, no TypeScript. No `package.json`.
+- Current form: `scripts/install.sh`, `verify.sh` + config templates for monorepo/nextjs/node/python project types
+- Wave 3 rewrite: TypeScript scaffolder that generates/installs config files programmatically rather than distributing static copies
+- Until Wave 3: templates bundled as wizard distributed assets alongside other shell tools
+- Migrated from `mosaic/quality-rails` (archive after wizard bundles it)
+
+#### `@mosaic/mosaic` _(Wave 2)_
+- **Entry point:** `npm i -g @mosaic/mosaic`
+- Onboarding wizard (migrated from `mosaic-wizard` in bootstrap)
+- Stages: detect installs → mode select → soul/user/tools setup → skill selection → runtime config → finalize
+- Copies shell tool suite (`tools/`, `guides/`, `runtime/`) to `~/.config/mosaic/`
+- Prompts which additional `@mosaic/*` packages to install
+- Config validation: checks OpenBrain connectivity, Gitea token, etc.
+- When Wave 3 completes: wizard shifts from "copy shell scripts" to "just install TypeScript packages"
+
+#### `@mosaic/openclaw-context` _(Wave 2, plugins/)_
+- OpenClaw → OpenBrain context engine plugin
+- Lives in monorepo for shared tooling, published standalone
+- Installable independently: `openclaw plugin install @mosaic/openclaw-context`
+- Migrated from `mosaic/openclaw-openbrain-context`
+
+#### `@mosaic/coord` _(Wave 3 — TypeScript rewrite)_
+- TypeScript rewrite of `tools/orchestrator/` bash suite
+- Mission lifecycle: `init`, `run`, `resume`, `status`, `drain`
+- Reads/writes `docs/TASKS.md` in target project
+- CLI subcommand: `mosaic coord <init|run|resume|status>`
+- Will replace shell scripts in the distributed toolkit
+- **Prerequisite:** Wave 1+2 complete and stable
+
+#### `@mosaic/prdy` _(Wave 3 — TypeScript rewrite)_
+- TypeScript rewrite of `tools/prdy/` bash suite
+- AI-assisted PRD generation, update, validation
+- CLI subcommand: `mosaic prdy <init|update|validate|status>`
+- Depends on LLM client (OpenAI / Anthropic / Z.ai — configured via env)
+- **Prerequisite:** Wave 1+2 complete and stable
+
+#### `@mosaic/cli` _(Wave 3)_
+- Unified `mosaic` binary — thin shell over all `@mosaic/*` packages
+- Plugin discovery: checks installed `@mosaic/*` packages, registers CLI contributions
+- Subcommand routing: `mosaic coord`, `mosaic prdy`, `mosaic queue`, `mosaic prdy`, etc.
+- Does NOT bundle everything — loads plugins dynamically
+
+---
+
+## 6. Tooling
+
+### Package Manager
+**pnpm workspaces** — single lockfile, workspace-linked packages during development, content-addressed store.
+
+### Build Orchestration
+**Turborepo** — affected-only builds, remote cache (optional), pipeline definitions per task type.
+
+```json
+// turbo.json (root)
+{
+  "pipeline": {
+    "build":     { "dependsOn": ["^build"], "outputs": ["dist/**"] },
+    "test":      { "dependsOn": ["^build"] },
+    "lint":      {},
+    "typecheck": { "dependsOn": ["^build"] },
+    "publish":   { "dependsOn": ["build", "test", "lint", "typecheck"] }
+  }
+}
+```
+
+### Shared Root Config (inherited by all packages)
+- `tsconfig.base.json` — strict mode, ESM, `NodeNext` module resolution
+- `eslint.config.js` — shared ruleset (from `tools/quality/templates/` in bootstrap — already exists!)
+- `vitest.workspace.ts` — workspace-level test runner
+
+Each package `tsconfig.json` extends `../../tsconfig.base.json`.
+
+### Versioning
+**Fixed (synchronized):** All packages share a version number. `@mosaic/queue@0.2.0` implies `@mosaic/types@0.2.0`. Revisit if packages diverge significantly in maturity.
+
+Tool: **Changesets** (`@changesets/cli`) — human-authored change descriptions → automated version bumps + changelogs on release.
+
+---
+
+## 7. Registry & CI
+
+### Gitea npm Registry
+Packages published to `https://git.mosaicstack.dev/api/packages/mosaic/npm`.
+
+`.npmrc` (root):
+```
+@mosaic:registry=https://git.mosaicstack.dev/api/packages/mosaic/npm
+//git.mosaicstack.dev/api/packages/mosaic/npm/:_authToken=${GITEA_TOKEN}
+```
+
+### CI Pipeline (Woodpecker)
+Trigger: push to `main`
+
+Steps:
+1. `pnpm install --frozen-lockfile`
+2. `pnpm turbo lint typecheck`
+3. `pnpm turbo test` (with Valkey service container for `@mosaic/queue` tests)
+4. `pnpm turbo build`
+5. Detect changed packages (Turborepo affected output)
+6. `pnpm changeset publish` → publish affected packages to Gitea registry
+
+PR pipeline (no publish): steps 1–4 only + `pnpm changeset status`.
+
+**Valkey service for queue tests:**
+```yaml
+# .woodpecker.yml
+services:
+  - name: valkey
+    image: valkey/valkey:8-alpine
+    ports: ["6379:6379"]
+```
+
+---
+
+## 8. Migration Plan
+
+### Prerequisites (check before starting)
+- [ ] `mosaic/queue` atomicity fix (issue #4) merged
+- [ ] `mosaic/openclaw-openbrain-context` compact/ingest fix (issue #4) merged
+- [ ] No active agent branches on repos being migrated
+
+---
+
+### Wave 1 — Foundation
+
+#### Phase 0: Scaffold Monorepo
+- [ ] Create `mosaic/mosaic` repo on Gitea
+- [ ] Initialize pnpm workspace + Turborepo
+- [ ] Add root `tsconfig.base.json`, `eslint.config.js`, `vitest.workspace.ts`
+- [ ] Add `.npmrc` pointing to Gitea registry
+- [ ] Smoke-test publish (empty `@mosaic/types@0.0.1`) to validate CI+registry auth
+- [ ] Add Woodpecker pipeline with Valkey service
+- [ ] Add root `CONTEXT.md`, `docs/TASKS.md`, `docs/PRD.md`
+
+#### Phase 1: `@mosaic/types`
+- [ ] Extract all shared types from `queue`, `coord`, `prdy`, `quality-rails` (audit each for duplicates)
+- [ ] Publish `@mosaic/types@0.1.0`
+- [ ] No other package migrates until types are published
+
+#### Phase 2: `@mosaic/queue`
+- [ ] Move `mosaic/queue/packages/queue/src` → `mosaic/mosaic/packages/queue/src`
+- [ ] Update imports to use `@mosaic/types`
+- [ ] All tests green including integration (Valkey CI service)
+- [ ] Publish `@mosaic/queue@0.1.0`
+- [ ] Archive `mosaic/queue` repo (read-only, redirect notice in README)
+
+---
+
+### Wave 2 — Entry Point
+
+#### Phase 4: `@mosaic/mosaic` (wizard migration)
+- [ ] Move `mosaic-bootstrap/src/` → `packages/mosaic/src/`
+- [ ] Update package name: `mosaic-wizard` → `@mosaic/mosaic`
+- [ ] Update wizard to reference monorepo packages where applicable
+- [ ] `postinstall` script: run wizard on first install
+- [ ] Shell tool suite (`tools/`, `guides/`, `runtime/`) bundled as static assets
+- [ ] Publish `@mosaic/mosaic@0.1.0`
+- [ ] Validate: `npm i -g @mosaic/mosaic` on clean machine completes setup
+- [ ] Archive `mosaic/bootstrap` (read-only, redirect notice)
+
+#### Phase 5: `@mosaic/openclaw-context`
+- [ ] Move `mosaic/openclaw-openbrain-context/src` → `plugins/openclaw-context/src`
+- [ ] Update shared tooling to use root configs
+- [ ] Publish `@mosaic/openclaw-context@0.1.0`
+- [ ] Validate standalone install works: `openclaw plugin install @mosaic/openclaw-context`
+- [ ] Archive `mosaic/openclaw-openbrain-context`
+
+---
+
+### Wave 3 — TypeScript Rewrites (future, no timeline set)
+
+#### Phase 6: `@mosaic/coord`
+- [ ] Design TypeScript API mirroring current bash behavior
+- [ ] Implement: `init`, `run`, `resume`, `status`, `drain`
+- [ ] CLI subcommand registration for `@mosaic/cli`
+- [ ] Migrate shell scripts to integration tests (parity verification)
+- [ ] Wizard updated: stop distributing orchestrator shell scripts once this ships
+- [ ] Publish `@mosaic/coord@0.1.0`
+
+#### Phase 7: `@mosaic/prdy`
+- [ ] Design TypeScript API mirroring current bash behavior
+- [ ] LLM client abstraction (supports Anthropic / OpenAI / Z.ai via config)
+- [ ] Implement: `init`, `update`, `validate`, `status`
+- [ ] CLI subcommand registration for `@mosaic/cli`
+- [ ] Wizard updated: stop distributing prdy shell scripts once this ships
+- [ ] Publish `@mosaic/prdy@0.1.0`
+
+#### Phase 8: `@mosaic/quality-rails`
+- [ ] TypeScript scaffolder API — generates ESLint, tsconfig, Woodpecker, husky, lint-staged for a project
+- [ ] Supports project types: monorepo, typescript-node, typescript-nextjs, python
+- [ ] CLI: `mosaic quality install [--type monorepo|node|nextjs|python]`, `mosaic quality verify`
+- [ ] Replaces static template distribution from wizard
+- [ ] Archive `mosaic/quality-rails`
+- [ ] Publish `@mosaic/quality-rails@0.1.0`
+
+#### Phase 9: `@mosaic/cli`
+- [ ] Unified binary with plugin discovery
+- [ ] Subcommands: `mosaic coord`, `mosaic prdy`, `mosaic queue`, `mosaic prdy`
+- [ ] Wizard updated to install `@mosaic/cli` as primary binary instead of path-based scripts
+- [ ] Publish `@mosaic/cli@0.1.0`
+
+---
+
+## 9. Open Questions
+
+| # | Question | Priority | Status |
+|---|---|---|---|
+| 1 | Bootstrap audit: what in `tools/` is safe to archive vs still needed? Shell scripts will coexist during Wave 1+2 | High | ✅ Answered — shell tools distributed by wizard until Wave 3 |
+| 2 | `quality-rails` audit complete: shell scripts + templates, no TypeScript, no package.json | High | ✅ Wave 3 rewrite |
+| 3 | `@mosaic/prdy` LLM client: peer dep or bundled? | Medium | Open |
+| 4 | Valkey CI service in Woodpecker: verify swarm runner can spin Docker services | High | Open |
+| 5 | Changesets vs manual semver? (Recommendation: Changesets) | Low | Open |
+| 6 | Matrix orchestrator (`tools/orchestrator-matrix/`, Python) — migrate to TypeScript in Wave 3 or keep Python? | Medium | Open |
+| 7 | `mosaic/agent-skills` — stays polyrepo, but should the wizard install it? | Low | Open |
+
+---
+
+## 10. Success Criteria
+
+### Wave 1+2 (MVP)
+- [ ] `npm i -g @mosaic/mosaic` on a clean machine completes setup end-to-end
+- [ ] `mosaic coord init`, `mosaic prdy init`, `mosaic queue list` all work (bash, distributed by wizard)
+- [ ] Zero duplicated TypeScript config files across Wave 1+2 packages
+- [ ] One PR for a shared type change, no follow-up PRs in sibling repos
+- [ ] CI publishes affected packages on every merge to `main`
+- [ ] All existing functionality preserved — no regressions
+
+### Wave 3 (Full TypeScript)
+- [ ] `mosaic coord` and `mosaic prdy` run as native TypeScript packages, no bash distribution needed
+- [ ] Wizard no longer ships shell scripts for coord/prdy
+- [ ] `@mosaic/cli` is the single `mosaic` binary entry point
+
+---
+
+## 11. Risks
+
+| Risk | Likelihood | Impact | Mitigation |
+|---|---|---|---|
+| `mosaic/quality-rails` is also shell (not TypeScript) | Medium | Medium | Audit before Phase 3; if shell, it becomes Wave 3 |
+| Wizard migration breaks existing installs on Jason's machines | Medium | High | Keep bootstrap installable alongside; wizard detects existing config and upgrades safely |
+| Turborepo cache invalidation surprises | Low | Low | Start without remote cache; add later |
+| Gitea npm registry auth in CI | Medium | High | Smoke-test publish in Phase 0 before any real migration |
+| Breaking installed tools mid-migration | Low | High | Keep old repos live until new packages verified working |
+| Matrix orchestrator (Python) creates a language gap in Wave 3 | Low | Medium | Decision deferred; can keep as Python microservice or wrap in TypeScript subprocess |
+
+---
+
+_Start: Phase 0 scaffold after current fix workers complete (queue + openclaw-context issue #4)._  
+_Wave 3 timeline: TBD — start after Wave 1+2 are stable in production._
--- a/docs/TASKS.md
+++ b/docs/TASKS.md
@@ -0,0 +1,19 @@
+# Mosaic Monorepo Tasks
+
+## Wave 1 - Foundation
+
+- [x] Phase 0: Scaffold monorepo (`mosaic/mosaic`)
+- [ ] Phase 1: Create `@mosaic/types` shared interfaces and publish `0.1.0`
+- [ ] Phase 2: Migrate `@mosaic/queue` into monorepo, adopt `@mosaic/types`, and publish `0.1.0`
+
+## Wave 2 - Entry Point
+
+- [ ] Phase 4: Migrate wizard into `@mosaic/mosaic`
+- [ ] Phase 5: Migrate plugin into `@mosaic/openclaw-context`
+
+## Wave 3 - TypeScript Rewrites
+
+- [ ] Phase 6: Rewrite `@mosaic/coord`
+- [ ] Phase 7: Rewrite `@mosaic/prdy`
+- [ ] Phase 8: Rewrite `@mosaic/quality-rails`
+- [ ] Phase 9: Introduce unified `@mosaic/cli`