diff --git a/docs/briefs/monorepo-consolidation.md b/docs/briefs/monorepo-consolidation.md new file mode 100644 index 00000000..157a0824 --- /dev/null +++ b/docs/briefs/monorepo-consolidation.md @@ -0,0 +1,231 @@ +# Brief: Monorepo Consolidation — mosaic/stack → mosaic/mosaic-stack + +## Source + +Architecture consolidation — merge the mosaic/stack repo (Forge pipeline, MACP protocol, framework tools) into mosaic/mosaic-stack (Harness Foundation platform). Two repos doing related work that need to converge. + +## Context + +**mosaic/stack** (OLD) contains: + +- Forge progressive refinement pipeline (stages, agents, personas, rails, debate protocol, brief classification) +- MACP protocol (JSON schemas, deterministic Python controller, dispatcher, event system, gate runner) +- Credential resolver (Python — OC config, mosaic files, ambient env, JSON5 parser) +- OC framework plugin (injects Mosaic rails into all agent sessions) +- Profiles (runtime-neutral context packs for tech stacks and domains) +- Stage adapter (Forge→MACP bridge) +- Board tasks (multi-agent board evaluation) +- OpenBrain specialist memory (learning capture/recall) +- 17 guides, 5 universal skills + +**mosaic/mosaic-stack** (NEW) contains: + +- Harness Foundation platform (NestJS gateway, Next.js web, Drizzle ORM, Pi SDK runtime) +- 5 provider adapters, task classifier, routing rules, model capability matrix +- MACP OC plugin (ACP runtime backend with Pi bridge) +- TS coord package (mission runner, tasks file manager, status tracker — 1635 lines) +- BullMQ job queue, OTEL telemetry, channel plugins (Discord, Telegram) +- CLI with TUI, 65/65 tasks done, v0.2.0 + +**Decision:** NEW repo is the base. All unique work from OLD gets ported into NEW as packages. + +## Scope + +### Work Package 1: Forge Pipeline Package (`packages/forge`) + +Port the entire Forge progressive refinement pipeline as a TypeScript package. + +**From OLD:** + +- `forge/pipeline/stages/*.md` — 11 stage definitions +- `forge/pipeline/agents/{board,generalists,specialists,cross-cutting}/*.md` — all persona definitions +- `forge/pipeline/rails/*.md` — debate protocol, dynamic composition, worker rails +- `forge/pipeline/gates/` — gate reviewer definitions +- `forge/pipeline/orchestrator/run-structure.md` — file-based observability spec +- `forge/templates/` — brief and PRD templates +- `forge/pipeline/orchestrator/board_tasks.py` → rewrite in TS +- `forge/pipeline/orchestrator/stage_adapter.py` → rewrite in TS +- `forge/pipeline/orchestrator/pipeline_runner.py` → rewrite in TS +- `forge/forge` CLI (Python) → rewrite in TS, integrate with `packages/cli` + +**Package structure:** + +``` +packages/forge/ +├── src/ +│ ├── index.ts # Public API +│ ├── pipeline-runner.ts # Orchestrates full pipeline run +│ ├── stage-adapter.ts # Maps stages to MACP/coord tasks +│ ├── board-tasks.ts # Multi-agent board evaluation task generator +│ ├── brief-classifier.ts # strategic/technical/hotfix classification +│ ├── types.ts # Stage specs, run manifest, gate results +│ └── constants.ts # Stage sequence, timeouts, labels +├── pipeline/ +│ ├── stages/ # .md stage definitions (copied) +│ ├── agents/ # .md persona definitions (copied) +│ │ ├── board/ +│ │ ├── cross-cutting/ +│ │ ├── generalists/ +│ │ └── specialists/ +│ │ ├── language/ +│ │ └── domain/ +│ ├── rails/ # .md rails (copied) +│ ├── gates/ # .md gate definitions (copied) +│ └── templates/ # brief + PRD templates (copied) +└── package.json +``` + +**Key design decisions:** + +- Pipeline markdown assets are runtime data, not compiled — ship as-is in the package +- `pipeline-runner.ts` calls into `packages/coord` for task execution (not a separate controller) +- Stage adapter generates coord-compatible tasks, not MACP JSON directly +- Board tasks use `depends_on_policy: "all_terminal"` for synthesis +- Per-stage timeouts from `STAGE_TIMEOUTS` map +- Brief classifier supports CLI flag, YAML frontmatter, and keyword auto-detection +- Run output goes to project-scoped `.forge/runs/{run-id}/` (not inside the Forge package) + +**Persona override system (new):** + +- Base personas ship with the package (read-only) +- Project-level overrides in `.forge/personas/{role}.md` extend (not replace) base personas +- Board composition configurable via `.forge/config.yaml`: + ```yaml + board: + additional_members: + - compliance-officer.md + skip_members: [] + specialists: + always_include: + - proxmox-expert + ``` +- OpenBrain integration for cross-run specialist memory (when enabled) + +### Work Package 2: MACP Protocol Package (`packages/macp`) + +Port the MACP protocol layer, event system, and gate runner as a TypeScript package. + +**From OLD:** + +- `tools/macp/protocol/task.schema.json` — task JSON schema +- `tools/macp/protocol/` — event schemas +- `tools/macp/controller/gate_runner.py` → rewrite in TS as `gate-runner.ts` +- `tools/macp/events/` — event watcher, webhook adapter, Discord formatter → rewrite in TS +- `tools/macp/dispatcher/credential_resolver.py` → rewrite in TS as `credential-resolver.ts` +- `tools/macp/memory/learning_capture.py` + `learning_recall.py` → rewrite in TS + +**Package structure:** + +``` +packages/macp/ +├── src/ +│ ├── index.ts # Public API +│ ├── types.ts # Task, event, result, gate types +│ ├── schemas/ # JSON schemas (copied) +│ ├── gate-runner.ts # Mechanical + AI review quality gates +│ ├── credential-resolver.ts # Provider credential resolution (mosaic files, OC config, ambient) +│ ├── event-emitter.ts # Append events to ndjson, structured event types +│ ├── event-watcher.ts # Poll events.ndjson with cursor persistence +│ ├── webhook-adapter.ts # POST events to configurable URL +│ ├── discord-formatter.ts # Human-readable event messages +│ └── learning.ts # OpenBrain capture + recall +└── package.json +``` + +**Integration with existing packages:** + +- `packages/coord` uses `packages/macp` for event emission, gate running, and credential resolution +- `plugins/macp` uses `packages/macp` for protocol types and credential resolution +- `packages/forge` uses `packages/macp` gate types for stage gates + +### Work Package 3: OC Framework Plugin (`plugins/mosaic-framework`) + +Port the OC framework plugin that injects Mosaic rails into all agent sessions. + +**From OLD:** + +- `oc-plugins/mosaic-framework/index.ts` — `before_agent_start` + `subagent_spawning` hooks +- `oc-plugins/mosaic-framework/openclaw.plugin.json` + +**Structure:** + +``` +plugins/mosaic-framework/ +├── src/ +│ └── index.ts # Plugin hooks +└── package.json +``` + +**This is separate from `plugins/macp`:** + +- `mosaic-framework` = injects Mosaic rails/contracts into every OC session (passive enforcement) +- `macp` = provides an ACP runtime backend for MACP task execution (active runtime) + +### Work Package 4: Profiles + Guides + Skills + +Port reference content as a documentation/config package or top-level directories. + +**From OLD:** + +- `profiles/domains/*.json` — HIPAA, fintech, crypto context packs +- `profiles/tech-stacks/*.json` — NestJS, Next.js, FastAPI, React conventions +- `profiles/workflows/*.json` — API development, frontend component, testing workflows +- `guides/*.md` — 17 guides (auth, backend, QA, orchestrator, PRD, etc.) +- `skills-universal/` — jarvis, macp, mosaic-standards, prd, setup-cicd skills + +**Destination:** + +``` +profiles/ # Top-level (same as OLD) +guides/ # Top-level (same as OLD) +skills/ # Top-level (renamed from skills-universal) +``` + +These are runtime-neutral assets consumed by any agent or profile loader — they don't belong in a compiled package. + +## Out of Scope + +- Rewriting the NestJS orchestrator app from OLD (`apps/orchestrator/`) — its functionality is subsumed by `packages/coord` + `apps/gateway` +- Porting the FastAPI coordinator from OLD (`apps/coordinator/`) — its functionality (webhook receiver, issue parser, quality orchestrator) is handled by `packages/coord` + `apps/gateway` in the new architecture +- Porting the Prisma schema or OLD's `apps/api` — Drizzle migration is complete +- Old Docker Compose configs (Traefik, Matrix, OpenBao) — NEW has its own infra setup + +## Success Criteria + +1. `packages/forge` exists with all 11 stage definitions, all persona markdowns, all rails, and TS implementations of pipeline-runner, stage-adapter, board-tasks, and brief-classifier +2. `packages/macp` exists with gate-runner, credential-resolver, event system, and learning capture/recall — all in TypeScript +3. `plugins/mosaic-framework` exists and registers OC hooks for rails injection +4. Profiles, guides, and skills are present at top-level +5. `packages/forge` integrates with `packages/coord` for task execution +6. `packages/macp` credential-resolver is used by `plugins/macp` Pi bridge +7. All existing tests pass (no regressions) +8. New packages have test coverage ≥85% +9. `pnpm lint && pnpm typecheck && pnpm build` passes +10. `.forge/runs/` project-scoped output directory works for at least one test run + +## Technical Constraints + +- All new code is ESM with NodeNext module resolution +- No Python in the new repo — everything rewrites to TypeScript +- Pipeline markdown assets (stages, personas, rails) are shipped as package data, not compiled +- Credential resolver must support: mosaic credential files, OC config (JSON5), ambient environment — same resolution order as the Python version +- Must preserve `depends_on_policy` semantics (all, any, all_terminal) +- Per-stage timeouts must be preserved +- JSON5 stripping must use the placeholder-extraction approach (not naive regex on string content) + +## Estimated Complexity + +High — crosses 4 work packages with protocol porting, TS rewrites, and integration wiring. Each work package is independently shippable. + +**Suggested execution order:** + +1. WP4 (profiles/guides/skills) — pure copy, no code, fast win +2. WP2 (packages/macp) — protocol foundation, needed by WP1 and WP3 +3. WP1 (packages/forge) — the big one, depends on WP2 +4. WP3 (plugins/mosaic-framework) — OC integration, can parallel with WP1 + +## Dependencies + +- `packages/coord` must be stable (it is — WP1 integrates with it) +- `plugins/macp` must be stable (it is — WP2 provides types/credentials to it) +- Pi SDK (`@mariozechner/pi-agent-core`) already in the dependency tree