feat: monorepo consolidation — forge pipeline, MACP protocol, framework plugin, profiles/guides/skills

Work packages completed: - WP1: packages/forge — pipeline runner, stage adapter, board tasks, brief classifier, persona loader with project-level overrides. 89 tests, 95.62% coverage. - WP2: packages/macp — credential resolver, gate runner, event emitter, protocol types. 65 tests, 96.24% coverage. Full Python-to-TS port preserving all behavior. - WP3: plugins/mosaic-framework — OC rails injection plugin (before_agent_start + subagent_spawning hooks for Mosaic contract enforcement). - WP4: profiles/ (domains, tech-stacks, workflows), guides/ (17 docs), skills/ (5 universal skills), forge pipeline assets (48 markdown files). Board deliberation: docs/reviews/consolidation-board-memo.md Brief: briefs/monorepo-consolidation.md Consolidates mosaic/stack (forge, MACP, bootstrap framework) into mosaic/mosaic-stack. 154 new tests total. Zero Python — all TypeScript/ESM.
2026-03-30 19:43:24 +00:00
parent 40c068fcbc
commit 10689a30d2
123 changed files with 18166 additions and 11 deletions
--- a/briefs/monorepo-consolidation.md
+++ b/briefs/monorepo-consolidation.md
@@ -0,0 +1,231 @@
+# Brief: Monorepo Consolidation — mosaic/stack → mosaic/mosaic-stack
+
+## Source
+
+Architecture consolidation — merge the mosaic/stack repo (Forge pipeline, MACP protocol, framework tools) into mosaic/mosaic-stack (Harness Foundation platform). Two repos doing related work that need to converge.
+
+## Context
+
+**mosaic/stack** (OLD) contains:
+
+- Forge progressive refinement pipeline (stages, agents, personas, rails, debate protocol, brief classification)
+- MACP protocol (JSON schemas, deterministic Python controller, dispatcher, event system, gate runner)
+- Credential resolver (Python — OC config, mosaic files, ambient env, JSON5 parser)
+- OC framework plugin (injects Mosaic rails into all agent sessions)
+- Profiles (runtime-neutral context packs for tech stacks and domains)
+- Stage adapter (Forge→MACP bridge)
+- Board tasks (multi-agent board evaluation)
+- OpenBrain specialist memory (learning capture/recall)
+- 17 guides, 5 universal skills
+
+**mosaic/mosaic-stack** (NEW) contains:
+
+- Harness Foundation platform (NestJS gateway, Next.js web, Drizzle ORM, Pi SDK runtime)
+- 5 provider adapters, task classifier, routing rules, model capability matrix
+- MACP OC plugin (ACP runtime backend with Pi bridge)
+- TS coord package (mission runner, tasks file manager, status tracker — 1635 lines)
+- BullMQ job queue, OTEL telemetry, channel plugins (Discord, Telegram)
+- CLI with TUI, 65/65 tasks done, v0.2.0
+
+**Decision:** NEW repo is the base. All unique work from OLD gets ported into NEW as packages.
+
+## Scope
+
+### Work Package 1: Forge Pipeline Package (`packages/forge`)
+
+Port the entire Forge progressive refinement pipeline as a TypeScript package.
+
+**From OLD:**
+
+- `forge/pipeline/stages/*.md` — 11 stage definitions
+- `forge/pipeline/agents/{board,generalists,specialists,cross-cutting}/*.md` — all persona definitions
+- `forge/pipeline/rails/*.md` — debate protocol, dynamic composition, worker rails
+- `forge/pipeline/gates/` — gate reviewer definitions
+- `forge/pipeline/orchestrator/run-structure.md` — file-based observability spec
+- `forge/templates/` — brief and PRD templates
+- `forge/pipeline/orchestrator/board_tasks.py` → rewrite in TS
+- `forge/pipeline/orchestrator/stage_adapter.py` → rewrite in TS
+- `forge/pipeline/orchestrator/pipeline_runner.py` → rewrite in TS
+- `forge/forge` CLI (Python) → rewrite in TS, integrate with `packages/cli`
+
+**Package structure:**
+
+```
+packages/forge/
+├── src/
+│   ├── index.ts              # Public API
+│   ├── pipeline-runner.ts    # Orchestrates full pipeline run
+│   ├── stage-adapter.ts      # Maps stages to MACP/coord tasks
+│   ├── board-tasks.ts        # Multi-agent board evaluation task generator
+│   ├── brief-classifier.ts   # strategic/technical/hotfix classification
+│   ├── types.ts              # Stage specs, run manifest, gate results
+│   └── constants.ts          # Stage sequence, timeouts, labels
+├── pipeline/
+│   ├── stages/               # .md stage definitions (copied)
+│   ├── agents/               # .md persona definitions (copied)
+│   │   ├── board/
+│   │   ├── cross-cutting/
+│   │   ├── generalists/
+│   │   └── specialists/
+│   │       ├── language/
+│   │       └── domain/
+│   ├── rails/                # .md rails (copied)
+│   ├── gates/                # .md gate definitions (copied)
+│   └── templates/            # brief + PRD templates (copied)
+└── package.json
+```
+
+**Key design decisions:**
+
+- Pipeline markdown assets are runtime data, not compiled — ship as-is in the package
+- `pipeline-runner.ts` calls into `packages/coord` for task execution (not a separate controller)
+- Stage adapter generates coord-compatible tasks, not MACP JSON directly
+- Board tasks use `depends_on_policy: "all_terminal"` for synthesis
+- Per-stage timeouts from `STAGE_TIMEOUTS` map
+- Brief classifier supports CLI flag, YAML frontmatter, and keyword auto-detection
+- Run output goes to project-scoped `.forge/runs/{run-id}/` (not inside the Forge package)
+
+**Persona override system (new):**
+
+- Base personas ship with the package (read-only)
+- Project-level overrides in `.forge/personas/{role}.md` extend (not replace) base personas
+- Board composition configurable via `.forge/config.yaml`:
+  ```yaml
+  board:
+    additional_members:
+      - compliance-officer.md
+    skip_members: []
+  specialists:
+    always_include:
+      - proxmox-expert
+  ```
+- OpenBrain integration for cross-run specialist memory (when enabled)
+
+### Work Package 2: MACP Protocol Package (`packages/macp`)
+
+Port the MACP protocol layer, event system, and gate runner as a TypeScript package.
+
+**From OLD:**
+
+- `tools/macp/protocol/task.schema.json` — task JSON schema
+- `tools/macp/protocol/` — event schemas
+- `tools/macp/controller/gate_runner.py` → rewrite in TS as `gate-runner.ts`
+- `tools/macp/events/` — event watcher, webhook adapter, Discord formatter → rewrite in TS
+- `tools/macp/dispatcher/credential_resolver.py` → rewrite in TS as `credential-resolver.ts`
+- `tools/macp/memory/learning_capture.py` + `learning_recall.py` → rewrite in TS
+
+**Package structure:**
+
+```
+packages/macp/
+├── src/
+│   ├── index.ts                # Public API
+│   ├── types.ts                # Task, event, result, gate types
+│   ├── schemas/                # JSON schemas (copied)
+│   ├── gate-runner.ts          # Mechanical + AI review quality gates
+│   ├── credential-resolver.ts  # Provider credential resolution (mosaic files, OC config, ambient)
+│   ├── event-emitter.ts        # Append events to ndjson, structured event types
+│   ├── event-watcher.ts        # Poll events.ndjson with cursor persistence
+│   ├── webhook-adapter.ts      # POST events to configurable URL
+│   ├── discord-formatter.ts    # Human-readable event messages
+│   └── learning.ts             # OpenBrain capture + recall
+└── package.json
+```
+
+**Integration with existing packages:**
+
+- `packages/coord` uses `packages/macp` for event emission, gate running, and credential resolution
+- `plugins/macp` uses `packages/macp` for protocol types and credential resolution
+- `packages/forge` uses `packages/macp` gate types for stage gates
+
+### Work Package 3: OC Framework Plugin (`plugins/mosaic-framework`)
+
+Port the OC framework plugin that injects Mosaic rails into all agent sessions.
+
+**From OLD:**
+
+- `oc-plugins/mosaic-framework/index.ts` — `before_agent_start` + `subagent_spawning` hooks
+- `oc-plugins/mosaic-framework/openclaw.plugin.json`
+
+**Structure:**
+
+```
+plugins/mosaic-framework/
+├── src/
+│   └── index.ts    # Plugin hooks
+└── package.json
+```
+
+**This is separate from `plugins/macp`:**
+
+- `mosaic-framework` = injects Mosaic rails/contracts into every OC session (passive enforcement)
+- `macp` = provides an ACP runtime backend for MACP task execution (active runtime)
+
+### Work Package 4: Profiles + Guides + Skills
+
+Port reference content as a documentation/config package or top-level directories.
+
+**From OLD:**
+
+- `profiles/domains/*.json` — HIPAA, fintech, crypto context packs
+- `profiles/tech-stacks/*.json` — NestJS, Next.js, FastAPI, React conventions
+- `profiles/workflows/*.json` — API development, frontend component, testing workflows
+- `guides/*.md` — 17 guides (auth, backend, QA, orchestrator, PRD, etc.)
+- `skills-universal/` — jarvis, macp, mosaic-standards, prd, setup-cicd skills
+
+**Destination:**
+
+```
+profiles/           # Top-level (same as OLD)
+guides/             # Top-level (same as OLD)
+skills/             # Top-level (renamed from skills-universal)
+```
+
+These are runtime-neutral assets consumed by any agent or profile loader — they don't belong in a compiled package.
+
+## Out of Scope
+
+- Rewriting the NestJS orchestrator app from OLD (`apps/orchestrator/`) — its functionality is subsumed by `packages/coord` + `apps/gateway`
+- Porting the FastAPI coordinator from OLD (`apps/coordinator/`) — its functionality (webhook receiver, issue parser, quality orchestrator) is handled by `packages/coord` + `apps/gateway` in the new architecture
+- Porting the Prisma schema or OLD's `apps/api` — Drizzle migration is complete
+- Old Docker Compose configs (Traefik, Matrix, OpenBao) — NEW has its own infra setup
+
+## Success Criteria
+
+1. `packages/forge` exists with all 11 stage definitions, all persona markdowns, all rails, and TS implementations of pipeline-runner, stage-adapter, board-tasks, and brief-classifier
+2. `packages/macp` exists with gate-runner, credential-resolver, event system, and learning capture/recall — all in TypeScript
+3. `plugins/mosaic-framework` exists and registers OC hooks for rails injection
+4. Profiles, guides, and skills are present at top-level
+5. `packages/forge` integrates with `packages/coord` for task execution
+6. `packages/macp` credential-resolver is used by `plugins/macp` Pi bridge
+7. All existing tests pass (no regressions)
+8. New packages have test coverage ≥85%
+9. `pnpm lint && pnpm typecheck && pnpm build` passes
+10. `.forge/runs/` project-scoped output directory works for at least one test run
+
+## Technical Constraints
+
+- All new code is ESM with NodeNext module resolution
+- No Python in the new repo — everything rewrites to TypeScript
+- Pipeline markdown assets (stages, personas, rails) are shipped as package data, not compiled
+- Credential resolver must support: mosaic credential files, OC config (JSON5), ambient environment — same resolution order as the Python version
+- Must preserve `depends_on_policy` semantics (all, any, all_terminal)
+- Per-stage timeouts must be preserved
+- JSON5 stripping must use the placeholder-extraction approach (not naive regex on string content)
+
+## Estimated Complexity
+
+High — crosses 4 work packages with protocol porting, TS rewrites, and integration wiring. Each work package is independently shippable.
+
+**Suggested execution order:**
+
+1. WP4 (profiles/guides/skills) — pure copy, no code, fast win
+2. WP2 (packages/macp) — protocol foundation, needed by WP1 and WP3
+3. WP1 (packages/forge) — the big one, depends on WP2
+4. WP3 (plugins/mosaic-framework) — OC integration, can parallel with WP1
+
+## Dependencies
+
+- `packages/coord` must be stable (it is — WP1 integrates with it)
+- `plugins/macp` must be stable (it is — WP2 provides types/credentials to it)
+- Pi SDK (`@mariozechner/pi-agent-core`) already in the dependency tree