stack/briefs/monorepo-consolidation.md

Brief: Monorepo Consolidation — mosaic/stack → mosaic/mosaic-stack

Source

Architecture consolidation — merge the mosaic/stack repo (Forge pipeline, MACP protocol, framework tools) into mosaic/mosaic-stack (Harness Foundation platform). Two repos doing related work that need to converge.

Context

mosaic/stack (OLD) contains:

• Forge progressive refinement pipeline (stages, agents, personas, rails, debate protocol, brief classification)
• MACP protocol (JSON schemas, deterministic Python controller, dispatcher, event system, gate runner)
• Credential resolver (Python — OC config, mosaic files, ambient env, JSON5 parser)
• OC framework plugin (injects Mosaic rails into all agent sessions)
• Profiles (runtime-neutral context packs for tech stacks and domains)
• Stage adapter (Forge→MACP bridge)
• Board tasks (multi-agent board evaluation)
• OpenBrain specialist memory (learning capture/recall)
• 17 guides, 5 universal skills

mosaic/mosaic-stack (NEW) contains:

• Harness Foundation platform (NestJS gateway, Next.js web, Drizzle ORM, Pi SDK runtime)
• 5 provider adapters, task classifier, routing rules, model capability matrix
• MACP OC plugin (ACP runtime backend with Pi bridge)
• TS coord package (mission runner, tasks file manager, status tracker — 1635 lines)
• BullMQ job queue, OTEL telemetry, channel plugins (Discord, Telegram)
• CLI with TUI, 65/65 tasks done, v0.2.0

Decision: NEW repo is the base. All unique work from OLD gets ported into NEW as packages.

Scope

Work Package 1: Forge Pipeline Package (packages/forge)

Port the entire Forge progressive refinement pipeline as a TypeScript package.

From OLD:

• forge/pipeline/stages/*.md — 11 stage definitions
• forge/pipeline/agents/{board,generalists,specialists,cross-cutting}/*.md — all persona definitions
• forge/pipeline/rails/*.md — debate protocol, dynamic composition, worker rails
• forge/pipeline/gates/ — gate reviewer definitions
• forge/pipeline/orchestrator/run-structure.md — file-based observability spec
• forge/templates/ — brief and PRD templates
• forge/pipeline/orchestrator/board_tasks.py → rewrite in TS
• forge/pipeline/orchestrator/stage_adapter.py → rewrite in TS
• forge/pipeline/orchestrator/pipeline_runner.py → rewrite in TS
• forge/forge CLI (Python) → rewrite in TS, integrate with packages/cli

Package structure:

packages/forge/
├── src/
│   ├── index.ts              # Public API
│   ├── pipeline-runner.ts    # Orchestrates full pipeline run
│   ├── stage-adapter.ts      # Maps stages to MACP/coord tasks
│   ├── board-tasks.ts        # Multi-agent board evaluation task generator
│   ├── brief-classifier.ts   # strategic/technical/hotfix classification
│   ├── types.ts              # Stage specs, run manifest, gate results
│   └── constants.ts          # Stage sequence, timeouts, labels
├── pipeline/
│   ├── stages/               # .md stage definitions (copied)
│   ├── agents/               # .md persona definitions (copied)
│   │   ├── board/
│   │   ├── cross-cutting/
│   │   ├── generalists/
│   │   └── specialists/
│   │       ├── language/
│   │       └── domain/
│   ├── rails/                # .md rails (copied)
│   ├── gates/                # .md gate definitions (copied)
│   └── templates/            # brief + PRD templates (copied)
└── package.json

Key design decisions:

• Pipeline markdown assets are runtime data, not compiled — ship as-is in the package
• pipeline-runner.ts calls into packages/coord for task execution (not a separate controller)
• Stage adapter generates coord-compatible tasks, not MACP JSON directly
• Board tasks use depends_on_policy: "all_terminal" for synthesis
• Per-stage timeouts from STAGE_TIMEOUTS map
• Brief classifier supports CLI flag, YAML frontmatter, and keyword auto-detection
• Run output goes to project-scoped .forge/runs/{run-id}/ (not inside the Forge package)

Persona override system (new):

• Base personas ship with the package (read-only)
• Project-level overrides in .forge/personas/{role}.md extend (not replace) base personas
• Board composition configurable via .forge/config.yaml:

  board:
    additional_members:
      - compliance-officer.md
    skip_members: []
  specialists:
    always_include:
      - proxmox-expert
  

• OpenBrain integration for cross-run specialist memory (when enabled)

Work Package 2: MACP Protocol Package (packages/macp)

Port the MACP protocol layer, event system, and gate runner as a TypeScript package.

From OLD:

• tools/macp/protocol/task.schema.json — task JSON schema
• tools/macp/protocol/ — event schemas
• tools/macp/controller/gate_runner.py → rewrite in TS as gate-runner.ts
• tools/macp/events/ — event watcher, webhook adapter, Discord formatter → rewrite in TS
• tools/macp/dispatcher/credential_resolver.py → rewrite in TS as credential-resolver.ts
• tools/macp/memory/learning_capture.py + learning_recall.py → rewrite in TS

Package structure:

packages/macp/
├── src/
│   ├── index.ts                # Public API
│   ├── types.ts                # Task, event, result, gate types
│   ├── schemas/                # JSON schemas (copied)
│   ├── gate-runner.ts          # Mechanical + AI review quality gates
│   ├── credential-resolver.ts  # Provider credential resolution (mosaic files, OC config, ambient)
│   ├── event-emitter.ts        # Append events to ndjson, structured event types
│   ├── event-watcher.ts        # Poll events.ndjson with cursor persistence
│   ├── webhook-adapter.ts      # POST events to configurable URL
│   ├── discord-formatter.ts    # Human-readable event messages
│   └── learning.ts             # OpenBrain capture + recall
└── package.json

Integration with existing packages:

• packages/coord uses packages/macp for event emission, gate running, and credential resolution
• plugins/macp uses packages/macp for protocol types and credential resolution
• packages/forge uses packages/macp gate types for stage gates

Work Package 3: OC Framework Plugin (plugins/mosaic-framework)

Port the OC framework plugin that injects Mosaic rails into all agent sessions.

From OLD:

• oc-plugins/mosaic-framework/index.ts — before_agent_start + subagent_spawning hooks
• oc-plugins/mosaic-framework/openclaw.plugin.json

Structure:

plugins/mosaic-framework/
├── src/
│   └── index.ts    # Plugin hooks
└── package.json

This is separate from plugins/macp:

• mosaic-framework = injects Mosaic rails/contracts into every OC session (passive enforcement)
• macp = provides an ACP runtime backend for MACP task execution (active runtime)

Work Package 4: Profiles + Guides + Skills

Port reference content as a documentation/config package or top-level directories.

From OLD:

• profiles/domains/*.json — HIPAA, fintech, crypto context packs
• profiles/tech-stacks/*.json — NestJS, Next.js, FastAPI, React conventions
• profiles/workflows/*.json — API development, frontend component, testing workflows
• guides/*.md — 17 guides (auth, backend, QA, orchestrator, PRD, etc.)
• skills-universal/ — jarvis, macp, mosaic-standards, prd, setup-cicd skills

Destination:

profiles/           # Top-level (same as OLD)
guides/             # Top-level (same as OLD)
skills/             # Top-level (renamed from skills-universal)

These are runtime-neutral assets consumed by any agent or profile loader — they don't belong in a compiled package.

Out of Scope

• Rewriting the NestJS orchestrator app from OLD (apps/orchestrator/) — its functionality is subsumed by packages/coord + apps/gateway
• Porting the FastAPI coordinator from OLD (apps/coordinator/) — its functionality (webhook receiver, issue parser, quality orchestrator) is handled by packages/coord + apps/gateway in the new architecture
• Porting the Prisma schema or OLD's apps/api — Drizzle migration is complete
• Old Docker Compose configs (Traefik, Matrix, OpenBao) — NEW has its own infra setup

Success Criteria

1. packages/forge exists with all 11 stage definitions, all persona markdowns, all rails, and TS implementations of pipeline-runner, stage-adapter, board-tasks, and brief-classifier
2. packages/macp exists with gate-runner, credential-resolver, event system, and learning capture/recall — all in TypeScript
3. plugins/mosaic-framework exists and registers OC hooks for rails injection
4. Profiles, guides, and skills are present at top-level
5. packages/forge integrates with packages/coord for task execution
6. packages/macp credential-resolver is used by plugins/macp Pi bridge
7. All existing tests pass (no regressions)
8. New packages have test coverage ≥85%
9. pnpm lint && pnpm typecheck && pnpm build passes
10. .forge/runs/ project-scoped output directory works for at least one test run

Technical Constraints

• All new code is ESM with NodeNext module resolution
• No Python in the new repo — everything rewrites to TypeScript
• Pipeline markdown assets (stages, personas, rails) are shipped as package data, not compiled
• Credential resolver must support: mosaic credential files, OC config (JSON5), ambient environment — same resolution order as the Python version
• Must preserve depends_on_policy semantics (all, any, all_terminal)
• Per-stage timeouts must be preserved
• JSON5 stripping must use the placeholder-extraction approach (not naive regex on string content)

Estimated Complexity

High — crosses 4 work packages with protocol porting, TS rewrites, and integration wiring. Each work package is independently shippable.

Suggested execution order:

1. WP4 (profiles/guides/skills) — pure copy, no code, fast win
2. WP2 (packages/macp) — protocol foundation, needed by WP1 and WP3
3. WP1 (packages/forge) — the big one, depends on WP2
4. WP3 (plugins/mosaic-framework) — OC integration, can parallel with WP1

Dependencies

• packages/coord must be stable (it is — WP1 integrates with it)
• plugins/macp must be stable (it is — WP2 provides types/credentials to it)
• Pi SDK (@mariozechner/pi-agent-core) already in the dependency tree