feat: monorepo consolidation — forge pipeline, MACP protocol, framework plugin, profiles/guides/skills
Work packages completed: - WP1: packages/forge — pipeline runner, stage adapter, board tasks, brief classifier, persona loader with project-level overrides. 89 tests, 95.62% coverage. - WP2: packages/macp — credential resolver, gate runner, event emitter, protocol types. 65 tests, 96.24% coverage. Full Python-to-TS port preserving all behavior. - WP3: plugins/mosaic-framework — OC rails injection plugin (before_agent_start + subagent_spawning hooks for Mosaic contract enforcement). - WP4: profiles/ (domains, tech-stacks, workflows), guides/ (17 docs), skills/ (5 universal skills), forge pipeline assets (48 markdown files). Board deliberation: docs/reviews/consolidation-board-memo.md Brief: briefs/monorepo-consolidation.md Consolidates mosaic/stack (forge, MACP, bootstrap framework) into mosaic/mosaic-stack. 154 new tests total. Zero Python — all TypeScript/ESM.
This commit is contained in:
Mos (Agent)
1256
docs/reviews/consolidation-board-memo.md
Normal file
1256
docs/reviews/consolidation-board-memo.md
Normal file
File diff suppressed because it is too large
Load Diff
265
docs/tasks/WP1-forge-package.md
Normal file
265
docs/tasks/WP1-forge-package.md
Normal file
@@ -0,0 +1,265 @@
|
||||
# WP1: packages/forge — Forge Pipeline Package
|
||||
|
||||
## Context
|
||||
|
||||
Port the Forge progressive refinement pipeline from Python (~/src/mosaic-stack/forge/) to TypeScript as `packages/forge` in this monorepo. The pipeline markdown assets (stages, agents, personas, rails, gates, templates) are already copied to `packages/forge/pipeline/`. This task is the TypeScript implementation layer.
|
||||
|
||||
**Board decisions that constrain this work:**
|
||||
|
||||
- Abstract TaskExecutor interface — packages/forge must NOT hard-import packages/coord. Define an abstract interface; coord satisfies it.
|
||||
- Clean index.ts exports, no internal path leakage, no hardcoded paths
|
||||
- 85% test coverage on TS implementation files (markdown assets excluded)
|
||||
- Test strategy for non-deterministic AI orchestration: fixture-based integration tests
|
||||
- OpenBrain is OUT OF SCOPE
|
||||
- ESM only, zero Python
|
||||
|
||||
**Dependencies available:**
|
||||
|
||||
- `@mosaic/macp` (packages/macp) is built and provides: GateEntry, GateResult, Task types, credential resolution, gate running, event emission
|
||||
|
||||
## Source Files (Python → TypeScript)
|
||||
|
||||
### 1. types.ts
|
||||
|
||||
Define all Forge-specific types:
|
||||
|
||||
```typescript
|
||||
// Stage specification
|
||||
interface StageSpec {
|
||||
number: string;
|
||||
title: string;
|
||||
dispatch: 'exec' | 'yolo' | 'pi';
|
||||
type: 'research' | 'review' | 'coding' | 'deploy';
|
||||
gate: string;
|
||||
promptFile: string;
|
||||
qualityGates: (string | GateEntry)[];
|
||||
}
|
||||
|
||||
// Brief classification
|
||||
type BriefClass = 'strategic' | 'technical' | 'hotfix';
|
||||
type ClassSource = 'cli' | 'frontmatter' | 'auto';
|
||||
|
||||
// Run manifest (persisted to disk)
|
||||
interface RunManifest {
|
||||
runId: string;
|
||||
brief: string;
|
||||
codebase: string;
|
||||
briefClass: BriefClass;
|
||||
classSource: ClassSource;
|
||||
forceBoard: boolean;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
currentStage: string;
|
||||
status: 'in_progress' | 'completed' | 'failed' | 'interrupted' | 'rejected';
|
||||
stages: Record<string, StageStatus>;
|
||||
}
|
||||
|
||||
// Abstract task executor (decouples from packages/coord)
|
||||
interface TaskExecutor {
|
||||
submitTask(task: ForgeTask): Promise<void>;
|
||||
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
|
||||
}
|
||||
|
||||
// Persona override config
|
||||
interface ForgeConfig {
|
||||
board?: {
|
||||
additionalMembers?: string[];
|
||||
skipMembers?: string[];
|
||||
};
|
||||
specialists?: {
|
||||
alwaysInclude?: string[];
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### 2. constants.ts
|
||||
|
||||
**Source:** Top of `~/src/mosaic-stack/forge/lib` (ALL_STAGES, LABELS, STAGE_SPECS equivalent) + `~/src/mosaic-stack/forge/pipeline/orchestrator/stage_adapter.py` (STAGE_TIMEOUTS)
|
||||
|
||||
```typescript
|
||||
export const STAGE_SEQUENCE = [
|
||||
'00-intake',
|
||||
'00b-discovery',
|
||||
'01-board',
|
||||
'01b-brief-analyzer',
|
||||
'02-planning-1',
|
||||
'03-planning-2',
|
||||
'04-planning-3',
|
||||
'05-coding',
|
||||
'06-review',
|
||||
'07-remediate',
|
||||
'08-test',
|
||||
'09-deploy',
|
||||
];
|
||||
|
||||
export const STAGE_TIMEOUTS: Record<string, number> = {
|
||||
'00-intake': 120,
|
||||
'00b-discovery': 300,
|
||||
'01-board': 120,
|
||||
'02-planning-1': 600,
|
||||
// ... etc
|
||||
};
|
||||
|
||||
export const STAGE_LABELS: Record<string, string> = {
|
||||
'00-intake': 'INTAKE',
|
||||
// ... etc
|
||||
};
|
||||
```
|
||||
|
||||
Also: STRATEGIC_KEYWORDS, TECHNICAL_KEYWORDS for brief classification.
|
||||
|
||||
### 3. brief-classifier.ts
|
||||
|
||||
**Source:** `classify_brief()`, `parse_brief_frontmatter()`, `stages_for_class()` from `~/src/mosaic-stack/forge/lib`
|
||||
|
||||
- Auto-classify brief by keyword analysis (strategic vs technical)
|
||||
- Parse YAML frontmatter for explicit `class:` field
|
||||
- CLI flag override
|
||||
- Return stage list based on classification (strategic = full pipeline, technical = skip board, hotfix = skip board + brief analyzer)
|
||||
|
||||
### 4. stage-adapter.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/stage_adapter.py`
|
||||
|
||||
- `mapStageToTask()`: Convert a Forge stage into a task compatible with TaskExecutor
|
||||
- Stage briefs written to `{runDir}/{stageName}/brief.md`
|
||||
- Result paths at `{runDir}/{stageName}/result.json`
|
||||
- Previous results read from disk at runtime (not baked into brief)
|
||||
- Per-stage timeouts from STAGE_TIMEOUTS
|
||||
- depends_on chain built from stage sequence
|
||||
|
||||
### 5. board-tasks.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/board_tasks.py`
|
||||
|
||||
- `loadBoardPersonas()`: Read all .md files from `pipeline/agents/board/`
|
||||
- `generateBoardTasks()`: One task per persona + synthesis task
|
||||
- Synthesis depends on all persona tasks with `depends_on_policy: 'all_terminal'`
|
||||
- Persona briefs include role description + brief under review
|
||||
- Synthesis script merges independent reviews into board memo
|
||||
|
||||
### 6. pipeline-runner.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/pipeline_runner.py` + `~/src/mosaic-stack/forge/lib` (cmd_run, cmd_resume, cmd_status)
|
||||
|
||||
- `runPipeline(briefPath, projectRoot, options)`: Full pipeline execution
|
||||
- Creates run directory at `{projectRoot}/.forge/runs/{runId}/`
|
||||
- Generates tasks for all stages, submits to TaskExecutor
|
||||
- Tracks manifest.json with stage statuses
|
||||
- `resumePipeline(runDir)`: Pick up from last incomplete stage
|
||||
- `getPipelineStatus(runDir)`: Read manifest and report
|
||||
|
||||
**Key difference from Python:** Run output goes to PROJECT-scoped `.forge/runs/`, not inside the Forge package.
|
||||
|
||||
### 7. Persona Override System (NEW — not in Python)
|
||||
|
||||
- Base personas read from `packages/forge/pipeline/agents/`
|
||||
- Project overrides read from `{projectRoot}/.forge/personas/{role}.md`
|
||||
- Merge strategy: project persona content APPENDED to base persona (not replaced)
|
||||
- Board composition configurable via `{projectRoot}/.forge/config.yaml`
|
||||
- If no project config exists, use defaults (all base personas, no overrides)
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
packages/forge/
|
||||
├── src/
|
||||
│ ├── index.ts
|
||||
│ ├── types.ts
|
||||
│ ├── constants.ts
|
||||
│ ├── brief-classifier.ts
|
||||
│ ├── stage-adapter.ts
|
||||
│ ├── board-tasks.ts
|
||||
│ ├── pipeline-runner.ts
|
||||
│ └── persona-loader.ts
|
||||
├── pipeline/ # Already copied (WP4) — markdown assets
|
||||
│ ├── stages/
|
||||
│ ├── agents/
|
||||
│ ├── rails/
|
||||
│ ├── gates/
|
||||
│ └── templates/
|
||||
├── __tests__/
|
||||
│ ├── brief-classifier.test.ts
|
||||
│ ├── stage-adapter.test.ts
|
||||
│ ├── board-tasks.test.ts
|
||||
│ ├── pipeline-runner.test.ts
|
||||
│ └── persona-loader.test.ts
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── vitest.config.ts
|
||||
```
|
||||
|
||||
## Package.json
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@mosaic/forge",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": "./src/index.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mosaic/macp": "workspace:*"
|
||||
},
|
||||
"devDependencies": {
|
||||
"vitest": "workspace:*",
|
||||
"typescript": "workspace:*"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Only dependency: @mosaic/macp (for gate types, event emission).
|
||||
|
||||
## Test Strategy (Board requirement)
|
||||
|
||||
**Deterministic code (brief-classifier, stage-adapter, board-tasks, persona-loader, constants):**
|
||||
|
||||
- Standard unit tests with known inputs/outputs
|
||||
- 100% of classification logic, stage mapping, persona loading covered
|
||||
|
||||
**Non-deterministic code (pipeline-runner):**
|
||||
|
||||
- Fixture-based integration tests using a mock TaskExecutor
|
||||
- Mock executor returns pre-recorded results for each stage
|
||||
- Tests verify: manifest progression, stage ordering, dependency enforcement, resume behavior, error handling
|
||||
- NO real AI calls in tests
|
||||
|
||||
**Markdown assets:** Excluded from coverage measurement (configure vitest to exclude `pipeline/` directory).
|
||||
|
||||
## ESM Requirements
|
||||
|
||||
- `"type": "module"` in package.json
|
||||
- NodeNext module resolution in tsconfig
|
||||
- `.js` extensions in all imports
|
||||
- No CommonJS
|
||||
|
||||
## Key Design: Abstract TaskExecutor
|
||||
|
||||
```typescript
|
||||
// In packages/forge/src/types.ts
|
||||
export interface TaskExecutor {
|
||||
submitTask(task: ForgeTask): Promise<void>;
|
||||
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
|
||||
getTaskStatus(taskId: string): Promise<TaskStatus>;
|
||||
}
|
||||
|
||||
// In packages/coord (or wherever the concrete impl lives)
|
||||
export class CoordTaskExecutor implements TaskExecutor {
|
||||
// ... uses packages/coord runner
|
||||
}
|
||||
```
|
||||
|
||||
This means packages/forge can be tested with a mock executor and deployed with any backend.
|
||||
|
||||
## Asset Resolution
|
||||
|
||||
Pipeline markdown assets (stages, personas, rails) must be resolved relative to the package installation, NOT hardcoded paths:
|
||||
|
||||
```typescript
|
||||
// Use import.meta.url to find package root
|
||||
const PACKAGE_ROOT = new URL('..', import.meta.url).pathname;
|
||||
const PIPELINE_DIR = path.join(PACKAGE_ROOT, 'pipeline');
|
||||
```
|
||||
|
||||
Project-level overrides resolved relative to projectRoot parameter.
|
||||
150
docs/tasks/WP2-macp-package.md
Normal file
150
docs/tasks/WP2-macp-package.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# WP2: packages/macp — MACP Protocol Package
|
||||
|
||||
## Context
|
||||
|
||||
Port the MACP protocol layer from Python (in ~/src/mosaic-stack/tools/macp/) to TypeScript as `packages/macp` in this monorepo. This package provides the foundational protocol types, quality gate execution, credential resolution, and event system that `packages/coord` and `plugins/macp` depend on.
|
||||
|
||||
**Board decisions that constrain this work:**
|
||||
|
||||
- No Python in the new repo — everything rewrites to TypeScript
|
||||
- OpenBrain learning capture/recall is OUT OF SCOPE (deferred to future brief)
|
||||
- 85% test coverage on TS implementation files
|
||||
- Credential resolver behavior must be captured as test fixtures BEFORE rewrite
|
||||
- Clean index.ts exports, no internal path leakage
|
||||
|
||||
## Source Files (Python → TypeScript)
|
||||
|
||||
### 1. credential-resolver.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/tools/macp/dispatcher/credential_resolver.py`
|
||||
|
||||
Resolution order (MUST preserve exactly):
|
||||
|
||||
1. Mosaic credential files (`~/.config/mosaic/credentials/{provider}.env`)
|
||||
2. OpenClaw config (`~/.openclaw/openclaw.json`) — env block + models.providers.{provider}.apiKey
|
||||
3. Ambient environment variables
|
||||
4. CredentialError (failure)
|
||||
|
||||
Key behaviors to preserve:
|
||||
|
||||
- Provider registry: anthropic, openai, zai → env var names + credential file paths + OC config paths
|
||||
- Dotenv parser: handles single/double quotes, comments, blank lines
|
||||
- JSON5 stripping: placeholder-extraction approach (NOT naive regex) — protects URLs and timestamps inside string values
|
||||
- OC config permission check: warn on world-readable, skip if wrong owner
|
||||
- Redacted marker detection: `__OPENCLAW_REDACTED__` values skipped
|
||||
- Task-level override via `credentials.provider_key_env`
|
||||
|
||||
### 2. gate-runner.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/tools/macp/controller/gate_runner.py`
|
||||
|
||||
Three gate types:
|
||||
|
||||
- `mechanical`: shell command, pass = exit code 0
|
||||
- `ai-review`: shell command producing JSON, parse findings, fail on blockers
|
||||
- `ci-pipeline`: placeholder (always passes for now)
|
||||
|
||||
Key behaviors:
|
||||
|
||||
- `normalize_gate()`: accepts string or dict, normalizes to gate entry
|
||||
- `run_gate()`: executes single gate, returns result with pass/fail
|
||||
- `run_gates()`: executes all gates, emits events, returns (all_passed, results)
|
||||
- AI review parsing: `_count_ai_findings()` reads stats.blockers or findings[].severity
|
||||
- `fail_on` modes: "blocker" (default) or "any"
|
||||
|
||||
### 3. event-emitter.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/tools/macp/controller/gate_runner.py` (emit_event, append_event functions) + `~/src/mosaic-stack/tools/macp/events/`
|
||||
|
||||
- Append structured events to ndjson file
|
||||
- Event types: task.assigned, task.started, task.completed, task.failed, task.escalated, task.gated, task.retry.scheduled, rail.check.started, rail.check.passed, rail.check.failed
|
||||
- Each event: event_id (uuid), event_type, task_id, status, timestamp, source, message, metadata
|
||||
|
||||
### 4. types.ts
|
||||
|
||||
**Source:** `~/src/mosaic-stack/tools/macp/protocol/task.schema.json`
|
||||
|
||||
TypeScript types for:
|
||||
|
||||
- Task (id, title, status, dispatch, runtime, depends_on, depends_on_policy, quality_gates, timeout_seconds, metadata, etc.)
|
||||
- Event (event_id, event_type, task_id, status, timestamp, source, message, metadata)
|
||||
- GateResult (command, exit_code, type, passed, output, findings, blockers)
|
||||
- TaskResult (task_id, status, completed_at, exit_code, gate_results, files_changed, etc.)
|
||||
- CredentialError, ProviderRegistry
|
||||
|
||||
### 5. schemas/ (copy)
|
||||
|
||||
Copy `~/src/mosaic-stack/tools/macp/protocol/task.schema.json` as-is.
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
packages/macp/
|
||||
├── src/
|
||||
│ ├── index.ts
|
||||
│ ├── types.ts
|
||||
│ ├── credential-resolver.ts
|
||||
│ ├── gate-runner.ts
|
||||
│ ├── event-emitter.ts
|
||||
│ └── schemas/
|
||||
│ └── task.schema.json
|
||||
├── __tests__/
|
||||
│ ├── credential-resolver.test.ts
|
||||
│ ├── gate-runner.test.ts
|
||||
│ └── event-emitter.test.ts
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── vitest.config.ts
|
||||
```
|
||||
|
||||
## Package.json
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@mosaic/macp",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": "./src/index.ts"
|
||||
},
|
||||
"dependencies": {},
|
||||
"devDependencies": {
|
||||
"vitest": "workspace:*",
|
||||
"typescript": "workspace:*"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Zero external dependencies. Uses node:fs, node:path, node:child_process, node:crypto only.
|
||||
|
||||
## Test Requirements
|
||||
|
||||
Port ALL existing Python tests as TypeScript equivalents:
|
||||
|
||||
- `test_resolve_from_file` → credential file resolution
|
||||
- `test_resolve_from_ambient` → ambient env resolution
|
||||
- `test_resolve_from_oc_config_env_block` → OC config env block
|
||||
- `test_resolve_from_oc_config_provider_apikey` → OC config provider
|
||||
- `test_oc_config_precedence` → mosaic file wins over OC config
|
||||
- `test_oc_config_missing_file` → graceful fallback
|
||||
- `test_json5_strip` → structural transforms
|
||||
- `test_json5_strip_urls_and_timestamps` → URLs/timestamps survive
|
||||
- `test_redacted_values_skipped` → redacted marker detection
|
||||
- `test_oc_config_permission_warning` → file permission check
|
||||
- `test_resolve_missing_raises` → CredentialError thrown
|
||||
- Gate runner: mechanical pass/fail, AI review parsing, ci-pipeline placeholder
|
||||
- Event emitter: append to ndjson, event structure validation
|
||||
|
||||
## ESM Requirements
|
||||
|
||||
- `"type": "module"` in package.json
|
||||
- NodeNext module resolution in tsconfig
|
||||
- `.js` extensions in all imports
|
||||
- No CommonJS (`require`, `module.exports`)
|
||||
|
||||
## Integration Points
|
||||
|
||||
After this package is built:
|
||||
|
||||
- `packages/coord` should import `@mosaic/macp` for event emission and gate types
|
||||
- `plugins/macp` should import `@mosaic/macp` for credential resolution and protocol types
|
||||
63
docs/tasks/WP3-mosaic-framework-plugin.md
Normal file
63
docs/tasks/WP3-mosaic-framework-plugin.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# WP3: plugins/mosaic-framework — OC Rails Injection Plugin
|
||||
|
||||
## Context
|
||||
|
||||
Port the OpenClaw framework plugin from ~/src/mosaic-stack/oc-plugins/mosaic-framework/ to `plugins/mosaic-framework` in this monorepo. This plugin injects Mosaic framework contracts (rails, completion gates, worktree requirements) into every OpenClaw agent session.
|
||||
|
||||
**This is SEPARATE from plugins/macp:**
|
||||
|
||||
- `mosaic-framework` = passive enforcement — injects rails into all OC sessions
|
||||
- `macp` = active runtime — provides ACP backend for MACP task execution
|
||||
|
||||
## Source Files
|
||||
|
||||
**Source:** `~/src/mosaic-stack/oc-plugins/mosaic-framework/`
|
||||
|
||||
- `index.ts` — plugin hooks (before_agent_start, subagent_spawning)
|
||||
- `openclaw.plugin.json` — plugin manifest
|
||||
- `package.json`
|
||||
|
||||
## What It Does
|
||||
|
||||
### For OC native agents (before_agent_start hook):
|
||||
|
||||
- Injects Mosaic global hard rules via `appendSystemContext`
|
||||
- Completion gates: code review ✓ | security review ✓ | tests GREEN ✓ | CI green ✓
|
||||
- Worker completion protocol: open PR → fire system event → EXIT — never merge
|
||||
- Worktree requirement: `~/src/{repo}-worktrees/{task-slug}`, never `/tmp`
|
||||
- Injects dynamic mission state via `prependContext` (reads from project's `.mosaic/orchestrator/mission.json`)
|
||||
|
||||
### For ACP coding workers (subagent_spawning hook):
|
||||
|
||||
- Writes `~/.codex/instructions.md` or `~/.claude/CLAUDE.md` BEFORE the process starts
|
||||
- Full runtime contract: mandatory load order, hard gates, mode declaration
|
||||
- Global framework rules + worktree + completion gate requirements
|
||||
|
||||
## Implementation
|
||||
|
||||
Port the TypeScript source, updating hardcoded paths to be configurable. The OC plugin SDK imports should reference the installed OpenClaw location dynamically (not hardcoded `/home/jarvis/` paths like the OLD version).
|
||||
|
||||
**Structure:**
|
||||
|
||||
```
|
||||
plugins/mosaic-framework/
|
||||
├── src/
|
||||
│ └── index.ts
|
||||
├── openclaw.plugin.json
|
||||
├── package.json
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
## Key Constraint
|
||||
|
||||
The plugin SDK imports in the OLD version use absolute paths:
|
||||
|
||||
```typescript
|
||||
import type { OpenClawPluginApi } from '/home/jarvis/.npm-global/lib/node_modules/openclaw/dist/plugin-sdk/index.js';
|
||||
```
|
||||
|
||||
This must be resolved dynamically or via a peer dependency. Check how `plugins/macp` handles this in the new repo and follow the same pattern.
|
||||
|
||||
## Tests
|
||||
|
||||
Minimal — plugin hooks are integration-tested against OC runtime. Unit test the context string builders and config resolution.
|
||||
Reference in New Issue
Block a user