mosaicstack/bootstrap
Archived
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: MACP Phase 1 — Core Protocol Implementation (#9)

Browse Source
This commit was merged in pull request #9.
This commit is contained in:
jason.woltje
2026-03-28 01:39:26 +00:00
parent 24496cea01
commit 28392914a7
19 changed files with 1724 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/DEVELOPER-GUIDE/README.md Normal file
View File

@@ -0,0 +1,5 @@
# Developer Guide
## Orchestrator Matrix
- [MACP Phase 1](./orchestrator-matrix/macp-phase1.md)

31
docs/DEVELOPER-GUIDE/orchestrator-matrix/macp-phase1.md Normal file
View File

@@ -0,0 +1,31 @@
# MACP Phase 1
MACP Phase 1 extends `tools/orchestrator-matrix/` without replacing the existing deterministic controller model.
## What Changed
1. Task and event schemas now describe MACP dispatch metadata, new lifecycle statuses, and dispatcher-originated events.
2. `tools/orchestrator-matrix/dispatcher/macp_dispatcher.py` manages task worktrees, dispatch command generation, result files, and cleanup.
3. `mosaic_orchestrator.py` routes MACP-aware tasks through the dispatcher while leaving legacy non-`dispatch` tasks on the original shell path.
4. `bin/mosaic-macp` adds manual submit, status, drain, and history operations for `.mosaic/orchestrator/`.
## Dispatch Modes
1. `exec`: runs the task's `command` directly inside the task worktree.
2. `yolo`: launches `mosaic yolo <runtime>` via a PTY wrapper and stages the brief in a temporary file so the brief body is not exposed in process arguments.
3. `acp`: escalates immediately with `ACP dispatch requires OpenClaw integration (Phase 2)` until real ACP/OpenClaw spawning exists.
## Result Contract
MACP writes task result JSON under `.mosaic/orchestrator/results/` by default. Result files capture:
1. Task status and timing
2. Attempt counters
3. Runtime and dispatch metadata
4. Changed files seen in the task worktree
5. Quality-gate command results
6. Error or escalation details
## Compatibility
Legacy tasks that omit `dispatch` still behave like the original matrix controller. `tasks_md_sync.py` only injects MACP fields when the corresponding markdown headers exist, which keeps existing `tasks.json` workflows functional while allowing orchestrators to opt into MACP incrementally.

98
docs/PRD.md Normal file
View File

@@ -0,0 +1,98 @@
# PRD: MACP Phase 1 Core Protocol Implementation
## Metadata
- Owner: Jarvis
- Date: 2026-03-27
- Status: in-progress
- Best-Guess Mode: true
## Problem Statement
The current orchestrator-matrix rail can queue shell-based worker tasks, but it does not yet expose a standardized protocol for dispatch selection, worktree-aware execution, structured results, or manual MACP queue operations. MACP Phase 1 extends the existing rail so orchestrators can delegate to multiple runtimes through a consistent task model while preserving current behavior for legacy tasks.
## Objectives
1. Extend the existing orchestrator-matrix protocol and controller to support MACP-aware task dispatch and status tracking.
2. Add a dispatcher layer that manages worktree lifecycle, runtime command generation, and standardized results.
3. Provide a CLI entrypoint for manual MACP submission, status inspection, queue draining, and history review.
## Scope
### In Scope
1. Extend the orchestrator task and event schemas and add a result schema.
2. Add a Python dispatcher module under `tools/orchestrator-matrix/dispatcher/`.
3. Update the controller to use the dispatcher for MACP-aware tasks while preserving legacy execution paths.
4. Update orchestrator config templates, task markdown sync logic, and CLI routing/scripts for MACP commands.
5. Add verification for backward compatibility, schema validity, imports, and basic MACP execution flow.
### Out of Scope
1. Rewriting the orchestrator controller architecture.
2. Changing Matrix transport behavior beyond schema compatibility.
3. Implementing real OpenClaw `sessions_spawn` execution beyond producing the config payload/command for callers.
4. Adding non-stdlib Python dependencies or npm-based tooling.
## User/Stakeholder Requirements
1. MACP must evolve the current orchestrator-matrix implementation rather than replace it.
2. Legacy task queues without `dispatch` fields must continue to run exactly as before.
3. MACP-aware tasks must support dispatch modes `yolo`, `acp`, and `exec`.
4. Results must be written in a structured JSON format suitable for audit and orchestration follow-up.
5. A manual `mosaic macp` CLI must expose submit, status, drain, and history flows.
## Functional Requirements
1. Task schema must include MACP dispatch, worktree, result, retry, branch, brief, issue/PR, and dependency fields.
2. Event schema must recognize `task.gated`, `task.escalated`, and `task.retry.scheduled`, plus a `dispatcher` source.
3. Dispatcher functions must set up worktrees, build commands, execute tasks, collect results, and clean up worktrees.
4. Controller `run_single_task()` must route MACP-aware tasks through the dispatcher and emit the correct lifecycle events/status transitions.
5. `tasks_md_sync.py` must map optional MACP table columns only when those headers are present in `docs/TASKS.md`; absent MACP headers must not inject MACP fields into legacy tasks.
6. `bin/mosaic` must route `mosaic macp ...` to a new `bin/mosaic-macp` script.
## Non-Functional Requirements
1. Security: no secrets embedded in generated commands, config, or results.
2. Performance: controller remains deterministic and synchronous with no async or thread-based orchestration.
3. Reliability: worktree creation/cleanup failures must be surfaced predictably and produce structured task failure/escalation states.
4. Observability: lifecycle events, logs, and result JSON must clearly show task outcome, attempts, gates, and errors.
## Acceptance Criteria
1. Existing legacy tasks without `dispatch` still run through the old shell path with unchanged behavior.
2. MACP-aware `exec` tasks run through the dispatcher and produce result JSON with gate outcomes.
3. New schemas validate task/event/result payload expectations for MACP fields and statuses.
4. `mosaic macp submit`, `status`, and `history` work from a bootstrapped repo state, and `drain` delegates to the existing orchestrator runner.
5. Python imports for the updated controller, dispatcher, and sync code complete without errors on Python 3.10+.
## Constraints and Dependencies
1. Python implementation must use stdlib only and support Python 3.10+.
2. Shell tooling must remain bash-based and fit the existing Mosaic CLI style.
3. Dispatch fallback rules must use `exec` when `dispatch` is absent and config/default runtime when `runtime` is absent.
4. Worktree convention must derive from the repository name and task metadata unless explicitly overridden by task fields.
## Risks and Open Questions
1. Risk: yolo command execution requires a PTY, so the dispatcher needs a safe wrapper that still behaves under `subprocess`.
2. Risk: worktree cleanup could remove a path unexpectedly if task metadata is malformed.
3. Risk: old queue consumers may assume only the original task statuses and event types.
4. Open Question: whether `task.gated` should be emitted by the dispatcher or controller once worker execution ends and quality gates begin.
## Testing and Verification Expectations
1. Baseline checks: Python import validation, targeted script execution checks, JSON syntax/schema validation, and any repo-local validation applicable to changed code paths.
2. Situational testing: legacy orchestrator run with old-style tasks, MACP `exec` flow including result file generation, CLI submit/status/history behavior, and worktree lifecycle validation.
3. Evidence format: command-level results captured in the scratchpad and summarized in the final delivery report.
## Milestone / Delivery Intent
1. Target milestone/version: 0.0.x bootstrap enhancement
2. Definition of done: code merged to `main`, CI terminal green, issue `#8` closed, and verification evidence recorded against all acceptance criteria.
## Assumptions
1. ASSUMPTION: A single issue can track the full Phase 1 implementation because the user requested one bounded feature delivery rather than separate independent tickets.
2. ASSUMPTION: For `acp` dispatch in Phase 1, the controller must escalate the task immediately with a clear reason instead of pretending work ran before OpenClaw integration exists.
3. ASSUMPTION: `task.gated` should be emitted by the controller as the transition into quality-gate execution, which keeps gate-state ownership in one place alongside the existing gate loop.

7
docs/SITEMAP.md Normal file
View File

@@ -0,0 +1,7 @@
# Sitemap
- [PRD](./PRD.md)
- [Tasks](./TASKS.md)
- [Developer Guide](./DEVELOPER-GUIDE/README.md)
- [Task Briefs](./tasks/MACP-PHASE1-brief.md)
- [Scratchpads](./scratchpads/macp-phase1.md)

17
docs/TASKS.md Normal file
View File

@@ -0,0 +1,17 @@
# TASKS
Canonical tracking for active work. Keep this file current.
## Rules
1. Update status as work progresses.
2. Link every non-trivial task to a provider issue (`#123`) or internal ref (`TASKS:T1`) if no provider is available.
3. Keep one row per active task.
4. Do not set `status=done` for source-code work until PR is merged, CI/pipeline is terminal green, and linked issue/ref is closed.
5. If merge/CI/issue closure fails, set `status=blocked` and record the exact failed wrapper command in `notes`.
## Tasks
| id | status | description | issue | repo | branch | depends_on | blocks | agent | started_at | completed_at | estimate | used | notes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MACP-PHASE1 | in-progress | Implement MACP Phase 1 across orchestrator schemas, dispatcher, controller, CLI, config, and task sync while preserving legacy queue behavior. | #8 | bootstrap | feat/macp-phase1 | | | Jarvis | 2026-03-27T23:00:00Z | | medium | in-progress | Review-fix pass started 2026-03-28T00:38:01Z to address backward-compatibility, ACP safety, result timing, and worktree/brief security findings on top of the blocked PR-create state. Prior blocker remains: `~/.config/mosaic/tools/git/pr-create.sh -t 'feat: implement MACP phase 1 core protocol' -b ... -B main -H feat/macp-phase1` failed with `Remote repository required: Specify ID via --repo or execute from a local git repo.` |

109
docs/scratchpads/macp-phase1.md Normal file
View File

@@ -0,0 +1,109 @@
# MACP Phase 1 Scratchpad
## Objective
Implement MACP Phase 1 in `mosaic-bootstrap` by extending the orchestrator-matrix protocol, adding a dispatcher layer, wiring the controller, updating sync/config/CLI behavior, and preserving backward compatibility for legacy tasks.
## Tracking
- Task ID: `MACP-PHASE1`
- Issue: `#8`
- Branch: `feat/macp-phase1`
- Repo: `bootstrap`
## Budget
- User hard token budget: not specified
- Working assumption: complete in one implementation pass with focused verification and no unnecessary parallelization
- Budget response plan: keep scope limited to the requested file map and verification set
## Plan
1. Extend task and event schemas and add the MACP result schema.
2. Build `dispatcher/macp_dispatcher.py` with isolated helpers for worktree, command, result, and cleanup logic.
3. Wire the controller to detect MACP-aware tasks, emit gated/escalated events, and preserve the legacy path for tasks without `dispatch`.
4. Add MACP defaults to config, add `mosaic macp` routing and shell script support, and extend `tasks_md_sync.py`.
5. Run backward-compatibility and MACP verification, then review/remediate before commit and PR flow.
## Assumptions
1. `dispatch` presence is the MACP-aware signal for the controller, matching the user brief.
2. `docs/tasks/MACP-PHASE1-brief.md` is an input artifact for this delivery and should remain available for reference.
3. A minimal developer-facing documentation update tied to the orchestrator README/CLI behavior will satisfy the documentation gate for this change surface.
## Risks
1. Worktree cleanup must be constrained to task-owned paths only.
2. Gate-state transitions need to remain compatible with current event consumers.
3. CLI submit behavior must avoid corrupting existing `tasks.json` structure.
## Verification Plan
| Acceptance Criterion | Verification Method | Evidence |
|---|---|---|
| AC-1 legacy tasks still work | Run controller against old-style tasks without `dispatch` | Passed: temp repo run completed legacy task and wrote completed result JSON |
| AC-2 MACP `exec` flow works | Run controller against MACP task queue and inspect result JSON/events/worktree cleanup | Passed: temp repo run produced `task.gated` and `task.completed`, wrote structured result JSON with `files_changed`, and removed the task worktree |
| AC-3 schemas are valid | Validate schema JSON loads and sample payloads parse/import cleanly | Passed: `jsonschema.validate(...)` succeeded for task, event, and result schema samples |
| AC-4 CLI works | Run `mosaic macp submit/status/history` against a test repo state | Passed: `bin/mosaic-macp` submit/status/history succeeded and `bin/mosaic --help` exposed `macp` |
| AC-5 imports are clean | Run `python3 -c "import ..."` for updated modules | Passed: importlib-based module loads and `py_compile` succeeded |
## Progress Log
- 2026-03-27: Loaded global/project/runtime contracts and required delivery guides.
- 2026-03-27: Created provider issue `#8`.
- 2026-03-27: Drafted PRD, task tracking, and this scratchpad before implementation.
- 2026-03-27: Implemented MACP schemas, dispatcher lifecycle, controller integration, CLI support, sync updates, and developer docs.
- 2026-03-27: Added explicit worker escalation handling via the `MACP_ESCALATE:` stdout marker.
- 2026-03-27: Committed and pushed branch `feat/macp-phase1` (`7ef49a3`, `fd6274f`).
- 2026-03-27: Blocked in PR workflow when `~/.config/mosaic/tools/git/pr-create.sh` failed to resolve the remote repository from this worktree.
- 2026-03-28: Resumed from blocked state for a review-fix pass covering 5 findings in `docs/tasks/MACP-PHASE1-fixes.md`.
## Review Fix Pass
### Scope
1. Restore legacy `tasks_md_sync.py` behavior so rows without MACP headers do not become MACP tasks.
2. Make ACP dispatch fail-safe via escalation instead of a no-op success path.
3. Move MACP result writes to the controller after quality gates determine the final task status.
4. Remove brief text from yolo command arguments by switching to file-based brief handoff.
5. Restrict worktree cleanup to validated paths under the configured worktree base.
### TDD / Test-First Decision
1. This is a bug-fix and security-hardening pass, so targeted reproducer verification is required.
2. Repo appears to use focused script-level verification rather than a Python test suite for this surface, so reproducer checks will be command-driven and recorded as evidence.
### Planned Verification Additions
| Finding | Verification |
|---|---|
| Legacy task reclassification | Sync `docs/TASKS.md` without MACP headers into `tasks.json` and confirm `dispatch` is absent so controller stays on `run_shell()` |
| ACP no-op success | Run controller/dispatcher with `dispatch=acp` and confirm `status=escalated`, exit path is non-zero, and `task.escalated` is emitted |
| Premature result write | Inspect result JSON after final controller state only; confirm gate results are present and no dispatcher pre-write remains |
| Brief exposure | Build yolo command and confirm the brief body is absent from the command text |
| Unsafe cleanup | Call cleanup against a path outside configured base and confirm it is refused |
## Tests Run
- `python3 -m py_compile tools/orchestrator-matrix/controller/mosaic_orchestrator.py tools/orchestrator-matrix/controller/tasks_md_sync.py tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`
- importlib module-load check for updated Python files
- `jsonschema.validate(...)` for task/event/result samples
- Temp repo legacy controller run with old-style `tasks.json`
- Temp repo MACP `exec` controller run with worktree/result/cleanup verification
- Temp repo CLI checks using `bin/mosaic-macp submit`, `status`, and `history`
- Temp repo `tasks_md_sync.py --apply` run with MACP columns
- `git diff --check`
- `bash -n bin/mosaic bin/mosaic-macp`
## Review
- Automated review attempted with `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted`
- Initial review attempt failed because generated `__pycache__` files introduced non-UTF-8 diff content; generated caches were removed
- Manual independent review completed on the dispatcher/controller/CLI delta
- Remediation applied: final MACP results are now written before worktree cleanup, and explicit worker escalation is handled instead of being ignored
## Residual Risks
- The repo-local `bin/mosaic` route was verified through help output; full runtime routing depends on installation into `$MOSAIC_HOME/bin`.
- Explicit worker escalation currently uses a stdout marker convention rather than a richer structured worker-to-controller handoff.
- Delivery is blocked on repository automation because PR creation failed before merge/CI/issue-closure stages could run.

324
docs/tasks/MACP-PHASE1-brief.md Normal file
View File

@@ -0,0 +1,324 @@
# MACP Phase 1 — Core Protocol Implementation
**Branch:** `feat/macp-phase1`
**Repo:** `mosaic-bootstrap` (worktree at `~/src/mosaic-bootstrap-worktrees/macp-phase1`)
---
## Objective
Extend the existing orchestrator-matrix into **MACP (Mosaic Agent Coordination Protocol)** — a standardized protocol that lets any orchestrator delegate work to any agent runtime, track progress via quality gates, and collect structured results.
This is an **evolution** of the existing code, not a rewrite. Extend existing schemas, enhance the controller, and add the dispatcher layer.
---
## Task 1: Extend Task Schema (`tools/orchestrator-matrix/protocol/task.schema.json`)
Add these new fields to the existing schema:
```json
{
"type": {
"type": "string",
"enum": ["coding", "deploy", "research", "review", "documentation", "infrastructure"],
"description": "Task type — determines dispatch strategy and gate requirements"
},
"dispatch": {
"type": "string",
"enum": ["yolo", "acp", "exec"],
"description": "Execution backend: yolo=mosaic yolo (full system), acp=OpenClaw sessions_spawn (sandboxed), exec=direct shell"
},
"worktree": {
"type": "string",
"description": "Path to git worktree for this task, e.g. ~/src/repo-worktrees/task-042"
},
"branch": {
"type": "string",
"description": "Git branch name for this task"
},
"brief_path": {
"type": "string",
"description": "Path to markdown task brief relative to repo root"
},
"result_path": {
"type": "string",
"description": "Path to JSON result file relative to .mosaic/orchestrator/"
},
"issue": {
"type": "string",
"description": "Issue reference (e.g. #42)"
},
"pr": {
"type": ["string", "null"],
"description": "PR number/URL once opened"
},
"depends_on": {
"type": "array",
"items": { "type": "string" },
"description": "List of task IDs this task depends on"
},
"max_attempts": {
"type": "integer",
"minimum": 1,
"default": 1
},
"attempts": {
"type": "integer",
"minimum": 0,
"default": 0
},
"timeout_seconds": {
"type": "integer",
"description": "Override default timeout for this task"
}
}
```
Extend the `status` enum to include: `"pending"`, `"running"`, `"gated"`, `"completed"`, `"failed"`, `"escalated"`
Keep all existing fields. Keep `additionalProperties: true`.
---
## Task 2: Extend Event Schema (`tools/orchestrator-matrix/protocol/event.schema.json`)
Add new event types to the enum:
- `task.gated` — Worker done coding, quality gates now running
- `task.escalated` — Requires human intervention
- `task.retry.scheduled` — Task will be retried (already emitted by controller, just not in schema)
Add new source type: `"dispatcher"`
Keep all existing event types and fields.
---
## Task 3: Create Result Schema (`tools/orchestrator-matrix/protocol/result.schema.json`)
New file — standardized worker completion report:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://mosaicstack.dev/schemas/orchestrator/result.schema.json",
"title": "MACP Task Result",
"type": "object",
"required": ["task_id", "status", "completed_at"],
"properties": {
"task_id": { "type": "string" },
"status": { "type": "string", "enum": ["completed", "failed", "escalated"] },
"completed_at": { "type": "string", "format": "date-time" },
"failed_at": { "type": ["string", "null"], "format": "date-time" },
"exit_code": { "type": ["integer", "null"] },
"attempt": { "type": "integer" },
"max_attempts": { "type": "integer" },
"runtime": { "type": "string" },
"dispatch": { "type": "string" },
"worktree": { "type": ["string", "null"] },
"branch": { "type": ["string", "null"] },
"pr": { "type": ["string", "null"] },
"summary": { "type": "string", "description": "Human-readable summary of what the worker did" },
"files_changed": { "type": "array", "items": { "type": "string" } },
"gate_results": {
"type": "array",
"items": {
"type": "object",
"properties": {
"command": { "type": "string" },
"exit_code": { "type": "integer" },
"type": { "type": "string", "enum": ["mechanical", "ai-review", "ci-pipeline"] }
}
}
},
"error": { "type": ["string", "null"] },
"escalation_reason": { "type": ["string", "null"] },
"metadata": { "type": "object" }
}
}
```
---
## Task 4: Build MACP Dispatcher (`tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`)
New Python module — the core of MACP. This translates task specs into execution commands and manages the full lifecycle.
### Dispatcher Responsibilities:
1. **Worktree setup** — `git worktree add` before dispatch (if task has `worktree` field)
2. **Command generation** — Build the right command based on `dispatch` type:
- `yolo`: `mosaic yolo codex|claude|opencode "<brief contents>"` (reads from `brief_path`)
- `acp`: Generates a command that would be used with OpenClaw sessions_spawn (just outputs the config JSON for the caller)
- `exec`: Direct shell command from task `command` field
3. **Result collection** — After worker exits, write structured result JSON to `results/<task-id>.json`
4. **Worktree cleanup** — After task completion (success or failure after max retries), remove the worktree
### Key Functions:
```python
def setup_worktree(task: dict, repo_root: Path) -> Path:
"""Create git worktree for task. Returns worktree path."""
def build_dispatch_command(task: dict, repo_root: Path) -> str:
"""Generate execution command based on task dispatch type and runtime."""
def collect_result(task: dict, exit_code: int, gate_results: list, orch_dir: Path) -> dict:
"""Build standardized result JSON after worker completion."""
def cleanup_worktree(task: dict) -> None:
"""Remove git worktree after task is done."""
def dispatch_task(task: dict, repo_root: Path, orch_dir: Path, config: dict) -> tuple[int, str]:
"""Full dispatch lifecycle: setup → execute → collect → cleanup. Returns (exit_code, output)."""
```
### Worktree Convention:
- Base path: `~/src/<repo-name>-worktrees/`
- Worktree name: task ID slugified (e.g., `task-042-auth`)
- Branch: `feat/<task-id>-<slugified-title>` (or use task `branch` field if specified)
- Created from: `origin/main`
### `mosaic yolo` Command Pattern:
The dispatcher needs to invoke `mosaic yolo` via PTY (it requires a terminal). The command pattern:
```bash
export PATH="$HOME/.config/mosaic/bin:$PATH"
cd <worktree_path>
mosaic yolo codex "<task brief contents>"
```
For the dispatcher, since it's called from the controller which uses `subprocess.Popen`, the command should be wrapped appropriately. Use `script -qec` or similar for PTY simulation if needed, but prefer passing the brief via a temp file and using the worker script pattern.
### Important Constraints:
- The dispatcher is a **library module** imported by the controller — NOT a standalone script
- It should be testable (pure functions where possible, side effects isolated)
- Error handling: if worktree creation fails, task goes to `failed` immediately
- If `dispatch` is not set on a task, fall back to `exec` (backward compatible)
- If `runtime` is not set, fall back to config's `worker.runtime`
---
## Task 5: Wire Controller to Use Dispatcher (`tools/orchestrator-matrix/controller/mosaic_orchestrator.py`)
Modify the existing controller to use the new dispatcher:
1. **Import the dispatcher module**
2. **In `run_single_task()`**:
- If task has `dispatch` field (or is MACP-aware), call `dispatcher.dispatch_task()` instead of raw `run_shell()`
- If task does NOT have `dispatch` field, keep existing `run_shell()` behavior (backward compatibility)
3. **Handle new statuses**: `gated` and `escalated`
- `gated`: Set when worker completes but gates are running (transition state between running and completed/failed)
- `escalated`: Set when task needs human intervention (gate failures after max retries, explicit worker escalation)
4. **Emit new event types**: `task.gated` and `task.escalated`
### Backward Compatibility is Critical:
- Existing tasks.json files with no `dispatch` field MUST continue to work exactly as before
- The controller should detect MACP-aware tasks by checking for the `dispatch` field
- All existing tests/workflows must remain functional
---
## Task 6: Update Config Template (`templates/repo/.mosaic/orchestrator/config.json`)
Add MACP dispatcher configuration:
```json
{
"enabled": false,
"transport": "matrix",
"macp": {
"worktree_base": "~/src/{repo}-worktrees",
"default_dispatch": "exec",
"default_runtime": "codex",
"cleanup_worktrees": true,
"brief_dir": "docs/tasks",
"result_dir": ".mosaic/orchestrator/results"
},
"worker": { ... existing ... },
"quality_gates": [ ... existing ... ]
}
```
---
## Task 7: Build `mosaic macp` CLI (`bin/mosaic-macp`)
Shell script that provides manual MACP interaction:
```bash
mosaic macp submit --task-id TASK-001 --title "..." --type coding --runtime codex --brief docs/tasks/TASK-001.md
mosaic macp status [--task-id TASK-001]
mosaic macp drain # Run all pending tasks sequentially
mosaic macp history [--task-id TASK-001] # Show events for a task
```
Implementation:
- `submit`: Adds a task to `tasks.json` with MACP fields populated
- `status`: Pretty-prints task queue state (pending/running/gated/completed/failed/escalated counts)
- `drain`: Calls `mosaic-orchestrator-run --until-drained`
- `history`: Reads `events.ndjson` and filters by task ID
Register in the main `bin/mosaic` dispatcher so `mosaic macp ...` works.
---
## Task 8: Update `tasks_md_sync.py`
Extend the sync script to handle MACP-specific columns in `docs/TASKS.md`:
Optional new columns: `type`, `dispatch`, `runtime`, `branch`
If these columns exist in the markdown table, map them to the task JSON. If they don't exist, use defaults from config.
---
## Verification
After all tasks are complete:
1. **Existing behavior preserved**: Run `python3 tools/orchestrator-matrix/controller/mosaic_orchestrator.py --repo /tmp/test-repo --once` with an old-style tasks.json — must work identically
2. **New MACP flow works**: Create a tasks.json with `dispatch: "exec"` tasks, run controller, verify worktree creation, execution, result collection, and cleanup
3. **Schema validation**: All JSON files should validate against their schemas
4. **CLI works**: `mosaic macp status` and `mosaic macp submit` produce correct output
5. **No broken imports**: All Python files should pass `python3 -c "import ..."`
---
## File Map (what goes where)
```
tools/orchestrator-matrix/
├── protocol/
│ ├── task.schema.json ← MODIFY (extend)
│ ├── event.schema.json ← MODIFY (extend)
│ └── result.schema.json ← NEW
├── dispatcher/
│ ├── __init__.py ← NEW
│ └── macp_dispatcher.py ← NEW
├── controller/
│ ├── mosaic_orchestrator.py ← MODIFY (wire dispatcher)
│ └── tasks_md_sync.py ← MODIFY (new columns)
├── transport/
│ └── matrix_transport.py ← UNCHANGED
└── adapters/
└── README.md ← UNCHANGED
templates/repo/.mosaic/orchestrator/
├── config.json ← MODIFY (add macp section)
├── tasks.json ← UNCHANGED
└── ...
bin/
├── mosaic ← MODIFY (add macp subcommand routing)
└── mosaic-macp ← NEW
```
---
## Ground Rules
- This is the `mosaic-bootstrap` repo — a CLI framework, NOT a web app
- No npm/pnpm — this is bash + Python (stdlib only, no pip dependencies)
- All Python must work with Python 3.10+
- Keep the deterministic controller pattern — no async, no threads, no external services
- Commit messages: `feat: <what changed>` (conventional commits)
- Push to `feat/macp-phase1` branch when done

64
docs/tasks/MACP-PHASE1-fixes.md Normal file
View File

@@ -0,0 +1,64 @@
# MACP Phase 1 — Review Fixes
**Branch:** `feat/macp-phase1` (amend/fix commits on top of existing)
**Repo worktree:** `~/src/mosaic-bootstrap-worktrees/macp-phase1`
These are 5 findings from code review and security review. Fix all of them.
---
## Fix 1 (BLOCKER): Legacy task reclassification in tasks_md_sync.py
**File:** `tools/orchestrator-matrix/controller/tasks_md_sync.py`
**Problem:** `build_task()` now always sets `task["dispatch"]` from `macp_defaults` even when the `docs/TASKS.md` table has no `dispatch` column. The controller's `is_macp_task()` check is simply `"dispatch" in task`, so ALL synced tasks get reclassified as MACP tasks and routed through the dispatcher instead of the legacy `run_shell()` path. This breaks backward compatibility.
**Fix:** Only populate `dispatch`, `type`, `runtime`, and `branch` from MACP defaults when the markdown table row **explicitly contains that column**. If the column is absent from the table header, do NOT inject MACP fields. Check whether the parsed headers include these column names before adding them.
---
## Fix 2 (BLOCKER): ACP dispatch is a no-op that falsely reports success
**File:** `tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`
**Problem:** For `dispatch == "acp"`, `build_dispatch_command()` returns a `python3 -c ...` command that only prints JSON but doesn't actually spawn an ACP session. The controller then marks the task completed even though no work ran.
**Fix:** For `acp` dispatch, do NOT generate a command that runs and exits 0. Instead, set the task status to `"escalated"` with `escalation_reason = "ACP dispatch requires OpenClaw integration (Phase 2)"` and return exit code 1. This makes it fail-safe — ACP tasks won't silently succeed. The controller should emit a `task.escalated` event for these.
---
## Fix 3 (SHOULD-FIX): Premature result write before quality gates
**File:** `tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`
**Problem:** `dispatch_task()` calls `collect_result(task, exit_code, [], orch_dir)` immediately after the worker exits, before quality gates run. If the controller crashes between this write and the final overwrite, a false "completed" result with empty gate_results sits on disk.
**Fix:** Remove the `collect_result()` call from `dispatch_task()`. Instead, have `dispatch_task()` return `(exit_code, output)` only. The controller in `mosaic_orchestrator.py` should call `collect_result()` AFTER quality gates have run, when the final status is known. Make sure the controller passes gate_results to collect_result.
---
## Fix 4 (SECURITY MEDIUM): Brief contents exposed in process arguments
**File:** `tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`
**Problem:** For `yolo` dispatch, `build_dispatch_command()` puts the full task brief text into the shell command as an argument: `mosaic yolo codex "<entire brief>"`. This is visible in `ps` output, shell logs, and crash reports.
**Fix:** Write the brief contents to a temporary file with restrictive permissions (0600), and pass the file path to the worker instead. The command should read: `mosaic yolo <runtime> "$(cat /path/to/brief-TASKID.tmp)"` or better, restructure so the worker script reads from a file path argument. Use the existing `orchestrator-worker.sh` pattern which already reads from a task file. Clean up the temp file after dispatch completes.
---
## Fix 5 (SECURITY MEDIUM): Unrestricted worktree cleanup path
**File:** `tools/orchestrator-matrix/dispatcher/macp_dispatcher.py`
**Problem:** `cleanup_worktree()` trusts `task['worktree']` without validating it belongs to the expected worktree base directory. A tampered task could cause deletion of unrelated worktrees.
**Fix:** Before running `git worktree remove`, validate that the resolved worktree path starts with the configured `worktree_base` (from `config.macp.worktree_base`, with `{repo}` expanded). If the path doesn't match the expected base, log a warning and refuse to clean up. Add a helper `_is_safe_worktree_path(worktree_path, config)` for this validation.
---
## Verification
After fixes:
1. `python3 -c "from tools.orchestrator_matrix.dispatcher import macp_dispatcher"` should not error (fix import path if needed, or just verify `python3 tools/orchestrator-matrix/dispatcher/macp_dispatcher.py` has no syntax errors)
2. Legacy tasks.json with no `dispatch` field must still work with the controller's `run_shell()` path
3. ACP dispatch should NOT mark tasks completed — should escalate
4. No brief text should appear in generated shell commands for yolo dispatch
5. `cleanup_worktree()` should refuse paths outside the configured base
Commit with: `fix: address review findings — backward compat, ACP safety, result timing, security`