bootstrap/docs/scratchpads/macp-phase1.md

MACP Phase 1 Scratchpad

Objective

Implement MACP Phase 1 in mosaic-bootstrap by extending the orchestrator-matrix protocol, adding a dispatcher layer, wiring the controller, updating sync/config/CLI behavior, and preserving backward compatibility for legacy tasks.

Tracking

• Task ID: MACP-PHASE1
• Issue: #8
• Branch: feat/macp-phase1
• Repo: bootstrap

Budget

• User hard token budget: not specified
• Working assumption: complete in one implementation pass with focused verification and no unnecessary parallelization
• Budget response plan: keep scope limited to the requested file map and verification set

Plan

1. Extend task and event schemas and add the MACP result schema.
2. Build dispatcher/macp_dispatcher.py with isolated helpers for worktree, command, result, and cleanup logic.
3. Wire the controller to detect MACP-aware tasks, emit gated/escalated events, and preserve the legacy path for tasks without dispatch.
4. Add MACP defaults to config, add mosaic macp routing and shell script support, and extend tasks_md_sync.py.
5. Run backward-compatibility and MACP verification, then review/remediate before commit and PR flow.

Assumptions

1. dispatch presence is the MACP-aware signal for the controller, matching the user brief.
2. docs/tasks/MACP-PHASE1-brief.md is an input artifact for this delivery and should remain available for reference.
3. A minimal developer-facing documentation update tied to the orchestrator README/CLI behavior will satisfy the documentation gate for this change surface.

Risks

1. Worktree cleanup must be constrained to task-owned paths only.
2. Gate-state transitions need to remain compatible with current event consumers.
3. CLI submit behavior must avoid corrupting existing tasks.json structure.

Verification Plan

Acceptance Criterion	Verification Method	Evidence
AC-1 legacy tasks still work	Run controller against old-style tasks without dispatch	Passed: temp repo run completed legacy task and wrote completed result JSON
AC-2 MACP exec flow works	Run controller against MACP task queue and inspect result JSON/events/worktree cleanup	Passed: temp repo run produced task.gated and task.completed, wrote structured result JSON with files_changed, and removed the task worktree
AC-3 schemas are valid	Validate schema JSON loads and sample payloads parse/import cleanly	Passed: jsonschema.validate(...) succeeded for task, event, and result schema samples
AC-4 CLI works	Run mosaic macp submit/status/history against a test repo state	Passed: bin/mosaic-macp submit/status/history succeeded and bin/mosaic --help exposed macp
AC-5 imports are clean	Run python3 -c "import ..." for updated modules	Passed: importlib-based module loads and py_compile succeeded

Progress Log

• 2026-03-27: Loaded global/project/runtime contracts and required delivery guides.
• 2026-03-27: Created provider issue #8.
• 2026-03-27: Drafted PRD, task tracking, and this scratchpad before implementation.
• 2026-03-27: Implemented MACP schemas, dispatcher lifecycle, controller integration, CLI support, sync updates, and developer docs.
• 2026-03-27: Added explicit worker escalation handling via the MACP_ESCALATE: stdout marker.
• 2026-03-27: Committed and pushed branch feat/macp-phase1 (7ef49a3, fd6274f).
• 2026-03-27: Blocked in PR workflow when ~/.config/mosaic/tools/git/pr-create.sh failed to resolve the remote repository from this worktree.
• 2026-03-28: Resumed from blocked state for a review-fix pass covering 5 findings in docs/tasks/MACP-PHASE1-fixes.md.

Review Fix Pass

Scope

1. Restore legacy tasks_md_sync.py behavior so rows without MACP headers do not become MACP tasks.
2. Make ACP dispatch fail-safe via escalation instead of a no-op success path.
3. Move MACP result writes to the controller after quality gates determine the final task status.
4. Remove brief text from yolo command arguments by switching to file-based brief handoff.
5. Restrict worktree cleanup to validated paths under the configured worktree base.

TDD / Test-First Decision

1. This is a bug-fix and security-hardening pass, so targeted reproducer verification is required.
2. Repo appears to use focused script-level verification rather than a Python test suite for this surface, so reproducer checks will be command-driven and recorded as evidence.

Planned Verification Additions

Finding	Verification
Legacy task reclassification	Sync docs/TASKS.md without MACP headers into tasks.json and confirm dispatch is absent so controller stays on run_shell()
ACP no-op success	Run controller/dispatcher with dispatch=acp and confirm status=escalated, exit path is non-zero, and task.escalated is emitted
Premature result write	Inspect result JSON after final controller state only; confirm gate results are present and no dispatcher pre-write remains
Brief exposure	Build yolo command and confirm the brief body is absent from the command text
Unsafe cleanup	Call cleanup against a path outside configured base and confirm it is refused

Tests Run

• python3 -m py_compile tools/orchestrator-matrix/controller/mosaic_orchestrator.py tools/orchestrator-matrix/controller/tasks_md_sync.py tools/orchestrator-matrix/dispatcher/macp_dispatcher.py
• importlib module-load check for updated Python files
• jsonschema.validate(...) for task/event/result samples
• Temp repo legacy controller run with old-style tasks.json
• Temp repo MACP exec controller run with worktree/result/cleanup verification
• Temp repo CLI checks using bin/mosaic-macp submit, status, and history
• Temp repo tasks_md_sync.py --apply run with MACP columns
• git diff --check
• bash -n bin/mosaic bin/mosaic-macp

Review

• Automated review attempted with ~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted
• Initial review attempt failed because generated __pycache__ files introduced non-UTF-8 diff content; generated caches were removed
• Manual independent review completed on the dispatcher/controller/CLI delta
• Remediation applied: final MACP results are now written before worktree cleanup, and explicit worker escalation is handled instead of being ignored

Residual Risks

• The repo-local bin/mosaic route was verified through help output; full runtime routing depends on installation into $MOSAIC_HOME/bin.
• Explicit worker escalation currently uses a stdout marker convention rather than a richer structured worker-to-controller handoff.
• Delivery is blocked on repository automation because PR creation failed before merge/CI/issue-closure stages could run.