8d8db47289
CRITICAL finding: Agents cannot trigger compaction - "compact and continue" does NOT work - Only user typing /compact in CLI works - Auto-compact at ~95% is too late Updated protocol: - Stop at 55-60% context usage - Output COMPACTION REQUIRED checkpoint - Wait for user to run /compact and say "continue" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
420 lines
15 KiB
Markdown
420 lines
15 KiB
Markdown
# Mosaic Stack Orchestrator Guide
|
||
|
||
> Platform-specific orchestrator protocol for Mosaic Stack.
|
||
|
||
## Overview
|
||
|
||
The orchestrator **cold-starts** with just a review report location and minimal kickstart. It autonomously:
|
||
|
||
1. Parses review reports to extract findings
|
||
2. Categorizes findings into phases by severity
|
||
3. Estimates token usage per task
|
||
4. Creates Gitea issues (phase-level)
|
||
5. Bootstraps `docs/tasks.md` from scratch
|
||
6. Coordinates completion using worker agents
|
||
|
||
**Key principle:** The orchestrator is the **sole writer** of `docs/tasks.md`. Worker agents execute tasks and report results — they never modify the tracking file.
|
||
|
||
---
|
||
|
||
## Bootstrap Templates
|
||
|
||
Use templates from `docs/templates/` (relative to repo root):
|
||
|
||
```bash
|
||
# Set environment variables
|
||
export PROJECT="mosaic-stack"
|
||
export MILESTONE="M6-Feature"
|
||
export CURRENT_DATETIME=$(date -Iseconds)
|
||
export TASK_PREFIX="MS-SEC"
|
||
export PHASE_ISSUE="#337"
|
||
export PHASE_BRANCH="fix/security"
|
||
|
||
# Create tasks.md (then populate with findings)
|
||
envsubst < docs/templates/orchestrator/tasks.md.template > docs/tasks.md
|
||
|
||
# Create learnings tracking
|
||
envsubst < docs/templates/orchestrator/orchestrator-learnings.json.template > docs/orchestrator-learnings.json
|
||
|
||
# Create review report structure (if doing new review)
|
||
./docs/templates/reports/review-report-scaffold.sh codebase-review mosaic-stack
|
||
```
|
||
|
||
**Available templates:**
|
||
|
||
| Template | Purpose |
|
||
| --------------------------------------------------- | ------------------------------- |
|
||
| `orchestrator/tasks.md.template` | Task tracking table with schema |
|
||
| `orchestrator/orchestrator-learnings.json.template` | Variance tracking |
|
||
| `orchestrator/phase-issue-body.md.template` | Gitea issue body |
|
||
| `orchestrator/compaction-summary.md.template` | 60% checkpoint format |
|
||
| `reports/review-report-scaffold.sh` | Creates report directory |
|
||
| `scratchpad.md.template` | Per-task working document |
|
||
|
||
See `docs/templates/README.md` for full documentation.
|
||
|
||
### CLI Tools
|
||
|
||
Git operations use `@mosaic/cli-tools` package (auto-detects Gitea vs GitHub):
|
||
|
||
```bash
|
||
# Issue operations
|
||
pnpm exec mosaic-issue-create -t "Title" -b "Body" -m "Milestone"
|
||
pnpm exec mosaic-issue-list -s open -m "Milestone"
|
||
|
||
# PR operations
|
||
pnpm exec mosaic-pr-create -t "Title" -b "Body" -B develop
|
||
pnpm exec mosaic-pr-merge -n 42 -m squash -d
|
||
|
||
# Milestone operations
|
||
pnpm exec mosaic-milestone-create -t "M7-Feature" -d "Description"
|
||
```
|
||
|
||
See `packages/cli-tools/README.md` for full command reference.
|
||
|
||
---
|
||
|
||
## Phase 1: Bootstrap
|
||
|
||
### Step 1: Parse Review Reports
|
||
|
||
Review reports follow this structure:
|
||
|
||
```
|
||
docs/reports/{report-name}/
|
||
├── 00-executive-summary.md # Start here - overview and counts
|
||
├── 01-security-review.md # Security findings with IDs like SEC-*
|
||
├── 02-code-quality-review.md # Code quality findings like CQ-*
|
||
├── 03-qa-test-coverage.md # Test coverage gaps like TEST-*
|
||
└── ...
|
||
```
|
||
|
||
**Extract findings by looking for:**
|
||
|
||
- Finding IDs (e.g., `SEC-API-1`, `CQ-WEB-3`, `TEST-001`)
|
||
- Severity labels: Critical, High, Medium, Low
|
||
- Affected files/components (use for `repo` column)
|
||
- Specific line numbers or code patterns
|
||
|
||
### Step 2: Categorize into Phases
|
||
|
||
| Severity | Phase | Focus | Branch Pattern |
|
||
| -------- | ----- | --------------------------------------- | ------------------- |
|
||
| Critical | 1 | Security vulnerabilities, data exposure | `fix/security` |
|
||
| High | 2 | Security hardening, auth gaps | `fix/security` |
|
||
| Medium | 3 | Code quality, performance, bugs | `fix/code-quality` |
|
||
| Low | 4 | Tests, documentation, cleanup | `fix/test-coverage` |
|
||
|
||
**Within each phase, order tasks by:**
|
||
|
||
1. Blockers first (tasks that unblock others)
|
||
2. Same-file tasks grouped together
|
||
3. Simpler fixes before complex ones
|
||
|
||
### Step 3: Estimate Token Usage
|
||
|
||
| Task Type | Estimate | Examples |
|
||
| --------------------- | -------- | ----------------------------------------- |
|
||
| Single-line fix | 3-5K | Typo, wrong operator, missing null check |
|
||
| Add guard/validation | 5-8K | Add auth decorator, input validation |
|
||
| Fix error handling | 8-12K | Proper try/catch, error propagation |
|
||
| Refactor pattern | 10-15K | Replace KEYS with SCAN, fix memory leak |
|
||
| Add new functionality | 15-25K | New service method, new component |
|
||
| Write tests | 15-25K | Unit tests for untested service |
|
||
| Complex refactor | 25-40K | Architectural change, multi-file refactor |
|
||
|
||
**Adjust estimates based on:**
|
||
|
||
- Number of files affected (+5K per additional file)
|
||
- Test requirements (+5-10K if tests needed)
|
||
- Documentation needs (+2-3K if docs needed)
|
||
|
||
### Step 4: Determine Dependencies
|
||
|
||
**Automatic dependency rules:**
|
||
|
||
1. All tasks in Phase N depend on the Phase N-1 verification task
|
||
2. Tasks touching the same file should be sequential (earlier blocks later)
|
||
3. Auth/security foundation tasks block tasks that rely on them
|
||
4. Each phase ends with a verification task that depends on all phase tasks
|
||
|
||
### Step 5: Create Gitea Issues (Phase-Level)
|
||
|
||
Create ONE issue per phase using `@mosaic/cli-tools`:
|
||
|
||
```bash
|
||
# Use mosaic CLI tools (auto-detects Gitea vs GitHub)
|
||
pnpm exec mosaic-issue-create \
|
||
-t "Phase 1: Critical Security Fixes" \
|
||
-b "## Findings
|
||
- SEC-API-1: Description
|
||
- SEC-WEB-2: Description
|
||
|
||
## Acceptance Criteria
|
||
- [ ] All critical findings remediated
|
||
- [ ] Quality gates passing" \
|
||
-l "security" \
|
||
-m "{milestone-name}"
|
||
```
|
||
|
||
**CLI tools location:** `packages/cli-tools/bin/` - see `packages/cli-tools/README.md` for full documentation.
|
||
|
||
### Step 6: Create docs/tasks.md
|
||
|
||
Create the file with this exact schema:
|
||
|
||
```markdown
|
||
# Tasks
|
||
|
||
| id | status | description | issue | repo | branch | depends_on | blocks | agent | started_at | completed_at | estimate | used |
|
||
| ---------- | ----------- | ---------------------------- | ----- | ---- | ------------ | ---------- | ---------- | ----- | ---------- | ------------ | -------- | ---- |
|
||
| MS-SEC-001 | not-started | SEC-API-1: Brief description | #337 | api | fix/security | | MS-SEC-002 | | | | 8K | |
|
||
```
|
||
|
||
**Column definitions:**
|
||
|
||
| Column | Format | Purpose |
|
||
| -------------- | ---------------------------------------------------- | ------------------------------------------- |
|
||
| `id` | `MS-{CAT}-{NNN}` | Unique task ID |
|
||
| `status` | `not-started` \| `in-progress` \| `done` \| `failed` | Current state |
|
||
| `description` | `{FindingID}: Brief summary` | What to fix |
|
||
| `issue` | `#NNN` | Gitea issue (phase-level) |
|
||
| `repo` | Workspace name | `api`, `web`, `orchestrator`, `coordinator` |
|
||
| `branch` | Branch name | `fix/security`, `fix/code-quality`, etc. |
|
||
| `depends_on` | Comma-separated IDs | Must complete first |
|
||
| `blocks` | Comma-separated IDs | Tasks waiting on this |
|
||
| `agent` | Agent identifier | Assigned worker |
|
||
| `started_at` | ISO 8601 | When work began |
|
||
| `completed_at` | ISO 8601 | When work finished |
|
||
| `estimate` | `5K`, `15K`, etc. | Predicted token usage |
|
||
| `used` | `4.2K`, `12.8K`, etc. | Actual usage |
|
||
|
||
### Step 7: Commit Bootstrap
|
||
|
||
```bash
|
||
git add docs/tasks.md docs/orchestrator-learnings.json
|
||
git commit -m "chore(orchestrator): Bootstrap tasks.md from review report
|
||
|
||
Parsed {N} findings into {M} tasks across {P} phases.
|
||
Estimated total: {X}K tokens."
|
||
git push
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 2: Execution Loop
|
||
|
||
```
|
||
1. git pull --rebase
|
||
2. Read docs/tasks.md
|
||
3. Find next task: status=not-started AND all depends_on are done
|
||
4. If no task available:
|
||
- All done? → Report success, run final retrospective, STOP
|
||
- Some blocked? → Report deadlock, STOP
|
||
5. Update tasks.md: status=in-progress, agent={identifier}, started_at={now}
|
||
6. Spawn worker agent (Task tool) with task details
|
||
7. Wait for worker completion
|
||
8. Parse worker result (JSON)
|
||
9. Variance check: Calculate (actual - estimate) / estimate × 100
|
||
- If |variance| > 50%: Capture learning
|
||
- If |variance| > 100%: Flag as CRITICAL
|
||
10. Update tasks.md: status=done/failed, completed_at={now}, used={actual}
|
||
11. Cleanup reports: Remove processed report files
|
||
12. Commit + push: git add docs/tasks.md && git commit && git push
|
||
13. If phase verification task: Run phase retrospective
|
||
14. Check context usage
|
||
15. If >= 55%: Output COMPACTION REQUIRED checkpoint, STOP, wait for user
|
||
16. If < 55%: Go to step 1
|
||
17. After user runs /compact and says "continue": Go to step 1
|
||
```
|
||
|
||
---
|
||
|
||
## Worker Prompt Template
|
||
|
||
````markdown
|
||
## Task Assignment: {id}
|
||
|
||
**Description:** {description}
|
||
**Repository:** apps/{repo}
|
||
**Branch:** {branch}
|
||
|
||
**Reference:** See `docs/reports/` for detailed finding description. Search for the finding ID.
|
||
|
||
## Workflow
|
||
|
||
1. Checkout branch: `git checkout {branch} || git checkout -b {branch} develop && git pull`
|
||
2. Read the finding details from the report
|
||
3. Implement the fix following existing code patterns
|
||
4. Run quality gates (ALL must pass):
|
||
```bash
|
||
pnpm lint && pnpm typecheck && pnpm test
|
||
```
|
||
5. If gates fail: Fix and retry. Do NOT report success with failures.
|
||
6. Commit: `git commit -m "fix({finding_id}): brief description"`
|
||
7. Push: `git push origin {branch}`
|
||
8. Report result as JSON (see format below)
|
||
|
||
## Result Format (MANDATORY)
|
||
|
||
```json
|
||
{
|
||
"task_id": "{id}",
|
||
"status": "success|failed",
|
||
"used": "5.2K",
|
||
"commit_sha": "abc123",
|
||
"notes": "Brief summary of what was done"
|
||
}
|
||
```
|
||
|
||
## Rules
|
||
|
||
- DO NOT modify docs/tasks.md
|
||
- DO NOT claim other tasks
|
||
- Complete this single task, report results, done
|
||
````
|
||
|
||
---
|
||
|
||
## Compaction Protocol
|
||
|
||
**Threshold:** 55-60% context usage
|
||
|
||
**CRITICAL:** Agents CANNOT trigger compaction. Only the user typing `/compact` works.
|
||
|
||
- ❌ "compact and continue" does NOT work (agent outputs summary but context is NOT compressed)
|
||
- ❌ Agent cannot invoke `/compact` programmatically
|
||
- ✅ User must type `/compact` directly in the CLI
|
||
|
||
**When approaching threshold (55-60%):**
|
||
|
||
1. Complete current task
|
||
2. Persist all state:
|
||
- Update docs/tasks.md with all progress
|
||
- Update docs/orchestrator-learnings.json with variances
|
||
- Commit and push both files
|
||
3. Output checkpoint using this EXACT format:
|
||
|
||
```
|
||
---
|
||
⚠️ COMPACTION REQUIRED
|
||
|
||
Context: {X}% — Cannot continue without compaction.
|
||
|
||
Progress: {completed}/{total} tasks ({percentage}%)
|
||
Next task: {task_id}
|
||
|
||
State persisted to:
|
||
- docs/tasks.md ✓
|
||
- docs/orchestrator-learnings.json ✓
|
||
|
||
ACTION REQUIRED:
|
||
1. Type `/compact` in the CLI (not in chat)
|
||
2. After compaction completes, say "continue"
|
||
|
||
I will resume with {task_id} after compaction.
|
||
---
|
||
```
|
||
|
||
4. **STOP COMPLETELY** — do not continue working
|
||
5. Wait for user to run `/compact` and say "continue"
|
||
6. Resume from next task
|
||
|
||
**Rules:**
|
||
|
||
- Do NOT output a summary and keep working
|
||
- Do NOT claim you can compact yourself
|
||
- Do NOT continue past 60% — the checkpoint is mandatory
|
||
- STOP means STOP — wait for user action
|
||
|
||
---
|
||
|
||
## Learning & Retrospective
|
||
|
||
### Variance Thresholds
|
||
|
||
| Variance | Action |
|
||
| -------- | ------------------------------------------------------ |
|
||
| 0-30% | Log only (acceptable) |
|
||
| 30-50% | Flag for review |
|
||
| 50-100% | Capture learning to `docs/orchestrator-learnings.json` |
|
||
| >100% | CRITICAL — review task classification |
|
||
|
||
### Task Type Classification
|
||
|
||
| Type | Keywords | Base Estimate |
|
||
| ------------ | -------------------------------------- | ---------------- |
|
||
| STYLE_FIX | "formatting", "prettier", "lint" | 3-5K |
|
||
| BULK_CLEANUP | "unused", "warnings", "~N files" | file_count × 550 |
|
||
| GUARD_ADD | "add guard", "decorator", "validation" | 5-8K |
|
||
| SECURITY_FIX | "sanitize", "injection", "XSS" | 8-12K × 2.5 |
|
||
| AUTH_ADD | "authentication", "auth" | 15-25K |
|
||
| REFACTOR | "refactor", "replace", "migrate" | 10-15K |
|
||
| TEST_ADD | "add tests", "coverage" | 15-25K |
|
||
|
||
---
|
||
|
||
## Report Cleanup
|
||
|
||
QA automation generates report files in `docs/reports/qa-automation/pending/`. Clean up after processing.
|
||
|
||
| Event | Action |
|
||
| ------------------ | --------------------------------------- |
|
||
| Task success | Delete matching reports from `pending/` |
|
||
| Task failed | Move reports to `escalated/` |
|
||
| Phase verification | Clean up all `pending/` reports |
|
||
| Milestone complete | Archive or delete `escalated/` |
|
||
|
||
---
|
||
|
||
## Stopping Criteria
|
||
|
||
**ONLY stop if:**
|
||
|
||
1. All tasks in docs/tasks.md are `done`
|
||
2. Critical blocker preventing progress (document and alert)
|
||
3. Context usage >= 55% — output COMPACTION REQUIRED checkpoint and wait
|
||
4. Absolute context limit reached AND cannot compact further
|
||
|
||
**DO NOT stop to ask "should I continue?"** — the answer is always YES.
|
||
**DO stop at 55-60%** — output the compaction checkpoint and wait for user to run `/compact`.
|
||
|
||
---
|
||
|
||
## Kickstart Message Format
|
||
|
||
```markdown
|
||
## Mission
|
||
|
||
Remediate findings from the codebase review.
|
||
|
||
## Setup
|
||
|
||
- Project: /home/localadmin/src/mosaic-stack
|
||
- Review: docs/reports/{report-name}/
|
||
- Quality gates: pnpm lint && pnpm typecheck && pnpm test
|
||
- Milestone: {milestone-name}
|
||
- Task prefix: MS
|
||
|
||
## Protocol
|
||
|
||
Read docs/claude/orchestrator.md for full instructions.
|
||
|
||
## Start
|
||
|
||
Bootstrap from the review report, then execute until complete.
|
||
```
|
||
|
||
---
|
||
|
||
## Quick Reference
|
||
|
||
| Phase | Action |
|
||
| --------- | ----------------------------------------------------------------------- |
|
||
| Bootstrap | Parse reports → Categorize → Estimate → Create issues → Create tasks.md |
|
||
| Execute | Loop: claim → spawn worker → update → commit |
|
||
| Compact | At 60%: summarize, clear history, continue |
|
||
| Stop | Queue empty, blocker, or context limit |
|
||
|
||
**Orchestrator owns tasks.md. Workers execute and report. Single writer eliminates conflicts.**
|