Merge M9: mission-level capability policy

Closes #30

BUILD-LOG.md

@@ -275,4 +275,32 @@ Adapter seam verified; harness boundary is now additive by construction. `main`
 
 M5 tagged `workspace-capabilities-v1`, M6 tagged `sessions-v1`, M7 tagged `operator-ergonomics-v1`; release 0.0.6 active. Tracker: docs/plans/2026-09-03_autonomous-run.md.
 
+---
+
+## Phase 12: Conductor loop — self-orchestration (M8)
+
+### Entry 12.1 — before
+
+- Timestamp: 2026-09-03
+- Intended action: Stand up the poor-man orchestration loop per docs/plans/CONDUCTOR.md: conductor (host, holds git) mirrors the repo into a worker workspace, dispatches a headless pi worker (session worker-1, tools read/write/edit/bash) to implement retry <runId>, reviews the diff, integrates, verifies (Gitea #25, #26, #27).
+- Reason: The owner asked for circular task processing with agent workers; the stack now has every primitive needed — this proves it on the stack itself.
+- Expected result: worker-authored retry merged with suites green and a live retry verified.
+
+### Entry 12.2 — after
+
+- Timestamp: 2026-09-03
+- Commands run: repo mirror clone; worker dispatch (tasks/worker-retry.json); diff review; apply; live retry; refinement dispatch (tasks/worker-retry-refine.json); second review; reverse+reapply combined patch; conductor interpolation fix; live retry; full sweep.
+- Observed result:
+  - Worker round 1: implemented retry correctly per spec in 2m28s; diff reviewed clean.
+  - Live retry exposed a spec gap (direct invocation lacks launcher env exports).
+  - Worker round 2 (same session, 59s): made spawnEnv self-sufficient, but used PI_* names where compose interpolates MOSAIC_*.
+  - Conductor hotfix: 3-line rename to MOSAIC_PROVIDER/MOSAIC_MODEL/MOSAIC_DATA_ROOT (too trivial for a worker round).
+  - Final: live retry succeeded (replied REMEMBERED, new run recorded); all suites green.
+- Failure or correction: three rounds total — one spec gap (conductor), one naming mismatch (worker), one trivial rename (conductor). Each was caught by mechanical verification (run record stderr), never by hope.
+- Attribution: feature authored by headless pi worker (glm-5.3-flash) in sessions worker-1; conductor reviewed, integrated, and hotfixed.
+
+## Result (M8)
+
+Conductor loop proven end-to-end on the stack itself. `main` merged with M8; release 0.0.6 remains active (retry is host-side only, no image change).
+
 

docs/plans/CONDUCTOR.md

@@ -0,0 +1,51 @@
+# Conductor protocol — poor-man orchestration loop
+
+How the stack orchestrates headless pi workers to do work on itself.
+
+## Roles
+
+| Role | Runs where | Powers | Never has |
+|---|---|---|---|
+| **Conductor** | host (assistant or owner) | git (clone/commit/push), task dispatch, review, verification suites, Gitea | nothing new |
+| **Worker** | container (headless pi via `scripts/run-task.sh`) | read/write/edit/bash inside its workspace; persistent session on request | git credentials, docker socket, host filesystem |
+
+## The loop
+
+1. **Decompose**: conductor turns a goal into worker tasks small enough to
+   specify completely in one prompt (file paths, acceptance criteria, style
+   constraints, verification the worker can run itself, e.g. `node --check`).
+2. **Mirror**: conductor maintains the repo clone at
+   `<dataRoot>/workspaces/stack-repo` (host-side git; workers see it read-write
+   through their workspace mount).
+3. **Dispatch**: `scripts/run-task.sh run <worker-task.json>` — worker edits the
+   clone. Session name `worker-<n>` keeps continuity across refinement rounds.
+4. **Extract**: `git -C <workspace> diff > patch` — the worker's entire output
+   is a reviewable diff. Run record (result.json, stderr.txt) is the receipt.
+5. **Review**: conductor reads the diff line by line. Bad output → refine the
+   prompt, re-dispatch (same session: "your patch had these problems…").
+6. **Integrate**: conductor applies the patch to the real repo, runs the full
+   suites, commits and pushes. Suites failing → revert apply, back to step 5.
+7. **Record**: update CURRENT.md, BUILD-LOG, close the Gitea issue.
+
+## Guardrails
+
+- Workers never receive credentials; they never run git; they never leave the
+  workspace (container is the boundary; tools allowlist is the gate).
+- Every worker diff is reviewed by the conductor before integration. No
+  auto-apply. (Auto-apply would be a capability-policy decision for later.)
+- Verification is mechanical: suites + `node --check` / `bash -n` gates.
+- Recursive decomposition = "fail → smaller task", never "hope."
+
+## Worker task template
+
+```json
+{
+  "taskVersion": 1,
+  "id": "t-worker-<name>",
+  "prompt": "<full spec: goal, files, constraints, acceptance, self-checks>",
+  "workspace": "stack-repo",
+  "capabilities": { "tools": ["read", "write", "edit", "bash"] },
+  "session": "worker-1",
+  "timeoutSeconds": 600
+}
+```

docs/plans/CURRENT.md

@@ -0,0 +1,36 @@
+# CURRENT — single source of "what happens next"
+
+This file always names exactly one next action. Any "continue" / "next" /
+"proceed" message means: execute the action below, fully (implement → test →
+verify against its acceptance criteria → commit → push → close the issue →
+update this file to the next action). No ambiguity, no re-planning.
+
+## Next action
+
+Owner decision on M9: mission-level capability policy (design sketch: missions may declare default tool sets; tasks inherit unless overridden; conductor validates the merge). Say "next" to proceed or name a different target.
+
+## Queue (ordered, not started)
+
+1. M9: mission-level capability policy
+2. Run-record retention/pruning policy
+3. Second real adapter (parked — owner focused on Pi)
+4. Auto-apply policy for worker patches (deferred until capability policy exists)
+
+## Rules
+
+- One action in flight. Update this file at the END of every action.
+- Blocked? Move the item to "Blocked" below with the reason and stop.
+- Completed actions move to the log at the bottom (date + issue + result).
+
+## Blocked
+
+(none)
+
+## Completed log
+
+- 2026-09-03 — M5 workspaces + capability envelope (#20, #21) — merged, tests green
+- 2026-09-03 — M6 named sessions + resume (#22, #23) — merged, teach/recall verified
+- 2026-09-03 — M7 run inspection + release 0.0.6 (#24) — merged, activated
+- 2026-09-03 — M8 conductor loop + worker-built retry (#25, #26, #27) — merged; worker authored retry in 2 refinement rounds, conductor fixed a 3-line interpolation rename; live retry verified
+- 2026-09-03 — retry lineage + relative mission resolution (#28) — merged, 36/24/14 suites + verify green
+- 2026-09-03 — M8 conductor loop + worker-built retry (#25, #26, #27) — merged; worker authored retry in 2 refinement rounds, conductor fixed a 3-line interpolation rename; live retry verified

scripts/mosaic-task.mjs

@@ -28,6 +28,7 @@
  */
 
 import fs from "node:fs";
+import os from "node:os";
 import path from "node:path";
 import process from "node:process";
 import { randomBytes } from "node:crypto";
@@ -83,7 +84,7 @@ function validateId(value, what) {
 
 function validateMission(document, file) {
   if (!isPlainObject(document)) fail(2, "mission must be a JSON object");
-  rejectUnknownKeys(document, ["missionVersion", "id", "objective", "directives"], "mission");
+  rejectUnknownKeys(document, ["missionVersion", "id", "objective", "directives", "capabilities"], "mission");
   if (document.missionVersion !== 1) {
     fail(2, `unsupported missionVersion: ${JSON.stringify(document.missionVersion)} (supported: 1)`);
   }
@@ -101,7 +102,29 @@ function validateMission(document, file) {
       return d;
     });
   }
-  return { missionVersion: document.missionVersion, id: document.id, objective: document.objective, directives };
+
+  // Governing capability constraints (M9): same validation as task
+  // capabilities; semantically these BOUND tasks (least-privilege
+  // intersection at run time), never grant beyond them.
+  let capabilities = null;
+  if (document.capabilities !== undefined && document.capabilities !== null) {
+    if (!isPlainObject(document.capabilities)) fail(2, 'mission "capabilities" must be a JSON object');
+    rejectUnknownKeys(document.capabilities, ["tools"], 'mission "capabilities"');
+    if (!Array.isArray(document.capabilities.tools) || document.capabilities.tools.length === 0) {
+      fail(2, 'mission "capabilities.tools" must be a non-empty array of tool names');
+    }
+    const seen = new Set();
+    for (const tool of document.capabilities.tools) {
+      if (!SUPPORTED_TOOLS.includes(tool)) {
+        fail(2, `unsupported mission tool: ${JSON.stringify(tool)} (supported: ${SUPPORTED_TOOLS.join(", ")})`);
+      }
+      if (seen.has(tool)) fail(2, `duplicate tool in mission capabilities.tools: ${tool}`);
+      seen.add(tool);
+    }
+    capabilities = { tools: [...seen] };
+  }
+
+  return { missionVersion: document.missionVersion, id: document.id, objective: document.objective, directives, capabilities };
 }
 
 function validateTask(document, file) {
@@ -233,7 +256,7 @@ function writeOnce(file, content) {
   }
 }
 
-function runTask(taskFile) {
+function runTask(taskFile, options = {}) {
   const resolved = JSON.parse(
     spawnSync(process.execPath, [path.join(PROJECT_ROOT, "scripts", "mosaic-config.mjs"), "validate"], {
       cwd: PROJECT_ROOT,
@@ -263,6 +286,15 @@ function runTask(taskFile) {
   // CONTAINER path (dataRoot maps to /var/lib/mosaic in the image).
   const spawnEnv = { ...process.env };
   spawnEnv.MOSAIC_ADAPTER = resolved.execution.adapter;
+  // Self-sufficient env: direct invocation (e.g. `retry`) skips the shell
+  // launcher exports, so derive them from the resolved config and release.
+  // (Names here are the compose interpolation consumers, not PI_*.)
+  spawnEnv.MOSAIC_PROVIDER = resolved.execution.provider;
+  spawnEnv.MOSAIC_MODEL = resolved.execution.model;
+  spawnEnv.MOSAIC_DATA_ROOT = resolved.dataRoot;
+  const release = fs.readFileSync(path.join(PROJECT_ROOT, "RELEASE"), "utf8").trim();
+  const piVersion = JSON.parse(fs.readFileSync(path.join(PROJECT_ROOT, "package.json"), "utf8")).dependencies["@earendil-works/pi-coding-agent"];
+  spawnEnv.MOSAIC_IMAGE_TAG = `mosaic-poc-agent:${piVersion}-r${release}`;
   if (task.missionSnapshot) {
     const relative = path.relative(resolved.dataRoot, runDir);
     if (relative.startsWith("..") || path.isAbsolute(relative)) {
@@ -281,7 +313,24 @@ function runTask(taskFile) {
     workspaceContainerPath = `/var/lib/mosaic/workspaces/${task.workspace}`;
   }
   if (workspaceContainerPath) spawnEnv.MOSAIC_WORKSPACE = workspaceContainerPath;
-  spawnEnv.MOSAIC_TOOLS = task.tools ? task.tools.join(",") : "";
+
+  // Capability policy (M9): least-privilege intersection. A task may narrow
+  // a mission's tool grant, never widen it. Empty intersection = tool-free.
+  let effectiveTools = task.tools;
+  let policyNote = null;
+  if (task.missionSnapshot?.capabilities) {
+    const missionTools = task.missionSnapshot.capabilities.tools;
+    if (effectiveTools) {
+      effectiveTools = effectiveTools.filter((t) => missionTools.includes(t));
+      if (effectiveTools.length === 0) {
+        policyNote = `capability policy: mission ${task.missionSnapshot.id} and task request no tools in common -> tool-free run`;
+      }
+    } else {
+      effectiveTools = [...missionTools];
+    }
+  }
+  if (policyNote) process.stderr.write(`mosaic-task: ${policyNote}\n`);
+  spawnEnv.MOSAIC_TOOLS = effectiveTools ? effectiveTools.join(",") : "";
 
   // Session (M6): persistent named session dir, passed as container path.
   if (task.session) {
@@ -335,8 +384,9 @@ function runTask(taskFile) {
     request: task.prompt,
     response,
     expectedExact: expected,
+    ...(options.retriedFrom ? { retriedFrom: options.retriedFrom } : {}),
     workspace: task.workspace,
-    tools: task.tools,
+    tools: effectiveTools,
     session: task.session,
     exitCode: proc.status,
     signal: proc.signal ?? null,
@@ -415,6 +465,7 @@ function showRun(runId) {
         `request:   ${JSON.stringify(result.request)}`,
         `response:  ${JSON.stringify(result.response)}`,
         result.expectedExact !== null && result.expectedExact !== undefined ? `expected:  ${JSON.stringify(result.expectedExact)}` : null,
+        result.retriedFrom ? `retriedFrom: ${result.retriedFrom}` : null,
         `timing:    ${result.startedAt} -> ${result.finishedAt} (${result.durationMs} ms)`,
         `exit:      ${result.exitCode}${result.signal ? ` signal=${result.signal}` : ""}`,
       ].filter((line) => line !== null).join("\n") + "\n",
@@ -428,6 +479,56 @@ function showRun(runId) {
   process.exit(0);
 }
 
+function retryRun(runId) {
+  if (!/^r-[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/.test(runId)) {
+    fail(4, `invalid run id: ${JSON.stringify(runId)} (expected r-<id>)`);
+  }
+  const resolved = loadConfig();
+  const dir = path.join(runsRoot(resolved), runId);
+  if (!fs.existsSync(dir)) {
+    fail(4, `run not found: ${runId}`);
+  }
+
+  let snapshot;
+  try {
+    snapshot = fs.readFileSync(path.join(dir, "task.json"), "utf8");
+  } catch {
+    fail(4, `run task snapshot unreadable: ${runId}`);
+  }
+
+  // Relative mission paths in a snapshot resolve against the ORIGINAL task
+  // location, which no longer exists here — rewrite them to the run's own
+  // recorded mission.json so retries stay faithful.
+  let snapshotDoc;
+  try {
+    snapshotDoc = JSON.parse(snapshot);
+  } catch {
+    fail(4, `run task snapshot is not valid JSON: ${runId}`);
+  }
+  if (snapshotDoc.mission && !path.isAbsolute(snapshotDoc.mission)) {
+    const recordedMission = path.join(dir, "mission.json");
+    if (!fs.existsSync(recordedMission)) {
+      fail(4, `cannot retry ${runId}: relative mission path but no mission.json snapshot in run dir`);
+    }
+    snapshotDoc.mission = recordedMission;
+    snapshot = `${JSON.stringify(snapshotDoc, null, 2)}\n`;
+  }
+
+  // A retry is a brand-new run: replay the recorded task snapshot through
+  // the ordinary run path; existing run records stay untouched.
+  const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "mosaic-retry-"));
+  const tempTaskFile = path.join(tempDir, "task.json");
+  fs.writeFileSync(tempTaskFile, snapshot);
+  process.on("exit", () => {
+    try {
+      fs.rmSync(tempDir, { recursive: true, force: true });
+    } catch {
+      // Best-effort cleanup only.
+    }
+  });
+  runTask(tempTaskFile, { retriedFrom: runId });
+}
+
 const operation = process.argv[2];
 const target = process.argv[3];
 
@@ -451,6 +552,10 @@ switch (operation) {
   case "list":
     listRuns();
     process.exit(0);
+  case "retry":
+    if (!target) fail(4, "usage: mosaic-task.mjs retry <runId>");
+    retryRun(target);
+    break;
   default:
-    fail(4, `unknown operation: ${JSON.stringify(operation ?? "")} (expected validate | run | show | list)`);
+    fail(4, `unknown operation: ${JSON.stringify(operation ?? "")} (expected validate | run | show | list | retry)`);
 }

scripts/test-task.sh

@@ -142,6 +142,50 @@ EOF
         && grep -q 'Seam directive A.' "$SANDBOX/data/system-prompt.md" \
         && check "mission section injected into generated prompt" 0 \
         || check "mission section injected into generated prompt" 1
+
+    # retry lineage + relative mission path resolution
+    expect_exit "retry of mission run succeeds" 0 -- \
+        env MOSAIC_CONFIG="$SANDBOX/mock-adapters.json" MOSAIC_MOCK_RESPONSE=MOCKED \
+        node scripts/mosaic-task.mjs retry "$(ls -dt "$SANDBOX/data/runs"/r-* | head -1 | xargs basename)"
+    RT="$(ls -dt "$SANDBOX/data/runs"/r-* | head -1 | xargs basename)"
+    ORIG="$(ls -dt "$SANDBOX/data/runs"/r-* | sed -n 2p | xargs basename)"
+    node -e 'const r=JSON.parse(require("fs").readFileSync(process.argv[1],"utf8"));process.exit(r.retriedFrom===process.argv[2]?0:1)' \
+        "$SANDBOX/data/runs/$RT/result.json" "$ORIG" \
+        && check "retriedFrom lineage recorded" 0 || check "retriedFrom lineage recorded" 1
+    grep -q 'MISSION (runtime)' "$SANDBOX/data/system-prompt.md" \
+        && check "mission section present after retry (relative path resolved)" 0 \
+        || check "mission section present after retry (relative path resolved)" 1
+    expect_exit "retry of missing run exits 4" 4 -- \
+        env MOSAIC_CONFIG="$SANDBOX/mock-adapters.json" node scripts/mosaic-task.mjs retry r-missing
+
+    # capability policy (M9): least-privilege intersection
+    POL="$SANDBOX/data/workspaces"; mkdir -p "$POL"
+    pol_run() { # missionTools(ABSENT|json) taskTools(ABSENT|json) -> stderr MOSAIC_TOOLS value
+        if [ "$1" = "ABSENT" ]; then
+            printf '{"missionVersion":1,"id":"m-pol","objective":"o"}' > "$SANDBOX/pol-m.json"
+        else
+            printf '{"missionVersion":1,"id":"m-pol","objective":"o","capabilities":{"tools":[%s]}}' "$1" > "$SANDBOX/pol-m.json"
+        fi
+        if [ "$2" = "ABSENT" ]; then
+            printf '{"taskVersion":1,"id":"t-pol","prompt":"x","mission":"pol-m.json"}' > "$SANDBOX/pol-t.json"
+        else
+            printf '{"taskVersion":1,"id":"t-pol","prompt":"x","mission":"pol-m.json","capabilities":{"tools":[%s]}}' "$2" > "$SANDBOX/pol-t.json"
+        fi
+        env MOSAIC_CONFIG="$SANDBOX/mock-adapters.json" MOSAIC_MOCK_RESPONSE=MOCKED \
+            scripts/run-task.sh run "$SANDBOX/pol-t.json" >/dev/null 2>&1
+        grep -o '^MOSAIC_TOOLS=.*' "$(ls -dt "$SANDBOX/data/runs"/r-* | head -1)/stderr.txt" 2>/dev/null
+    }
+    [ "$(pol_run '"read","bash"' ABSENT)" = 'MOSAIC_TOOLS=read,bash' ] \
+        && check "policy: mission only -> mission tools" 0 || check "policy: mission only -> mission tools" 1
+    [ "$(pol_run ABSENT '"read","bash"')" = 'MOSAIC_TOOLS=read,bash' ] \
+        && check "policy: task only -> task tools" 0 || check "policy: task only -> task tools" 1
+    [ "$(pol_run '"read"' '"bash","read"')" = 'MOSAIC_TOOLS=read' ] \
+        && check "policy: both -> intersection (task narrowed)" 0 || check "policy: both -> intersection (task narrowed)" 1
+    [ "$(pol_run '"grep"' '"bash","read"')" = 'MOSAIC_TOOLS=' ] \
+        && check "policy: empty intersection -> tool-free" 0 || check "policy: empty intersection -> tool-free" 1
+    printf '{"missionVersion":1,"id":"m-pol","objective":"o","capabilities":{"tools":["sudo"]}}' > "$SANDBOX/pol-m.json"
+    expect_exit "invalid mission capabilities rejected" 2 -- \
+        env MOSAIC_CONFIG="$SANDBOX/mock-adapters.json" $TASK validate "$SANDBOX/pol-t.json"
 else
     echo "skip adapter seam cases (docker daemon unavailable)"
 fi

tasks/worker-retry-refine.json

@@ -0,0 +1,9 @@
+{
+  "taskVersion": 1,
+  "id": "t-worker-retry-refine",
+  "prompt": "Follow-up on your last change to scripts/mosaic-task.mjs (the retry operation). Conductor review found one defect: runTask builds spawnEnv from process.env, so a direct 'node scripts/mosaic-task.mjs retry <runId>' invocation lacks the launcher-exported variables and compose fails with: required variable MOSAIC_IMAGE_TAG is missing.\n\nFIX (in runTask, not retryRun): make spawnEnv self-sufficient —\n1. spawnEnv.PI_PROVIDER = resolved.execution.provider; spawnEnv.PI_MODEL = resolved.execution.model (from the already-loaded resolved config).\n2. spawnEnv.MOSAIC_IMAGE_TAG = 'mosaic-poc-agent:' + <pinned pi version> + '-r' + <RELEASE file content trimmed>, reading RELEASE and package.json the same way scripts/common.sh load_release does.\nKeep the change minimal; do not touch other files; keep code style.\n\nSELF-CHECK: node --check scripts/mosaic-task.mjs must pass.\nWhen finished, reply with exactly: REFINE_DONE",
+  "workspace": "stack-repo",
+  "capabilities": { "tools": ["read", "write", "edit", "bash"] },
+  "session": "worker-1",
+  "timeoutSeconds": 600
+}

tasks/worker-retry.json

@@ -0,0 +1,9 @@
+{
+  "taskVersion": 1,
+  "id": "t-worker-retry",
+  "prompt": "You are working alone inside a git repository checkout (your current working directory). Implement a new feature in scripts/mosaic-task.mjs.\n\nFEATURE: add a 'retry <runId>' operation alongside the existing 'validate | run | show | list' operations.\n\nBEHAVIOR: 'retry <runId>' re-executes a previously recorded run. Steps: (1) validate the runId argument with the same pattern showRun uses; (2) load the config exactly like showRun does; (3) locate the run directory under the runs root; if it does not exist, fail with exit code 4 and the message 'run not found: <runId>'; (4) read that run directory's task.json snapshot; if unreadable, fail with exit code 4; (5) write the snapshot to a temp file and execute it through the EXISTING runTask(path) function so a brand-new run is created — do not duplicate any execution logic; (6) exit with whatever exit code the new run produced.\n\nCONSTRAINTS: reuse runTask; do not refactor unrelated code; do not touch any file other than scripts/mosaic-task.mjs; keep the existing code style; update the default usage error message to include retry.\n\nSELF-CHECK (run it yourself with the bash tool): node --check scripts/mosaic-task.mjs must pass.\n\nWhen finished, reply with exactly: RETRY_DONE",
+  "workspace": "stack-repo",
+  "capabilities": { "tools": ["read", "write", "edit", "bash"] },
+  "session": "worker-1",
+  "timeoutSeconds": 600
+}