Collaboration record (conductor loop, docs/plans/CONDUCTOR.md): - round 1 (worker session worker-1, 2m28s): retry implemented per spec - conductor live test exposed spec gap: direct invocation lacked launcher env exports - round 2 (same worker session, 59s): spawnEnv made self-sufficient, but used PI_* where compose interpolates MOSAIC_* - conductor hotfix: 3-line rename to MOSAIC_PROVIDER/MOSAIC_MODEL/ MOSAIC_DATA_ROOT Final: node scripts/mosaic-task.mjs retry <runId> re-executes a run's task snapshot as a new run; live retry replied REMEMBERED; all suites green (24/32/14 + verify). Known limitation: retrying a run whose task used a RELATIVE mission path resolves it against the temp dir; lineage tracking deferred. Closes #25, closes #26, closes #27
This commit is contained in:
@@ -275,4 +275,32 @@ Adapter seam verified; harness boundary is now additive by construction. `main`
|
||||
|
||||
M5 tagged `workspace-capabilities-v1`, M6 tagged `sessions-v1`, M7 tagged `operator-ergonomics-v1`; release 0.0.6 active. Tracker: docs/plans/2026-09-03_autonomous-run.md.
|
||||
|
||||
---
|
||||
|
||||
## Phase 12: Conductor loop — self-orchestration (M8)
|
||||
|
||||
### Entry 12.1 — before
|
||||
|
||||
- Timestamp: 2026-09-03
|
||||
- Intended action: Stand up the poor-man orchestration loop per docs/plans/CONDUCTOR.md: conductor (host, holds git) mirrors the repo into a worker workspace, dispatches a headless pi worker (session worker-1, tools read/write/edit/bash) to implement retry <runId>, reviews the diff, integrates, verifies (Gitea #25, #26, #27).
|
||||
- Reason: The owner asked for circular task processing with agent workers; the stack now has every primitive needed — this proves it on the stack itself.
|
||||
- Expected result: worker-authored retry merged with suites green and a live retry verified.
|
||||
|
||||
### Entry 12.2 — after
|
||||
|
||||
- Timestamp: 2026-09-03
|
||||
- Commands run: repo mirror clone; worker dispatch (tasks/worker-retry.json); diff review; apply; live retry; refinement dispatch (tasks/worker-retry-refine.json); second review; reverse+reapply combined patch; conductor interpolation fix; live retry; full sweep.
|
||||
- Observed result:
|
||||
- Worker round 1: implemented retry correctly per spec in 2m28s; diff reviewed clean.
|
||||
- Live retry exposed a spec gap (direct invocation lacks launcher env exports).
|
||||
- Worker round 2 (same session, 59s): made spawnEnv self-sufficient, but used PI_* names where compose interpolates MOSAIC_*.
|
||||
- Conductor hotfix: 3-line rename to MOSAIC_PROVIDER/MOSAIC_MODEL/MOSAIC_DATA_ROOT (too trivial for a worker round).
|
||||
- Final: live retry succeeded (replied REMEMBERED, new run recorded); all suites green.
|
||||
- Failure or correction: three rounds total — one spec gap (conductor), one naming mismatch (worker), one trivial rename (conductor). Each was caught by mechanical verification (run record stderr), never by hope.
|
||||
- Attribution: feature authored by headless pi worker (glm-5.3-flash) in sessions worker-1; conductor reviewed, integrated, and hotfixed.
|
||||
|
||||
## Result (M8)
|
||||
|
||||
Conductor loop proven end-to-end on the stack itself. `main` merged with M8; release 0.0.6 remains active (retry is host-side only, no image change).
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Conductor protocol — poor-man orchestration loop
|
||||
|
||||
How the stack orchestrates headless pi workers to do work on itself.
|
||||
|
||||
## Roles
|
||||
|
||||
| Role | Runs where | Powers | Never has |
|
||||
|---|---|---|---|
|
||||
| **Conductor** | host (assistant or owner) | git (clone/commit/push), task dispatch, review, verification suites, Gitea | nothing new |
|
||||
| **Worker** | container (headless pi via `scripts/run-task.sh`) | read/write/edit/bash inside its workspace; persistent session on request | git credentials, docker socket, host filesystem |
|
||||
|
||||
## The loop
|
||||
|
||||
1. **Decompose**: conductor turns a goal into worker tasks small enough to
|
||||
specify completely in one prompt (file paths, acceptance criteria, style
|
||||
constraints, verification the worker can run itself, e.g. `node --check`).
|
||||
2. **Mirror**: conductor maintains the repo clone at
|
||||
`<dataRoot>/workspaces/stack-repo` (host-side git; workers see it read-write
|
||||
through their workspace mount).
|
||||
3. **Dispatch**: `scripts/run-task.sh run <worker-task.json>` — worker edits the
|
||||
clone. Session name `worker-<n>` keeps continuity across refinement rounds.
|
||||
4. **Extract**: `git -C <workspace> diff > patch` — the worker's entire output
|
||||
is a reviewable diff. Run record (result.json, stderr.txt) is the receipt.
|
||||
5. **Review**: conductor reads the diff line by line. Bad output → refine the
|
||||
prompt, re-dispatch (same session: "your patch had these problems…").
|
||||
6. **Integrate**: conductor applies the patch to the real repo, runs the full
|
||||
suites, commits and pushes. Suites failing → revert apply, back to step 5.
|
||||
7. **Record**: update CURRENT.md, BUILD-LOG, close the Gitea issue.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Workers never receive credentials; they never run git; they never leave the
|
||||
workspace (container is the boundary; tools allowlist is the gate).
|
||||
- Every worker diff is reviewed by the conductor before integration. No
|
||||
auto-apply. (Auto-apply would be a capability-policy decision for later.)
|
||||
- Verification is mechanical: suites + `node --check` / `bash -n` gates.
|
||||
- Recursive decomposition = "fail → smaller task", never "hope."
|
||||
|
||||
## Worker task template
|
||||
|
||||
```json
|
||||
{
|
||||
"taskVersion": 1,
|
||||
"id": "t-worker-<name>",
|
||||
"prompt": "<full spec: goal, files, constraints, acceptance, self-checks>",
|
||||
"workspace": "stack-repo",
|
||||
"capabilities": { "tools": ["read", "write", "edit", "bash"] },
|
||||
"session": "worker-1",
|
||||
"timeoutSeconds": 600
|
||||
}
|
||||
```
|
||||
@@ -11,9 +11,9 @@ Gitea issue: (none queued — awaiting owner direction)
|
||||
|
||||
## Queue (ordered, not started)
|
||||
|
||||
1. Owner review of M5–M7 (workspaces, capabilities, sessions) — hands-on
|
||||
2. M8 candidate: mission-level capability policy (defaults per mission)
|
||||
3. M9 candidate: run-record retention + `retry <runId>` convenience
|
||||
1. Owner review of M5–M8 (workspaces, capabilities, sessions, conductor loop)
|
||||
2. M8 follow-up: retry lineage (retriedFrom field) + relative-mission-path retry limitation
|
||||
3. M9 candidates: mission-level capability policy; run-record retention; decision on auto-apply policy for worker patches
|
||||
|
||||
## Rules
|
||||
|
||||
@@ -30,3 +30,4 @@ Gitea issue: (none queued — awaiting owner direction)
|
||||
- 2026-09-03 — M5 workspaces + capability envelope (#20, #21) — merged, tests green
|
||||
- 2026-09-03 — M6 named sessions + resume (#22, #23) — merged, teach/recall verified
|
||||
- 2026-09-03 — M7 run inspection + release 0.0.6 (#24) — merged, activated
|
||||
- 2026-09-03 — M8 conductor loop + worker-built retry (#25, #26, #27) — merged; worker authored retry in 2 refinement rounds, conductor fixed a 3-line interpolation rename; live retry verified
|
||||
|
||||
+47
-1
@@ -28,6 +28,7 @@
|
||||
*/
|
||||
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import process from "node:process";
|
||||
import { randomBytes } from "node:crypto";
|
||||
@@ -263,6 +264,15 @@ function runTask(taskFile) {
|
||||
// CONTAINER path (dataRoot maps to /var/lib/mosaic in the image).
|
||||
const spawnEnv = { ...process.env };
|
||||
spawnEnv.MOSAIC_ADAPTER = resolved.execution.adapter;
|
||||
// Self-sufficient env: direct invocation (e.g. `retry`) skips the shell
|
||||
// launcher exports, so derive them from the resolved config and release.
|
||||
// (Names here are the compose interpolation consumers, not PI_*.)
|
||||
spawnEnv.MOSAIC_PROVIDER = resolved.execution.provider;
|
||||
spawnEnv.MOSAIC_MODEL = resolved.execution.model;
|
||||
spawnEnv.MOSAIC_DATA_ROOT = resolved.dataRoot;
|
||||
const release = fs.readFileSync(path.join(PROJECT_ROOT, "RELEASE"), "utf8").trim();
|
||||
const piVersion = JSON.parse(fs.readFileSync(path.join(PROJECT_ROOT, "package.json"), "utf8")).dependencies["@earendil-works/pi-coding-agent"];
|
||||
spawnEnv.MOSAIC_IMAGE_TAG = `mosaic-poc-agent:${piVersion}-r${release}`;
|
||||
if (task.missionSnapshot) {
|
||||
const relative = path.relative(resolved.dataRoot, runDir);
|
||||
if (relative.startsWith("..") || path.isAbsolute(relative)) {
|
||||
@@ -428,6 +438,38 @@ function showRun(runId) {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
function retryRun(runId) {
|
||||
if (!/^r-[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/.test(runId)) {
|
||||
fail(4, `invalid run id: ${JSON.stringify(runId)} (expected r-<id>)`);
|
||||
}
|
||||
const resolved = loadConfig();
|
||||
const dir = path.join(runsRoot(resolved), runId);
|
||||
if (!fs.existsSync(dir)) {
|
||||
fail(4, `run not found: ${runId}`);
|
||||
}
|
||||
|
||||
let snapshot;
|
||||
try {
|
||||
snapshot = fs.readFileSync(path.join(dir, "task.json"), "utf8");
|
||||
} catch {
|
||||
fail(4, `run task snapshot unreadable: ${runId}`);
|
||||
}
|
||||
|
||||
// A retry is a brand-new run: replay the recorded task snapshot through
|
||||
// the ordinary run path; existing run records stay untouched.
|
||||
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "mosaic-retry-"));
|
||||
const tempTaskFile = path.join(tempDir, "task.json");
|
||||
fs.writeFileSync(tempTaskFile, snapshot);
|
||||
process.on("exit", () => {
|
||||
try {
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
// Best-effort cleanup only.
|
||||
}
|
||||
});
|
||||
runTask(tempTaskFile);
|
||||
}
|
||||
|
||||
const operation = process.argv[2];
|
||||
const target = process.argv[3];
|
||||
|
||||
@@ -451,6 +493,10 @@ switch (operation) {
|
||||
case "list":
|
||||
listRuns();
|
||||
process.exit(0);
|
||||
case "retry":
|
||||
if (!target) fail(4, "usage: mosaic-task.mjs retry <runId>");
|
||||
retryRun(target);
|
||||
break;
|
||||
default:
|
||||
fail(4, `unknown operation: ${JSON.stringify(operation ?? "")} (expected validate | run | show | list)`);
|
||||
fail(4, `unknown operation: ${JSON.stringify(operation ?? "")} (expected validate | run | show | list | retry)`);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"taskVersion": 1,
|
||||
"id": "t-worker-retry-refine",
|
||||
"prompt": "Follow-up on your last change to scripts/mosaic-task.mjs (the retry operation). Conductor review found one defect: runTask builds spawnEnv from process.env, so a direct 'node scripts/mosaic-task.mjs retry <runId>' invocation lacks the launcher-exported variables and compose fails with: required variable MOSAIC_IMAGE_TAG is missing.\n\nFIX (in runTask, not retryRun): make spawnEnv self-sufficient —\n1. spawnEnv.PI_PROVIDER = resolved.execution.provider; spawnEnv.PI_MODEL = resolved.execution.model (from the already-loaded resolved config).\n2. spawnEnv.MOSAIC_IMAGE_TAG = 'mosaic-poc-agent:' + <pinned pi version> + '-r' + <RELEASE file content trimmed>, reading RELEASE and package.json the same way scripts/common.sh load_release does.\nKeep the change minimal; do not touch other files; keep code style.\n\nSELF-CHECK: node --check scripts/mosaic-task.mjs must pass.\nWhen finished, reply with exactly: REFINE_DONE",
|
||||
"workspace": "stack-repo",
|
||||
"capabilities": { "tools": ["read", "write", "edit", "bash"] },
|
||||
"session": "worker-1",
|
||||
"timeoutSeconds": 600
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"taskVersion": 1,
|
||||
"id": "t-worker-retry",
|
||||
"prompt": "You are working alone inside a git repository checkout (your current working directory). Implement a new feature in scripts/mosaic-task.mjs.\n\nFEATURE: add a 'retry <runId>' operation alongside the existing 'validate | run | show | list' operations.\n\nBEHAVIOR: 'retry <runId>' re-executes a previously recorded run. Steps: (1) validate the runId argument with the same pattern showRun uses; (2) load the config exactly like showRun does; (3) locate the run directory under the runs root; if it does not exist, fail with exit code 4 and the message 'run not found: <runId>'; (4) read that run directory's task.json snapshot; if unreadable, fail with exit code 4; (5) write the snapshot to a temp file and execute it through the EXISTING runTask(path) function so a brand-new run is created — do not duplicate any execution logic; (6) exit with whatever exit code the new run produced.\n\nCONSTRAINTS: reuse runTask; do not refactor unrelated code; do not touch any file other than scripts/mosaic-task.mjs; keep the existing code style; update the default usage error message to include retry.\n\nSELF-CHECK (run it yourself with the bash tool): node --check scripts/mosaic-task.mjs must pass.\n\nWhen finished, reply with exactly: RETRY_DONE",
|
||||
"workspace": "stack-repo",
|
||||
"capabilities": { "tools": ["read", "write", "edit", "bash"] },
|
||||
"session": "worker-1",
|
||||
"timeoutSeconds": 600
|
||||
}
|
||||
Reference in New Issue
Block a user