diff --git a/packages/mosaic/framework/tools/orchestrator/README.md b/packages/mosaic/framework/tools/orchestrator/README.md new file mode 100644 index 00000000..d57804d7 --- /dev/null +++ b/packages/mosaic/framework/tools/orchestrator/README.md @@ -0,0 +1,80 @@ +# orchestrator/ tools + +Helper scripts for r0 coordinator / orchestrator sessions — mission lifecycle, +session health, continuation, and board maintenance. See +`framework/guides/ORCHESTRATOR-PROTOCOL.md` for the surrounding process. + +| Script | Purpose | +|--------|---------| +| `mission-init.sh` | Initialize a new orchestration mission (manifest, scratchpad, TASKS.md). | +| `mission-status.sh` | Show the mission progress dashboard. | +| `session-run.sh` | Generate continuation context and launch the target runtime. | +| `session-resume.sh` | Crash recovery for dead orchestrator sessions. | +| `session-status.sh` | Check agent session health. | +| `continue-prompt.sh` | Generate the continuation prompt for the next session. | +| `board-roll.sh` | Keep a LIVE orchestration board under its byte cap by rolling the oldest entries to its LEDGER. | +| `smoke-test.sh` | Behavior smoke checks for the coord continue/run workflows. | +| `test-board-roll.sh` | Regression harness for `board-roll.sh`. | +| `_lib.sh` | Shared functions sourced by the above (state files, TASKS.md parsing, locks). | + +## board-roll.sh + +Coordinator boards (`MOS-ORCHESTRATION-BOARD-LIVE.md`, `MS-LEAD-BOARD-LIVE.md`) +follow a **"< 8 KB LIVE"** discipline: the LIVE board is the only file loaded on +resume, so it must stay small, and history lives in an append-only LEDGER. When a +board write would push LIVE over its cap, coordinators otherwise hand-trim and +retry every time — an observed 38 ABORT-OVER-CAP cycles in one 24 h window. +`board-roll.sh` automates that trim mechanically and reversibly: the audit trail +is moved to the LEDGER instead of being hand-deleted. + +### Contract (conservative — it never guesses what is safe to move) + +The LIVE board opts in by wrapping its aging archival ticks in an explicit roll +zone. Everything **outside** the markers (title, protocol blockquote, curated +always-current `##` sections) is pinned and never touched: + +```markdown +# MOS ORCHESTRATION BOARD — LIVE state +> protocol blockquote … (pinned) + +## 🟦 Curated always-current section (pinned) +… + + +### 2026-07-22 (mid²²) — newest tick, stays longest +… +### 2026-07-20 (dawn) — oldest tick, rolled first +… + +``` + +Inside the zone, entries are delimited by a heading marker (default `### `) and +are assumed **newest-first (top) → oldest-last (bottom)**. `board-roll.sh` moves +whole oldest (bottom-most) entry blocks out of the zone and appends them verbatim +to the LEDGER, one at a time, until LIVE is back under the cap or the zone is +empty. If the board has no markers, it exits `3` and changes nothing — adding the +markers is a deliberate opt-in by the board owner. + +### Usage + +```bash +board-roll.sh --live --ledger [options] + + --live LIVE board file (required) + --ledger append-only LEDGER file (required; created if absent) + --cap size ceiling for LIVE (default 8192) + --marker entry-heading prefix inside the roll zone (default "### ") + --dry-run report what would move; change nothing + -h, --help show help and exit 0 +``` + +Exit codes: `0` LIVE under cap (already, or after rolling) — on `--dry-run`, a +plan exists or nothing to do · `2` usage / argument / IO error · `3` cannot meet +the cap (no markers, or the pinned sections alone exceed the cap and need a manual +trim). + +Writes are atomic (temp file + `mv`, LEDGER first) so a failure never leaves a +board half-written; line endings are normalized to LF on rewrite. `--dry-run` +first is recommended when wiring it into a board update protocol. + +Run the regression suite with `bash test-board-roll.sh`. diff --git a/packages/mosaic/framework/tools/orchestrator/board-roll.sh b/packages/mosaic/framework/tools/orchestrator/board-roll.sh new file mode 100644 index 00000000..1c69fe4c --- /dev/null +++ b/packages/mosaic/framework/tools/orchestrator/board-roll.sh @@ -0,0 +1,257 @@ +#!/usr/bin/env bash +# +# board-roll.sh — keep a LIVE orchestration board under its byte cap by rolling +# the oldest archival entries out to its append-only LEDGER. +# +# WHY: coordinator boards (MOS-ORCHESTRATION-BOARD-LIVE.md, MS-LEAD-BOARD-LIVE.md) +# enforce a "< 8 KB LIVE" discipline via a self-guard that ABORTs the board write +# when the file exceeds the cap. In practice the LIVE board keeps bumping the cap, +# so coordinators hand-trim + retry every time (observed: 38 ABORT-OVER-CAP cycles +# in a 24h window on one coordinator). This automates that trim, mechanically and +# reversibly, so the audit trail is preserved in the LEDGER instead of hand-deleted. +# +# CONTRACT (conservative by design — it NEVER guesses what is safe to move): +# The LIVE board must declare an explicit ROLL ZONE with HTML-comment markers: +# +# +# ### 2026-07-22 (newest tick — stays longest) +# ... +# ### 2026-07-19 (oldest tick — rolled first) +# ... +# +# +# Everything OUTSIDE the markers (title, protocol blockquote, curated always-current +# `##` sections) is PINNED and never touched. Inside the zone, entries are delimited +# by a heading marker (default `### `) and are assumed newest-first (top) → oldest-last +# (bottom), matching board convention. board-roll moves whole oldest (bottom-most) +# entry blocks out of the zone and APPENDS them verbatim to the LEDGER, one block at a +# time, until the LIVE file is back under the cap or the zone is empty. +# +# If no markers are present, it exits 3 without changing anything (safe default — +# adding the markers is a deliberate opt-in by the board owner). +# +# USAGE: +# board-roll.sh --live --ledger [options] +# +# OPTIONS: +# --live LIVE board file (required) +# --ledger append-only LEDGER file (required; created if absent) +# --cap size ceiling for LIVE (default 8192) +# --marker entry-heading prefix inside the roll zone (default "### ") +# --dry-run report what would move + resulting size; change nothing +# -h, --help print usage and exit 0 +# +# EXIT CODES: +# 0 LIVE is under cap (already, or after rolling); on --dry-run, 0 = a plan exists +# (or nothing to do) +# 2 usage / argument / IO error (bad flag, missing file, unwritable target) +# 3 cannot satisfy the cap: no roll markers present, OR the zone was emptied and +# LIVE is still over cap (curated pinned sections need a manual trim) +# +# NOTE: line endings are normalized to LF on rewrite (boards are LF markdown); a +# trailing newline is always ensured. Writes are atomic (temp file + mv) so a +# failure never leaves LIVE or LEDGER half-written. + +set -euo pipefail + +START_MARK='' +END_MARK='' + +usage() { + cat <<'EOF' +Usage: board-roll.sh --live --ledger [options] + +Roll the oldest entries out of a LIVE orchestration board into its LEDGER +until the LIVE file is under a byte cap. Conservative: only content inside +explicit / markers is moved. + +Options: + --live LIVE board file (required) + --ledger append-only LEDGER file (required; created if absent) + --cap size ceiling for LIVE (default 8192) + --marker entry-heading prefix inside the roll zone (default "### ") + --dry-run report what would move; change nothing + -h, --help show this help and exit 0 + +Exit: 0 under cap (or dry-run plan) · 2 usage/IO error · 3 cannot meet cap + (no markers, or pinned sections alone exceed the cap). +EOF +} + +die() { echo "board-roll: $*" >&2; exit 2; } + +LIVE=""; LEDGER=""; CAP=8192; MARKER='### '; DRYRUN=0 +while [[ $# -gt 0 ]]; do + case "$1" in + --live) LIVE="${2:-}"; shift 2 || die "--live needs a value" ;; + --ledger) LEDGER="${2:-}"; shift 2 || die "--ledger needs a value" ;; + --cap) CAP="${2:-}"; shift 2 || die "--cap needs a value" ;; + --marker) MARKER="${2:-}"; shift 2 || die "--marker needs a value" ;; + --dry-run) DRYRUN=1; shift ;; + -h|--help) usage; exit 0 ;; + *) usage >&2; die "unknown option: $1" ;; + esac +done + +[[ -n "$LIVE" ]] || { usage >&2; die "--live is required"; } +[[ -n "$LEDGER" ]] || { usage >&2; die "--ledger is required"; } +[[ -f "$LIVE" ]] || die "LIVE file not found: $LIVE" +[[ "$CAP" =~ ^[0-9]+$ ]] || die "--cap must be a non-negative integer, got: $CAP" + +# --- read LIVE into a line array (newlines stripped; re-added on write) --------- +mapfile -t LINES < "$LIVE" + +# byte size of an array rendered as LF-terminated text +render_size() { + if [[ $# -eq 0 ]]; then printf 0; return; fi + printf '%s\n' "$@" | wc -c +} + +orig_size=$(render_size "${LINES[@]}") + +# --- already under cap → nothing to do ----------------------------------------- +if (( orig_size < CAP )); then + echo "board-roll: LIVE is ${orig_size}B (< cap ${CAP}B) — nothing to roll." + exit 0 +fi + +# --- locate the roll-zone markers ---------------------------------------------- +start_idx=-1; end_idx=-1 +for i in "${!LINES[@]}"; do + [[ "${LINES[$i]}" == "$START_MARK" && $start_idx -eq -1 ]] && start_idx=$i + [[ "${LINES[$i]}" == "$END_MARK" ]] && end_idx=$i +done +if (( start_idx < 0 || end_idx < 0 || end_idx <= start_idx )); then + echo "board-roll: LIVE is ${orig_size}B (>= cap ${CAP}B) but no usable roll zone" >&2 + echo " (need '$START_MARK' then '$END_MARK'). Add the markers around the" >&2 + echo " archival tick section to opt this board into automatic rolling." >&2 + exit 3 +fi + +# preamble = lines [0 .. start_idx] (inclusive of START marker) +# zone = lines (start_idx .. end_idx) (exclusive of both markers) +# footer = lines [end_idx .. end] (inclusive of END marker) +preamble=(); zone=(); footer=() +for i in "${!LINES[@]}"; do + if (( i <= start_idx )); then preamble+=("${LINES[$i]}") + elif (( i < end_idx )); then zone+=("${LINES[$i]}") + else footer+=("${LINES[$i]}") + fi +done + +# --- split the zone into a fixed head + entry blocks ---------------------------- +# zone_head = any zone lines before the first entry marker (kept, never rolled). +# blocks[k] = newline-joined text of entry k (marker line .. line before next marker). +zone_head=(); declare -a block_start=() +first_block=-1 +for i in "${!zone[@]}"; do + if [[ "${zone[$i]}" == "$MARKER"* ]]; then + [[ $first_block -eq -1 ]] && first_block=$i + block_start+=("$i") + fi +done +if (( first_block == -1 )); then + echo "board-roll: LIVE is ${orig_size}B (>= cap ${CAP}B) but the roll zone has no" >&2 + echo " '${MARKER}' entries to move. Trim the pinned sections manually." >&2 + exit 3 +fi +for (( i=0; i= CAP && keep > 0 )); do + keep=$(( keep - 1 )) + current_size=$(build_live_size) +done + +moved_count=$(( nblocks - keep )) +if (( moved_count == 0 )); then + # zone had entries but none movable brought us under (shouldn't happen: keep hits 0) + echo "board-roll: could not reduce LIVE below cap (${current_size}B >= ${CAP}B)." >&2 + exit 3 +fi + +# --- dry-run report ------------------------------------------------------------- +plan_headers() { + local k + for (( k=keep; k= CAP )); then + echo " WARNING: still >= cap after emptying the zone; pinned sections need a manual trim." >&2 + exit 3 + fi + exit 0 +fi + +# --- commit the roll atomically ------------------------------------------------- +live_tmp="$(mktemp "${LIVE}.roll.XXXXXX")" || die "cannot create temp next to LIVE" +ledger_tmp="" +# shellcheck disable=SC2329 # invoked indirectly via `trap cleanup EXIT` +cleanup() { rm -f "$live_tmp" "$ledger_tmp" 2>/dev/null || true; } +trap cleanup EXIT + +# new LIVE = preamble + zone_head + kept blocks + footer +{ + printf '%s\n' "${preamble[@]}" "${zone_head[@]}" + for (( k=0; k "$live_tmp" + +# LEDGER gets the moved blocks appended verbatim, in original top→bottom order, +# under a provenance separator. LEDGER is append-only, so we only ever add at EOF. +ledger_tmp="$(mktemp "${LEDGER}.roll.XXXXXX")" || die "cannot create temp next to LEDGER" +if [[ -f "$LEDGER" ]]; then cat "$LEDGER" > "$ledger_tmp"; fi +# ensure a trailing newline on existing content before appending +if [[ -s "$ledger_tmp" && -n "$(tail -c1 "$ledger_tmp")" ]]; then printf '\n' >> "$ledger_tmp"; fi +{ + printf '\n\n' \ + "$moved_count" "$([[ $moved_count -eq 1 ]] && echo y || echo ies)" "$(basename "$LIVE")" + for (( k=keep; k> "$ledger_tmp" + +# atomic swap (both, LEDGER first so a crash never drops content that left LIVE) +mv "$ledger_tmp" "$LEDGER"; ledger_tmp="" +mv "$live_tmp" "$LIVE"; live_tmp="" +trap - EXIT + +# read the real on-disk size back (truthful, not the predicted value) +final_size=$(wc -c < "$LIVE") +echo "board-roll: rolled ${moved_count} entr$([[ $moved_count -eq 1 ]] && echo y || echo ies) to $(basename "$LEDGER"); LIVE ${orig_size}B → ${final_size}B (cap ${CAP}B)." +if (( final_size >= CAP )); then + echo "board-roll: still >= cap after rolling all zone entries; pinned sections need a manual trim." >&2 + exit 3 +fi +exit 0 diff --git a/packages/mosaic/framework/tools/orchestrator/test-board-roll.sh b/packages/mosaic/framework/tools/orchestrator/test-board-roll.sh new file mode 100644 index 00000000..ca75c789 --- /dev/null +++ b/packages/mosaic/framework/tools/orchestrator/test-board-roll.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# Regression harness for board-roll.sh — rolling oldest LIVE-board entries to LEDGER. +# +# Asserts: +# 1. Under cap → no-op, exit 0, files unchanged. +# 2. Over cap, no roll markers → exit 3, LIVE unchanged (never guesses). +# 3. Over cap, markers present → rolls the fewest oldest entries to get under cap, +# LIVE ends under cap, pinned preamble/footer + newest entries preserved. +# 4. Rolled blocks land in the LEDGER verbatim, oldest set in original order. +# 5. --dry-run changes nothing and reports a plan. +# 6. Zone emptied but pinned sections alone exceed cap → exit 3. +# 7. --help exits 0 and prints usage; an unknown flag exits nonzero (#701 discipline). + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +SUT="$SCRIPT_DIR/board-roll.sh" + +fail=0 +note() { echo "FAIL: $*" >&2; fail=1; } + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +# builds a LIVE board: pinned preamble + roll zone with N dated entries (newest first), +# each entry padded to be individually large so the cap math is predictable. +make_board() { # $1 file $2 n_entries $3 with_markers(1/0) $4 pad_bytes + local f=$1 n=$2 markers=$3 pad=$4 i padtxt + padtxt=$(head -c "$pad" < /dev/zero | tr '\0' 'x') + { + echo "# BOARD — LIVE" + echo "> pinned protocol blockquote, never rolled." + echo + echo "## Curated always-current section (pinned)" + echo "- this stays no matter what" + echo + [[ "$markers" == 1 ]] && echo '' + # newest first (i=n .. 1); oldest (i=1) ends at the bottom + for (( i=n; i>=1; i-- )); do + echo "### 2026-07-$(printf '%02d' $i) tick number $i" + echo "- detail $i $padtxt" + echo + done + [[ "$markers" == 1 ]] && echo '' + } > "$f" + return 0 +} + +# ── 1. under cap → no-op ─────────────────────────────────────────────────────── +L="$WORK/live1.md"; G="$WORK/ledger1.md"; : > "$G" +make_board "$L" 2 1 10 +before=$(cat "$L") +if ! out=$(bash "$SUT" --live "$L" --ledger "$G" --cap 100000 2>&1); then + note "under-cap should exit 0 (got nonzero): $out" +fi +[[ "$(cat "$L")" == "$before" ]] || note "under-cap modified LIVE" +[[ -s "$G" ]] && note "under-cap wrote to LEDGER" + +# ── 2. over cap, no markers → exit 3, unchanged ──────────────────────────────── +L="$WORK/live2.md"; G="$WORK/ledger2.md"; : > "$G" +make_board "$L" 6 0 400 +before=$(cat "$L") +set +e; bash "$SUT" --live "$L" --ledger "$G" --cap 800 >/dev/null 2>&1; rc=$?; set -e +[[ "$rc" -eq 3 ]] || note "no-markers over-cap should exit 3 (got $rc)" +[[ "$(cat "$L")" == "$before" ]] || note "no-markers run modified LIVE (must never guess)" + +# ── 3+4. over cap with markers → rolls oldest, LIVE under cap, LEDGER gets them ─ +L="$WORK/live3.md"; G="$WORK/ledger3.md"; echo "# LEDGER" > "$G" +make_board "$L" 6 1 400 # 6 entries, each ~>400B +big=$(wc -c < "$L") +[[ "$big" -ge 2000 ]] || note "fixture too small to test rolling ($big B)" +if ! out=$(bash "$SUT" --live "$L" --ledger "$G" --cap 2000 2>&1); then + note "marker roll should exit 0 when it can get under cap: $out" +fi +after=$(wc -c < "$L") +[[ "$after" -lt 2000 ]] || note "LIVE still >= cap after roll ($after B)" +# pinned content survives +grep -q "Curated always-current section" "$L" || note "roll dropped pinned section" +grep -q 'BOARD-ROLL:START' "$L" || note "roll dropped START marker" +grep -q 'BOARD-ROLL:END' "$L" || note "roll dropped END marker" +# newest entry (07-06) stays; oldest (07-01) is the first to leave +grep -q "### 2026-07-06 tick number 6" "$L" || note "roll dropped the newest entry" +grep -q "### 2026-07-01 tick number 1" "$L" && note "oldest entry not rolled out of LIVE" +# oldest went to LEDGER +grep -q "### 2026-07-01 tick number 1" "$G" || note "oldest entry not appended to LEDGER" +grep -q "board-roll:.*rolled from live3.md" "$G" || note "LEDGER missing provenance separator" +# a rolled entry must not be duplicated (present in exactly one of LIVE/LEDGER) +if grep -q "### 2026-07-01 tick number 1" "$L"; then note "rolled entry duplicated in LIVE"; fi +# LEDGER original content preserved +grep -q "^# LEDGER" "$G" || note "roll clobbered existing LEDGER content" + +# ── 5. --dry-run changes nothing ─────────────────────────────────────────────── +L="$WORK/live5.md"; G="$WORK/ledger5.md"; echo "# LEDGER" > "$G" +make_board "$L" 6 1 400 +before_l=$(cat "$L"); before_g=$(cat "$G") +out=$(bash "$SUT" --live "$L" --ledger "$G" --cap 2000 --dry-run 2>&1) || note "dry-run exited nonzero: $out" +echo "$out" | grep -qi "dry run" || note "dry-run did not announce itself" +echo "$out" | grep -q "would roll" || note "dry-run did not report a plan" +[[ "$(cat "$L")" == "$before_l" ]] || note "dry-run modified LIVE" +[[ "$(cat "$G")" == "$before_g" ]] || note "dry-run modified LEDGER" + +# ── 6. zone emptied, pinned alone over cap → exit 3 ──────────────────────────── +# cap 120 is below the pinned preamble+footer size (~180B), so even after rolling +# every zone entry the LIVE file stays over cap → must report the unsatisfiable case. +L="$WORK/live6.md"; G="$WORK/ledger6.md"; echo "# LEDGER" > "$G" +make_board "$L" 3 1 50 +set +e; bash "$SUT" --live "$L" --ledger "$G" --cap 120 >/dev/null 2>&1; rc=$?; set -e +[[ "$rc" -eq 3 ]] || note "unsatisfiable cap should exit 3 (got $rc)" + +# ── 7. help exits 0, unknown flag exits nonzero (#701) ───────────────────────── +if ! out=$(bash "$SUT" --help 2>&1); then note "--help exited nonzero"; fi +[[ "$out" == Usage:* ]] || note "--help did not print usage" +bash "$SUT" -h >/dev/null 2>&1 || note "-h exited nonzero" +if bash "$SUT" --not-a-real-flag >/dev/null 2>&1; then note "unknown flag was accepted"; fi +if bash "$SUT" --live "$WORK/live3.md" >/dev/null 2>&1; then note "missing --ledger was accepted"; fi + +if [[ "$fail" -eq 0 ]]; then + echo "board-roll regression passed (7 groups)" +fi +exit "$fail"