feat(wake): W2 — three-cursor durable store + RECEIVED/CONSUMED ack-wrapper + watch-list schema
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Implements W2 of the wake/heartbeat canon (EPIC #892), honoring the
CONVERGED-DESIGN contracts exactly.

A2 store.sh — three-cursor durable queue (§1.2/§2.3/§2.4):
  - observed_seq authoritative monotonic cursor (detector assigns, lib records);
    SHAs are descriptors, not the cursor.
  - durable pending-inbox {observed_seq, locators, class, emit_ts, hmac}; hmac is
    an UNSIGNED placeholder (signing is W3/A5).
  - consumed_seq advances ONLY on a CONSUMED ack over a CONTIGUOUS gapless prefix
    <=N; never on enqueue/exit-0/paste.
  - coalesce: digest REPLACES the pending cumulative-state entry; actionable/human
    APPEND; ALL classes durable (durability never bypassed); absent class => actionable.
  - drain returns the deliverable set; every mutation is atomic write-tmp+rename.

A4 ack.sh — RECEIVED/CONSUMED wrapper (§2.2):
  - RECEIVED dedups DELIVERY via wake_id (dup = re-RECEIVE, never re-action).
  - CONSUMED N local-write-only to the XDG ledger, cumulative, gapless-prefix
    enforced; background sync ships the ack (never blocks on network).
  - `embed` prints the one copy-run line a digest embeds.

Watch-list SCHEMA contract (defaults/wake-watch-list.schema.json) with
schema_version; wake manifest.txt declares component semver + supported schema
range (schema_min/schema_max, Gate B — version metadata only, NOT path ownership).

RED-first shell harness wired into test:framework-shell covers each invariant
(three-cursor, coalesce-vs-append, contiguous-prefix CONSUMED, wake_id dedup,
atomic tmp+rename crash-safety, durability-across-restart, ack-never-blocks-on-net).
shellcheck clean; sanitization gate + operator firewall pass.

Part of #892

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
2026-07-25 18:28:45 -05:00
parent ab6e8e80dc
commit 0f1a37ed2d
7 changed files with 1163 additions and 1 deletions

View File

@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# _wake-common.sh — shared state resolution + atomic-write primitive for the
# wake/heartbeat durable queue (W2 of the wake canon, EPIC #892).
#
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
# §1.2 three-cursor durable queue; ALL state XDG, atomic write-tmp+rename.
# §2.3 durability is NEVER bypassed.
#
# This file is sourced by store.sh and ack.sh. It defines NO top-level actions;
# sourcing it is side-effect-free except for setting readonly path vars.
#
# Operator-agnostic (framework firewall): state location is derived purely from
# XDG / env. No operator paths, names, or secrets appear here.
# ---------------------------------------------------------------------------
# State layout (XDG). §1.2: "all state XDG".
# base = ${WAKE_STATE_HOME:-${XDG_STATE_HOME:-$HOME/.local/state}/mosaic/wake}
# agent = ${WAKE_AGENT:-default} (per-agent queue namespace)
# STATE_DIR = <base>/<agent>
# ---------------------------------------------------------------------------
wake_state_dir() {
local base agent
base="${WAKE_STATE_HOME:-${XDG_STATE_HOME:-$HOME/.local/state}/mosaic/wake}"
agent="${WAKE_AGENT:-default}"
printf '%s/%s' "$base" "$agent"
}
# File names within STATE_DIR.
# observed_seq — highest observed_seq ever recorded (max monotonic int).
# consumed_seq — consumer cursor: top of the contiguous consumed prefix.
# observed.set — observed seqs in the live window (> consumed_seq), one int
# per line; the gap-detector's source of truth. Immune to
# coalescing (coalescing removes a pending ENTRY, never the
# fact that its seq was observed).
# pending.jsonl — the durable pending-inbox: one entry JSON object per line,
# {observed_seq, locators, class, emit_ts, hmac}.
# ack-ledger.jsonl — local-write-only ack ledger (RECEIVED / CONSUMED).
# ack-sync.state — background-sync bookkeeping (last shipped / outage flag).
# _wake_tmp_glob DIR — the glob used for atomic-write temp files, so readers can
# ignore in-flight/crashed writes. A crash leaves one of these; it is NEVER the
# live file (only rename promotes content), so it can never corrupt a read.
_wake_tmp_prefix='.wake.tmp.'
# _atomic_write TARGET (content on stdin)
# §1.2 / task: EVERY state mutation is atomic write-tmp+rename. Write a temp
# file in the SAME directory (so mv is a same-filesystem atomic rename), fsync
# is best-effort, then rename over the target. A crash before the rename leaves
# the old target fully intact and a stale .wake.tmp.* that readers ignore.
_atomic_write() {
local target="$1" dir tmp
dir="$(dirname "$target")"
[ -d "$dir" ] || mkdir -p "$dir"
tmp="$(mktemp "$dir/${_wake_tmp_prefix}XXXXXX")" || return 1
if ! cat >"$tmp"; then
rm -f "$tmp"
return 1
fi
# Best-effort durability of the temp file before the rename. Not fatal if the
# platform lacks it — the rename atomicity is the load-bearing guarantee.
sync "$tmp" 2>/dev/null || true
if ! mv -f "$tmp" "$target"; then
rm -f "$tmp"
return 1
fi
}
# _wake_clean_stale_tmp DIR — remove leftover atomic-write temp files (e.g. from
# a crash mid-write). Safe: these are never the live store.
_wake_clean_stale_tmp() {
local dir="$1"
[ -d "$dir" ] || return 0
find "$dir" -maxdepth 1 -name "${_wake_tmp_prefix}*" -type f -delete 2>/dev/null || true
}
# _wake_read_int FILE DEFAULT — read a single integer from FILE, or DEFAULT.
_wake_read_int() {
local file="$1" def="$2" val
if [ -f "$file" ]; then
val="$(tr -d '[:space:]' <"$file")"
case "$val" in
'' | *[!0-9]*) printf '%s' "$def" ;;
*) printf '%s' "$val" ;;
esac
else
printf '%s' "$def"
fi
}
# _wake_init_dir STATE_DIR — ensure the state layout exists; idempotent.
_wake_init_dir() {
local dir="$1"
mkdir -p "$dir" || return 1
_wake_clean_stale_tmp "$dir"
[ -f "$dir/observed_seq" ] || printf '0' | _atomic_write "$dir/observed_seq"
[ -f "$dir/consumed_seq" ] || printf '0' | _atomic_write "$dir/consumed_seq"
[ -f "$dir/observed.set" ] || printf '' | _atomic_write "$dir/observed.set"
[ -f "$dir/pending.jsonl" ] || printf '' | _atomic_write "$dir/pending.jsonl"
[ -f "$dir/ack-ledger.jsonl" ] || printf '' | _atomic_write "$dir/ack-ledger.jsonl"
}

View File

@@ -0,0 +1,279 @@
#!/usr/bin/env bash
# ack.sh — A4 of the wake canon (EPIC #892, W2): the RECEIVED / CONSUMED
# ack-wrapper.
#
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
# §2.2 ack protocol: RECEIVED vs CONSUMED; local-write + async ship.
# §1.2 three-cursor: consumed_seq advances ONLY on a consumer CONSUMED ack.
#
# RECEIVED — delivery happened (paste landed). `wake_id` dedups DELIVERY ONLY:
# a duplicate delivery is a re-RECEIVE, NEVER a re-action.
# CONSUMED N — emitted ONLY after durable capture OR no-op disposition of a
# CONTIGUOUS prefix <=N. first-action-before-capture is FORBIDDEN
# (caller discipline: capture BEFORE invoking this). Cannot ack N
# while N-1 is unconsumed (the store enforces the gapless prefix).
# Acks are CUMULATIVE: CONSUMED N implies all <=N.
#
# LOCAL-WRITE-ONLY: the ack path writes the XDG ledger and advances the cursor
# with NO network call. A BACKGROUND sync ships the ack (WAKE_ACK_SYNC_CMD).
# Ack-path outage => a single re-wake after timeout then fall back to cadence
# (no retry spam) — the background sync runs ONCE and records an outage marker;
# it never loops.
#
# The `embed` subcommand prints the one copy-run line meant to be EMBEDDED in
# each digest (W3 renders it into the digest body).
#
# Operator-agnostic: ledger via XDG/env only; no operator paths/names/secrets.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=./_wake-common.sh disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
STATE_DIR="$(wake_state_dir)"
STORE_SH="$SCRIPT_DIR/store.sh"
_need_jq() {
command -v jq >/dev/null 2>&1 || {
echo "ack.sh: jq is required" >&2
exit 3
}
}
usage() {
cat >&2 <<'EOF'
Usage: ack.sh <command> [options]
Commands:
received --wake-id ID Record a RECEIVED ack (delivery). The
wake_id DEDUPS DELIVERY only: a repeat
is a re-RECEIVE (prints DUP), never a
re-action. Local-write only.
consumed --upto N [--wake-id ID] Record a CONSUMED ack for the contiguous
prefix <=N and advance consumed_seq.
Local-write only; a background sync
ships it (never blocks on network).
--no-sync suppresses the background ship.
embed --upto N [--wake-id ID] Print the copy-run ack line to EMBED in
a digest (does not perform the ack).
status Print ledger tail + cursor state.
Environment:
WAKE_STATE_HOME override base state dir (XDG by default).
WAKE_AGENT per-agent queue namespace (default: default).
WAKE_ACK_SYNC_CMD command run in the BACKGROUND to ship an ack. Receives the
ack JSON on stdin. Never invoked on the synchronous path.
WAKE_ACK_CLI override the embedded copy-run invocation prefix (default:
the resolved path to this script).
EOF
}
# _ledger_append JSON — append one ack record to the local ledger, atomically.
# §2.2: LOCAL-WRITE-ONLY. No network here.
_ledger_append() {
local record="$1"
_wake_init_dir "$STATE_DIR"
{
cat "$STATE_DIR/ack-ledger.jsonl" 2>/dev/null
printf '%s\n' "$record"
} | grep -v '^[[:space:]]*$' | _atomic_write "$STATE_DIR/ack-ledger.jsonl"
}
# _wake_id_seen ID — true if a RECEIVED record for this wake_id already exists.
# Count-based (jq -s slurp): an EMPTY ledger slurps to [] => length 0. (jq -e
# over a zero-input file returns exit 0, which would false-positive here — so
# never rely on -e exit semantics for presence.)
_wake_id_seen() {
local id="$1" n
[ -f "$STATE_DIR/ack-ledger.jsonl" ] || return 1
n="$(jq -s --arg id "$id" \
'[.[] | select(.type=="RECEIVED" and .wake_id==$id)] | length' \
"$STATE_DIR/ack-ledger.jsonl" 2>/dev/null || echo 0)"
[ "${n:-0}" -gt 0 ]
}
# _ship_async JSON — fire the background sync ONCE, fully detached, so the ack
# path returns immediately and NEVER blocks on the network (§2.2). No retry
# loop: a failure records an outage marker and stops (single re-wake + cadence
# fallback is the escalation policy, handled off this path).
_ship_async() {
local record="$1"
[ -n "${WAKE_ACK_SYNC_CMD:-}" ] || return 0
local marker="$STATE_DIR/ack-sync.state"
(
if printf '%s\n' "$record" | sh -c "$WAKE_ACK_SYNC_CMD" >/dev/null 2>&1; then
printf 'shipped %s\n' "$(date +%s)" >"$marker" 2>/dev/null || true
else
printf 'outage %s\n' "$(date +%s)" >"$marker" 2>/dev/null || true
fi
) </dev/null >/dev/null 2>&1 &
# Detach so no shell job-control state ties the ack turn to the ship.
disown 2>/dev/null || true
}
cmd_received() {
_need_jq
local wake_id=''
while [ $# -gt 0 ]; do
case "$1" in
--wake-id)
wake_id="${2:-}"
shift 2
;;
*)
echo "ack.sh received: unknown option '$1'" >&2
exit 2
;;
esac
done
[ -n "$wake_id" ] || {
echo "ack.sh received: --wake-id is required" >&2
exit 2
}
_wake_init_dir "$STATE_DIR"
# wake_id dedups DELIVERY only. A duplicate delivery is a re-RECEIVE — logged,
# but flagged dup so no consumer re-action is triggered. It NEVER advances
# consumed_seq (delivery != consumption).
local dup="false" verb="RECEIVED"
if _wake_id_seen "$wake_id"; then
dup="true"
verb="DUP"
fi
local record
record="$(jq -cn \
--arg id "$wake_id" \
--argjson dup "$dup" \
--argjson ts "$(date +%s)" \
'{type:"RECEIVED", wake_id:$id, dup:$dup, ts:$ts}')"
_ledger_append "$record"
echo "$verb"
}
cmd_consumed() {
_need_jq
local upto='' wake_id='' do_sync="1"
while [ $# -gt 0 ]; do
case "$1" in
--upto)
upto="${2:-}"
shift 2
;;
--wake-id)
wake_id="${2:-}"
shift 2
;;
--no-sync)
do_sync="0"
shift
;;
*)
echo "ack.sh consumed: unknown option '$1'" >&2
exit 2
;;
esac
done
case "$upto" in
'' | *[!0-9]*)
echo "ack.sh consumed: --upto must be a non-negative integer" >&2
exit 2
;;
esac
# Advance consumed_seq via the store. The store enforces the CONTIGUOUS
# gapless-prefix rule and rejects a gap (cannot ack N while N-1 unconsumed).
# This is a LOCAL-WRITE cursor advance — no network.
local new_cursor
if ! new_cursor="$("$STORE_SH" consume --upto "$upto" 2>&1)"; then
echo "ack.sh consumed: refused — $new_cursor" >&2
exit 1
fi
# Record the CONSUMED ack in the local ledger (still no network).
local record
record="$(jq -cn \
--argjson upto "$upto" \
--arg id "$wake_id" \
--argjson ts "$(date +%s)" \
'{type:"CONSUMED", upto:$upto, wake_id:$id, ts:$ts}')"
_ledger_append "$record"
# Only now, OFF the ack path, ship it in the background. The synchronous
# portion is already complete and durable.
if [ "$do_sync" = "1" ]; then
_ship_async "$record"
fi
echo "CONSUMED $new_cursor"
}
cmd_embed() {
local upto='' wake_id=''
while [ $# -gt 0 ]; do
case "$1" in
--upto)
upto="${2:-}"
shift 2
;;
--wake-id)
wake_id="${2:-}"
shift 2
;;
*)
echo "ack.sh embed: unknown option '$1'" >&2
exit 2
;;
esac
done
case "$upto" in
'' | *[!0-9]*)
echo "ack.sh embed: --upto must be a non-negative integer" >&2
exit 2
;;
esac
# The one copy-run line a digest embeds. WAKE_ACK_CLI lets an installer point
# it at the deployed path; default is this script's resolved path.
local cli="${WAKE_ACK_CLI:-$SCRIPT_DIR/ack.sh}"
if [ -n "$wake_id" ]; then
printf '%s consumed --upto %s --wake-id %s\n' "$cli" "$upto" "$wake_id"
else
printf '%s consumed --upto %s\n' "$cli" "$upto"
fi
}
cmd_status() {
_wake_init_dir "$STATE_DIR"
echo "# cursors"
"$STORE_SH" cursors
echo "# ack-ledger (tail)"
tail -n 10 "$STATE_DIR/ack-ledger.jsonl" 2>/dev/null || true
if [ -f "$STATE_DIR/ack-sync.state" ]; then
echo "# sync"
cat "$STATE_DIR/ack-sync.state"
fi
}
main() {
[ $# -ge 1 ] || {
usage
exit 2
}
local cmd="$1"
shift
case "$cmd" in
received) cmd_received "$@" ;;
consumed) cmd_consumed "$@" ;;
embed) cmd_embed "$@" ;;
status) cmd_status "$@" ;;
-h | --help | help) usage ;;
*)
echo "ack.sh: unknown command '$cmd'" >&2
usage
exit 2
;;
esac
}
main "$@"

View File

@@ -0,0 +1,29 @@
# Mosaic wake component — VERSION metadata manifest (Gate B).
#
# EPIC #892, W2 of the wake/heartbeat canon.
#
# SCOPE — THIS FILE IS VERSION METADATA ONLY. It declares the wake component's
# semantic version and the RANGE of watch-list schema versions it supports. It
# does NOT authorize file/path ownership: path-ownership remains the sole domain
# of packages/mosaic/framework/framework-manifest.txt (Gate A). Do not read any
# ownership meaning into this file.
#
# Format: KEY=VALUE, one per line. '#' and blank lines ignored.
# Component identity + semantic version (the store+drain lib + ack-wrapper).
component=wake
version=0.1.0
# Watch-list schema this component consumes, and the INCLUSIVE range of
# schema_version values it supports. A wake-watch-list.json whose schema_version
# falls outside [schema_min, schema_max] is rejected by the component (fail-loud),
# never silently coerced.
schema=wake-watch-list
schema_min=1
schema_max=1
# W2 pieces shipped by this component version (informational):
# store.sh A2 — three-cursor durable store + drain lib.
# ack.sh A4 — RECEIVED/CONSUMED ack-wrapper (local-write + async ship).
# Out of W2 scope (later waves): detector (W4), digest renderer/HMAC (W3),
# FN-oracle/reconciler (W5), beacon (W6), installer (W7).

View File

@@ -0,0 +1,319 @@
#!/usr/bin/env bash
# store.sh — A2 of the wake canon (EPIC #892, W2): the THREE-CURSOR durable
# queue (store + drain lib).
#
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
# §1.2 three-cursor data-flow: observe -> durable store -> coalesce -> drain.
# §2.3 classes & durability: ALL classes durable; coalescing is optional.
# §2.4 cursor semantics: observed_seq is authoritative; SHAs are descriptors.
#
# Three cursors:
# observed_seq (detector-owned; this lib RECORDS/stores it) — monotonic int
# assigned at observation. Authoritative. Source SHAs are
# descriptors, NOT the cursor (§2.4).
# pending-inbox (durable) — append store; retains until CONSUMED; survives
# pane death / park / restart.
# consumed_seq (consumer-owned) — advances ONLY on a CONSUMED ack, over a
# CONTIGUOUS gapless prefix <=N. NEVER on sender exit-0 / paste.
#
# Coalesce (§1.2/§2.3): class=digest REPLACES the pending cumulative-state entry
# (newest subsumes prior). Non-coalescible classes (actionable, human, ...)
# APPEND. ALL classes are stored durably — durability is NEVER bypassed.
# Absent class => treated as `actionable` (fail-safe).
#
# HMAC: entries carry an `hmac` field left as an UNSIGNED PLACEHOLDER ("").
# Signing is W3/A5, explicitly NOT W2.
#
# Every state mutation is atomic write-tmp+rename (see _wake-common.sh).
#
# Operator-agnostic: state via XDG/env only; no operator paths/names/secrets.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=./_wake-common.sh disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
STATE_DIR="$(wake_state_dir)"
_need_jq() {
command -v jq >/dev/null 2>&1 || {
echo "store.sh: jq is required" >&2
exit 3
}
}
usage() {
cat >&2 <<'EOF'
Usage: store.sh <command> [options]
Commands:
init Create/repair the XDG state layout.
enqueue --seq N --class C [opts] Record an observation into the durable
pending-inbox (coalesce or append).
--seq N observed_seq (detector-assigned monotonic int). Required.
--class C digest|actionable|human|terminal-log|reaction.
Absent/empty => actionable (fail-safe).
--locators JSON JSON value for the entry's locators (default {}).
--emit-ts TS epoch seconds (default: now).
drain [--require-idle-cmd CMD] Emit the deliverable set (pending-inbox,
coalesced) as JSONL. WHAT to deliver;
the actual paste is out of scope.
If --require-idle-cmd is given and it
exits non-zero, emit nothing (not idle).
consume --upto N Advance consumed_seq over the contiguous
gapless prefix <=N; drop consumed
entries. Rejects a gap (cannot ack N
while N-1 is unconsumed). Cumulative &
idempotent.
cursors Print observed_seq / consumed_seq / depth.
Environment:
WAKE_STATE_HOME override base (default ${XDG_STATE_HOME:-$HOME/.local/state}/mosaic/wake)
WAKE_AGENT per-agent queue namespace (default: default)
EOF
}
cmd_init() {
_wake_init_dir "$STATE_DIR" || {
echo "store.sh: failed to init $STATE_DIR" >&2
exit 1
}
echo "$STATE_DIR"
}
cmd_enqueue() {
_need_jq
local seq='' class='' locators='{}' emit_ts=''
while [ $# -gt 0 ]; do
case "$1" in
--seq)
seq="${2:-}"
shift 2
;;
--class)
class="${2:-}"
shift 2
;;
--locators)
locators="${2:-}"
shift 2
;;
--emit-ts)
emit_ts="${2:-}"
shift 2
;;
*)
echo "store.sh enqueue: unknown option '$1'" >&2
exit 2
;;
esac
done
case "$seq" in
'' | *[!0-9]*)
echo "store.sh enqueue: --seq must be a non-negative integer" >&2
exit 2
;;
esac
# Absent class => actionable (fail-safe, §2.3).
[ -n "$class" ] || class="actionable"
case "$class" in
digest | actionable | human | terminal-log | reaction) : ;;
*)
echo "store.sh enqueue: unknown class '$class'" >&2
exit 2
;;
esac
[ -n "$emit_ts" ] || emit_ts="$(date +%s)"
case "$emit_ts" in
*[!0-9]*)
echo "store.sh enqueue: --emit-ts must be epoch seconds" >&2
exit 2
;;
esac
# Validate locators is well-formed JSON.
if ! printf '%s' "$locators" | jq -e . >/dev/null 2>&1; then
echo "store.sh enqueue: --locators must be valid JSON" >&2
exit 2
fi
_wake_init_dir "$STATE_DIR"
local consumed
consumed="$(_wake_read_int "$STATE_DIR/consumed_seq" 0)"
# An observation already inside the consumed prefix is idempotently ignored
# (§1.2: cursor never regresses; a re-observed already-consumed seq is a no-op).
if [ "$seq" -le "$consumed" ]; then
return 0
fi
# Build the durable entry. `hmac` is an UNSIGNED PLACEHOLDER (W3/A5 signs it).
local entry
entry="$(jq -cn \
--argjson seq "$seq" \
--arg class "$class" \
--argjson locators "$locators" \
--argjson emit_ts "$emit_ts" \
'{observed_seq:$seq, locators:$locators, class:$class, emit_ts:$emit_ts, hmac:""}')" || {
echo "store.sh enqueue: failed to build entry" >&2
exit 1
}
# --- durable store write (coalesce vs append) -----------------------------
# digest: REPLACE the single pending cumulative-state entry (drop any existing
# digest lines, then append the newest). Newest subsumes prior.
# others: APPEND (never replaced).
# In BOTH cases the entry lands in the durable store.
local new_pending
if [ "$class" = "digest" ]; then
new_pending="$(jq -c 'select(.class != "digest")' "$STATE_DIR/pending.jsonl" 2>/dev/null || true)"
new_pending="$(printf '%s\n%s\n' "$new_pending" "$entry" | grep -v '^[[:space:]]*$' || true)"
else
new_pending="$(cat "$STATE_DIR/pending.jsonl" 2>/dev/null; printf '%s\n' "$entry")"
new_pending="$(printf '%s' "$new_pending" | grep -v '^[[:space:]]*$' || true)"
fi
printf '%s\n' "$new_pending" | grep -v '^[[:space:]]*$' | _atomic_write "$STATE_DIR/pending.jsonl"
# --- observed.set: record the seq in the live window (gap-detector SoT) ----
{
cat "$STATE_DIR/observed.set" 2>/dev/null
printf '%s\n' "$seq"
} | grep -v '^[[:space:]]*$' | sort -n | uniq | _atomic_write "$STATE_DIR/observed.set"
# --- observed_seq cursor: max monotonic int -------------------------------
local observed
observed="$(_wake_read_int "$STATE_DIR/observed_seq" 0)"
if [ "$seq" -gt "$observed" ]; then
printf '%s' "$seq" | _atomic_write "$STATE_DIR/observed_seq"
fi
}
cmd_drain() {
local idle_cmd=''
while [ $# -gt 0 ]; do
case "$1" in
--require-idle-cmd)
idle_cmd="${2:-}"
shift 2
;;
*)
echo "store.sh drain: unknown option '$1'" >&2
exit 2
;;
esac
done
# Drain returns the deliverable set ONLY when the pane is idle-at-prompt. The
# actual idle detection + paste is out of W2 scope (send-message.sh); the
# caller supplies an idle predicate. No predicate => emit unconditionally.
if [ -n "$idle_cmd" ]; then
if ! sh -c "$idle_cmd" >/dev/null 2>&1; then
return 0 # not idle-at-prompt: deliver nothing this cycle.
fi
fi
[ -f "$STATE_DIR/pending.jsonl" ] || return 0
grep -v '^[[:space:]]*$' "$STATE_DIR/pending.jsonl" 2>/dev/null || true
}
cmd_consume() {
local upto=''
while [ $# -gt 0 ]; do
case "$1" in
--upto)
upto="${2:-}"
shift 2
;;
*)
echo "store.sh consume: unknown option '$1'" >&2
exit 2
;;
esac
done
case "$upto" in
'' | *[!0-9]*)
echo "store.sh consume: --upto must be a non-negative integer" >&2
exit 2
;;
esac
_need_jq
_wake_init_dir "$STATE_DIR"
local consumed observed
consumed="$(_wake_read_int "$STATE_DIR/consumed_seq" 0)"
observed="$(_wake_read_int "$STATE_DIR/observed_seq" 0)"
# Cumulative & idempotent: CONSUMED N implies all <=N. N already covered is a
# no-op success (the sender checks cursor-advanced-past-X, not per-wake).
if [ "$upto" -le "$consumed" ]; then
echo "$consumed"
return 0
fi
# Never consume beyond what was observed.
if [ "$upto" -gt "$observed" ]; then
echo "store.sh consume: cannot CONSUMED $upto beyond observed_seq $observed" >&2
exit 1
fi
# CONTIGUOUS gapless prefix: every seq in (consumed, upto] must have been
# observed. A missing interior seq is a GAP => reject (cannot ack N while N-1
# is unconsumed). observed.set is coalescing-immune, so a subsumed digest seq
# still counts as observed and does not read as a gap.
local k
k=$((consumed + 1))
while [ "$k" -le "$upto" ]; do
if ! grep -qxF "$k" "$STATE_DIR/observed.set" 2>/dev/null; then
echo "store.sh consume: gap at seq $k — cannot CONSUMED $upto while $k is unobserved/unconsumed" >&2
exit 1
fi
k=$((k + 1))
done
# Advance the consumer cursor and drop the now-consumed prefix from the
# durable store (retain-until-CONSUMED is satisfied).
jq -c "select(.observed_seq > $upto)" "$STATE_DIR/pending.jsonl" 2>/dev/null |
grep -v '^[[:space:]]*$' | _atomic_write "$STATE_DIR/pending.jsonl" || true
awk -v c="$upto" 'NF && $1+0 > c' "$STATE_DIR/observed.set" 2>/dev/null |
_atomic_write "$STATE_DIR/observed.set" || true
printf '%s' "$upto" | _atomic_write "$STATE_DIR/consumed_seq"
echo "$upto"
}
cmd_cursors() {
_wake_init_dir "$STATE_DIR"
local observed consumed depth
observed="$(_wake_read_int "$STATE_DIR/observed_seq" 0)"
consumed="$(_wake_read_int "$STATE_DIR/consumed_seq" 0)"
depth="$(grep -cv '^[[:space:]]*$' "$STATE_DIR/pending.jsonl" 2>/dev/null || echo 0)"
printf 'observed_seq=%s\nconsumed_seq=%s\npending_depth=%s\n' "$observed" "$consumed" "$depth"
}
main() {
[ $# -ge 1 ] || {
usage
exit 2
}
local cmd="$1"
shift
case "$cmd" in
init) cmd_init "$@" ;;
enqueue) cmd_enqueue "$@" ;;
drain) cmd_drain "$@" ;;
consume) cmd_consume "$@" ;;
cursors) cmd_cursors "$@" ;;
-h | --help | help) usage ;;
*)
echo "store.sh: unknown command '$cmd'" >&2
usage
exit 2
;;
esac
}
main "$@"

View File

@@ -0,0 +1,269 @@
#!/usr/bin/env bash
# test-wake-store-ack.sh — RED-FIRST invariant harness for W2 (EPIC #892):
# three-cursor durable store (store.sh, A2) + RECEIVED/CONSUMED ack-wrapper
# (ack.sh, A4).
#
# Each test asserts ONE CONVERGED-DESIGN invariant and is designed to go RED if
# that invariant regresses:
# T1 three-cursor advancement (§1.2/§2.4)
# T2 digest-coalesce-REPLACE vs actionable-APPEND (§1.2/§2.3)
# T3 contiguous-prefix CONSUMED / gap rejection (§1.2/§2.2)
# T4 wake_id DELIVERY-dedup (never re-action) (§2.2)
# T5 atomic write-tmp+rename survives crash mid-write (§1.2)
# T6 durability survives restart (retain until CONSUMED) (§2.3)
# T7 ack local-write NEVER blocks on network (§2.2)
#
# Isolated: every test runs against a fresh WAKE_STATE_HOME temp dir.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
STORE="$SCRIPT_DIR/store.sh"
ACK="$SCRIPT_DIR/ack.sh"
command -v jq >/dev/null 2>&1 || {
echo "SKIP: jq not available" >&2
exit 0
}
TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT
# Failures are recorded to a marker FILE, not a shell var: each test runs in a
# subshell (for env isolation) and a subshell cannot mutate a parent variable,
# so a var-based counter would silently swallow failures (the exact bug this
# harness must never have).
FAILFILE="$TMP_ROOT/failures"
: >"$FAILFILE"
pass=0
fail_msg() {
echo " FAIL: $*" >&2
echo "x" >>"$FAILFILE"
}
ok() { pass=$((pass + 1)); }
# jq_any FILE FILTER — true iff any JSONL record in FILE matches FILTER. Uses
# slurp (-s), NOT `jq -e select` (which reflects only the LAST input's value
# over a multi-line JSONL and would misreport presence).
jq_any() {
local file="$1" filter="$2"
[ -f "$file" ] || return 1
[ "$(jq -s "[.[] | select($filter)] | length" "$file" 2>/dev/null || echo 0)" -gt 0 ]
}
# stream_any — same, but reads JSONL from stdin.
stream_any() {
local filter="$1"
[ "$(jq -s "[.[] | select($filter)] | length" 2>/dev/null || echo 0)" -gt 0 ]
}
# fresh_state NAME — echoes a fresh isolated state home for export.
fresh_state() {
local d="$TMP_ROOT/$1"
rm -rf "$d"
mkdir -p "$d"
printf '%s' "$d"
}
echo "== T1: three-cursor advancement =="
(
WAKE_STATE_HOME="$(fresh_state t1)"
export WAKE_STATE_HOME
unset WAKE_AGENT
"$STORE" enqueue --seq 1 --class actionable --locators '{"repo":"r","issue":1}' >/dev/null
"$STORE" enqueue --seq 2 --class actionable --locators '{"repo":"r","issue":2}' >/dev/null
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'observed_seq=2' || fail_msg "T1: observed_seq should be 2 after enqueue 1,2 [$cur]"
echo "$cur" | grep -q 'consumed_seq=0' || fail_msg "T1: consumed_seq must NOT advance on enqueue (only on CONSUMED ack) [$cur]"
echo "$cur" | grep -q 'pending_depth=2' || fail_msg "T1: pending depth should be 2 [$cur]"
# consumed_seq advances only on CONSUMED.
"$STORE" consume --upto 2 >/dev/null
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'consumed_seq=2' || fail_msg "T1: consumed_seq should be 2 after CONSUMED 2 [$cur]"
echo "$cur" | grep -q 'pending_depth=0' || fail_msg "T1: pending drained after CONSUMED [$cur]"
) && ok
echo "== T2: digest coalesce-REPLACE vs actionable APPEND =="
(
WAKE_STATE_HOME="$(fresh_state t2)"
export WAKE_STATE_HOME
unset WAKE_AGENT
"$STORE" enqueue --seq 1 --class digest --locators '{"head":"aaa"}' >/dev/null
"$STORE" enqueue --seq 2 --class actionable --locators '{"issue":7}' >/dev/null
"$STORE" enqueue --seq 3 --class digest --locators '{"head":"bbb"}' >/dev/null
"$STORE" enqueue --seq 4 --class human --locators '{"from":"peer"}' >/dev/null
out="$("$STORE" drain)"
ndigest="$(printf '%s\n' "$out" | jq -c 'select(.class=="digest")' | grep -c . || true)"
[ "$ndigest" = "1" ] || fail_msg "T2: digest must COALESCE to a single pending entry, got $ndigest"
head="$(printf '%s\n' "$out" | jq -r 'select(.class=="digest") | .locators.head')"
[ "$head" = "bbb" ] || fail_msg "T2: newest digest must REPLACE prior (expected bbb, got $head)"
nactionable="$(printf '%s\n' "$out" | jq -c 'select(.class=="actionable")' | grep -c . || true)"
nhuman="$(printf '%s\n' "$out" | jq -c 'select(.class=="human")' | grep -c . || true)"
[ "$nactionable" = "1" ] || fail_msg "T2: actionable must APPEND (never replaced), got $nactionable"
[ "$nhuman" = "1" ] || fail_msg "T2: human must APPEND / stay durable, got $nhuman"
# Durability never bypassed: all classes present in the durable store.
total="$(printf '%s\n' "$out" | grep -c . || true)"
[ "$total" = "3" ] || fail_msg "T2: durable store should hold digest+actionable+human = 3, got $total"
) && ok
echo "== T3: contiguous-prefix CONSUMED (reject ack N while N-1 unconsumed) =="
(
WAKE_STATE_HOME="$(fresh_state t3)"
export WAKE_STATE_HOME
unset WAKE_AGENT
# Gap: observe seq 1 and 3, but NOT 2.
"$STORE" enqueue --seq 1 --class actionable --locators '{}' >/dev/null
"$STORE" enqueue --seq 3 --class actionable --locators '{}' >/dev/null
if "$STORE" consume --upto 3 >/dev/null 2>&1; then
fail_msg "T3: CONSUMED 3 must be REJECTED while seq 2 is a gap (unconsumed)"
fi
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'consumed_seq=0' || fail_msg "T3: rejected CONSUMED must NOT advance the cursor [$cur]"
# Also reject via the ack wrapper.
if "$ACK" consumed --upto 3 --no-sync >/dev/null 2>&1; then
fail_msg "T3: ack.sh CONSUMED 3 must be REJECTED over a gap"
fi
# Fill the gap; now the contiguous prefix is ackable.
"$STORE" enqueue --seq 2 --class actionable --locators '{}' >/dev/null
"$ACK" consumed --upto 3 --no-sync >/dev/null 2>&1 || fail_msg "T3: CONSUMED 3 should succeed once gap at 2 is filled"
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'consumed_seq=3' || fail_msg "T3: consumed_seq should be 3 after contiguous prefix filled [$cur]"
# Cumulative + can't regress: CONSUMED 2 after 3 is an idempotent no-op.
"$ACK" consumed --upto 2 --no-sync >/dev/null 2>&1 || fail_msg "T3: cumulative CONSUMED 2 (<=3) should be an idempotent success"
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'consumed_seq=3' || fail_msg "T3: cursor must not regress below 3 [$cur]"
) && ok
echo "== T4: wake_id DELIVERY-dedup (dup delivery = re-RECEIVE, never re-action) =="
(
WAKE_STATE_HOME="$(fresh_state t4)"
export WAKE_STATE_HOME
unset WAKE_AGENT
"$STORE" enqueue --seq 1 --class digest --locators '{}' >/dev/null
r1="$("$ACK" received --wake-id WID-abc)"
[ "$r1" = "RECEIVED" ] || fail_msg "T4: first delivery should be RECEIVED, got '$r1'"
r2="$("$ACK" received --wake-id WID-abc)"
[ "$r2" = "DUP" ] || fail_msg "T4: duplicate delivery of same wake_id should be DUP (re-RECEIVE), got '$r2'"
# RECEIVED (delivery) must NEVER advance consumed_seq (delivery != consumption).
cur="$("$STORE" cursors)"
echo "$cur" | grep -q 'consumed_seq=0' || fail_msg "T4: RECEIVED must not advance consumed_seq [$cur]"
) && ok
echo "== T5: atomic write-tmp+rename survives crash mid-write =="
(
WAKE_STATE_HOME="$(fresh_state t5)"
export WAKE_STATE_HOME
unset WAKE_AGENT
"$STORE" enqueue --seq 1 --class actionable --locators '{"issue":1}' >/dev/null
STATE_DIR="$WAKE_STATE_HOME/default"
before="$(cat "$STATE_DIR/pending.jsonl")"
# Simulate a crash mid-write: a leftover atomic-write temp file with GARBAGE
# that was never renamed over the live file.
printf '{"observed_seq": 999, TRUNCATED-GARBAGE' >"$STATE_DIR/.wake.tmp.crash12"
# The live store must be untouched and still valid JSON.
after="$(cat "$STATE_DIR/pending.jsonl")"
[ "$before" = "$after" ] || fail_msg "T5: live pending.jsonl changed by a crash temp file"
drained="$("$STORE" drain)"
printf '%s\n' "$drained" | jq -e . >/dev/null 2>&1 || fail_msg "T5: drain returned non-JSON — garbage temp corrupted the store"
printf '%s\n' "$drained" | stream_any '.observed_seq==1' || fail_msg "T5: committed entry (seq 1) lost after simulated crash"
printf '%s\n' "$drained" | grep -q 999 && fail_msg "T5: uncommitted garbage (seq 999) leaked into the live store"
# A subsequent real mutation must succeed and clean the stale temp.
"$STORE" enqueue --seq 2 --class actionable --locators '{"issue":2}' >/dev/null || fail_msg "T5: enqueue after crash temp failed"
[ -e "$STATE_DIR/.wake.tmp.crash12" ] && fail_msg "T5: stale crash temp not cleaned by next mutation"
d2="$("$STORE" drain)"
[ "$(printf '%s\n' "$d2" | grep -c .)" = "2" ] || fail_msg "T5: store not healthy after crash+recovery (expected 2 entries)"
) && ok
echo "== T5b: _atomic_write is tmp+rename (killing the writer mid-write leaves the target intact) =="
(
WAKE_STATE_HOME="$(fresh_state t5b)"
export WAKE_STATE_HOME
unset WAKE_AGENT
# shellcheck disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
d="$WAKE_STATE_HOME/atomic"
mkdir -p "$d"
target="$d/file"
printf 'GOOD\n' | _atomic_write "$target"
# Feed a writer via a FIFO but NEVER send EOF, then kill it mid-write. With
# true tmp+rename the partial bytes land in a temp file and the rename never
# runs, so `target` keeps its committed content. A direct (non-atomic) write
# would have streamed the garbage straight into `target`.
fifo="$d/fifo"
mkfifo "$fifo"
( _atomic_write "$target" <"$fifo" ) &
wpid=$!
exec 9>"$fifo"
printf 'PARTIAL-GARBAGE-NO-EOF' >&9
sleep 0.3
pkill -9 -P "$wpid" 2>/dev/null || true
kill -9 "$wpid" 2>/dev/null || true
exec 9>&-
wait "$wpid" 2>/dev/null || true
got="$(cat "$target" 2>/dev/null)"
[ "$got" = "GOOD" ] || fail_msg "T5b: target corrupted by a killed mid-write (got '$got') — write is not tmp+rename atomic"
) && ok
echo "== T6: durability survives restart (retain until CONSUMED) =="
(
WAKE_STATE_HOME="$(fresh_state t6)"
export WAKE_STATE_HOME
unset WAKE_AGENT
# "Session 1": enqueue then vanish (no in-memory state carried).
"$STORE" enqueue --seq 1 --class human --locators '{"from":"peer"}' >/dev/null
"$STORE" enqueue --seq 2 --class actionable --locators '{"issue":9}' >/dev/null
# "Session 2": a brand-new process (this subshell invocation of store.sh) must
# see the persisted entries — nothing was held in memory.
d="$("$STORE" drain)"
[ "$(printf '%s\n' "$d" | grep -c .)" = "2" ] || fail_msg "T6: entries not retained across restart (expected 2)"
printf '%s\n' "$d" | stream_any '.class=="human"' || fail_msg "T6: a human message was lost across restart (durability bypassed)"
# Retained UNTIL consumed: after CONSUMED they are released.
"$STORE" consume --upto 2 >/dev/null
d2="$("$STORE" drain)"
n2="$(printf '%s\n' "$d2" | grep -c . || true)"
[ "$n2" = "0" ] || fail_msg "T6: entries should be released only AFTER CONSUMED (still $n2 pending)"
) && ok
echo "== T7: ack local-write NEVER blocks on network =="
(
WAKE_STATE_HOME="$(fresh_state t7)"
export WAKE_STATE_HOME
unset WAKE_AGENT
"$STORE" enqueue --seq 1 --class digest --locators '{}' >/dev/null
# A network-shaped shim on PATH: if the SYNCHRONOUS ack path ever invoked the
# network directly, this marker would appear before ack returns.
bin="$TMP_ROOT/t7bin"
mkdir -p "$bin"
for netcmd in curl wget nc ssh; do
cat >"$bin/$netcmd" <<EOF
#!/usr/bin/env bash
touch "$WAKE_STATE_HOME/NET_CALLED_SYNC"
exit 0
EOF
chmod +x "$bin/$netcmd"
done
# Background sync deliberately SLOW: proves the ack does not wait for it.
export WAKE_ACK_SYNC_CMD="sleep 3; touch '$WAKE_STATE_HOME/SYNC_DONE'"
start="$(date +%s)"
out="$(PATH="$bin:$PATH" "$ACK" consumed --upto 1 --wake-id WID-1)"
end="$(date +%s)"
elapsed=$((end - start))
[ "$elapsed" -lt 2 ] || fail_msg "T7: ack path blocked ${elapsed}s — must return without waiting on the background sync"
echo "$out" | grep -q 'CONSUMED 1' || fail_msg "T7: CONSUMED not reported [$out]"
# Local write is durable IMMEDIATELY (no network needed to record the ack).
STATE_DIR="$WAKE_STATE_HOME/default"
jq_any "$STATE_DIR/ack-ledger.jsonl" '.type=="CONSUMED" and .upto==1' ||
fail_msg "T7: CONSUMED not written to the local ledger synchronously"
consumed_now="$("$STORE" cursors | sed -n 's/consumed_seq=//p')"
[ "$consumed_now" = "1" ] || fail_msg "T7: consumed_seq not advanced by the local ack write (got '$consumed_now')"
# The synchronous path must NOT have touched the network.
[ -e "$WAKE_STATE_HOME/NET_CALLED_SYNC" ] && fail_msg "T7: a network command was invoked on the SYNCHRONOUS ack path"
# And the slow sync must still be in flight (proves it was truly backgrounded).
[ -e "$WAKE_STATE_HOME/SYNC_DONE" ] && fail_msg "T7: background sync finished synchronously — it was not detached"
true
) && ok
echo
if [ -s "$FAILFILE" ]; then
echo "wake store/ack harness: FAILED ($(grep -c . "$FAILFILE") assertion(s))" >&2
exit 1
fi
echo "wake store/ack harness: all invariants passed ($pass groups)"