Files
stack/packages/mosaic/framework/tools/wake/reconcile.sh
T
mos-dt-0andMos 8d1d6e5e76
ci/woodpecker/push/publish Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
feat(wake): #958 A11 preimage.sh — durable provenance for the operator-side preimage definition (#964)
Closes #958.

The preimage definition (source-adapter.sh) is the single most consequential file in the wake pipeline — every observed_hash is a sha256 of what it emits — and it was UNVERSIONED: no git history, no backup. When it was edited at 07:27 on 2026-07-30, attribution was recoverable only because an agent transcript happened to still be on disk. A11 gives that file durable provenance (option (2) of #958: recorded content-addressed, not in-band).

DESIGN, per the pre-registration:
- Provenance is OUT-OF-BAND (never on the adapter's stdout) — an in-band record would advance observed_hash for every source at once and manufacture the re-baseline it exists to explain (B3).
- The obligation never depends on the provenance path: a missing/corrupt store cannot halt the detector or swallow a wake (#940 advisory-fields precedent; B4).
- DESC_FMT=d1 is NOT the provenance record — the tag versions the descriptor FORMAT; a behaviour change that keeps descriptor shape re-baselines every hash and leaves the tag unchanged (B6).

CREDENTIAL HARD GATE (rebuilt after the first verdict FAILED it): byte capture is now RECORD-ONLY BY DEFAULT (extras opt in via WAKE_PREIMAGE_CAPTURE), not allow-by-default-refuse-on-shape — because a shape list can only refuse the secrets someone already enumerated, and the tool's own usage text recommended adding detector.env (where HMAC material lives). Deny is evaluated on BOTH raw and resolved path forms with resolved anchors, ordered before allow — closing the realpath-before-deny ordering defect that let a renamed symlink target through.

VERIFICATION (reviewer, mos-dt, independent of the author's claims):
- Seven decoy cases by planted-marker-then-grep-whole-state-dir: known cred path / same-name symlink / RENAMED-target symlink / prefixed secret / prefixless secret / opted-in-symlink-to-DENIED-target all REFUSED; opted-in-symlink-to-ALLOWED-target CAPTURED (positive control that the harness can capture at all, and that C was not closed by breaking every symlink).
- Polarity-completeness self-test RE-RUN with the shape list stubbed always-allow AND both deny lists stubbed — case D still safe: the flip is complete, the shape list is not load-bearing. Each stub proven live first (a stub that silently fails to apply reports the dangerous state as safe).
- B11: rm-then-change fails LOUD (rc=1), refuses to re-baseline, leaves the ledger absent; absent-with-emptied-objects still first-installs cleanly (absent-is-not-corrupt not paid for by breaking first install).
- RED-first reproduced exactly P13-P16 pre-fix; each refusal corroborated three ways (loud stderr, ledger row captured:false WITH a hash so attribution survives refusal, objects/ holding only the adapter).

KNOWN RESIDUAL (filed #969, non-gating): the deny check is both-forms but the suite needles only the resolved form — a deny reduced to resolved-only survives 17/17 green and would leak a renamed-symlink case. No reachable leak at this head (shipped code correct on all seven decoys); it constrains a FUTURE edit. Doctrine: a both-forms fix needs a needle per form; a fixture that satisfies its assertion through a DIFFERENT rule is testing the rule it did not mean to test.

Authored by pepper (sb-it-1-dt); independently reviewed by mos-dt (sb-it-1-dt) under diff-blind pre-registration (7242688b1, predating first read) — NOT CLEAR on the first verdict (B2/B11 failed by decoy), CLEAR at 8aff7d8 after the polarity rebuild. Manifest version 0.7.0.

Co-authored-by: mos-dt-0 <[email protected]>
2026-07-30 21:48:06 +00:00

584 lines
24 KiB
Bash
Executable File

#!/usr/bin/env bash
# reconcile.sh — A7 of the wake canon (EPIC #892, W5): the source-parity
# reconciler.
#
# CONTRACT ANCHORS (docs/scratchpads/heartbeat-planning/CONVERGED-DESIGN.md):
# §4/G3 Source parity / reconciliation — the GO condition has TWO ordered
# parts:
# (i) a source-coverage PARITY INVENTORY FIRST: a lane-by-lane
# DECLARED inventory of every operational source the lane
# depends on, so an OMITTED source cannot make the §4 vector pass
# VACUOUSLY. An incomplete / undeclared inventory FLAGS (it must
# NOT silently pass).
# (ii) then a periodic full RECONCILE: enumerate declared+configured
# source state vs observed_seq/inbox => 0 UNACCOUNTED. Any source
# state not reflected in observed_seq/inbox is a gap => FLAG.
# §4 vector { source-parity inventory complete AND reconcile = 0 unaccounted }
# §7-res5 FN-oracle + reconciliation are a real, NON-OPTIONAL operating cost.
# W4 review division: the detector's first-seen-baseline does NOT wake; the
# RECONCILER is what enumerates PRE-EXISTING / startup source state into
# the durable store (at startup and periodically). Without it, state
# that already existed when the detector first saw it (baselined
# silently) would never enter the inbox and would be lost.
#
# SCOPE: calls the PUBLIC APIs of store.sh (cursors/drain/enqueue) only. It does
# NOT reimplement or modify the store, ack, digest, signer, or detector. It
# OBSERVES current source state through the SAME operator adapter contract the
# detector uses (WAKE_DETECTOR_SOURCE_CMD <kind> <id>, def on stdin), with the
# SAME fail-loud (G2a) discipline: a source error / ambiguous-empty is never
# silently "no state".
#
# CONTRACT NOTES (flagged, not silently guessed — see PR body):
# * observed_seq DUAL-ALLOCATOR HAZARD — RESOLVED by #908 (single store-side
# allocator). HISTORY: observed_seq once had TWO INDEPENDENT allocators — the
# detector's PRIVATE counter (W4) and the STORE cursor (observed_seq+offset)
# this reconciler used to enumerate from. Because the detector's next seq was
# (private_counter+1), NOT (store_observed_seq+1), the two were decoupled and
# even serializing them did NOT prevent collision: the reconciler enumerating
# alpha->seq1, beta->seq2 while the detector's private counter was still 0
# meant the detector's next delta re-allocated seq1, ALIASING alpha, and a
# consumer acking CONSUMED 1 silently dropped a distinct obligation.
# #908 DISSOLVES this at the root: store.sh enqueue is now the SOLE allocator
# (reads its own observed_seq cursor, next=+1, under an exclusive lock). This
# reconciler now enumerates by calling store.sh enqueue WITHOUT --seq — the
# SAME single allocator the detector uses — so co-feeding one store is SAFE:
# both paths draw distinct, contiguous seqs from the one cursor. The former
# fail-closed refusal (and the reconciler-SOLE-FEEDER assertion) are therefore
# RETIRED; --allow-enumerate / WAKE_RECONCILE_ALLOW_ENUMERATE remain accepted
# as deprecated no-ops for caller compatibility. The G3 accounting invariants
# (0-UNACCOUNTED, vacuous-pass prevention) are unaffected and still enforced.
# * "accounted" is judged against THREE sources: observed_seq/inbox (§4/G3
# wording), the reconciler's OWN durable reconciled-state ledger, and (#932)
# the STORE's last-consumed record (consumed-hashes.jsonl, store-written at
# consume-truncation) — NOT the detector's hash-file. Judging by the detector
# baseline would let a silent first-seen baseline swallow a pre-existing
# obligation (the exact hole G3 closes). The 3rd source suppresses ONLY states
# the store RECORDED as consumed (structurally cannot be hash-advance-without-
# enqueue), so a truly-unaccounted state still re-enumerates (G3 teeth intact).
# * enumeration uses the non-coalescible fail-safe class `actionable` (§2.3)
# so two distinct pre-existing obligations can never coalesce into one.
#
# Operator-agnostic: all state via XDG/env; no operator paths/names/secrets.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=./_wake-common.sh disable=SC1091
. "$SCRIPT_DIR/_wake-common.sh"
STORE_SH="$SCRIPT_DIR/store.sh"
MANIFEST="$SCRIPT_DIR/manifest.txt"
STATE_DIR="$(wake_state_dir)"
# Reconciler-local state (its durable reconciled-state ledger) lives in its own
# subdir under the store's STATE_DIR so it never collides with store/detector.
RECON_DIR="$STATE_DIR/reconciler"
_need_jq() {
command -v jq >/dev/null 2>&1 || {
echo "reconcile.sh: jq is required" >&2
exit 3
}
}
_hash() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum | awk '{print $1}'
elif command -v shasum >/dev/null 2>&1; then
shasum -a 256 | awk '{print $1}'
elif command -v openssl >/dev/null 2>&1; then
openssl dgst -sha256 | awk '{print $NF}'
else
echo "reconcile.sh: no sha256 tool (sha256sum/shasum/openssl) available" >&2
exit 3
fi
}
usage() {
cat >&2 <<'EOF'
Usage: reconcile.sh <command> [options]
Commands:
inventory
Part (i) — source-coverage PARITY INVENTORY. Prints a lane-by-lane
declared inventory and FLAGS any incompleteness that could let the §4
vector pass VACUOUSLY:
* a declared operational source (repos/board_files/lane_anchors) that
NO watch covers (an OMITTED source);
* a watch that references a source with no declared definition (dangling);
* a lane with zero sources; an empty watch-list;
* if a lane declares `required_sources`, any id missing from its sources.
Exit 0 IFF the inventory is complete; non-zero (loud) on any flag.
reconcile [--no-enumerate] [--allow-enumerate]
Part (ii) — periodic full reconcile. Enumerates each covered source's
CURRENT state and compares it against observed_seq/inbox (+ the
reconciler's reconciled-state ledger). Any source state NOT reflected =>
UNACCOUNTED => FLAG. Unless --no-enumerate, each unaccounted source is
ENUMERATED into the durable store (the startup/pre-existing obligation
path). Exit 0 IFF 0 unaccounted were FOUND this pass.
SINGLE STORE-SIDE ALLOCATOR (#908, resolved): enumeration allocates
observed_seq by calling store.sh enqueue WITHOUT --seq — the SAME sole
allocator the detector uses (store cursor +1 under an exclusive lock). A
detector co-feeding this store is therefore SAFE: both draw distinct,
contiguous seqs from the one cursor, so no aliasing is possible. The old
fail-closed refusal is retired; --allow-enumerate (and
WAKE_RECONCILE_ALLOW_ENUMERATE=1) are accepted as deprecated no-ops.
--no-enumerate (and `check`) never write, so they remain side-effect-free.
check
The full §4/G3 gate, side-effect-free: `inventory` AND
`reconcile --no-enumerate`. Exit 0 IFF the inventory is complete AND 0
unaccounted. Use this to evaluate the vector's G3 clause without writing.
Environment:
WAKE_WATCH_LIST Operator watch-list JSON (required).
WAKE_DETECTOR_SOURCE_CMD Source adapter (required for reconcile/check; same
contract as the detector: <cmd> <kind> <id>, def on
stdin; exit!=0 or empty => FAIL LOUD).
WAKE_STATE_HOME/WAKE_AGENT store namespace (see store.sh).
WAKE_RECONCILE_ALLOW_ENUMERATE DEPRECATED no-op (#908 made co-feeding safe via
the single store-side allocator); accepted for
backward compatibility, no longer required.
EOF
}
_manifest_val() {
local key="$1"
[ -f "$MANIFEST" ] || return 0
sed -n "s/^${key}=//p" "$MANIFEST" | head -n1 | tr -d '[:space:]'
}
# _load_watchlist — validate path + JSON + shape + Gate B schema range (mirrors
# the detector's fail-loud validation so the reconciler certifies the SAME
# watch-list the detector runs). Echoes validated JSON on stdout.
_load_watchlist() {
_need_jq
local wl="${WAKE_WATCH_LIST:-}"
if [ -z "$wl" ]; then
echo "reconcile.sh: WAKE_WATCH_LIST is not set (no watch-list to reconcile)" >&2
return 2
fi
if [ ! -f "$wl" ]; then
echo "reconcile.sh: watch-list not found: $wl" >&2
return 2
fi
local json
if ! json="$(jq -e . "$wl" 2>/dev/null)"; then
echo "reconcile.sh: watch-list is not valid JSON: $wl" >&2
return 2
fi
if ! printf '%s' "$json" | jq -e 'has("schema_version") and has("watches")' >/dev/null 2>&1; then
echo "reconcile.sh: watch-list missing required 'schema_version' or 'watches'" >&2
return 2
fi
local ver smin smax
ver="$(printf '%s' "$json" | jq -r '.schema_version')"
smin="$(_manifest_val schema_min)"
smax="$(_manifest_val schema_max)"
case "$ver" in
'' | *[!0-9]*)
echo "reconcile.sh: watch-list schema_version must be an integer (got '$ver')" >&2
return 2
;;
esac
if [ -z "$smin" ] || [ -z "$smax" ]; then
echo "reconcile.sh: manifest is missing schema_min/schema_max (cannot validate Gate B)" >&2
return 2
fi
if [ "$ver" -lt "$smin" ] || [ "$ver" -gt "$smax" ]; then
echo "reconcile.sh: FAIL LOUD (Gate B) — watch-list schema_version $ver is OUTSIDE the supported range [$smin, $smax]" >&2
return 2
fi
printf '%s' "$json"
}
# _coll_for_kind KIND — the top-level collection a source kind is defined in.
_coll_for_kind() {
case "$1" in
repo) printf 'repos' ;;
board_file) printf 'board_files' ;;
lane_anchor) printf 'lane_anchors' ;;
*) return 1 ;;
esac
}
# --- part (i): source-coverage parity inventory ----------------------------
cmd_inventory() {
local json
json="$(_load_watchlist)" || exit $?
local flags=0
# An empty watch-list can never certify coverage of anything.
local nwatch
nwatch="$(printf '%s' "$json" | jq -r '.watches | length')"
if [ "${nwatch:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — watch-list declares NO watches; coverage is vacuous." >&2
flags=$((flags + 1))
fi
echo "# source-coverage parity inventory (lane-by-lane)"
# Per-lane declared inventory + empty-lane + required_sources checks.
local nlanes li
nlanes="$(printf '%s' "$json" | jq -r '.watches | length')"
li=0
while [ "$li" -lt "${nlanes:-0}" ]; do
local lane nsrc
lane="$(printf '%s' "$json" | jq -r --argjson i "$li" '.watches[$i].lane // "(unnamed)"')"
nsrc="$(printf '%s' "$json" | jq -r --argjson i "$li" '.watches[$i].sources | length')"
printf ' lane %s: %s declared source(s)\n' "$lane" "${nsrc:-0}"
if [ "${nsrc:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — lane '$lane' declares ZERO sources (empty coverage)." >&2
flags=$((flags + 1))
fi
# required_sources (optional, additive; ignored by the detector). If the
# operator declares the lane's true dependency set here, every id MUST be
# covered by the lane's sources — an OMITTED dependency FLAGS.
local missing
missing="$(printf '%s' "$json" | jq -r --argjson i "$li" '
(.watches[$i].required_sources // []) as $req
| ($req - [ .watches[$i].sources[].id ]) | .[]')"
if [ -n "$missing" ]; then
local m
while IFS= read -r m; do
[ -n "$m" ] || continue
echo "reconcile.sh: FLAG (G3 parity) — lane '$lane' requires source '$m' but it is OMITTED from the lane's watched sources (vacuous-pass prevented)." >&2
flags=$((flags + 1))
done <<EOF
$missing
EOF
fi
li=$((li + 1))
done
# Dangling: every watched (kind,id) MUST have a declared definition.
local pairs kind id
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
while IFS=$'\t' read -r kind id; do
[ -n "$kind" ] || continue
local coll def
if ! coll="$(_coll_for_kind "$kind")"; then
echo "reconcile.sh: FLAG (G3 parity) — unknown source kind '$kind' in a watch." >&2
flags=$((flags + 1))
continue
fi
def="$(printf '%s' "$json" | jq -c --arg c "$coll" --arg id "$id" '(.[$c] // []) | map(select(.id == $id)) | .[0] // empty')"
if [ -z "$def" ]; then
echo "reconcile.sh: FLAG (G3 parity) — watch references '$kind/$id' but no such entry is declared in '$coll' (dangling)." >&2
flags=$((flags + 1))
fi
done <<EOF
$pairs
EOF
# Omitted-from-coverage: every DECLARED operational source must be covered by
# at least one watch. A declared-but-unwatched source is exactly the OMITTED
# source that would let the vector pass VACUOUSLY (the detector never checks it).
local coll kinds
kinds="repos:repo board_files:board_file lane_anchors:lane_anchor"
for coll in $kinds; do
local cname ckind ids
cname="${coll%%:*}"
ckind="${coll##*:}"
ids="$(printf '%s' "$json" | jq -r --arg c "$cname" '(.[$c] // [])[].id')"
local sid covered
while IFS= read -r sid; do
[ -n "$sid" ] || continue
covered="$(printf '%s' "$json" | jq -r --arg k "$ckind" --arg id "$sid" \
'[ .watches[].sources[] | select(.kind==$k and .id==$id) ] | length')"
if [ "${covered:-0}" -eq 0 ]; then
echo "reconcile.sh: FLAG (G3 parity) — declared source '$ckind/$sid' is NOT covered by any watch (OMITTED from coverage; vacuous-pass prevented)." >&2
flags=$((flags + 1))
fi
done <<EOF
$ids
EOF
done
if [ "$flags" -ne 0 ]; then
echo "reconcile.sh: source-parity inventory INCOMPLETE — $flags flag(s). The §4 vector's G3 clause does NOT pass." >&2
return 1
fi
echo "reconcile.sh: source-parity inventory COMPLETE (every declared source is covered; no dangling/empty/omitted)."
return 0
}
# --- part (ii): full reconcile (0 unaccounted) -----------------------------
# _scope_anchor ANCHOR (content on stdin) — mirror of the detector's anchor
# scoping so the reconciler hashes the SAME region the detector would. Extract
# from the first line containing the anchor up to the next heading (or EOF).
_scope_anchor() {
local anchor="$1"
awk -v a="$anchor" '
index($0, a) > 0 && !inzone { inzone=1; print; next }
inzone && /^#/ { exit }
inzone { print }
'
}
# _recon_key KIND ID — stable per-source ledger key.
_recon_key() {
printf '%s\037%s' "$1" "$2" | _hash
}
# _observe_source KIND ID DEF — observe current scoped state; echo the hash on
# success (rc 0). rc 1 = FAIL LOUD (source error / ambiguous-empty), same G2a
# discipline as the detector: an error is never silently "no state".
_observe_source() {
local kind="$1" id="$2" def="$3"
local anchor raw rc deftmp
anchor="$(printf '%s' "$def" | jq -r '.anchor // empty')"
mkdir -p "$RECON_DIR"
# shellcheck disable=SC2154
deftmp="$(mktemp "$RECON_DIR/${_wake_tmp_prefix}defXXXXXX")" || return 1
printf '%s' "$def" >"$deftmp"
raw="$("$WAKE_DETECTOR_SOURCE_CMD" "$kind" "$id" <"$deftmp" 2>/dev/null)"
rc=$?
rm -f "$deftmp"
if [ "$rc" -ne 0 ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' errored (adapter exit $rc). Cannot certify parity; not treated as 'no state'." >&2
return 1
fi
if [ -z "$raw" ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' returned AMBIGUOUS-EMPTY (empty-that-might-mean-hidden is never 'no state')." >&2
return 1
fi
local scoped
if [ -n "$anchor" ]; then
scoped="$(printf '%s' "$raw" | _scope_anchor "$anchor")"
if [ -z "$scoped" ]; then
echo "reconcile.sh: FAIL LOUD (G2a) — source '$kind/$id' anchor '$anchor' not present (ambiguous section state)." >&2
return 1
fi
else
scoped="$raw"
fi
printf '%s' "$scoped" | _hash
}
# _inbox_has KIND ID HASH — true iff a pending inbox entry reflects this exact
# source state (observed_seq/inbox is the §4/G3 accounting universe).
_inbox_has() {
local kind="$1" id="$2" h="$3" n
n="$("$STORE_SH" drain 2>/dev/null | jq -s --arg k "$kind" --arg id "$id" --arg h "$h" \
'[ .[] | select(.locators.kind==$k and .locators.id==$id and .locators.observed_hash==$h) ] | length' 2>/dev/null || echo 0)"
[ "${n:-0}" -gt 0 ]
}
# _store_consumed_has KIND ID HASH — #932 THIRD accounting source. True iff the
# STORE's last-consumed record (consumed-hashes.jsonl, written by store.sh at
# consume-truncation) records HASH as the last-consumed observed_hash for
# (kind,id). After consume truncates the pending prefix, a consumed state matches
# NEITHER _inbox_has (inbox truncated) NOR the reconciler's own seen-ledger (it
# only covers the reconciler's OWN enumerations) — so without this check the
# reconciler re-enumerates the just-consumed state: one DUPLICATE orientation
# wake + one SPURIOUS rc=1 CRITICAL per detector-active window per cycle (#932).
#
# TRUST BOUNDARY (load-bearing — do NOT violate): this consults ONLY the
# store-written record under STATE_DIR — NEVER a detector-owned hash-file. The
# store record is trustworthy BY CONSTRUCTION: it is written only at CONSUME of a
# durably-enqueued entry, so its existence implies the state WAS durably enqueued,
# and it structurally cannot exhibit the §5 swallow-hole signature (hash-advance-
# WITHOUT-enqueue). Trusting DETECTOR hash-files STAYS REJECTED (it reopens §5).
#
# G3 is NOT weakened: this suppresses ONLY states the store has RECORDED as
# consumed. A genuinely-unaccounted state (durably enqueued but not yet consumed
# — still in the inbox — OR a real gap) has no matching consumed record, so it is
# still UNACCOUNTED and still re-enumerates + alarms.
_store_consumed_has() {
local kind="$1" id="$2" h="$3" n
[ -f "$STATE_DIR/consumed-hashes.jsonl" ] || return 1
n="$(jq -s --arg k "$kind" --arg id "$id" --arg h "$h" \
'[ .[] | select(.kind==$k and .id==$id and .observed_hash==$h) ] | length' \
"$STATE_DIR/consumed-hashes.jsonl" 2>/dev/null || echo 0)"
[ "${n:-0}" -gt 0 ]
}
cmd_reconcile() {
local enumerate=1 allow_enumerate=0
while [ $# -gt 0 ]; do
case "$1" in
--no-enumerate)
enumerate=0
shift
;;
--allow-enumerate)
allow_enumerate=1
shift
;;
*)
echo "reconcile.sh reconcile: unknown option '$1'" >&2
exit 2
;;
esac
done
[ "${WAKE_RECONCILE_ALLOW_ENUMERATE:-0}" = "1" ] && allow_enumerate=1
local json
json="$(_load_watchlist)" || exit $?
if [ -z "${WAKE_DETECTOR_SOURCE_CMD:-}" ]; then
echo "reconcile.sh: WAKE_DETECTOR_SOURCE_CMD is not set (no adapter to observe source state)" >&2
exit 2
fi
"$STORE_SH" init >/dev/null 2>&1 || true
mkdir -p "$RECON_DIR"
# --- SINGLE STORE-SIDE ALLOCATOR (#908) -----------------------------------
# Enumeration allocates observed_seq by calling store.sh enqueue WITHOUT --seq
# (the sole allocator the detector also uses). Co-feeding one store is safe —
# both paths draw distinct, contiguous seqs from the one store cursor under its
# exclusive lock — so the former dual-allocator fail-closed refusal is retired.
# allow_enumerate is now a deprecated no-op, retained only so old callers that
# pass --allow-enumerate / WAKE_RECONCILE_ALLOW_ENUMERATE=1 still work.
: "$allow_enumerate"
local pairs kind id failed=0 unaccounted=0 enumerated=0
# #958 preimage provenance: same pre-step as the detector's poll tick, for
# the path where the preimage changed while the detector was down — without
# it, a changed adapter surfaces here only as N UNACCOUNTED enumerations
# with no first-class cause line. Run BEFORE observing any source so the
# cause entry's observed_seq precedes the enumerations it explains. Loud
# infrastructure failure marks the reconcile failed but does not stop it.
if ! "$SCRIPT_DIR/preimage.sh" check --enqueue; then
echo "reconcile.sh: FAIL LOUD — preimage provenance check failed (see preimage.sh above); reconcile continues but exits non-zero." >&2
failed=1
fi
pairs="$(printf '%s' "$json" | jq -r '[ .watches[].sources[] | "\(.kind)\t\(.id)" ] | unique | .[]')"
if [ -z "$pairs" ]; then
echo "reconcile.sh: watch-list declares no sources under watches[].sources[]" >&2
exit 2
fi
echo "# full reconcile (declared+configured source state vs observed_seq/inbox)"
while IFS=$'\t' read -r kind id; do
[ -n "$kind" ] || continue
local coll def
if ! coll="$(_coll_for_kind "$kind")"; then
echo "reconcile.sh: FAIL LOUD — unknown source kind '$kind' in watch-list" >&2
failed=1
continue
fi
def="$(printf '%s' "$json" | jq -c --arg c "$coll" --arg id "$id" '(.[$c] // []) | map(select(.id == $id)) | .[0] // empty')"
if [ -z "$def" ]; then
echo "reconcile.sh: FAIL LOUD (G3 parity) — watch references '$kind/$id' but no such entry is declared in '$coll'" >&2
failed=1
continue
fi
local curhash key ledger accounted
if ! curhash="$(_observe_source "$kind" "$id" "$def")"; then
failed=1
continue
fi
key="$(_recon_key "$kind" "$id")"
ledger="$RECON_DIR/seen-$key"
accounted=0
# Reflected in the durable inbox (delivered, awaiting consume)?
if _inbox_has "$kind" "$id" "$curhash"; then
accounted=1
# Or already reconciled to this exact state before (durable ledger)?
elif [ -f "$ledger" ] && [ "$(tr -d '[:space:]' <"$ledger")" = "$curhash" ]; then
accounted=1
# Or the STORE recorded this exact state as ALREADY CONSUMED (#932 — the THIRD
# accounting source; a consume-truncated state matches neither the inbox nor
# the seen-ledger, so without this it would spuriously re-enumerate). Consults
# ONLY the store-written record, never a detector hash-file (§5 stays closed).
elif _store_consumed_has "$kind" "$id" "$curhash"; then
accounted=1
fi
if [ "$accounted" -eq 1 ]; then
printf ' %s/%s: ACCOUNTED\n' "$kind" "$id"
continue
fi
# UNACCOUNTED: a source state not reflected in observed_seq/inbox => a gap.
unaccounted=$((unaccounted + 1))
printf ' %s/%s: UNACCOUNTED (current state not reflected in observed_seq/inbox)\n' "$kind" "$id" >&2
if [ "$enumerate" -eq 1 ]; then
# ENUMERATE the pre-existing/startup obligation into the durable store via
# the SINGLE store-side allocator (#908): store.sh enqueue WITHOUT --seq
# allocates the next observed_seq under the store's lock and PRINTS it. Both
# the detector and this reconciler draw from that one cursor, so a co-fed
# store cannot alias. Class = non-coalescible fail-safe `actionable` (§2.3)
# so distinct pre-existing obligations never collapse together.
local seq locators
locators="$(jq -cn --arg kind "$kind" --arg id "$id" --arg hash "$curhash" \
--argjson sdef "$def" \
'{kind:$kind, id:$id, observed_hash:$hash, reconciled:true}
+ ( $sdef | {repo, path, anchor, remote, branches} | with_entries(select(.value != null)) )')"
if seq="$("$STORE_SH" enqueue --class actionable --locators "$locators")"; then
printf '%s' "$curhash" | _atomic_write "$ledger"
enumerated=$((enumerated + 1))
printf ' -> enumerated into store as observed_seq=%s (class=actionable)\n' "$seq"
else
echo "reconcile.sh: enumerate FAILED for '$kind/$id'" >&2
failed=1
fi
fi
done <<EOF
$pairs
EOF
echo "reconcile.sh: UNACCOUNTED=$unaccounted ENUMERATED=$enumerated"
if [ "$failed" -ne 0 ]; then
echo "reconcile.sh: reconcile FAILED (a source could not be observed/enumerated)." >&2
return 1
fi
if [ "$unaccounted" -ne 0 ]; then
if [ "$enumerate" -eq 1 ]; then
echo "reconcile.sh: FLAG — $unaccounted unaccounted source-state(s) found and enumerated. Re-run to confirm 0." >&2
else
echo "reconcile.sh: FLAG — $unaccounted unaccounted source-state(s) (audit mode; nothing enumerated)." >&2
fi
return 1
fi
echo "reconcile.sh: reconcile CLEAN — 0 unaccounted (every declared source state is reflected in observed_seq/inbox)."
return 0
}
cmd_check() {
# The full §4/G3 gate, side-effect-free.
local rc=0
cmd_inventory || rc=1
cmd_reconcile --no-enumerate || rc=1
if [ "$rc" -eq 0 ]; then
echo "reconcile.sh: G3 GATE PASS — inventory complete AND 0 unaccounted."
else
echo "reconcile.sh: G3 GATE FAIL — inventory incomplete OR unaccounted state present." >&2
fi
return "$rc"
}
main() {
[ $# -ge 1 ] || {
usage
exit 2
}
local cmd="$1"
shift
case "$cmd" in
inventory) cmd_inventory "$@" ;;
reconcile) cmd_reconcile "$@" ;;
check) cmd_check "$@" ;;
-h | --help | help) usage ;;
*)
echo "reconcile.sh: unknown command '$cmd'" >&2
usage
exit 2
;;
esac
}
main "$@"