fix(wake): #912 exercise the digest/HMAC trust suite in real CI
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Make the wake digest/HMAC suite RUN and PASS in the real Woodpecker CI
runner (Alpine/musl, root), then hard-require the HMAC legs in CI.

Root cause of the runner-only D1/D4/D5/D6 failures (a masked-local run
passed, so it was runner-specific): a TOOLCHAIN divergence, not locale or
root. digest.sh's _scrub_ctrl used GNU-sed `\xNN` hex-escape byte matching.
The CI runner is node:24-alpine, whose sed is BusyBox — BusyBox sed REJECTS
a `\xNN` character range ("bad regex ... Invalid character range"), aborting
the whole scrub sed and silently VOIDING the scrub. Every scrubbed value
collapsed to empty, cascading into D1 (blank locators), D4 (no scrub/redact,
SHA blanked), D5 (blank agent prefix), D6 (blank [digest] class). Confirmed
by reproducing the exact 9-assertion failure in the ci-base image as root.

Fix (at the correct layer — a wake digest must render identically on any
runner):
- digest.sh _scrub_ctrl: patterns are now LITERAL bytes (printf %b), matching
  byte-identically under GNU sed (glibc dev) and BusyBox sed (Alpine CI).
  Verified identical output on both. Contract preserved: two-tier trust,
  exit-4 hard-locator FAIL-LOUD, secret-scrub, and 40-hex SHA preservation
  all unchanged — deterministic, not weakened.
- test-wake-digest-hmac.sh D4: replaced PCRE `grep -qP` (BusyBox grep has no
  -P; the `&&` silently skipped the check in CI) with portable literal-byte
  `grep -E` ranges (two disjoint bidi/zero-width ranges, excluding legit
  U+2014 em-dash).

CI enablement:
- Dockerfile.ci + .woodpecker/ci.yml test step: add openssl (the non-circular
  HMAC signer) so H1/H2, beacon B12, install I8 can run. The apk add in the
  test step covers PR pipelines before ci-base rebuilds.
- Flip the 3 openssl skip-guards (digest whole-file, beacon B12, install I8)
  to HARD-REQUIRE openssl when CI is set (Woodpecker CI=woodpecker) and FAIL
  loud if absent; KEEP the skip for openssl-less local dev.
- manifest.txt: wake 0.6.2 -> 0.6.3 (digest.sh scrub portability; precedent).

Red-first verified in the ci-base container (root): D4 catches a broken
redaction, H1 catches a tamper that doesn't break the MAC, B12/I8 catch a
corrupted signer.

Closes #912
Part of #892

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
2026-07-26 02:49:44 -05:00
parent d967a4a926
commit ef43cc3be8
7 changed files with 99 additions and 18 deletions

View File

@@ -88,16 +88,37 @@ EOF
# _scrub_ctrl (stdin) — strip ANSI escape sequences, Unicode bidi controls,
# zero-width characters, and C0/C1 control bytes. Byte-exact under LC_ALL=C so a
# multibyte control sequence cannot slip through a locale-dependent class.
#
# PORTABILITY (#912): the byte patterns are LITERAL bytes (materialized via
# `printf %b`), NOT GNU-sed `\xNN` hex escapes. `\xNN` is a GNU-sed extension;
# BusyBox sed (the Alpine/musl CI runner, running as root) REJECTS a `\xNN`
# character range with "bad regex ... Invalid character range", which aborted
# the whole sed and silently VOIDED the scrub in CI — the digest suite's D1/D4/
# D5/D6 all failed only in the Woodpecker runner because every scrubbed value
# collapsed to empty. Literal bytes match identically under GNU sed (glibc dev)
# and BusyBox sed (Alpine CI): a wake digest must render byte-for-byte the same
# regardless of the runner's sed implementation. LC_ALL=C keeps every match
# byte-exact (no locale-dependent multibyte class).
_scrub_ctrl() {
local ESC p280 p281 aa ae a6 a9 x8b x8f a0 bom alm
ESC="$(printf '%b' '\x1b')" # U+001B ESC
p280="$(printf '%b' '\xe2\x80')" # UTF-8 lead bytes for U+2000..U+203F
p281="$(printf '%b' '\xe2\x81')" # UTF-8 lead bytes for U+2040..U+207F
aa="$(printf '%b' '\xaa')"; ae="$(printf '%b' '\xae')" # U+202A..U+202E bidi
a6="$(printf '%b' '\xa6')"; a9="$(printf '%b' '\xa9')" # U+2066..U+2069 isolates
x8b="$(printf '%b' '\x8b')"; x8f="$(printf '%b' '\x8f')" # U+200B..U+200F zero-width
a0="$(printf '%b' '\xa0')" # U+2060 word joiner
bom="$(printf '%b' '\xef\xbb\xbf')" # U+FEFF BOM/ZWNBSP
alm="$(printf '%b' '\xd8\x9c')" # U+061C arabic letter mark
LC_ALL=C sed -E \
-e 's/\x1b\[[0-9;?]*[ -/]*[@-~]//g' \
-e 's/\x1b[@-Z\\-_]//g' \
-e 's/\xe2\x80[\xaa-\xae]//g' \
-e 's/\xe2\x81[\xa6-\xa9]//g' \
-e 's/\xe2\x80[\x8b-\x8f]//g' \
-e 's/\xe2\x81\xa0//g' \
-e 's/\xef\xbb\xbf//g' \
-e 's/\xd8\x9c//g' |
-e 's/'"$ESC"'\[[0-9;?]*[ -/]*[@-~]//g' \
-e 's/'"$ESC"'[@-Z\\-_]//g' \
-e 's/'"$p280"'['"$aa"'-'"$ae"']//g' \
-e 's/'"$p281"'['"$a6"'-'"$a9"']//g' \
-e 's/'"$p280"'['"$x8b"'-'"$x8f"']//g' \
-e 's/'"$p281$a0"'//g' \
-e 's/'"$bom"'//g' \
-e 's/'"$alm"'//g' |
LC_ALL=C tr -d '\000-\010\013\014\016-\037\177'
}

View File

@@ -45,8 +45,20 @@
# pointer carries a usable (soft) locator instead of rendering
# empty. Display-only: the ACTIONABLE-tier hard-locator FAIL-LOUD
# gate (_has_hard_locator, exit 4) is unchanged.
# 0.6.3 #912 digest.sh scrub PORTABILITY (no contract change): _scrub_ctrl's
# control/bidi/zero-width byte patterns are now LITERAL bytes (via
# printf %b) instead of GNU-sed `\xNN` hex escapes. BusyBox sed (the
# Alpine/musl CI runner, running as root) rejects a `\xNN` character
# range, which aborted the whole scrub sed and silently VOIDED the
# scrub in CI — collapsing every scrubbed value to empty and failing
# the digest suite's D1/D4/D5/D6 only in the Woodpecker runner. The
# scrub now renders byte-identically under GNU sed (glibc dev) and
# BusyBox sed (Alpine CI). The two-tier trust, exit-4 hard-locator
# FAIL-LOUD, and the secret-scrub/SHA-preservation contract are all
# unchanged — this makes the existing scrub deterministic across
# runners, it does not weaken it.
component=wake
version=0.6.2
version=0.6.3
# Watch-list schema this component consumes, and the INCLUSIVE range of
# schema_version values it supports. A wake-watch-list.json whose schema_version

View File

@@ -296,8 +296,15 @@ echo "== B11: staleness from monitor ingested_ts -> a far-future emit_ts STILL g
) && ok
echo "== B12: HMAC-verify at record -> a spoofed (bad-sig) beacon is REJECTED =="
# #912: hard-require openssl in CI (Woodpecker sets CI=woodpecker) so the beacon
# HMAC-verify leg is actually exercised; keep the skip for openssl-less local dev.
if ! command -v openssl >/dev/null 2>&1; then
echo "SKIP: openssl not available" >&2
if [ -n "${CI:-}" ]; then
echo " FAIL: B12 requires openssl in CI (#912) but it is not on PATH — the CI image must provide it" >&2
echo "x" >>"$FAILFILE"
else
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
fi
else
(
H="$(fresh_home b12)"

View File

@@ -41,11 +41,20 @@ command -v jq >/dev/null 2>&1 || {
echo "SKIP: jq not available" >&2
exit 0
}
# NOTE: whole-suite openssl-skip retained; unmasking D1-D6 to run in CI is tracked in #912 (they currently fail under the CI env).
command -v openssl >/dev/null 2>&1 || {
echo "SKIP: openssl not available" >&2
# openssl gates the HMAC legs (H1/H2). #912: the wake trust layer MUST be
# exercised in real CI, so when running under CI (Woodpecker sets CI=woodpecker)
# openssl is HARD-REQUIRED — a missing openssl fails the suite LOUD rather than
# silently skipping the signer (the §4 G6 evidence must come from an
# actually-run HMAC leg, not a skipped one). In an openssl-less LOCAL DEV env
# the whole suite still skips so `pnpm test` stays runnable without openssl.
if ! command -v openssl >/dev/null 2>&1; then
if [ -n "${CI:-}" ]; then
echo "FATAL (#912): openssl is REQUIRED in CI to exercise the wake digest/HMAC trust layer, but is not on PATH. The CI image must provide openssl (see .woodpecker/ci-image.yml)." >&2
exit 1
fi
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
exit 0
}
fi
TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT
@@ -209,10 +218,26 @@ echo "== D4: SCRUB — secret-canary + ANSI/bidi/zero-width in source free-text
# ANSI escape / CSI must be gone.
printf '%s' "$out" | LC_ALL=C grep -q "$(printf '\x1b')" && fail_msg "D4: ANSI ESC survived the scrub"
# bidi/zero-width/BOM UTF-8 sequences must be gone.
printf '%s' "$out" | LC_ALL=C grep -qP '\xe2\x80[\x8b-\x8f\xaa-\xae]|\xef\xbb\xbf' &&
# #912: patterns are LITERAL bytes + `grep -E`, NOT PCRE `grep -P`. BusyBox
# grep (Alpine/musl CI) has no `-P` — a `grep -qP` there errors
# ("unrecognized option: P"), returns non-zero, and the `&&` silently skips
# the assertion, so the scrub was NEVER checked in CI. Literal-byte ranges
# under `grep -E` + LC_ALL=C match identically on BusyBox and GNU grep.
# Two DISJOINT byte ranges: U+200B..U+200F (E2 80 8B..8F, zero-width) and
# U+202A..U+202E (E2 80 AA..AE, bidi). NOT a single 8B..AE range — that would
# wrongly flag legitimate E2 80 xx punctuation in between, e.g. U+2014 EM DASH
# (E2 80 94) which the digest body uses.
_b280="$(printf '%b' '\xe2\x80')"
_b8b="$(printf '%b' '\x8b')"; _b8f="$(printf '%b' '\x8f')"
_baa="$(printf '%b' '\xaa')"; _bae="$(printf '%b' '\xae')"
_bbom="$(printf '%b' '\xef\xbb\xbf')"
printf '%s' "$out" | LC_ALL=C grep -qE "${_b280}[${_b8b}-${_b8f}${_baa}-${_bae}]|${_bbom}" &&
fail_msg "D4: bidi/zero-width/BOM survived the scrub"
# C0 control bytes (except tab/newline) must be gone.
printf '%s' "$out" | LC_ALL=C grep -qP '[\x00-\x08\x0e-\x1f\x7f]' && fail_msg "D4: a C0 control byte survived the scrub"
_c00="$(printf '%b' '\x01')"; _c08="$(printf '%b' '\x08')"
_c0e="$(printf '%b' '\x0e')"; _c1f="$(printf '%b' '\x1f')"; _c7f="$(printf '%b' '\x7f')"
printf '%s' "$out" | LC_ALL=C grep -qE "[${_c00}-${_c08}${_c0e}-${_c1f}${_c7f}]" &&
fail_msg "D4: a C0 control byte survived the scrub"
# Secret canaries must be redacted, never inlined.
printf '%s' "$out" | grep -q 'ghp_0123456789' && fail_msg "D4: GitHub-token canary LEAKED into the digest"
printf '%s' "$out" | grep -q 'AKIAIOSFODNN7EXAMPLE' && fail_msg "D4: AWS-key canary LEAKED into the digest"

View File

@@ -219,8 +219,15 @@ echo "== I7: ingested_ts staleness — a far-future emit_ts still goes stale (re
) && ok
echo "== I8: beacon HMAC-verify at record — a spoofed (bad-sig) beacon is REJECTED =="
# #912: hard-require openssl in CI (Woodpecker sets CI=woodpecker) so the install
# beacon-sign leg is actually exercised; keep the skip for openssl-less local dev.
if ! command -v openssl >/dev/null 2>&1; then
echo "SKIP: openssl not available" >&2
if [ -n "${CI:-}" ]; then
echo " FAIL: I8 requires openssl in CI (#912) but it is not on PATH — the CI image must provide it" >&2
echo "x" >>"$FAILFILE"
else
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
fi
else
(
H="$(fresh i8)"