fix(wake): #912 exercise the digest/HMAC trust suite in real CI
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Make the wake digest/HMAC suite RUN and PASS in the real Woodpecker CI
runner (Alpine/musl, root), then hard-require the HMAC legs in CI.

Root cause of the runner-only D1/D4/D5/D6 failures (a masked-local run
passed, so it was runner-specific): a TOOLCHAIN divergence, not locale or
root. digest.sh's _scrub_ctrl used GNU-sed `\xNN` hex-escape byte matching.
The CI runner is node:24-alpine, whose sed is BusyBox — BusyBox sed REJECTS
a `\xNN` character range ("bad regex ... Invalid character range"), aborting
the whole scrub sed and silently VOIDING the scrub. Every scrubbed value
collapsed to empty, cascading into D1 (blank locators), D4 (no scrub/redact,
SHA blanked), D5 (blank agent prefix), D6 (blank [digest] class). Confirmed
by reproducing the exact 9-assertion failure in the ci-base image as root.

Fix (at the correct layer — a wake digest must render identically on any
runner):
- digest.sh _scrub_ctrl: patterns are now LITERAL bytes (printf %b), matching
  byte-identically under GNU sed (glibc dev) and BusyBox sed (Alpine CI).
  Verified identical output on both. Contract preserved: two-tier trust,
  exit-4 hard-locator FAIL-LOUD, secret-scrub, and 40-hex SHA preservation
  all unchanged — deterministic, not weakened.
- test-wake-digest-hmac.sh D4: replaced PCRE `grep -qP` (BusyBox grep has no
  -P; the `&&` silently skipped the check in CI) with portable literal-byte
  `grep -E` ranges (two disjoint bidi/zero-width ranges, excluding legit
  U+2014 em-dash).

CI enablement:
- Dockerfile.ci + .woodpecker/ci.yml test step: add openssl (the non-circular
  HMAC signer) so H1/H2, beacon B12, install I8 can run. The apk add in the
  test step covers PR pipelines before ci-base rebuilds.
- Flip the 3 openssl skip-guards (digest whole-file, beacon B12, install I8)
  to HARD-REQUIRE openssl when CI is set (Woodpecker CI=woodpecker) and FAIL
  loud if absent; KEEP the skip for openssl-less local dev.
- manifest.txt: wake 0.6.2 -> 0.6.3 (digest.sh scrub portability; precedent).

Red-first verified in the ci-base container (root): D4 catches a broken
redaction, H1 catches a tamper that doesn't break the MAC, B12/I8 catch a
corrupted signer.

Closes #912
Part of #892

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
This commit is contained in:
mosaic-coder
2026-07-26 02:49:44 -05:00
parent d967a4a926
commit ef43cc3be8
7 changed files with 99 additions and 18 deletions

View File

@@ -98,6 +98,12 @@ steps:
DATABASE_URL: postgresql://mosaic:mosaic@ci-postgres:5432/mosaic DATABASE_URL: postgresql://mosaic:mosaic@ci-postgres:5432/mosaic
commands: commands:
- *enable_pnpm - *enable_pnpm
# openssl (#912) is the wake HMAC signer: the digest H1/H2, beacon B12,
# and install I8 legs hard-require it in CI. It is baked into ci-base via
# Dockerfile.ci, but ci-base only rebuilds on push-to-main/tag — this
# `apk add` guarantees openssl is present on PR pipelines too (and is a
# fast no-op once the rebuilt image already ships it).
- apk add --no-cache openssl
# postgresql-client (pg_isready) is baked into ci-base. # postgresql-client (pg_isready) is baked into ci-base.
# Wait up to 60s for CI postgres to be ready; fail fast if it never comes up. # Wait up to 60s for CI postgres to be ready; fail fast if it never comes up.
- | - |

View File

@@ -25,7 +25,10 @@ FROM node:24-alpine
# postgresql-client used by the test step's pg_isready readiness probe. `bash`, # postgresql-client used by the test step's pg_isready readiness probe. `bash`,
# `git`, and `jq` are baked here too — framework shell tests and the shipped # `git`, and `jq` are baked here too — framework shell tests and the shipped
# Codex review wrappers require them without per-run installation in ci.yml. # Codex review wrappers require them without per-run installation in ci.yml.
RUN apk add --no-cache python3 make g++ postgresql-client bash git jq # `openssl` (#912) is the non-circular HMAC signer for the wake trust layer:
# the digest H1/H2, beacon B12, and install I8 legs hard-require it in CI so the
# §4 G6 evidence comes from an actually-run HMAC leg, not a skipped one.
RUN apk add --no-cache python3 make g++ postgresql-client bash git jq openssl
# Pin pnpm to the repo's packageManager version via corepack. # Pin pnpm to the repo's packageManager version via corepack.
RUN corepack enable && corepack prepare pnpm@10.6.2 --activate RUN corepack enable && corepack prepare pnpm@10.6.2 --activate

View File

@@ -88,16 +88,37 @@ EOF
# _scrub_ctrl (stdin) — strip ANSI escape sequences, Unicode bidi controls, # _scrub_ctrl (stdin) — strip ANSI escape sequences, Unicode bidi controls,
# zero-width characters, and C0/C1 control bytes. Byte-exact under LC_ALL=C so a # zero-width characters, and C0/C1 control bytes. Byte-exact under LC_ALL=C so a
# multibyte control sequence cannot slip through a locale-dependent class. # multibyte control sequence cannot slip through a locale-dependent class.
#
# PORTABILITY (#912): the byte patterns are LITERAL bytes (materialized via
# `printf %b`), NOT GNU-sed `\xNN` hex escapes. `\xNN` is a GNU-sed extension;
# BusyBox sed (the Alpine/musl CI runner, running as root) REJECTS a `\xNN`
# character range with "bad regex ... Invalid character range", which aborted
# the whole sed and silently VOIDED the scrub in CI — the digest suite's D1/D4/
# D5/D6 all failed only in the Woodpecker runner because every scrubbed value
# collapsed to empty. Literal bytes match identically under GNU sed (glibc dev)
# and BusyBox sed (Alpine CI): a wake digest must render byte-for-byte the same
# regardless of the runner's sed implementation. LC_ALL=C keeps every match
# byte-exact (no locale-dependent multibyte class).
_scrub_ctrl() { _scrub_ctrl() {
local ESC p280 p281 aa ae a6 a9 x8b x8f a0 bom alm
ESC="$(printf '%b' '\x1b')" # U+001B ESC
p280="$(printf '%b' '\xe2\x80')" # UTF-8 lead bytes for U+2000..U+203F
p281="$(printf '%b' '\xe2\x81')" # UTF-8 lead bytes for U+2040..U+207F
aa="$(printf '%b' '\xaa')"; ae="$(printf '%b' '\xae')" # U+202A..U+202E bidi
a6="$(printf '%b' '\xa6')"; a9="$(printf '%b' '\xa9')" # U+2066..U+2069 isolates
x8b="$(printf '%b' '\x8b')"; x8f="$(printf '%b' '\x8f')" # U+200B..U+200F zero-width
a0="$(printf '%b' '\xa0')" # U+2060 word joiner
bom="$(printf '%b' '\xef\xbb\xbf')" # U+FEFF BOM/ZWNBSP
alm="$(printf '%b' '\xd8\x9c')" # U+061C arabic letter mark
LC_ALL=C sed -E \ LC_ALL=C sed -E \
-e 's/\x1b\[[0-9;?]*[ -/]*[@-~]//g' \ -e 's/'"$ESC"'\[[0-9;?]*[ -/]*[@-~]//g' \
-e 's/\x1b[@-Z\\-_]//g' \ -e 's/'"$ESC"'[@-Z\\-_]//g' \
-e 's/\xe2\x80[\xaa-\xae]//g' \ -e 's/'"$p280"'['"$aa"'-'"$ae"']//g' \
-e 's/\xe2\x81[\xa6-\xa9]//g' \ -e 's/'"$p281"'['"$a6"'-'"$a9"']//g' \
-e 's/\xe2\x80[\x8b-\x8f]//g' \ -e 's/'"$p280"'['"$x8b"'-'"$x8f"']//g' \
-e 's/\xe2\x81\xa0//g' \ -e 's/'"$p281$a0"'//g' \
-e 's/\xef\xbb\xbf//g' \ -e 's/'"$bom"'//g' \
-e 's/\xd8\x9c//g' | -e 's/'"$alm"'//g' |
LC_ALL=C tr -d '\000-\010\013\014\016-\037\177' LC_ALL=C tr -d '\000-\010\013\014\016-\037\177'
} }

View File

@@ -45,8 +45,20 @@
# pointer carries a usable (soft) locator instead of rendering # pointer carries a usable (soft) locator instead of rendering
# empty. Display-only: the ACTIONABLE-tier hard-locator FAIL-LOUD # empty. Display-only: the ACTIONABLE-tier hard-locator FAIL-LOUD
# gate (_has_hard_locator, exit 4) is unchanged. # gate (_has_hard_locator, exit 4) is unchanged.
# 0.6.3 #912 digest.sh scrub PORTABILITY (no contract change): _scrub_ctrl's
# control/bidi/zero-width byte patterns are now LITERAL bytes (via
# printf %b) instead of GNU-sed `\xNN` hex escapes. BusyBox sed (the
# Alpine/musl CI runner, running as root) rejects a `\xNN` character
# range, which aborted the whole scrub sed and silently VOIDED the
# scrub in CI — collapsing every scrubbed value to empty and failing
# the digest suite's D1/D4/D5/D6 only in the Woodpecker runner. The
# scrub now renders byte-identically under GNU sed (glibc dev) and
# BusyBox sed (Alpine CI). The two-tier trust, exit-4 hard-locator
# FAIL-LOUD, and the secret-scrub/SHA-preservation contract are all
# unchanged — this makes the existing scrub deterministic across
# runners, it does not weaken it.
component=wake component=wake
version=0.6.2 version=0.6.3
# Watch-list schema this component consumes, and the INCLUSIVE range of # Watch-list schema this component consumes, and the INCLUSIVE range of
# schema_version values it supports. A wake-watch-list.json whose schema_version # schema_version values it supports. A wake-watch-list.json whose schema_version

View File

@@ -296,8 +296,15 @@ echo "== B11: staleness from monitor ingested_ts -> a far-future emit_ts STILL g
) && ok ) && ok
echo "== B12: HMAC-verify at record -> a spoofed (bad-sig) beacon is REJECTED ==" echo "== B12: HMAC-verify at record -> a spoofed (bad-sig) beacon is REJECTED =="
# #912: hard-require openssl in CI (Woodpecker sets CI=woodpecker) so the beacon
# HMAC-verify leg is actually exercised; keep the skip for openssl-less local dev.
if ! command -v openssl >/dev/null 2>&1; then if ! command -v openssl >/dev/null 2>&1; then
echo "SKIP: openssl not available" >&2 if [ -n "${CI:-}" ]; then
echo " FAIL: B12 requires openssl in CI (#912) but it is not on PATH — the CI image must provide it" >&2
echo "x" >>"$FAILFILE"
else
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
fi
else else
( (
H="$(fresh_home b12)" H="$(fresh_home b12)"

View File

@@ -41,11 +41,20 @@ command -v jq >/dev/null 2>&1 || {
echo "SKIP: jq not available" >&2 echo "SKIP: jq not available" >&2
exit 0 exit 0
} }
# NOTE: whole-suite openssl-skip retained; unmasking D1-D6 to run in CI is tracked in #912 (they currently fail under the CI env). # openssl gates the HMAC legs (H1/H2). #912: the wake trust layer MUST be
command -v openssl >/dev/null 2>&1 || { # exercised in real CI, so when running under CI (Woodpecker sets CI=woodpecker)
echo "SKIP: openssl not available" >&2 # openssl is HARD-REQUIRED — a missing openssl fails the suite LOUD rather than
# silently skipping the signer (the §4 G6 evidence must come from an
# actually-run HMAC leg, not a skipped one). In an openssl-less LOCAL DEV env
# the whole suite still skips so `pnpm test` stays runnable without openssl.
if ! command -v openssl >/dev/null 2>&1; then
if [ -n "${CI:-}" ]; then
echo "FATAL (#912): openssl is REQUIRED in CI to exercise the wake digest/HMAC trust layer, but is not on PATH. The CI image must provide openssl (see .woodpecker/ci-image.yml)." >&2
exit 1
fi
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
exit 0 exit 0
} fi
TMP_ROOT="$(mktemp -d)" TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT trap 'rm -rf "$TMP_ROOT"' EXIT
@@ -209,10 +218,26 @@ echo "== D4: SCRUB — secret-canary + ANSI/bidi/zero-width in source free-text
# ANSI escape / CSI must be gone. # ANSI escape / CSI must be gone.
printf '%s' "$out" | LC_ALL=C grep -q "$(printf '\x1b')" && fail_msg "D4: ANSI ESC survived the scrub" printf '%s' "$out" | LC_ALL=C grep -q "$(printf '\x1b')" && fail_msg "D4: ANSI ESC survived the scrub"
# bidi/zero-width/BOM UTF-8 sequences must be gone. # bidi/zero-width/BOM UTF-8 sequences must be gone.
printf '%s' "$out" | LC_ALL=C grep -qP '\xe2\x80[\x8b-\x8f\xaa-\xae]|\xef\xbb\xbf' && # #912: patterns are LITERAL bytes + `grep -E`, NOT PCRE `grep -P`. BusyBox
# grep (Alpine/musl CI) has no `-P` — a `grep -qP` there errors
# ("unrecognized option: P"), returns non-zero, and the `&&` silently skips
# the assertion, so the scrub was NEVER checked in CI. Literal-byte ranges
# under `grep -E` + LC_ALL=C match identically on BusyBox and GNU grep.
# Two DISJOINT byte ranges: U+200B..U+200F (E2 80 8B..8F, zero-width) and
# U+202A..U+202E (E2 80 AA..AE, bidi). NOT a single 8B..AE range — that would
# wrongly flag legitimate E2 80 xx punctuation in between, e.g. U+2014 EM DASH
# (E2 80 94) which the digest body uses.
_b280="$(printf '%b' '\xe2\x80')"
_b8b="$(printf '%b' '\x8b')"; _b8f="$(printf '%b' '\x8f')"
_baa="$(printf '%b' '\xaa')"; _bae="$(printf '%b' '\xae')"
_bbom="$(printf '%b' '\xef\xbb\xbf')"
printf '%s' "$out" | LC_ALL=C grep -qE "${_b280}[${_b8b}-${_b8f}${_baa}-${_bae}]|${_bbom}" &&
fail_msg "D4: bidi/zero-width/BOM survived the scrub" fail_msg "D4: bidi/zero-width/BOM survived the scrub"
# C0 control bytes (except tab/newline) must be gone. # C0 control bytes (except tab/newline) must be gone.
printf '%s' "$out" | LC_ALL=C grep -qP '[\x00-\x08\x0e-\x1f\x7f]' && fail_msg "D4: a C0 control byte survived the scrub" _c00="$(printf '%b' '\x01')"; _c08="$(printf '%b' '\x08')"
_c0e="$(printf '%b' '\x0e')"; _c1f="$(printf '%b' '\x1f')"; _c7f="$(printf '%b' '\x7f')"
printf '%s' "$out" | LC_ALL=C grep -qE "[${_c00}-${_c08}${_c0e}-${_c1f}${_c7f}]" &&
fail_msg "D4: a C0 control byte survived the scrub"
# Secret canaries must be redacted, never inlined. # Secret canaries must be redacted, never inlined.
printf '%s' "$out" | grep -q 'ghp_0123456789' && fail_msg "D4: GitHub-token canary LEAKED into the digest" printf '%s' "$out" | grep -q 'ghp_0123456789' && fail_msg "D4: GitHub-token canary LEAKED into the digest"
printf '%s' "$out" | grep -q 'AKIAIOSFODNN7EXAMPLE' && fail_msg "D4: AWS-key canary LEAKED into the digest" printf '%s' "$out" | grep -q 'AKIAIOSFODNN7EXAMPLE' && fail_msg "D4: AWS-key canary LEAKED into the digest"

View File

@@ -219,8 +219,15 @@ echo "== I7: ingested_ts staleness — a far-future emit_ts still goes stale (re
) && ok ) && ok
echo "== I8: beacon HMAC-verify at record — a spoofed (bad-sig) beacon is REJECTED ==" echo "== I8: beacon HMAC-verify at record — a spoofed (bad-sig) beacon is REJECTED =="
# #912: hard-require openssl in CI (Woodpecker sets CI=woodpecker) so the install
# beacon-sign leg is actually exercised; keep the skip for openssl-less local dev.
if ! command -v openssl >/dev/null 2>&1; then if ! command -v openssl >/dev/null 2>&1; then
echo "SKIP: openssl not available" >&2 if [ -n "${CI:-}" ]; then
echo " FAIL: I8 requires openssl in CI (#912) but it is not on PATH — the CI image must provide it" >&2
echo "x" >>"$FAILFILE"
else
echo "SKIP: openssl not available (local dev; CI hard-requires it)" >&2
fi
else else
( (
H="$(fresh i8)" H="$(fresh i8)"