Hermes Agent
2a2a87251a
ci/woodpecker/pr/ci Pipeline failed
Round-four review, three more absence-driven allows. 1. The guard read the command as TYPED. A backslash before a newline is removed before anything else happens, so an endpoint token split across the join (`.../iss\` + newline + `ues/1/comments`) executed the comments endpoint while the literal token never appeared in the text. Continuations are now joined before every check, because the joined form IS the command. This is the same defect as the split-across-variables case, minus the excuse: there the token genuinely does not exist until the shell expands it, here it was sitting in the input the whole time and the guard chose the wrong reading of it. 2. Transparent prefixes take option VALUES. `sudo -u root curl` hid a live write because `root` was a word the prefix list did not know. Enumerating option grammars per prefix is the wrong game, so what is skipped is an option and at most one value for it, plus a bare duration for `timeout` — never an arbitrary word. `xargs echo curl ...` therefore stays ALLOWED, because there the command is echo and the client is its argument. 3. `find -exec` runs the client. It opens command position the same way an operator does, and now reads that way. All seven reviewer repros are fixtures, each with its counter-case in the allowed direction: a continuation inside a heredoc document stays a document, `xargs echo curl` stays allowed, `sudo apt-get install curl` stays allowed, a prefixed READ stays allowed. 48/48, and the 18-command ordinary sweep still blocks none. Gates: sanitization, resident budget, test enumeration, tools-index (self-test 4/4, git suite 100%), prettier.
143 lines
12 KiB
Bash
Executable File
143 lines
12 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# test-wrapper-guard.sh — hermetic behavioural regression for wrapper-guard.sh.
|
|
#
|
|
# Resolves no credentials, touches no network, and creates no repository: the
|
|
# guard reads a hook payload on stdin and answers with an exit code, so the whole
|
|
# contract is testable from fixtures.
|
|
#
|
|
# The fixtures are written to a temp file rather than passed inline, and this is
|
|
# not stylistic. The guard inspects the literal text of the Bash command it is
|
|
# handed. A test that embeds `git clone ... $HOME` inside its own command line
|
|
# trips the guard on the harness instead of on the fixture — which is exactly
|
|
# what happened the first time this was checked by hand. Substring matching over
|
|
# whole command text is the guard's deliberate fail-closed posture; a test that
|
|
# does not account for it silently measures the wrong thing.
|
|
#
|
|
# Exit: 0 = every fixture behaved as specified · 1 = at least one did not
|
|
|
|
set -uo pipefail
|
|
|
|
HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
|
GUARD="${1:-$HERE/wrapper-guard.sh}"
|
|
[ -x "$GUARD" ] || { printf 'test-wrapper-guard: not executable: %s\n' "$GUARD" >&2; exit 2; }
|
|
|
|
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
|
FIXTURES="$TMP/fixtures.tsv"
|
|
|
|
# Each line: <expected-exit> TAB <hook payload> TAB <what it proves>
|
|
# 0 = allowed, 2 = blocked.
|
|
{
|
|
printf '2\t{"tool_input":{"command":"git clone https://example.invalid/x ~/wt"}}\tcheckout into $HOME is refused\n'
|
|
printf '2\t{"tool_input":{"command":"git worktree add ~/wt topic"}}\tworktree into $HOME is refused\n'
|
|
printf '0\t{"tool_input":{"command":"git clone https://example.invalid/x /src/wt"}}\tcheckout onto a work filesystem is fine\n'
|
|
printf '0\t{"tool_input":{"command":"curl -s -X GET https://git.example.invalid/api/v1/repos/a/b/pulls/1"}}\treads are never blocked\n'
|
|
printf '2\t{"tool_input":{"command":"curl -X POST -d @b https://git.example.invalid/api/v1/repos/a/b/pulls/1/reviews"}}\treview write has a wrapper\n'
|
|
printf '2\t{"tool_input":{"command":"curl -X POST -d @b https://git.example.invalid/api/v1/repos/a/b/pulls/1/merge"}}\tmerge write has a wrapper\n'
|
|
printf '2\t{"tool_input":{"command":"curl -X POST -d @b https://api.github.com/repos/a/b/issues"}}\tGitHub host is covered too\n'
|
|
printf '0\t{"tool_input":{"command":"curl -X POST -d @b https://git.example.invalid/api/v1/repos/a/b/releases"}}\tan endpoint with no wrapper passes\n'
|
|
printf '2\t{"tool_input":{"command":"curl -X POST -d {\\"event\\":\\"APPROVE\\"} https://example.invalid/x"}}\tthe APPROVE token is caught anywhere\n'
|
|
printf '0\t{"tool_input":{"command":"ls -la /src"}}\tordinary commands are untouched\n'
|
|
printf '0\t{"tool_input":{"command":"MOSAIC_WRAPPER_OVERRIDE=1 curl -X POST -d @b https://git.example.invalid/api/v1/repos/a/b/pulls"}}\tbreak-glass works\n'
|
|
printf '0\t{"tool_input":{}}\tan empty payload does not block the session\n'
|
|
# --- bypasses an independent reviewer demonstrated against the first version.
|
|
# Each of these returned 0 (allowed) and each is a real write. They are pinned
|
|
# as fixtures rather than fixed-and-forgotten because the class is recurring:
|
|
# the guard reads text, so every spelling it does not know is a hole.
|
|
printf '2\t{"tool_input":{"command":"curl -d@b https://git.example.invalid/api/v1/repos/a/b/pulls/1/reviews"}}\t-d@body with no space is still a body\n'
|
|
printf '2\t{"tool_input":{"command":"curl --request=POST -d@b https://git.example.invalid/api/v1/repos/a/b/pulls/1/reviews"}}\t--request=POST equals-form is still a method\n'
|
|
printf '2\t{"tool_input":{"command":"p=/api/v1/repo; q=s/a/b/pulls/1/reviews; curl -d@b https://git.example.invalid${p}${q}"}}\ta path split across variables is still that path\n'
|
|
printf '2\t{"tool_input":{"command":"curl --data-binary @b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\t--data-binary is a body\n'
|
|
printf '2\t{"tool_input":{"command":"curl -F f=@b https://git.example.invalid/api/v1/repos/a/b/issues"}}\t-F multipart is a body\n'
|
|
# Reads must survive every one of those broadenings, or the guard gets disabled.
|
|
printf '0\t{"tool_input":{"command":"curl -s https://git.example.invalid/api/v1/repos/a/b/pulls/1/reviews"}}\tno body and no verb is a read\n'
|
|
printf '0\t{"tool_input":{"command":"grep -rn /pulls/ src/ | head -20"}}\ta path fragment in a grep is not an API call\n'
|
|
printf '0\t{"tool_input":{"command":"curl -X POST -d @b https://registry.example.invalid/v2/x/manifests/latest"}}\tan unwrapped API is not this guard'"'"'s business\n'
|
|
# --- round two of the same review. Splitting the ENDPOINT TOKEN defeats any
|
|
# amount of fragment matching, because the endpoint does not exist until the
|
|
# shell expands it. The guard now refuses to clear a write whose URL it cannot
|
|
# read, rather than pretending it read one.
|
|
printf '2\t{"tool_input":{"command":"a=/api/v1/repos/a/b/iss; b=ues/1/comments; curl -d@body https://git.example.invalid${a}${b}"}}\tan endpoint token split across variables is unreadable, not absent\n'
|
|
printf '2\t{"tool_input":{"command":"a=/api/v1/repos/a/b/pu; b=lls/1/reviews; curl -d@body https://git.example.invalid${a}${b}"}}\tsame split, review endpoint\n'
|
|
printf '0\t{"tool_input":{"command":"curl -X POST -d @payload https://hooks.example.invalid/services/${WEBHOOK_ID}"}}\tan opaque URL that is not forge-shaped stays allowed\n'
|
|
# And the other direction, which is the failure mode that gets a hook deleted:
|
|
# discussing a call is not making one. In each of these the client sits behind
|
|
# a quote, never at command position.
|
|
printf '0\t{"tool_input":{"command":"grep -R \\"curl -d https://git.example.invalid/api/v1/repos/a/b/issues\\" docs/"}}\tgrepping for an example is not calling it\n'
|
|
printf '0\t{"tool_input":{"command":"echo \\"curl -d https://git.example.invalid/api/v1/repos/a/b/pulls\\" > note.txt"}}\twriting an example into a file is not calling it\n'
|
|
printf '0\t{"tool_input":{"command":"python3 -c '"'"'print(\\"curl -d https://git.example.invalid/api/v1/repos/a/b/issues\\")'"'"'"}}\tprinting an example is not calling it\n'
|
|
# Command position must still catch the real thing behind operators and env.
|
|
printf '2\t{"tool_input":{"command":"cd /tmp && GITEA_TOKEN=$T curl -d@b https://git.example.invalid/api/v1/repos/a/b/pulls/1/merge"}}\ta real call behind && and an assignment is still a call\n'
|
|
# --- and the case the AUTHOR hit, one level in from the reported one: an
|
|
# operator INSIDE a quoted string is not an operator. This blocked a message
|
|
# that merely quoted the fixture above. Position is judged on the skeleton.
|
|
printf '0\t{"tool_input":{"command":"send.sh -m \\"repro was: cd /tmp && curl -d@b https://git.example.invalid/api/v1/repos/a/b/pulls/1/merge\\""}}\tan operator inside a quoted string is not an operator\n'
|
|
printf '0\t{"tool_input":{"command":"cat >> notes.md <<EOF\\nwe ran: curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues\\nEOF"}}\ta heredoc body is data, not code\n'
|
|
# ...but quotes stop being data the moment something executes them.
|
|
printf '2\t{"tool_input":{"command":"bash -c \\"curl -d@b https://git.example.invalid/api/v1/repos/a/b/pulls/1/merge\\""}}\tbash -c makes the quoted text code again\n'
|
|
# --- round three. The skeleton was right about position and wrong about which
|
|
# words hold it. A word in front of a command does not displace the command:
|
|
# each of these four is a real write that the bare-name match could not see.
|
|
printf '2\t{"tool_input":{"command":"env GITEA_TOKEN=$T curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\tenv VAR=... in front of the client is still the client\n'
|
|
printf '2\t{"tool_input":{"command":"command curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\tcommand in front of the client is still the client\n'
|
|
printf '2\t{"tool_input":{"command":"timeout 10 curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\ttimeout N in front of the client is still the client\n'
|
|
printf '2\t{"tool_input":{"command":"/usr/bin/curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\tan absolute path to the client is still the client\n'
|
|
# ...and the prefix list must stay a list of prefixes. `echo` is not one.
|
|
printf '0\t{"tool_input":{"command":"echo timeout 10 curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments >> notes.md"}}\tnaming a command after echo is not running it\n'
|
|
# A shell standing between quoted data and execution makes that data code,
|
|
# and the pipe is the form agents actually use. Filing it as data allowed the
|
|
# call to vanish from the skeleton while still running.
|
|
printf '2\t{"tool_input":{"command":"printf '"'"'%%s\\\\n'"'"' '"'"'curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments'"'"' | sh"}}\tquoted code piped to a shell is code\n'
|
|
printf '2\t{"tool_input":{"command":"cat <<EOF | sh\\ncurl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments\\nEOF"}}\ta heredoc piped to a shell is code\n'
|
|
printf '2\t{"tool_input":{"command":"sh -s <<EOF\\ncurl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments\\nEOF"}}\tsh -s reads its script from the heredoc\n'
|
|
# ...but a pipe to anything that is not a shell leaves the data as data.
|
|
printf '0\t{"tool_input":{"command":"grep -R \\"curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments\\" docs/ | wc -l"}}\tpiping an example to wc is not executing it\n'
|
|
# ...and a shell on ONE line does not execute a string on another. Switching
|
|
# the whole command into code because it contains an unrelated `sh -c` is how
|
|
# this blocked its author a second time.
|
|
printf '0\t{"tool_input":{"command":"docker run --rm alpine sh -c '"'"'echo hi'"'"'\\necho \\"example: curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments\\" >> notes.md"}}\tan unrelated shell on another line does not promote quoted prose to code\n'
|
|
# --- round four. The guard was still reading the command as typed rather than
|
|
# as the shell will run it: a backslash before a newline is removed before
|
|
# anything else happens, so the endpoint token can be split across the join.
|
|
printf '2\t{"tool_input":{"command":"curl -d@b https://git.example.invalid/api/v1/repos/a/b/iss\\\\\\nues/1/comments"}}\ta line continuation inside the endpoint token is still that endpoint\n'
|
|
printf '2\t{"tool_input":{"command":"curl -d@b https://git.example.invalid/api/v1/repos/a/b/pu\\\\\\nlls/1/reviews"}}\tsame join, review endpoint\n'
|
|
printf '0\t{"tool_input":{"command":"cat >> notes.md <<EOF\\nwe ran: curl -d@b https://git.example.invalid/api/v1/repos/a/b/iss\\\\\\nues/1/comments\\nEOF"}}\tjoining lines does not turn a document into a command\n'
|
|
# Transparent prefixes take option VALUES, and the value was a word the list
|
|
# did not know — so the client went missing again behind an ordinary `sudo -u`.
|
|
printf '2\t{"tool_input":{"command":"sudo -u root curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\tan option value after a prefix does not hide the client\n'
|
|
printf '2\t{"tool_input":{"command":"timeout --signal TERM 10 curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\tan option pair plus a duration does not hide the client\n'
|
|
# ...but only an OPTION and its value are skipped, never any word: in
|
|
# `xargs echo curl ...` the command is echo and the client is its argument.
|
|
printf '0\t{"tool_input":{"command":"xargs echo curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments"}}\ta client passed as an argument to echo is not a call\n'
|
|
printf '0\t{"tool_input":{"command":"sudo apt-get install curl"}}\tinstalling the client is not calling it\n'
|
|
# Execution through another command still opens command position.
|
|
printf '2\t{"tool_input":{"command":"find . -maxdepth 0 -exec curl -d@b https://git.example.invalid/api/v1/repos/a/b/issues/1/comments ;"}}\tfind -exec runs the client\n'
|
|
} > "$FIXTURES"
|
|
|
|
fail=0 n=0
|
|
while IFS=$'\t' read -r want payload why; do
|
|
[ -n "${want:-}" ] || continue
|
|
n=$((n + 1))
|
|
printf '%s' "$payload" | "$GUARD" >/dev/null 2>&1
|
|
got=$?
|
|
if [ "$got" = "$want" ]; then
|
|
printf 'ok %s\n' "$why"
|
|
else
|
|
printf 'FAIL %s (want exit %s, got %s)\n' "$why" "$want" "$got"
|
|
fail=1
|
|
fi
|
|
done < "$FIXTURES"
|
|
|
|
printf '\n'
|
|
if [ "$fail" -eq 0 ]; then
|
|
printf 'wrapper-guard: %d/%d fixtures behaved as specified.\n' "$n" "$n"
|
|
else
|
|
cat <<'EOF'
|
|
wrapper-guard drifted from its contract.
|
|
|
|
A guard that blocks too much gets routed around, and a guard that blocks too
|
|
little is decoration. Both directions are failures here, which is why the
|
|
allowed cases are asserted as hard as the blocked ones.
|
|
EOF
|
|
fi
|
|
exit "$fail"
|