Files
stack/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh
Hermes Agent 16481ece3d
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
fix(tools): attribute read-back to acting identity and paginate fully (#865)
Round 2 remediation for the read-back verification in issue-comment.sh and
pr-review.sh.

BLOCKER A (invocation attribution): id-above-boundary + content/state match
only proves temporal ordering — a concurrent write from a different identity
could satisfy it while this tea invocation created nothing. Both wrappers now
resolve the acting identity once via curl GET /api/v1/user and additionally
require the accepted record's author login to equal that identity. Residual
same-identity same-body/state concurrency is documented in-code (tea 0.11.1
emits no reliable created-record id to close it further).

BLOCKER B (pagination): the comments and reviews list reads now walk every
page (?limit=&page=1,2,… until a short/empty page) for both the pre-write
boundary and the post-write read-back, so a record beyond page 1 is still
found.

Adds regressions: concurrent different-identity write fails closed (comments
and reviews); a matching review beyond page 1 is still found. README updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:02:25 -05:00

266 lines
9.5 KiB
Bash
Executable File

#!/usr/bin/env bash
# Regression harness for issue-comment.sh top-level `tea comment` invocation and
# its BOUNDED, INVOCATION-ATTRIBUTED, PAGINATED read-back verification (#865).
#
# The #865 bug: `tea issue comment ...` (a nonexistent subcommand on tea
# v0.11.1) silently no-ops and exits 0, so a comment is never posted. A naive
# read-back that matches ANY historical comment by body would falsely report
# success whenever an identically-bodied comment already exists from a prior
# run. Merely bounding by "id > pre-write max" is also insufficient: it accepts
# ANY newer matching comment, including one a CONCURRENT DIFFERENT identity
# posted while this tea invocation created nothing. This harness proves the
# wrapper:
# 1. uses the top-level `tea comment` form (never `tea issue comment`);
# 2. records the pre-write maximum comment id as a boundary and requires a
# strictly-newer comment on read-back, so a pre-existing identical body
# does NOT satisfy verification (fails closed);
# 3. attributes the matched comment to the acting identity (GET /user login),
# so a concurrent DIFFERENT-identity write does NOT satisfy verification;
# 4. reports success only when a genuinely new comment (id > boundary) with
# the exact body AND the acting author appears.
#
# The `tea` stub NEVER creates a comment (it mimics the silent no-op); the
# "server" comment state is modeled entirely by the curl stub's responses, so
# the fresh-success vs. no-op distinction is driven purely by whether the
# post-write read-back surfaces a new, correctly-attributed id.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/issue-comment-readback}"
REPO_DIR="$WORK_DIR/repo"
BIN_DIR="$WORK_DIR/bin"
TEA_LOG="$WORK_DIR/tea.log"
CURL_LOG="$WORK_DIR/curl.log"
OUTPUT_FILE="$WORK_DIR/output.log"
CREDENTIALS_FILE="$WORK_DIR/credentials.json"
CALLS_FILE="$WORK_DIR/comment_calls"
cleanup() {
rm -rf "$WORK_DIR"
}
trap cleanup EXIT
mkdir -p "$REPO_DIR" "$BIN_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git
ISSUE_NUMBER=7
API_BASE="https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack"
API_ROOT="https://git.mosaicstack.dev/api/v1"
BODY='durable "note" -- marker'
ACTING_LOGIN="primary-reviewer"
FOREIGN_LOGIN="other-writer"
CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY'
import json
import os
import sys
with open(sys.argv[1], "w", encoding="utf-8") as credentials:
json.dump({
"gitea": {
"mosaicstack": {
"url": os.environ["CONFIGURED_GITEA_URL"],
"token": "test-only-placeholder",
}
}
}, credentials)
PY
# tea stub: resolves the login list, and treats `tea comment ...` as a silent
# no-op (exit 0 without creating anything) to mimic the real failure mode.
cat > "$BIN_DIR/tea" <<'SH'
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' "$*" >> "$ISSUE_COMMENT_TEA_LOG"
if [[ "$*" == "login list --output json" ]]; then
printf '%s\n' '[{"name":"mosaicstack","url":"https://git.mosaicstack.dev"}]'
exit 0
fi
# The wrapper must use the TOP-LEVEL `tea comment` form; the broken
# `tea issue comment` subcommand must never be invoked.
if [[ "$*" == issue\ comment* ]]; then
echo "wrapper invoked nonexistent 'tea issue comment' subcommand" >&2
exit 90
fi
if [[ "$*" == comment\ * ]]; then
# Mimic tea v0.11.1: exit 0. Whether a comment actually lands is modeled
# by the curl stub's post-write read-back response, not here.
exit 0
fi
echo "Unexpected tea command: $*" >&2
exit 92
SH
chmod +x "$BIN_DIR/tea"
# curl stub: serves GET /user (acting identity) and GET .../issues/7/comments
# (paginated: ?limit=&page=). The comments endpoint's first call = pre-write
# boundary, second call = post-write read-back. The boundary always contains a
# pre-existing comment (id 50) whose body is IDENTICAL to the one under test,
# which is exactly the condition a body-only match would trip over.
cat > "$BIN_DIR/curl" <<'SH'
#!/usr/bin/env bash
set -euo pipefail
output_file=""
method="GET"
url=""
while [[ $# -gt 0 ]]; do
case "$1" in
-o) output_file="$2"; shift 2 ;;
-w|-H) shift 2 ;;
-X) method="$2"; shift 2 ;;
-d|--data) shift 2 ;;
-s|-S|-sS) shift ;;
http://*|https://*) url="$1"; shift ;;
*) shift ;;
esac
done
# Strip any query string so pagination params don't defeat path matching, but
# still log the full URL (including ?limit=&page=) so the test can assert the
# read-back paginated.
path="${url%%\?*}"
printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG"
write_response() {
local status="$1" body="$2"
[[ -n "$output_file" ]] || exit 96
printf '%s' "$body" > "$output_file"
printf '%s' "$status"
}
if [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_ROOT/user" ]]; then
write_response 200 "$(ISSUE_COMMENT_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY'
import json
import os
print(json.dumps({"login": os.environ["ISSUE_COMMENT_LOGIN"]}))
PY
)"
elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then
calls_file="$ISSUE_COMMENT_CALLS"
if [[ -f "$calls_file" ]]; then
# post-write read-back
response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \
ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" \
ISSUE_COMMENT_FOREIGN_LOGIN="$ISSUE_COMMENT_FOREIGN_LOGIN" \
ISSUE_COMMENT_TEST_MODE="$ISSUE_COMMENT_TEST_MODE" python3 - <<'PY'
import json
import os
body = os.environ["ISSUE_COMMENT_BODY"]
acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"]
foreign = os.environ["ISSUE_COMMENT_FOREIGN_LOGIN"]
mode = os.environ["ISSUE_COMMENT_TEST_MODE"]
if mode == "fresh-success":
# A genuinely new comment (id 60 > boundary 50) authored by the acting
# identity.
records = [
{"id": 50, "body": body, "user": {"login": acting}},
{"id": 60, "body": body, "user": {"login": acting}},
]
elif mode == "foreign-identity":
# A concurrent new comment (id 60 > boundary 50) with the SAME body but a
# DIFFERENT author. tea created nothing; attribution must reject this.
records = [
{"id": 50, "body": body, "user": {"login": acting}},
{"id": 60, "body": body, "user": {"login": foreign}},
]
else:
# no-op: nothing new landed; the pre-existing id-50 comment remains.
records = [{"id": 50, "body": body, "user": {"login": acting}}]
print(json.dumps(records))
PY
)
else
: > "$calls_file"
response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \
ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY'
import json
import os
body = os.environ["ISSUE_COMMENT_BODY"]
acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"]
print(json.dumps([{"id": 50, "body": body, "user": {"login": acting}}]))
PY
)
fi
write_response 200 "$response"
else
echo "Unexpected curl request: $method $url" >&2
exit 97
fi
SH
chmod +x "$BIN_DIR/curl"
run_comment() {
local mode="$1"
: > "$TEA_LOG"
: > "$CURL_LOG"
: > "$OUTPUT_FILE"
rm -f "$CALLS_FILE"
(
cd "$REPO_DIR"
PATH="$BIN_DIR:$PATH" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \
ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \
ISSUE_COMMENT_CALLS="$CALLS_FILE" \
ISSUE_COMMENT_TEST_MODE="$mode" \
ISSUE_COMMENT_EXPECTED_BODY="$BODY" \
ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \
ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \
ISSUE_COMMENT_API_BASE="$API_BASE" \
ISSUE_COMMENT_API_ROOT="$API_ROOT" \
"$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY"
) > "$OUTPUT_FILE" 2>&1
}
# Case 1: silent no-op with a pre-existing identical body must FAIL CLOSED.
if run_comment noop-preexisting; then
echo "FAIL: wrapper reported success when tea no-opped but an identical body pre-existed" >&2
cat "$OUTPUT_FILE" >&2
exit 1
fi
if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then
echo "FAIL: read-back matched a pre-existing comment by body only" >&2
exit 1
fi
# The wrapper must have used the top-level form and read comments back twice
# (boundary + post-write).
grep -q "^comment 7 " "$TEA_LOG"
if grep -q '^issue comment' "$TEA_LOG"; then
echo "FAIL: wrapper used the broken 'tea issue comment' subcommand" >&2
exit 1
fi
[[ "$(grep -c "^GET $API_BASE/issues/7/comments?" "$CURL_LOG")" == "2" ]]
# Read-back must be paginated (limit + page query params present).
grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=1$" "$CURL_LOG"
# Attribution must have resolved the acting identity via GET /user.
grep -q "^GET $API_ROOT/user$" "$CURL_LOG"
# Case 2: a concurrent DIFFERENT-identity write (id 60 > boundary, same body,
# foreign author) must FAIL CLOSED — temporal ordering is not attribution.
if run_comment foreign-identity; then
echo "FAIL: wrapper accepted a concurrent comment authored by a different identity" >&2
cat "$OUTPUT_FILE" >&2
exit 1
fi
if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then
echo "FAIL: read-back matched a different-identity comment (attribution bypassed)" >&2
exit 1
fi
# Case 3: a genuinely new comment (id 60 > boundary 50, acting author) verifies.
run_comment fresh-success
grep -q 'Added and verified comment on Gitea issue #7 (comment ID 60)' "$OUTPUT_FILE"
grep -q "^comment 7 " "$TEA_LOG"
echo "issue-comment.sh bounded + attributed read-back regression passed"