Files
stack/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh
Hermes Agent 9384f0bc0a
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
fix(tools): write Gitea reviews/comments via REST POST and verify by exact created id (#865)
Replace the tea-based write + boundary/author read-back with a direct Gitea
REST POST that returns the created record's id, and verify that exact record.

BLOCKER 2 (credential ordering): resolve the acting identity, the write token,
and the read-back token from the SAME effective login. A --login override now
selects the credential used for the POST, GET /user, and the GET-by-id
read-back, so an overridden write is verified against the identity that
performed it -- not the host default. Login-name resolution is best-effort and
non-fatal (the override always wins; otherwise fall back to the host
credential), so exotic/ported hosts still resolve a token.

BLOCKER 1+3 (attribution + tautological tests): the write is now
POST /issues/{n}/comments or POST /pulls/{n}/reviews (event + body + commit_id
== PR head), parsing the provider-returned created id. Verification GETs that
exact id and checks author == acting identity and body (comments) or state +
commit_id (reviews). Keying on the created id closes the concurrency window:
a no-op create yields no id and fails closed with no list-scan fallback, and a
concurrent same-identity record has a different id. The review body travels in
the review submit, removing the separate detached comment.

Tests: the curl stub now models a real server with persistent on-disk
review/comment state -- a POST actually creates+persists a record and returns
its id, and the read-back reads that same state (no fabricated record for the
wrapper to find). Adds same-identity no-op-concurrent and author-mismatch
fail-closed cases for both comments and reviews, and >page-1 pagination
coverage for both. README "Durable review provenance" refreshed for the REST
mechanism.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:46:32 -05:00

332 lines
12 KiB
Bash
Executable File

#!/usr/bin/env bash
# Regression harness for issue-comment.sh's Gitea comment write + verification
# (#865).
#
# The #865 defect class: tea 0.11.1's `tea issue comment ...` (a nonexistent
# subcommand) silently no-ops yet exits 0, and tea cannot emit the id of a
# record it created — so an exit code is worthless as proof of a durable write.
# The wrapper therefore does NOT write via tea at all. It POSTs the comment to
# the Gitea REST API (which returns the created comment object, including its
# id), then GETs THAT EXACT id back and requires it to match on id, author
# (acting identity), body, and issue. Because verification is keyed to the id
# the create returned, no concurrent comment can masquerade as this write, and a
# suppressed/no-op create yields no id and fails closed.
#
# This harness models a REAL server: the curl stub keeps persistent comment
# state on disk, the POST actually CREATES and PERSISTS a record and returns its
# id, and the read-back GET reads that same state. There is no independently
# fabricated record for the wrapper to "find" — the only way verification
# passes is if the POST genuinely created the record the read-back retrieves.
# It proves the wrapper:
# 1. never shells out to tea to write (no `tea comment` / `tea issue comment`);
# 2. creates the comment via REST POST and learns the provider-returned id;
# 3. verifies THAT EXACT id by direct GET, attributed to the acting identity;
# 4. fails closed when the write is a no-op even though a concurrent
# SAME-IDENTITY comment with the same body already exists (the closed
# concurrency window — no fallback list scan can rescue a no-op);
# 5. fails closed when the created record is not authored by the acting
# identity;
# 6. enumerates the created id in the issue's FULLY PAGINATED comment list,
# finding it even when it lands beyond page 1.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/issue-comment-readback}"
REPO_DIR="$WORK_DIR/repo"
BIN_DIR="$WORK_DIR/bin"
XDG_DIR="$WORK_DIR/xdg"
TEA_LOG="$WORK_DIR/tea.log"
CURL_LOG="$WORK_DIR/curl.log"
OUTPUT_FILE="$WORK_DIR/output.log"
CREDENTIALS_FILE="$WORK_DIR/credentials.json"
STATE_FILE="$WORK_DIR/comments.json"
cleanup() {
rm -rf "$WORK_DIR"
}
trap cleanup EXIT
mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR"
git -C "$REPO_DIR" init -q
git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git
ISSUE_NUMBER=7
REPO_SLUG="mosaicstack/stack"
API_BASE="https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack"
API_ROOT="https://git.mosaicstack.dev/api/v1"
BODY='durable "note" -- marker'
ACTING_LOGIN="primary-reviewer"
FOREIGN_LOGIN="other-writer"
CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY'
import json
import os
import sys
with open(sys.argv[1], "w", encoding="utf-8") as credentials:
json.dump({
"gitea": {
"mosaicstack": {
"url": os.environ["CONFIGURED_GITEA_URL"],
"token": "test-only-placeholder",
}
}
}, credentials)
PY
# tea stub: only ever answers the login list (used to resolve the default login
# name). It must NEVER be asked to write a comment — the wrapper writes via REST.
cat > "$BIN_DIR/tea" <<'SH'
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' "$*" >> "$ISSUE_COMMENT_TEA_LOG"
if [[ "$*" == "login list --output json" ]]; then
printf '%s\n' '[{"name":"mosaicstack","url":"https://git.mosaicstack.dev"}]'
exit 0
fi
echo "Unexpected tea command (wrapper must not write via tea): $*" >&2
exit 92
SH
chmod +x "$BIN_DIR/tea"
# curl stub: a small REST server backed by persistent on-disk comment state.
# GET /user -> acting identity
# POST /issues/7/comments -> CREATE + PERSIST, return created object
# GET /issues/comments/{id} -> read the persisted record by exact id
# GET /issues/7/comments?page=&.. -> paginated listing of persisted state
cat > "$BIN_DIR/curl" <<'SH'
#!/usr/bin/env bash
set -euo pipefail
output_file=""
method="GET"
url=""
data=""
while [[ $# -gt 0 ]]; do
case "$1" in
-o) output_file="$2"; shift 2 ;;
-w|-H) shift 2 ;;
-X) method="$2"; shift 2 ;;
-d|--data) data="$2"; shift 2 ;;
-s|-S|-sS) shift ;;
http://*|https://*) url="$1"; shift ;;
*) shift ;;
esac
done
path="${url%%\?*}"
query="${url#*\?}"
[[ "$query" == "$url" ]] && query=""
printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG"
write_response() {
local status="$1" body="$2"
[[ -n "$output_file" ]] || exit 96
printf '%s' "$body" > "$output_file"
printf '%s' "$status"
}
if [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_ROOT/user" ]]; then
write_response 200 "$(ISSUE_COMMENT_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY'
import json
import os
print(json.dumps({"login": os.environ["ISSUE_COMMENT_LOGIN"]}))
PY
)"
elif [[ "$method" == "POST" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then
result=$(ISSUE_COMMENT_DATA="$data" python3 - <<'PY'
import json
import os
state_path = os.environ["ISSUE_COMMENT_STATE"]
mode = os.environ["ISSUE_COMMENT_TEST_MODE"]
acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"]
foreign = os.environ["ISSUE_COMMENT_FOREIGN_LOGIN"]
repo = os.environ["ISSUE_COMMENT_REPO_SLUG"]
body = json.loads(os.environ["ISSUE_COMMENT_DATA"]).get("body")
with open(state_path, encoding="utf-8") as handle:
comments = json.load(handle)
# no-op-concurrent: the wrapper's own write is SUPPRESSED (returns 200 with no
# created object) even though a concurrent same-identity comment already exists
# in state. Nothing is persisted; there is no created id to verify.
if mode == "no-op-concurrent":
print("200")
print(json.dumps({}))
raise SystemExit(0)
author = foreign if mode == "author-mismatch" else acting
new_id = (max((c["id"] for c in comments), default=0)) + 1
record = {
"id": new_id,
"body": body,
"user": {"login": author},
"issue_url": f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7",
}
comments.append(record)
with open(state_path, "w", encoding="utf-8") as handle:
json.dump(comments, handle)
print("201")
print(json.dumps(record))
PY
)
write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)"
elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE"/issues/comments/* ]]; then
result=$(ISSUE_COMMENT_GET_ID="${path##*/}" python3 - <<'PY'
import json
import os
state_path = os.environ["ISSUE_COMMENT_STATE"]
wanted = int(os.environ["ISSUE_COMMENT_GET_ID"])
with open(state_path, encoding="utf-8") as handle:
comments = json.load(handle)
match = next((c for c in comments if c["id"] == wanted), None)
if match is None:
print("404")
print(json.dumps({"message": "not found"}))
else:
print("200")
print(json.dumps(match))
PY
)
write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)"
elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then
result=$(ISSUE_COMMENT_QUERY="$query" python3 - <<'PY'
import json
import os
from urllib.parse import parse_qs
state_path = os.environ["ISSUE_COMMENT_STATE"]
params = parse_qs(os.environ["ISSUE_COMMENT_QUERY"])
limit = int(params.get("limit", ["50"])[0])
page = int(params.get("page", ["1"])[0])
with open(state_path, encoding="utf-8") as handle:
comments = json.load(handle)
start = (page - 1) * limit
print("200")
print(json.dumps(comments[start:start + limit]))
PY
)
write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)"
else
echo "Unexpected curl request: $method $url" >&2
exit 97
fi
SH
chmod +x "$BIN_DIR/curl"
# Seed persistent server state for a mode, then run the wrapper against it.
seed_state() {
local mode="$1"
ISSUE_COMMENT_SEED_MODE="$mode" ISSUE_COMMENT_SEED_BODY="$BODY" \
ISSUE_COMMENT_SEED_ACTING="$ACTING_LOGIN" ISSUE_COMMENT_SEED_REPO="$REPO_SLUG" \
python3 - "$STATE_FILE" <<'PY'
import json
import os
import sys
mode = os.environ["ISSUE_COMMENT_SEED_MODE"]
body = os.environ["ISSUE_COMMENT_SEED_BODY"]
acting = os.environ["ISSUE_COMMENT_SEED_ACTING"]
repo = os.environ["ISSUE_COMMENT_SEED_REPO"]
issue_url = f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7"
if mode == "fresh-success":
# 50 pre-existing comments fill page 1 (limit 50); the comment this run
# creates becomes id 51 and lands ALONE on page 2, exercising >page-1
# pagination in the enumeration check.
comments = [
{"id": i, "body": f"prior {i}", "user": {"login": acting}, "issue_url": issue_url}
for i in range(1, 51)
]
elif mode == "no-op-concurrent":
# A concurrent SAME-IDENTITY comment with the IDENTICAL body already exists.
# The wrapper's own write will be a no-op; it must still fail closed because
# no created id is returned — it must not scan and accept this record.
comments = [
{"id": 55, "body": body, "user": {"login": acting}, "issue_url": issue_url}
]
else: # author-mismatch
comments = []
with open(sys.argv[1], "w", encoding="utf-8") as handle:
json.dump(comments, handle)
PY
}
run_comment() {
local mode="$1"
: > "$TEA_LOG"
: > "$CURL_LOG"
: > "$OUTPUT_FILE"
seed_state "$mode"
(
cd "$REPO_DIR"
PATH="$BIN_DIR:$PATH" \
XDG_CONFIG_HOME="$XDG_DIR" \
MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \
ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \
ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \
ISSUE_COMMENT_STATE="$STATE_FILE" \
ISSUE_COMMENT_TEST_MODE="$mode" \
ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \
ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \
ISSUE_COMMENT_REPO_SLUG="$REPO_SLUG" \
ISSUE_COMMENT_API_BASE="$API_BASE" \
ISSUE_COMMENT_API_ROOT="$API_ROOT" \
"$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY"
) > "$OUTPUT_FILE" 2>&1
}
# Case 1: a genuine REST create (id 51) is verified end to end via its exact
# provider-returned id and enumerated on page 2 of the paginated listing.
run_comment fresh-success
grep -q 'Added and verified comment on Gitea issue #7 (comment ID 51)' "$OUTPUT_FILE"
# The write is a REST POST, never a tea comment.
grep -q "^POST $API_BASE/issues/7/comments$" "$CURL_LOG"
if grep -Eq '^comment |^issue comment ' "$TEA_LOG"; then
echo "FAIL: wrapper wrote a comment via tea instead of REST" >&2
exit 1
fi
# Read-back is a DIRECT GET of the exact created id.
grep -q "^GET $API_BASE/issues/comments/51$" "$CURL_LOG"
# Acting identity resolved via GET /user.
grep -q "^GET $API_ROOT/user$" "$CURL_LOG"
# Enumeration paginated beyond page 1 to find the created comment.
grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=2$" "$CURL_LOG"
# Case 2: a no-op write with a concurrent SAME-IDENTITY, same-body comment
# already present must FAIL CLOSED — the closed concurrency window.
if run_comment no-op-concurrent; then
echo "FAIL: wrapper reported success when its write no-opped but a concurrent same-identity comment existed" >&2
cat "$OUTPUT_FILE" >&2
exit 1
fi
if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then
echo "FAIL: wrapper accepted a concurrent record for a no-op write (window not closed)" >&2
exit 1
fi
# It must NOT have fallen back to a list scan that could find the concurrent id.
if grep -q "^GET $API_BASE/issues/comments/55$" "$CURL_LOG"; then
echo "FAIL: wrapper read back the concurrent comment id 55 (illegitimate fallback)" >&2
exit 1
fi
# Case 3: a created record NOT authored by the acting identity must FAIL CLOSED.
if run_comment author-mismatch; then
echo "FAIL: wrapper accepted a created comment authored by a different identity" >&2
cat "$OUTPUT_FILE" >&2
exit 1
fi
if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then
echo "FAIL: read-back did not enforce acting-identity authorship" >&2
exit 1
fi
echo "issue-comment.sh REST create + exact-id read-back regression passed"