From a27f1fa7df723004498ae5d106b57e5710fcb5ae Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 17:50:43 -0500 Subject: [PATCH 01/15] fix(tools): use top-level tea comment invocation and formalize --login passthrough (#865) issue-comment.sh called the non-existent `tea issue comment` subcommand form; tea 0.11.1 silently no-ops and exits 0 instead of erroring, producing a false-success write. Switch to the top-level `tea comment ` form and add fail-closed REST read-back verification so the wrapper no longer trusts tea's exit code alone. pr-review.sh's comment path was already fixed for this bug by #812/#835 (routes through a read-back-verified REST comment API instead of any tea comment subcommand); this change formalizes an explicit --login override flag there too and documents the after-detection last-wins --login ordering, consistent with issue-comment.sh. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 13 +++ .../framework/tools/git/issue-comment.sh | 105 +++++++++++++++++- .../mosaic/framework/tools/git/pr-review.sh | 50 ++++++++- 3 files changed, 160 insertions(+), 8 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index c463de14..a148a86e 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -7,3 +7,16 @@ These scripts provide host-aware GitHub and Gitea issue, pull-request, milestone A successful provider write command—or a wrapper message based only on that command's exit code—is **not** durable review provenance. Review comments count as durable provenance only after the wrapper reads the created provider record back and verifies that it belongs to the intended repository and pull request and contains the exact submitted body (or verifies the provider-returned record ID). `pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. + +`issue-comment.sh` applies the same fail-closed read-back verification to issue comments: after posting via `tea comment`, it independently re-fetches the issue's comments via the Gitea REST API and confirms one matches the submitted body before reporting success. + +## `tea` invocation notes (Gitea) + +- tea v0.11.1 has **no `comment` subcommand under `tea pr` or `tea issue`**. The correct invocation is the **top-level** `tea comment [--repo ...] [--login ...]`. The `tea pr comment` / `tea issue comment` forms don't error — tea silently falls through to a no-op and still exits 0, producing a false-success write (#865). Always use the top-level form. +- `tea pr approve` and `tea pr reject` take an optional review comment/reason as a **trailing positional argument**, not a `--comment`/`-comment` flag (that flag does not exist on those subcommands). `pr-review.sh` avoids this positional form entirely for the approve/reject actions and instead posts any review comment through the same durable, read-back-verified comment API used for the `comment` action (see #835/#812) — the trailing-positional form remains available to callers who invoke `tea` directly, but is not used by these wrappers. + +### `--login` passthrough + +Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea `tea` login for that single invocation. The override is appended to the `tea` command line **after** the detected default (`get_gitea_repo_args()` / `get_gitea_login[_for_host]()`), because tea honors only the **last** `--login` flag on its command line — an override placed before the default would be silently clobbered by it. Callers who need a different login than the host default should pass `--login ` rather than relying on ordering tricks or re-invoking `tea login` globally. + +As a durable successor to this mechanism, consider giving each reviewer/approver slot its own dedicated Gitea login credential, so that author≠reviewer holds at the credential level rather than relying on wrapper-level `--login` bookkeeping. This is a recommendation for future hardening, not something implemented by this flag. diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index 5fef417e..0d0491b9 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -1,6 +1,23 @@ #!/bin/bash # issue-comment.sh - Add a comment to an issue on GitHub or Gitea -# Usage: issue-comment.sh -i -c +# Usage: issue-comment.sh -i -c [--login ] +# +# tea v0.11.1 defines no `comment` subcommand under `tea issue` (or `tea pr`); +# the correct invocation is the TOP-LEVEL `tea comment ` form. +# Calling the non-existent `tea issue comment ...` form does not error — tea +# silently falls through to a no-op and still exits 0, so a caller trusting +# the exit code alone believes a comment was posted when it was not (#865). +# Because that failure mode is silent, this script never trusts tea's exit +# code alone: after posting, it independently re-fetches the issue's comments +# via the Gitea REST API (curl — urllib is blocked by Cloudflare on this +# host) and fails closed if the posted body cannot be found. +# +# --login override: the default `--login` is resolved from the local `tea` +# login list for this repo's host (get_gitea_login). Pass --login to +# override that default for this invocation only. The override is appended +# to the tea command line AFTER the detected default, because tea honors +# only the LAST `--login` flag on the command line — a flag placed before +# the default would be silently clobbered by it. set -e @@ -10,6 +27,7 @@ source "$SCRIPT_DIR/detect-platform.sh" # Parse arguments ISSUE_NUMBER="" COMMENT="" +LOGIN_OVERRIDE="" while [[ $# -gt 0 ]]; do case $1 in @@ -21,12 +39,17 @@ while [[ $# -gt 0 ]]; do COMMENT="$2" shift 2 ;; + -l|--login) + LOGIN_OVERRIDE="$2" + shift 2 + ;; -h|--help) - echo "Usage: issue-comment.sh -i -c " + echo "Usage: issue-comment.sh -i -c [--login ]" echo "" echo "Options:" echo " -i, --issue Issue number (required)" echo " -c, --comment Comment text (required)" + echo " -l, --login Override the detected Gitea tea login for this call" echo " -h, --help Show this help" exit 0 ;; @@ -49,6 +72,68 @@ fi detect_platform >/dev/null +# Independently re-fetch the issue's comments via the Gitea REST API and +# confirm one matches the body we just posted (see header comment: tea's +# exit code is not trustworthy evidence of a durable write on its own). +# Prints the matched comment ID to stdout on success. +gitea_verify_comment_posted() { + local issue_number="$1" comment_body="$2" + local host token configured_url repo api_base readback_response_file + + host=$(get_remote_host) + token=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for comment read-back verification" >&2 + return 1 + } + configured_url=$(get_gitea_url_for_host "$host") || { + echo "Error: Configured Gitea URL not found for comment read-back verification" >&2 + return 1 + } + repo=$(get_gitea_repo_slug_for_url "$configured_url") || { + echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 + return 1 + } + api_base="${configured_url%/}/api/v1/repos/$repo" + + readback_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") + trap 'rm -f "$readback_response_file"' RETURN + + if ! readback_status=$(curl -sS -o "$readback_response_file" -w '%{http_code}' \ + -H "Authorization: token $token" \ + "$api_base/issues/$issue_number/comments"); then + echo "Error: Gitea comment read-back transport failed" >&2 + return 1 + fi + if [[ "$readback_status" != "200" ]]; then + echo "Error: Gitea comment read-back failed with HTTP $readback_status" >&2 + return 1 + fi + + EXPECTED_COMMENT_BODY="$comment_body" python3 - "$readback_response_file" <<'PY' +import json +import os +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + comments = json.load(response) + if not isinstance(comments, list): + raise ValueError("response is not a comment list") + expected_body = os.environ["EXPECTED_COMMENT_BODY"] + matches = [c for c in comments if isinstance(c, dict) and c.get("body") == expected_body] + if not matches: + raise ValueError("no matching comment found on read-back") + best = max(matches, key=lambda c: c.get("id") or 0) + comment_id = best.get("id") + if not isinstance(comment_id, int) or comment_id <= 0: + raise ValueError("matching comment has no usable id") +except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: + print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) + raise SystemExit(1) +print(comment_id) +PY +} + if [[ "$PLATFORM" == "github" ]]; then gh issue comment "$ISSUE_NUMBER" --body "$COMMENT" echo "Added comment to GitHub issue #$ISSUE_NUMBER" @@ -61,8 +146,20 @@ elif [[ "$PLATFORM" == "gitea" ]]; then echo "Error: could not resolve a Gitea login for this repo; cannot comment on issue #$ISSUE_NUMBER." >&2 exit 1 } - tea issue comment "$ISSUE_NUMBER" "$COMMENT" --repo "$REPO_SLUG" --login "$GITEA_LOGIN_NAME" - echo "Added comment to Gitea issue #$ISSUE_NUMBER" + TEA_ARGS=(comment "$ISSUE_NUMBER" "$COMMENT" --repo "$REPO_SLUG" --login "$GITEA_LOGIN_NAME") + # --login override goes LAST: tea honors only the final --login on its + # command line, so an override placed before the detected default above + # would be silently clobbered by it. + if [[ -n "$LOGIN_OVERRIDE" ]]; then + TEA_ARGS+=(--login "$LOGIN_OVERRIDE") + fi + tea "${TEA_ARGS[@]}" + + comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT") || { + echo "Error: could not verify comment landed on Gitea issue #$ISSUE_NUMBER via read-back; treating tea's exit code as untrustworthy (#865)." >&2 + exit 1 + } + echo "Added and verified comment on Gitea issue #$ISSUE_NUMBER (comment ID $comment_id)" else echo "Error: Unknown platform" exit 1 diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index 9b95c0ed..1e0e15c0 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -1,6 +1,17 @@ #!/bin/bash # pr-review.sh - Review a pull request on GitHub or Gitea -# Usage: pr-review.sh -n -a [-c ] +# Usage: pr-review.sh -n -a [-c ] [--login ] +# +# --login override: approve/request-changes on Gitea invoke `tea pr +# approve`/`tea pr reject` with a `--login` resolved from the local tea +# login list for this repo's host (get_gitea_login_for_host). Pass +# --login to override that default for this invocation only. The +# override is appended to the tea command line AFTER the detected default +# (get_gitea_repo_args()-equivalent resolution happens first), because tea +# honors only the LAST `--login` flag on its command line — a flag placed +# before the default would be silently clobbered by it. The `comment` +# action does not shell out to `tea` at all (see gitea_post_verified_comment +# below), so --login has no effect on it. set -e @@ -12,6 +23,7 @@ source "$SCRIPT_DIR/detect-platform.sh" PR_NUMBER="" ACTION="" COMMENT="" +LOGIN_OVERRIDE="" while [[ $# -gt 0 ]]; do case $1 in @@ -27,13 +39,18 @@ while [[ $# -gt 0 ]]; do COMMENT="$2" shift 2 ;; + -l|--login) + LOGIN_OVERRIDE="$2" + shift 2 + ;; -h|--help) - echo "Usage: pr-review.sh -n -a [-c ]" + echo "Usage: pr-review.sh -n -a [-c ] [--login ]" echo "" echo "Options:" echo " -n, --number PR number (required)" echo " -a, --action Review action: approve, request-changes, comment (required)" echo " -c, --comment Review comment (required for request-changes)" + echo " -l, --login Override the detected Gitea tea login (approve/request-changes only)" echo " -h, --help Show this help" exit 0 ;; @@ -210,8 +227,22 @@ elif [[ "$PLATFORM" == "gitea" ]]; then login=$(get_gitea_login_for_host "$host") # tea v0.11.1 defines no --comment/-comment flag on `pr approve`; # route any review body via the durable comment API instead (#835). - tea pr approve "$PR_NUMBER" --repo "$repo" --login "$login" + TEA_ARGS=(pr approve "$PR_NUMBER" --repo "$repo" --login "$login") + # --login override goes LAST: tea honors only the final --login on + # its command line, so an override placed before the detected + # default above would be silently clobbered by it. + if [[ -n "$LOGIN_OVERRIDE" ]]; then + TEA_ARGS+=(--login "$LOGIN_OVERRIDE") + fi + tea "${TEA_ARGS[@]}" echo "Approved Gitea PR #$PR_NUMBER" + # TODO(#865): this trusts tea's exit code for the approval STATE + # itself (no read-back of the review's approved status via the + # Gitea REST API). Only the optional accompanying COMMENT text + # below is independently read-back verified. Add a review-state + # read-back (e.g. GET /repos/{repo}/pulls/{pr}/reviews) if the + # approval state itself needs the same durable-provenance + # guarantee as comments. if [[ -n "$COMMENT" ]]; then comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" @@ -227,8 +258,19 @@ elif [[ "$PLATFORM" == "gitea" ]]; then login=$(get_gitea_login_for_host "$host") # tea v0.11.1 defines no --comment/-comment flag on `pr reject`; # route the review body via the durable comment API instead (#835). - tea pr reject "$PR_NUMBER" --repo "$repo" --login "$login" + TEA_ARGS=(pr reject "$PR_NUMBER" --repo "$repo" --login "$login") + # --login override goes LAST: tea honors only the final --login on + # its command line, so an override placed before the detected + # default above would be silently clobbered by it. + if [[ -n "$LOGIN_OVERRIDE" ]]; then + TEA_ARGS+=(--login "$LOGIN_OVERRIDE") + fi + tea "${TEA_ARGS[@]}" echo "Requested changes on Gitea PR #$PR_NUMBER" + # TODO(#865): this trusts tea's exit code for the rejection STATE + # itself (no read-back of the review's rejected/changes-requested + # status via the Gitea REST API). Only the required accompanying + # COMMENT text below is independently read-back verified. comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" ;; -- 2.49.1 From 10fdd49e32f2f30f92c90ca6944a650a146af833 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 18:21:02 -0500 Subject: [PATCH 02/15] fix(tools): bound Gitea read-back to this write; verify approve/reject state (#865) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review remediation for two correctness holes in the #865 fix: BLOCKER 1 — issue-comment.sh read-back was body-only across all history: if `tea comment` silently no-opped (the #865 bug) while an identically bodied comment already existed from a prior run, the read-back matched the OLD comment and falsely reported success. Now record the pre-write maximum comment id as a boundary and require a comment with id > boundary AND exact body match; monotonic Gitea ids make id > boundary mean "created by this write". Fails closed otherwise. BLOCKER 2 — pr-review.sh approve/reject trusted tea's exit code for the review STATE (same never-trust-exit-zero defect class as #865). Removed the TODO deferral and added a real bounded read-back: record the max review id before `tea pr approve`/`reject`, then require a review with id > boundary, the expected state (APPROVED / REQUEST_CHANGES), and commit_id equal to the PR's current head. Fails closed if absent. Tests: extended test-pr-review-gitea-comment.sh to model and assert the new review-state read-back (guardrails preserved, assertions added). Added test-issue-comment-readback.sh proving the pre-existing-identical-body false positive now fails closed and a genuinely new comment verifies. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 6 +- .../framework/tools/git/issue-comment.sh | 111 +++++++-- .../mosaic/framework/tools/git/pr-review.sh | 198 ++++++++++++++-- .../tools/git/test-issue-comment-readback.sh | 216 ++++++++++++++++++ .../tools/git/test-pr-review-gitea-comment.sh | 41 +++- 5 files changed, 529 insertions(+), 43 deletions(-) create mode 100755 packages/mosaic/framework/tools/git/test-issue-comment-readback.sh diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index a148a86e..4db7da6a 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -4,11 +4,11 @@ These scripts provide host-aware GitHub and Gitea issue, pull-request, milestone ## Durable review provenance -A successful provider write command—or a wrapper message based only on that command's exit code—is **not** durable review provenance. Review comments count as durable provenance only after the wrapper reads the created provider record back and verifies that it belongs to the intended repository and pull request and contains the exact submitted body (or verifies the provider-returned record ID). +A successful provider write command—or a wrapper message based only on that command's exit code—is **not** durable review provenance. Review comments, approvals, and change requests count as durable provenance only after the wrapper reads the created provider record back and verifies that it was created by _this_ write. -`pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. +`pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. The `approve` and `request-changes` actions apply the same discipline to the review **state** itself: they record the maximum existing review id _before_ invoking `tea pr approve`/`reject`, then require a review whose id is strictly greater than that boundary, whose state matches the requested action (`APPROVED` / `REQUEST_CHANGES`), and whose reviewed commit equals the PR's current head. tea's exit code alone is never treated as evidence the review landed. -`issue-comment.sh` applies the same fail-closed read-back verification to issue comments: after posting via `tea comment`, it independently re-fetches the issue's comments via the Gitea REST API and confirms one matches the submitted body before reporting success. +`issue-comment.sh` applies the same fail-closed, boundary-bounded read-back to issue comments: it records the maximum existing comment id _before_ posting via `tea comment`, then re-fetches the issue's comments via the Gitea REST API and requires a comment whose id is strictly greater than that boundary **and** whose body exactly matches what was submitted. Bounding the read-back by the pre-write id is essential — a body-only match across all history would falsely report success if `tea comment` silently no-ops (the #865 bug) while an identically-bodied comment already existed from a prior run. Gitea comment and review ids are monotonic, so `id > boundary` reliably means "created after this write began". ## `tea` invocation notes (Gitea) diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index 0d0491b9..e7d4a11a 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -72,16 +72,14 @@ fi detect_platform >/dev/null -# Independently re-fetch the issue's comments via the Gitea REST API and -# confirm one matches the body we just posted (see header comment: tea's -# exit code is not trustworthy evidence of a durable write on its own). -# Prints the matched comment ID to stdout on success. -gitea_verify_comment_posted() { - local issue_number="$1" comment_body="$2" - local host token configured_url repo api_base readback_response_file +# Resolve and cache the Gitea REST endpoint + token for the current remote. +# Populates GITEA_API_BASE and GITEA_API_TOKEN. Returns non-zero (with a +# clear stderr message) if any part of the resolution fails. +gitea_resolve_api() { + local host configured_url repo host=$(get_remote_host) - token=$(get_gitea_token "$host") || { + GITEA_API_TOKEN=$(get_gitea_token "$host") || { echo "Error: Gitea token not found for comment read-back verification" >&2 return 1 } @@ -93,23 +91,76 @@ gitea_verify_comment_posted() { echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 return 1 } - api_base="${configured_url%/}/api/v1/repos/$repo" + GITEA_API_BASE="${configured_url%/}/api/v1/repos/$repo" + return 0 +} + +# Print the maximum existing comment id on an issue (0 if none). This is the +# pre-write BOUNDARY: Gitea comment ids are monotonic, so any comment created +# by a subsequent write has an id strictly greater than this value. Bounding +# the read-back this way is what distinguishes a genuine fresh write from a +# pre-existing comment that merely happens to share the same body — the exact +# false-positive a body-only, whole-history match would miss when `tea +# comment` silently no-ops (#865). +gitea_max_comment_id() { + local issue_number="$1" response_file status + + response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-boundary.XXXXXX") + trap 'rm -f "$response_file"' RETURN + + if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/issues/$issue_number/comments"); then + echo "Error: Gitea comment boundary read transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + echo "Error: Gitea comment boundary read failed with HTTP $status" >&2 + return 1 + fi + + python3 - "$response_file" <<'PY' +import json +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + comments = json.load(response) + if not isinstance(comments, list): + raise ValueError("response is not a comment list") + ids = [c.get("id") for c in comments if isinstance(c, dict) and isinstance(c.get("id"), int)] + print(max(ids) if ids else 0) +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not compute Gitea comment boundary: {error}", file=sys.stderr) + raise SystemExit(1) +PY +} + +# Independently re-fetch the issue's comments via the Gitea REST API and +# require a comment that was created by THIS write: its id must be strictly +# greater than the pre-write boundary AND its body must exactly match what we +# submitted (see header comment: tea's exit code is not trustworthy evidence +# of a durable write on its own). Prints the matched comment ID on success. +gitea_verify_comment_posted() { + local issue_number="$1" comment_body="$2" boundary="$3" + local readback_response_file status readback_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") trap 'rm -f "$readback_response_file"' RETURN - if ! readback_status=$(curl -sS -o "$readback_response_file" -w '%{http_code}' \ - -H "Authorization: token $token" \ - "$api_base/issues/$issue_number/comments"); then + if ! status=$(curl -sS -o "$readback_response_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/issues/$issue_number/comments"); then echo "Error: Gitea comment read-back transport failed" >&2 return 1 fi - if [[ "$readback_status" != "200" ]]; then - echo "Error: Gitea comment read-back failed with HTTP $readback_status" >&2 + if [[ "$status" != "200" ]]; then + echo "Error: Gitea comment read-back failed with HTTP $status" >&2 return 1 fi - EXPECTED_COMMENT_BODY="$comment_body" python3 - "$readback_response_file" <<'PY' + EXPECTED_COMMENT_BODY="$comment_body" BOUNDARY_COMMENT_ID="$boundary" \ + python3 - "$readback_response_file" <<'PY' import json import os import sys @@ -120,13 +171,21 @@ try: if not isinstance(comments, list): raise ValueError("response is not a comment list") expected_body = os.environ["EXPECTED_COMMENT_BODY"] - matches = [c for c in comments if isinstance(c, dict) and c.get("body") == expected_body] + boundary = int(os.environ["BOUNDARY_COMMENT_ID"]) + # Require both: created-after-boundary (fresh write) AND exact body match. + matches = [ + c for c in comments + if isinstance(c, dict) + and isinstance(c.get("id"), int) + and c.get("id") > boundary + and c.get("body") == expected_body + ] if not matches: - raise ValueError("no matching comment found on read-back") - best = max(matches, key=lambda c: c.get("id") or 0) - comment_id = best.get("id") - if not isinstance(comment_id, int) or comment_id <= 0: - raise ValueError("matching comment has no usable id") + raise ValueError( + "no comment created by this write matched (id > boundary and exact body); " + "tea may have silently no-opped (#865)" + ) + comment_id = max(c["id"] for c in matches) except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) @@ -146,6 +205,12 @@ elif [[ "$PLATFORM" == "gitea" ]]; then echo "Error: could not resolve a Gitea login for this repo; cannot comment on issue #$ISSUE_NUMBER." >&2 exit 1 } + + # Resolve the REST endpoint and record the pre-write boundary BEFORE the + # write, so the read-back can require a strictly-newer comment id. + gitea_resolve_api || exit 1 + boundary=$(gitea_max_comment_id "$ISSUE_NUMBER") || exit 1 + TEA_ARGS=(comment "$ISSUE_NUMBER" "$COMMENT" --repo "$REPO_SLUG" --login "$GITEA_LOGIN_NAME") # --login override goes LAST: tea honors only the final --login on its # command line, so an override placed before the detected default above @@ -155,8 +220,8 @@ elif [[ "$PLATFORM" == "gitea" ]]; then fi tea "${TEA_ARGS[@]}" - comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT") || { - echo "Error: could not verify comment landed on Gitea issue #$ISSUE_NUMBER via read-back; treating tea's exit code as untrustworthy (#865)." >&2 + comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT" "$boundary") || { + echo "Error: could not verify comment landed on Gitea issue #$ISSUE_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 exit 1 } echo "Added and verified comment on Gitea issue #$ISSUE_NUMBER (comment ID $comment_id)" diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index 1e0e15c0..b57fd43b 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -192,6 +192,171 @@ PY return 0 } +# Resolve and cache the Gitea REST endpoint + token for the current remote. +# Populates GITEA_API_BASE and GITEA_API_TOKEN. Returns non-zero (with a +# clear stderr message) on any resolution failure. +gitea_resolve_api() { + local host configured_url repo + + host=$(get_remote_host) + GITEA_API_TOKEN=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for review read-back verification" >&2 + return 1 + } + configured_url=$(get_gitea_url_for_host "$host") || { + echo "Error: Configured Gitea URL not found for review read-back verification" >&2 + return 1 + } + repo=$(get_gitea_repo_slug_for_url "$configured_url") || { + echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 + return 1 + } + GITEA_API_BASE="${configured_url%/}/api/v1/repos/$repo" + return 0 +} + +# Print the maximum existing review id on a PR (0 if none). This is the +# pre-write BOUNDARY: Gitea pull-review ids are monotonic, so any review +# submitted by a subsequent `tea pr approve`/`reject` has an id strictly +# greater than this value. Bounding the read-back this way is what turns the +# check into a genuine write-verification rather than a match against any +# historical review — the same never-trust-exit-zero discipline #865 requires +# for comments, applied to the review STATE itself. +gitea_max_review_id() { + local pr_number="$1" response_file status + + response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-boundary.XXXXXX") + trap 'rm -f "$response_file"' RETURN + + if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/pulls/$pr_number/reviews"); then + echo "Error: Gitea review boundary read transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + echo "Error: Gitea review boundary read failed with HTTP $status" >&2 + return 1 + fi + + python3 - "$response_file" <<'PY' +import json +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + reviews = json.load(response) + if not isinstance(reviews, list): + raise ValueError("response is not a review list") + ids = [r.get("id") for r in reviews if isinstance(r, dict) and isinstance(r.get("id"), int)] + print(max(ids) if ids else 0) +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not compute Gitea review boundary: {error}", file=sys.stderr) + raise SystemExit(1) +PY +} + +# Independently verify that `tea pr approve`/`reject` produced a durable review +# record — never trust tea's exit code alone (#865, same defect class). Require +# a review that was created by THIS action: its id must be strictly greater +# than the pre-write boundary, its state must equal the expected state +# (APPROVED / REQUEST_CHANGES), and it must have been submitted against the +# PR's current head commit. Prints the matched review id on success; fails +# closed (non-zero, clear stderr) if no such review is found. +# +# Args: $1 = PR number, $2 = expected state (APPROVED|REQUEST_CHANGES), +# $3 = pre-write boundary review id. +gitea_verify_review_submitted() { + local pr_number="$1" expected_state="$2" boundary="$3" + local pr_response_file reviews_response_file status head_sha review_id + + pr_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") + reviews_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-state.XXXXXX") + trap 'rm -f "$pr_response_file" "$reviews_response_file"' RETURN + + # Resolve the PR's current head commit so the review can be pinned to it. + if ! status=$(curl -sS -o "$pr_response_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/pulls/$pr_number"); then + echo "Error: Gitea PR head read transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + echo "Error: Gitea PR head read failed with HTTP $status" >&2 + return 1 + fi + head_sha=$(python3 - "$pr_response_file" <<'PY' +import json +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + pr = json.load(response) + head_sha = pr.get("head", {}).get("sha") if isinstance(pr, dict) else None + if not isinstance(head_sha, str) or not head_sha: + raise ValueError("missing PR head sha") +except (OSError, json.JSONDecodeError, AttributeError, TypeError, ValueError) as error: + print(f"Error: could not resolve PR head commit: {error}", file=sys.stderr) + raise SystemExit(1) +print(head_sha) +PY +) || return 1 + + if ! status=$(curl -sS -o "$reviews_response_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/pulls/$pr_number/reviews"); then + echo "Error: Gitea review read-back transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + echo "Error: Gitea review read-back failed with HTTP $status" >&2 + return 1 + fi + + review_id=$(EXPECTED_STATE="$expected_state" BOUNDARY_REVIEW_ID="$boundary" EXPECTED_HEAD_SHA="$head_sha" \ + python3 - "$reviews_response_file" <<'PY' +import json +import os +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + reviews = json.load(response) + if not isinstance(reviews, list): + raise ValueError("response is not a review list") + expected_state = os.environ["EXPECTED_STATE"] + boundary = int(os.environ["BOUNDARY_REVIEW_ID"]) + expected_head = os.environ["EXPECTED_HEAD_SHA"] + # Require all of: created-after-boundary (this action's write), the + # expected review state, and pinned to the PR's current head commit. + # The monotonic id boundary is what proves "submitted by this action" + # rather than matching some pre-existing historical review. + matches = [ + r for r in reviews + if isinstance(r, dict) + and isinstance(r.get("id"), int) + and r.get("id") > boundary + and r.get("state") == expected_state + and r.get("commit_id") == expected_head + ] + if not matches: + raise ValueError( + f"no {expected_state} review created by this action found " + "(id > boundary, expected state, current head); " + "tea may have silently failed (#865 defect class)" + ) + review_id = max(r["id"] for r in matches) +except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: + print(f"Error: Gitea review persistence verification failed: {error}", file=sys.stderr) + raise SystemExit(1) +print(review_id) +PY +) || return 1 + + echo "$review_id" + return 0 +} + if [[ "$PLATFORM" == "github" ]]; then case $ACTION in approve) @@ -225,6 +390,12 @@ elif [[ "$PLATFORM" == "gitea" ]]; then repo=$(get_repo_slug) host=$(get_remote_host) login=$(get_gitea_login_for_host "$host") + # Resolve the REST endpoint and record the pre-write review-id + # boundary BEFORE the write, so the read-back can require a + # strictly-newer review created by THIS action (never trust tea's + # exit code alone — #865 defect class applies to the review state). + gitea_resolve_api || exit 1 + review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 # tea v0.11.1 defines no --comment/-comment flag on `pr approve`; # route any review body via the durable comment API instead (#835). TEA_ARGS=(pr approve "$PR_NUMBER" --repo "$repo" --login "$login") @@ -235,14 +406,11 @@ elif [[ "$PLATFORM" == "gitea" ]]; then TEA_ARGS+=(--login "$LOGIN_OVERRIDE") fi tea "${TEA_ARGS[@]}" - echo "Approved Gitea PR #$PR_NUMBER" - # TODO(#865): this trusts tea's exit code for the approval STATE - # itself (no read-back of the review's approved status via the - # Gitea REST API). Only the optional accompanying COMMENT text - # below is independently read-back verified. Add a review-state - # read-back (e.g. GET /repos/{repo}/pulls/{pr}/reviews) if the - # approval state itself needs the same durable-provenance - # guarantee as comments. + review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "APPROVED" "$review_boundary") || { + echo "Error: could not verify an APPROVED review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 + exit 1 + } + echo "Approved and verified Gitea PR #$PR_NUMBER (review ID $review_id)" if [[ -n "$COMMENT" ]]; then comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" @@ -256,6 +424,10 @@ elif [[ "$PLATFORM" == "gitea" ]]; then repo=$(get_repo_slug) host=$(get_remote_host) login=$(get_gitea_login_for_host "$host") + # Record the pre-write review-id boundary BEFORE the write (see the + # approve path above for the rationale). + gitea_resolve_api || exit 1 + review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 # tea v0.11.1 defines no --comment/-comment flag on `pr reject`; # route the review body via the durable comment API instead (#835). TEA_ARGS=(pr reject "$PR_NUMBER" --repo "$repo" --login "$login") @@ -266,11 +438,11 @@ elif [[ "$PLATFORM" == "gitea" ]]; then TEA_ARGS+=(--login "$LOGIN_OVERRIDE") fi tea "${TEA_ARGS[@]}" - echo "Requested changes on Gitea PR #$PR_NUMBER" - # TODO(#865): this trusts tea's exit code for the rejection STATE - # itself (no read-back of the review's rejected/changes-requested - # status via the Gitea REST API). Only the required accompanying - # COMMENT text below is independently read-back verified. + review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "REQUEST_CHANGES" "$review_boundary") || { + echo "Error: could not verify a REQUEST_CHANGES review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 + exit 1 + } + echo "Requested changes and verified on Gitea PR #$PR_NUMBER (review ID $review_id)" comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" ;; diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh new file mode 100755 index 00000000..8dd8a06b --- /dev/null +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -0,0 +1,216 @@ +#!/usr/bin/env bash +# Regression harness for issue-comment.sh top-level `tea comment` invocation and +# its BOUNDED read-back verification (#865). +# +# The #865 bug: `tea issue comment ...` (a nonexistent subcommand on tea +# v0.11.1) silently no-ops and exits 0, so a comment is never posted. A naive +# read-back that matches ANY historical comment by body would falsely report +# success whenever an identically-bodied comment already exists from a prior +# run. This harness proves the wrapper: +# 1. uses the top-level `tea comment` form (never `tea issue comment`); +# 2. records the pre-write maximum comment id as a boundary and requires a +# strictly-newer comment on read-back, so a pre-existing identical body +# does NOT satisfy verification (fails closed); +# 3. reports success only when a genuinely new comment (id > boundary) with +# the exact body appears. +# +# The `tea` stub NEVER creates a comment (it mimics the silent no-op); the +# "server" comment state is modeled entirely by the curl stub's responses, so +# the fresh-success vs. no-op distinction is driven purely by whether the +# post-write read-back surfaces a new id. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/issue-comment-readback}" +REPO_DIR="$WORK_DIR/repo" +BIN_DIR="$WORK_DIR/bin" +TEA_LOG="$WORK_DIR/tea.log" +CURL_LOG="$WORK_DIR/curl.log" +OUTPUT_FILE="$WORK_DIR/output.log" +CREDENTIALS_FILE="$WORK_DIR/credentials.json" +CALLS_FILE="$WORK_DIR/comment_calls" + +cleanup() { + rm -rf "$WORK_DIR" +} +trap cleanup EXIT + +mkdir -p "$REPO_DIR" "$BIN_DIR" +git -C "$REPO_DIR" init -q +git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git + +ISSUE_NUMBER=7 +API_BASE="https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack" +BODY='durable "note" -- marker' + +CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY' +import json +import os +import sys + +with open(sys.argv[1], "w", encoding="utf-8") as credentials: + json.dump({ + "gitea": { + "mosaicstack": { + "url": os.environ["CONFIGURED_GITEA_URL"], + "token": "test-only-placeholder", + } + } + }, credentials) +PY + +# tea stub: resolves the login list, and treats `tea comment ...` as a silent +# no-op (exit 0 without creating anything) to mimic the real failure mode. +cat > "$BIN_DIR/tea" <<'SH' +#!/usr/bin/env bash +set -euo pipefail + +printf '%s\n' "$*" >> "$ISSUE_COMMENT_TEA_LOG" + +if [[ "$*" == "login list --output json" ]]; then + printf '%s\n' '[{"name":"mosaicstack","url":"https://git.mosaicstack.dev"}]' + exit 0 +fi + +# The wrapper must use the TOP-LEVEL `tea comment` form; the broken +# `tea issue comment` subcommand must never be invoked. +if [[ "$*" == issue\ comment* ]]; then + echo "wrapper invoked nonexistent 'tea issue comment' subcommand" >&2 + exit 90 +fi + +if [[ "$*" == comment\ * ]]; then + # Mimic tea v0.11.1: exit 0. Whether a comment actually lands is modeled + # by the curl stub's post-write read-back response, not here. + exit 0 +fi + +echo "Unexpected tea command: $*" >&2 +exit 92 +SH +chmod +x "$BIN_DIR/tea" + +# curl stub: serves GET .../issues/7/comments. First call = pre-write boundary, +# second call = post-write read-back. The boundary always contains a +# pre-existing comment (id 50) whose body is IDENTICAL to the one under test, +# which is exactly the condition a body-only match would trip over. +cat > "$BIN_DIR/curl" <<'SH' +#!/usr/bin/env bash +set -euo pipefail + +output_file="" +method="GET" +url="" +while [[ $# -gt 0 ]]; do + case "$1" in + -o) output_file="$2"; shift 2 ;; + -w|-H) shift 2 ;; + -X) method="$2"; shift 2 ;; + -d|--data) shift 2 ;; + -s|-S|-sS) shift ;; + http://*|https://*) url="$1"; shift ;; + *) shift ;; + esac +done + +printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG" + +write_response() { + local status="$1" body="$2" + [[ -n "$output_file" ]] || exit 96 + printf '%s' "$body" > "$output_file" + printf '%s' "$status" +} + +if [[ "$method" == "GET" && "$url" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then + calls_file="$ISSUE_COMMENT_CALLS" + if [[ -f "$calls_file" ]]; then + # post-write read-back + if [[ "$ISSUE_COMMENT_TEST_MODE" == "fresh-success" ]]; then + response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' +import json +import os + +body = os.environ["ISSUE_COMMENT_BODY"] +print(json.dumps([ + {"id": 50, "body": body}, + {"id": 60, "body": body}, +])) +PY +) + else + # no-op: nothing new landed; the pre-existing id-50 comment remains. + response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' +import json +import os + +body = os.environ["ISSUE_COMMENT_BODY"] +print(json.dumps([{"id": 50, "body": body}])) +PY +) + fi + else + : > "$calls_file" + response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' +import json +import os + +body = os.environ["ISSUE_COMMENT_BODY"] +print(json.dumps([{"id": 50, "body": body}])) +PY +) + fi + write_response 200 "$response" +else + echo "Unexpected curl request: $method $url" >&2 + exit 97 +fi +SH +chmod +x "$BIN_DIR/curl" + +run_comment() { + local mode="$1" + : > "$TEA_LOG" + : > "$CURL_LOG" + : > "$OUTPUT_FILE" + rm -f "$CALLS_FILE" + ( + cd "$REPO_DIR" + PATH="$BIN_DIR:$PATH" \ + MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ + ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \ + ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \ + ISSUE_COMMENT_CALLS="$CALLS_FILE" \ + ISSUE_COMMENT_TEST_MODE="$mode" \ + ISSUE_COMMENT_EXPECTED_BODY="$BODY" \ + ISSUE_COMMENT_API_BASE="$API_BASE" \ + "$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY" + ) > "$OUTPUT_FILE" 2>&1 +} + +# Case 1: silent no-op with a pre-existing identical body must FAIL CLOSED. +if run_comment noop-preexisting; then + echo "FAIL: wrapper reported success when tea no-opped but an identical body pre-existed" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: read-back matched a pre-existing comment by body only" >&2 + exit 1 +fi +# The wrapper must have used the top-level form and read comments back twice +# (boundary + post-write). +grep -q "^comment 7 " "$TEA_LOG" +if grep -q '^issue comment' "$TEA_LOG"; then + echo "FAIL: wrapper used the broken 'tea issue comment' subcommand" >&2 + exit 1 +fi +[[ "$(grep -c "^GET $API_BASE/issues/7/comments$" "$CURL_LOG")" == "2" ]] + +# Case 2: a genuinely new comment (id 60 > boundary 50) verifies successfully. +run_comment fresh-success +grep -q 'Added and verified comment on Gitea issue #7 (comment ID 60)' "$OUTPUT_FILE" +grep -q "^comment 7 " "$TEA_LOG" + +echo "issue-comment.sh bounded read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index 77f3d3d6..bf32ba4d 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -145,7 +145,33 @@ case "${PR_REVIEW_TEST_MODE:-}" in write_response 500 '{"message":"simulated rejection"}' ;; approve|request-changes|comment-success|http-success|prefix-success|subpath-success|port-success|scp-ssh-success|url-ssh-success|ssh-transport-port-success|explicit-default-port-success|readback-failure) - if [[ "$method" == "POST" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then + if [[ "$method" == "GET" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then + # First GET = pre-write review-id BOUNDARY; second GET = post-write + # read-back that must surface a strictly-newer review created by + # THIS action (id 200 > boundary 100), with the expected state and + # pinned to the PR's current head commit. + calls_file="${PR_REVIEW_REVIEW_CALLS:-/dev/null}" + if [[ -f "$calls_file" ]]; then + state="APPROVED" + [[ "$PR_REVIEW_TEST_MODE" == "request-changes" ]] && state="REQUEST_CHANGES" + response=$(PR_REVIEW_STATE="$state" python3 - <<'PY' +import json +import os + +print(json.dumps([ + {"id": 100, "state": "COMMENT", "commit_id": "oldsha0000"}, + {"id": 200, "state": os.environ["PR_REVIEW_STATE"], "commit_id": "HEADSHA_FEEDFACE"}, +])) +PY +) + else + : > "$calls_file" + response='[{"id":100,"state":"COMMENT","commit_id":"oldsha0000"}]' + fi + write_response 200 "$response" + elif [[ "$method" == "GET" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123" ]]; then + write_response 200 '{"head":{"sha":"HEADSHA_FEEDFACE"}}' + elif [[ "$method" == "POST" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' import json import os @@ -201,12 +227,14 @@ run_review() { : > "$TEA_LOG" : > "$CURL_LOG" : > "$OUTPUT_FILE" + rm -f "$WORK_DIR/review_calls" ( cd "$REPO_DIR" PATH="$BIN_DIR:$PATH" \ MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ PR_REVIEW_TEA_LOG="$TEA_LOG" \ PR_REVIEW_CURL_LOG="$CURL_LOG" \ + PR_REVIEW_REVIEW_CALLS="$WORK_DIR/review_calls" \ PR_REVIEW_TEST_MODE="$mode" \ PR_REVIEW_EXPECTED_BODY="$comment" \ PR_REVIEW_EXPECTED_API_BASE="$expected_api_base" \ @@ -216,7 +244,11 @@ run_review() { run_review approve approve grep -q '^pr approve 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Approved Gitea PR #123' "$OUTPUT_FILE" +grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" +# #865: the approval STATE itself is read back — a pre-write boundary GET and a +# post-write read-back GET on the reviews endpoint, plus a PR head lookup. +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG" if grep -q 'comment' "$TEA_LOG"; then echo "Plain approve (no review body) unexpectedly touched comment persistence" >&2 exit 1 @@ -227,7 +259,7 @@ fi # comment REST API instead of being passed to `tea` directly. run_review approve approve approve-note grep -q '^pr approve 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Approved Gitea PR #123' "$OUTPUT_FILE" +grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$OUTPUT_FILE" @@ -235,7 +267,8 @@ grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$ # #835: same for `pr reject` (request-changes), where a comment is required. run_review request-changes request-changes changes-required grep -q '^pr reject 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Requested changes on Gitea PR #123' "$OUTPUT_FILE" +grep -q 'Requested changes and verified on Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$OUTPUT_FILE" -- 2.49.1 From 16481ece3df8d6839d260d960fbc36c3731e41b6 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 19:02:25 -0500 Subject: [PATCH 03/15] fix(tools): attribute read-back to acting identity and paginate fully (#865) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 2 remediation for the read-back verification in issue-comment.sh and pr-review.sh. BLOCKER A (invocation attribution): id-above-boundary + content/state match only proves temporal ordering — a concurrent write from a different identity could satisfy it while this tea invocation created nothing. Both wrappers now resolve the acting identity once via curl GET /api/v1/user and additionally require the accepted record's author login to equal that identity. Residual same-identity same-body/state concurrency is documented in-code (tea 0.11.1 emits no reliable created-record id to close it further). BLOCKER B (pagination): the comments and reviews list reads now walk every page (?limit=&page=1,2,… until a short/empty page) for both the pre-write boundary and the post-write read-back, so a record beyond page 1 is still found. Adds regressions: concurrent different-identity write fails closed (comments and reviews); a matching review beyond page 1 is still found. README updated. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 8 +- .../framework/tools/git/issue-comment.sh | 175 ++++++++++++----- .../mosaic/framework/tools/git/pr-review.sh | 185 +++++++++++++----- .../tools/git/test-issue-comment-readback.sh | 109 ++++++++--- .../tools/git/test-pr-review-gitea-comment.sh | 130 +++++++++--- 5 files changed, 461 insertions(+), 146 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index 4db7da6a..912de1bb 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -6,9 +6,13 @@ These scripts provide host-aware GitHub and Gitea issue, pull-request, milestone A successful provider write command—or a wrapper message based only on that command's exit code—is **not** durable review provenance. Review comments, approvals, and change requests count as durable provenance only after the wrapper reads the created provider record back and verifies that it was created by _this_ write. -`pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. The `approve` and `request-changes` actions apply the same discipline to the review **state** itself: they record the maximum existing review id _before_ invoking `tea pr approve`/`reject`, then require a review whose id is strictly greater than that boundary, whose state matches the requested action (`APPROVED` / `REQUEST_CHANGES`), and whose reviewed commit equals the PR's current head. tea's exit code alone is never treated as evidence the review landed. +`pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. The `approve` and `request-changes` actions apply the same discipline to the review **state** itself: they record the maximum existing review id _before_ invoking `tea pr approve`/`reject`, then require a review whose id is strictly greater than that boundary, whose **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), whose state matches the requested action (`APPROVED` / `REQUEST_CHANGES`), and whose reviewed commit equals the PR's current head. tea's exit code alone is never treated as evidence the review landed. -`issue-comment.sh` applies the same fail-closed, boundary-bounded read-back to issue comments: it records the maximum existing comment id _before_ posting via `tea comment`, then re-fetches the issue's comments via the Gitea REST API and requires a comment whose id is strictly greater than that boundary **and** whose body exactly matches what was submitted. Bounding the read-back by the pre-write id is essential — a body-only match across all history would falsely report success if `tea comment` silently no-ops (the #865 bug) while an identically-bodied comment already existed from a prior run. Gitea comment and review ids are monotonic, so `id > boundary` reliably means "created after this write began". +`issue-comment.sh` applies the same fail-closed, boundary-bounded read-back to issue comments: it records the maximum existing comment id _before_ posting via `tea comment`, then re-fetches the issue's comments via the Gitea REST API and requires a comment whose id is strictly greater than that boundary, **whose author login equals the acting identity**, **and** whose body exactly matches what was submitted. Bounding the read-back by the pre-write id is essential — a body-only match across all history would falsely report success if `tea comment` silently no-ops (the #865 bug) while an identically-bodied comment already existed from a prior run. Gitea comment and review ids are monotonic, so `id > boundary` reliably means "created after this write began". + +**Invocation attribution, not just temporal ordering.** `id > boundary` alone only proves a record was created after the write began; it would still be satisfied by a _concurrent_ write from a _different_ identity while this `tea` invocation created nothing. Both wrappers therefore additionally require the accepted record's author login to equal the identity the API token authenticates as, narrowing the match to this invocation's writer. The one residual window — a concurrent write by the _same_ identity with an identical body/state inside the boundary window — cannot be eliminated without a tea-emitted created-record id, which tea 0.11.1 does not reliably provide; it is strictly narrower than temporal-only matching and is documented in-code. + +**Full pagination.** Gitea paginates list endpoints, so a single-page read of the comments or reviews list would false-negative once the freshly created record lands beyond the first page. Both the pre-write boundary computation and the post-write read-back walk every page (`?limit=&page=1,2,…` until a short/empty page) so the match is exhaustive regardless of how many comments or reviews already exist. ## `tea` invocation notes (Gitea) diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index e7d4a11a..64f1a147 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -73,8 +73,9 @@ fi detect_platform >/dev/null # Resolve and cache the Gitea REST endpoint + token for the current remote. -# Populates GITEA_API_BASE and GITEA_API_TOKEN. Returns non-zero (with a -# clear stderr message) if any part of the resolution fails. +# Populates GITEA_API_ROOT (…/api/v1), GITEA_API_BASE (…/api/v1/repos/), +# and GITEA_API_TOKEN. Returns non-zero (with a clear stderr message) if any +# part of the resolution fails. gitea_resolve_api() { local host configured_url repo @@ -91,31 +92,86 @@ gitea_resolve_api() { echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 return 1 } - GITEA_API_BASE="${configured_url%/}/api/v1/repos/$repo" + GITEA_API_ROOT="${configured_url%/}/api/v1" + GITEA_API_BASE="$GITEA_API_ROOT/repos/$repo" return 0 } -# Print the maximum existing comment id on an issue (0 if none). This is the -# pre-write BOUNDARY: Gitea comment ids are monotonic, so any comment created -# by a subsequent write has an id strictly greater than this value. Bounding -# the read-back this way is what distinguishes a genuine fresh write from a -# pre-existing comment that merely happens to share the same body — the exact -# false-positive a body-only, whole-history match would miss when `tea -# comment` silently no-ops (#865). -gitea_max_comment_id() { - local issue_number="$1" response_file status +# Fetch every page of a Gitea list endpoint into $2 (merged into one JSON +# array). Gitea paginates list responses, so a single-page read would +# false-negative once a newly created record lands beyond page 1. Walks +# page=1,2,… until a short page (fewer than the requested limit) or an empty +# page is returned, so the merged array is exhaustive. $1 is the endpoint URL +# with NO query string. Returns non-zero (clear stderr) on any transport / +# HTTP / parse failure. +gitea_fetch_all() { + local base_url="$1" dest="$2" page=1 limit=50 status page_file count - response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-boundary.XXXXXX") + printf '[]' > "$dest" + while :; do + page_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-page.XXXXXX") + if ! status=$(curl -sS -o "$page_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "${base_url}?limit=${limit}&page=${page}"); then + rm -f "$page_file" + echo "Error: Gitea list read transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + rm -f "$page_file" + echo "Error: Gitea list read failed with HTTP $status" >&2 + return 1 + fi + count=$(DEST="$dest" python3 - "$page_file" <<'PY' +import json +import os +import sys + +try: + with open(os.environ["DEST"], encoding="utf-8") as merged_file: + merged = json.load(merged_file) + with open(sys.argv[1], encoding="utf-8") as page_file: + page = json.load(page_file) + if not isinstance(page, list): + raise ValueError("page response is not a list") + merged.extend(item for item in page if isinstance(item, dict)) + with open(os.environ["DEST"], "w", encoding="utf-8") as merged_file: + json.dump(merged, merged_file) +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not merge Gitea list page: {error}", file=sys.stderr) + raise SystemExit(1) +print(len(page)) +PY +) || { rm -f "$page_file"; return 1; } + rm -f "$page_file" + [[ "$count" -lt "$limit" ]] && break + page=$((page + 1)) + if [[ "$page" -gt 1000 ]]; then + echo "Error: Gitea list pagination exceeded 1000 pages" >&2 + return 1 + fi + done + return 0 +} + +# Resolve the login of the identity the API token authenticates as (GET +# /user). Used to attribute a read-back record to THIS invocation's writer so +# a concurrent write from a DIFFERENT identity cannot satisfy verification. +# Prints the login on success. +gitea_authenticated_login() { + local response_file status + + response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-whoami.XXXXXX") trap 'rm -f "$response_file"' RETURN if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ -H "Authorization: token $GITEA_API_TOKEN" \ - "$GITEA_API_BASE/issues/$issue_number/comments"); then - echo "Error: Gitea comment boundary read transport failed" >&2 + "$GITEA_API_ROOT/user"); then + echo "Error: Gitea authenticated-identity read transport failed" >&2 return 1 fi if [[ "$status" != "200" ]]; then - echo "Error: Gitea comment boundary read failed with HTTP $status" >&2 + echo "Error: Gitea authenticated-identity read failed with HTTP $status" >&2 return 1 fi @@ -123,11 +179,37 @@ gitea_max_comment_id() { import json import sys +try: + with open(sys.argv[1], encoding="utf-8") as response: + user = json.load(response) + login = user.get("login") if isinstance(user, dict) else None + if not isinstance(login, str) or not login: + raise ValueError("missing authenticated login") +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not resolve authenticated Gitea identity: {error}", file=sys.stderr) + raise SystemExit(1) +print(login) +PY +} + +# Print the maximum existing comment id on an issue (0 if none). This is the +# pre-write BOUNDARY: Gitea comment ids are monotonic, so any comment created +# by a subsequent write has an id strictly greater than this value. +gitea_max_comment_id() { + local issue_number="$1" merged_file + + merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-boundary.XXXXXX") + trap 'rm -f "$merged_file"' RETURN + + gitea_fetch_all "$GITEA_API_BASE/issues/$issue_number/comments" "$merged_file" || return 1 + + python3 - "$merged_file" <<'PY' +import json +import sys + try: with open(sys.argv[1], encoding="utf-8") as response: comments = json.load(response) - if not isinstance(comments, list): - raise ValueError("response is not a comment list") ids = [c.get("id") for c in comments if isinstance(c, dict) and isinstance(c.get("id"), int)] print(max(ids) if ids else 0) except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: @@ -136,31 +218,29 @@ except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: PY } -# Independently re-fetch the issue's comments via the Gitea REST API and -# require a comment that was created by THIS write: its id must be strictly -# greater than the pre-write boundary AND its body must exactly match what we -# submitted (see header comment: tea's exit code is not trustworthy evidence -# of a durable write on its own). Prints the matched comment ID on success. +# Independently re-fetch (all pages of) the issue's comments and require a +# comment attributable to THIS invocation: id strictly greater than the +# pre-write boundary AND author login equal to the acting identity AND exact +# body match. tea's exit code is not trustworthy evidence of a durable write +# on its own (#865); id-above-boundary alone is only temporal ordering, so the +# author-login check is what excludes a concurrent write by a DIFFERENT +# identity. Prints the matched comment ID on success. +# +# Residual (documented, not eliminable without a tea-emitted created-record +# id, which tea 0.11.1 does not reliably provide): a concurrent write by the +# SAME identity with an identical body inside the boundary window could still +# be accepted. That is a strictly narrower window than temporal-only matching. gitea_verify_comment_posted() { - local issue_number="$1" comment_body="$2" boundary="$3" - local readback_response_file status + local issue_number="$1" comment_body="$2" boundary="$3" acting_login="$4" + local merged_file - readback_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") - trap 'rm -f "$readback_response_file"' RETURN + merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") + trap 'rm -f "$merged_file"' RETURN - if ! status=$(curl -sS -o "$readback_response_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ - "$GITEA_API_BASE/issues/$issue_number/comments"); then - echo "Error: Gitea comment read-back transport failed" >&2 - return 1 - fi - if [[ "$status" != "200" ]]; then - echo "Error: Gitea comment read-back failed with HTTP $status" >&2 - return 1 - fi + gitea_fetch_all "$GITEA_API_BASE/issues/$issue_number/comments" "$merged_file" || return 1 - EXPECTED_COMMENT_BODY="$comment_body" BOUNDARY_COMMENT_ID="$boundary" \ - python3 - "$readback_response_file" <<'PY' + EXPECTED_COMMENT_BODY="$comment_body" BOUNDARY_COMMENT_ID="$boundary" ACTING_LOGIN="$acting_login" \ + python3 - "$merged_file" <<'PY' import json import os import sys @@ -172,17 +252,22 @@ try: raise ValueError("response is not a comment list") expected_body = os.environ["EXPECTED_COMMENT_BODY"] boundary = int(os.environ["BOUNDARY_COMMENT_ID"]) - # Require both: created-after-boundary (fresh write) AND exact body match. + acting_login = os.environ["ACTING_LOGIN"] + # Attribution to THIS write: created-after-boundary AND authored by the + # acting identity AND exact body match. The author check excludes a + # concurrent DIFFERENT-identity writer that id+body alone would admit. matches = [ c for c in comments if isinstance(c, dict) and isinstance(c.get("id"), int) and c.get("id") > boundary + and (c.get("user") or {}).get("login") == acting_login and c.get("body") == expected_body ] if not matches: raise ValueError( - "no comment created by this write matched (id > boundary and exact body); " + "no comment attributable to this write matched " + "(id > boundary, acting identity, exact body); " "tea may have silently no-opped (#865)" ) comment_id = max(c["id"] for c in matches) @@ -206,9 +291,11 @@ elif [[ "$PLATFORM" == "gitea" ]]; then exit 1 } - # Resolve the REST endpoint and record the pre-write boundary BEFORE the - # write, so the read-back can require a strictly-newer comment id. + # Resolve the REST endpoint, the acting identity, and the pre-write + # boundary BEFORE the write, so the read-back can require a strictly-newer + # comment id authored by this identity. gitea_resolve_api || exit 1 + ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 boundary=$(gitea_max_comment_id "$ISSUE_NUMBER") || exit 1 TEA_ARGS=(comment "$ISSUE_NUMBER" "$COMMENT" --repo "$REPO_SLUG" --login "$GITEA_LOGIN_NAME") @@ -220,7 +307,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then fi tea "${TEA_ARGS[@]}" - comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT" "$boundary") || { + comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT" "$boundary" "$ACTING_LOGIN") || { echo "Error: could not verify comment landed on Gitea issue #$ISSUE_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 exit 1 } diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index b57fd43b..0ba60760 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -193,8 +193,9 @@ PY } # Resolve and cache the Gitea REST endpoint + token for the current remote. -# Populates GITEA_API_BASE and GITEA_API_TOKEN. Returns non-zero (with a -# clear stderr message) on any resolution failure. +# Populates GITEA_API_ROOT (…/api/v1), GITEA_API_BASE (…/api/v1/repos/), +# and GITEA_API_TOKEN. Returns non-zero (with a clear stderr message) on any +# resolution failure. gitea_resolve_api() { local host configured_url repo @@ -211,31 +212,85 @@ gitea_resolve_api() { echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 return 1 } - GITEA_API_BASE="${configured_url%/}/api/v1/repos/$repo" + GITEA_API_ROOT="${configured_url%/}/api/v1" + GITEA_API_BASE="$GITEA_API_ROOT/repos/$repo" return 0 } -# Print the maximum existing review id on a PR (0 if none). This is the -# pre-write BOUNDARY: Gitea pull-review ids are monotonic, so any review -# submitted by a subsequent `tea pr approve`/`reject` has an id strictly -# greater than this value. Bounding the read-back this way is what turns the -# check into a genuine write-verification rather than a match against any -# historical review — the same never-trust-exit-zero discipline #865 requires -# for comments, applied to the review STATE itself. -gitea_max_review_id() { - local pr_number="$1" response_file status +# Fetch every page of a Gitea list endpoint into $2 (merged into one JSON +# array). Gitea paginates list responses, so a single-page read would +# false-negative once a newly created review/comment lands beyond page 1. +# Walks page=1,2,… until a short or empty page is returned so the merged array +# is exhaustive. $1 is the endpoint URL with NO query string. Returns non-zero +# (clear stderr) on any transport / HTTP / parse failure. +gitea_fetch_all() { + local base_url="$1" dest="$2" page=1 limit=50 status page_file count - response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-boundary.XXXXXX") + printf '[]' > "$dest" + while :; do + page_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-page.XXXXXX") + if ! status=$(curl -sS -o "$page_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "${base_url}?limit=${limit}&page=${page}"); then + rm -f "$page_file" + echo "Error: Gitea list read transport failed" >&2 + return 1 + fi + if [[ "$status" != "200" ]]; then + rm -f "$page_file" + echo "Error: Gitea list read failed with HTTP $status" >&2 + return 1 + fi + count=$(DEST="$dest" python3 - "$page_file" <<'PY' +import json +import os +import sys + +try: + with open(os.environ["DEST"], encoding="utf-8") as merged_file: + merged = json.load(merged_file) + with open(sys.argv[1], encoding="utf-8") as page_file: + page = json.load(page_file) + if not isinstance(page, list): + raise ValueError("page response is not a list") + merged.extend(item for item in page if isinstance(item, dict)) + with open(os.environ["DEST"], "w", encoding="utf-8") as merged_file: + json.dump(merged, merged_file) +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not merge Gitea list page: {error}", file=sys.stderr) + raise SystemExit(1) +print(len(page)) +PY +) || { rm -f "$page_file"; return 1; } + rm -f "$page_file" + [[ "$count" -lt "$limit" ]] && break + page=$((page + 1)) + if [[ "$page" -gt 1000 ]]; then + echo "Error: Gitea list pagination exceeded 1000 pages" >&2 + return 1 + fi + done + return 0 +} + +# Resolve the login of the identity the API token authenticates as (GET +# /user). Used to attribute a read-back review to THIS action's reviewer so a +# concurrent review from a DIFFERENT identity cannot satisfy verification. +# Prints the login on success. +gitea_authenticated_login() { + local response_file status + + response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-whoami.XXXXXX") trap 'rm -f "$response_file"' RETURN if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ -H "Authorization: token $GITEA_API_TOKEN" \ - "$GITEA_API_BASE/pulls/$pr_number/reviews"); then - echo "Error: Gitea review boundary read transport failed" >&2 + "$GITEA_API_ROOT/user"); then + echo "Error: Gitea authenticated-identity read transport failed" >&2 return 1 fi if [[ "$status" != "200" ]]; then - echo "Error: Gitea review boundary read failed with HTTP $status" >&2 + echo "Error: Gitea authenticated-identity read failed with HTTP $status" >&2 return 1 fi @@ -243,11 +298,39 @@ gitea_max_review_id() { import json import sys +try: + with open(sys.argv[1], encoding="utf-8") as response: + user = json.load(response) + login = user.get("login") if isinstance(user, dict) else None + if not isinstance(login, str) or not login: + raise ValueError("missing authenticated login") +except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: + print(f"Error: could not resolve authenticated Gitea identity: {error}", file=sys.stderr) + raise SystemExit(1) +print(login) +PY +} + +# Print the maximum existing review id on a PR (0 if none). This is the +# pre-write BOUNDARY: Gitea pull-review ids are monotonic, so any review +# submitted by a subsequent `tea pr approve`/`reject` has an id strictly +# greater than this value. Paginates fully so a boundary review beyond page 1 +# is still counted. +gitea_max_review_id() { + local pr_number="$1" merged_file + + merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-boundary.XXXXXX") + trap 'rm -f "$merged_file"' RETURN + + gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$merged_file" || return 1 + + python3 - "$merged_file" <<'PY' +import json +import sys + try: with open(sys.argv[1], encoding="utf-8") as response: reviews = json.load(response) - if not isinstance(reviews, list): - raise ValueError("response is not a review list") ids = [r.get("id") for r in reviews if isinstance(r, dict) and isinstance(r.get("id"), int)] print(max(ids) if ids else 0) except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: @@ -258,21 +341,32 @@ PY # Independently verify that `tea pr approve`/`reject` produced a durable review # record — never trust tea's exit code alone (#865, same defect class). Require -# a review that was created by THIS action: its id must be strictly greater -# than the pre-write boundary, its state must equal the expected state -# (APPROVED / REQUEST_CHANGES), and it must have been submitted against the -# PR's current head commit. Prints the matched review id on success; fails -# closed (non-zero, clear stderr) if no such review is found. +# a review attributable to THIS action: its id must be strictly greater than +# the pre-write boundary, its author login must equal the acting identity, its +# state must equal the expected state (APPROVED / REQUEST_CHANGES), and it must +# have been submitted against the PR's current head commit. The reviews list is +# paginated fully so a matching review beyond page 1 is still found. Prints the +# matched review id on success; fails closed (non-zero, clear stderr) if no +# such review is found. +# +# id-above-boundary alone is only temporal ordering; the author-login check is +# what excludes a concurrent review submitted by a DIFFERENT identity. +# +# Residual (documented, not eliminable without a tea-emitted created-record id, +# which tea 0.11.1 does not reliably provide for approve/reject): a concurrent +# review by the SAME identity with the same state against the same head inside +# the boundary window could still be accepted. That is strictly narrower than +# temporal-only matching. # # Args: $1 = PR number, $2 = expected state (APPROVED|REQUEST_CHANGES), -# $3 = pre-write boundary review id. +# $3 = pre-write boundary review id, $4 = acting reviewer login. gitea_verify_review_submitted() { - local pr_number="$1" expected_state="$2" boundary="$3" - local pr_response_file reviews_response_file status head_sha review_id + local pr_number="$1" expected_state="$2" boundary="$3" acting_login="$4" + local pr_response_file reviews_merged_file status head_sha review_id pr_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") - reviews_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-state.XXXXXX") - trap 'rm -f "$pr_response_file" "$reviews_response_file"' RETURN + reviews_merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-state.XXXXXX") + trap 'rm -f "$pr_response_file" "$reviews_merged_file"' RETURN # Resolve the PR's current head commit so the review can be pinned to it. if ! status=$(curl -sS -o "$pr_response_file" -w '%{http_code}' \ @@ -302,19 +396,10 @@ print(head_sha) PY ) || return 1 - if ! status=$(curl -sS -o "$reviews_response_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ - "$GITEA_API_BASE/pulls/$pr_number/reviews"); then - echo "Error: Gitea review read-back transport failed" >&2 - return 1 - fi - if [[ "$status" != "200" ]]; then - echo "Error: Gitea review read-back failed with HTTP $status" >&2 - return 1 - fi + gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$reviews_merged_file" || return 1 - review_id=$(EXPECTED_STATE="$expected_state" BOUNDARY_REVIEW_ID="$boundary" EXPECTED_HEAD_SHA="$head_sha" \ - python3 - "$reviews_response_file" <<'PY' + review_id=$(EXPECTED_STATE="$expected_state" BOUNDARY_REVIEW_ID="$boundary" EXPECTED_HEAD_SHA="$head_sha" ACTING_LOGIN="$acting_login" \ + python3 - "$reviews_merged_file" <<'PY' import json import os import sys @@ -327,22 +412,24 @@ try: expected_state = os.environ["EXPECTED_STATE"] boundary = int(os.environ["BOUNDARY_REVIEW_ID"]) expected_head = os.environ["EXPECTED_HEAD_SHA"] - # Require all of: created-after-boundary (this action's write), the - # expected review state, and pinned to the PR's current head commit. - # The monotonic id boundary is what proves "submitted by this action" - # rather than matching some pre-existing historical review. + acting_login = os.environ["ACTING_LOGIN"] + # Attribution to THIS action: created-after-boundary AND submitted by the + # acting reviewer identity AND expected state AND pinned to the PR's + # current head commit. The author check excludes a concurrent + # DIFFERENT-identity review that id+state+head alone would admit. matches = [ r for r in reviews if isinstance(r, dict) and isinstance(r.get("id"), int) and r.get("id") > boundary + and (r.get("user") or {}).get("login") == acting_login and r.get("state") == expected_state and r.get("commit_id") == expected_head ] if not matches: raise ValueError( - f"no {expected_state} review created by this action found " - "(id > boundary, expected state, current head); " + f"no {expected_state} review attributable to this action found " + "(id > boundary, acting identity, expected state, current head); " "tea may have silently failed (#865 defect class)" ) review_id = max(r["id"] for r in matches) @@ -395,6 +482,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # strictly-newer review created by THIS action (never trust tea's # exit code alone — #865 defect class applies to the review state). gitea_resolve_api || exit 1 + ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 # tea v0.11.1 defines no --comment/-comment flag on `pr approve`; # route any review body via the durable comment API instead (#835). @@ -406,7 +494,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then TEA_ARGS+=(--login "$LOGIN_OVERRIDE") fi tea "${TEA_ARGS[@]}" - review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "APPROVED" "$review_boundary") || { + review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "APPROVED" "$review_boundary" "$ACTING_LOGIN") || { echo "Error: could not verify an APPROVED review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 exit 1 } @@ -427,6 +515,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # Record the pre-write review-id boundary BEFORE the write (see the # approve path above for the rationale). gitea_resolve_api || exit 1 + ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 # tea v0.11.1 defines no --comment/-comment flag on `pr reject`; # route the review body via the durable comment API instead (#835). @@ -438,7 +527,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then TEA_ARGS+=(--login "$LOGIN_OVERRIDE") fi tea "${TEA_ARGS[@]}" - review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "REQUEST_CHANGES" "$review_boundary") || { + review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "REQUEST_CHANGES" "$review_boundary" "$ACTING_LOGIN") || { echo "Error: could not verify a REQUEST_CHANGES review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 exit 1 } diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index 8dd8a06b..b036b8e4 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -1,23 +1,28 @@ #!/usr/bin/env bash # Regression harness for issue-comment.sh top-level `tea comment` invocation and -# its BOUNDED read-back verification (#865). +# its BOUNDED, INVOCATION-ATTRIBUTED, PAGINATED read-back verification (#865). # # The #865 bug: `tea issue comment ...` (a nonexistent subcommand on tea # v0.11.1) silently no-ops and exits 0, so a comment is never posted. A naive # read-back that matches ANY historical comment by body would falsely report # success whenever an identically-bodied comment already exists from a prior -# run. This harness proves the wrapper: +# run. Merely bounding by "id > pre-write max" is also insufficient: it accepts +# ANY newer matching comment, including one a CONCURRENT DIFFERENT identity +# posted while this tea invocation created nothing. This harness proves the +# wrapper: # 1. uses the top-level `tea comment` form (never `tea issue comment`); # 2. records the pre-write maximum comment id as a boundary and requires a # strictly-newer comment on read-back, so a pre-existing identical body # does NOT satisfy verification (fails closed); -# 3. reports success only when a genuinely new comment (id > boundary) with -# the exact body appears. +# 3. attributes the matched comment to the acting identity (GET /user login), +# so a concurrent DIFFERENT-identity write does NOT satisfy verification; +# 4. reports success only when a genuinely new comment (id > boundary) with +# the exact body AND the acting author appears. # # The `tea` stub NEVER creates a comment (it mimics the silent no-op); the # "server" comment state is modeled entirely by the curl stub's responses, so # the fresh-success vs. no-op distinction is driven purely by whether the -# post-write read-back surfaces a new id. +# post-write read-back surfaces a new, correctly-attributed id. set -euo pipefail @@ -42,7 +47,10 @@ git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/sta ISSUE_NUMBER=7 API_BASE="https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack" +API_ROOT="https://git.mosaicstack.dev/api/v1" BODY='durable "note" -- marker' +ACTING_LOGIN="primary-reviewer" +FOREIGN_LOGIN="other-writer" CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY' import json @@ -91,8 +99,9 @@ exit 92 SH chmod +x "$BIN_DIR/tea" -# curl stub: serves GET .../issues/7/comments. First call = pre-write boundary, -# second call = post-write read-back. The boundary always contains a +# curl stub: serves GET /user (acting identity) and GET .../issues/7/comments +# (paginated: ?limit=&page=). The comments endpoint's first call = pre-write +# boundary, second call = post-write read-back. The boundary always contains a # pre-existing comment (id 50) whose body is IDENTICAL to the one under test, # which is exactly the condition a body-only match would trip over. cat > "$BIN_DIR/curl" <<'SH' @@ -114,6 +123,10 @@ while [[ $# -gt 0 ]]; do esac done +# Strip any query string so pagination params don't defeat path matching, but +# still log the full URL (including ?limit=&page=) so the test can assert the +# read-back paginated. +path="${url%%\?*}" printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG" write_response() { @@ -123,41 +136,58 @@ write_response() { printf '%s' "$status" } -if [[ "$method" == "GET" && "$url" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then +if [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_ROOT/user" ]]; then + write_response 200 "$(ISSUE_COMMENT_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY' +import json +import os +print(json.dumps({"login": os.environ["ISSUE_COMMENT_LOGIN"]})) +PY +)" +elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then calls_file="$ISSUE_COMMENT_CALLS" if [[ -f "$calls_file" ]]; then # post-write read-back - if [[ "$ISSUE_COMMENT_TEST_MODE" == "fresh-success" ]]; then - response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' + response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \ + ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" \ + ISSUE_COMMENT_FOREIGN_LOGIN="$ISSUE_COMMENT_FOREIGN_LOGIN" \ + ISSUE_COMMENT_TEST_MODE="$ISSUE_COMMENT_TEST_MODE" python3 - <<'PY' import json import os body = os.environ["ISSUE_COMMENT_BODY"] -print(json.dumps([ - {"id": 50, "body": body}, - {"id": 60, "body": body}, -])) -PY -) - else - # no-op: nothing new landed; the pre-existing id-50 comment remains. - response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' -import json -import os +acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"] +foreign = os.environ["ISSUE_COMMENT_FOREIGN_LOGIN"] +mode = os.environ["ISSUE_COMMENT_TEST_MODE"] -body = os.environ["ISSUE_COMMENT_BODY"] -print(json.dumps([{"id": 50, "body": body}])) +if mode == "fresh-success": + # A genuinely new comment (id 60 > boundary 50) authored by the acting + # identity. + records = [ + {"id": 50, "body": body, "user": {"login": acting}}, + {"id": 60, "body": body, "user": {"login": acting}}, + ] +elif mode == "foreign-identity": + # A concurrent new comment (id 60 > boundary 50) with the SAME body but a + # DIFFERENT author. tea created nothing; attribution must reject this. + records = [ + {"id": 50, "body": body, "user": {"login": acting}}, + {"id": 60, "body": body, "user": {"login": foreign}}, + ] +else: + # no-op: nothing new landed; the pre-existing id-50 comment remains. + records = [{"id": 50, "body": body, "user": {"login": acting}}] +print(json.dumps(records)) PY ) - fi else : > "$calls_file" - response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" python3 - <<'PY' + response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \ + ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY' import json import os - body = os.environ["ISSUE_COMMENT_BODY"] -print(json.dumps([{"id": 50, "body": body}])) +acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"] +print(json.dumps([{"id": 50, "body": body, "user": {"login": acting}}])) PY ) fi @@ -184,7 +214,10 @@ run_comment() { ISSUE_COMMENT_CALLS="$CALLS_FILE" \ ISSUE_COMMENT_TEST_MODE="$mode" \ ISSUE_COMMENT_EXPECTED_BODY="$BODY" \ + ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \ + ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ ISSUE_COMMENT_API_BASE="$API_BASE" \ + ISSUE_COMMENT_API_ROOT="$API_ROOT" \ "$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY" ) > "$OUTPUT_FILE" 2>&1 } @@ -206,11 +239,27 @@ if grep -q '^issue comment' "$TEA_LOG"; then echo "FAIL: wrapper used the broken 'tea issue comment' subcommand" >&2 exit 1 fi -[[ "$(grep -c "^GET $API_BASE/issues/7/comments$" "$CURL_LOG")" == "2" ]] +[[ "$(grep -c "^GET $API_BASE/issues/7/comments?" "$CURL_LOG")" == "2" ]] +# Read-back must be paginated (limit + page query params present). +grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=1$" "$CURL_LOG" +# Attribution must have resolved the acting identity via GET /user. +grep -q "^GET $API_ROOT/user$" "$CURL_LOG" -# Case 2: a genuinely new comment (id 60 > boundary 50) verifies successfully. +# Case 2: a concurrent DIFFERENT-identity write (id 60 > boundary, same body, +# foreign author) must FAIL CLOSED — temporal ordering is not attribution. +if run_comment foreign-identity; then + echo "FAIL: wrapper accepted a concurrent comment authored by a different identity" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: read-back matched a different-identity comment (attribution bypassed)" >&2 + exit 1 +fi + +# Case 3: a genuinely new comment (id 60 > boundary 50, acting author) verifies. run_comment fresh-success grep -q 'Added and verified comment on Gitea issue #7 (comment ID 60)' "$OUTPUT_FILE" grep -q "^comment 7 " "$TEA_LOG" -echo "issue-comment.sh bounded read-back regression passed" +echo "issue-comment.sh bounded + attributed read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index bf32ba4d..c28310c3 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -23,6 +23,9 @@ cleanup() { } trap cleanup EXIT +ACTING_LOGIN="review-bot" +FOREIGN_LOGIN="other-writer" + mkdir -p "$REPO_DIR" "$BIN_DIR" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git @@ -67,7 +70,7 @@ if [[ "$*" == *" -comment "* || "$*" == *" --comment "* || "$*" == *" -comment" fi case "${PR_REVIEW_TEST_MODE:-}" in - approve) + approve|paginated-approve|foreign-review) [[ "$*" == "pr approve 123 --repo mosaicstack/stack --login mosaicstack" ]] || exit 90 ;; request-changes) @@ -127,6 +130,12 @@ while [[ $# -gt 0 ]]; do esac done +# Strip any query string so pagination params (?limit=&page=) don't defeat +# path matching, but keep the full URL in the log so tests can assert that the +# read-back paginated. +path="${url%%\?*}" +page="${url##*page=}" +[[ "$page" == "$url" ]] && page=1 printf '%s %s\n' "$method" "$url" >> "$PR_REVIEW_CURL_LOG" write_response() { @@ -144,32 +153,83 @@ case "${PR_REVIEW_TEST_MODE:-}" in write-http-failure) write_response 500 '{"message":"simulated rejection"}' ;; - approve|request-changes|comment-success|http-success|prefix-success|subpath-success|port-success|scp-ssh-success|url-ssh-success|ssh-transport-port-success|explicit-default-port-success|readback-failure) - if [[ "$method" == "GET" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then - # First GET = pre-write review-id BOUNDARY; second GET = post-write - # read-back that must surface a strictly-newer review created by - # THIS action (id 200 > boundary 100), with the expected state and - # pinned to the PR's current head commit. + approve|request-changes|paginated-approve|foreign-review|comment-success|http-success|prefix-success|subpath-success|port-success|scp-ssh-success|url-ssh-success|ssh-transport-port-success|explicit-default-port-success|readback-failure) + if [[ "$method" == "GET" && "$path" == "$PR_REVIEW_API_ROOT/user" ]]; then + # Acting reviewer identity used for invocation attribution. + write_response 200 "$(PR_REVIEW_LOGIN="$PR_REVIEW_ACTING_LOGIN" python3 - <<'PY' +import json +import os +print(json.dumps({"login": os.environ["PR_REVIEW_LOGIN"]})) +PY +)" + elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then + # First (boundary) call precedes the write; later calls are the + # post-write read-back that must surface a strictly-newer review + # created by THIS action (id 200 > boundary 100), authored by the + # acting identity, with the expected state and pinned to the PR's + # current head commit. The list is served PAGINATED so a target + # beyond page 1 is only found by a fully-paginating read-back. calls_file="${PR_REVIEW_REVIEW_CALLS:-/dev/null}" if [[ -f "$calls_file" ]]; then - state="APPROVED" - [[ "$PR_REVIEW_TEST_MODE" == "request-changes" ]] && state="REQUEST_CHANGES" - response=$(PR_REVIEW_STATE="$state" python3 - <<'PY' + phase="post" + else + : > "$calls_file" + phase="boundary" + fi + state="APPROVED" + [[ "$PR_REVIEW_TEST_MODE" == "request-changes" ]] && state="REQUEST_CHANGES" + response=$(PR_REVIEW_PHASE="$phase" PR_REVIEW_PAGE="$page" \ + PR_REVIEW_MODE="$PR_REVIEW_TEST_MODE" PR_REVIEW_STATE="$state" \ + PR_REVIEW_ACTING_LOGIN="$PR_REVIEW_ACTING_LOGIN" \ + PR_REVIEW_FOREIGN_LOGIN="$PR_REVIEW_FOREIGN_LOGIN" python3 - <<'PY' import json import os -print(json.dumps([ - {"id": 100, "state": "COMMENT", "commit_id": "oldsha0000"}, - {"id": 200, "state": os.environ["PR_REVIEW_STATE"], "commit_id": "HEADSHA_FEEDFACE"}, -])) +phase = os.environ["PR_REVIEW_PHASE"] +page = int(os.environ["PR_REVIEW_PAGE"]) +mode = os.environ["PR_REVIEW_MODE"] +state = os.environ["PR_REVIEW_STATE"] +acting = os.environ["PR_REVIEW_ACTING_LOGIN"] +foreign = os.environ["PR_REVIEW_FOREIGN_LOGIN"] + + +def review(review_id, review_state, commit, login): + return { + "id": review_id, + "state": review_state, + "commit_id": commit, + "user": {"login": login}, + } + + +if phase == "boundary": + records = [review(100, "COMMENT", "oldsha0000", acting)] if page == 1 else [] +elif mode == "paginated-approve": + # A full first page (50 non-matching records) forces the read-back to + # request page 2, where the genuine matching review lives. + if page == 1: + records = [review(101 + i, "COMMENT", "oldsha0000", acting) for i in range(50)] + elif page == 2: + records = [review(200, state, "HEADSHA_FEEDFACE", acting)] + else: + records = [] +elif mode == "foreign-review": + # A concurrent APPROVED review at the current head, but authored by a + # DIFFERENT identity. tea created nothing; attribution must reject this. + records = [review(200, state, "HEADSHA_FEEDFACE", foreign)] if page == 1 else [] +else: + if page == 1: + records = [ + review(100, "COMMENT", "oldsha0000", acting), + review(200, state, "HEADSHA_FEEDFACE", acting), + ] + else: + records = [] +print(json.dumps(records)) PY ) - else - : > "$calls_file" - response='[{"id":100,"state":"COMMENT","commit_id":"oldsha0000"}]' - fi write_response 200 "$response" - elif [[ "$method" == "GET" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123" ]]; then + elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123" ]]; then write_response 200 '{"head":{"sha":"HEADSHA_FEEDFACE"}}' elif [[ "$method" == "POST" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' @@ -222,6 +282,7 @@ run_review() { local remote_url="${5:-https://git.mosaicstack.dev/mosaicstack/stack.git}" local expected_repo="${6:-mosaicstack/stack}" local expected_api_base="${configured_url%/}/api/v1/repos/$expected_repo" + local expected_api_root="${configured_url%/}/api/v1" git -C "$REPO_DIR" remote set-url origin "$remote_url" write_credentials "$configured_url" : > "$TEA_LOG" @@ -238,6 +299,9 @@ run_review() { PR_REVIEW_TEST_MODE="$mode" \ PR_REVIEW_EXPECTED_BODY="$comment" \ PR_REVIEW_EXPECTED_API_BASE="$expected_api_base" \ + PR_REVIEW_API_ROOT="$expected_api_root" \ + PR_REVIEW_ACTING_LOGIN="$ACTING_LOGIN" \ + PR_REVIEW_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ "$SCRIPT_DIR/pr-review.sh" -n 123 -a "$action" ${comment:+-c "$comment"} ) > "$OUTPUT_FILE" 2>&1 } @@ -246,14 +310,36 @@ run_review approve approve grep -q '^pr approve 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" # #865: the approval STATE itself is read back — a pre-write boundary GET and a -# post-write read-back GET on the reviews endpoint, plus a PR head lookup. -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" +# post-write read-back GET on the (paginated) reviews endpoint, plus a PR head +# lookup and an acting-identity (GET /user) resolution for attribution. +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/user$' "$CURL_LOG" if grep -q 'comment' "$TEA_LOG"; then echo "Plain approve (no review body) unexpectedly touched comment persistence" >&2 exit 1 fi +# #865 (invocation attribution): a concurrent APPROVED review at the current +# head, authored by a DIFFERENT identity while tea created nothing, must NOT +# satisfy verification — id-above-boundary + state + head is only temporal +# ordering, not proof THIS reviewer wrote it. +if run_review foreign-review approve; then + echo "FAIL: approve accepted a review authored by a different identity" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: read-back matched a different-identity review (attribution bypassed)" >&2 + exit 1 +fi + +# #865 (pagination): a genuine matching review that lands beyond page 1 of the +# reviews list must still be found by a fully-paginating read-back. +run_review paginated-approve approve +grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=2$' "$CURL_LOG" + # #835: tea v0.11.1 defines no --comment/-comment flag on `pr approve`. A # review body supplied alongside approve must be routed through the durable # comment REST API instead of being passed to `tea` directly. @@ -268,7 +354,7 @@ grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$ run_review request-changes request-changes changes-required grep -q '^pr reject 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" grep -q 'Requested changes and verified on Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$OUTPUT_FILE" -- 2.49.1 From 9384f0bc0aad87779519e6745848521f1ec7a63e Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 19:46:32 -0500 Subject: [PATCH 04/15] fix(tools): write Gitea reviews/comments via REST POST and verify by exact created id (#865) Replace the tea-based write + boundary/author read-back with a direct Gitea REST POST that returns the created record's id, and verify that exact record. BLOCKER 2 (credential ordering): resolve the acting identity, the write token, and the read-back token from the SAME effective login. A --login override now selects the credential used for the POST, GET /user, and the GET-by-id read-back, so an overridden write is verified against the identity that performed it -- not the host default. Login-name resolution is best-effort and non-fatal (the override always wins; otherwise fall back to the host credential), so exotic/ported hosts still resolve a token. BLOCKER 1+3 (attribution + tautological tests): the write is now POST /issues/{n}/comments or POST /pulls/{n}/reviews (event + body + commit_id == PR head), parsing the provider-returned created id. Verification GETs that exact id and checks author == acting identity and body (comments) or state + commit_id (reviews). Keying on the created id closes the concurrency window: a no-op create yields no id and fails closed with no list-scan fallback, and a concurrent same-identity record has a different id. The review body travels in the review submit, removing the separate detached comment. Tests: the curl stub now models a real server with persistent on-disk review/comment state -- a POST actually creates+persists a record and returns its id, and the read-back reads that same state (no fabricated record for the wrapper to find). Adds same-identity no-op-concurrent and author-mismatch fail-closed cases for both comments and reviews, and >page-1 pagination coverage for both. README "Durable review provenance" refreshed for the REST mechanism. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 19 +- .../framework/tools/git/detect-platform.sh | 48 ++ .../framework/tools/git/issue-comment.sh | 290 +++++---- .../mosaic/framework/tools/git/pr-review.sh | 475 ++++++++------- .../tools/git/test-issue-comment-readback.sh | 322 ++++++---- .../tools/git/test-pr-review-gitea-comment.sh | 557 ++++++++++-------- 6 files changed, 1024 insertions(+), 687 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index 912de1bb..8cf68280 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -6,21 +6,24 @@ These scripts provide host-aware GitHub and Gitea issue, pull-request, milestone A successful provider write command—or a wrapper message based only on that command's exit code—is **not** durable review provenance. Review comments, approvals, and change requests count as durable provenance only after the wrapper reads the created provider record back and verifies that it was created by _this_ write. -`pr-review.sh` therefore fails closed when a Gitea comment cannot be written, its created comment ID cannot be identified, or provider read-back does not match. It reports comment success only after that read-back verification passes. The `approve` and `request-changes` actions apply the same discipline to the review **state** itself: they record the maximum existing review id _before_ invoking `tea pr approve`/`reject`, then require a review whose id is strictly greater than that boundary, whose **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), whose state matches the requested action (`APPROVED` / `REQUEST_CHANGES`), and whose reviewed commit equals the PR's current head. tea's exit code alone is never treated as evidence the review landed. +**The write is a direct Gitea REST `POST` that returns the created record's id.** Neither wrapper writes through `tea` — tea 0.11.1 can silently no-op while exiting 0 and cannot emit the id of a record it creates, so its exit code is worthless as proof of a durable write (#865). Instead: -`issue-comment.sh` applies the same fail-closed, boundary-bounded read-back to issue comments: it records the maximum existing comment id _before_ posting via `tea comment`, then re-fetches the issue's comments via the Gitea REST API and requires a comment whose id is strictly greater than that boundary, **whose author login equals the acting identity**, **and** whose body exactly matches what was submitted. Bounding the read-back by the pre-write id is essential — a body-only match across all history would falsely report success if `tea comment` silently no-ops (the #865 bug) while an identically-bodied comment already existed from a prior run. Gitea comment and review ids are monotonic, so `id > boundary` reliably means "created after this write began". +- Comments (`issue-comment.sh`, and the `comment` action of `pr-review.sh`) `POST /api/v1/repos/{owner}/{repo}/issues/{index}/comments`, requiring a `201` and parsing the created comment's `id` from the response body. +- Reviews (`approve` / `request-changes`) `POST /api/v1/repos/{owner}/{repo}/pulls/{index}/reviews` with the `event` (`APPROVED` / `REQUEST_CHANGES`), the review `body`, and `commit_id` pinned to the PR's current head, then parse the created review's `id`. The review body travels _in the review submit itself_ — there is no separate detached comment to reconcile (a Gitea `REQUEST_CHANGES` review requires a non-empty body, which the submit carries). -**Invocation attribution, not just temporal ordering.** `id > boundary` alone only proves a record was created after the write began; it would still be satisfied by a _concurrent_ write from a _different_ identity while this `tea` invocation created nothing. Both wrappers therefore additionally require the accepted record's author login to equal the identity the API token authenticates as, narrowing the match to this invocation's writer. The one residual window — a concurrent write by the _same_ identity with an identical body/state inside the boundary window — cannot be eliminated without a tea-emitted created-record id, which tea 0.11.1 does not reliably provide; it is strictly narrower than temporal-only matching and is documented in-code. +**Verification keys on that exact provider-returned id.** The wrapper then `GET`s that one record directly — `GET /issues/comments/{id}` or `GET /pulls/{n}/reviews/{id}` — and requires that its `id` equals the created id, its **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), and, for comments, its body exactly matches what was submitted, or, for reviews, its state matches the requested action and its reviewed `commit_id` equals the PR head. The write, the `/user` identity lookup, and the read-back all use the **same** credential — the effective login's token, or the host credential when no login is named — so the write is verified against the identity that actually performed it. -**Full pagination.** Gitea paginates list endpoints, so a single-page read of the comments or reviews list would false-negative once the freshly created record lands beyond the first page. Both the pre-write boundary computation and the post-write read-back walk every page (`?limit=&page=1,2,…` until a short/empty page) so the match is exhaustive regardless of how many comments or reviews already exist. +**This closes the concurrency window rather than documenting it.** Because verification keys on the id the create returned, a no-op create yields no id and fails closed with no list-scan fallback, and a _concurrent_ record — even one written by the _same_ identity with an identical body/state — has a _different_ id and cannot be mistaken for this write. There is no residual same-identity window: the earlier boundary-and-author heuristic (accept any `id > pre-write-max` with a matching author) is replaced entirely by exact-id attribution. + +**Full pagination.** After the exact-id read-back, each wrapper also confirms the created id is enumerable in the record list, walking every page (`?limit=&page=1,2,…` until a short/empty page) so a record that lands beyond the first page is still found regardless of how many comments or reviews already exist. ## `tea` invocation notes (Gitea) -- tea v0.11.1 has **no `comment` subcommand under `tea pr` or `tea issue`**. The correct invocation is the **top-level** `tea comment [--repo ...] [--login ...]`. The `tea pr comment` / `tea issue comment` forms don't error — tea silently falls through to a no-op and still exits 0, producing a false-success write (#865). Always use the top-level form. -- `tea pr approve` and `tea pr reject` take an optional review comment/reason as a **trailing positional argument**, not a `--comment`/`-comment` flag (that flag does not exist on those subcommands). `pr-review.sh` avoids this positional form entirely for the approve/reject actions and instead posts any review comment through the same durable, read-back-verified comment API used for the `comment` action (see #835/#812) — the trailing-positional form remains available to callers who invoke `tea` directly, but is not used by these wrappers. +- tea v0.11.1 has **no `comment` subcommand under `tea pr` or `tea issue`** — the `tea pr comment` / `tea issue comment` forms don't error, they silently fall through to a no-op and still exit 0, producing a false-success write (#865). tea's write subcommands (`tea comment`, `tea pr approve`/`reject`) also cannot report the id of the record they create, so their exit code cannot prove a durable write. These wrappers therefore do **not** write reviews or comments through `tea` at all; they use direct Gitea REST `POST`s that return the created record's id (see "Durable review provenance" above). `tea` is consulted only to enumerate the login list for host→login resolution. +- Because the review body is carried in the `POST …/reviews` submit itself, there is no separate detached review comment, and the historical `tea pr approve`/`reject` trailing-positional-argument vs. nonexistent `--comment`/`-comment` flag hazard (#835) no longer applies to these wrappers — no review comment is ever passed to `tea`. -### `--login` passthrough +### `--login` override -Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea `tea` login for that single invocation. The override is appended to the `tea` command line **after** the detected default (`get_gitea_repo_args()` / `get_gitea_login[_for_host]()`), because tea honors only the **last** `--login` flag on its command line — an override placed before the default would be silently clobbered by it. Callers who need a different login than the host default should pass `--login ` rather than relying on ordering tricks or re-invoking `tea login` globally. +Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea login for that single invocation. The override selects **which credential the REST write, the `/user` identity lookup, and the read-back all use** — its token is resolved from the tea config for that login name (`get_gitea_token_for_login`), falling back to the repo host's credential when no login is named. Resolving the acting identity and the read-back from the _same_ login that performs the write is essential: a write performed under an overridden login must be verified against that login's identity, not the host default's. Callers who need a different login than the host default should pass `--login `. As a durable successor to this mechanism, consider giving each reviewer/approver slot its own dedicated Gitea login credential, so that author≠reviewer holds at the credential level rather than relying on wrapper-level `--login` bookkeeping. This is a recommendation for future hardening, not something implemented by this flag. diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 3111eb42..5ee119e2 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -563,6 +563,54 @@ get_gitea_token() { return 1 } +# Resolve the API token for a SPECIFIC tea login name from tea's own config +# (the same store tea itself writes/reads for `--login `). This is what +# lets a REST write be performed AS the selected --login identity: tea keys its +# per-login tokens by `name` in $XDG_CONFIG_HOME/tea/config.yml (default +# ~/.config/tea/config.yml), exactly as the `tea` CLI resolves them, so a +# --login override and its REST read-back bind to the SAME credential/identity. +# Prints the token on success; returns non-zero (no output) if the config or a +# matching login token cannot be found. Callers must not log the result. +get_gitea_token_for_login() { + local login_name="$1" config_file + [[ -n "$login_name" ]] || return 1 + config_file="${XDG_CONFIG_HOME:-$HOME/.config}/tea/config.yml" + [[ -f "$config_file" ]] || return 1 + + LOGIN_NAME="$login_name" python3 - "$config_file" <<'PY' +import os +import sys + +try: + import yaml +except ImportError: + raise SystemExit(1) + +try: + with open(sys.argv[1], encoding="utf-8") as handle: + config = yaml.safe_load(handle) +except (OSError, yaml.YAMLError): + raise SystemExit(1) + +wanted = os.environ["LOGIN_NAME"] +logins = config.get("logins") if isinstance(config, dict) else None +if not isinstance(logins, list): + raise SystemExit(1) + +for login in logins: + if not isinstance(login, dict): + continue + if str(login.get("name") or "") == wanted: + token = login.get("token") + if isinstance(token, str) and token: + print(token) + raise SystemExit(0) + break + +raise SystemExit(1) +PY +} + # Resolve HTTPS basic auth credentials for a Gitea host from ~/.git-credentials. # Prints "username:password" for direct curl -u consumption. Callers must not log it. get_gitea_basic_auth() { diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index 64f1a147..ce08d023 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -3,21 +3,24 @@ # Usage: issue-comment.sh -i -c [--login ] # # tea v0.11.1 defines no `comment` subcommand under `tea issue` (or `tea pr`); -# the correct invocation is the TOP-LEVEL `tea comment ` form. -# Calling the non-existent `tea issue comment ...` form does not error — tea -# silently falls through to a no-op and still exits 0, so a caller trusting -# the exit code alone believes a comment was posted when it was not (#865). -# Because that failure mode is silent, this script never trusts tea's exit -# code alone: after posting, it independently re-fetches the issue's comments -# via the Gitea REST API (curl — urllib is blocked by Cloudflare on this -# host) and fails closed if the posted body cannot be found. +# the non-existent `tea issue comment ...` form does not error — tea silently +# no-ops and still exits 0, so a caller trusting the exit code believes a +# comment was posted when it was not (#865). tea 0.11.1 also cannot reliably +# emit the id of a record it created, so an exit code is the ONLY signal it +# offers — and that signal is untrustworthy. This script therefore does not +# write via tea at all: it POSTs the comment through the Gitea REST API (which +# returns the created comment object, including its id), then GETs that exact +# id back and fails closed unless it matches. Keying verification to the +# provider-returned created id means a concurrent comment cannot masquerade as +# this write and a no-op create simply yields no id to verify. # -# --login override: the default `--login` is resolved from the local `tea` -# login list for this repo's host (get_gitea_login). Pass --login to -# override that default for this invocation only. The override is appended -# to the tea command line AFTER the detected default, because tea honors -# only the LAST `--login` flag on the command line — a flag placed before -# the default would be silently clobbered by it. +# --login override: the default login is resolved from the local `tea` login +# list for this repo's host (get_gitea_login). Pass --login to override +# it for this invocation only. The REST write, the /user identity read, and the +# read-back are ALL performed with the token of the EFFECTIVE login (the +# override when given), so the write and its verification bind to the same +# identity — a --login override is never written under one credential and +# verified under a different default one. set -e @@ -72,16 +75,25 @@ fi detect_platform >/dev/null -# Resolve and cache the Gitea REST endpoint + token for the current remote. -# Populates GITEA_API_ROOT (…/api/v1), GITEA_API_BASE (…/api/v1/repos/), -# and GITEA_API_TOKEN. Returns non-zero (with a clear stderr message) if any -# part of the resolution fails. -gitea_resolve_api() { - local host configured_url repo +# Resolve and cache the Gitea REST endpoint + token for the current remote, +# bound to a SPECIFIC login identity ($1). Populates GITEA_API_ROOT (…/api/v1), +# GITEA_API_BASE (…/api/v1/repos/), and GITEA_API_TOKEN. +# +# The token is resolved for the EFFECTIVE login (the --login override when +# given, otherwise the detected default) so that the single credential used for +# the write ALSO drives the /user identity read and the read-back — write token +# and read-back token are the same identity by construction (this is the +# credential-ordering fix: a --login override is no longer written under one +# credential and verified under a different default one). Falls back to the +# host-scoped credential only when the login has no token in tea's own config. +# Returns non-zero (clear stderr) on any resolution failure. +gitea_resolve_api_for_login() { + local effective_login="$1" host configured_url repo host=$(get_remote_host) - GITEA_API_TOKEN=$(get_gitea_token "$host") || { - echo "Error: Gitea token not found for comment read-back verification" >&2 + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + || GITEA_API_TOKEN=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for login '$effective_login' (comment write/read-back)" >&2 return 1 } configured_url=$(get_gitea_url_for_host "$host") || { @@ -192,54 +204,22 @@ print(login) PY } -# Print the maximum existing comment id on an issue (0 if none). This is the -# pre-write BOUNDARY: Gitea comment ids are monotonic, so any comment created -# by a subsequent write has an id strictly greater than this value. -gitea_max_comment_id() { - local issue_number="$1" merged_file - - merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-boundary.XXXXXX") - trap 'rm -f "$merged_file"' RETURN - - gitea_fetch_all "$GITEA_API_BASE/issues/$issue_number/comments" "$merged_file" || return 1 - - python3 - "$merged_file" <<'PY' -import json -import sys - -try: - with open(sys.argv[1], encoding="utf-8") as response: - comments = json.load(response) - ids = [c.get("id") for c in comments if isinstance(c, dict) and isinstance(c.get("id"), int)] - print(max(ids) if ids else 0) -except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: - print(f"Error: could not compute Gitea comment boundary: {error}", file=sys.stderr) - raise SystemExit(1) -PY -} - -# Independently re-fetch (all pages of) the issue's comments and require a -# comment attributable to THIS invocation: id strictly greater than the -# pre-write boundary AND author login equal to the acting identity AND exact -# body match. tea's exit code is not trustworthy evidence of a durable write -# on its own (#865); id-above-boundary alone is only temporal ordering, so the -# author-login check is what excludes a concurrent write by a DIFFERENT -# identity. Prints the matched comment ID on success. -# -# Residual (documented, not eliminable without a tea-emitted created-record -# id, which tea 0.11.1 does not reliably provide): a concurrent write by the -# SAME identity with an identical body inside the boundary window could still -# be accepted. That is a strictly narrower window than temporal-only matching. -gitea_verify_comment_posted() { - local issue_number="$1" comment_body="$2" boundary="$3" acting_login="$4" - local merged_file +# Confirm that the comment CREATED by this invocation ($2 = its provider id) is +# enumerable in the issue's full, paginated comment listing and is authored by +# the acting identity. Gitea paginates list responses, so a comment created +# beyond page 1 must still be found; walking every page also proves the created +# id is durably indexed against THIS issue rather than merely retrievable by id. +# Returns non-zero (clear stderr) if the exact created id is not present with a +# matching author. +gitea_confirm_comment_enumerable() { + local issue_number="$1" created_id="$2" acting_login="$3" merged_file merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") trap 'rm -f "$merged_file"' RETURN gitea_fetch_all "$GITEA_API_BASE/issues/$issue_number/comments" "$merged_file" || return 1 - EXPECTED_COMMENT_BODY="$comment_body" BOUNDARY_COMMENT_ID="$boundary" ACTING_LOGIN="$acting_login" \ + CREATED_COMMENT_ID="$created_id" ACTING_LOGIN="$acting_login" \ python3 - "$merged_file" <<'PY' import json import os @@ -250,65 +230,161 @@ try: comments = json.load(response) if not isinstance(comments, list): raise ValueError("response is not a comment list") - expected_body = os.environ["EXPECTED_COMMENT_BODY"] - boundary = int(os.environ["BOUNDARY_COMMENT_ID"]) + created_id = int(os.environ["CREATED_COMMENT_ID"]) acting_login = os.environ["ACTING_LOGIN"] - # Attribution to THIS write: created-after-boundary AND authored by the - # acting identity AND exact body match. The author check excludes a - # concurrent DIFFERENT-identity writer that id+body alone would admit. - matches = [ - c for c in comments - if isinstance(c, dict) - and isinstance(c.get("id"), int) - and c.get("id") > boundary - and (c.get("user") or {}).get("login") == acting_login - and c.get("body") == expected_body - ] - if not matches: + match = next( + ( + c for c in comments + if isinstance(c, dict) + and c.get("id") == created_id + and (c.get("user") or {}).get("login") == acting_login + ), + None, + ) + if match is None: raise ValueError( - "no comment attributable to this write matched " - "(id > boundary, acting identity, exact body); " - "tea may have silently no-opped (#865)" + f"created comment id {created_id} is not enumerable in the issue's " + "paginated comment list under the acting identity" ) - comment_id = max(c["id"] for c in matches) +except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: + print(f"Error: Gitea comment enumeration check failed: {error}", file=sys.stderr) + raise SystemExit(1) +PY +} + +# Post a comment to a Gitea issue via the supported REST API and verify it +# durably against a PROVIDER-RETURNED created id — never trust an exit code +# (#865 defect class: tea's non-existent `tea issue comment` no-ops yet exits +# 0). The write is a direct POST that returns the created comment object, so we +# learn the exact id of THIS write; we then GET that exact id and require +# id == created id AND author == acting identity AND exact body AND that it +# belongs to this issue. Because verification is keyed to the id the create +# returned, a concurrent comment (even same identity, same body) CANNOT +# masquerade as this write, and a suppressed/no-op write yields no created id +# and fails closed — there is no fallback list scan that a concurrent record +# could satisfy. Prints the created comment id on success. +# +# Args: $1 = issue number, $2 = comment body, $3 = acting identity login. +gitea_create_comment_verified() { + local issue_number="$1" comment_body="$2" acting_login="$3" + local payload write_file readback_file write_status readback_status created_id + + payload=$(COMMENT_BODY="$comment_body" python3 -c ' +import json +import os + +print(json.dumps({"body": os.environ["COMMENT_BODY"]})) +') + write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-write.XXXXXX") + readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-getid.XXXXXX") + trap 'rm -f "$write_file" "$readback_file"' RETURN + + if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ + -X POST \ + -H "Authorization: token $GITEA_API_TOKEN" \ + -H 'Content-Type: application/json' \ + -d "$payload" \ + "$GITEA_API_BASE/issues/$issue_number/comments"); then + echo "Error: Gitea comment write transport failed" >&2 + return 1 + fi + if [[ "$write_status" != "201" ]]; then + echo "Error: Gitea comment write failed with HTTP $write_status (#865: no durable comment created)" >&2 + return 1 + fi + + created_id=$(python3 - "$write_file" <<'PY' +import json +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + comment = json.load(response) + created_id = comment.get("id") if isinstance(comment, dict) else None + if not isinstance(created_id, int) or created_id <= 0: + raise ValueError("create response carried no positive comment id") +except (OSError, json.JSONDecodeError, ValueError) as error: + print(f"Error: could not identify created Gitea comment: {error}", file=sys.stderr) + raise SystemExit(1) +print(created_id) +PY +) || return 1 + + if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/issues/comments/$created_id"); then + echo "Error: Gitea comment read-back transport failed" >&2 + return 1 + fi + if [[ "$readback_status" != "200" ]]; then + echo "Error: Gitea comment read-back failed with HTTP $readback_status" >&2 + return 1 + fi + + EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ + ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ + EXPECTED_ISSUE_NUMBER="$issue_number" \ + python3 - "$readback_file" <<'PY' || return 1 +import json +import os +import sys +from urllib.parse import urlparse + +try: + with open(sys.argv[1], encoding="utf-8") as response: + comment = json.load(response) + if not isinstance(comment, dict): + raise ValueError("response is not a comment object") + expected_id = int(os.environ["EXPECTED_COMMENT_ID"]) + expected_body = os.environ["EXPECTED_COMMENT_BODY"] + acting_login = os.environ["ACTING_LOGIN"] + expected_suffix = ( + f"/repos/{os.environ['EXPECTED_REPO_SLUG']}" + f"/issues/{os.environ['EXPECTED_ISSUE_NUMBER']}" + ) + issue_path = urlparse(comment.get("issue_url", "")).path.rstrip("/") + if comment.get("id") != expected_id: + raise ValueError("read-back id does not match the created id") + if (comment.get("user") or {}).get("login") != acting_login: + raise ValueError("created comment is not authored by the acting identity") + if comment.get("body") != expected_body: + raise ValueError("created comment body does not match") + if not issue_path.endswith(expected_suffix): + raise ValueError("created comment does not belong to this issue") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) -print(comment_id) PY + + gitea_confirm_comment_enumerable "$issue_number" "$created_id" "$acting_login" || return 1 + + echo "$created_id" + return 0 } if [[ "$PLATFORM" == "github" ]]; then gh issue comment "$ISSUE_NUMBER" --body "$COMMENT" echo "Added comment to GitHub issue #$ISSUE_NUMBER" elif [[ "$PLATFORM" == "gitea" ]]; then - # Build the invocation as an argv array (not unquoted $(get_gitea_repo_args) - # word-splitting) so the comment body — including Markdown backticks, $(...), - # and quotes — is passed verbatim and never re-split or shell-evaluated. - REPO_SLUG=$(get_repo_slug) - GITEA_LOGIN_NAME=$(get_gitea_login) || { - echo "Error: could not resolve a Gitea login for this repo; cannot comment on issue #$ISSUE_NUMBER." >&2 - exit 1 - } + # Resolve the login this comment should be attributed to: the --login + # override when given, otherwise the detected default for this repo's host. + # A --login override always wins. Otherwise name this repo host's login only + # as a best effort: the login name merely selects a per-login token, and + # gitea_resolve_api_for_login falls back to the host credential + # (get_gitea_token) when no tea login is named, so the default credential + # still resolves even when the host tea has no matching login entry. + EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" + [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login 2>/dev/null || true) - # Resolve the REST endpoint, the acting identity, and the pre-write - # boundary BEFORE the write, so the read-back can require a strictly-newer - # comment id authored by this identity. - gitea_resolve_api || exit 1 + # Bind the REST endpoint + token to the effective login, then derive the + # acting identity from that SAME credential (GET /user). The write below and + # its read-back both use this credential, so the write is verified against + # the identity that actually performed it. + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 - boundary=$(gitea_max_comment_id "$ISSUE_NUMBER") || exit 1 - TEA_ARGS=(comment "$ISSUE_NUMBER" "$COMMENT" --repo "$REPO_SLUG" --login "$GITEA_LOGIN_NAME") - # --login override goes LAST: tea honors only the final --login on its - # command line, so an override placed before the detected default above - # would be silently clobbered by it. - if [[ -n "$LOGIN_OVERRIDE" ]]; then - TEA_ARGS+=(--login "$LOGIN_OVERRIDE") - fi - tea "${TEA_ARGS[@]}" - - comment_id=$(gitea_verify_comment_posted "$ISSUE_NUMBER" "$COMMENT" "$boundary" "$ACTING_LOGIN") || { - echo "Error: could not verify comment landed on Gitea issue #$ISSUE_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 + comment_id=$(gitea_create_comment_verified "$ISSUE_NUMBER" "$COMMENT" "$ACTING_LOGIN") || { + echo "Error: could not create and verify a comment on Gitea issue #$ISSUE_NUMBER via a provider-returned created id (#865)." >&2 exit 1 } echo "Added and verified comment on Gitea issue #$ISSUE_NUMBER (comment ID $comment_id)" diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index 0ba60760..ca2315d1 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -2,16 +2,20 @@ # pr-review.sh - Review a pull request on GitHub or Gitea # Usage: pr-review.sh -n -a [-c ] [--login ] # -# --login override: approve/request-changes on Gitea invoke `tea pr -# approve`/`tea pr reject` with a `--login` resolved from the local tea -# login list for this repo's host (get_gitea_login_for_host). Pass -# --login to override that default for this invocation only. The -# override is appended to the tea command line AFTER the detected default -# (get_gitea_repo_args()-equivalent resolution happens first), because tea -# honors only the LAST `--login` flag on its command line — a flag placed -# before the default would be silently clobbered by it. The `comment` -# action does not shell out to `tea` at all (see gitea_post_verified_comment -# below), so --login has no effect on it. +# Gitea reviews and comments are written through the supported REST API, not +# `tea`: tea 0.11.1 cannot emit the id of a record it creates and can silently +# no-op while exiting 0 (#865 defect class), so an exit code is the only — and +# untrustworthy — signal it offers. approve/request-changes POST to +# /pulls/{n}/reviews (returns the created review with its id); the `comment` +# action POSTs to /issues/{n}/comments (returns the created comment with its +# id). Each write is then verified by GETting that exact returned id, so a +# concurrent record cannot masquerade as this write and a no-op fails closed. +# +# --login override: the default login is resolved from the local tea login list +# for this repo's host (get_gitea_login_for_host). Pass --login to +# override it for this invocation only. The REST write, the /user identity read, +# and every read-back are ALL performed with the token of the EFFECTIVE login, +# so the write and its verification bind to the same identity. set -e @@ -73,51 +77,37 @@ fi detect_platform >/dev/null -# Post a review comment body to a Gitea PR via the supported comments REST API -# and verify it durably via provider read-back (see docs on durable review -# provenance in README.md). Used by the `comment` action and, since `tea` -# v0.11.1 defines no `--comment`/`-comment` flag on `pr approve`/`pr reject`, -# also by the `approve` and `request-changes` actions to carry an optional -# review body that `tea` itself cannot attach. +# Post a comment to a Gitea PR (PR comments ARE issue comments) via the +# supported REST API and verify it against a PROVIDER-RETURNED created id. The +# write is a direct POST that returns the created comment object, so we learn +# the exact id of THIS write; we GET that exact id and require id == created id +# AND author == acting identity AND exact body AND that it belongs to this PR. +# Keying to the returned id means no concurrent comment (even same identity / +# body) can masquerade as this write, and a no-op create yields no id and fails +# closed. Requires GITEA_API_BASE / GITEA_API_TOKEN to be resolved first (via +# gitea_resolve_api_for_login). Prints the created comment id on success. # -# Args: $1 = PR number, $2 = comment body -# On success: prints only the created comment ID to stdout, returns 0. -# On failure: prints an error to stderr, returns 1. -gitea_post_verified_comment() { - local pr_number="$1" comment_body="$2" - local host token configured_url repo api_base payload - local write_response_file readback_response_file comment_id +# Args: $1 = PR number, $2 = comment body, $3 = acting identity login. +gitea_create_comment_verified() { + local pr_number="$1" comment_body="$2" acting_login="$3" + local payload write_file readback_file write_status readback_status created_id - host=$(get_remote_host) - token=$(get_gitea_token "$host") || { - echo "Error: Gitea token not found for comment persistence" >&2 - return 1 - } - configured_url=$(get_gitea_url_for_host "$host") || { - echo "Error: Configured Gitea URL not found for comment persistence" >&2 - return 1 - } - repo=$(get_gitea_repo_slug_for_url "$configured_url") || { - echo "Error: Could not resolve Gitea owner/repository relative to configured URL" >&2 - return 1 - } - api_base="${configured_url%/}/api/v1/repos/$repo" payload=$(COMMENT_BODY="$comment_body" python3 -c ' import json import os print(json.dumps({"body": os.environ["COMMENT_BODY"]})) ') - write_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-write.XXXXXX") - readback_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-readback.XXXXXX") - trap 'rm -f "$write_response_file" "$readback_response_file"' RETURN + write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-write.XXXXXX") + readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-getid.XXXXXX") + trap 'rm -f "$write_file" "$readback_file"' RETURN - if ! write_status=$(curl -sS -o "$write_response_file" -w '%{http_code}' \ + if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ -X POST \ - -H "Authorization: token $token" \ + -H "Authorization: token $GITEA_API_TOKEN" \ -H 'Content-Type: application/json' \ -d "$payload" \ - "$api_base/issues/$pr_number/comments"); then + "$GITEA_API_BASE/issues/$pr_number/comments"); then echo "Error: Gitea comment write transport failed" >&2 return 1 fi @@ -126,26 +116,26 @@ print(json.dumps({"body": os.environ["COMMENT_BODY"]})) return 1 fi - comment_id=$(python3 - "$write_response_file" <<'PY' + created_id=$(python3 - "$write_file" <<'PY' import json import sys try: with open(sys.argv[1], encoding="utf-8") as response: comment = json.load(response) - comment_id = comment.get("id") if isinstance(comment, dict) else None - if not isinstance(comment_id, int) or comment_id <= 0: - raise ValueError("missing positive comment id") + created_id = comment.get("id") if isinstance(comment, dict) else None + if not isinstance(created_id, int) or created_id <= 0: + raise ValueError("create response carried no positive comment id") except (OSError, json.JSONDecodeError, ValueError) as error: print(f"Error: could not identify created Gitea comment: {error}", file=sys.stderr) raise SystemExit(1) -print(comment_id) +print(created_id) PY ) || return 1 - if ! readback_status=$(curl -sS -o "$readback_response_file" -w '%{http_code}' \ - -H "Authorization: token $token" \ - "$api_base/issues/comments/$comment_id"); then + if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/issues/comments/$created_id"); then echo "Error: Gitea comment read-back transport failed" >&2 return 1 fi @@ -154,8 +144,10 @@ PY return 1 fi - if EXPECTED_COMMENT_ID="$comment_id" EXPECTED_COMMENT_BODY="$comment_body" EXPECTED_REPO="$repo" EXPECTED_PR_NUMBER="$pr_number" \ - python3 - "$readback_response_file" <<'PY' + EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ + ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ + EXPECTED_PR_NUMBER="$pr_number" \ + python3 - "$readback_file" <<'PY' || return 1 import json import os import sys @@ -168,40 +160,48 @@ try: raise ValueError("response is not a comment object") expected_id = int(os.environ["EXPECTED_COMMENT_ID"]) expected_body = os.environ["EXPECTED_COMMENT_BODY"] - expected_repo = os.environ["EXPECTED_REPO"] - expected_pr = os.environ["EXPECTED_PR_NUMBER"] + acting_login = os.environ["ACTING_LOGIN"] + expected_suffix = ( + f"/repos/{os.environ['EXPECTED_REPO_SLUG']}" + f"/issues/{os.environ['EXPECTED_PR_NUMBER']}" + ) issue_path = urlparse(comment.get("issue_url", "")).path.rstrip("/") - expected_suffix = f"/repos/{expected_repo}/issues/{expected_pr}" if comment.get("id") != expected_id: - raise ValueError("comment id mismatch") + raise ValueError("read-back id does not match the created id") + if (comment.get("user") or {}).get("login") != acting_login: + raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: - raise ValueError("comment body mismatch") + raise ValueError("created comment body does not match") if not issue_path.endswith(expected_suffix): - raise ValueError("repository or PR mismatch") + raise ValueError("created comment does not belong to this PR") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) PY - then - true - else - return 1 - fi - echo "$comment_id" + echo "$created_id" return 0 } -# Resolve and cache the Gitea REST endpoint + token for the current remote. -# Populates GITEA_API_ROOT (…/api/v1), GITEA_API_BASE (…/api/v1/repos/), -# and GITEA_API_TOKEN. Returns non-zero (with a clear stderr message) on any -# resolution failure. -gitea_resolve_api() { - local host configured_url repo +# Resolve and cache the Gitea REST endpoint + token for the current remote, +# bound to a SPECIFIC login identity ($1). Populates GITEA_API_ROOT (…/api/v1), +# GITEA_API_BASE (…/api/v1/repos/), and GITEA_API_TOKEN. +# +# The token is resolved for the EFFECTIVE login (the --login override when +# given, otherwise the detected default), so the one credential used to submit +# the review/comment ALSO drives the /user identity read and every read-back — +# write token and read-back token are the same identity by construction. This +# is the credential-ordering fix: a --login override is no longer submitted +# under one credential and verified under a different default one. Falls back to +# the host-scoped credential only when the login has no token in tea's config. +# Returns non-zero (clear stderr) on any resolution failure. +gitea_resolve_api_for_login() { + local effective_login="$1" host configured_url repo host=$(get_remote_host) - GITEA_API_TOKEN=$(get_gitea_token "$host") || { - echo "Error: Gitea token not found for review read-back verification" >&2 + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + || GITEA_API_TOKEN=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for login '$effective_login' (review write/read-back)" >&2 return 1 } configured_url=$(get_gitea_url_for_host "$host") || { @@ -311,65 +311,17 @@ print(login) PY } -# Print the maximum existing review id on a PR (0 if none). This is the -# pre-write BOUNDARY: Gitea pull-review ids are monotonic, so any review -# submitted by a subsequent `tea pr approve`/`reject` has an id strictly -# greater than this value. Paginates fully so a boundary review beyond page 1 -# is still counted. -gitea_max_review_id() { - local pr_number="$1" merged_file +# Resolve the PR's current head commit SHA (GET /pulls/{n}). The review is +# submitted against — and later verified as pinned to — this exact commit, so a +# stale review left over from an earlier push cannot be mistaken for this one. +# Prints the head SHA on success. +gitea_pr_head_sha() { + local pr_number="$1" pr_file status - merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-boundary.XXXXXX") - trap 'rm -f "$merged_file"' RETURN + pr_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") + trap 'rm -f "$pr_file"' RETURN - gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$merged_file" || return 1 - - python3 - "$merged_file" <<'PY' -import json -import sys - -try: - with open(sys.argv[1], encoding="utf-8") as response: - reviews = json.load(response) - ids = [r.get("id") for r in reviews if isinstance(r, dict) and isinstance(r.get("id"), int)] - print(max(ids) if ids else 0) -except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: - print(f"Error: could not compute Gitea review boundary: {error}", file=sys.stderr) - raise SystemExit(1) -PY -} - -# Independently verify that `tea pr approve`/`reject` produced a durable review -# record — never trust tea's exit code alone (#865, same defect class). Require -# a review attributable to THIS action: its id must be strictly greater than -# the pre-write boundary, its author login must equal the acting identity, its -# state must equal the expected state (APPROVED / REQUEST_CHANGES), and it must -# have been submitted against the PR's current head commit. The reviews list is -# paginated fully so a matching review beyond page 1 is still found. Prints the -# matched review id on success; fails closed (non-zero, clear stderr) if no -# such review is found. -# -# id-above-boundary alone is only temporal ordering; the author-login check is -# what excludes a concurrent review submitted by a DIFFERENT identity. -# -# Residual (documented, not eliminable without a tea-emitted created-record id, -# which tea 0.11.1 does not reliably provide for approve/reject): a concurrent -# review by the SAME identity with the same state against the same head inside -# the boundary window could still be accepted. That is strictly narrower than -# temporal-only matching. -# -# Args: $1 = PR number, $2 = expected state (APPROVED|REQUEST_CHANGES), -# $3 = pre-write boundary review id, $4 = acting reviewer login. -gitea_verify_review_submitted() { - local pr_number="$1" expected_state="$2" boundary="$3" acting_login="$4" - local pr_response_file reviews_merged_file status head_sha review_id - - pr_response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") - reviews_merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-state.XXXXXX") - trap 'rm -f "$pr_response_file" "$reviews_merged_file"' RETURN - - # Resolve the PR's current head commit so the review can be pinned to it. - if ! status=$(curl -sS -o "$pr_response_file" -w '%{http_code}' \ + if ! status=$(curl -sS -o "$pr_file" -w '%{http_code}' \ -H "Authorization: token $GITEA_API_TOKEN" \ "$GITEA_API_BASE/pulls/$pr_number"); then echo "Error: Gitea PR head read transport failed" >&2 @@ -379,7 +331,7 @@ gitea_verify_review_submitted() { echo "Error: Gitea PR head read failed with HTTP $status" >&2 return 1 fi - head_sha=$(python3 - "$pr_response_file" <<'PY' + python3 - "$pr_file" <<'PY' import json import sys @@ -394,12 +346,25 @@ except (OSError, json.JSONDecodeError, AttributeError, TypeError, ValueError) as raise SystemExit(1) print(head_sha) PY -) || return 1 +} - gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$reviews_merged_file" || return 1 +# Confirm that the review CREATED by this action ($2 = its provider id) is +# enumerable in the PR's full, paginated review listing, authored by the acting +# identity, in the expected state. Gitea paginates review lists, so a review +# created beyond page 1 must still be found; walking every page also proves the +# created id is durably indexed against THIS PR rather than merely retrievable +# by id. Returns non-zero (clear stderr) if the exact created id is absent or +# does not match author/state. +gitea_confirm_review_enumerable() { + local pr_number="$1" created_id="$2" expected_state="$3" acting_login="$4" merged_file - review_id=$(EXPECTED_STATE="$expected_state" BOUNDARY_REVIEW_ID="$boundary" EXPECTED_HEAD_SHA="$head_sha" ACTING_LOGIN="$acting_login" \ - python3 - "$reviews_merged_file" <<'PY' + merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-list.XXXXXX") + trap 'rm -f "$merged_file"' RETURN + + gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$merged_file" || return 1 + + CREATED_REVIEW_ID="$created_id" EXPECTED_STATE="$expected_state" ACTING_LOGIN="$acting_login" \ + python3 - "$merged_file" <<'PY' import json import os import sys @@ -409,38 +374,138 @@ try: reviews = json.load(response) if not isinstance(reviews, list): raise ValueError("response is not a review list") + created_id = int(os.environ["CREATED_REVIEW_ID"]) expected_state = os.environ["EXPECTED_STATE"] - boundary = int(os.environ["BOUNDARY_REVIEW_ID"]) - expected_head = os.environ["EXPECTED_HEAD_SHA"] acting_login = os.environ["ACTING_LOGIN"] - # Attribution to THIS action: created-after-boundary AND submitted by the - # acting reviewer identity AND expected state AND pinned to the PR's - # current head commit. The author check excludes a concurrent - # DIFFERENT-identity review that id+state+head alone would admit. - matches = [ - r for r in reviews - if isinstance(r, dict) - and isinstance(r.get("id"), int) - and r.get("id") > boundary - and (r.get("user") or {}).get("login") == acting_login - and r.get("state") == expected_state - and r.get("commit_id") == expected_head - ] - if not matches: + match = next( + ( + r for r in reviews + if isinstance(r, dict) + and r.get("id") == created_id + and (r.get("user") or {}).get("login") == acting_login + and r.get("state") == expected_state + ), + None, + ) + if match is None: raise ValueError( - f"no {expected_state} review attributable to this action found " - "(id > boundary, acting identity, expected state, current head); " - "tea may have silently failed (#865 defect class)" + f"created review id {created_id} is not enumerable in the PR's " + "paginated review list under the acting identity/state" ) - review_id = max(r["id"] for r in matches) except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: - print(f"Error: Gitea review persistence verification failed: {error}", file=sys.stderr) + print(f"Error: Gitea review enumeration check failed: {error}", file=sys.stderr) raise SystemExit(1) -print(review_id) +PY +} + +# Submit a review to a Gitea PR via the supported REST API and verify it against +# a PROVIDER-RETURNED created id. tea 0.11.1's `pr approve`/`reject` cannot emit +# the id of the review it created and can silently no-op while exiting 0 (#865 +# defect class), so this does NOT shell out to tea: it POSTs to +# /pulls/{n}/reviews with the event (APPROVED / REQUEST_CHANGES), the PR head +# commit_id, and the review body, which returns the created review object +# including its id. It then GETs that exact review id and requires +# id == created id AND author == acting identity AND state == expected AND +# commit_id == PR head. Keying to the returned id means no concurrent review +# (even same identity/state/head) can masquerade as this one, and a no-op +# submit yields no id and fails closed. Prints the created review id on success. +# +# Args: $1 = PR number, $2 = event (APPROVED|REQUEST_CHANGES), +# $3 = review body (may be empty for APPROVED), $4 = acting login, +# $5 = PR head sha. +gitea_submit_review_verified() { + local pr_number="$1" event="$2" review_body="$3" acting_login="$4" head_sha="$5" + local payload write_file readback_file write_status readback_status created_id + + payload=$(REVIEW_EVENT="$event" REVIEW_BODY="$review_body" REVIEW_COMMIT="$head_sha" python3 -c ' +import json +import os + +print(json.dumps({ + "event": os.environ["REVIEW_EVENT"], + "body": os.environ["REVIEW_BODY"], + "commit_id": os.environ["REVIEW_COMMIT"], +})) +') + write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-submit.XXXXXX") + readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-getid.XXXXXX") + trap 'rm -f "$write_file" "$readback_file"' RETURN + + if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ + -X POST \ + -H "Authorization: token $GITEA_API_TOKEN" \ + -H 'Content-Type: application/json' \ + -d "$payload" \ + "$GITEA_API_BASE/pulls/$pr_number/reviews"); then + echo "Error: Gitea review submit transport failed" >&2 + return 1 + fi + # Gitea returns 200 (occasionally 201) with the created review object. + if [[ "$write_status" != "200" && "$write_status" != "201" ]]; then + echo "Error: Gitea review submit failed with HTTP $write_status (#865: no durable review created)" >&2 + return 1 + fi + + created_id=$(python3 - "$write_file" <<'PY' +import json +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + review = json.load(response) + created_id = review.get("id") if isinstance(review, dict) else None + if not isinstance(created_id, int) or created_id <= 0: + raise ValueError("submit response carried no positive review id") +except (OSError, json.JSONDecodeError, ValueError) as error: + print(f"Error: could not identify created Gitea review: {error}", file=sys.stderr) + raise SystemExit(1) +print(created_id) PY ) || return 1 - echo "$review_id" + if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ + -H "Authorization: token $GITEA_API_TOKEN" \ + "$GITEA_API_BASE/pulls/$pr_number/reviews/$created_id"); then + echo "Error: Gitea review read-back transport failed" >&2 + return 1 + fi + if [[ "$readback_status" != "200" ]]; then + echo "Error: Gitea review read-back failed with HTTP $readback_status" >&2 + return 1 + fi + + EXPECTED_REVIEW_ID="$created_id" EXPECTED_STATE="$event" ACTING_LOGIN="$acting_login" \ + EXPECTED_HEAD_SHA="$head_sha" \ + python3 - "$readback_file" <<'PY' || return 1 +import json +import os +import sys + +try: + with open(sys.argv[1], encoding="utf-8") as response: + review = json.load(response) + if not isinstance(review, dict): + raise ValueError("response is not a review object") + expected_id = int(os.environ["EXPECTED_REVIEW_ID"]) + expected_state = os.environ["EXPECTED_STATE"] + acting_login = os.environ["ACTING_LOGIN"] + expected_head = os.environ["EXPECTED_HEAD_SHA"] + if review.get("id") != expected_id: + raise ValueError("read-back id does not match the created id") + if (review.get("user") or {}).get("login") != acting_login: + raise ValueError("created review is not authored by the acting identity") + if review.get("state") != expected_state: + raise ValueError("created review is not in the expected state") + if review.get("commit_id") != expected_head: + raise ValueError("created review is not pinned to the PR head commit") +except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: + print(f"Error: Gitea review persistence verification failed: {error}", file=sys.stderr) + raise SystemExit(1) +PY + + gitea_confirm_review_enumerable "$pr_number" "$created_id" "$event" "$acting_login" || return 1 + + echo "$created_id" return 0 } @@ -474,74 +539,76 @@ if [[ "$PLATFORM" == "github" ]]; then elif [[ "$PLATFORM" == "gitea" ]]; then case $ACTION in approve) - repo=$(get_repo_slug) host=$(get_remote_host) - login=$(get_gitea_login_for_host "$host") - # Resolve the REST endpoint and record the pre-write review-id - # boundary BEFORE the write, so the read-back can require a - # strictly-newer review created by THIS action (never trust tea's - # exit code alone — #865 defect class applies to the review state). - gitea_resolve_api || exit 1 + # A --login override always wins. Otherwise name this host's login + # only as a best effort: the login name merely selects a per-login + # token, and gitea_resolve_api_for_login falls back to the host + # credential (get_gitea_token) when no tea login is named — so a host + # tea's login list need not enumerate exotic (e.g. ported) hosts for + # the default credential to resolve. The single resolved token is + # then used for the write, the /user identity, and the read-back. + EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" + [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login_for_host "$host" 2>/dev/null || true) + # Bind the REST endpoint + token to the effective login, then derive + # the acting identity from that SAME credential so the review submit + # and its read-back verify against the identity that performed them. + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 - review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 - # tea v0.11.1 defines no --comment/-comment flag on `pr approve`; - # route any review body via the durable comment API instead (#835). - TEA_ARGS=(pr approve "$PR_NUMBER" --repo "$repo" --login "$login") - # --login override goes LAST: tea honors only the final --login on - # its command line, so an override placed before the detected - # default above would be silently clobbered by it. - if [[ -n "$LOGIN_OVERRIDE" ]]; then - TEA_ARGS+=(--login "$LOGIN_OVERRIDE") - fi - tea "${TEA_ARGS[@]}" - review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "APPROVED" "$review_boundary" "$ACTING_LOGIN") || { - echo "Error: could not verify an APPROVED review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 + head_sha=$(gitea_pr_head_sha "$PR_NUMBER") || exit 1 + # The review body (if any) travels with the review itself in the REST + # submit — the created review record carries it — so there is no + # separate detached comment to reconcile. + review_id=$(gitea_submit_review_verified "$PR_NUMBER" "APPROVED" "$COMMENT" "$ACTING_LOGIN" "$head_sha") || { + echo "Error: could not submit and verify an APPROVED review on Gitea PR #$PR_NUMBER via a provider-returned created id (#865)." >&2 exit 1 } echo "Approved and verified Gitea PR #$PR_NUMBER (review ID $review_id)" - if [[ -n "$COMMENT" ]]; then - comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 - echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" - fi ;; request-changes) if [[ -z "$COMMENT" ]]; then echo "Error: Comment required for request-changes" exit 1 fi - repo=$(get_repo_slug) host=$(get_remote_host) - login=$(get_gitea_login_for_host "$host") - # Record the pre-write review-id boundary BEFORE the write (see the - # approve path above for the rationale). - gitea_resolve_api || exit 1 + # A --login override always wins. Otherwise name this host's login + # only as a best effort: the login name merely selects a per-login + # token, and gitea_resolve_api_for_login falls back to the host + # credential (get_gitea_token) when no tea login is named — so a host + # tea's login list need not enumerate exotic (e.g. ported) hosts for + # the default credential to resolve. The single resolved token is + # then used for the write, the /user identity, and the read-back. + EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" + [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login_for_host "$host" 2>/dev/null || true) + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 - review_boundary=$(gitea_max_review_id "$PR_NUMBER") || exit 1 - # tea v0.11.1 defines no --comment/-comment flag on `pr reject`; - # route the review body via the durable comment API instead (#835). - TEA_ARGS=(pr reject "$PR_NUMBER" --repo "$repo" --login "$login") - # --login override goes LAST: tea honors only the final --login on - # its command line, so an override placed before the detected - # default above would be silently clobbered by it. - if [[ -n "$LOGIN_OVERRIDE" ]]; then - TEA_ARGS+=(--login "$LOGIN_OVERRIDE") - fi - tea "${TEA_ARGS[@]}" - review_id=$(gitea_verify_review_submitted "$PR_NUMBER" "REQUEST_CHANGES" "$review_boundary" "$ACTING_LOGIN") || { - echo "Error: could not verify a REQUEST_CHANGES review landed on Gitea PR #$PR_NUMBER via bounded read-back; treating tea's exit code as untrustworthy (#865)." >&2 + head_sha=$(gitea_pr_head_sha "$PR_NUMBER") || exit 1 + review_id=$(gitea_submit_review_verified "$PR_NUMBER" "REQUEST_CHANGES" "$COMMENT" "$ACTING_LOGIN" "$head_sha") || { + echo "Error: could not submit and verify a REQUEST_CHANGES review on Gitea PR #$PR_NUMBER via a provider-returned created id (#865)." >&2 exit 1 } echo "Requested changes and verified on Gitea PR #$PR_NUMBER (review ID $review_id)" - comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 - echo "Added and verified review comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" ;; comment) if [[ -z "$COMMENT" ]]; then echo "Error: Comment required" exit 1 fi - - comment_id=$(gitea_post_verified_comment "$PR_NUMBER" "$COMMENT") || exit 1 + host=$(get_remote_host) + # A --login override always wins. Otherwise name this host's login + # only as a best effort: the login name merely selects a per-login + # token, and gitea_resolve_api_for_login falls back to the host + # credential (get_gitea_token) when no tea login is named — so a host + # tea's login list need not enumerate exotic (e.g. ported) hosts for + # the default credential to resolve. The single resolved token is + # then used for the write, the /user identity, and the read-back. + EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" + [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login_for_host "$host" 2>/dev/null || true) + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 + ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 + comment_id=$(gitea_create_comment_verified "$PR_NUMBER" "$COMMENT" "$ACTING_LOGIN") || { + echo "Error: could not create and verify a comment on Gitea PR #$PR_NUMBER via a provider-returned created id (#865)." >&2 + exit 1 + } echo "Added and verified comment on Gitea PR #$PR_NUMBER (comment ID $comment_id)" ;; *) diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index b036b8e4..d32662ea 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -1,28 +1,33 @@ #!/usr/bin/env bash -# Regression harness for issue-comment.sh top-level `tea comment` invocation and -# its BOUNDED, INVOCATION-ATTRIBUTED, PAGINATED read-back verification (#865). +# Regression harness for issue-comment.sh's Gitea comment write + verification +# (#865). # -# The #865 bug: `tea issue comment ...` (a nonexistent subcommand on tea -# v0.11.1) silently no-ops and exits 0, so a comment is never posted. A naive -# read-back that matches ANY historical comment by body would falsely report -# success whenever an identically-bodied comment already exists from a prior -# run. Merely bounding by "id > pre-write max" is also insufficient: it accepts -# ANY newer matching comment, including one a CONCURRENT DIFFERENT identity -# posted while this tea invocation created nothing. This harness proves the -# wrapper: -# 1. uses the top-level `tea comment` form (never `tea issue comment`); -# 2. records the pre-write maximum comment id as a boundary and requires a -# strictly-newer comment on read-back, so a pre-existing identical body -# does NOT satisfy verification (fails closed); -# 3. attributes the matched comment to the acting identity (GET /user login), -# so a concurrent DIFFERENT-identity write does NOT satisfy verification; -# 4. reports success only when a genuinely new comment (id > boundary) with -# the exact body AND the acting author appears. +# The #865 defect class: tea 0.11.1's `tea issue comment ...` (a nonexistent +# subcommand) silently no-ops yet exits 0, and tea cannot emit the id of a +# record it created — so an exit code is worthless as proof of a durable write. +# The wrapper therefore does NOT write via tea at all. It POSTs the comment to +# the Gitea REST API (which returns the created comment object, including its +# id), then GETs THAT EXACT id back and requires it to match on id, author +# (acting identity), body, and issue. Because verification is keyed to the id +# the create returned, no concurrent comment can masquerade as this write, and a +# suppressed/no-op create yields no id and fails closed. # -# The `tea` stub NEVER creates a comment (it mimics the silent no-op); the -# "server" comment state is modeled entirely by the curl stub's responses, so -# the fresh-success vs. no-op distinction is driven purely by whether the -# post-write read-back surfaces a new, correctly-attributed id. +# This harness models a REAL server: the curl stub keeps persistent comment +# state on disk, the POST actually CREATES and PERSISTS a record and returns its +# id, and the read-back GET reads that same state. There is no independently +# fabricated record for the wrapper to "find" — the only way verification +# passes is if the POST genuinely created the record the read-back retrieves. +# It proves the wrapper: +# 1. never shells out to tea to write (no `tea comment` / `tea issue comment`); +# 2. creates the comment via REST POST and learns the provider-returned id; +# 3. verifies THAT EXACT id by direct GET, attributed to the acting identity; +# 4. fails closed when the write is a no-op even though a concurrent +# SAME-IDENTITY comment with the same body already exists (the closed +# concurrency window — no fallback list scan can rescue a no-op); +# 5. fails closed when the created record is not authored by the acting +# identity; +# 6. enumerates the created id in the issue's FULLY PAGINATED comment list, +# finding it even when it lands beyond page 1. set -euo pipefail @@ -30,22 +35,24 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/issue-comment-readback}" REPO_DIR="$WORK_DIR/repo" BIN_DIR="$WORK_DIR/bin" +XDG_DIR="$WORK_DIR/xdg" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" -CALLS_FILE="$WORK_DIR/comment_calls" +STATE_FILE="$WORK_DIR/comments.json" cleanup() { rm -rf "$WORK_DIR" } trap cleanup EXIT -mkdir -p "$REPO_DIR" "$BIN_DIR" +mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git ISSUE_NUMBER=7 +REPO_SLUG="mosaicstack/stack" API_BASE="https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack" API_ROOT="https://git.mosaicstack.dev/api/v1" BODY='durable "note" -- marker' @@ -68,8 +75,8 @@ with open(sys.argv[1], "w", encoding="utf-8") as credentials: }, credentials) PY -# tea stub: resolves the login list, and treats `tea comment ...` as a silent -# no-op (exit 0 without creating anything) to mimic the real failure mode. +# tea stub: only ever answers the login list (used to resolve the default login +# name). It must NEVER be asked to write a comment — the wrapper writes via REST. cat > "$BIN_DIR/tea" <<'SH' #!/usr/bin/env bash set -euo pipefail @@ -81,29 +88,16 @@ if [[ "$*" == "login list --output json" ]]; then exit 0 fi -# The wrapper must use the TOP-LEVEL `tea comment` form; the broken -# `tea issue comment` subcommand must never be invoked. -if [[ "$*" == issue\ comment* ]]; then - echo "wrapper invoked nonexistent 'tea issue comment' subcommand" >&2 - exit 90 -fi - -if [[ "$*" == comment\ * ]]; then - # Mimic tea v0.11.1: exit 0. Whether a comment actually lands is modeled - # by the curl stub's post-write read-back response, not here. - exit 0 -fi - -echo "Unexpected tea command: $*" >&2 +echo "Unexpected tea command (wrapper must not write via tea): $*" >&2 exit 92 SH chmod +x "$BIN_DIR/tea" -# curl stub: serves GET /user (acting identity) and GET .../issues/7/comments -# (paginated: ?limit=&page=). The comments endpoint's first call = pre-write -# boundary, second call = post-write read-back. The boundary always contains a -# pre-existing comment (id 50) whose body is IDENTICAL to the one under test, -# which is exactly the condition a body-only match would trip over. +# curl stub: a small REST server backed by persistent on-disk comment state. +# GET /user -> acting identity +# POST /issues/7/comments -> CREATE + PERSIST, return created object +# GET /issues/comments/{id} -> read the persisted record by exact id +# GET /issues/7/comments?page=&.. -> paginated listing of persisted state cat > "$BIN_DIR/curl" <<'SH' #!/usr/bin/env bash set -euo pipefail @@ -111,22 +105,22 @@ set -euo pipefail output_file="" method="GET" url="" +data="" while [[ $# -gt 0 ]]; do case "$1" in -o) output_file="$2"; shift 2 ;; -w|-H) shift 2 ;; -X) method="$2"; shift 2 ;; - -d|--data) shift 2 ;; + -d|--data) data="$2"; shift 2 ;; -s|-S|-sS) shift ;; http://*|https://*) url="$1"; shift ;; *) shift ;; esac done -# Strip any query string so pagination params don't defeat path matching, but -# still log the full URL (including ?limit=&page=) so the test can assert the -# read-back paginated. path="${url%%\?*}" +query="${url#*\?}" +[[ "$query" == "$url" ]] && query="" printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG" write_response() { @@ -143,55 +137,82 @@ import os print(json.dumps({"login": os.environ["ISSUE_COMMENT_LOGIN"]})) PY )" -elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then - calls_file="$ISSUE_COMMENT_CALLS" - if [[ -f "$calls_file" ]]; then - # post-write read-back - response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \ - ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" \ - ISSUE_COMMENT_FOREIGN_LOGIN="$ISSUE_COMMENT_FOREIGN_LOGIN" \ - ISSUE_COMMENT_TEST_MODE="$ISSUE_COMMENT_TEST_MODE" python3 - <<'PY' +elif [[ "$method" == "POST" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then + result=$(ISSUE_COMMENT_DATA="$data" python3 - <<'PY' import json import os -body = os.environ["ISSUE_COMMENT_BODY"] +state_path = os.environ["ISSUE_COMMENT_STATE"] +mode = os.environ["ISSUE_COMMENT_TEST_MODE"] acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"] foreign = os.environ["ISSUE_COMMENT_FOREIGN_LOGIN"] -mode = os.environ["ISSUE_COMMENT_TEST_MODE"] +repo = os.environ["ISSUE_COMMENT_REPO_SLUG"] +body = json.loads(os.environ["ISSUE_COMMENT_DATA"]).get("body") -if mode == "fresh-success": - # A genuinely new comment (id 60 > boundary 50) authored by the acting - # identity. - records = [ - {"id": 50, "body": body, "user": {"login": acting}}, - {"id": 60, "body": body, "user": {"login": acting}}, - ] -elif mode == "foreign-identity": - # A concurrent new comment (id 60 > boundary 50) with the SAME body but a - # DIFFERENT author. tea created nothing; attribution must reject this. - records = [ - {"id": 50, "body": body, "user": {"login": acting}}, - {"id": 60, "body": body, "user": {"login": foreign}}, - ] -else: - # no-op: nothing new landed; the pre-existing id-50 comment remains. - records = [{"id": 50, "body": body, "user": {"login": acting}}] -print(json.dumps(records)) +with open(state_path, encoding="utf-8") as handle: + comments = json.load(handle) + +# no-op-concurrent: the wrapper's own write is SUPPRESSED (returns 200 with no +# created object) even though a concurrent same-identity comment already exists +# in state. Nothing is persisted; there is no created id to verify. +if mode == "no-op-concurrent": + print("200") + print(json.dumps({})) + raise SystemExit(0) + +author = foreign if mode == "author-mismatch" else acting +new_id = (max((c["id"] for c in comments), default=0)) + 1 +record = { + "id": new_id, + "body": body, + "user": {"login": author}, + "issue_url": f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7", +} +comments.append(record) +with open(state_path, "w", encoding="utf-8") as handle: + json.dump(comments, handle) +print("201") +print(json.dumps(record)) PY ) - else - : > "$calls_file" - response=$(ISSUE_COMMENT_BODY="$ISSUE_COMMENT_EXPECTED_BODY" \ - ISSUE_COMMENT_ACTING_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY' + write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" +elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE"/issues/comments/* ]]; then + result=$(ISSUE_COMMENT_GET_ID="${path##*/}" python3 - <<'PY' import json import os -body = os.environ["ISSUE_COMMENT_BODY"] -acting = os.environ["ISSUE_COMMENT_ACTING_LOGIN"] -print(json.dumps([{"id": 50, "body": body, "user": {"login": acting}}])) + +state_path = os.environ["ISSUE_COMMENT_STATE"] +wanted = int(os.environ["ISSUE_COMMENT_GET_ID"]) +with open(state_path, encoding="utf-8") as handle: + comments = json.load(handle) +match = next((c for c in comments if c["id"] == wanted), None) +if match is None: + print("404") + print(json.dumps({"message": "not found"})) +else: + print("200") + print(json.dumps(match)) PY ) - fi - write_response 200 "$response" + write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" +elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then + result=$(ISSUE_COMMENT_QUERY="$query" python3 - <<'PY' +import json +import os +from urllib.parse import parse_qs + +state_path = os.environ["ISSUE_COMMENT_STATE"] +params = parse_qs(os.environ["ISSUE_COMMENT_QUERY"]) +limit = int(params.get("limit", ["50"])[0]) +page = int(params.get("page", ["1"])[0]) +with open(state_path, encoding="utf-8") as handle: + comments = json.load(handle) +start = (page - 1) * limit +print("200") +print(json.dumps(comments[start:start + limit])) +PY +) + write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" else echo "Unexpected curl request: $method $url" >&2 exit 97 @@ -199,67 +220,112 @@ fi SH chmod +x "$BIN_DIR/curl" +# Seed persistent server state for a mode, then run the wrapper against it. +seed_state() { + local mode="$1" + ISSUE_COMMENT_SEED_MODE="$mode" ISSUE_COMMENT_SEED_BODY="$BODY" \ + ISSUE_COMMENT_SEED_ACTING="$ACTING_LOGIN" ISSUE_COMMENT_SEED_REPO="$REPO_SLUG" \ + python3 - "$STATE_FILE" <<'PY' +import json +import os +import sys + +mode = os.environ["ISSUE_COMMENT_SEED_MODE"] +body = os.environ["ISSUE_COMMENT_SEED_BODY"] +acting = os.environ["ISSUE_COMMENT_SEED_ACTING"] +repo = os.environ["ISSUE_COMMENT_SEED_REPO"] +issue_url = f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7" + +if mode == "fresh-success": + # 50 pre-existing comments fill page 1 (limit 50); the comment this run + # creates becomes id 51 and lands ALONE on page 2, exercising >page-1 + # pagination in the enumeration check. + comments = [ + {"id": i, "body": f"prior {i}", "user": {"login": acting}, "issue_url": issue_url} + for i in range(1, 51) + ] +elif mode == "no-op-concurrent": + # A concurrent SAME-IDENTITY comment with the IDENTICAL body already exists. + # The wrapper's own write will be a no-op; it must still fail closed because + # no created id is returned — it must not scan and accept this record. + comments = [ + {"id": 55, "body": body, "user": {"login": acting}, "issue_url": issue_url} + ] +else: # author-mismatch + comments = [] + +with open(sys.argv[1], "w", encoding="utf-8") as handle: + json.dump(comments, handle) +PY +} + run_comment() { local mode="$1" : > "$TEA_LOG" : > "$CURL_LOG" : > "$OUTPUT_FILE" - rm -f "$CALLS_FILE" + seed_state "$mode" ( cd "$REPO_DIR" PATH="$BIN_DIR:$PATH" \ + XDG_CONFIG_HOME="$XDG_DIR" \ MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \ ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \ - ISSUE_COMMENT_CALLS="$CALLS_FILE" \ + ISSUE_COMMENT_STATE="$STATE_FILE" \ ISSUE_COMMENT_TEST_MODE="$mode" \ - ISSUE_COMMENT_EXPECTED_BODY="$BODY" \ ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \ ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ + ISSUE_COMMENT_REPO_SLUG="$REPO_SLUG" \ ISSUE_COMMENT_API_BASE="$API_BASE" \ ISSUE_COMMENT_API_ROOT="$API_ROOT" \ "$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY" ) > "$OUTPUT_FILE" 2>&1 } -# Case 1: silent no-op with a pre-existing identical body must FAIL CLOSED. -if run_comment noop-preexisting; then - echo "FAIL: wrapper reported success when tea no-opped but an identical body pre-existed" >&2 - cat "$OUTPUT_FILE" >&2 - exit 1 -fi -if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then - echo "FAIL: read-back matched a pre-existing comment by body only" >&2 - exit 1 -fi -# The wrapper must have used the top-level form and read comments back twice -# (boundary + post-write). -grep -q "^comment 7 " "$TEA_LOG" -if grep -q '^issue comment' "$TEA_LOG"; then - echo "FAIL: wrapper used the broken 'tea issue comment' subcommand" >&2 - exit 1 -fi -[[ "$(grep -c "^GET $API_BASE/issues/7/comments?" "$CURL_LOG")" == "2" ]] -# Read-back must be paginated (limit + page query params present). -grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=1$" "$CURL_LOG" -# Attribution must have resolved the acting identity via GET /user. -grep -q "^GET $API_ROOT/user$" "$CURL_LOG" - -# Case 2: a concurrent DIFFERENT-identity write (id 60 > boundary, same body, -# foreign author) must FAIL CLOSED — temporal ordering is not attribution. -if run_comment foreign-identity; then - echo "FAIL: wrapper accepted a concurrent comment authored by a different identity" >&2 - cat "$OUTPUT_FILE" >&2 - exit 1 -fi -if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then - echo "FAIL: read-back matched a different-identity comment (attribution bypassed)" >&2 - exit 1 -fi - -# Case 3: a genuinely new comment (id 60 > boundary 50, acting author) verifies. +# Case 1: a genuine REST create (id 51) is verified end to end via its exact +# provider-returned id and enumerated on page 2 of the paginated listing. run_comment fresh-success -grep -q 'Added and verified comment on Gitea issue #7 (comment ID 60)' "$OUTPUT_FILE" -grep -q "^comment 7 " "$TEA_LOG" +grep -q 'Added and verified comment on Gitea issue #7 (comment ID 51)' "$OUTPUT_FILE" +# The write is a REST POST, never a tea comment. +grep -q "^POST $API_BASE/issues/7/comments$" "$CURL_LOG" +if grep -Eq '^comment |^issue comment ' "$TEA_LOG"; then + echo "FAIL: wrapper wrote a comment via tea instead of REST" >&2 + exit 1 +fi +# Read-back is a DIRECT GET of the exact created id. +grep -q "^GET $API_BASE/issues/comments/51$" "$CURL_LOG" +# Acting identity resolved via GET /user. +grep -q "^GET $API_ROOT/user$" "$CURL_LOG" +# Enumeration paginated beyond page 1 to find the created comment. +grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=2$" "$CURL_LOG" -echo "issue-comment.sh bounded + attributed read-back regression passed" +# Case 2: a no-op write with a concurrent SAME-IDENTITY, same-body comment +# already present must FAIL CLOSED — the closed concurrency window. +if run_comment no-op-concurrent; then + echo "FAIL: wrapper reported success when its write no-opped but a concurrent same-identity comment existed" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: wrapper accepted a concurrent record for a no-op write (window not closed)" >&2 + exit 1 +fi +# It must NOT have fallen back to a list scan that could find the concurrent id. +if grep -q "^GET $API_BASE/issues/comments/55$" "$CURL_LOG"; then + echo "FAIL: wrapper read back the concurrent comment id 55 (illegitimate fallback)" >&2 + exit 1 +fi + +# Case 3: a created record NOT authored by the acting identity must FAIL CLOSED. +if run_comment author-mismatch; then + echo "FAIL: wrapper accepted a created comment authored by a different identity" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: read-back did not enforce acting-identity authorship" >&2 + exit 1 +fi + +echo "issue-comment.sh REST create + exact-id read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index c28310c3..25f36cb3 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -1,11 +1,23 @@ #!/usr/bin/env bash -# Regression harness for durable Gitea PR review comments (#812) and for the -# approve/reject `--comment` flag removal (#835). The `tea` stub below rejects -# any `-comment`/`--comment` flag on `pr approve`/`pr reject` exactly like real -# `tea` v0.11.1 does ("flag provided but not defined: -comment"), so this -# harness fails RED against the pre-#835 wrapper (which passed that flag) and -# only passes once the wrapper routes the review body through the durable -# comment REST API instead. +# Regression harness for pr-review.sh's Gitea review + comment writes (#865, +# #812, #835). +# +# The #865 defect class: tea 0.11.1 can silently no-op while exiting 0 and +# cannot emit the id of a record it creates, so its exit code is worthless as +# proof of a durable write. The wrapper therefore does NOT write reviews or +# comments via tea. approve/request-changes POST to /pulls/{n}/reviews (with the +# event, the PR head commit_id, and the review body) and read the created review +# back by its EXACT provider-returned id; the `comment` action POSTs to +# /issues/{n}/comments and reads that created comment back by its exact id. +# Because verification keys on the id the create returned, no concurrent record +# can masquerade as this write and a no-op create fails closed. tea is only ever +# consulted for the login list. +# +# The curl stub models a REAL server with persistent review/comment state on +# disk: a POST actually CREATES and PERSISTS a record and returns its id, and +# the read-back reads that same state. There is no independently fabricated +# record for the wrapper to "find" — verification passes only when the POST +# genuinely created the record the read-back retrieves. set -euo pipefail @@ -13,6 +25,11 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/pr-review-gitea-comment}" REPO_DIR="$WORK_DIR/repo" BIN_DIR="$WORK_DIR/bin" +XDG_DIR="$WORK_DIR/xdg" +STATE_DIR="$WORK_DIR/state" +REVIEWS_FILE="$STATE_DIR/reviews.json" +COMMENTS_FILE="$STATE_DIR/comments.json" +SUBMIT_PAYLOAD_FILE="$STATE_DIR/review_payload.json" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" OUTPUT_FILE="$WORK_DIR/output.log" @@ -25,8 +42,9 @@ trap cleanup EXIT ACTING_LOGIN="review-bot" FOREIGN_LOGIN="other-writer" +HEAD_SHA="HEADSHA_FEEDFACE" -mkdir -p "$REPO_DIR" "$BIN_DIR" +mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$STATE_DIR" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git @@ -49,6 +67,9 @@ with open(sys.argv[1], "w", encoding="utf-8") as credentials: PY } +# tea stub: only ever answers the login list. The wrapper must never write a +# review or comment through tea (#865 defect class); any other tea invocation is +# an error. cat > "$BIN_DIR/tea" <<'SH' #!/usr/bin/env bash set -euo pipefail @@ -56,42 +77,17 @@ set -euo pipefail printf '%s\n' "$*" >> "$PR_REVIEW_TEA_LOG" if [[ "$*" == "login list --output json" ]]; then - printf '%s\n' '[{"name":"mosaicstack","url":"https://git.mosaicstack.dev"}]' + printf '[{"name":"mosaicstack","url":"%s"}]\n' "$PR_REVIEW_LOGIN_URL" exit 0 fi -# tea v0.11.1 defines no --comment/-comment flag on `pr approve` or `pr -# reject`; it fails closed with this exact message and a nonzero exit. Any -# regression that reintroduces the flag on those subcommands must hit this -# branch and fail RED (#835). -if [[ "$*" == *" -comment "* || "$*" == *" --comment "* || "$*" == *" -comment" || "$*" == *" --comment" ]]; then - echo "flag provided but not defined: -comment" >&2 - exit 1 -fi - -case "${PR_REVIEW_TEST_MODE:-}" in - approve|paginated-approve|foreign-review) - [[ "$*" == "pr approve 123 --repo mosaicstack/stack --login mosaicstack" ]] || exit 90 - ;; - request-changes) - [[ "$*" == "pr reject 123 --repo mosaicstack/stack --login mosaicstack" ]] || exit 91 - ;; - legacy-fallback|comment-success|http-success|prefix-success|subpath-success|port-success|scp-ssh-success|url-ssh-success|ssh-transport-port-success|explicit-default-port-success|write-transport-failure|write-http-failure|readback-failure) - if [[ "$*" == pr\ comment* ]]; then - # tea v0.11.1 treats the nonexistent subcommand as `tea pr list` and exits 0. - printf '%s\n' 'INDEX TITLE STATE' - exit 0 - fi - echo "Unexpected tea command: $*" >&2 - exit 92 - ;; - *) - exit 95 - ;; -esac +echo "Unexpected tea command (wrapper must not write via tea): $*" >&2 +exit 92 SH chmod +x "$BIN_DIR/tea" +# curl stub: a small REST server backed by persistent on-disk review/comment +# state. cat > "$BIN_DIR/curl" <<'SH' #!/usr/bin/env bash set -euo pipefail @@ -102,40 +98,19 @@ payload="" url="" while [[ $# -gt 0 ]]; do case "$1" in - -o) - output_file="$2" - shift 2 - ;; - -w|-H) - shift 2 - ;; - -X) - method="$2" - shift 2 - ;; - -d|--data) - payload="$2" - shift 2 - ;; - -s|-S|-sS) - shift - ;; - http://*|https://*) - url="$1" - shift - ;; - *) - shift - ;; + -o) output_file="$2"; shift 2 ;; + -w|-H) shift 2 ;; + -X) method="$2"; shift 2 ;; + -d|--data) payload="$2"; shift 2 ;; + -s|-S|-sS) shift ;; + http://*|https://*) url="$1"; shift ;; + *) shift ;; esac done -# Strip any query string so pagination params (?limit=&page=) don't defeat -# path matching, but keep the full URL in the log so tests can assert that the -# read-back paginated. path="${url%%\?*}" -page="${url##*page=}" -[[ "$page" == "$url" ]] && page=1 +query="${url#*\?}" +[[ "$query" == "$url" ]] && query="" printf '%s %s\n' "$method" "$url" >> "$PR_REVIEW_CURL_LOG" write_response() { @@ -145,137 +120,202 @@ write_response() { printf '%s' "$status" } -case "${PR_REVIEW_TEST_MODE:-}" in - legacy-fallback|write-transport-failure) - echo "simulated transport failure" >&2 - exit 7 - ;; - write-http-failure) - write_response 500 '{"message":"simulated rejection"}' - ;; - approve|request-changes|paginated-approve|foreign-review|comment-success|http-success|prefix-success|subpath-success|port-success|scp-ssh-success|url-ssh-success|ssh-transport-port-success|explicit-default-port-success|readback-failure) - if [[ "$method" == "GET" && "$path" == "$PR_REVIEW_API_ROOT/user" ]]; then - # Acting reviewer identity used for invocation attribution. - write_response 200 "$(PR_REVIEW_LOGIN="$PR_REVIEW_ACTING_LOGIN" python3 - <<'PY' +emit() { + # Split a two-line "status\n" python result into the response. + local result="$1" + write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" +} + +mode="${PR_REVIEW_TEST_MODE:-}" + +if [[ "$method" == "GET" && "$path" == "$PR_REVIEW_API_ROOT/user" ]]; then + write_response 200 "$(PR_REVIEW_LOGIN="$PR_REVIEW_ACTING_LOGIN" python3 - <<'PY' import json import os print(json.dumps({"login": os.environ["PR_REVIEW_LOGIN"]})) PY )" - elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then - # First (boundary) call precedes the write; later calls are the - # post-write read-back that must surface a strictly-newer review - # created by THIS action (id 200 > boundary 100), authored by the - # acting identity, with the expected state and pinned to the PR's - # current head commit. The list is served PAGINATED so a target - # beyond page 1 is only found by a fully-paginating read-back. - calls_file="${PR_REVIEW_REVIEW_CALLS:-/dev/null}" - if [[ -f "$calls_file" ]]; then - phase="post" - else - : > "$calls_file" - phase="boundary" - fi - state="APPROVED" - [[ "$PR_REVIEW_TEST_MODE" == "request-changes" ]] && state="REQUEST_CHANGES" - response=$(PR_REVIEW_PHASE="$phase" PR_REVIEW_PAGE="$page" \ - PR_REVIEW_MODE="$PR_REVIEW_TEST_MODE" PR_REVIEW_STATE="$state" \ - PR_REVIEW_ACTING_LOGIN="$PR_REVIEW_ACTING_LOGIN" \ - PR_REVIEW_FOREIGN_LOGIN="$PR_REVIEW_FOREIGN_LOGIN" python3 - <<'PY' +elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123" ]]; then + write_response 200 "$(PR_REVIEW_HEAD_SHA="$PR_REVIEW_HEAD_SHA" python3 - <<'PY' +import json +import os +print(json.dumps({"head": {"sha": os.environ["PR_REVIEW_HEAD_SHA"]}})) +PY +)" +elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then + printf '%s' "$payload" > "$PR_REVIEW_SUBMIT_PAYLOAD" + emit "$(PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' import json import os -phase = os.environ["PR_REVIEW_PHASE"] -page = int(os.environ["PR_REVIEW_PAGE"]) -mode = os.environ["PR_REVIEW_MODE"] -state = os.environ["PR_REVIEW_STATE"] +state_path = os.environ["PR_REVIEW_REVIEWS"] +mode = os.environ["PR_REVIEW_TEST_MODE"] acting = os.environ["PR_REVIEW_ACTING_LOGIN"] foreign = os.environ["PR_REVIEW_FOREIGN_LOGIN"] +submitted = json.loads(os.environ["PR_REVIEW_PAYLOAD"]) +with open(state_path, encoding="utf-8") as handle: + reviews = json.load(handle) -def review(review_id, review_state, commit, login): - return { - "id": review_id, - "state": review_state, - "commit_id": commit, - "user": {"login": login}, - } +# no-op-concurrent-review: the wrapper's own submit is SUPPRESSED (200, no +# created object) even though a concurrent same-identity, same-state review at +# the same head already exists. Nothing is persisted; no created id to verify. +if mode == "no-op-concurrent-review": + print("200") + print(json.dumps({})) + raise SystemExit(0) +author = foreign if mode == "author-mismatch-review" else acting +new_id = (max((r["id"] for r in reviews), default=0)) + 1 +record = { + "id": new_id, + "state": submitted.get("event"), + "commit_id": submitted.get("commit_id"), + "body": submitted.get("body"), + "user": {"login": author}, +} +reviews.append(record) +with open(state_path, "w", encoding="utf-8") as handle: + json.dump(reviews, handle) +print("201") +print(json.dumps(record)) +PY +)" +elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE"/pulls/123/reviews/* ]]; then + emit "$(PR_REVIEW_GET_ID="${path##*/}" python3 - <<'PY' +import json +import os -if phase == "boundary": - records = [review(100, "COMMENT", "oldsha0000", acting)] if page == 1 else [] -elif mode == "paginated-approve": - # A full first page (50 non-matching records) forces the read-back to - # request page 2, where the genuine matching review lives. - if page == 1: - records = [review(101 + i, "COMMENT", "oldsha0000", acting) for i in range(50)] - elif page == 2: - records = [review(200, state, "HEADSHA_FEEDFACE", acting)] - else: - records = [] -elif mode == "foreign-review": - # A concurrent APPROVED review at the current head, but authored by a - # DIFFERENT identity. tea created nothing; attribution must reject this. - records = [review(200, state, "HEADSHA_FEEDFACE", foreign)] if page == 1 else [] +state_path = os.environ["PR_REVIEW_REVIEWS"] +wanted = int(os.environ["PR_REVIEW_GET_ID"]) +with open(state_path, encoding="utf-8") as handle: + reviews = json.load(handle) +match = next((r for r in reviews if r["id"] == wanted), None) +if match is None: + print("404") + print(json.dumps({"message": "not found"})) else: - if page == 1: - records = [ - review(100, "COMMENT", "oldsha0000", acting), - review(200, state, "HEADSHA_FEEDFACE", acting), - ] - else: - records = [] -print(json.dumps(records)) + print("200") + print(json.dumps(match)) PY -) - write_response 200 "$response" - elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123" ]]; then - write_response 200 '{"head":{"sha":"HEADSHA_FEEDFACE"}}' - elif [[ "$method" == "POST" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then - PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' +)" +elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then + emit "$(PR_REVIEW_QUERY="$query" python3 - <<'PY' +import json +import os +from urllib.parse import parse_qs + +state_path = os.environ["PR_REVIEW_REVIEWS"] +params = parse_qs(os.environ["PR_REVIEW_QUERY"]) +limit = int(params.get("limit", ["50"])[0]) +page = int(params.get("page", ["1"])[0]) +with open(state_path, encoding="utf-8") as handle: + reviews = json.load(handle) +start = (page - 1) * limit +print("200") +print(json.dumps(reviews[start:start + limit])) +PY +)" +elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then + case "$mode" in + write-transport-failure) + echo "simulated transport failure" >&2 + exit 7 + ;; + write-http-failure) + write_response 500 '{"message":"simulated rejection"}' + ;; + *) + emit "$(PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' import json import os -assert json.loads(os.environ["PR_REVIEW_PAYLOAD"]) == {"body": os.environ["PR_REVIEW_EXPECTED_BODY"]} -PY - response=$(python3 - <<'PY' -import json -import os - -print(json.dumps({"id": 456, "body": os.environ["PR_REVIEW_EXPECTED_BODY"]})) -PY -) - write_response 201 "$response" - elif [[ "$method" == "GET" && "$url" == "$PR_REVIEW_EXPECTED_API_BASE/issues/comments/456" ]]; then - if [[ "$PR_REVIEW_TEST_MODE" == "readback-failure" ]]; then - body="different-body" - else - body="$PR_REVIEW_EXPECTED_BODY" - fi - response=$(PR_REVIEW_BODY="$body" python3 - <<'PY' -import json -import os - -print(json.dumps({ +state_path = os.environ["PR_REVIEW_COMMENTS"] +acting = os.environ["PR_REVIEW_ACTING_LOGIN"] +base = os.environ["PR_REVIEW_EXPECTED_API_BASE"] +body = json.loads(os.environ["PR_REVIEW_PAYLOAD"]).get("body") +record = { "id": 456, - "body": os.environ["PR_REVIEW_BODY"], - "issue_url": os.environ["PR_REVIEW_EXPECTED_API_BASE"] + "/issues/123", -})) + "body": body, + "user": {"login": acting}, + "issue_url": f"{base}/issues/123", +} +with open(state_path, "w", encoding="utf-8") as handle: + json.dump([record], handle) +print("201") +print(json.dumps(record)) PY -) - write_response 200 "$response" - else - echo "Unexpected curl request: $method $url" >&2 - exit 97 - fi - ;; - *) - exit 98 - ;; -esac +)" + ;; + esac +elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE"/issues/comments/* ]]; then + emit "$(PR_REVIEW_GET_ID="${path##*/}" python3 - <<'PY' +import json +import os + +state_path = os.environ["PR_REVIEW_COMMENTS"] +mode = os.environ["PR_REVIEW_TEST_MODE"] +wanted = int(os.environ["PR_REVIEW_GET_ID"]) +with open(state_path, encoding="utf-8") as handle: + comments = json.load(handle) +match = next((c for c in comments if c["id"] == wanted), None) +if match is None: + print("404") + print(json.dumps({"message": "not found"})) + raise SystemExit(0) +if mode == "readback-failure": + # The server returns a DIFFERENT body than was created — a genuine + # provider-side mismatch the wrapper must reject. + match = dict(match, body="different-body") +print("200") +print(json.dumps(match)) +PY +)" +else + echo "Unexpected curl request: $method $url" >&2 + exit 97 +fi SH chmod +x "$BIN_DIR/curl" +# Seed persistent server state for a mode before the wrapper runs. +seed_state() { + local mode="$1" + printf '[]' > "$COMMENTS_FILE" + rm -f "$SUBMIT_PAYLOAD_FILE" + PR_REVIEW_SEED_MODE="$mode" PR_REVIEW_SEED_ACTING="$ACTING_LOGIN" \ + PR_REVIEW_SEED_HEAD="$HEAD_SHA" python3 - "$REVIEWS_FILE" <<'PY' +import json +import os +import sys + +mode = os.environ["PR_REVIEW_SEED_MODE"] +acting = os.environ["PR_REVIEW_SEED_ACTING"] +head = os.environ["PR_REVIEW_SEED_HEAD"] + + +def review(rid, state, commit, login): + return {"id": rid, "state": state, "commit_id": commit, "user": {"login": login}} + + +if mode == "paginated-approve": + # 50 pre-existing reviews fill page 1 (limit 50); the review this run submits + # becomes id 51 and lands ALONE on page 2, exercising >page-1 pagination in + # the enumeration check. + reviews = [review(i, "COMMENT", "oldsha0000", acting) for i in range(1, 51)] +elif mode == "no-op-concurrent-review": + # A concurrent SAME-IDENTITY APPROVED review at the CURRENT head already + # exists. The wrapper's own submit will be a no-op; it must fail closed + # because no created id is returned — it must not scan and accept this one. + reviews = [review(77, "APPROVED", head, acting)] +else: + reviews = [review(100, "COMMENT", "oldsha0000", acting)] + +with open(sys.argv[1], "w", encoding="utf-8") as handle: + json.dump(reviews, handle) +PY +} + run_review() { local mode="$1" action="$2" comment="${3:-}" local configured_url="${4:-https://git.mosaicstack.dev}" @@ -288,101 +328,144 @@ run_review() { : > "$TEA_LOG" : > "$CURL_LOG" : > "$OUTPUT_FILE" - rm -f "$WORK_DIR/review_calls" + seed_state "$mode" ( cd "$REPO_DIR" PATH="$BIN_DIR:$PATH" \ + XDG_CONFIG_HOME="$XDG_DIR" \ MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ PR_REVIEW_TEA_LOG="$TEA_LOG" \ + PR_REVIEW_LOGIN_URL="${configured_url%/}" \ PR_REVIEW_CURL_LOG="$CURL_LOG" \ - PR_REVIEW_REVIEW_CALLS="$WORK_DIR/review_calls" \ + PR_REVIEW_REVIEWS="$REVIEWS_FILE" \ + PR_REVIEW_COMMENTS="$COMMENTS_FILE" \ + PR_REVIEW_SUBMIT_PAYLOAD="$SUBMIT_PAYLOAD_FILE" \ PR_REVIEW_TEST_MODE="$mode" \ PR_REVIEW_EXPECTED_BODY="$comment" \ PR_REVIEW_EXPECTED_API_BASE="$expected_api_base" \ PR_REVIEW_API_ROOT="$expected_api_root" \ + PR_REVIEW_HEAD_SHA="$HEAD_SHA" \ PR_REVIEW_ACTING_LOGIN="$ACTING_LOGIN" \ PR_REVIEW_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ "$SCRIPT_DIR/pr-review.sh" -n 123 -a "$action" ${comment:+-c "$comment"} ) > "$OUTPUT_FILE" 2>&1 } +assert_no_tea_write() { + # tea must only ever be used for the login list, never to write. + if grep -qvE '^login list --output json$' "$TEA_LOG"; then + echo "FAIL: wrapper invoked tea for something other than the login list" >&2 + cat "$TEA_LOG" >&2 + exit 1 + fi +} + +# Case 1: a plain approve submits a review via REST and verifies it by its exact +# provider-returned id (id 101), attributed to the acting identity, pinned to +# the PR head, with no separate comment. run_review approve approve -grep -q '^pr approve 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" -# #865: the approval STATE itself is read back — a pre-write boundary GET and a -# post-write read-back GET on the (paginated) reviews endpoint, plus a PR head -# lookup and an acting-identity (GET /user) resolution for attribution. -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG" +grep -q 'Approved and verified Gitea PR #123 (review ID 101)' "$OUTPUT_FILE" grep -q '^GET https://git.mosaicstack.dev/api/v1/user$' "$CURL_LOG" -if grep -q 'comment' "$TEA_LOG"; then - echo "Plain approve (no review body) unexpectedly touched comment persistence" >&2 +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG" +grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/101$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" +assert_no_tea_write +# The submitted review payload carries the event and the PR head commit_id. +PR_REVIEW_HEAD_SHA="$HEAD_SHA" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' +import json +import os +import sys + +payload = json.load(open(sys.argv[1], encoding="utf-8")) +assert payload["event"] == "APPROVED", payload +assert payload["commit_id"] == os.environ["PR_REVIEW_HEAD_SHA"], payload +PY +# A plain approve (no body) must not POST a comment. +if grep -q '/issues/123/comments' "$CURL_LOG"; then + echo "FAIL: plain approve unexpectedly posted a comment" >&2 exit 1 fi -# #865 (invocation attribution): a concurrent APPROVED review at the current -# head, authored by a DIFFERENT identity while tea created nothing, must NOT -# satisfy verification — id-above-boundary + state + head is only temporal -# ordering, not proof THIS reviewer wrote it. -if run_review foreign-review approve; then +# Case 2: a submitted review NOT authored by the acting identity must FAIL +# CLOSED — the exact-id read-back enforces authorship. +if run_review author-mismatch-review approve; then echo "FAIL: approve accepted a review authored by a different identity" >&2 cat "$OUTPUT_FILE" >&2 exit 1 fi if grep -q 'Approved and verified' "$OUTPUT_FILE"; then - echo "FAIL: read-back matched a different-identity review (attribution bypassed)" >&2 + echo "FAIL: read-back did not enforce acting-identity authorship" >&2 exit 1 fi -# #865 (pagination): a genuine matching review that lands beyond page 1 of the -# reviews list must still be found by a fully-paginating read-back. -run_review paginated-approve approve -grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=2$' "$CURL_LOG" - -# #835: tea v0.11.1 defines no --comment/-comment flag on `pr approve`. A -# review body supplied alongside approve must be routed through the durable -# comment REST API instead of being passed to `tea` directly. -run_review approve approve approve-note -grep -q '^pr approve 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Approved and verified Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" -grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" -grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$OUTPUT_FILE" - -# #835: same for `pr reject` (request-changes), where a comment is required. -run_review request-changes request-changes changes-required -grep -q '^pr reject 123 --repo mosaicstack/stack --login mosaicstack$' "$TEA_LOG" -grep -q 'Requested changes and verified on Gitea PR #123 (review ID 200)' "$OUTPUT_FILE" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" -grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" -grep -q 'Added and verified review comment on Gitea PR #123 (comment ID 456)' "$OUTPUT_FILE" - -if run_review legacy-fallback comment durable-body; then - echo "The old nonexistent tea pr comment fallback returned success" >&2 +# Case 3: a no-op submit with a concurrent SAME-IDENTITY, same-state review at +# the current head already present must FAIL CLOSED — the closed concurrency +# window. The wrapper must not read back (or accept) the concurrent id 77. +if run_review no-op-concurrent-review approve; then + echo "FAIL: approve reported success when its submit no-opped but a concurrent review existed" >&2 cat "$OUTPUT_FILE" >&2 exit 1 fi -if grep -q '^pr comment ' "$TEA_LOG"; then - echo "Wrapper invoked unsupported tea pr comment" >&2 +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: approve accepted a concurrent review for a no-op submit (window not closed)" >&2 exit 1 fi -if grep -q 'Added comment to Gitea PR' "$OUTPUT_FILE"; then - echo "Wrapper reported success without durable persistence" >&2 +if grep -q '/pulls/123/reviews/77$' "$CURL_LOG"; then + echo "FAIL: wrapper read back the concurrent review id 77 (illegitimate fallback)" >&2 exit 1 fi +# Case 4: a genuine matching review that lands beyond page 1 of the reviews list +# must still be found by the fully-paginating enumeration check. +run_review paginated-approve approve +grep -q 'Approved and verified Gitea PR #123 (review ID 51)' "$OUTPUT_FILE" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/51$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=2$' "$CURL_LOG" + +# Case 5: an approve WITH a body carries that body in the review submit itself — +# there is no separate detached comment POST. +run_review approve approve approve-note +grep -q 'Approved and verified Gitea PR #123 (review ID 101)' "$OUTPUT_FILE" +PR_REVIEW_EXPECTED_BODY="approve-note" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' +import json +import os +import sys + +payload = json.load(open(sys.argv[1], encoding="utf-8")) +assert payload["body"] == os.environ["PR_REVIEW_EXPECTED_BODY"], payload +PY +if grep -q '/issues/123/comments' "$CURL_LOG"; then + echo "FAIL: approve-with-body posted a separate comment instead of carrying the body on the review" >&2 + exit 1 +fi + +# Case 6: request-changes requires a body and carries it on the REQUEST_CHANGES +# review submit. +run_review request-changes request-changes changes-required +grep -q 'Requested changes and verified on Gitea PR #123 (review ID 101)' "$OUTPUT_FILE" +grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" +grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/101$' "$CURL_LOG" +PR_REVIEW_EXPECTED_BODY="changes-required" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' +import json +import os +import sys + +payload = json.load(open(sys.argv[1], encoding="utf-8")) +assert payload["event"] == "REQUEST_CHANGES", payload +assert payload["body"] == os.environ["PR_REVIEW_EXPECTED_BODY"], payload +PY +assert_no_tea_write + +# Case 7: the `comment` action creates a comment via REST and verifies it by its +# exact created id, attributed to the acting identity. This also exercises +# owner/repo + base-URL resolution across clone-URL shapes. complex_body=$'durable "body"\n-- marker' run_review comment-success comment "$complex_body" grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" grep -q 'Added and verified comment on Gitea PR #123' "$OUTPUT_FILE" -if [[ -s "$TEA_LOG" ]]; then - echo "REST comment path unexpectedly invoked tea" >&2 - cat "$TEA_LOG" >&2 - exit 1 -fi +assert_no_tea_write run_review http-success comment durable-body http://git.mosaicstack.dev grep -q '^POST http://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" @@ -410,23 +493,17 @@ grep -q '^POST https://git.example/api/v1/repos/owner/repo/issues/123/comments$' run_review url-ssh-success comment durable-body https://git.example ssh://git@git.example/owner/repo.git owner/repo grep -q '^POST https://git.example/api/v1/repos/owner/repo/issues/123/comments$' "$CURL_LOG" -# #850 (follow-up to #812): an SSH remote's transport port (e.g. `ssh:// -# git@host:2222/...`) must NOT be compared against the configured HTTP(S) API -# URL's port -- they identify unrelated properties (SSH daemon port vs. HTTP(S) -# provider port) of the same Gitea host. Before the fix, host-match required -# the configured URL to carry the identical port, so this failed closed even -# though both remote and configured URL name the same host. +# #850: an SSH remote's transport port must not be compared against the +# configured HTTP(S) API URL's port. run_review ssh-transport-port-success comment durable-body https://git.example ssh://git@git.example:2222/owner/repo.git owner/repo grep -q '^POST https://git.example/api/v1/repos/owner/repo/issues/123/comments$' "$CURL_LOG" -# #850 (follow-up to #812): an explicit default HTTP(S) port on the remote -# (`https://host:443/...`) must be treated as equal to an implicit -# (portless) configured URL on BOTH sides -- the pre-fix comparison only -# normalized the default port when the REMOTE side was portless, so the -# inverse (explicit remote, implicit configured) form failed closed. +# #850: an explicit default HTTP(S) port on the remote must equal an implicit +# (portless) configured URL. run_review explicit-default-port-success comment durable-body https://git.example https://git.example:443/owner/repo.git owner/repo grep -q '^POST https://git.example/api/v1/repos/owner/repo/issues/123/comments$' "$CURL_LOG" +# Comment write/read-back failure modes must all fail closed. if run_review write-transport-failure comment durable-body; then echo "Expected provider transport failure to return nonzero" >&2 exit 1 @@ -444,4 +521,4 @@ if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then exit 1 fi -echo "pr-review.sh durable Gitea comment regression passed" +echo "pr-review.sh REST review + comment create/read-back regression passed" -- 2.49.1 From 6168f9ac86e52c28441b6563cddb94230ffdb303 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 20:07:00 -0500 Subject: [PATCH 05/15] fix(git-tools): fail closed on unresolvable explicit --login override (#865) gitea_resolve_api_for_login silently fell back to the host-default identity whenever the named login could not be resolved for ANY reason -- including when the name came from an EXPLICIT --login override. A caller passing a dedicated per-role credential could thus have its write attributed to the shared default identity while being told it succeeded as requested. Thread an "override was explicit" signal into gitea_resolve_api_for_login (second param, "explicit" when LOGIN_OVERRIDE is non-empty). When the override is explicit and that login's token cannot be resolved, FAIL CLOSED (return 1, clear error naming the login, no host-default fallback). The best-effort host-default fallback now applies ONLY on the no-override default path. Applied symmetrically to issue-comment.sh and all three pr-review.sh dispatch sites (approve / request-changes / comment). Tests: both scripts' write flows now assert credential attribution via a token->identity seam in the curl stub -- (a) resolvable --login override drives the entire write/read-back chain under THAT login's token, nothing under the default; (b) unresolvable --login override fails closed (nonzero, no success line, no write, no default-identity request); (c) no-override default path still succeeds under the host-default best-effort credential. Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/issue-comment.sh | 34 ++++-- .../mosaic/framework/tools/git/pr-review.sh | 32 ++++-- .../tools/git/test-issue-comment-readback.sh | 102 ++++++++++++++++- .../tools/git/test-pr-review-gitea-comment.sh | 107 +++++++++++++++++- 4 files changed, 245 insertions(+), 30 deletions(-) diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index ce08d023..f225844b 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -85,17 +85,29 @@ detect_platform >/dev/null # and read-back token are the same identity by construction (this is the # credential-ordering fix: a --login override is no longer written under one # credential and verified under a different default one). Falls back to the -# host-scoped credential only when the login has no token in tea's own config. -# Returns non-zero (clear stderr) on any resolution failure. +# host-scoped credential ONLY when NO --login override was supplied (the +# best-effort default path). When $2 is "explicit" the login came from a +# caller-supplied --login: that exact login's token MUST resolve, and we FAIL +# CLOSED rather than silently downgrading the write to the host default +# identity — otherwise a caller relying on a dedicated per-role credential would +# be told the write succeeded as requested while it was attributed to the shared +# default. Returns non-zero (clear stderr) on any resolution failure. gitea_resolve_api_for_login() { - local effective_login="$1" host configured_url repo + local effective_login="$1" override_explicit="${2:-}" host configured_url repo host=$(get_remote_host) - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ - || GITEA_API_TOKEN=$(get_gitea_token "$host") || { - echo "Error: Gitea token not found for login '$effective_login' (comment write/read-back)" >&2 - return 1 - } + if [[ -n "$override_explicit" ]]; then + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") || { + echo "Error: could not resolve a Gitea token for --login '$effective_login'; refusing to fall back to the host default identity (comment write/read-back)" >&2 + return 1 + } + else + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + || GITEA_API_TOKEN=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for login '$effective_login' (comment write/read-back)" >&2 + return 1 + } + fi configured_url=$(get_gitea_url_for_host "$host") || { echo "Error: Configured Gitea URL not found for comment read-back verification" >&2 return 1 @@ -379,8 +391,10 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # Bind the REST endpoint + token to the effective login, then derive the # acting identity from that SAME credential (GET /user). The write below and # its read-back both use this credential, so the write is verified against - # the identity that actually performed it. - gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 + # the identity that actually performed it. Passing "explicit" when --login + # was supplied forbids the host-default fallback: an unresolvable explicit + # override fails closed instead of writing under the default identity. + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" "${LOGIN_OVERRIDE:+explicit}" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 comment_id=$(gitea_create_comment_verified "$ISSUE_NUMBER" "$COMMENT" "$ACTING_LOGIN") || { diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index ca2315d1..555c6e6a 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -193,17 +193,27 @@ PY # write token and read-back token are the same identity by construction. This # is the credential-ordering fix: a --login override is no longer submitted # under one credential and verified under a different default one. Falls back to -# the host-scoped credential only when the login has no token in tea's config. -# Returns non-zero (clear stderr) on any resolution failure. +# the host-scoped credential ONLY when NO --login override was supplied (the +# best-effort default path). When $2 is "explicit" the login came from a +# caller-supplied --login: that exact login's token MUST resolve, and we FAIL +# CLOSED rather than silently downgrading the review/comment to the host default +# identity. Returns non-zero (clear stderr) on any resolution failure. gitea_resolve_api_for_login() { - local effective_login="$1" host configured_url repo + local effective_login="$1" override_explicit="${2:-}" host configured_url repo host=$(get_remote_host) - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ - || GITEA_API_TOKEN=$(get_gitea_token "$host") || { - echo "Error: Gitea token not found for login '$effective_login' (review write/read-back)" >&2 - return 1 - } + if [[ -n "$override_explicit" ]]; then + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") || { + echo "Error: could not resolve a Gitea token for --login '$effective_login'; refusing to fall back to the host default identity (review write/read-back)" >&2 + return 1 + } + else + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + || GITEA_API_TOKEN=$(get_gitea_token "$host") || { + echo "Error: Gitea token not found for login '$effective_login' (review write/read-back)" >&2 + return 1 + } + fi configured_url=$(get_gitea_url_for_host "$host") || { echo "Error: Configured Gitea URL not found for review read-back verification" >&2 return 1 @@ -552,7 +562,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # Bind the REST endpoint + token to the effective login, then derive # the acting identity from that SAME credential so the review submit # and its read-back verify against the identity that performed them. - gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" "${LOGIN_OVERRIDE:+explicit}" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 head_sha=$(gitea_pr_head_sha "$PR_NUMBER") || exit 1 # The review body (if any) travels with the review itself in the REST @@ -579,7 +589,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # then used for the write, the /user identity, and the read-back. EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login_for_host "$host" 2>/dev/null || true) - gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" "${LOGIN_OVERRIDE:+explicit}" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 head_sha=$(gitea_pr_head_sha "$PR_NUMBER") || exit 1 review_id=$(gitea_submit_review_verified "$PR_NUMBER" "REQUEST_CHANGES" "$COMMENT" "$ACTING_LOGIN" "$head_sha") || { @@ -603,7 +613,7 @@ elif [[ "$PLATFORM" == "gitea" ]]; then # then used for the write, the /user identity, and the read-back. EFFECTIVE_LOGIN="$LOGIN_OVERRIDE" [[ -n "$EFFECTIVE_LOGIN" ]] || EFFECTIVE_LOGIN=$(get_gitea_login_for_host "$host" 2>/dev/null || true) - gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" || exit 1 + gitea_resolve_api_for_login "$EFFECTIVE_LOGIN" "${LOGIN_OVERRIDE:+explicit}" || exit 1 ACTING_LOGIN=$(gitea_authenticated_login) || exit 1 comment_id=$(gitea_create_comment_verified "$PR_NUMBER" "$COMMENT" "$ACTING_LOGIN") || { echo "Error: could not create and verify a comment on Gitea PR #$PR_NUMBER via a provider-returned created id (#865)." >&2 diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index d32662ea..6eb3bdfe 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -27,7 +27,14 @@ # 5. fails closed when the created record is not authored by the acting # identity; # 6. enumerates the created id in the issue's FULLY PAGINATED comment list, -# finding it even when it lands beyond page 1. +# finding it even when it lands beyond page 1; +# 7. with a RESOLVABLE --login override, performs the write, the /user identity +# lookup, and the read-back ALL under THAT login's token/identity — never +# the host default; +# 8. with an UNRESOLVABLE --login override, FAILS CLOSED (nonzero, no write, no +# success line) instead of silently downgrading to the host default +# identity — the token seam maps each bearer token to the identity it +# authenticates as, so a misattributed write is caught. set -euo pipefail @@ -38,6 +45,7 @@ BIN_DIR="$WORK_DIR/bin" XDG_DIR="$WORK_DIR/xdg" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" +AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" STATE_FILE="$WORK_DIR/comments.json" @@ -58,6 +66,27 @@ API_ROOT="https://git.mosaicstack.dev/api/v1" BODY='durable "note" -- marker' ACTING_LOGIN="primary-reviewer" FOREIGN_LOGIN="other-writer" +# A dedicated per-role --login override identity, with its own token stored in +# tea's config (exactly the author-not-equal-reviewer hardening path). +OVERRIDE_LOGIN="delegated-reviewer" +DEFAULT_TOKEN="test-only-placeholder" +OVERRIDE_TOKEN="override-token-placeholder" + +# tea config: the override login has its own token here (as tea itself stores +# per-login tokens). The default login name ("mosaicstack") is deliberately NOT +# present, so the no-override default path resolves via the host credential +# fallback while an explicit --login must resolve from this file or fail closed. +mkdir -p "$XDG_DIR/tea" +OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" python3 - "$XDG_DIR/tea/config.yml" <<'PY' +import os +import sys + +with open(sys.argv[1], "w", encoding="utf-8") as handle: + handle.write("logins:\n") + handle.write(f" - name: {os.environ['OVERRIDE_LOGIN']}\n") + handle.write(" url: https://git.mosaicstack.dev\n") + handle.write(f" token: {os.environ['OVERRIDE_TOKEN']}\n") +PY CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY' import json @@ -106,10 +135,14 @@ output_file="" method="GET" url="" data="" +auth_token="" while [[ $# -gt 0 ]]; do case "$1" in -o) output_file="$2"; shift 2 ;; - -w|-H) shift 2 ;; + -H) + [[ "$2" == Authorization:* ]] && auth_token="${2##* }" + shift 2 ;; + -w) shift 2 ;; -X) method="$2"; shift 2 ;; -d|--data) data="$2"; shift 2 ;; -s|-S|-sS) shift ;; @@ -123,6 +156,17 @@ query="${url#*\?}" [[ "$query" == "$url" ]] && query="" printf '%s %s\n' "$method" "$url" >> "$ISSUE_COMMENT_CURL_LOG" +# Map the presented bearer token to the identity it authenticates as — the same +# derivation Gitea's own /user does. The wrapper's write, /user lookup, and +# read-back must all carry the SAME token, so the acting identity recorded here +# reveals which credential actually performed the request. +acting_identity="" +case "$auth_token" in + "$ISSUE_COMMENT_DEFAULT_TOKEN") acting_identity="$ISSUE_COMMENT_ACTING_LOGIN" ;; + "$ISSUE_COMMENT_OVERRIDE_TOKEN") acting_identity="$ISSUE_COMMENT_OVERRIDE_LOGIN" ;; +esac +printf '%s %s %s\n' "$method" "$path" "${acting_identity:-}" >> "$ISSUE_COMMENT_AUTH_LOG" + write_response() { local status="$1" body="$2" [[ -n "$output_file" ]] || exit 96 @@ -131,14 +175,15 @@ write_response() { } if [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_ROOT/user" ]]; then - write_response 200 "$(ISSUE_COMMENT_LOGIN="$ISSUE_COMMENT_ACTING_LOGIN" python3 - <<'PY' + [[ -n "$acting_identity" ]] || { write_response 401 '{"message":"unauthenticated"}'; exit 0; } + write_response 200 "$(ISSUE_COMMENT_LOGIN="$acting_identity" python3 - <<'PY' import json import os print(json.dumps({"login": os.environ["ISSUE_COMMENT_LOGIN"]})) PY )" elif [[ "$method" == "POST" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then - result=$(ISSUE_COMMENT_DATA="$data" python3 - <<'PY' + result=$(ISSUE_COMMENT_ACTING_LOGIN="${acting_identity:-$ISSUE_COMMENT_ACTING_LOGIN}" ISSUE_COMMENT_DATA="$data" python3 - <<'PY' import json import os @@ -261,8 +306,10 @@ PY run_comment() { local mode="$1" + shift : > "$TEA_LOG" : > "$CURL_LOG" + : > "$AUTH_LOG" : > "$OUTPUT_FILE" seed_state "$mode" ( @@ -272,14 +319,18 @@ run_comment() { MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \ ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \ + ISSUE_COMMENT_AUTH_LOG="$AUTH_LOG" \ ISSUE_COMMENT_STATE="$STATE_FILE" \ ISSUE_COMMENT_TEST_MODE="$mode" \ ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \ ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ + ISSUE_COMMENT_OVERRIDE_LOGIN="$OVERRIDE_LOGIN" \ + ISSUE_COMMENT_DEFAULT_TOKEN="$DEFAULT_TOKEN" \ + ISSUE_COMMENT_OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ ISSUE_COMMENT_REPO_SLUG="$REPO_SLUG" \ ISSUE_COMMENT_API_BASE="$API_BASE" \ ISSUE_COMMENT_API_ROOT="$API_ROOT" \ - "$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY" + "$SCRIPT_DIR/issue-comment.sh" -i "$ISSUE_NUMBER" -c "$BODY" "$@" ) > "$OUTPUT_FILE" 2>&1 } @@ -299,6 +350,10 @@ grep -q "^GET $API_BASE/issues/comments/51$" "$CURL_LOG" grep -q "^GET $API_ROOT/user$" "$CURL_LOG" # Enumeration paginated beyond page 1 to find the created comment. grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=2$" "$CURL_LOG" +# Default path (no --login): the host credential fallback resolves, and the +# write is performed AND self-verified under the host-default acting identity. +grep -q "^POST $API_BASE/issues/7/comments $ACTING_LOGIN$" "$AUTH_LOG" +grep -q "^GET $API_BASE/issues/comments/51 $ACTING_LOGIN$" "$AUTH_LOG" # Case 2: a no-op write with a concurrent SAME-IDENTITY, same-body comment # already present must FAIL CLOSED — the closed concurrency window. @@ -328,4 +383,41 @@ if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then exit 1 fi +# Case 4: a RESOLVABLE --login override — the write, the /user identity lookup, +# and the read-back must ALL be performed under THAT login's token/identity, not +# the host default. The override login has id 1 (empty seed). +run_comment override-success --login "$OVERRIDE_LOGIN" +grep -q 'Added and verified comment on Gitea issue #7 (comment ID 1)' "$OUTPUT_FILE" +grep -q "^GET $API_ROOT/user $OVERRIDE_LOGIN$" "$AUTH_LOG" +grep -q "^POST $API_BASE/issues/7/comments $OVERRIDE_LOGIN$" "$AUTH_LOG" +grep -q "^GET $API_BASE/issues/comments/1 $OVERRIDE_LOGIN$" "$AUTH_LOG" +# The host-default identity must NOT have performed ANY request in this run. +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: an explicit --login override request was performed under the host default identity" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi + +# Case 5: an UNRESOLVABLE --login override (name absent from tea config) must +# FAIL CLOSED — no silent downgrade to the host default identity: nonzero exit, +# no success line, and NO write performed. +if run_comment override-unresolvable --login "nonexistent-typo-login"; then + echo "FAIL: unresolvable --login override did not fail closed" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: unresolvable --login override reported success" >&2 + exit 1 +fi +if grep -q "^POST $API_BASE/issues/7/comments" "$CURL_LOG"; then + echo "FAIL: unresolvable --login override still performed a write" >&2 + exit 1 +fi +# And it must not have silently fallen back to the host default identity. +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: unresolvable --login override fell back to the host default identity" >&2 + exit 1 +fi + echo "issue-comment.sh REST create + exact-id read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index 25f36cb3..3108235e 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -18,6 +18,14 @@ # the read-back reads that same state. There is no independently fabricated # record for the wrapper to "find" — verification passes only when the POST # genuinely created the record the read-back retrieves. +# +# #865 Round-4: the curl stub also maps the presented bearer token to the +# identity it authenticates as and logs it per request, so tests can prove +# credential attribution. An explicit --login override must drive the entire +# write→read-back chain under THAT login's token (resolvable case) or FAIL +# CLOSED (unresolvable case) — never silently downgrade to the host-default +# identity. The host-default best-effort fallback is reserved for the +# no-override default path. set -euo pipefail @@ -32,6 +40,7 @@ COMMENTS_FILE="$STATE_DIR/comments.json" SUBMIT_PAYLOAD_FILE="$STATE_DIR/review_payload.json" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" +AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" @@ -43,11 +52,32 @@ trap cleanup EXIT ACTING_LOGIN="review-bot" FOREIGN_LOGIN="other-writer" HEAD_SHA="HEADSHA_FEEDFACE" +# A dedicated per-role --login override identity with its own token in tea's +# config (the author-not-equal-reviewer hardening path). +OVERRIDE_LOGIN="primary-reviewer" +DEFAULT_TOKEN="test-only-placeholder" +OVERRIDE_TOKEN="override-token-placeholder" mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$STATE_DIR" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git +# tea config: the override login carries its own token here. The default login +# name ("mosaicstack") is deliberately absent, so the no-override default path +# resolves via the host credential fallback while an explicit --login must +# resolve from this file or fail closed. +mkdir -p "$XDG_DIR/tea" +OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" python3 - "$XDG_DIR/tea/config.yml" <<'PY' +import os +import sys + +with open(sys.argv[1], "w", encoding="utf-8") as handle: + handle.write("logins:\n") + handle.write(f" - name: {os.environ['OVERRIDE_LOGIN']}\n") + handle.write(" url: https://git.mosaicstack.dev\n") + handle.write(f" token: {os.environ['OVERRIDE_TOKEN']}\n") +PY + write_credentials() { local configured_url="$1" CONFIGURED_GITEA_URL="$configured_url" python3 - "$CREDENTIALS_FILE" <<'PY' @@ -96,10 +126,14 @@ output_file="" method="GET" payload="" url="" +auth_token="" while [[ $# -gt 0 ]]; do case "$1" in -o) output_file="$2"; shift 2 ;; - -w|-H) shift 2 ;; + -H) + [[ "$2" == Authorization:* ]] && auth_token="${2##* }" + shift 2 ;; + -w) shift 2 ;; -X) method="$2"; shift 2 ;; -d|--data) payload="$2"; shift 2 ;; -s|-S|-sS) shift ;; @@ -113,6 +147,17 @@ query="${url#*\?}" [[ "$query" == "$url" ]] && query="" printf '%s %s\n' "$method" "$url" >> "$PR_REVIEW_CURL_LOG" +# Map the presented bearer token to the identity it authenticates as (as Gitea's +# /user does). The write, /user lookup, and read-back must all carry the SAME +# token, so the identity logged here reveals which credential performed each +# request — proving an explicit --login override is honored, not downgraded. +acting_identity="" +case "$auth_token" in + "$PR_REVIEW_DEFAULT_TOKEN") acting_identity="$PR_REVIEW_ACTING_LOGIN" ;; + "$PR_REVIEW_OVERRIDE_TOKEN") acting_identity="$PR_REVIEW_OVERRIDE_LOGIN" ;; +esac +printf '%s %s %s\n' "$method" "$path" "${acting_identity:-}" >> "$PR_REVIEW_AUTH_LOG" + write_response() { local status="$1" body="$2" [[ -n "$output_file" ]] || exit 96 @@ -129,7 +174,8 @@ emit() { mode="${PR_REVIEW_TEST_MODE:-}" if [[ "$method" == "GET" && "$path" == "$PR_REVIEW_API_ROOT/user" ]]; then - write_response 200 "$(PR_REVIEW_LOGIN="$PR_REVIEW_ACTING_LOGIN" python3 - <<'PY' + [[ -n "$acting_identity" ]] || { write_response 401 '{"message":"unauthenticated"}'; exit 0; } + write_response 200 "$(PR_REVIEW_LOGIN="$acting_identity" python3 - <<'PY' import json import os print(json.dumps({"login": os.environ["PR_REVIEW_LOGIN"]})) @@ -144,7 +190,7 @@ PY )" elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then printf '%s' "$payload" > "$PR_REVIEW_SUBMIT_PAYLOAD" - emit "$(PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' + emit "$(PR_REVIEW_ACTING_LOGIN="${acting_identity:-$PR_REVIEW_ACTING_LOGIN}" PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' import json import os @@ -321,12 +367,14 @@ run_review() { local configured_url="${4:-https://git.mosaicstack.dev}" local remote_url="${5:-https://git.mosaicstack.dev/mosaicstack/stack.git}" local expected_repo="${6:-mosaicstack/stack}" + local login_override="${7:-}" local expected_api_base="${configured_url%/}/api/v1/repos/$expected_repo" local expected_api_root="${configured_url%/}/api/v1" git -C "$REPO_DIR" remote set-url origin "$remote_url" write_credentials "$configured_url" : > "$TEA_LOG" : > "$CURL_LOG" + : > "$AUTH_LOG" : > "$OUTPUT_FILE" seed_state "$mode" ( @@ -337,6 +385,7 @@ run_review() { PR_REVIEW_TEA_LOG="$TEA_LOG" \ PR_REVIEW_LOGIN_URL="${configured_url%/}" \ PR_REVIEW_CURL_LOG="$CURL_LOG" \ + PR_REVIEW_AUTH_LOG="$AUTH_LOG" \ PR_REVIEW_REVIEWS="$REVIEWS_FILE" \ PR_REVIEW_COMMENTS="$COMMENTS_FILE" \ PR_REVIEW_SUBMIT_PAYLOAD="$SUBMIT_PAYLOAD_FILE" \ @@ -347,7 +396,10 @@ run_review() { PR_REVIEW_HEAD_SHA="$HEAD_SHA" \ PR_REVIEW_ACTING_LOGIN="$ACTING_LOGIN" \ PR_REVIEW_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ - "$SCRIPT_DIR/pr-review.sh" -n 123 -a "$action" ${comment:+-c "$comment"} + PR_REVIEW_OVERRIDE_LOGIN="$OVERRIDE_LOGIN" \ + PR_REVIEW_DEFAULT_TOKEN="$DEFAULT_TOKEN" \ + PR_REVIEW_OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ + "$SCRIPT_DIR/pr-review.sh" -n 123 -a "$action" ${comment:+-c "$comment"} ${login_override:+--login "$login_override"} ) > "$OUTPUT_FILE" 2>&1 } @@ -370,6 +422,10 @@ grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/1 grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/101$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" +# No-override default path: the write, /user lookup, and read-back all resolve +# via the host-default credential and authenticate as the acting identity. +grep -q "^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews $ACTING_LOGIN\$" "$AUTH_LOG" +grep -q "^GET https://git.mosaicstack.dev/api/v1/user $ACTING_LOGIN\$" "$AUTH_LOG" assert_no_tea_write # The submitted review payload carries the event and the PR head commit_id. PR_REVIEW_HEAD_SHA="$HEAD_SHA" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' @@ -521,4 +577,47 @@ if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then exit 1 fi +# Case 8 (#865 Round-4): a RESOLVABLE explicit --login override must attribute +# the entire write→read-back chain to THAT login's token/identity, never the +# host-default identity. The override login carries its own token in the tea +# config, so /user, the review POST, and the exact-id read-back all authenticate +# as the override identity — and NOTHING is performed under the default identity. +run_review override-success approve "" https://git.mosaicstack.dev \ + https://git.mosaicstack.dev/mosaicstack/stack.git mosaicstack/stack "$OVERRIDE_LOGIN" +grep -q 'Approved and verified Gitea PR #123 (review ID 101)' "$OUTPUT_FILE" +grep -q "^GET https://git.mosaicstack.dev/api/v1/user $OVERRIDE_LOGIN\$" "$AUTH_LOG" +grep -q "^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews $OVERRIDE_LOGIN\$" "$AUTH_LOG" +grep -q "^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/101 $OVERRIDE_LOGIN\$" "$AUTH_LOG" +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: an explicit --login override was silently downgraded to the host-default identity" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +assert_no_tea_write + +# Case 9 (#865 Round-4): an UNRESOLVABLE explicit --login override (a name absent +# from the tea config) must FAIL CLOSED — nonzero exit, no success line, no review +# POST, and above all NO request performed under the host-default identity. The +# host-default best-effort fallback is reserved for the no-override path only. +if run_review override-unresolvable approve "" https://git.mosaicstack.dev \ + https://git.mosaicstack.dev/mosaicstack/stack.git mosaicstack/stack "nonexistent-typo-login"; then + echo "FAIL: an unresolvable --login override was not rejected (silently used the host default)" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: unresolvable --login override reported success" >&2 + exit 1 +fi +if grep -q '/pulls/123/reviews ' "$AUTH_LOG" && grep -qE '^POST .*/pulls/123/reviews ' "$AUTH_LOG"; then + echo "FAIL: unresolvable --login override performed a review POST" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: unresolvable --login override fell back to the host-default identity" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi + echo "pr-review.sh REST review + comment create/read-back regression passed" -- 2.49.1 From 2bb3ac4549d64975a310bfa8bb98ab5cca1a2a2c Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 20:49:48 -0500 Subject: [PATCH 06/15] fix(git-tools): env-robust token parse, host-bound creds, real Gitea URL verify (#865 round-5) CI-red root cause (classification a: my round-4 change fails in the clean/cold CI env): get_gitea_token_for_login hard-required PyYAML (`import yaml`), which is absent on CI's node:24-alpine (python3 without py3-yaml). Round-4's --login override cases were the first to exercise that path, turning the mosaic package test (test:framework-shell -> test-pr-review-gitea-comment.sh) RED. Fix: add an indentation-aware line-parser fallback that resolves the SAME per-name token PyYAML would from tea's flat `logins:` list; PyYAML stays the fast path. This also repairs a latent production defect (--login overrides were silently unusable on any PyYAML-less host). Auditor blockers folded into the same round-5: 1. issue_url vs pull_request_url shape (correctness): Gitea populates WEB (html) URLs in issue_url/pull_request_url, not API paths, and a PR-conversation comment carries pull_request_url (issue_url empty). Verification now accepts either web shape scoped to the repo slug + number, so a durable write is never rejected for URL shape. Test stubs now emit the REAL Gitea web shapes. 2. Cross-host credential binding (security): get_gitea_token_for_login now takes the repo host and requires the matched login's configured URL host to equal it; an override login configured for a different host FAILS CLOSED instead of sending a cross-host credential. Regression tests added to both suites. 3. Non-exhaustive enumeration (false-fail): removed the redundant, non-exhaustive post-verification list enumeration (gitea_fetch_all + confirm_*_enumerable) from both wrappers; the exact-id GET is authoritative. Pagination cases dropped; a guard asserts no list enumeration is performed. 4. Trap clobbering / temp-file leak (security/hygiene): removing the nested enumeration eliminates the RETURN-trap nesting that clobbered caller cleanup; remaining RETURN traps are single/non-nested and clean up on all exit paths. Temp-file leak regression tests (success + failure paths) added to both suites. 5. README: corrected the exhaustive-pagination claim and documented host-bound --login selection. Preserves every round-2/3/4 fix (explicit --login fail-closed at all write sites, token->identity attribution seam). Gates: cold `pnpm turbo run test --filter=@mosaicstack/mosaic` green (14/14); full test-*.sh suite green with AND without PyYAML; bash -n, shellcheck -x -S warning, prettier --check README clean. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 4 +- .../framework/tools/git/detect-platform.sh | 147 ++++++++++++++--- .../framework/tools/git/issue-comment.sh | 134 ++------------- .../mosaic/framework/tools/git/pr-review.sh | 135 ++------------- .../tools/git/test-issue-comment-readback.sh | 155 +++++++++++++----- .../tools/git/test-pr-review-gitea-comment.sh | 135 +++++++++++---- 6 files changed, 373 insertions(+), 337 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index 8cf68280..6ffaf45f 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -15,7 +15,7 @@ A successful provider write command—or a wrapper message based only on that co **This closes the concurrency window rather than documenting it.** Because verification keys on the id the create returned, a no-op create yields no id and fails closed with no list-scan fallback, and a _concurrent_ record — even one written by the _same_ identity with an identical body/state — has a _different_ id and cannot be mistaken for this write. There is no residual same-identity window: the earlier boundary-and-author heuristic (accept any `id > pre-write-max` with a matching author) is replaced entirely by exact-id attribution. -**Full pagination.** After the exact-id read-back, each wrapper also confirms the created id is enumerable in the record list, walking every page (`?limit=&page=1,2,…` until a short/empty page) so a record that lands beyond the first page is still found regardless of how many comments or reviews already exist. +**Exact-id read-back is the sole authority.** Verification is a direct `GET` of the one record the create returned; there is no follow-up list enumeration. An earlier redundant pass that re-listed the record's page (`?limit=&page=1,2,…`) was removed: server-capped page sizes and list-pagination quirks made it a false-failure source (a durable, exact-id-verified record could be missed by a non-exhaustive enumeration), and it added nothing over the authoritative exact-id `GET`. ## `tea` invocation notes (Gitea) @@ -24,6 +24,6 @@ A successful provider write command—or a wrapper message based only on that co ### `--login` override -Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea login for that single invocation. The override selects **which credential the REST write, the `/user` identity lookup, and the read-back all use** — its token is resolved from the tea config for that login name (`get_gitea_token_for_login`), falling back to the repo host's credential when no login is named. Resolving the acting identity and the read-back from the _same_ login that performs the write is essential: a write performed under an overridden login must be verified against that login's identity, not the host default's. Callers who need a different login than the host default should pass `--login `. +Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea login for that single invocation. The override selects **which credential the REST write, the `/user` identity lookup, and the read-back all use** — its token is resolved from the tea config for that login name (`get_gitea_token_for_login`), falling back to the repo host's credential when no login is named. The resolved login is **host-bound**: the login's configured URL host must match the repo remote's host, so a login name shared across hosts (or an override configured for a different Gitea) can never send one host's credential to another — a host mismatch fails closed rather than leaking a cross-host token. Resolving the acting identity and the read-back from the _same_ login that performs the write is essential: a write performed under an overridden login must be verified against that login's identity, not the host default's. Callers who need a different login than the host default should pass `--login `. As a durable successor to this mechanism, consider giving each reviewer/approver slot its own dedicated Gitea login credential, so that author≠reviewer holds at the credential level rather than relying on wrapper-level `--login` bookkeeping. This is a recommendation for future hardening, not something implemented by this flag. diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 5ee119e2..2e0a5194 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -569,44 +569,137 @@ get_gitea_token() { # per-login tokens by `name` in $XDG_CONFIG_HOME/tea/config.yml (default # ~/.config/tea/config.yml), exactly as the `tea` CLI resolves them, so a # --login override and its REST read-back bind to the SAME credential/identity. -# Prints the token on success; returns non-zero (no output) if the config or a -# matching login token cannot be found. Callers must not log the result. +# +# $2 (repo host) binds the selected credential to the TARGET host: a tea login +# also records the `url` it authenticates against, and the matched login's URL +# host MUST equal the repo host. This fails closed when an override login is +# configured for a DIFFERENT host than the repo remote, so a login name shared +# across hosts (or a mistargeted override) can never send one host's credential +# to another host (cross-host credential leak). When $2 is empty the host bind +# is skipped (host-agnostic lookup) — callers that write should always pass it. +# +# Prints the token on success; returns non-zero (no output) if the config, a +# matching login token, or the host bind cannot be satisfied. Callers must not +# log the result. get_gitea_token_for_login() { - local login_name="$1" config_file + local login_name="$1" repo_host="${2:-}" config_file [[ -n "$login_name" ]] || return 1 config_file="${XDG_CONFIG_HOME:-$HOME/.config}/tea/config.yml" [[ -f "$config_file" ]] || return 1 - LOGIN_NAME="$login_name" python3 - "$config_file" <<'PY' + LOGIN_NAME="$login_name" REPO_HOST="$repo_host" python3 - "$config_file" <<'PY' import os +import re import sys - -try: - import yaml -except ImportError: - raise SystemExit(1) - -try: - with open(sys.argv[1], encoding="utf-8") as handle: - config = yaml.safe_load(handle) -except (OSError, yaml.YAMLError): - raise SystemExit(1) +from urllib.parse import urlparse wanted = os.environ["LOGIN_NAME"] -logins = config.get("logins") if isinstance(config, dict) else None -if not isinstance(logins, list): +repo_host = os.environ.get("REPO_HOST", "").strip().lower() +config_path = sys.argv[1] + + +def _strip_scalar(value): + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in ("'", '"'): + value = value[1:-1] + return value + + +def _host_of(url): + if not isinstance(url, str) or not url: + return None + parsed = urlparse(url if "//" in url else f"//{url}") + host = parsed.hostname + return host.lower() if host else None + + +def _accept(token, url): + # Enforce the host bind before surfacing a token. When a repo host is given, + # the login's recorded URL host must match it exactly; a login with no + # usable URL (or a mismatched one) is rejected (fail closed) so a cross-host + # credential is never emitted. + if not isinstance(token, str) or not token: + return None + if repo_host: + if _host_of(url) != repo_host: + return None + return token + + +def _token_via_pyyaml(): + # Preferred, fully general path when PyYAML is installed. Raises ImportError + # (caught by the caller) when the module is unavailable so the environment + # -robust fallback can take over instead of failing closed on every host + # that lacks PyYAML. + import yaml + + with open(config_path, encoding="utf-8") as handle: + config = yaml.safe_load(handle) + logins = config.get("logins") if isinstance(config, dict) else None + if not isinstance(logins, list): + return None + for login in logins: + if isinstance(login, dict) and str(login.get("name") or "") == wanted: + return _accept(login.get("token"), login.get("url")) + return None + + +def _token_via_lines(): + # Conservative fallback for hosts without PyYAML. tea writes config.yml in a + # fixed, flat shape (a `logins:` list of maps with scalar name/url/token + # fields), so a small indentation-aware scan resolves the SAME token PyYAML + # would. It only ever returns the `token` of the entry whose `name` EXACTLY + # equals the requested login AND whose url host matches the repo host, so it + # cannot misattribute to another identity or host; anything it cannot parse + # yields None (fail closed). + with open(config_path, encoding="utf-8") as handle: + lines = handle.read().splitlines() + + logins_indent = None + start = len(lines) + for index, line in enumerate(lines): + match = re.match(r"^(\s*)logins\s*:\s*$", line) + if match: + logins_indent = len(match.group(1)) + start = index + 1 + break + if logins_indent is None: + return None + + entries = [] + current = None + for line in lines[start:]: + if not line.strip() or line.lstrip().startswith("#"): + continue + indent = len(line) - len(line.lstrip(" ")) + if indent <= logins_indent: + break + item = re.match(r"^\s*-\s*(.*)$", line) + rest = item.group(1) if item else line + if item: + current = {} + entries.append(current) + pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", rest.strip()) + if pair and current is not None: + current[pair.group(1)] = _strip_scalar(pair.group(2)) + + for entry in entries: + if str(entry.get("name") or "") == wanted: + return _accept(entry.get("token"), entry.get("url")) + return None + + +try: + try: + token = _token_via_pyyaml() + except ImportError: + token = _token_via_lines() +except Exception: raise SystemExit(1) -for login in logins: - if not isinstance(login, dict): - continue - if str(login.get("name") or "") == wanted: - token = login.get("token") - if isinstance(token, str) and token: - print(token) - raise SystemExit(0) - break - +if isinstance(token, str) and token: + print(token) + raise SystemExit(0) raise SystemExit(1) PY } diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index f225844b..08fd8738 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -97,12 +97,12 @@ gitea_resolve_api_for_login() { host=$(get_remote_host) if [[ -n "$override_explicit" ]]; then - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") || { - echo "Error: could not resolve a Gitea token for --login '$effective_login'; refusing to fall back to the host default identity (comment write/read-back)" >&2 + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login" "$host") || { + echo "Error: could not resolve a host-matched Gitea token for --login '$effective_login' on host '$host'; refusing to fall back to the host default identity or a cross-host credential (comment write/read-back)" >&2 return 1 } else - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login" "$host") \ || GITEA_API_TOKEN=$(get_gitea_token "$host") || { echo "Error: Gitea token not found for login '$effective_login' (comment write/read-back)" >&2 return 1 @@ -121,63 +121,6 @@ gitea_resolve_api_for_login() { return 0 } -# Fetch every page of a Gitea list endpoint into $2 (merged into one JSON -# array). Gitea paginates list responses, so a single-page read would -# false-negative once a newly created record lands beyond page 1. Walks -# page=1,2,… until a short page (fewer than the requested limit) or an empty -# page is returned, so the merged array is exhaustive. $1 is the endpoint URL -# with NO query string. Returns non-zero (clear stderr) on any transport / -# HTTP / parse failure. -gitea_fetch_all() { - local base_url="$1" dest="$2" page=1 limit=50 status page_file count - - printf '[]' > "$dest" - while :; do - page_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-page.XXXXXX") - if ! status=$(curl -sS -o "$page_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ - "${base_url}?limit=${limit}&page=${page}"); then - rm -f "$page_file" - echo "Error: Gitea list read transport failed" >&2 - return 1 - fi - if [[ "$status" != "200" ]]; then - rm -f "$page_file" - echo "Error: Gitea list read failed with HTTP $status" >&2 - return 1 - fi - count=$(DEST="$dest" python3 - "$page_file" <<'PY' -import json -import os -import sys - -try: - with open(os.environ["DEST"], encoding="utf-8") as merged_file: - merged = json.load(merged_file) - with open(sys.argv[1], encoding="utf-8") as page_file: - page = json.load(page_file) - if not isinstance(page, list): - raise ValueError("page response is not a list") - merged.extend(item for item in page if isinstance(item, dict)) - with open(os.environ["DEST"], "w", encoding="utf-8") as merged_file: - json.dump(merged, merged_file) -except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: - print(f"Error: could not merge Gitea list page: {error}", file=sys.stderr) - raise SystemExit(1) -print(len(page)) -PY -) || { rm -f "$page_file"; return 1; } - rm -f "$page_file" - [[ "$count" -lt "$limit" ]] && break - page=$((page + 1)) - if [[ "$page" -gt 1000 ]]; then - echo "Error: Gitea list pagination exceeded 1000 pages" >&2 - return 1 - fi - done - return 0 -} - # Resolve the login of the identity the API token authenticates as (GET # /user). Used to attribute a read-back record to THIS invocation's writer so # a concurrent write from a DIFFERENT identity cannot satisfy verification. @@ -216,54 +159,6 @@ print(login) PY } -# Confirm that the comment CREATED by this invocation ($2 = its provider id) is -# enumerable in the issue's full, paginated comment listing and is authored by -# the acting identity. Gitea paginates list responses, so a comment created -# beyond page 1 must still be found; walking every page also proves the created -# id is durably indexed against THIS issue rather than merely retrievable by id. -# Returns non-zero (clear stderr) if the exact created id is not present with a -# matching author. -gitea_confirm_comment_enumerable() { - local issue_number="$1" created_id="$2" acting_login="$3" merged_file - - merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-readback.XXXXXX") - trap 'rm -f "$merged_file"' RETURN - - gitea_fetch_all "$GITEA_API_BASE/issues/$issue_number/comments" "$merged_file" || return 1 - - CREATED_COMMENT_ID="$created_id" ACTING_LOGIN="$acting_login" \ - python3 - "$merged_file" <<'PY' -import json -import os -import sys - -try: - with open(sys.argv[1], encoding="utf-8") as response: - comments = json.load(response) - if not isinstance(comments, list): - raise ValueError("response is not a comment list") - created_id = int(os.environ["CREATED_COMMENT_ID"]) - acting_login = os.environ["ACTING_LOGIN"] - match = next( - ( - c for c in comments - if isinstance(c, dict) - and c.get("id") == created_id - and (c.get("user") or {}).get("login") == acting_login - ), - None, - ) - if match is None: - raise ValueError( - f"created comment id {created_id} is not enumerable in the issue's " - "paginated comment list under the acting identity" - ) -except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: - print(f"Error: Gitea comment enumeration check failed: {error}", file=sys.stderr) - raise SystemExit(1) -PY -} - # Post a comment to a Gitea issue via the supported REST API and verify it # durably against a PROVIDER-RETURNED created id — never trust an exit code # (#865 defect class: tea's non-existent `tea issue comment` no-ops yet exits @@ -335,7 +230,7 @@ PY EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ - EXPECTED_ISSUE_NUMBER="$issue_number" \ + EXPECTED_NUMBER="$issue_number" \ python3 - "$readback_file" <<'PY' || return 1 import json import os @@ -350,26 +245,31 @@ try: expected_id = int(os.environ["EXPECTED_COMMENT_ID"]) expected_body = os.environ["EXPECTED_COMMENT_BODY"] acting_login = os.environ["ACTING_LOGIN"] - expected_suffix = ( - f"/repos/{os.environ['EXPECTED_REPO_SLUG']}" - f"/issues/{os.environ['EXPECTED_ISSUE_NUMBER']}" - ) - issue_path = urlparse(comment.get("issue_url", "")).path.rstrip("/") + slug = os.environ["EXPECTED_REPO_SLUG"] + number = os.environ["EXPECTED_NUMBER"] + # Gitea populates WEB (html) URLs here, not API paths. A plain issue comment + # carries issue_url = ///issues/ (pull_request_url + # empty); a comment posted to a PR's conversation carries + # pull_request_url = ///pulls/ (issue_url empty). + # Accept whichever the provider populated — scoped to THIS repo slug and + # number — so a genuine write is never rejected merely for URL shape. + issue_suffix = f"/{slug}/issues/{number}" + pr_suffix = f"/{slug}/pulls/{number}" + issue_path = urlparse(comment.get("issue_url") or "").path.rstrip("/") + pr_path = urlparse(comment.get("pull_request_url") or "").path.rstrip("/") if comment.get("id") != expected_id: raise ValueError("read-back id does not match the created id") if (comment.get("user") or {}).get("login") != acting_login: raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: raise ValueError("created comment body does not match") - if not issue_path.endswith(expected_suffix): + if not (issue_path.endswith(issue_suffix) or pr_path.endswith(pr_suffix)): raise ValueError("created comment does not belong to this issue") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) PY - gitea_confirm_comment_enumerable "$issue_number" "$created_id" "$acting_login" || return 1 - echo "$created_id" return 0 } diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index 555c6e6a..4c7bddb3 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -146,7 +146,7 @@ PY EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ - EXPECTED_PR_NUMBER="$pr_number" \ + EXPECTED_NUMBER="$pr_number" \ python3 - "$readback_file" <<'PY' || return 1 import json import os @@ -161,18 +161,25 @@ try: expected_id = int(os.environ["EXPECTED_COMMENT_ID"]) expected_body = os.environ["EXPECTED_COMMENT_BODY"] acting_login = os.environ["ACTING_LOGIN"] - expected_suffix = ( - f"/repos/{os.environ['EXPECTED_REPO_SLUG']}" - f"/issues/{os.environ['EXPECTED_PR_NUMBER']}" - ) - issue_path = urlparse(comment.get("issue_url", "")).path.rstrip("/") + slug = os.environ["EXPECTED_REPO_SLUG"] + number = os.environ["EXPECTED_NUMBER"] + # Gitea populates WEB (html) URLs here, not API paths. A PR-conversation + # comment carries pull_request_url = ///pulls/ (with + # issue_url empty), while a plain issue comment carries + # issue_url = ///issues/ (with pull_request_url empty). + # Accept whichever the provider populated — scoped to THIS repo slug and + # number — so a genuine write is never rejected merely for URL shape. + issue_suffix = f"/{slug}/issues/{number}" + pr_suffix = f"/{slug}/pulls/{number}" + issue_path = urlparse(comment.get("issue_url") or "").path.rstrip("/") + pr_path = urlparse(comment.get("pull_request_url") or "").path.rstrip("/") if comment.get("id") != expected_id: raise ValueError("read-back id does not match the created id") if (comment.get("user") or {}).get("login") != acting_login: raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: raise ValueError("created comment body does not match") - if not issue_path.endswith(expected_suffix): + if not (issue_path.endswith(issue_suffix) or pr_path.endswith(pr_suffix)): raise ValueError("created comment does not belong to this PR") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) @@ -203,12 +210,12 @@ gitea_resolve_api_for_login() { host=$(get_remote_host) if [[ -n "$override_explicit" ]]; then - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") || { - echo "Error: could not resolve a Gitea token for --login '$effective_login'; refusing to fall back to the host default identity (review write/read-back)" >&2 + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login" "$host") || { + echo "Error: could not resolve a host-matched Gitea token for --login '$effective_login' on host '$host'; refusing to fall back to the host default identity or a cross-host credential (review write/read-back)" >&2 return 1 } else - GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login") \ + GITEA_API_TOKEN=$(get_gitea_token_for_login "$effective_login" "$host") \ || GITEA_API_TOKEN=$(get_gitea_token "$host") || { echo "Error: Gitea token not found for login '$effective_login' (review write/read-back)" >&2 return 1 @@ -227,62 +234,6 @@ gitea_resolve_api_for_login() { return 0 } -# Fetch every page of a Gitea list endpoint into $2 (merged into one JSON -# array). Gitea paginates list responses, so a single-page read would -# false-negative once a newly created review/comment lands beyond page 1. -# Walks page=1,2,… until a short or empty page is returned so the merged array -# is exhaustive. $1 is the endpoint URL with NO query string. Returns non-zero -# (clear stderr) on any transport / HTTP / parse failure. -gitea_fetch_all() { - local base_url="$1" dest="$2" page=1 limit=50 status page_file count - - printf '[]' > "$dest" - while :; do - page_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-page.XXXXXX") - if ! status=$(curl -sS -o "$page_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ - "${base_url}?limit=${limit}&page=${page}"); then - rm -f "$page_file" - echo "Error: Gitea list read transport failed" >&2 - return 1 - fi - if [[ "$status" != "200" ]]; then - rm -f "$page_file" - echo "Error: Gitea list read failed with HTTP $status" >&2 - return 1 - fi - count=$(DEST="$dest" python3 - "$page_file" <<'PY' -import json -import os -import sys - -try: - with open(os.environ["DEST"], encoding="utf-8") as merged_file: - merged = json.load(merged_file) - with open(sys.argv[1], encoding="utf-8") as page_file: - page = json.load(page_file) - if not isinstance(page, list): - raise ValueError("page response is not a list") - merged.extend(item for item in page if isinstance(item, dict)) - with open(os.environ["DEST"], "w", encoding="utf-8") as merged_file: - json.dump(merged, merged_file) -except (OSError, json.JSONDecodeError, TypeError, ValueError) as error: - print(f"Error: could not merge Gitea list page: {error}", file=sys.stderr) - raise SystemExit(1) -print(len(page)) -PY -) || { rm -f "$page_file"; return 1; } - rm -f "$page_file" - [[ "$count" -lt "$limit" ]] && break - page=$((page + 1)) - if [[ "$page" -gt 1000 ]]; then - echo "Error: Gitea list pagination exceeded 1000 pages" >&2 - return 1 - fi - done - return 0 -} - # Resolve the login of the identity the API token authenticates as (GET # /user). Used to attribute a read-back review to THIS action's reviewer so a # concurrent review from a DIFFERENT identity cannot satisfy verification. @@ -358,56 +309,6 @@ print(head_sha) PY } -# Confirm that the review CREATED by this action ($2 = its provider id) is -# enumerable in the PR's full, paginated review listing, authored by the acting -# identity, in the expected state. Gitea paginates review lists, so a review -# created beyond page 1 must still be found; walking every page also proves the -# created id is durably indexed against THIS PR rather than merely retrievable -# by id. Returns non-zero (clear stderr) if the exact created id is absent or -# does not match author/state. -gitea_confirm_review_enumerable() { - local pr_number="$1" created_id="$2" expected_state="$3" acting_login="$4" merged_file - - merged_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-list.XXXXXX") - trap 'rm -f "$merged_file"' RETURN - - gitea_fetch_all "$GITEA_API_BASE/pulls/$pr_number/reviews" "$merged_file" || return 1 - - CREATED_REVIEW_ID="$created_id" EXPECTED_STATE="$expected_state" ACTING_LOGIN="$acting_login" \ - python3 - "$merged_file" <<'PY' -import json -import os -import sys - -try: - with open(sys.argv[1], encoding="utf-8") as response: - reviews = json.load(response) - if not isinstance(reviews, list): - raise ValueError("response is not a review list") - created_id = int(os.environ["CREATED_REVIEW_ID"]) - expected_state = os.environ["EXPECTED_STATE"] - acting_login = os.environ["ACTING_LOGIN"] - match = next( - ( - r for r in reviews - if isinstance(r, dict) - and r.get("id") == created_id - and (r.get("user") or {}).get("login") == acting_login - and r.get("state") == expected_state - ), - None, - ) - if match is None: - raise ValueError( - f"created review id {created_id} is not enumerable in the PR's " - "paginated review list under the acting identity/state" - ) -except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: - print(f"Error: Gitea review enumeration check failed: {error}", file=sys.stderr) - raise SystemExit(1) -PY -} - # Submit a review to a Gitea PR via the supported REST API and verify it against # a PROVIDER-RETURNED created id. tea 0.11.1's `pr approve`/`reject` cannot emit # the id of the review it created and can silently no-op while exiting 0 (#865 @@ -513,8 +414,6 @@ except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error raise SystemExit(1) PY - gitea_confirm_review_enumerable "$pr_number" "$created_id" "$event" "$acting_login" || return 1 - echo "$created_id" return 0 } diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index 6eb3bdfe..a232de43 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -26,15 +26,22 @@ # concurrency window — no fallback list scan can rescue a no-op); # 5. fails closed when the created record is not authored by the acting # identity; -# 6. enumerates the created id in the issue's FULLY PAGINATED comment list, -# finding it even when it lands beyond page 1; +# 6. treats the exact-id GET as the SOLE authority — it performs NO follow-up +# list enumeration (the stub exposes no comment-list endpoint, so any +# residual enumeration attempt would fail the run); # 7. with a RESOLVABLE --login override, performs the write, the /user identity # lookup, and the read-back ALL under THAT login's token/identity — never # the host default; # 8. with an UNRESOLVABLE --login override, FAILS CLOSED (nonzero, no write, no # success line) instead of silently downgrading to the host default # identity — the token seam maps each bearer token to the identity it -# authenticates as, so a misattributed write is caught. +# authenticates as, so a misattributed write is caught; +# 9. with a --login override whose tea config URL is a DIFFERENT host than the +# repo remote, FAILS CLOSED (host-bound token selection) rather than sending +# that other host's credential cross-host; +# 10. leaves NO temp files behind (POST/GET bodies + metadata) on either the +# success or the failure path — nested function-scoped RETURN traps do not +# clobber each other and every scratch file is removed on all exit paths. set -euo pipefail @@ -49,13 +56,16 @@ AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" STATE_FILE="$WORK_DIR/comments.json" +# A dedicated scratch dir the wrapper is pointed at via TMPDIR, so the leak +# check can assert every POST/GET body + metadata temp file is cleaned up. +TMP_SCRATCH="$WORK_DIR/scratch" cleanup() { rm -rf "$WORK_DIR" } trap cleanup EXIT -mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" +mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$TMP_SCRATCH" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git @@ -71,13 +81,22 @@ FOREIGN_LOGIN="other-writer" OVERRIDE_LOGIN="delegated-reviewer" DEFAULT_TOKEN="test-only-placeholder" OVERRIDE_TOKEN="override-token-placeholder" +# A --login override whose tea config URL points at a DIFFERENT Gitea host than +# the repo remote (git.mosaicstack.dev). Its token must NEVER be sent to the +# repo host: host-bound selection must fail closed on the host mismatch. +CROSS_HOST_LOGIN="foreign-host-reviewer" +CROSS_HOST_TOKEN="cross-host-token-placeholder" # tea config: the override login has its own token here (as tea itself stores # per-login tokens). The default login name ("mosaicstack") is deliberately NOT # present, so the no-override default path resolves via the host credential # fallback while an explicit --login must resolve from this file or fail closed. +# A second login is configured for a DIFFERENT host to exercise host-bound +# rejection. mkdir -p "$XDG_DIR/tea" -OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" python3 - "$XDG_DIR/tea/config.yml" <<'PY' +OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ + CROSS_HOST_LOGIN="$CROSS_HOST_LOGIN" CROSS_HOST_TOKEN="$CROSS_HOST_TOKEN" \ + python3 - "$XDG_DIR/tea/config.yml" <<'PY' import os import sys @@ -86,6 +105,9 @@ with open(sys.argv[1], "w", encoding="utf-8") as handle: handle.write(f" - name: {os.environ['OVERRIDE_LOGIN']}\n") handle.write(" url: https://git.mosaicstack.dev\n") handle.write(f" token: {os.environ['OVERRIDE_TOKEN']}\n") + handle.write(f" - name: {os.environ['CROSS_HOST_LOGIN']}\n") + handle.write(" url: https://git.uscllc.com\n") + handle.write(f" token: {os.environ['CROSS_HOST_TOKEN']}\n") PY CONFIGURED_GITEA_URL="https://git.mosaicstack.dev" python3 - "$CREDENTIALS_FILE" <<'PY' @@ -123,10 +145,12 @@ SH chmod +x "$BIN_DIR/tea" # curl stub: a small REST server backed by persistent on-disk comment state. -# GET /user -> acting identity -# POST /issues/7/comments -> CREATE + PERSIST, return created object -# GET /issues/comments/{id} -> read the persisted record by exact id -# GET /issues/7/comments?page=&.. -> paginated listing of persisted state +# GET /user -> acting identity +# POST /issues/7/comments -> CREATE + PERSIST, return created object +# GET /issues/comments/{id} -> read the persisted record by exact id +# There is deliberately NO comment-LIST endpoint: exact-id read-back is the sole +# authority, so any residual list enumeration attempt hits the unexpected-request +# guard and fails the test. cat > "$BIN_DIR/curl" <<'SH' #!/usr/bin/env bash set -euo pipefail @@ -164,6 +188,7 @@ acting_identity="" case "$auth_token" in "$ISSUE_COMMENT_DEFAULT_TOKEN") acting_identity="$ISSUE_COMMENT_ACTING_LOGIN" ;; "$ISSUE_COMMENT_OVERRIDE_TOKEN") acting_identity="$ISSUE_COMMENT_OVERRIDE_LOGIN" ;; + "$ISSUE_COMMENT_CROSS_HOST_TOKEN") acting_identity="$ISSUE_COMMENT_CROSS_HOST_LOGIN" ;; esac printf '%s %s %s\n' "$method" "$path" "${acting_identity:-}" >> "$ISSUE_COMMENT_AUTH_LOG" @@ -211,7 +236,10 @@ record = { "id": new_id, "body": body, "user": {"login": author}, - "issue_url": f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7", + # REAL Gitea comment shape: issue_url is the WEB (html) path, not an API + # path, and a plain issue comment leaves pull_request_url empty. + "issue_url": f"https://git.mosaicstack.dev/{repo}/issues/7", + "pull_request_url": "", } comments.append(record) with open(state_path, "w", encoding="utf-8") as handle: @@ -238,24 +266,6 @@ else: print("200") print(json.dumps(match)) PY -) - write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" -elif [[ "$method" == "GET" && "$path" == "$ISSUE_COMMENT_API_BASE/issues/7/comments" ]]; then - result=$(ISSUE_COMMENT_QUERY="$query" python3 - <<'PY' -import json -import os -from urllib.parse import parse_qs - -state_path = os.environ["ISSUE_COMMENT_STATE"] -params = parse_qs(os.environ["ISSUE_COMMENT_QUERY"]) -limit = int(params.get("limit", ["50"])[0]) -page = int(params.get("page", ["1"])[0]) -with open(state_path, encoding="utf-8") as handle: - comments = json.load(handle) -start = (page - 1) * limit -print("200") -print(json.dumps(comments[start:start + limit])) -PY ) write_response "$(printf '%s' "$result" | head -n1)" "$(printf '%s' "$result" | tail -n +2)" else @@ -279,23 +289,30 @@ mode = os.environ["ISSUE_COMMENT_SEED_MODE"] body = os.environ["ISSUE_COMMENT_SEED_BODY"] acting = os.environ["ISSUE_COMMENT_SEED_ACTING"] repo = os.environ["ISSUE_COMMENT_SEED_REPO"] -issue_url = f"https://git.mosaicstack.dev/api/v1/repos/{repo}/issues/7" +# REAL Gitea comment shape: issue_url is the WEB path, pull_request_url empty. +issue_url = f"https://git.mosaicstack.dev/{repo}/issues/7" + + +def comment(cid, text, author): + return { + "id": cid, + "body": text, + "user": {"login": author}, + "issue_url": issue_url, + "pull_request_url": "", + } + if mode == "fresh-success": - # 50 pre-existing comments fill page 1 (limit 50); the comment this run - # creates becomes id 51 and lands ALONE on page 2, exercising >page-1 - # pagination in the enumeration check. - comments = [ - {"id": i, "body": f"prior {i}", "user": {"login": acting}, "issue_url": issue_url} - for i in range(1, 51) - ] + # 50 pre-existing comments already exist; the comment this run creates + # becomes id 51, proving exact-id read-back works regardless of how many + # comments precede it (no list enumeration is involved). + comments = [comment(i, f"prior {i}", acting) for i in range(1, 51)] elif mode == "no-op-concurrent": # A concurrent SAME-IDENTITY comment with the IDENTICAL body already exists. # The wrapper's own write will be a no-op; it must still fail closed because # no created id is returned — it must not scan and accept this record. - comments = [ - {"id": 55, "body": body, "user": {"login": acting}, "issue_url": issue_url} - ] + comments = [comment(55, body, acting)] else: # author-mismatch comments = [] @@ -315,6 +332,7 @@ run_comment() { ( cd "$REPO_DIR" PATH="$BIN_DIR:$PATH" \ + TMPDIR="$TMP_SCRATCH" \ XDG_CONFIG_HOME="$XDG_DIR" \ MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \ @@ -325,8 +343,10 @@ run_comment() { ISSUE_COMMENT_ACTING_LOGIN="$ACTING_LOGIN" \ ISSUE_COMMENT_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ ISSUE_COMMENT_OVERRIDE_LOGIN="$OVERRIDE_LOGIN" \ + ISSUE_COMMENT_CROSS_HOST_LOGIN="$CROSS_HOST_LOGIN" \ ISSUE_COMMENT_DEFAULT_TOKEN="$DEFAULT_TOKEN" \ ISSUE_COMMENT_OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ + ISSUE_COMMENT_CROSS_HOST_TOKEN="$CROSS_HOST_TOKEN" \ ISSUE_COMMENT_REPO_SLUG="$REPO_SLUG" \ ISSUE_COMMENT_API_BASE="$API_BASE" \ ISSUE_COMMENT_API_ROOT="$API_ROOT" \ @@ -334,8 +354,21 @@ run_comment() { ) > "$OUTPUT_FILE" 2>&1 } +# Assert the wrapper left no scratch temp files behind in TMPDIR (POST/GET +# request bodies + metadata). Called after both success and failure paths so a +# clobbered/leaked RETURN trap is caught on every exit route. +assert_no_temp_leak() { + local context="$1" leaked + leaked=$(find "$TMP_SCRATCH" -type f -name 'mosaic-issue-comment-*' 2>/dev/null || true) + if [[ -n "$leaked" ]]; then + echo "FAIL: issue-comment temp files leaked ($context):" >&2 + printf '%s\n' "$leaked" >&2 + exit 1 + fi +} + # Case 1: a genuine REST create (id 51) is verified end to end via its exact -# provider-returned id and enumerated on page 2 of the paginated listing. +# provider-returned id — no list enumeration is involved. run_comment fresh-success grep -q 'Added and verified comment on Gitea issue #7 (comment ID 51)' "$OUTPUT_FILE" # The write is a REST POST, never a tea comment. @@ -348,12 +381,17 @@ fi grep -q "^GET $API_BASE/issues/comments/51$" "$CURL_LOG" # Acting identity resolved via GET /user. grep -q "^GET $API_ROOT/user$" "$CURL_LOG" -# Enumeration paginated beyond page 1 to find the created comment. -grep -q "^GET $API_BASE/issues/7/comments?limit=[0-9]*&page=2$" "$CURL_LOG" +# No comment-list enumeration is performed — the exact-id GET is authoritative. +if grep -Eq "^GET $API_BASE/issues/7/comments(\?|$)" "$CURL_LOG"; then + echo "FAIL: wrapper performed a redundant comment-list enumeration" >&2 + exit 1 +fi # Default path (no --login): the host credential fallback resolves, and the # write is performed AND self-verified under the host-default acting identity. grep -q "^POST $API_BASE/issues/7/comments $ACTING_LOGIN$" "$AUTH_LOG" grep -q "^GET $API_BASE/issues/comments/51 $ACTING_LOGIN$" "$AUTH_LOG" +# Success path leaves no scratch temp files behind. +assert_no_temp_leak "fresh-success" # Case 2: a no-op write with a concurrent SAME-IDENTITY, same-body comment # already present must FAIL CLOSED — the closed concurrency window. @@ -382,6 +420,9 @@ if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then echo "FAIL: read-back did not enforce acting-identity authorship" >&2 exit 1 fi +# Failure-after-read-back path must ALSO leave no scratch temp files behind +# (proves the RETURN traps clean up on the error-return route, not just success). +assert_no_temp_leak "author-mismatch" # Case 4: a RESOLVABLE --login override — the write, the /user identity lookup, # and the read-back must ALL be performed under THAT login's token/identity, not @@ -420,4 +461,34 @@ if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then exit 1 fi +# Case 6: a --login override that IS present in tea config but whose URL is a +# DIFFERENT host than the repo remote must FAIL CLOSED (host-bound selection). +# The cross-host token must NEVER be sent to the repo host, and no write occurs. +if run_comment cross-host --login "$CROSS_HOST_LOGIN"; then + echo "FAIL: cross-host --login override did not fail closed" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: cross-host --login override reported success" >&2 + exit 1 +fi +# The cross-host credential must not have performed ANY request against the repo +# host — no request may be attributed to the cross-host identity. +if grep -q " $CROSS_HOST_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: cross-host credential was sent to the repo host (cross-host leak)" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +if grep -q "^POST $API_BASE/issues/7/comments" "$CURL_LOG"; then + echo "FAIL: cross-host --login override still performed a write" >&2 + exit 1 +fi +# It must not have silently downgraded to the host default identity either. +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: cross-host --login override fell back to the host default identity" >&2 + exit 1 +fi +assert_no_temp_leak "cross-host" + echo "issue-comment.sh REST create + exact-id read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index 3108235e..bd2e03de 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -26,6 +26,13 @@ # CLOSED (unresolvable case) — never silently downgrade to the host-default # identity. The host-default best-effort fallback is reserved for the # no-override default path. +# +# #865 Round-5: the exact-id read-back is the SOLE authority — the wrapper does +# NO follow-up list enumeration (the stub exposes no review/comment list +# endpoint, so a residual enumeration would fail the run). A --login override is +# host-bound: an override configured for a DIFFERENT host than the repo remote +# FAILS CLOSED rather than leaking a cross-host credential. And every run leaves +# no scratch temp files behind on any exit path (POST/GET bodies + metadata). set -euo pipefail @@ -43,6 +50,9 @@ CURL_LOG="$WORK_DIR/curl.log" AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" +# A dedicated scratch dir the wrapper is pointed at via TMPDIR, so the leak +# check can assert every POST/GET body + metadata temp file is cleaned up. +TMP_SCRATCH="$WORK_DIR/scratch" cleanup() { rm -rf "$WORK_DIR" @@ -57,17 +67,25 @@ HEAD_SHA="HEADSHA_FEEDFACE" OVERRIDE_LOGIN="primary-reviewer" DEFAULT_TOKEN="test-only-placeholder" OVERRIDE_TOKEN="override-token-placeholder" +# A --login override whose tea config URL points at a DIFFERENT Gitea host than +# the repo remote (git.mosaicstack.dev). Host-bound selection must reject it +# rather than send its token cross-host. +CROSS_HOST_LOGIN="foreign-host-reviewer" +CROSS_HOST_TOKEN="cross-host-token-placeholder" -mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$STATE_DIR" +mkdir -p "$REPO_DIR" "$BIN_DIR" "$XDG_DIR" "$STATE_DIR" "$TMP_SCRATCH" git -C "$REPO_DIR" init -q git -C "$REPO_DIR" remote add origin https://git.mosaicstack.dev/mosaicstack/stack.git # tea config: the override login carries its own token here. The default login # name ("mosaicstack") is deliberately absent, so the no-override default path # resolves via the host credential fallback while an explicit --login must -# resolve from this file or fail closed. +# resolve from this file or fail closed. A second login is configured for a +# DIFFERENT host to exercise host-bound rejection. mkdir -p "$XDG_DIR/tea" -OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" python3 - "$XDG_DIR/tea/config.yml" <<'PY' +OVERRIDE_LOGIN="$OVERRIDE_LOGIN" OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ + CROSS_HOST_LOGIN="$CROSS_HOST_LOGIN" CROSS_HOST_TOKEN="$CROSS_HOST_TOKEN" \ + python3 - "$XDG_DIR/tea/config.yml" <<'PY' import os import sys @@ -76,6 +94,9 @@ with open(sys.argv[1], "w", encoding="utf-8") as handle: handle.write(f" - name: {os.environ['OVERRIDE_LOGIN']}\n") handle.write(" url: https://git.mosaicstack.dev\n") handle.write(f" token: {os.environ['OVERRIDE_TOKEN']}\n") + handle.write(f" - name: {os.environ['CROSS_HOST_LOGIN']}\n") + handle.write(" url: https://git.uscllc.com\n") + handle.write(f" token: {os.environ['CROSS_HOST_TOKEN']}\n") PY write_credentials() { @@ -155,6 +176,7 @@ acting_identity="" case "$auth_token" in "$PR_REVIEW_DEFAULT_TOKEN") acting_identity="$PR_REVIEW_ACTING_LOGIN" ;; "$PR_REVIEW_OVERRIDE_TOKEN") acting_identity="$PR_REVIEW_OVERRIDE_LOGIN" ;; + "$PR_REVIEW_CROSS_HOST_TOKEN") acting_identity="$PR_REVIEW_CROSS_HOST_LOGIN" ;; esac printf '%s %s %s\n' "$method" "$path" "${acting_identity:-}" >> "$PR_REVIEW_AUTH_LOG" @@ -245,23 +267,6 @@ else: print(json.dumps(match)) PY )" -elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then - emit "$(PR_REVIEW_QUERY="$query" python3 - <<'PY' -import json -import os -from urllib.parse import parse_qs - -state_path = os.environ["PR_REVIEW_REVIEWS"] -params = parse_qs(os.environ["PR_REVIEW_QUERY"]) -limit = int(params.get("limit", ["50"])[0]) -page = int(params.get("page", ["1"])[0]) -with open(state_path, encoding="utf-8") as handle: - reviews = json.load(handle) -start = (page - 1) * limit -print("200") -print(json.dumps(reviews[start:start + limit])) -PY -)" elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/issues/123/comments" ]]; then case "$mode" in write-transport-failure) @@ -278,13 +283,18 @@ import os state_path = os.environ["PR_REVIEW_COMMENTS"] acting = os.environ["PR_REVIEW_ACTING_LOGIN"] -base = os.environ["PR_REVIEW_EXPECTED_API_BASE"] +web_base = os.environ["PR_REVIEW_WEB_BASE"] body = json.loads(os.environ["PR_REVIEW_PAYLOAD"]).get("body") +# REAL Gitea shape for a comment posted to a PR's conversation +# (/issues/{n}/comments on a PR): pull_request_url is the WEB pulls path and +# issue_url is left empty. This is what the wrapper must tolerate — it must NOT +# require an API-shaped issue_url. record = { "id": 456, "body": body, "user": {"login": acting}, - "issue_url": f"{base}/issues/123", + "issue_url": "", + "pull_request_url": f"{web_base}/pulls/123", } with open(state_path, "w", encoding="utf-8") as handle: json.dump([record], handle) @@ -344,10 +354,10 @@ def review(rid, state, commit, login): return {"id": rid, "state": state, "commit_id": commit, "user": {"login": login}} -if mode == "paginated-approve": - # 50 pre-existing reviews fill page 1 (limit 50); the review this run submits - # becomes id 51 and lands ALONE on page 2, exercising >page-1 pagination in - # the enumeration check. +if mode == "many-prior-approve": + # 50 pre-existing reviews already exist; the review this run submits becomes + # id 51, proving exact-id read-back works regardless of how many reviews + # precede it (no list enumeration is involved). reviews = [review(i, "COMMENT", "oldsha0000", acting) for i in range(1, 51)] elif mode == "no-op-concurrent-review": # A concurrent SAME-IDENTITY APPROVED review at the CURRENT head already @@ -370,6 +380,7 @@ run_review() { local login_override="${7:-}" local expected_api_base="${configured_url%/}/api/v1/repos/$expected_repo" local expected_api_root="${configured_url%/}/api/v1" + local expected_web_base="${configured_url%/}/$expected_repo" git -C "$REPO_DIR" remote set-url origin "$remote_url" write_credentials "$configured_url" : > "$TEA_LOG" @@ -380,6 +391,7 @@ run_review() { ( cd "$REPO_DIR" PATH="$BIN_DIR:$PATH" \ + TMPDIR="$TMP_SCRATCH" \ XDG_CONFIG_HOME="$XDG_DIR" \ MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ PR_REVIEW_TEA_LOG="$TEA_LOG" \ @@ -393,16 +405,32 @@ run_review() { PR_REVIEW_EXPECTED_BODY="$comment" \ PR_REVIEW_EXPECTED_API_BASE="$expected_api_base" \ PR_REVIEW_API_ROOT="$expected_api_root" \ + PR_REVIEW_WEB_BASE="$expected_web_base" \ PR_REVIEW_HEAD_SHA="$HEAD_SHA" \ PR_REVIEW_ACTING_LOGIN="$ACTING_LOGIN" \ PR_REVIEW_FOREIGN_LOGIN="$FOREIGN_LOGIN" \ PR_REVIEW_OVERRIDE_LOGIN="$OVERRIDE_LOGIN" \ + PR_REVIEW_CROSS_HOST_LOGIN="$CROSS_HOST_LOGIN" \ PR_REVIEW_DEFAULT_TOKEN="$DEFAULT_TOKEN" \ PR_REVIEW_OVERRIDE_TOKEN="$OVERRIDE_TOKEN" \ + PR_REVIEW_CROSS_HOST_TOKEN="$CROSS_HOST_TOKEN" \ "$SCRIPT_DIR/pr-review.sh" -n 123 -a "$action" ${comment:+-c "$comment"} ${login_override:+--login "$login_override"} ) > "$OUTPUT_FILE" 2>&1 } +# Assert the wrapper left no scratch temp files behind in TMPDIR (POST/GET +# request bodies + metadata). Called after both success and failure paths so a +# clobbered/leaked RETURN trap is caught on every exit route. +assert_no_temp_leak() { + local context="$1" leaked + leaked=$(find "$TMP_SCRATCH" -type f -name 'mosaic-pr-review-*' 2>/dev/null || true) + if [[ -n "$leaked" ]]; then + echo "FAIL: pr-review temp files leaked ($context):" >&2 + printf '%s\n' "$leaked" >&2 + exit 1 + fi +} + assert_no_tea_write() { # tea must only ever be used for the login list, never to write. if grep -qvE '^login list --output json$' "$TEA_LOG"; then @@ -421,12 +449,17 @@ grep -q '^GET https://git.mosaicstack.dev/api/v1/user$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG" grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews$' "$CURL_LOG" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/101$' "$CURL_LOG" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=1$' "$CURL_LOG" +# No review-list enumeration is performed — the exact-id GET is authoritative. +if grep -Eq '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews(\?|$)' "$CURL_LOG"; then + echo "FAIL: wrapper performed a redundant review-list enumeration" >&2 + exit 1 +fi # No-override default path: the write, /user lookup, and read-back all resolve # via the host-default credential and authenticate as the acting identity. grep -q "^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews $ACTING_LOGIN\$" "$AUTH_LOG" grep -q "^GET https://git.mosaicstack.dev/api/v1/user $ACTING_LOGIN\$" "$AUTH_LOG" assert_no_tea_write +assert_no_temp_leak "approve" # The submitted review payload carries the event and the PR head commit_id. PR_REVIEW_HEAD_SHA="$HEAD_SHA" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' import json @@ -454,6 +487,8 @@ if grep -q 'Approved and verified' "$OUTPUT_FILE"; then echo "FAIL: read-back did not enforce acting-identity authorship" >&2 exit 1 fi +# Failure-after-read-back path must ALSO leave no scratch temp files behind. +assert_no_temp_leak "author-mismatch-review" # Case 3: a no-op submit with a concurrent SAME-IDENTITY, same-state review at # the current head already present must FAIL CLOSED — the closed concurrency @@ -472,12 +507,16 @@ if grep -q '/pulls/123/reviews/77$' "$CURL_LOG"; then exit 1 fi -# Case 4: a genuine matching review that lands beyond page 1 of the reviews list -# must still be found by the fully-paginating enumeration check. -run_review paginated-approve approve +# Case 4: a genuine matching review (id 51) created after 50 pre-existing reviews +# is still verified by its EXACT provider-returned id — no list enumeration is +# needed regardless of how many reviews precede it. +run_review many-prior-approve approve grep -q 'Approved and verified Gitea PR #123 (review ID 51)' "$OUTPUT_FILE" grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews/51$' "$CURL_LOG" -grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews?limit=[0-9]*&page=2$' "$CURL_LOG" +if grep -Eq '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123/reviews(\?|$)' "$CURL_LOG"; then + echo "FAIL: wrapper performed a redundant review-list enumeration" >&2 + exit 1 +fi # Case 5: an approve WITH a body carries that body in the review submit itself — # there is no separate detached comment POST. @@ -522,6 +561,7 @@ grep -q '^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues grep -q '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/comments/456$' "$CURL_LOG" grep -q 'Added and verified comment on Gitea PR #123' "$OUTPUT_FILE" assert_no_tea_write +assert_no_temp_leak "comment-success" run_review http-success comment durable-body http://git.mosaicstack.dev grep -q '^POST http://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/issues/123/comments$' "$CURL_LOG" @@ -620,4 +660,37 @@ if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then exit 1 fi +# Case 10 (#865 Round-5): a --login override that IS present in tea config but +# whose URL is a DIFFERENT host than the repo remote must FAIL CLOSED (host-bound +# selection). The cross-host token must NEVER be sent to the repo host, and no +# review POST occurs. +if run_review cross-host approve "" https://git.mosaicstack.dev \ + https://git.mosaicstack.dev/mosaicstack/stack.git mosaicstack/stack "$CROSS_HOST_LOGIN"; then + echo "FAIL: cross-host --login override did not fail closed" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: cross-host --login override reported success" >&2 + exit 1 +fi +# The cross-host credential must not have performed ANY request against the repo +# host — no request may be attributed to the cross-host identity. +if grep -q " $CROSS_HOST_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: cross-host credential was sent to the repo host (cross-host leak)" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +if grep -qE '^POST .*/pulls/123/reviews ' "$AUTH_LOG"; then + echo "FAIL: cross-host --login override performed a review POST" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then + echo "FAIL: cross-host --login override fell back to the host-default identity" >&2 + cat "$AUTH_LOG" >&2 + exit 1 +fi +assert_no_temp_leak "cross-host" + echo "pr-review.sh REST review + comment create/read-back regression passed" -- 2.49.1 From 99856c55679c1ed7d65dca19af926df58994bf2f Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 21:49:59 -0500 Subject: [PATCH 07/15] fix(git-tools): scope-aware YAML fallback, port-bound creds, origin-pinned URL + review-body verification (#865) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-6 remediation for PR #866 addressing four cross-host exact-diff audit blockers (REQUEST_CHANGES governs): Blocker 1 (detect-platform.sh get_gitea_token_for_login line parser): the PyYAML-absence fallback attributed any `key: value` at any depth to the current login, so a token from a nested sub-map or a mis-indented line could be selected where PyYAML fails closed, and inline comments were not stripped. The fallback is now scope-aware — a field attaches only at the entry's own direct-field indentation, only list items at the login list's own dash indent open an entry — and _strip_scalar strips a trailing inline comment like PyYAML. It is therefore only ever MORE conservative than PyYAML, never less. Blocker 2 (detect-platform.sh host bind): credential binding compared parsed.hostname only, dropping the port, so a :9443 login satisfied a portless host and a matching :8443 login was rejected. Binding now normalizes scheme + host + effective port (scheme default applied symmetrically) exactly like gitea_url_matches_host. Blocker 3 (issue-comment.sh + pr-review.sh read-back URL check): verification used path.endswith, accepting a look-alike host or a decoy path prefix. It now pins the returned issue_url/pull_request_url ORIGIN (scheme+host+effective-port) and FULL path (deployment prefix + exact owner/repo + kind + number). A new GITEA_WEB_BASE is exported from gitea_resolve_api_for_login for this. Blocker 4 (pr-review.sh gitea_submit_review_verified): the submitted review body was not verified, so a finalized/reused pending review id carrying foreign Content passed. The persisted body is now bound to the exact submitted body. Tests: added forced-PyYAML-absence parser-equivalence fixtures (nested sub-map, sibling, mis-indent, inline comment, tab-indent fail-closed, port match/mismatch) to test-gitea-login-resolution.sh; URL-forgery fail-closed cases (wrong-host/owner/repo + prefix injection) to both write suites; and a reused-review-id body-mismatch case to the pr-review suite. Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 4 +- .../framework/tools/git/detect-platform.sh | 108 ++++++++++++--- .../framework/tools/git/issue-comment.sh | 52 +++++-- .../mosaic/framework/tools/git/pr-review.sh | 62 +++++++-- .../tools/git/test-gitea-login-resolution.sh | 128 ++++++++++++++++++ .../tools/git/test-issue-comment-readback.sh | 41 +++++- .../tools/git/test-pr-review-gitea-comment.sh | 76 ++++++++++- 7 files changed, 420 insertions(+), 51 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index 6ffaf45f..4b5e83dd 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -11,7 +11,7 @@ A successful provider write command—or a wrapper message based only on that co - Comments (`issue-comment.sh`, and the `comment` action of `pr-review.sh`) `POST /api/v1/repos/{owner}/{repo}/issues/{index}/comments`, requiring a `201` and parsing the created comment's `id` from the response body. - Reviews (`approve` / `request-changes`) `POST /api/v1/repos/{owner}/{repo}/pulls/{index}/reviews` with the `event` (`APPROVED` / `REQUEST_CHANGES`), the review `body`, and `commit_id` pinned to the PR's current head, then parse the created review's `id`. The review body travels _in the review submit itself_ — there is no separate detached comment to reconcile (a Gitea `REQUEST_CHANGES` review requires a non-empty body, which the submit carries). -**Verification keys on that exact provider-returned id.** The wrapper then `GET`s that one record directly — `GET /issues/comments/{id}` or `GET /pulls/{n}/reviews/{id}` — and requires that its `id` equals the created id, its **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), and, for comments, its body exactly matches what was submitted, or, for reviews, its state matches the requested action and its reviewed `commit_id` equals the PR head. The write, the `/user` identity lookup, and the read-back all use the **same** credential — the effective login's token, or the host credential when no login is named — so the write is verified against the identity that actually performed it. +**Verification keys on that exact provider-returned id.** The wrapper then `GET`s that one record directly — `GET /issues/comments/{id}` or `GET /pulls/{n}/reviews/{id}` — and requires that its `id` equals the created id, its **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), and, for comments, its body exactly matches what was submitted **and its returned web URL belongs to this exact provider and repository** (the `issue_url` / `pull_request_url` origin — scheme, host, and effective port — and full path, i.e. deployment prefix + exact `owner/repo` + kind + number, must match; a suffix/`endsWith` test would accept a look-alike host or a decoy path prefix, so the whole normalized URL is compared), or, for reviews, its state matches the requested action, its reviewed `commit_id` equals the PR head, **and its persisted body equals the submitted body** (Gitea can finalize/reuse a pending review id whose stored content was authored elsewhere, so the body is bound too). The write, the `/user` identity lookup, and the read-back all use the **same** credential — the effective login's token, or the host credential when no login is named — so the write is verified against the identity that actually performed it. **This closes the concurrency window rather than documenting it.** Because verification keys on the id the create returned, a no-op create yields no id and fails closed with no list-scan fallback, and a _concurrent_ record — even one written by the _same_ identity with an identical body/state — has a _different_ id and cannot be mistaken for this write. There is no residual same-identity window: the earlier boundary-and-author heuristic (accept any `id > pre-write-max` with a matching author) is replaced entirely by exact-id attribution. @@ -24,6 +24,6 @@ A successful provider write command—or a wrapper message based only on that co ### `--login` override -Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea login for that single invocation. The override selects **which credential the REST write, the `/user` identity lookup, and the read-back all use** — its token is resolved from the tea config for that login name (`get_gitea_token_for_login`), falling back to the repo host's credential when no login is named. The resolved login is **host-bound**: the login's configured URL host must match the repo remote's host, so a login name shared across hosts (or an override configured for a different Gitea) can never send one host's credential to another — a host mismatch fails closed rather than leaking a cross-host token. Resolving the acting identity and the read-back from the _same_ login that performs the write is essential: a write performed under an overridden login must be verified against that login's identity, not the host default's. Callers who need a different login than the host default should pass `--login `. +Both `pr-review.sh` and `issue-comment.sh` accept an optional `--login ` flag that overrides the automatically detected Gitea login for that single invocation. The override selects **which credential the REST write, the `/user` identity lookup, and the read-back all use** — its token is resolved from the tea config for that login name (`get_gitea_token_for_login`), falling back to the repo host's credential when no login is named. The resolved login is **host- and port-bound**: the login's configured URL host **and effective port** (the scheme's default port — 80 for `http`, 443 for `https` — applies when a port is omitted, symmetrically on both sides) must match the repo remote's, so a login name shared across hosts (or an override configured for a different Gitea, including one on a different port of the same host) can never send one host's credential to another — a host or port mismatch fails closed rather than leaking a cross-host token. Resolving the acting identity and the read-back from the _same_ login that performs the write is essential: a write performed under an overridden login must be verified against that login's identity, not the host default's. Callers who need a different login than the host default should pass `--login `. As a durable successor to this mechanism, consider giving each reviewer/approver slot its own dedicated Gitea login credential, so that author≠reviewer holds at the credential level rather than relying on wrapper-level `--login` bookkeeping. This is a recommendation for future hardening, not something implemented by this flag. diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 2e0a5194..db5b6db8 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -599,29 +599,61 @@ config_path = sys.argv[1] def _strip_scalar(value): + # Resolve a YAML flow scalar the way PyYAML would for tea's simple scalars: + # honor surrounding quotes and strip a trailing inline comment. A quoted + # scalar keeps its literal contents (any '#' inside is data, not a comment); + # an unquoted scalar ends at the first whitespace-preceded '#' (a YAML + # comment must be preceded by whitespace or line start), so "abc#def" stays + # literal while "abc # note" becomes "abc". value = value.strip() - if len(value) >= 2 and value[0] == value[-1] and value[0] in ("'", '"'): - value = value[1:-1] - return value + if not value: + return value + if value[0] in ("'", '"'): + quote = value[0] + end = value.find(quote, 1) + if end != -1: + return value[1:end] + # Unterminated quote: PyYAML would error; return best-effort remainder so + # the (more conservative) caller still compares against the wanted name. + return value[1:] + for index, char in enumerate(value): + if char == "#" and (index == 0 or value[index - 1] in (" ", "\t")): + value = value[:index] + break + return value.strip() -def _host_of(url): +def _url_matches_repo_host(url): + # Mirror gitea_url_matches_host (detect-platform.sh): the login's recorded + # URL must name the SAME host AND the SAME effective port as the repo remote + # host, not merely the same hostname. A login configured for an explicit, + # non-default provider port (e.g. :9443) must NOT satisfy a portless (default + # -port) repo host, and a login on the matching port (e.g. :8443) must NOT be + # rejected. Ports are normalized by applying the login URL's scheme default + # (80 for http, else 443) to whichever side omits the port, symmetrically, so + # an implicit port and its explicit default-port form compare equal. if not isinstance(url, str) or not url: - return None - parsed = urlparse(url if "//" in url else f"//{url}") - host = parsed.hostname - return host.lower() if host else None + return False + configured = urlparse(url if "//" in url else f"//{url}") + remote = urlparse(f"//{repo_host}") + configured_host = configured.hostname + if not configured_host or configured_host.lower() != (remote.hostname or "").lower(): + return False + default_port = 80 if configured.scheme == "http" else 443 + configured_port = configured.port if configured.port is not None else default_port + remote_port = remote.port if remote.port is not None else default_port + return configured_port == remote_port def _accept(token, url): # Enforce the host bind before surfacing a token. When a repo host is given, - # the login's recorded URL host must match it exactly; a login with no - # usable URL (or a mismatched one) is rejected (fail closed) so a cross-host - # credential is never emitted. + # the login's recorded URL host AND port must match it; a login with no + # usable URL (or a mismatched host/port) is rejected (fail closed) so a + # cross-host (or cross-port) credential is never emitted. if not isinstance(token, str) or not token: return None if repo_host: - if _host_of(url) != repo_host: + if not _url_matches_repo_host(url): return None return token @@ -647,11 +679,17 @@ def _token_via_pyyaml(): def _token_via_lines(): # Conservative fallback for hosts without PyYAML. tea writes config.yml in a # fixed, flat shape (a `logins:` list of maps with scalar name/url/token - # fields), so a small indentation-aware scan resolves the SAME token PyYAML - # would. It only ever returns the `token` of the entry whose `name` EXACTLY - # equals the requested login AND whose url host matches the repo host, so it - # cannot misattribute to another identity or host; anything it cannot parse - # yields None (fail closed). + # fields). This scan is SCOPE-AWARE: a field is attributed to a login entry + # ONLY when it sits at that entry's own direct-field indentation. A field + # nested inside a deeper sub-map (e.g. `extra:\n token: X`) or a mis-indented + # line is NEVER attached to the entry — exactly the cases where PyYAML resolves + # the entry's own `token` to None (or errors) and thus fails closed. Likewise + # only list items at the login list's own dash indent open a new entry, so a + # nested list item cannot masquerade as a sibling login. It returns the + # `token` of the entry whose `name` EXACTLY equals the requested login AND + # whose url host/port matches the repo host; anything else yields None (fail + # closed). This can only ever be MORE conservative than PyYAML (it never + # selects a token where PyYAML would refuse), never less. with open(config_path, encoding="utf-8") as handle: lines = handle.read().splitlines() @@ -668,19 +706,47 @@ def _token_via_lines(): entries = [] current = None + item_indent = None # dash column of the login list's own items + field_indent = None # exact column of the current entry's direct fields for line in lines[start:]: if not line.strip() or line.lstrip().startswith("#"): continue indent = len(line) - len(line.lstrip(" ")) if indent <= logins_indent: break - item = re.match(r"^\s*-\s*(.*)$", line) - rest = item.group(1) if item else line + item = re.match(r"^(\s*)-(\s*)(.*)$", line) if item: + dash_indent = len(item.group(1)) + if item_indent is None: + item_indent = dash_indent + if dash_indent != item_indent: + # A more-deeply-indented (nested) or dedented list item: not a + # direct login entry. Ignore it and its scope. + continue current = {} entries.append(current) - pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", rest.strip()) - if pair and current is not None: + content = item.group(3) + if content: + # First field shares this line; its column is the direct-field + # indent for the rest of the entry. + field_indent = dash_indent + 1 + len(item.group(2)) + pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", content) + if pair: + current[pair.group(1)] = _strip_scalar(pair.group(2)) + else: + # Bare "-": the first following field line establishes the indent. + field_indent = None + continue + if current is None: + continue + if field_indent is None: + field_indent = indent + if indent != field_indent: + # Deeper => a nested sub-map's field (not this entry's own); anything + # else at an unexpected column is not a direct field. Skip either way. + continue + pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", line.strip()) + if pair: current[pair.group(1)] = _strip_scalar(pair.group(2)) for entry in entries: diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index 08fd8738..cbf684ad 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -118,6 +118,11 @@ gitea_resolve_api_for_login() { } GITEA_API_ROOT="${configured_url%/}/api/v1" GITEA_API_BASE="$GITEA_API_ROOT/repos/$repo" + # The provider WEB base (scheme + host + effective port + any deployment path + # prefix) that Gitea uses to build a comment's html issue_url/pull_request_url. + # Read-back verification pins the returned URL's origin + path prefix to THIS, + # not just a repo/issue suffix. + GITEA_WEB_BASE="${configured_url%/}" return 0 } @@ -230,13 +235,26 @@ PY EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ - EXPECTED_NUMBER="$issue_number" \ + EXPECTED_NUMBER="$issue_number" EXPECTED_WEB_BASE="$GITEA_WEB_BASE" \ python3 - "$readback_file" <<'PY' || return 1 import json import os import sys from urllib.parse import urlparse + +def _origin_and_path(url): + # Normalize a URL to (scheme, host, effective-port) + comment path. The port + # defaults to the scheme's default (80 http / 443 otherwise) so an implicit + # port and its explicit default form compare equal. + parsed = urlparse(url or "") + scheme = (parsed.scheme or "").lower() + host = (parsed.hostname or "").lower() + default_port = 80 if scheme == "http" else 443 + port = parsed.port if parsed.port is not None else default_port + return (scheme, host, port), parsed.path.rstrip("/") + + try: with open(sys.argv[1], encoding="utf-8") as response: comment = json.load(response) @@ -247,24 +265,36 @@ try: acting_login = os.environ["ACTING_LOGIN"] slug = os.environ["EXPECTED_REPO_SLUG"] number = os.environ["EXPECTED_NUMBER"] + web_base = os.environ["EXPECTED_WEB_BASE"] # Gitea populates WEB (html) URLs here, not API paths. A plain issue comment - # carries issue_url = ///issues/ (pull_request_url + # carries issue_url = ///issues/ (pull_request_url # empty); a comment posted to a PR's conversation carries - # pull_request_url = ///pulls/ (issue_url empty). - # Accept whichever the provider populated — scoped to THIS repo slug and - # number — so a genuine write is never rejected merely for URL shape. - issue_suffix = f"/{slug}/issues/{number}" - pr_suffix = f"/{slug}/pulls/{number}" - issue_path = urlparse(comment.get("issue_url") or "").path.rstrip("/") - pr_path = urlparse(comment.get("pull_request_url") or "").path.rstrip("/") + # pull_request_url = ///pulls/ (issue_url empty). + # Pin the returned URL's ORIGIN (scheme+host+port) and its FULL path to this + # provider + repo + kind + number — an endswith/suffix test would accept a + # look-alike host (evil.example/deceptive//issues/N) or a same-host + # decoy prefix (/other//issues/N), so compare the whole thing. + base_origin, base_path = _origin_and_path(web_base) + expected_issue_path = f"{base_path}/{slug}/issues/{number}" + expected_pr_path = f"{base_path}/{slug}/pulls/{number}" + + def _belongs(url, expected_path): + if not url: + return False + origin, path = _origin_and_path(url) + return origin == base_origin and path == expected_path + if comment.get("id") != expected_id: raise ValueError("read-back id does not match the created id") if (comment.get("user") or {}).get("login") != acting_login: raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: raise ValueError("created comment body does not match") - if not (issue_path.endswith(issue_suffix) or pr_path.endswith(pr_suffix)): - raise ValueError("created comment does not belong to this issue") + if not ( + _belongs(comment.get("issue_url"), expected_issue_path) + or _belongs(comment.get("pull_request_url"), expected_pr_path) + ): + raise ValueError("created comment does not belong to this issue on this provider/repo") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index 4c7bddb3..d652540d 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -146,13 +146,26 @@ PY EXPECTED_COMMENT_ID="$created_id" EXPECTED_COMMENT_BODY="$comment_body" \ ACTING_LOGIN="$acting_login" EXPECTED_REPO_SLUG="${GITEA_API_BASE##*/repos/}" \ - EXPECTED_NUMBER="$pr_number" \ + EXPECTED_NUMBER="$pr_number" EXPECTED_WEB_BASE="$GITEA_WEB_BASE" \ python3 - "$readback_file" <<'PY' || return 1 import json import os import sys from urllib.parse import urlparse + +def _origin_and_path(url): + # Normalize a URL to (scheme, host, effective-port) + comment path. The port + # defaults to the scheme's default (80 http / 443 otherwise) so an implicit + # port and its explicit default form compare equal. + parsed = urlparse(url or "") + scheme = (parsed.scheme or "").lower() + host = (parsed.hostname or "").lower() + default_port = 80 if scheme == "http" else 443 + port = parsed.port if parsed.port is not None else default_port + return (scheme, host, port), parsed.path.rstrip("/") + + try: with open(sys.argv[1], encoding="utf-8") as response: comment = json.load(response) @@ -163,24 +176,36 @@ try: acting_login = os.environ["ACTING_LOGIN"] slug = os.environ["EXPECTED_REPO_SLUG"] number = os.environ["EXPECTED_NUMBER"] + web_base = os.environ["EXPECTED_WEB_BASE"] # Gitea populates WEB (html) URLs here, not API paths. A PR-conversation - # comment carries pull_request_url = ///pulls/ (with + # comment carries pull_request_url = ///pulls/ (with # issue_url empty), while a plain issue comment carries - # issue_url = ///issues/ (with pull_request_url empty). - # Accept whichever the provider populated — scoped to THIS repo slug and - # number — so a genuine write is never rejected merely for URL shape. - issue_suffix = f"/{slug}/issues/{number}" - pr_suffix = f"/{slug}/pulls/{number}" - issue_path = urlparse(comment.get("issue_url") or "").path.rstrip("/") - pr_path = urlparse(comment.get("pull_request_url") or "").path.rstrip("/") + # issue_url = ///issues/ (with pull_request_url empty). + # Pin the returned URL's ORIGIN (scheme+host+port) and its FULL path to this + # provider + repo + kind + number — an endswith/suffix test would accept a + # look-alike host (evil.example/deceptive//pulls/N) or a same-host + # decoy prefix (/other//pulls/N), so compare the whole thing. + base_origin, base_path = _origin_and_path(web_base) + expected_issue_path = f"{base_path}/{slug}/issues/{number}" + expected_pr_path = f"{base_path}/{slug}/pulls/{number}" + + def _belongs(url, expected_path): + if not url: + return False + origin, path = _origin_and_path(url) + return origin == base_origin and path == expected_path + if comment.get("id") != expected_id: raise ValueError("read-back id does not match the created id") if (comment.get("user") or {}).get("login") != acting_login: raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: raise ValueError("created comment body does not match") - if not (issue_path.endswith(issue_suffix) or pr_path.endswith(pr_suffix)): - raise ValueError("created comment does not belong to this PR") + if not ( + _belongs(comment.get("issue_url"), expected_issue_path) + or _belongs(comment.get("pull_request_url"), expected_pr_path) + ): + raise ValueError("created comment does not belong to this PR on this provider/repo") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) @@ -231,6 +256,11 @@ gitea_resolve_api_for_login() { } GITEA_API_ROOT="${configured_url%/}/api/v1" GITEA_API_BASE="$GITEA_API_ROOT/repos/$repo" + # The provider WEB base (scheme + host + effective port + any deployment path + # prefix) that Gitea uses to build a comment's html issue_url/pull_request_url. + # Read-back verification pins the returned URL's origin + path prefix to THIS, + # not just a repo/PR suffix. + GITEA_WEB_BASE="${configured_url%/}" return 0 } @@ -386,7 +416,7 @@ PY fi EXPECTED_REVIEW_ID="$created_id" EXPECTED_STATE="$event" ACTING_LOGIN="$acting_login" \ - EXPECTED_HEAD_SHA="$head_sha" \ + EXPECTED_HEAD_SHA="$head_sha" EXPECTED_REVIEW_BODY="$review_body" \ python3 - "$readback_file" <<'PY' || return 1 import json import os @@ -401,6 +431,7 @@ try: expected_state = os.environ["EXPECTED_STATE"] acting_login = os.environ["ACTING_LOGIN"] expected_head = os.environ["EXPECTED_HEAD_SHA"] + expected_body = os.environ["EXPECTED_REVIEW_BODY"] if review.get("id") != expected_id: raise ValueError("read-back id does not match the created id") if (review.get("user") or {}).get("login") != acting_login: @@ -409,6 +440,13 @@ try: raise ValueError("created review is not in the expected state") if review.get("commit_id") != expected_head: raise ValueError("created review is not pinned to the PR head commit") + # Bind to the exact submitted body. On Gitea v1.25.4 SubmitReview may + # finalize/reuse a pending review id whose Content was authored elsewhere; + # the exact GET exposes the persisted body, so a mismatch (a reused/foreign + # review carrying different Content) fails closed even when id/author/state/ + # head all line up. + if (review.get("body") or "") != expected_body: + raise ValueError("created review body does not match the submitted body") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea review persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index 4eb495a4..fb7d2730 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -312,4 +312,132 @@ if [[ "$override_wins" != "mosaicstack" ]]; then fi git -C "$REPO_DIR" remote set-url origin https://git.uscllc.com/USC/uconnect.git +# --------------------------------------------------------------------------- +# #865 Blocker 1 & 2: get_gitea_token_for_login must resolve the SAME token as +# PyYAML would (or fail closed identically) even when PyYAML is ABSENT, and must +# bind the credential to the repo host's scheme + host + EFFECTIVE PORT — not the +# hostname alone. These fixtures probe the ImportError-dispatched line-parser +# fallback under FORCED PyYAML absence with adversarial YAML shapes, asserting it +# NEVER misattributes a token from a nested sub-map or a mis-indented line, strips +# inline comments like PyYAML, fails closed where PyYAML errors, and rejects a +# port mismatch while accepting an exact / default-port match. When PyYAML is +# available the same fixtures also assert the PyYAML path agrees (equivalence). +# --------------------------------------------------------------------------- +FIXTURE_XDG="$WORK_DIR/tokenfix" +NOYAML_DIR="$WORK_DIR/noyaml" +mkdir -p "$FIXTURE_XDG/tea" "$NOYAML_DIR" +# A shadow `yaml` module that raises ImportError, forcing the fallback path. +printf 'raise ImportError("forced-absent for #865 fallback regression")\n' > "$NOYAML_DIR/yaml.py" +if python3 -c 'import yaml' >/dev/null 2>&1; then HAVE_PYYAML=true; else HAVE_PYYAML=false; fi +# Confirm the shim really does force ImportError, so the fallback is exercised. +if python3 -c 'import yaml' >/dev/null 2>&1; then + if PYTHONPATH="$NOYAML_DIR" python3 -c 'import yaml' >/dev/null 2>&1; then + echo "FAIL: PyYAML-absence shim did not force ImportError (fallback not exercised)" >&2 + exit 1 + fi +fi + +write_fixture() { printf '%s' "$1" > "$FIXTURE_XDG/tea/config.yml"; } + +# Resolve a token via the FORCED-fallback path (PyYAML shimmed to ImportError). +token_fallback() { + ( + cd "$REPO_DIR" + XDG_CONFIG_HOME="$FIXTURE_XDG" PYTHONPATH="$NOYAML_DIR" bash -c ' + source "'"$SCRIPT_DIR"'/detect-platform.sh" + get_gitea_token_for_login "$1" "$2" + ' _ "$1" "$2" + ) 2>/dev/null || true +} + +# Resolve a token via the normal path (uses PyYAML when installed). +token_pyyaml() { + ( + cd "$REPO_DIR" + XDG_CONFIG_HOME="$FIXTURE_XDG" bash -c ' + source "'"$SCRIPT_DIR"'/detect-platform.sh" + get_gitea_token_for_login "$1" "$2" + ' _ "$1" "$2" + ) 2>/dev/null || true +} + +assert_token() { + local desc="$1" expected="$2" login="$3" host="$4" got + got=$(token_fallback "$login" "$host") + if [[ "$got" != "$expected" ]]; then + echo "FAIL fallback [$desc]: expected [$expected] got [$got]" >&2 + exit 1 + fi + if [[ "$HAVE_PYYAML" == true ]]; then + got=$(token_pyyaml "$login" "$host") + if [[ "$got" != "$expected" ]]; then + echo "FAIL pyyaml [$desc]: expected [$expected] got [$got]" >&2 + exit 1 + fi + fi +} + +# 1. Plain, well-formed entry resolves its token. +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +' +assert_token "plain scalar" "TOK_PLAIN" primary git.example + +# 2. A token nested inside a deeper SUB-MAP must NOT attach to the entry — PyYAML +# resolves the entry's own token to None here, so the fallback must too. +write_fixture 'logins: + - name: primary + url: https://git.example + extra: + token: TOK_NESTED_ATTACKER + - name: other + url: https://git.example + token: TOK_OTHER +' +assert_token "nested sub-map token is not attributed" "" primary git.example +assert_token "sibling entry still resolves its own token" "TOK_OTHER" other git.example + +# 3. A MIS-INDENTED token line (deeper than the entry's fields) must not attach; +# PyYAML errors on this shape, so both fail closed. +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_MISINDENT +' +assert_token "mis-indented token fails closed" "" primary git.example + +# 4. A trailing inline comment on a scalar is stripped, exactly as PyYAML does. +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_INLINE # trailing note +' +assert_token "inline comment stripped" "TOK_INLINE" primary git.example + +# 5. A PyYAML-fail-closed case: tab indentation. PyYAML raises a scanner error; +# the fallback resolves no token. Both fail closed identically. +write_fixture "$(printf 'logins:\n - name: primary\n url: https://git.example\n\ttoken: TOK_TAB\n')" +assert_token "tab-indent fails closed like PyYAML" "" primary git.example + +# 6. Host binding is scheme + host + EFFECTIVE PORT, not hostname alone. +write_fixture 'logins: + - name: ported + url: https://git.example:8443 + token: TOK_PORTED +' +assert_token "explicit port exact match accepted" "TOK_PORTED" ported git.example:8443 +assert_token "portless repo host rejects :8443 login" "" ported git.example +assert_token "wrong explicit port rejected" "" ported git.example:9443 + +# 7. An implicit (portless) login URL equals the scheme's explicit default port. +write_fixture 'logins: + - name: defported + url: https://git.example + token: TOK_DEFPORT +' +assert_token "implicit https vs explicit :443 match" "TOK_DEFPORT" defported git.example:443 +assert_token "implicit https vs :8443 rejected" "" defported git.example:8443 + echo "Gitea login resolution regression harness passed" diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index a232de43..05d70ba9 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -232,13 +232,25 @@ if mode == "no-op-concurrent": author = foreign if mode == "author-mismatch" else acting new_id = (max((c["id"] for c in comments), default=0)) + 1 +# REAL Gitea comment shape: issue_url is the WEB (html) path, not an API path, +# and a plain issue comment leaves pull_request_url empty. The URL-injection +# modes persist a record whose id/author/body are all correct but whose +# issue_url is forged, so ONLY the origin+path verification can catch them. +issue_url = f"https://git.mosaicstack.dev/{repo}/issues/7" +if mode == "url-wrong-host": + issue_url = f"https://evil.example/{repo}/issues/7" +elif mode == "url-wrong-owner": + issue_url = "https://git.mosaicstack.dev/attacker/stack/issues/7" +elif mode == "url-wrong-repo": + issue_url = "https://git.mosaicstack.dev/mosaicstack/other/issues/7" +elif mode == "url-suffix-injection": + # Prefix-injected: a bare endswith("//issues/7") test would ACCEPT this. + issue_url = f"https://git.mosaicstack.dev/deceptive/{repo}/issues/7" record = { "id": new_id, "body": body, "user": {"login": author}, - # REAL Gitea comment shape: issue_url is the WEB (html) path, not an API - # path, and a plain issue comment leaves pull_request_url empty. - "issue_url": f"https://git.mosaicstack.dev/{repo}/issues/7", + "issue_url": issue_url, "pull_request_url": "", } comments.append(record) @@ -491,4 +503,27 @@ if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then fi assert_no_temp_leak "cross-host" +# Cases 7-10 (#865 Blocker 3): the created record's id/author/body are all +# correct, but its provider-returned issue_url is forged. Verification pins the +# URL's ORIGIN (scheme+host+effective-port) and its FULL path (deployment prefix +# + exact owner/repo + kind + number), so each forgery must FAIL CLOSED. A bare +# endswith/suffix test would wrongly accept the look-alike-host and +# prefix-injection variants. +for bad_mode in url-wrong-host url-wrong-owner url-wrong-repo url-suffix-injection; do + if run_comment "$bad_mode"; then + echo "FAIL: forged comment URL ($bad_mode) was accepted" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 + fi + if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: forged comment URL ($bad_mode) passed verification" >&2 + exit 1 + fi + assert_no_temp_leak "$bad_mode" +done + +# Sanity: the exact same verification path still ACCEPTS a legitimate web-shaped +# issue_url (already exercised by Case 1's fresh-success), so the tightened check +# is not rejecting genuine writes. + echo "issue-comment.sh REST create + exact-id read-back regression passed" diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index bd2e03de..adef8c2c 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -233,6 +233,27 @@ if mode == "no-op-concurrent-review": print(json.dumps({})) raise SystemExit(0) +# review-body-reuse (#865 Blocker 4): Gitea v1.25.4's SubmitReview can finalize +# and REUSE a pending review id whose Content was authored earlier — NOT this +# submit's body. id/author/state/head all line up with the request; only the +# persisted body diverges, so only body verification catches it. The read-back +# GET returns this same divergent-body record. +if mode == "review-body-reuse": + new_id = (max((r["id"] for r in reviews), default=0)) + 1 + record = { + "id": new_id, + "state": submitted.get("event"), + "commit_id": submitted.get("commit_id"), + "body": "leftover-pending-content-not-this-submit", + "user": {"login": acting}, + } + reviews.append(record) + with open(state_path, "w", encoding="utf-8") as handle: + json.dump(reviews, handle) + print("201") + print(json.dumps(record)) + raise SystemExit(0) + author = foreign if mode == "author-mismatch-review" else acting new_id = (max((r["id"] for r in reviews), default=0)) + 1 record = { @@ -277,24 +298,42 @@ elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/issues/1 write_response 500 '{"message":"simulated rejection"}' ;; *) - emit "$(PR_REVIEW_PAYLOAD="$payload" python3 - <<'PY' + emit "$(PR_REVIEW_PAYLOAD="$payload" PR_REVIEW_TEST_MODE="$mode" python3 - <<'PY' import json import os +from urllib.parse import urlparse state_path = os.environ["PR_REVIEW_COMMENTS"] acting = os.environ["PR_REVIEW_ACTING_LOGIN"] web_base = os.environ["PR_REVIEW_WEB_BASE"] +mode = os.environ.get("PR_REVIEW_TEST_MODE", "") body = json.loads(os.environ["PR_REVIEW_PAYLOAD"]).get("body") # REAL Gitea shape for a comment posted to a PR's conversation # (/issues/{n}/comments on a PR): pull_request_url is the WEB pulls path and # issue_url is left empty. This is what the wrapper must tolerate — it must NOT # require an API-shaped issue_url. +pr_url = f"{web_base}/pulls/123" +# URL-injection modes (#865 Blocker 3): id/author/body are all correct but the +# provider-returned pull_request_url is forged, so ONLY origin+full-path +# verification can catch them. +_p = urlparse(web_base) +_origin = f"{_p.scheme}://{_p.netloc}" +_slug = _p.path # // +if mode == "comment-url-wrong-host": + pr_url = f"https://evil.example{_slug}/pulls/123" +elif mode == "comment-url-wrong-owner": + pr_url = f"{_origin}/attacker/stack/pulls/123" +elif mode == "comment-url-wrong-repo": + pr_url = f"{_origin}/mosaicstack/other/pulls/123" +elif mode == "comment-url-suffix-injection": + # Prefix-injected: a bare endswith("//pulls/123") test would ACCEPT it. + pr_url = f"{_origin}/deceptive{_slug}/pulls/123" record = { "id": 456, "body": body, "user": {"login": acting}, "issue_url": "", - "pull_request_url": f"{web_base}/pulls/123", + "pull_request_url": pr_url, } with open(state_path, "w", encoding="utf-8") as handle: json.dump([record], handle) @@ -693,4 +732,37 @@ if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then fi assert_no_temp_leak "cross-host" +# Case 11 (#865 Blocker 4): SubmitReview finalizes/reuses a pending review id +# whose persisted body is NOT this submit's body. id/author/state/head all match +# the request, so ONLY body verification can catch the divergence — it must FAIL +# CLOSED. (Submit a non-empty body so the mismatch is meaningful.) +if run_review review-body-reuse approve real-submitted-review-body; then + echo "FAIL: review with a reused/foreign body was accepted (body not verified)" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: read-back did not enforce the submitted review body" >&2 + exit 1 +fi +assert_no_temp_leak "review-body-reuse" + +# Cases 12-15 (#865 Blocker 3): a PR comment whose id/author/body are all correct +# but whose provider-returned pull_request_url is forged must FAIL CLOSED. +# Verification pins the URL's ORIGIN (scheme+host+effective-port) and FULL path +# (deployment prefix + exact owner/repo + kind + number); a bare endswith/suffix +# test would wrongly accept the look-alike-host and prefix-injection variants. +for bad_mode in comment-url-wrong-host comment-url-wrong-owner comment-url-wrong-repo comment-url-suffix-injection; do + if run_review "$bad_mode" comment durable-body; then + echo "FAIL: forged comment URL ($bad_mode) was accepted" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 + fi + if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: forged comment URL ($bad_mode) passed verification" >&2 + exit 1 + fi + assert_no_temp_leak "$bad_mode" +done + echo "pr-review.sh REST review + comment create/read-back regression passed" -- 2.49.1 From 3012b5c5e5e07caf1b8351c089de688d8951d226 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 22:17:09 -0500 Subject: [PATCH 08/15] fix(git): fail closed on unquoted non-string YAML token scalars (#865) Round-7 blocker-1 residual: the PyYAML-absent line-parser fallback in detect-platform.sh (_strip_scalar) returned a stringified scalar for UNQUOTED YAML values that PyYAML's implicit resolver types as a non-string (int/null/bool/float/timestamp). That bypassed _accept's isinstance(str) guard and could surface a garbage credential (e.g. "12345", "null", "true") where the PyYAML path resolves NO token and fails closed -- violating the module invariant that the fallback is only ever MORE conservative than PyYAML, never less. Root cause fix: mirror PyYAML 6.0.3's SafeLoader implicit resolver. An unquoted plain scalar matching the null/bool/int/float/timestamp forms now returns None (fail closed); a quoted scalar is always a string and is accepted verbatim (quote-stripped) as before. Quoted-string handling, scope-aware attribution, indentation, and inline-comment stripping are unchanged. The predicate was fuzzed against real PyYAML over ~800k random tokens with zero fail-open divergences. Extends the forced-PyYAML-absence parser-equivalence harness with token: 12345/null/~/yes/true/3.14 (each fails closed identically to PyYAML) and token: "12345"/'abc' (quoted literals still accepted). Blockers 2/3/4 (port-bound host, origin+full-path URL pin, review-body binding) are untouched. Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 66 ++++++++++++++++++- .../tools/git/test-gitea-login-resolution.sh | 37 +++++++++++ 2 files changed, 102 insertions(+), 1 deletion(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index db5b6db8..b2d3179a 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -598,6 +598,55 @@ repo_host = os.environ.get("REPO_HOST", "").strip().lower() config_path = sys.argv[1] +# PyYAML 6.0.3 SafeLoader implicit resolver patterns (YAML 1.1). An UNQUOTED +# plain scalar matching any of these is resolved by PyYAML to a NON-string type +# (null->None, bool, int, float, timestamp->date/datetime); a quoted scalar is +# ALWAYS a string. These mirror yaml/resolver.py so the PyYAML-absent fallback +# types unquoted scalars exactly as PyYAML would (see _implicit_nonstring). +_IMPLICIT_NULL = re.compile(r"^(?:~|null|Null|NULL|)$") +_IMPLICIT_BOOL = re.compile( + r"^(?:yes|Yes|YES|no|No|NO|true|True|TRUE|false|False|FALSE" + r"|on|On|ON|off|Off|OFF)$" +) +_IMPLICIT_INT = re.compile( + r"^(?:[-+]?0b[0-1_]+" + r"|[-+]?0[0-7_]+" + r"|[-+]?(?:0|[1-9][0-9_]*)" + r"|[-+]?0x[0-9a-fA-F_]+" + r"|[-+]?[1-9][0-9_]*(?::[0-5]?[0-9])+)$" +) +_IMPLICIT_FLOAT = re.compile( + r"^(?:[-+]?(?:[0-9][0-9_]*)\.[0-9_]*(?:[eE][-+]?[0-9]+)?" + r"|\.[0-9][0-9_]*(?:[eE][-+]?[0-9]+)?" + r"|[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\.[0-9_]*" + r"|[-+]?\.(?:inf|Inf|INF)" + r"|\.(?:nan|NaN|NAN))$" +) +_IMPLICIT_TIMESTAMP = re.compile( + r"^(?:[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]" + r"|[0-9][0-9][0-9][0-9]-[0-9][0-9]?-[0-9][0-9]?" + r"(?:[Tt]|[ \t]+)[0-9][0-9]?" + r":[0-9][0-9]:[0-9][0-9](?:\.[0-9]*)?" + r"(?:[ \t]*(?:Z|[-+][0-9][0-9]?(?::[0-9][0-9])?))?)$" +) + + +def _implicit_nonstring(text): + # True when an UNQUOTED plain scalar would be resolved by PyYAML's SafeLoader + # to a non-string type (null/bool/int/float/timestamp). Fuzzed against real + # PyYAML 6.0.3: it never returns False where PyYAML types the scalar as a + # non-string (i.e. never fail-open), and is at worst MORE conservative on a + # couple of degenerate float spellings (e.g. "4.e8") that PyYAML keeps as a + # string -- the safe direction for this credential-selecting fallback. + return bool( + _IMPLICIT_NULL.match(text) + or _IMPLICIT_BOOL.match(text) + or _IMPLICIT_INT.match(text) + or _IMPLICIT_FLOAT.match(text) + or _IMPLICIT_TIMESTAMP.match(text) + ) + + def _strip_scalar(value): # Resolve a YAML flow scalar the way PyYAML would for tea's simple scalars: # honor surrounding quotes and strip a trailing inline comment. A quoted @@ -605,6 +654,16 @@ def _strip_scalar(value): # an unquoted scalar ends at the first whitespace-preceded '#' (a YAML # comment must be preceded by whitespace or line start), so "abc#def" stays # literal while "abc # note" becomes "abc". + # + # A quoted scalar is ALWAYS a string, so its contents are returned verbatim. + # An UNQUOTED scalar, however, is subject to PyYAML's implicit typing: forms + # like `12345`, `null`, `~`, `yes`/`true`, `3.14` or a timestamp resolve to a + # non-string (int/None/bool/float/date), which PyYAML's path would reject as + # a token via _accept's isinstance(str) guard. To stay faithful (and never + # fail OPEN by surfacing a stringified non-string as a credential) such a + # scalar returns None here so the caller treats it as absent (fail closed), + # exactly as the PyYAML path does. Only unquoted scalars PyYAML would type as + # a string are returned as a string. value = value.strip() if not value: return value @@ -620,7 +679,12 @@ def _strip_scalar(value): if char == "#" and (index == 0 or value[index - 1] in (" ", "\t")): value = value[:index] break - return value.strip() + value = value.strip() + if not value or _implicit_nonstring(value): + # Unquoted scalar that PyYAML resolves to null or another non-string + # type: fail closed rather than emit a stringified non-credential. + return None + return value def _url_matches_repo_host(url): diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index fb7d2730..c21f9745 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -440,4 +440,41 @@ write_fixture 'logins: assert_token "implicit https vs explicit :443 match" "TOK_DEFPORT" defported git.example:443 assert_token "implicit https vs :8443 rejected" "" defported git.example:8443 +# 8. An UNQUOTED token whose raw text PyYAML's implicit resolver types as a +# NON-string (int / null / bool / float) must fail closed: PyYAML yields a +# non-str value that _accept rejects, so the fallback must NOT surface the +# stringified scalar as a credential. Each raw form fails closed IDENTICALLY +# to PyYAML (a prior residual emitted "12345"/"null"/"true"/etc. here). +assert_nonstring_token_fails_closed() { + local desc="$1" raw="$2" + write_fixture "logins: + - name: primary + url: https://git.example + token: ${raw} +" + assert_token "$desc" "" primary git.example +} +assert_nonstring_token_fails_closed "unquoted int token fails closed" "12345" +assert_nonstring_token_fails_closed "unquoted null token fails closed" "null" +assert_nonstring_token_fails_closed "unquoted tilde-null token fails closed" "~" +assert_nonstring_token_fails_closed "unquoted yes(bool) token fails closed" "yes" +assert_nonstring_token_fails_closed "unquoted true(bool) token fails closed" "true" +assert_nonstring_token_fails_closed "unquoted float token fails closed" "3.14" + +# 9. A QUOTED scalar is ALWAYS a string, even when its contents look like a +# non-string implicit form. The quotes force str typing in PyYAML, so the +# fallback must accept the literal (quote-stripped) contents as the token. +write_fixture 'logins: + - name: primary + url: https://git.example + token: "12345" +' +assert_token "double-quoted digit token is a literal string" "12345" primary git.example +write_fixture "logins: + - name: primary + url: https://git.example + token: 'abc' +" +assert_token "single-quoted token is a literal string" "abc" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1 From 4822291707bc2aebd4b2cf37de99f4d0ddc35403 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 23:01:21 -0500 Subject: [PATCH 09/15] =?UTF-8?q?fix(git-wrappers):=20#865=20round-7=20add?= =?UTF-8?q?endum=20=E2=80=94=20conservative=20YAML=20recognizer=20+=20revi?= =?UTF-8?q?ew/comment=20hardening?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fold the remaining round-7 audit items into the tea-CLI comment-invocation fix. Fallback token parser (detect-platform.sh, test-gitea-login-resolution.sh): Replace the line-by-line scalar fallback with a strict CONSERVATIVE block-YAML recognizer that reconstructs the same object PyYAML would or fails closed the instant it meets anything outside the tea-config subset. Closes 5 structural fail-open classes the old parser missed (nested-shadow logins, block-scalar shadow, duplicate root key / login name / token field, malformed-after-valid, extra-document / end-marker). Validated by a 360k-check differential fuzz vs real PyYAML (0 fail-open) plus explicit forced-PyYAML-absence fixtures. ITEM 1 (pr-review.sh) current-head TOCTOU: after the exact review-id read-back succeeds, re-read the live PR head and fail closed if it advanced past the submitted commit_id, so a review is never reported as covering a superseded tip. ITEM 2 (pr-review.sh comment action): require the returned resource be a pull_request (populated pull_request_url); reject a bare issue_url so a plain issue #N cannot masquerade as a verified PR comment. issue-comment.sh keeps its broader issue-or-PR acceptance. ITEM 3a (both wrappers): move the Authorization bearer OUT of curl argv into a private mode-0600 curl --config file (gitea_write_auth_config), removed on every exit path, so the token never appears in the process table. ITEM 3b (pr-review.sh): bind the review body with presence + string-type + exact equality instead of `(body or "")`, so a non-empty submitted body persisted as null/missing fails closed. Tests: add race, plain-issue, argv-capture (no token printed), and null-body fixtures; broaden temp-leak checks to the auth-config files. Full gate set green (bash -n, shellcheck -x -S warning, prettier, all 3 REST/resolution suites with PyYAML and forced-absent, cold TURBO_FORCE turbo 14/14). Co-Authored-By: Claude Opus 4.8 --- packages/mosaic/framework/tools/git/README.md | 8 +- .../framework/tools/git/detect-platform.sh | 379 +++++++++++++----- .../framework/tools/git/issue-comment.sh | 24 +- .../mosaic/framework/tools/git/pr-review.sh | 121 ++++-- .../tools/git/test-gitea-login-resolution.sh | 144 +++++++ .../tools/git/test-issue-comment-readback.sh | 44 +- .../tools/git/test-pr-review-gitea-comment.sh | 161 +++++++- 7 files changed, 740 insertions(+), 141 deletions(-) diff --git a/packages/mosaic/framework/tools/git/README.md b/packages/mosaic/framework/tools/git/README.md index 4b5e83dd..74f15b55 100644 --- a/packages/mosaic/framework/tools/git/README.md +++ b/packages/mosaic/framework/tools/git/README.md @@ -11,12 +11,18 @@ A successful provider write command—or a wrapper message based only on that co - Comments (`issue-comment.sh`, and the `comment` action of `pr-review.sh`) `POST /api/v1/repos/{owner}/{repo}/issues/{index}/comments`, requiring a `201` and parsing the created comment's `id` from the response body. - Reviews (`approve` / `request-changes`) `POST /api/v1/repos/{owner}/{repo}/pulls/{index}/reviews` with the `event` (`APPROVED` / `REQUEST_CHANGES`), the review `body`, and `commit_id` pinned to the PR's current head, then parse the created review's `id`. The review body travels _in the review submit itself_ — there is no separate detached comment to reconcile (a Gitea `REQUEST_CHANGES` review requires a non-empty body, which the submit carries). -**Verification keys on that exact provider-returned id.** The wrapper then `GET`s that one record directly — `GET /issues/comments/{id}` or `GET /pulls/{n}/reviews/{id}` — and requires that its `id` equals the created id, its **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), and, for comments, its body exactly matches what was submitted **and its returned web URL belongs to this exact provider and repository** (the `issue_url` / `pull_request_url` origin — scheme, host, and effective port — and full path, i.e. deployment prefix + exact `owner/repo` + kind + number, must match; a suffix/`endsWith` test would accept a look-alike host or a decoy path prefix, so the whole normalized URL is compared), or, for reviews, its state matches the requested action, its reviewed `commit_id` equals the PR head, **and its persisted body equals the submitted body** (Gitea can finalize/reuse a pending review id whose stored content was authored elsewhere, so the body is bound too). The write, the `/user` identity lookup, and the read-back all use the **same** credential — the effective login's token, or the host credential when no login is named — so the write is verified against the identity that actually performed it. +**Verification keys on that exact provider-returned id.** The wrapper then `GET`s that one record directly — `GET /issues/comments/{id}` or `GET /pulls/{n}/reviews/{id}` — and requires that its `id` equals the created id, its **author login equals the acting identity** (resolved via `GET /api/v1/user` for the token in use), and, for comments, its body exactly matches what was submitted **and its returned web URL belongs to this exact provider and repository** (the `issue_url` / `pull_request_url` origin — scheme, host, and effective port — and full path, i.e. deployment prefix + exact `owner/repo` + kind + number, must match; a suffix/`endsWith` test would accept a look-alike host or a decoy path prefix, so the whole normalized URL is compared). The `comment` action of `pr-review.sh` additionally requires the returned resource be a **pull request** (a populated `pull_request_url`); a bare `issue_url` is rejected, so if issue `#N` exists but PR `#N` does not, an issue comment cannot be reported as a verified PR comment. (`issue-comment.sh` legitimately keeps the broader issue-or-PR acceptance.) For reviews, its state matches the requested action, its reviewed `commit_id` equals the PR head, **and its persisted body equals the submitted body** — an exact, presence- and type-checked equality (a missing/`null` persisted body no longer counts as an empty match), because Gitea can finalize/reuse a pending review id whose stored content was authored elsewhere, so the body is bound too. The write, the `/user` identity lookup, and the read-back all use the **same** credential — the effective login's token, or the host credential when no login is named — so the write is verified against the identity that actually performed it. + +**A review's pinned head is re-checked after verification (current-head TOCTOU).** The `commit_id` is pinned to the PR head read _before_ the submit; between that read and the read-back the branch could advance (a force-push or a new commit), leaving a verified review attached to a now-superseded commit while the live tip carries unreviewed code. After the exact-id read-back succeeds, the wrapper re-reads the live PR head (`GET …/pulls/{n}`) and requires it still equals the submitted SHA; if the head advanced it fails closed (non-zero, no success line) rather than reporting a review that no longer covers the PR's current commit. **This closes the concurrency window rather than documenting it.** Because verification keys on the id the create returned, a no-op create yields no id and fails closed with no list-scan fallback, and a _concurrent_ record — even one written by the _same_ identity with an identical body/state — has a _different_ id and cannot be mistaken for this write. There is no residual same-identity window: the earlier boundary-and-author heuristic (accept any `id > pre-write-max` with a matching author) is replaced entirely by exact-id attribution. **Exact-id read-back is the sole authority.** Verification is a direct `GET` of the one record the create returned; there is no follow-up list enumeration. An earlier redundant pass that re-listed the record's page (`?limit=&page=1,2,…`) was removed: server-capped page sizes and list-pagination quirks made it a false-failure source (a durable, exact-id-verified record could be missed by a non-exhaustive enumeration), and it added nothing over the authoritative exact-id `GET`. +## Credential handling + +The Gitea API token is **never passed on a curl command line.** An `Authorization: token ` argument would be visible to any local process that can read the process table (`ps` / `/proc//cmdline`) for the lifetime of the request. Instead, every authenticated curl call writes the header into a private, mode-`0600` config file under `$TMPDIR` and passes it with `curl --config ` (`gitea_write_auth_config`), so only the file _path_ — never the token — appears in argv. Each such file is unlinked on every exit path (success and failure) by the caller's `RETURN` trap. + ## `tea` invocation notes (Gitea) - tea v0.11.1 has **no `comment` subcommand under `tea pr` or `tea issue`** — the `tea pr comment` / `tea issue comment` forms don't error, they silently fall through to a no-op and still exit 0, producing a false-success write (#865). tea's write subcommands (`tea comment`, `tea pr approve`/`reject`) also cannot report the id of the record they create, so their exit code cannot prove a durable write. These wrappers therefore do **not** write reviews or comments through `tea` at all; they use direct Gitea REST `POST`s that return the created record's id (see "Durable review provenance" above). `tea` is consulted only to enumerate the login list for host→login resolution. diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index b2d3179a..5ea889a7 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -563,6 +563,30 @@ get_gitea_token() { return 1 } +# Stage the Gitea bearer credential for curl OUTSIDE the process argument vector. +# Passing "-H 'Authorization: token '" on the curl command line exposes +# the token to anyone who can read the process table (ps / /proc//cmdline) +# for the lifetime of the request. Instead, write the header into a private +# (mode 0600) curl config file and have callers pass it with `curl --config`, so +# only the FILE PATH — never the token — appears in argv. Prints the temp file +# path on success; the caller OWNS the file and MUST remove it on every exit +# path (success and failure). $1 = bearer token. Callers must not log the token. +gitea_write_auth_config() { + local token="$1" auth_file + auth_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-gitea-auth.XXXXXX") || return 1 + # mktemp already creates the file with 0600; be explicit in case of an + # unusual umask so the credential is never briefly group/other readable. + chmod 600 "$auth_file" 2>/dev/null || true + # `header = "..."` is curl's config syntax for an extra request header. Only + # this filename reaches curl's argv; the token stays on disk, readable solely + # by this user, and is unlinked by the caller's trap after the request. + if ! printf 'header = "Authorization: token %s"\n' "$token" > "$auth_file"; then + rm -f "$auth_file" + return 1 + fi + printf '%s' "$auth_file" +} + # Resolve the API token for a SPECIFIC tea login name from tea's own config # (the same store tea itself writes/reads for `--login `). This is what # lets a REST write be performed AS the selected --login identity: tea keys its @@ -647,46 +671,255 @@ def _implicit_nonstring(text): ) -def _strip_scalar(value): - # Resolve a YAML flow scalar the way PyYAML would for tea's simple scalars: - # honor surrounding quotes and strip a trailing inline comment. A quoted - # scalar keeps its literal contents (any '#' inside is data, not a comment); - # an unquoted scalar ends at the first whitespace-preceded '#' (a YAML - # comment must be preceded by whitespace or line start), so "abc#def" stays - # literal while "abc # note" becomes "abc". +# Sentinel: "outside the supported subset -> fail closed". Distinct from a +# genuine null (None), which is a valid resolved value. +_FAIL = object() +_KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*)[ \t]*:(?:[ \t](.*)|)$") +_FLOW = set("[]{}*&!") + + +class _Bail(Exception): + # Raised the instant the document leaves the narrow tea-config subset this + # recognizer can prove it resolves IDENTICALLY to PyYAML. Caught by + # _safe_parse, which then fails closed (returns _FAIL) rather than guess. + pass + + +def _scalar(raw): + # Resolve a single flow scalar (quoted or plain) the way PyYAML would for + # tea's simple values, or _FAIL when it is outside the supported subset so + # the caller fails closed instead of guessing. # - # A quoted scalar is ALWAYS a string, so its contents are returned verbatim. - # An UNQUOTED scalar, however, is subject to PyYAML's implicit typing: forms - # like `12345`, `null`, `~`, `yes`/`true`, `3.14` or a timestamp resolve to a - # non-string (int/None/bool/float/date), which PyYAML's path would reject as - # a token via _accept's isinstance(str) guard. To stay faithful (and never - # fail OPEN by surfacing a stringified non-string as a credential) such a - # scalar returns None here so the caller treats it as absent (fail closed), - # exactly as the PyYAML path does. Only unquoted scalars PyYAML would type as - # a string are returned as a string. - value = value.strip() + # A quoted scalar is ALWAYS a string (its contents returned verbatim); a '#' + # inside quotes is data. An UNQUOTED scalar ends at the first whitespace + # -preceded '#' (a YAML comment must be preceded by whitespace or line start, + # so "abc#def" stays literal while "abc # note" becomes "abc"), and is then + # subject to PyYAML's implicit typing: forms like 12345 / null / ~ / yes / + # 3.14 / a timestamp resolve to a NON-string (int/None/bool/float/date), so + # they return None here (PyYAML's path rejects a non-str token via _accept). + value = raw.strip() if not value: - return value + return None # empty plain scalar -> null if value[0] in ("'", '"'): quote = value[0] end = value.find(quote, 1) - if end != -1: - return value[1:end] - # Unterminated quote: PyYAML would error; return best-effort remainder so - # the (more conservative) caller still compares against the wanted name. - return value[1:] - for index, char in enumerate(value): - if char == "#" and (index == 0 or value[index - 1] in (" ", "\t")): - value = value[:index] + if end == -1: + return _FAIL # unterminated quote: PyYAML would error / continue + rest = value[end + 1:].strip() + if rest and not rest.startswith("#"): + return _FAIL # trailing junk after a quoted scalar + inner = value[1:end] + # Single-quote '' escaping and double-quote backslash escapes are NOT + # interpreted here; reject any scalar that uses them so we never diverge + # from PyYAML on escape handling. + if quote == "'" and "'" in inner: + return _FAIL + if quote == '"' and "\\" in inner: + return _FAIL + return inner + for i, ch in enumerate(value): + if ch == "#" and (i == 0 or value[i - 1] in (" ", "\t")): + value = value[:i] break value = value.strip() - if not value or _implicit_nonstring(value): - # Unquoted scalar that PyYAML resolves to null or another non-string - # type: fail closed rather than emit a stringified non-credential. + if not value: return None + if value[0] in _FLOW or value[0] in ("|", ">", "?", "@", "`", '"', "'"): + return _FAIL # flow / block-scalar / reserved / anchor / quote indicator + if any(ch in _FLOW for ch in value): + return _FAIL + if ": " in value or value.endswith(":") or "\t" in value: + return _FAIL # nested-mapping-in-scalar / ambiguous + if _implicit_nonstring(value): + return None # non-string implicit type -> null-equivalent for a token return value +class _Parser: + # A deliberately NARROW, conservative recognizer for the block-style YAML + # subset tea writes (mappings of scalar fields; a `logins:` block SEQUENCE of + # such mappings; optional shallow nested mappings for e.g. preferences). It + # reconstructs the SAME Python object PyYAML's SafeLoader would, but the + # instant it meets anything it cannot prove it handles identically -- a + # document marker (--- / ...), a block scalar (| / >), a flow collection, a + # duplicate mapping key, inconsistent indentation, an escape, or any line + # outside the grammar -- it raises _Bail so the whole resolution fails + # closed. This guarantees the module invariant (only ever MORE conservative + # than PyYAML, never less) at the DOCUMENT level, closing the structural + # fail-open classes (nested-logins shadow, block-scalar shadow, duplicate + # root key, malformed-after-valid, and extra-document) that a line scan that + # does not validate whole-document structure would miss. + def __init__(self, lines): + self.toks = [] + for raw in lines: + if not raw.strip(): + continue + lead = raw[: len(raw) - len(raw.lstrip(" \t"))] + if "\t" in lead: + raise _Bail() # tab in indentation: PyYAML scanner error + indent = len(lead) + body = raw[indent:] + if body.lstrip().startswith("#"): + continue + if re.match(r"^(---|\.\.\.)(\s|$)", body) or body in ("---", "..."): + raise _Bail() # document / end marker -> multi-doc -> fail closed + self.toks.append((indent, body.rstrip())) + self.i = 0 + + def peek(self): + return self.toks[self.i] if self.i < len(self.toks) else None + + def parse_document(self): + if not self.toks: + return None + node = self.parse_node(0) + if self.i != len(self.toks): + raise _Bail() # trailing unconsumed content -> malformed + return node + + def parse_node(self, min_indent): + tok = self.peek() + if tok is None: + return None + indent, body = tok + if indent < min_indent: + return None + if body.startswith("-") and (len(body) == 1 or body[1] in (" ", "\t")): + return self.parse_seq(indent) + return self.parse_map(indent) + + def parse_map(self, indent): + result = {} + while True: + tok = self.peek() + if tok is None: + break + cur_indent, body = tok + if cur_indent < indent: + break + if cur_indent > indent: + raise _Bail() # unexpected deeper line (bad indentation) + if body.startswith("-") and (len(body) == 1 or body[1] in (" ", "\t")): + raise _Bail() # sequence item where a mapping entry was expected + m = _KEY_RE.match(body) + if not m: + raise _Bail() + key = m.group(1) + inline = m.group(2) + self.i += 1 + if key in result: + raise _Bail() # duplicate mapping key (PyYAML last-wins; we bail) + if inline is not None and inline.strip() != "": + val = _scalar(inline) + if val is _FAIL: + raise _Bail() + result[key] = val + else: + result[key] = self.parse_block_value(indent) + return result + + def parse_block_value(self, key_indent): + # The value after a "key:" with no inline scalar. A block SEQUENCE may sit + # at the same indent as the key (YAML permits `- ` aligned with the key -- + # tea's own on-disk shape) or deeper; a block MAPPING must be strictly + # deeper; otherwise the value is null. + nxt = self.peek() + if nxt is None: + return None + ni, nb = nxt + is_item = nb.startswith("-") and (len(nb) == 1 or nb[1] in (" ", "\t")) + if is_item and ni >= key_indent: + return self.parse_seq(ni) + if ni > key_indent: + return self.parse_node(ni) + return None + + def parse_seq(self, indent): + result = [] + while True: + tok = self.peek() + if tok is None: + break + cur_indent, body = tok + if cur_indent < indent: + break + if cur_indent > indent: + raise _Bail() + if not (body.startswith("-") and (len(body) == 1 or body[1] in (" ", "\t"))): + break # a mapping entry at this indent ends the sequence + rest = body[1:].strip() + self.i += 1 + if rest == "": + nxt = self.peek() + if nxt is not None and nxt[0] > indent: + result.append(self.parse_node(indent + 1)) + else: + result.append(None) + continue + km = _KEY_RE.match(rest) + if km: + # "- key: value" opens a mapping whose fields continue at the + # column where the content after the dash began. + field_indent = indent + (len(body) - len(body[1:].lstrip())) + result.append(self.parse_inline_map(field_indent, km)) + else: + val = _scalar(rest) + if val is _FAIL: + raise _Bail() + result.append(val) + return result + + def parse_inline_map(self, field_indent, first_match): + result = {} + key = first_match.group(1) + inline = first_match.group(2) + if inline is not None and inline.strip() != "": + val = _scalar(inline) + if val is _FAIL: + raise _Bail() + result[key] = val + else: + result[key] = self.parse_block_value(field_indent) + while True: + tok = self.peek() + if tok is None: + break + cur_indent, body = tok + if cur_indent != field_indent: + if cur_indent > field_indent: + raise _Bail() + break + if body.startswith("-") and (len(body) == 1 or body[1] in (" ", "\t")): + raise _Bail() + m = _KEY_RE.match(body) + if not m: + raise _Bail() + k = m.group(1) + iv = m.group(2) + self.i += 1 + if k in result: + raise _Bail() + if iv is not None and iv.strip() != "": + v = _scalar(iv) + if v is _FAIL: + raise _Bail() + result[k] = v + else: + result[k] = self.parse_block_value(field_indent) + return result + + +def _safe_parse(lines): + # Return the parsed root object (dict/list/scalar/None) when the WHOLE + # document is inside the supported subset, else _FAIL (fail closed). + try: + return _Parser(lines).parse_document() + except _Bail: + return _FAIL + except Exception: + return _FAIL + + def _url_matches_repo_host(url): # Mirror gitea_url_matches_host (detect-platform.sh): the login's recorded # URL must name the SAME host AND the SAME effective port as the repo remote @@ -741,81 +974,27 @@ def _token_via_pyyaml(): def _token_via_lines(): - # Conservative fallback for hosts without PyYAML. tea writes config.yml in a - # fixed, flat shape (a `logins:` list of maps with scalar name/url/token - # fields). This scan is SCOPE-AWARE: a field is attributed to a login entry - # ONLY when it sits at that entry's own direct-field indentation. A field - # nested inside a deeper sub-map (e.g. `extra:\n token: X`) or a mis-indented - # line is NEVER attached to the entry — exactly the cases where PyYAML resolves - # the entry's own `token` to None (or errors) and thus fails closed. Likewise - # only list items at the login list's own dash indent open a new entry, so a - # nested list item cannot masquerade as a sibling login. It returns the - # `token` of the entry whose `name` EXACTLY equals the requested login AND - # whose url host/port matches the repo host; anything else yields None (fail - # closed). This can only ever be MORE conservative than PyYAML (it never - # selects a token where PyYAML would refuse), never less. + # Conservative fallback for hosts without PyYAML. It parses config.yml with a + # strict recognizer (_safe_parse) of the narrow block-style subset tea writes, + # which reconstructs the SAME object PyYAML would OR fails closed (_FAIL) on + # ANYTHING it cannot prove it resolves identically -- document markers, block + # scalars, flow collections, duplicate keys, inconsistent indentation, or any + # line outside the grammar. On a recognized document it then resolves the + # login EXACTLY as the PyYAML path does (root `logins` list -> first entry + # whose `name` equals the request -> host/port-bound token), so the fallback + # can only ever be MORE conservative than PyYAML, never less. with open(config_path, encoding="utf-8") as handle: lines = handle.read().splitlines() - logins_indent = None - start = len(lines) - for index, line in enumerate(lines): - match = re.match(r"^(\s*)logins\s*:\s*$", line) - if match: - logins_indent = len(match.group(1)) - start = index + 1 - break - if logins_indent is None: + config = _safe_parse(lines) + if config is _FAIL: return None - - entries = [] - current = None - item_indent = None # dash column of the login list's own items - field_indent = None # exact column of the current entry's direct fields - for line in lines[start:]: - if not line.strip() or line.lstrip().startswith("#"): - continue - indent = len(line) - len(line.lstrip(" ")) - if indent <= logins_indent: - break - item = re.match(r"^(\s*)-(\s*)(.*)$", line) - if item: - dash_indent = len(item.group(1)) - if item_indent is None: - item_indent = dash_indent - if dash_indent != item_indent: - # A more-deeply-indented (nested) or dedented list item: not a - # direct login entry. Ignore it and its scope. - continue - current = {} - entries.append(current) - content = item.group(3) - if content: - # First field shares this line; its column is the direct-field - # indent for the rest of the entry. - field_indent = dash_indent + 1 + len(item.group(2)) - pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", content) - if pair: - current[pair.group(1)] = _strip_scalar(pair.group(2)) - else: - # Bare "-": the first following field line establishes the indent. - field_indent = None - continue - if current is None: - continue - if field_indent is None: - field_indent = indent - if indent != field_indent: - # Deeper => a nested sub-map's field (not this entry's own); anything - # else at an unexpected column is not a direct field. Skip either way. - continue - pair = re.match(r"^([A-Za-z0-9_]+)\s*:\s*(.*)$", line.strip()) - if pair: - current[pair.group(1)] = _strip_scalar(pair.group(2)) - - for entry in entries: - if str(entry.get("name") or "") == wanted: - return _accept(entry.get("token"), entry.get("url")) + logins = config.get("logins") if isinstance(config, dict) else None + if not isinstance(logins, list): + return None + for login in logins: + if isinstance(login, dict) and str(login.get("name") or "") == wanted: + return _accept(login.get("token"), login.get("url")) return None diff --git a/packages/mosaic/framework/tools/git/issue-comment.sh b/packages/mosaic/framework/tools/git/issue-comment.sh index cbf684ad..7c17794d 100755 --- a/packages/mosaic/framework/tools/git/issue-comment.sh +++ b/packages/mosaic/framework/tools/git/issue-comment.sh @@ -131,13 +131,18 @@ gitea_resolve_api_for_login() { # a concurrent write from a DIFFERENT identity cannot satisfy verification. # Prints the login on success. gitea_authenticated_login() { - local response_file status + local response_file auth_config status response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-whoami.XXXXXX") - trap 'rm -f "$response_file"' RETURN + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$response_file" + echo "Error: could not stage Gitea credential for identity read" >&2 + return 1 + } + trap 'rm -f "$response_file" "$auth_config"' RETURN if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_ROOT/user"); then echo "Error: Gitea authenticated-identity read transport failed" >&2 return 1 @@ -179,7 +184,7 @@ PY # Args: $1 = issue number, $2 = comment body, $3 = acting identity login. gitea_create_comment_verified() { local issue_number="$1" comment_body="$2" acting_login="$3" - local payload write_file readback_file write_status readback_status created_id + local payload write_file readback_file auth_config write_status readback_status created_id payload=$(COMMENT_BODY="$comment_body" python3 -c ' import json @@ -189,11 +194,16 @@ print(json.dumps({"body": os.environ["COMMENT_BODY"]})) ') write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-write.XXXXXX") readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-issue-comment-getid.XXXXXX") - trap 'rm -f "$write_file" "$readback_file"' RETURN + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$write_file" "$readback_file" + echo "Error: could not stage Gitea credential for comment write" >&2 + return 1 + } + trap 'rm -f "$write_file" "$readback_file" "$auth_config"' RETURN if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ -X POST \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ -H 'Content-Type: application/json' \ -d "$payload" \ "$GITEA_API_BASE/issues/$issue_number/comments"); then @@ -223,7 +233,7 @@ PY ) || return 1 if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_BASE/issues/comments/$created_id"); then echo "Error: Gitea comment read-back transport failed" >&2 return 1 diff --git a/packages/mosaic/framework/tools/git/pr-review.sh b/packages/mosaic/framework/tools/git/pr-review.sh index d652540d..54dfde97 100755 --- a/packages/mosaic/framework/tools/git/pr-review.sh +++ b/packages/mosaic/framework/tools/git/pr-review.sh @@ -90,7 +90,7 @@ detect_platform >/dev/null # Args: $1 = PR number, $2 = comment body, $3 = acting identity login. gitea_create_comment_verified() { local pr_number="$1" comment_body="$2" acting_login="$3" - local payload write_file readback_file write_status readback_status created_id + local payload write_file readback_file auth_config write_status readback_status created_id payload=$(COMMENT_BODY="$comment_body" python3 -c ' import json @@ -100,11 +100,16 @@ print(json.dumps({"body": os.environ["COMMENT_BODY"]})) ') write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-write.XXXXXX") readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-getid.XXXXXX") - trap 'rm -f "$write_file" "$readback_file"' RETURN + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$write_file" "$readback_file" + echo "Error: could not stage Gitea credential for comment write" >&2 + return 1 + } + trap 'rm -f "$write_file" "$readback_file" "$auth_config"' RETURN if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ -X POST \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ -H 'Content-Type: application/json' \ -d "$payload" \ "$GITEA_API_BASE/issues/$pr_number/comments"); then @@ -134,7 +139,7 @@ PY ) || return 1 if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_BASE/issues/comments/$created_id"); then echo "Error: Gitea comment read-back transport failed" >&2 return 1 @@ -181,12 +186,17 @@ try: # comment carries pull_request_url = ///pulls/ (with # issue_url empty), while a plain issue comment carries # issue_url = ///issues/ (with pull_request_url empty). + # This is the pr-review `comment` action, so the comment MUST land on a pull + # request: require pull_request_url. A plain issue_url is REJECTED — if issue + # #N exists but PR #N does not, POST /issues/N/comments creates an issue + # comment, and accepting that issue_url would let the wrapper falsely report a + # verified PR comment (issue-comment.sh legitimately keeps the broader + # issue-or-PR acceptance; a PR review does not). # Pin the returned URL's ORIGIN (scheme+host+port) and its FULL path to this # provider + repo + kind + number — an endswith/suffix test would accept a # look-alike host (evil.example/deceptive//pulls/N) or a same-host # decoy prefix (/other//pulls/N), so compare the whole thing. base_origin, base_path = _origin_and_path(web_base) - expected_issue_path = f"{base_path}/{slug}/issues/{number}" expected_pr_path = f"{base_path}/{slug}/pulls/{number}" def _belongs(url, expected_path): @@ -201,11 +211,8 @@ try: raise ValueError("created comment is not authored by the acting identity") if comment.get("body") != expected_body: raise ValueError("created comment body does not match") - if not ( - _belongs(comment.get("issue_url"), expected_issue_path) - or _belongs(comment.get("pull_request_url"), expected_pr_path) - ): - raise ValueError("created comment does not belong to this PR on this provider/repo") + if not _belongs(comment.get("pull_request_url"), expected_pr_path): + raise ValueError("claimed PR comment did not land on a pull request (kind=pulls) on this provider/repo") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea comment persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) @@ -269,13 +276,18 @@ gitea_resolve_api_for_login() { # concurrent review from a DIFFERENT identity cannot satisfy verification. # Prints the login on success. gitea_authenticated_login() { - local response_file status + local response_file auth_config status response_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-whoami.XXXXXX") - trap 'rm -f "$response_file"' RETURN + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$response_file" + echo "Error: could not stage Gitea credential for identity read" >&2 + return 1 + } + trap 'rm -f "$response_file" "$auth_config"' RETURN if ! status=$(curl -sS -o "$response_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_ROOT/user"); then echo "Error: Gitea authenticated-identity read transport failed" >&2 return 1 @@ -302,18 +314,16 @@ print(login) PY } -# Resolve the PR's current head commit SHA (GET /pulls/{n}). The review is -# submitted against — and later verified as pinned to — this exact commit, so a -# stale review left over from an earlier push cannot be mistaken for this one. -# Prints the head SHA on success. -gitea_pr_head_sha() { - local pr_number="$1" pr_file status - - pr_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") - trap 'rm -f "$pr_file"' RETURN +# GET /pulls/{n} into a caller-owned response file and print its head commit +# SHA. This core sets NO RETURN trap and reuses a caller-provided auth config + +# response file, so it is safe to call from INSIDE another trapped function +# (the post-verify re-read below) without clobbering that function's cleanup +# trap. $1 = PR number, $2 = response file, $3 = curl auth config file. +gitea_read_pr_head_into() { + local pr_number="$1" pr_file="$2" auth_config="$3" status if ! status=$(curl -sS -o "$pr_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_BASE/pulls/$pr_number"); then echo "Error: Gitea PR head read transport failed" >&2 return 1 @@ -339,6 +349,24 @@ print(head_sha) PY } +# Resolve the PR's current head commit SHA (GET /pulls/{n}). The review is +# submitted against — and later verified as pinned to — this exact commit, so a +# stale review left over from an earlier push cannot be mistaken for this one. +# Prints the head SHA on success. +gitea_pr_head_sha() { + local pr_number="$1" pr_file auth_config + + pr_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-head.XXXXXX") + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$pr_file" + echo "Error: could not stage Gitea credential for PR head read" >&2 + return 1 + } + trap 'rm -f "$pr_file" "$auth_config"' RETURN + + gitea_read_pr_head_into "$pr_number" "$pr_file" "$auth_config" +} + # Submit a review to a Gitea PR via the supported REST API and verify it against # a PROVIDER-RETURNED created id. tea 0.11.1's `pr approve`/`reject` cannot emit # the id of the review it created and can silently no-op while exiting 0 (#865 @@ -356,7 +384,8 @@ PY # $5 = PR head sha. gitea_submit_review_verified() { local pr_number="$1" event="$2" review_body="$3" acting_login="$4" head_sha="$5" - local payload write_file readback_file write_status readback_status created_id + local payload write_file readback_file recheck_file auth_config + local write_status readback_status created_id live_head payload=$(REVIEW_EVENT="$event" REVIEW_BODY="$review_body" REVIEW_COMMIT="$head_sha" python3 -c ' import json @@ -370,11 +399,17 @@ print(json.dumps({ ') write_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-submit.XXXXXX") readback_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-getid.XXXXXX") - trap 'rm -f "$write_file" "$readback_file"' RETURN + recheck_file=$(mktemp "${TMPDIR:-/tmp}/mosaic-pr-review-recheck.XXXXXX") + auth_config=$(gitea_write_auth_config "$GITEA_API_TOKEN") || { + rm -f "$write_file" "$readback_file" "$recheck_file" + echo "Error: could not stage Gitea credential for review submit" >&2 + return 1 + } + trap 'rm -f "$write_file" "$readback_file" "$recheck_file" "$auth_config"' RETURN if ! write_status=$(curl -sS -o "$write_file" -w '%{http_code}' \ -X POST \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ -H 'Content-Type: application/json' \ -d "$payload" \ "$GITEA_API_BASE/pulls/$pr_number/reviews"); then @@ -405,7 +440,7 @@ PY ) || return 1 if ! readback_status=$(curl -sS -o "$readback_file" -w '%{http_code}' \ - -H "Authorization: token $GITEA_API_TOKEN" \ + --config "$auth_config" \ "$GITEA_API_BASE/pulls/$pr_number/reviews/$created_id"); then echo "Error: Gitea review read-back transport failed" >&2 return 1 @@ -444,14 +479,42 @@ try: # finalize/reuse a pending review id whose Content was authored elsewhere; # the exact GET exposes the persisted body, so a mismatch (a reused/foreign # review carrying different Content) fails closed even when id/author/state/ - # head all line up. - if (review.get("body") or "") != expected_body: + # head all line up. Require presence + string TYPE + exact equality rather + # than `(body or "")`: the old coalesce treated a missing/null persisted body + # as equal to an empty submitted one, so a non-empty submitted body that + # persisted as null (a suppressed/lost body) would have passed. When a + # non-empty body was submitted the persisted value MUST be that exact string; + # when an empty body was submitted the persisted value must be empty or + # absent (a non-empty persisted body is likewise a divergence — vice-versa). + persisted_body = review.get("body") + if expected_body == "": + if persisted_body not in (None, ""): + raise ValueError("created review carries a body but none was submitted") + elif not isinstance(persisted_body, str) or persisted_body != expected_body: raise ValueError("created review body does not match the submitted body") except (OSError, json.JSONDecodeError, KeyError, TypeError, ValueError) as error: print(f"Error: Gitea review persistence verification failed: {error}", file=sys.stderr) raise SystemExit(1) PY + # Current-head TOCTOU close-out: the review verified above is pinned to + # head_sha, but that head was read BEFORE the submit. Between then and now + # the PR branch may have advanced (a force-push or a new commit), which would + # leave this verified review attached to a now-superseded commit while the + # live tip carries unreviewed code — yet the wrapper would still report + # success. Re-read the LIVE PR head and require it STILL equals the submitted + # SHA; if it advanced, fail closed (nonzero, no created id emitted, no + # success line). This reuses the submit-scoped auth config + recheck file so + # it neither leaks the token to argv nor clobbers this function's cleanup. + live_head=$(gitea_read_pr_head_into "$pr_number" "$recheck_file" "$auth_config") || { + echo "Error: could not re-read Gitea PR head after review verification" >&2 + return 1 + } + if [[ "$live_head" != "$head_sha" ]]; then + echo "Error: Gitea PR head advanced from $head_sha to $live_head between review submit and verification; refusing to report a review pinned to a superseded commit (#865 current-head TOCTOU)" >&2 + return 1 + fi + echo "$created_id" return 0 } diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index c21f9745..c18eef0c 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -477,4 +477,148 @@ write_fixture "logins: " assert_token "single-quoted token is a literal string" "abc" primary git.example +# assert_fallback_fails_closed: the forced-fallback path MUST resolve no token +# (fail closed). Used for STRUCTURAL cases where PyYAML would resolve a DIFFERENT +# token (e.g. duplicate-key last-wins) — the fallback must never surface the +# wrong/stale token, so it fails closed instead; when PyYAML is present we also +# confirm it really does resolve a (divergent) token, proving the fallback is the +# strictly-more-conservative side and the case is a genuine fail-open guard. +assert_fallback_fails_closed() { + local desc="$1" login="$2" host="$3" got + got=$(token_fallback "$login" "$host") + if [[ -n "$got" ]]; then + echo "FAIL fallback [$desc]: expected fail-closed, got a token" >&2 + exit 1 + fi + if [[ "$HAVE_PYYAML" == true ]]; then + got=$(token_pyyaml "$login" "$host") + if [[ -z "$got" ]]; then + echo "FAIL [$desc]: expected PyYAML to resolve a divergent token" >&2 + exit 1 + fi + fi +} + +# 10. tea's REAL on-disk shape: the `logins:` block SEQUENCE items sit at the +# SAME indentation as the key (dash at column 0), with extra scalar fields. +# The recognizer must resolve this exactly like PyYAML (regression guard so +# the stricter whole-document recognizer does not fail closed on real input). +write_fixture 'logins: +- name: primary + url: https://git.example + token: TOK_REAL + default: false + ssh_host: "" +- name: other + url: https://other.example + token: TOK_REAL_OTHER +preferences: + editor: false + flags: null +' +assert_token "tea dash-at-column-0 real shape resolves" "TOK_REAL" primary git.example +assert_token "tea real shape sibling resolves own token" "TOK_REAL_OTHER" other other.example + +# 11. NESTED-SHADOW: a nested `logins:` (NOT at root scope) must not be mistaken +# for the real root logins. The recognizer parses whole-document structure, +# so it selects the ROOT logins token exactly as PyYAML does — never the +# nested attacker token. (A prior line scan matched the FIRST logins at ANY +# indent and returned ATTACKER.) +write_fixture 'outer: + logins: + - name: primary + url: https://git.example + token: ATTACKER_NESTED +logins: + - name: primary + url: https://git.example + token: ROOT_TOK +' +assert_token "nested logins shadow selects ROOT token" "ROOT_TOK" primary git.example + +# 12. BLOCK-SCALAR-SHADOW: text inside a YAML literal/folded block ( | or > ) is +# an OPAQUE scalar to PyYAML (so `logins` is a string, not a list) and must +# not be scanned as live logins entries. Both fail closed. +write_fixture 'logins: | + - name: primary + url: https://git.example + token: ATTACKER_BLOCK +' +assert_token "block-scalar logins value fails closed" "" primary git.example +# A folded/literal block scalar anywhere is outside the recognizer's subset, so +# the fallback fails closed (conservative) even though PyYAML can still resolve +# the real root token past the opaque scalar. Fail-closed is the safe side. +write_fixture 'note: > + logins: + - name: primary + token: ATTACKER_FOLDED +logins: + - name: primary + url: https://git.example + token: ROOT_OK +' +assert_fallback_fails_closed "folded block scalar present fails closed" primary git.example + +# 13. DUPLICATE-ROOT / DUPLICATE-FIELD: a duplicated `logins:` root key (PyYAML +# last-wins) or a duplicated field within a login must fail closed rather +# than take the FIRST (stale) value. PyYAML resolves the LAST; the fallback +# refuses to guess. +write_fixture 'logins: + - name: primary + url: https://git.example + token: FIRST_DUP +logins: + - name: primary + url: https://git.example + token: LAST_DUP +' +assert_fallback_fails_closed "duplicate root logins key fails closed" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: FIRST_FIELD + token: SECOND_FIELD +' +assert_fallback_fails_closed "duplicate token field fails closed" primary git.example + +# 14. MALFORMED-AFTER-VALID: a syntax error LATER in the file makes PyYAML reject +# the WHOLE document; the recognizer must too (not emit the earlier token). +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +broken: a: b: c +' +assert_token "malformed line after valid login fails closed" "" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +broken: [unclosed +' +assert_token "unclosed flow after valid login fails closed" "" primary git.example + +# 15. EXTRA-DOCUMENT: a multi-document file (--- separator, or ... end marker) +# makes PyYAML safe_load reject multi-document input; the recognizer fails +# closed on ANY document marker rather than emit the first doc's token. +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +--- +logins: + - name: primary + url: https://git.example + token: SECOND_DOC +' +assert_token "second document (--- separator) fails closed" "" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +... +trailing: 1 +' +assert_token "end marker then more content fails closed" "" primary git.example + echo "Gitea login resolution regression harness passed" diff --git a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh index 05d70ba9..b0178385 100755 --- a/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh +++ b/packages/mosaic/framework/tools/git/test-issue-comment-readback.sh @@ -52,6 +52,8 @@ BIN_DIR="$WORK_DIR/bin" XDG_DIR="$WORK_DIR/xdg" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" +# Full curl argv per invocation — proves the bearer token never rides in argv. +CURL_ARGV_LOG="$WORK_DIR/curl-argv.log" AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" @@ -155,17 +157,24 @@ cat > "$BIN_DIR/curl" <<'SH' #!/usr/bin/env bash set -euo pipefail +# Record the FULL argv exactly as spawned, before consumption. The bearer token +# must NOT appear here — it is delivered via a curl --config file (#865 ITEM 3a), +# so only the config file PATH may show up. +printf '%s\n' "$*" >> "$ISSUE_COMMENT_CURL_ARGV_LOG" + output_file="" method="GET" url="" data="" auth_token="" +config_file="" while [[ $# -gt 0 ]]; do case "$1" in -o) output_file="$2"; shift 2 ;; -H) [[ "$2" == Authorization:* ]] && auth_token="${2##* }" shift 2 ;; + -K|--config) config_file="$2"; shift 2 ;; -w) shift 2 ;; -X) method="$2"; shift 2 ;; -d|--data) data="$2"; shift 2 ;; @@ -175,6 +184,17 @@ while [[ $# -gt 0 ]]; do esac done +# Resolve the bearer token from the curl --config file (its real, secure source); +# fall back to an -H header only for defense in depth. The config line is +# `header = "Authorization: token "`. +if [[ -z "$auth_token" && -n "$config_file" && -f "$config_file" ]]; then + config_hdr="$(grep -i 'Authorization' "$config_file" 2>/dev/null || true)" + if [[ "$config_hdr" == *"token "* ]]; then + auth_token="${config_hdr##*token }" + auth_token="${auth_token%\"}" + fi +fi + path="${url%%\?*}" query="${url#*\?}" [[ "$query" == "$url" ]] && query="" @@ -338,6 +358,7 @@ run_comment() { shift : > "$TEA_LOG" : > "$CURL_LOG" + : > "$CURL_ARGV_LOG" : > "$AUTH_LOG" : > "$OUTPUT_FILE" seed_state "$mode" @@ -349,6 +370,7 @@ run_comment() { MOSAIC_CREDENTIALS_FILE="$CREDENTIALS_FILE" \ ISSUE_COMMENT_TEA_LOG="$TEA_LOG" \ ISSUE_COMMENT_CURL_LOG="$CURL_LOG" \ + ISSUE_COMMENT_CURL_ARGV_LOG="$CURL_ARGV_LOG" \ ISSUE_COMMENT_AUTH_LOG="$AUTH_LOG" \ ISSUE_COMMENT_STATE="$STATE_FILE" \ ISSUE_COMMENT_TEST_MODE="$mode" \ @@ -371,7 +393,9 @@ run_comment() { # clobbered/leaked RETURN trap is caught on every exit route. assert_no_temp_leak() { local context="$1" leaked - leaked=$(find "$TMP_SCRATCH" -type f -name 'mosaic-issue-comment-*' 2>/dev/null || true) + # Includes the curl auth-config files (mosaic-gitea-auth-*), which carry the + # bearer token and must be unlinked on every exit path. + leaked=$(find "$TMP_SCRATCH" -type f \( -name 'mosaic-issue-comment-*' -o -name 'mosaic-gitea-auth-*' \) 2>/dev/null || true) if [[ -n "$leaked" ]]; then echo "FAIL: issue-comment temp files leaked ($context):" >&2 printf '%s\n' "$leaked" >&2 @@ -379,6 +403,21 @@ assert_no_temp_leak() { fi } +# Assert the presented bearer token NEVER appeared in curl's argv (it must travel +# via a curl --config file), and that --config auth was actually used. On the +# expected path grep matches nothing, so no token value is ever printed. +assert_token_not_in_argv() { + local context="$1" + if grep -qF -e "$DEFAULT_TOKEN" -e "$OVERRIDE_TOKEN" -e "$CROSS_HOST_TOKEN" "$CURL_ARGV_LOG"; then + echo "FAIL: a Gitea bearer token leaked into curl argv ($context)" >&2 + exit 1 + fi + if ! grep -q -- '--config' "$CURL_ARGV_LOG"; then + echo "FAIL: curl was not invoked with --config file auth ($context)" >&2 + exit 1 + fi +} + # Case 1: a genuine REST create (id 51) is verified end to end via its exact # provider-returned id — no list enumeration is involved. run_comment fresh-success @@ -404,6 +443,9 @@ grep -q "^POST $API_BASE/issues/7/comments $ACTING_LOGIN$" "$AUTH_LOG" grep -q "^GET $API_BASE/issues/comments/51 $ACTING_LOGIN$" "$AUTH_LOG" # Success path leaves no scratch temp files behind. assert_no_temp_leak "fresh-success" +# ITEM 3a: the token drove the write/read-back chain but never appeared in curl +# argv — it was passed via a curl --config file. +assert_token_not_in_argv "fresh-success default-token" # Case 2: a no-op write with a concurrent SAME-IDENTITY, same-body comment # already present must FAIL CLOSED — the closed concurrency window. diff --git a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh index adef8c2c..90f57c85 100644 --- a/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh +++ b/packages/mosaic/framework/tools/git/test-pr-review-gitea-comment.sh @@ -45,8 +45,13 @@ STATE_DIR="$WORK_DIR/state" REVIEWS_FILE="$STATE_DIR/reviews.json" COMMENTS_FILE="$STATE_DIR/comments.json" SUBMIT_PAYLOAD_FILE="$STATE_DIR/review_payload.json" +# Counts GET /pulls/{n} calls within a single run so a race mode can advance the +# reported head between the pre-submit read and the post-verify re-read. +HEAD_CALLS_FILE="$STATE_DIR/head_calls" TEA_LOG="$WORK_DIR/tea.log" CURL_LOG="$WORK_DIR/curl.log" +# Full curl argv per invocation — proves the bearer token never rides in argv. +CURL_ARGV_LOG="$WORK_DIR/curl-argv.log" AUTH_LOG="$WORK_DIR/auth.log" OUTPUT_FILE="$WORK_DIR/output.log" CREDENTIALS_FILE="$WORK_DIR/credentials.json" @@ -143,17 +148,24 @@ cat > "$BIN_DIR/curl" <<'SH' #!/usr/bin/env bash set -euo pipefail +# Record the FULL argv exactly as spawned, BEFORE any consumption. The bearer +# token must NOT appear here — it is delivered via a curl --config file, so only +# the config file PATH may show up. (#865 ITEM 3a credential-in-argv exposure.) +printf '%s\n' "$*" >> "$PR_REVIEW_CURL_ARGV_LOG" + output_file="" method="GET" payload="" url="" auth_token="" +config_file="" while [[ $# -gt 0 ]]; do case "$1" in -o) output_file="$2"; shift 2 ;; -H) [[ "$2" == Authorization:* ]] && auth_token="${2##* }" shift 2 ;; + -K|--config) config_file="$2"; shift 2 ;; -w) shift 2 ;; -X) method="$2"; shift 2 ;; -d|--data) payload="$2"; shift 2 ;; @@ -163,6 +175,17 @@ while [[ $# -gt 0 ]]; do esac done +# Resolve the bearer token from the curl --config file (its real, secure source); +# only fall back to an -H header for defense in depth. The config line is +# `header = "Authorization: token "`. +if [[ -z "$auth_token" && -n "$config_file" && -f "$config_file" ]]; then + config_hdr="$(grep -i 'Authorization' "$config_file" 2>/dev/null || true)" + if [[ "$config_hdr" == *"token "* ]]; then + auth_token="${config_hdr##*token }" + auth_token="${auth_token%\"}" + fi +fi + path="${url%%\?*}" query="${url#*\?}" [[ "$query" == "$url" ]] && query="" @@ -207,7 +230,25 @@ elif [[ "$method" == "GET" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123 write_response 200 "$(PR_REVIEW_HEAD_SHA="$PR_REVIEW_HEAD_SHA" python3 - <<'PY' import json import os -print(json.dumps({"head": {"sha": os.environ["PR_REVIEW_HEAD_SHA"]}})) + +head = os.environ["PR_REVIEW_HEAD_SHA"] +mode = os.environ.get("PR_REVIEW_TEST_MODE", "") +calls_path = os.environ.get("PR_REVIEW_HEAD_CALLS", "") +# Count GET /pulls/{n} calls within this run: call 1 is the pre-submit head read +# that pins the review; call 2+ is the post-verify re-read (current-head TOCTOU +# close-out). In the race mode the branch "advances" after the pin. +n = 1 +if calls_path: + try: + with open(calls_path, encoding="utf-8") as handle: + n = int(handle.read() or "0") + 1 + except (OSError, ValueError): + n = 1 + with open(calls_path, "w", encoding="utf-8") as handle: + handle.write(str(n)) +if mode == "head-advanced-race" and n >= 2: + head = "HEADSHA_ADVANCED_DEADBEEF" +print(json.dumps({"head": {"sha": head}})) PY )" elif [[ "$method" == "POST" && "$path" == "$PR_REVIEW_EXPECTED_API_BASE/pulls/123/reviews" ]]; then @@ -254,6 +295,26 @@ if mode == "review-body-reuse": print(json.dumps(record)) raise SystemExit(0) +# review-body-null (#865 ITEM 3b): a non-empty body was submitted but the +# persisted review carries body == null. id/author/state/head all line up; only +# strict presence + string-type body verification catches the lost body. The old +# `(body or "")` coalesce would have treated null as an empty string and passed. +if mode == "review-body-null": + new_id = (max((r["id"] for r in reviews), default=0)) + 1 + record = { + "id": new_id, + "state": submitted.get("event"), + "commit_id": submitted.get("commit_id"), + "body": None, + "user": {"login": acting}, + } + reviews.append(record) + with open(state_path, "w", encoding="utf-8") as handle: + json.dump(reviews, handle) + print("201") + print(json.dumps(record)) + raise SystemExit(0) + author = foreign if mode == "author-mismatch-review" else acting new_id = (max((r["id"] for r in reviews), default=0)) + 1 record = { @@ -313,6 +374,23 @@ body = json.loads(os.environ["PR_REVIEW_PAYLOAD"]).get("body") # issue_url is left empty. This is what the wrapper must tolerate — it must NOT # require an API-shaped issue_url. pr_url = f"{web_base}/pulls/123" +# comment-plain-issue (#865 ITEM 2): #123 is a plain ISSUE, not a PR. POST +# /issues/123/comments lands an issue comment whose issue_url is set and +# pull_request_url is empty. The pr-review `comment` action MUST reject this — it +# claimed a PR comment, so a bare issue_url is not acceptable proof. +if mode == "comment-plain-issue": + record = { + "id": 456, + "body": body, + "user": {"login": acting}, + "issue_url": f"{web_base}/issues/123", + "pull_request_url": "", + } + with open(state_path, "w", encoding="utf-8") as handle: + json.dump([record], handle) + print("201") + print(json.dumps(record)) + raise SystemExit(0) # URL-injection modes (#865 Blocker 3): id/author/body are all correct but the # provider-returned pull_request_url is forged, so ONLY origin+full-path # verification can catch them. @@ -377,7 +455,7 @@ chmod +x "$BIN_DIR/curl" seed_state() { local mode="$1" printf '[]' > "$COMMENTS_FILE" - rm -f "$SUBMIT_PAYLOAD_FILE" + rm -f "$SUBMIT_PAYLOAD_FILE" "$HEAD_CALLS_FILE" PR_REVIEW_SEED_MODE="$mode" PR_REVIEW_SEED_ACTING="$ACTING_LOGIN" \ PR_REVIEW_SEED_HEAD="$HEAD_SHA" python3 - "$REVIEWS_FILE" <<'PY' import json @@ -424,6 +502,7 @@ run_review() { write_credentials "$configured_url" : > "$TEA_LOG" : > "$CURL_LOG" + : > "$CURL_ARGV_LOG" : > "$AUTH_LOG" : > "$OUTPUT_FILE" seed_state "$mode" @@ -436,10 +515,12 @@ run_review() { PR_REVIEW_TEA_LOG="$TEA_LOG" \ PR_REVIEW_LOGIN_URL="${configured_url%/}" \ PR_REVIEW_CURL_LOG="$CURL_LOG" \ + PR_REVIEW_CURL_ARGV_LOG="$CURL_ARGV_LOG" \ PR_REVIEW_AUTH_LOG="$AUTH_LOG" \ PR_REVIEW_REVIEWS="$REVIEWS_FILE" \ PR_REVIEW_COMMENTS="$COMMENTS_FILE" \ PR_REVIEW_SUBMIT_PAYLOAD="$SUBMIT_PAYLOAD_FILE" \ + PR_REVIEW_HEAD_CALLS="$HEAD_CALLS_FILE" \ PR_REVIEW_TEST_MODE="$mode" \ PR_REVIEW_EXPECTED_BODY="$comment" \ PR_REVIEW_EXPECTED_API_BASE="$expected_api_base" \ @@ -462,7 +543,9 @@ run_review() { # clobbered/leaked RETURN trap is caught on every exit route. assert_no_temp_leak() { local context="$1" leaked - leaked=$(find "$TMP_SCRATCH" -type f -name 'mosaic-pr-review-*' 2>/dev/null || true) + # Includes the curl auth-config files (mosaic-gitea-auth-*), which carry the + # bearer token and must be unlinked on every exit path. + leaked=$(find "$TMP_SCRATCH" -type f \( -name 'mosaic-pr-review-*' -o -name 'mosaic-gitea-auth-*' \) 2>/dev/null || true) if [[ -n "$leaked" ]]; then echo "FAIL: pr-review temp files leaked ($context):" >&2 printf '%s\n' "$leaked" >&2 @@ -470,6 +553,21 @@ assert_no_temp_leak() { fi } +# Assert the presented bearer token NEVER appeared in curl's argv (it must travel +# via a curl --config file), and that --config auth was actually used. On the +# expected path grep matches nothing, so no token value is ever printed. +assert_token_not_in_argv() { + local context="$1" + if grep -qF -e "$DEFAULT_TOKEN" -e "$OVERRIDE_TOKEN" -e "$CROSS_HOST_TOKEN" "$CURL_ARGV_LOG"; then + echo "FAIL: a Gitea bearer token leaked into curl argv ($context)" >&2 + exit 1 + fi + if ! grep -q -- '--config' "$CURL_ARGV_LOG"; then + echo "FAIL: curl was not invoked with --config file auth ($context)" >&2 + exit 1 + fi +} + assert_no_tea_write() { # tea must only ever be used for the login list, never to write. if grep -qvE '^login list --output json$' "$TEA_LOG"; then @@ -499,6 +597,9 @@ grep -q "^POST https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/ grep -q "^GET https://git.mosaicstack.dev/api/v1/user $ACTING_LOGIN\$" "$AUTH_LOG" assert_no_tea_write assert_no_temp_leak "approve" +# ITEM 3a: the host-default token drove this whole chain, yet never appeared in +# any curl argv — it was passed via a curl --config file. +assert_token_not_in_argv "approve default-token" # The submitted review payload carries the event and the PR head commit_id. PR_REVIEW_HEAD_SHA="$HEAD_SHA" python3 - "$SUBMIT_PAYLOAD_FILE" <<'PY' import json @@ -673,6 +774,8 @@ if grep -q " $ACTING_LOGIN\$" "$AUTH_LOG"; then exit 1 fi assert_no_tea_write +# ITEM 3a: the override token likewise never leaked into curl argv. +assert_token_not_in_argv "override-success override-token" # Case 9 (#865 Round-4): an UNRESOLVABLE explicit --login override (a name absent # from the tea config) must FAIL CLOSED — nonzero exit, no success line, no review @@ -765,4 +868,56 @@ for bad_mode in comment-url-wrong-host comment-url-wrong-owner comment-url-wrong assert_no_temp_leak "$bad_mode" done +# Case 16 (#865 ITEM 1, current-head TOCTOU): the PR head advances between the +# pre-submit head read (which pins the review) and the post-verify re-read. The +# review is genuinely created and verified as pinned to the OLD head, but the +# live tip has moved on, so the wrapper must FAIL CLOSED rather than report a +# review that no longer covers the PR's current commit. +if run_review head-advanced-race approve; then + echo "FAIL: approve reported success though the PR head advanced after submit" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: current-head TOCTOU close-out did not fail closed on an advanced head" >&2 + exit 1 +fi +# The head was re-read after the submit/verify (2nd GET /pulls/123). +if [[ "$(grep -c '^GET https://git.mosaicstack.dev/api/v1/repos/mosaicstack/stack/pulls/123$' "$CURL_LOG")" -lt 2 ]]; then + echo "FAIL: wrapper did not re-read the PR head after review verification" >&2 + cat "$CURL_LOG" >&2 + exit 1 +fi +assert_no_temp_leak "head-advanced-race" + +# Case 17 (#865 ITEM 2): a claimed PR comment that actually lands as a plain +# ISSUE comment (issue #123 exists, PR #123 does not — issue_url set, +# pull_request_url empty) must FAIL CLOSED. The pr-review `comment` verifier +# requires a pull_request_url (kind=pulls) and rejects a bare issue_url. +if run_review comment-plain-issue comment durable-body; then + echo "FAIL: pr-review accepted a plain issue comment as a verified PR comment" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Added and verified comment' "$OUTPUT_FILE"; then + echo "FAIL: a plain issue_url satisfied the PR comment verifier" >&2 + exit 1 +fi +assert_no_temp_leak "comment-plain-issue" + +# Case 18 (#865 ITEM 3b): a non-empty review body submitted but persisted as null +# must FAIL CLOSED. id/author/state/head all match; only strict presence + +# string-type + exact body equality (not the old `(body or "")` coalesce) catches +# the lost body. +if run_review review-body-null approve real-submitted-review-body; then + echo "FAIL: review whose non-empty body persisted as null was accepted" >&2 + cat "$OUTPUT_FILE" >&2 + exit 1 +fi +if grep -q 'Approved and verified' "$OUTPUT_FILE"; then + echo "FAIL: a null persisted body passed strict review-body verification" >&2 + exit 1 +fi +assert_no_temp_leak "review-body-null" + echo "pr-review.sh REST review + comment create/read-back regression passed" -- 2.49.1 From 044744339b6d08b1c69fe8da75b6343a9540852b Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Tue, 21 Jul 2026 23:42:31 -0500 Subject: [PATCH 10/15] fix(865): fail closed on non-constructible typed scalars and bare indicators Round-8: the PyYAML-absent conservative recognizer proved STRUCTURE and implicit-resolver TYPE but not CONSTRUCTOR VALIDITY. PyYAML safe_load raises a constructor ValueError on the WHOLE document when a plain scalar matches a typed implicit resolver but is not constructible (e.g. bad calendar date 2023-99-99, empty-radix ints 0b_/0x_), yielding no token; the fallback instead treated such a scalar as a null-equivalent, ignored the malformed key, and still emitted the valid login token -- a credential fail-open in the dangerous direction. _scalar now checks constructibility via _constructible (replicating PyYAML 6.0.3 construct_yaml_int/float/timestamp, stdlib-only): a typed scalar that is not constructor-valid returns _FAIL, which the parser turns into _Bail so the WHOLE document fails closed exactly as PyYAML does. int and timestamp resolver patterns are byte-identical to PyYAML's; the float pattern is a strict superset whose extras are all float()-constructible (fuzz-verified 0/400k raise), so it never fails closed where PyYAML would emit. Also fail closed on plain scalars beginning with an indicator a plain scalar may not start with: '%' (directive) and ',' (flow), and the conditional block indicators '-'/'?'/':' when followed by whitespace or end-of-value (bare sequence/complex-key/value indicators PyYAML rejects), while '-x'/'-1'/'?x'/':x' remain valid plain strings. Differential fuzz through the real function: 0 fail-opens across 1499 diverse non-tab cases; targeted fixtures assert fallback == PyYAML == fail-closed for 2023-99-99, 2023-13-01, bad-hour timestamp, 0b_, 0x_, 0x__, %broken, ',bad', bare '-'/'- '/'? key', and assert the token STILL resolves for constructible/ string look-alikes (valid date/datetime, 0o_, 4.e8, 0x1f, -x, :x). Both PyYAML present and forced-absent. Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 156 +++++++++++++++++- .../tools/git/test-gitea-login-resolution.sh | 98 +++++++++++ 2 files changed, 251 insertions(+), 3 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 5ea889a7..0281b5ca 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -612,6 +612,7 @@ get_gitea_token_for_login() { [[ -f "$config_file" ]] || return 1 LOGIN_NAME="$login_name" REPO_HOST="$repo_host" python3 - "$config_file" <<'PY' +import datetime import os import re import sys @@ -671,6 +672,138 @@ def _implicit_nonstring(text): ) +# PyYAML 6.0.3 SafeConstructor timestamp regexp (yaml/constructor.py). The +# constructor RE-parses a timestamp-tagged scalar with THIS pattern and then +# builds a datetime.date/datetime, which raises ValueError for an out-of-range +# calendar field (e.g. month 99, hour 25). _IMPLICIT_TIMESTAMP (the RESOLVER +# pattern) is byte-identical to PyYAML's resolver, so anything it tags is also +# tagged by PyYAML and re-matched here. +_TIMESTAMP_CONSTRUCT = re.compile( + r"""^(?P[0-9][0-9][0-9][0-9]) + -(?P[0-9][0-9]?) + -(?P[0-9][0-9]?) + (?:(?:[Tt]|[ \t]+) + (?P[0-9][0-9]?) + :(?P[0-9][0-9]) + :(?P[0-9][0-9]) + (?:\.(?P[0-9]*))? + (?:[ \t]*(?PZ|(?P[-+])(?P[0-9][0-9]?) + (?::(?P[0-9][0-9]))?))?)?$""", + re.X, +) + + +def _int_constructible(text): + # Replicate PyYAML SafeConstructor.construct_yaml_int and report whether it + # would succeed. A resolver-tagged int whose radix body is empty after + # underscore removal (e.g. "0b_", "0x_", "0x__") makes int(base) raise, so + # PyYAML fails the WHOLE document -> the fallback must fail closed too. + value = text.replace("_", "") + if value[:1] in ("+", "-"): + value = value[1:] + if value == "0": + return True + try: + if value.startswith("0b"): + int(value[2:], 2) + elif value.startswith("0x"): + int(value[2:], 16) + elif value[:1] == "0": + int(value, 8) + elif ":" in value: + [int(part) for part in value.split(":")] + else: + int(value) + except ValueError: + return False + return True + + +def _float_constructible(text): + # Replicate PyYAML SafeConstructor.construct_yaml_float. Retained for the + # WHOLE-document invariant even though PyYAML's float-tagged set is always + # float()-constructible: the fallback's float RESOLVER pattern is a strict + # superset of PyYAML's (it also matches unsigned-exponent spellings PyYAML + # keeps as strings), and every such extra is likewise constructible, so this + # never fails closed where PyYAML would emit a token. + value = text.replace("_", "").lower() + if value[:1] in ("+", "-"): + value = value[1:] + if value in (".inf", ".nan"): + return True + try: + if ":" in value: + [float(part) for part in value.split(":")] + else: + float(value) + except ValueError: + return False + return True + + +def _timestamp_constructible(text): + # Replicate PyYAML SafeConstructor.construct_yaml_timestamp: build the same + # datetime.date/datetime and report whether it raises. Returns False for an + # out-of-range calendar field (month/day/hour/...), matching PyYAML's + # whole-document ValueError. + match = _TIMESTAMP_CONSTRUCT.match(text) + if not match: + return False + values = match.groupdict() + try: + year = int(values["year"]) + month = int(values["month"]) + day = int(values["day"]) + if not values["hour"]: + datetime.date(year, month, day) + return True + hour = int(values["hour"]) + minute = int(values["minute"]) + second = int(values["second"]) + fraction = 0 + if values["fraction"]: + frac = values["fraction"][:6] + frac += "0" * (6 - len(frac)) + fraction = int(frac) + tzinfo = None + if values["tz_sign"]: + tz_hour = int(values["tz_hour"]) + tz_minute = int(values["tz_minute"] or 0) + delta = datetime.timedelta(hours=tz_hour, minutes=tz_minute) + if values["tz_sign"] == "-": + delta = -delta + tzinfo = datetime.timezone(delta) + elif values["tz"]: + tzinfo = datetime.timezone.utc + datetime.datetime( + year, month, day, hour, minute, second, fraction, tzinfo=tzinfo + ) + except (ValueError, OverflowError): + return False + return True + + +def _constructible(text): + # WHOLE-DOCUMENT INVARIANT: the fallback resolves the SAME login token as + # PyYAML safe_load or fails closed -- never less conservative -- INCLUDING + # when PyYAML raises a CONSTRUCTOR error anywhere in the document. A plain + # scalar can match a typed implicit resolver (int/float/timestamp) yet NOT be + # constructible (e.g. 2023-99-99, 0b_, 0x_); PyYAML then raises on the whole + # load and yields no token, so the fallback MUST fail closed for the whole + # document too. This returns True only when PyYAML's constructor would build + # the scalar (null/bool token sets are always constructible), else False so + # the caller fails closed. `text` is assumed to satisfy _implicit_nonstring. + if _IMPLICIT_NULL.match(text) or _IMPLICIT_BOOL.match(text): + return True + if _IMPLICIT_TIMESTAMP.match(text): + return _timestamp_constructible(text) + if _IMPLICIT_INT.match(text): + return _int_constructible(text) + if _IMPLICIT_FLOAT.match(text): + return _float_constructible(text) + return True + + # Sentinel: "outside the supported subset -> fail closed". Distinct from a # genuine null (None), which is a valid resolved value. _FAIL = object() @@ -724,14 +857,31 @@ def _scalar(raw): value = value.strip() if not value: return None - if value[0] in _FLOW or value[0] in ("|", ">", "?", "@", "`", '"', "'"): - return _FAIL # flow / block-scalar / reserved / anchor / quote indicator + if value[0] in _FLOW or value[0] in ("|", ">", "?", "@", "`", '"', "'", "%", ","): + return _FAIL # flow / block-scalar / reserved / directive / anchor / quote + if value[0] in ("-", "?", ":") and (len(value) == 1 or value[1] in (" ", "\t")): + # A bare block indicator, not a plain scalar: '-'/'- ' opens a sequence + # entry, '?'/'? ' a complex mapping key, ':'/': ' a mapping value -- all + # illegal in a value position, where PyYAML raises a scanner error on the + # whole document. "-x"/"-1"/"?x"/":x" (indicator NOT followed by space) + # remain valid plain scalars and fall through. Fail closed on the bare + # indicator so the fallback never emits a token PyYAML would refuse. + return _FAIL if any(ch in _FLOW for ch in value): return _FAIL if ": " in value or value.endswith(":") or "\t" in value: return _FAIL # nested-mapping-in-scalar / ambiguous if _implicit_nonstring(value): - return None # non-string implicit type -> null-equivalent for a token + # A plain scalar PyYAML would tag as a non-string (null/bool/int/float/ + # timestamp). If PyYAML's CONSTRUCTOR would build it, the value is a + # non-string -> null-equivalent for a token field: return None and keep + # parsing (as before). But if it matches a typed implicit resolver yet is + # NOT constructible (e.g. 2023-99-99, 0b_, 0x_), PyYAML raises on the + # WHOLE document and yields no token, so the fallback MUST fail closed + # for the whole document too -> _FAIL (which the caller turns into _Bail). + if not _constructible(value): + return _FAIL + return None return value diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index c18eef0c..ad571b57 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -621,4 +621,102 @@ trailing: 1 ' assert_token "end marker then more content fails closed" "" primary git.example +# 16. CONSTRUCTOR-VALIDITY / INVALID-INDICATOR: a plain scalar can match a typed +# implicit resolver (int/float/timestamp) yet be NON-constructible, or begin +# with an indicator a plain scalar may not start with. PyYAML then RAISES on +# the WHOLE document (constructor error / scanner error) and yields NO token, +# so the fallback must ALSO fail closed for the whole document -- even though +# the (unrelated) malformed key sits alongside an otherwise-valid logins +# block whose token is itself well-formed. A prior residual proved STRUCTURE +# and implicit TYPE but not constructor validity, so it ignored the malformed +# key and still emitted the valid login token (fail-open in the dangerous +# direction). assert_both_fail_closed asserts fallback == PyYAML == no token. +assert_both_fail_closed() { + local desc="$1" login="$2" host="$3" got + got=$(token_fallback "$login" "$host") + if [[ -n "$got" ]]; then + echo "FAIL fallback [$desc]: expected fail-closed, got a token" >&2 + exit 1 + fi + if [[ "$HAVE_PYYAML" == true ]]; then + got=$(token_pyyaml "$login" "$host") + if [[ -n "$got" ]]; then + echo "FAIL pyyaml [$desc]: expected PyYAML to also fail closed (raise/no token), got a token" >&2 + exit 1 + fi + fi +} + +# write_bad_key_fixture: an unrelated root key carrying $1 as its plain scalar, +# followed by an otherwise-valid logins block whose token is well-formed. +write_bad_key_fixture() { + write_fixture "bad: $1 +logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +" +} + +# Non-constructible TIMESTAMP-tagged scalars: match the resolver, but the +# calendar field is out of range so PyYAML's datetime construction raises. +write_bad_key_fixture '2023-99-99' # month 99 / day 99 invalid +assert_both_fail_closed "bad-date 2023-99-99 fails closed like PyYAML" primary git.example +write_bad_key_fixture '2023-13-01' # month 13 invalid +assert_both_fail_closed "bad-month 2023-13-01 fails closed like PyYAML" primary git.example +write_bad_key_fixture '2023-01-15T25:00:00' # hour 25 invalid +assert_both_fail_closed "bad-hour timestamp fails closed like PyYAML" primary git.example + +# Non-constructible INT-tagged scalars: match the int resolver, but the radix +# body is empty after underscore removal so int(base) raises. +write_bad_key_fixture '0b_' +assert_both_fail_closed "empty-binary 0b_ fails closed like PyYAML" primary git.example +write_bad_key_fixture '0x_' +assert_both_fail_closed "empty-hex 0x_ fails closed like PyYAML" primary git.example +write_bad_key_fixture '0x__' +assert_both_fail_closed "empty-hex 0x__ (multi-underscore) fails closed" primary git.example + +# Invalid plain-scalar INDICATOR forms: a plain scalar may not begin with '%' +# (directive) or ',' (flow) -- PyYAML raises a scanner/parser error on the whole +# document, so the fallback fails closed on the leading indicator. +write_bad_key_fixture '%broken' +assert_both_fail_closed "leading-%% directive indicator fails closed" primary git.example +write_bad_key_fixture ',bad' +assert_both_fail_closed "leading-comma flow indicator fails closed" primary git.example +# Bare block indicators in a value position ('-'/'- ', '?'/'? ', ':'/': '): +# PyYAML raises a scanner error on the whole document, so the fallback must fail +# closed rather than accept the indicator as a plain-scalar string. +write_bad_key_fixture '-' +assert_both_fail_closed "bare dash (seq indicator) fails closed" primary git.example +write_bad_key_fixture '- x' +assert_both_fail_closed "dash-space (seq entry) fails closed" primary git.example +write_bad_key_fixture '? key' +assert_both_fail_closed "question-space (complex key) fails closed" primary git.example +# ...but an indicator NOT followed by whitespace is a valid plain scalar string, +# so the token still resolves (no over-broad fail-close). +write_bad_key_fixture '-x' +assert_token "dash-not-space is a plain string, token resolves" "TOK_PLAIN" primary git.example +write_bad_key_fixture ':x' +assert_token "colon-not-space is a plain string, token resolves" "TOK_PLAIN" primary git.example + +# NOT over-broad: a genuinely CONSTRUCTIBLE typed scalar (or a look-alike PyYAML +# keeps as a plain string) leaves the document valid, so BOTH still resolve the +# login token -- the fix must not fail closed on these. +write_bad_key_fixture '2023-01-15' +assert_token "valid date unrelated key still resolves token" "TOK_PLAIN" primary git.example +write_bad_key_fixture '2023-01-15 10:00:00' +assert_token "valid datetime unrelated key still resolves token" "TOK_PLAIN" primary git.example +# '0o_' is NOT matched by PyYAML's int resolver (YAML 1.1 octal is 0[0-7]+, not +# 0o...), so PyYAML keeps it a STRING and resolves the token; the fallback must +# agree (no spurious fail-close). +write_bad_key_fixture '0o_' +assert_token "0o_ is a plain string in PyYAML, token still resolves" "TOK_PLAIN" primary git.example +# '4.e8' matches the fallback's (superset) float pattern but PyYAML keeps it a +# string; either way it is constructible, so the token still resolves in both. +write_bad_key_fixture '4.e8' +assert_token "4.e8 float look-alike still resolves token" "TOK_PLAIN" primary git.example +# A valid radix int as an unrelated key must not fail closed. +write_bad_key_fixture '0x1f' +assert_token "valid hex int unrelated key still resolves token" "TOK_PLAIN" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1 From 6053e1ee4c531ea8906b9b8bb291300295ebf202 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 22 Jul 2026 00:05:39 -0500 Subject: [PATCH 11/15] fix(git): fail closed on tabs the YAML fallback recognizer would swallow (#865) PyYAML raises a ScannerError on a tab used anywhere outside a quoted scalar (leading/trailing/embedded in a plain value, immediately before or after a key colon, or as indentation) and yields no token, accepting tabs ONLY inside single/double-quoted scalars. The conservative block-YAML fallback in get_gitea_token_for_login normalized those tabs away -- via _scalar's top-level .strip(), _KEY_RE's [ \t] separators, _Parser.__init__'s body.rstrip(), parse_seq's body[1:].strip(), and the inline-map emptiness checks -- and still emitted the login token: a credential fail-open in the dangerous direction (less conservative than PyYAML). Extend the whole-document "only ever more conservative than PyYAML, never less" invariant to tab/scanner parity: - _KEY_RE now uses SPACE-only separators (` *:` / `[ ](.*)`), so a tab in a key/value separator makes the line fail to match and the caller fails closed. - Every whitespace-normalization site strips SPACES only (strip(" ")/rstrip(" ")) so a tab survives to a fail-closed guard instead of being silently removed: _scalar top-level strip, quoted-trailing strip, post-comment strip, _Parser.__init__ body rstrip, parse_seq item strip, and the three inline emptiness checks. - _scalar fails closed on any tab remaining in a plain scalar. Tabs strictly inside quoted scalars are preserved verbatim (unchanged parity), matching exactly what PyYAML accepts. Verified empirically against PyYAML 6.0.3: ScannerError for each rejected tab position; string-preserved for quoted inner tabs. Differential fuzz with tab re-included: 0 fail-opens over ~6000 inputs; 0 over-rejection across 220 PyYAML-accepted quoted-tab cases. Adds section 17 to the regression harness (fail-close fixtures for trailing/leading/embedded/after-colon/indentation tabs; parity fixtures for double- and single-quoted inner tabs). Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 37 ++++++++---- .../tools/git/test-gitea-login-resolution.sh | 57 +++++++++++++++++++ 2 files changed, 84 insertions(+), 10 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 0281b5ca..e552743f 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -807,7 +807,12 @@ def _constructible(text): # Sentinel: "outside the supported subset -> fail closed". Distinct from a # genuine null (None), which is a valid resolved value. _FAIL = object() -_KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*)[ \t]*:(?:[ \t](.*)|)$") +# Separators are SPACE-only: PyYAML rejects a tab used as key/value whitespace +# (before or after the ':') with a scanner error, so a tab there must NOT be +# treated as a benign separator. Space before the colon and one space after it +# stay valid (PyYAML strips a plain key's trailing spaces); a tab in either +# position makes the whole line fail to match -> the caller fails closed. +_KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*) *:(?:[ ](.*)|)$") _FLOW = set("[]{}*&!") @@ -830,7 +835,10 @@ def _scalar(raw): # subject to PyYAML's implicit typing: forms like 12345 / null / ~ / yes / # 3.14 / a timestamp resolve to a NON-string (int/None/bool/float/date), so # they return None here (PyYAML's path rejects a non-str token via _accept). - value = raw.strip() + # Strip SPACES only, never tabs: PyYAML raises a scanner error on a tab in a + # plain/leading/trailing scalar position, so a tab must be PRESERVED here to + # trip the fail-closed guard below rather than be silently normalized away. + value = raw.strip(" ") if not value: return None # empty plain scalar -> null if value[0] in ("'", '"'): @@ -838,7 +846,7 @@ def _scalar(raw): end = value.find(quote, 1) if end == -1: return _FAIL # unterminated quote: PyYAML would error / continue - rest = value[end + 1:].strip() + rest = value[end + 1:].strip(" ") if rest and not rest.startswith("#"): return _FAIL # trailing junk after a quoted scalar inner = value[1:end] @@ -854,7 +862,7 @@ def _scalar(raw): if ch == "#" and (i == 0 or value[i - 1] in (" ", "\t")): value = value[:i] break - value = value.strip() + value = value.strip(" ") # spaces only; a tab must survive to fail closed if not value: return None if value[0] in _FLOW or value[0] in ("|", ">", "?", "@", "`", '"', "'", "%", ","): @@ -869,7 +877,13 @@ def _scalar(raw): return _FAIL if any(ch in _FLOW for ch in value): return _FAIL - if ": " in value or value.endswith(":") or "\t" in value: + if "\t" in value: + # A tab anywhere in a plain scalar (leading, trailing, or embedded) is a + # PyYAML scanner error on the whole document -- it accepts tabs ONLY + # inside quoted scalars (handled above, returned verbatim). Fail closed + # so the fallback never emits a token PyYAML would refuse over a tab. + return _FAIL + if ": " in value or value.endswith(":"): return _FAIL # nested-mapping-in-scalar / ambiguous if _implicit_nonstring(value): # A plain scalar PyYAML would tag as a non-string (null/bool/int/float/ @@ -913,7 +927,10 @@ class _Parser: continue if re.match(r"^(---|\.\.\.)(\s|$)", body) or body in ("---", "..."): raise _Bail() # document / end marker -> multi-doc -> fail closed - self.toks.append((indent, body.rstrip())) + # rstrip SPACES only: a trailing tab is a PyYAML scanner error, so it + # must be kept on the token body to reach the fail-closed guards + # (rstrip() would swallow it and let a bad line resolve a token). + self.toks.append((indent, body.rstrip(" "))) self.i = 0 def peek(self): @@ -959,7 +976,7 @@ class _Parser: self.i += 1 if key in result: raise _Bail() # duplicate mapping key (PyYAML last-wins; we bail) - if inline is not None and inline.strip() != "": + if inline is not None and inline.strip(" ") != "": val = _scalar(inline) if val is _FAIL: raise _Bail() @@ -997,7 +1014,7 @@ class _Parser: raise _Bail() if not (body.startswith("-") and (len(body) == 1 or body[1] in (" ", "\t"))): break # a mapping entry at this indent ends the sequence - rest = body[1:].strip() + rest = body[1:].strip(" ") # spaces only; a tab must survive to bail self.i += 1 if rest == "": nxt = self.peek() @@ -1023,7 +1040,7 @@ class _Parser: result = {} key = first_match.group(1) inline = first_match.group(2) - if inline is not None and inline.strip() != "": + if inline is not None and inline.strip(" ") != "": val = _scalar(inline) if val is _FAIL: raise _Bail() @@ -1049,7 +1066,7 @@ class _Parser: self.i += 1 if k in result: raise _Bail() - if iv is not None and iv.strip() != "": + if iv is not None and iv.strip(" ") != "": v = _scalar(iv) if v is _FAIL: raise _Bail() diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index ad571b57..c6d4bdb6 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -719,4 +719,61 @@ assert_token "4.e8 float look-alike still resolves token" "TOK_PLAIN" primary gi write_bad_key_fixture '0x1f' assert_token "valid hex int unrelated key still resolves token" "TOK_PLAIN" primary git.example +# 17. TAB / SCANNER PARITY: PyYAML raises a ScannerError on a tab used anywhere +# outside a quoted scalar -- leading, trailing, or embedded in a plain value, +# immediately after a key colon, before a key colon, or as indentation -- and +# yields NO token, accepting tabs ONLY inside single/double-quoted scalars +# (where the tab is preserved as string content). A prior fallback swallowed +# those tabs (via .strip()/.rstrip() normalization and [ \t] key separators) +# and still emitted the login token -- a fail-open in the dangerous direction. +# The recognizer now fails CLOSED for the whole document on any tab PyYAML +# rejects, while preserving the tabs PyYAML keeps (inside quotes). All tab +# positions were verified empirically against PyYAML 6.0.3 (ScannerError for +# each rejected position; string-preserved for quoted inner tabs). +TAB=$'\t' +# Fail-close: a tab in a plain value position (trailing / leading / embedded). +write_bad_key_fixture "l4o${TAB}" +assert_both_fail_closed "trailing tab in plain value fails closed" primary git.example +write_bad_key_fixture "${TAB}9" +assert_both_fail_closed "leading tab in plain value fails closed" primary git.example +write_bad_key_fixture "a${TAB}b" +assert_both_fail_closed "embedded tab in plain value fails closed" primary git.example +# Fail-close: a tab immediately after the key colon (no separating space). +write_fixture "bad:${TAB}9 +logins: + - name: primary + url: https://git.example + token: TOK_PLAIN +" +assert_both_fail_closed "tab immediately after key colon fails closed" primary git.example +# Fail-close: a tab used as indentation (before a sequence dash). +write_fixture "logins: +${TAB}- name: primary + url: https://git.example + token: TOK_PLAIN +" +assert_both_fail_closed "tab used as indentation fails closed" primary git.example +# Fail-close: a tab trailing a sequence-mapping field value. +write_fixture "logins: + - name: primary + url: https://git.example + token: TOK_PLAIN${TAB} +" +assert_both_fail_closed "tab trailing a seq field value fails closed" primary git.example +# NOT over-broad: a tab strictly INSIDE a quoted scalar is valid YAML (PyYAML +# keeps it as string content), so the document parses and the login token still +# resolves in BOTH paths -- double-quoted and single-quoted. +write_bad_key_fixture "\"a${TAB}b\"" +assert_token "tab inside a double-quoted value still resolves token" "TOK_PLAIN" primary git.example +write_bad_key_fixture "'a${TAB}b'" +assert_token "tab inside a single-quoted value still resolves token" "TOK_PLAIN" primary git.example +# ...and a quoted token value carrying an inner tab resolves to the exact string +# (tab preserved), identical to PyYAML's construction. +write_fixture "logins: + - name: primary + url: https://git.example + token: \"T${TAB}OK\" +" +assert_token "quoted token with inner tab resolves verbatim" "T${TAB}OK" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1 From acf7955f872589cb4f0319fc5f537f020dd63f2d Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 22 Jul 2026 00:39:37 -0500 Subject: [PATCH 12/15] fix(git): fail closed on forbidden control chars, fix float exponent + '?' over-rejection (#865) Round-9 auditor blockers, both fixed at root cause in the PyYAML-absent line-parser fallback of get_gitea_token_for_login: 1. Control-character fail-open (dangerous): PyYAML 6.0.3's Reader rejects the WHOLE document (ReaderError) if a forbidden C0/DEL control byte {0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F, 0x7F} appears ANYWHERE in the raw stream -- plain scalar, inside quotes, or a comment -- regardless of position, verified empirically against the real installed PyYAML 6.0.3. The fallback previously only guarded tabs and emitted the login token from such documents. Added a whole-document _FORBIDDEN_CONTROL scan on the raw text (before splitlines(), which itself splits on some of the same bytes) so the fallback fails closed identically to PyYAML. 2. Unsigned-exponent float over-rejection: PyYAML's implicit float resolver requires an EXPLICIT sign on the exponent; unsigned-exponent spellings (1.0e10, +1.0e10, -1.0e10, 1.0E10, .5e10, 4.e8) are PyYAML STRINGS, not floats. The fallback's _IMPLICIT_FLOAT pattern allowed an optional sign, misclassifying these as floats and dropping the token. Tightened the regex to match PyYAML's resolver exactly. Also folds in a residual over-rejection found via this round's wide differential fuzz (1767 cases, 0 fail-opens / 0 over-rejections after the fix): a plain scalar starting with "?" not followed by whitespace (e.g. "?x") is a valid PyYAML string, but the fallback's blanket-reject set treated every leading "?" as illegal. Narrowed to match the existing space-aware handling already used for "-" and ":". Adds regression fixtures to test-gitea-login-resolution.sh covering all three fixes under both PyYAML-present and forced-ImportError-absent runs. Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 66 ++++++++-- .../tools/git/test-gitea-login-resolution.sh | 117 ++++++++++++++++++ 2 files changed, 171 insertions(+), 12 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index e552743f..7d55ad10 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -640,9 +640,14 @@ _IMPLICIT_INT = re.compile( r"|[-+]?0x[0-9a-fA-F_]+" r"|[-+]?[1-9][0-9_]*(?::[0-5]?[0-9])+)$" ) +# NOTE: the exponent group requires an EXPLICIT sign ([eE][-+][0-9]+), matching +# PyYAML 6.0.3's resolver.py float regex exactly. An unsigned exponent (e.g. +# "1.0e10", ".5e10", "4.e8") is NOT matched by PyYAML's implicit float resolver +# -- PyYAML resolves those as plain strings -- so this pattern must not match +# them either, or the fallback over-rejects a token PyYAML would emit verbatim. _IMPLICIT_FLOAT = re.compile( - r"^(?:[-+]?(?:[0-9][0-9_]*)\.[0-9_]*(?:[eE][-+]?[0-9]+)?" - r"|\.[0-9][0-9_]*(?:[eE][-+]?[0-9]+)?" + r"^(?:[-+]?(?:[0-9][0-9_]*)\.[0-9_]*(?:[eE][-+][0-9]+)?" + r"|\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?" r"|[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\.[0-9_]*" r"|[-+]?\.(?:inf|Inf|INF)" r"|\.(?:nan|NaN|NAN))$" @@ -660,9 +665,10 @@ def _implicit_nonstring(text): # True when an UNQUOTED plain scalar would be resolved by PyYAML's SafeLoader # to a non-string type (null/bool/int/float/timestamp). Fuzzed against real # PyYAML 6.0.3: it never returns False where PyYAML types the scalar as a - # non-string (i.e. never fail-open), and is at worst MORE conservative on a - # couple of degenerate float spellings (e.g. "4.e8") that PyYAML keeps as a - # string -- the safe direction for this credential-selecting fallback. + # non-string (i.e. never fail-open), and (post exponent-sign fix) no longer + # over-rejects unsigned-exponent spellings like "4.e8" or "1.0e10" -- those + # match PyYAML's own implicit float resolver exactly (explicit sign only), + # so PyYAML keeps them as strings and this function now agrees. return bool( _IMPLICIT_NULL.match(text) or _IMPLICIT_BOOL.match(text) @@ -721,11 +727,10 @@ def _int_constructible(text): def _float_constructible(text): # Replicate PyYAML SafeConstructor.construct_yaml_float. Retained for the - # WHOLE-document invariant even though PyYAML's float-tagged set is always - # float()-constructible: the fallback's float RESOLVER pattern is a strict - # superset of PyYAML's (it also matches unsigned-exponent spellings PyYAML - # keeps as strings), and every such extra is likewise constructible, so this - # never fails closed where PyYAML would emit a token. + # WHOLE-document invariant: _IMPLICIT_FLOAT now matches PyYAML's resolver + # pattern exactly (explicit-sign exponent only), and every scalar it tags + # is float()-constructible, so this never fails closed where PyYAML would + # emit a token. value = text.replace("_", "").lower() if value[:1] in ("+", "-"): value = value[1:] @@ -804,6 +809,22 @@ def _constructible(text): return True +# PyYAML's Reader.check_printable scans the ENTIRE raw input stream (not just +# scalar contents, and NOT scoped by quoting) for bytes outside its printable +# set and raises ReaderError -- a whole-document reject -- the instant one is +# found, no matter where it sits (an unrelated field, a comment, inside or +# outside quotes). Empirically verified against real PyYAML 6.0.3: EVERY C0 +# control byte below plus DEL (0x7F) rejects in plain scalars, inside +# double-quoted scalars, and inside single-quoted scalars alike; only +# TAB(0x09), LF(0x0A), and CR(0x0D) are accepted among the C0 range (LF/CR are +# line separators, handled elsewhere; TAB has its own narrower, position-aware +# guard above since PyYAML accepts it in some contexts). This set therefore +# excludes 0x09/0x0A/0x0D and matches the forbidden C0 set {0x00-0x08, 0x0B, +# 0x0C, 0x0E-0x1F} plus DEL (0x7F). +_FORBIDDEN_CONTROL = re.compile( + "[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]" +) + # Sentinel: "outside the supported subset -> fail closed". Distinct from a # genuine null (None), which is a valid resolved value. _FAIL = object() @@ -865,8 +886,15 @@ def _scalar(raw): value = value.strip(" ") # spaces only; a tab must survive to fail closed if not value: return None - if value[0] in _FLOW or value[0] in ("|", ">", "?", "@", "`", '"', "'", "%", ","): + if value[0] in _FLOW or value[0] in ("|", ">", "@", "`", '"', "'", "%", ","): return _FAIL # flow / block-scalar / reserved / directive / anchor / quote + # NOTE: "?" is deliberately NOT in the blanket-reject set above. Empirically + # verified against real PyYAML 6.0.3: "?x" (indicator immediately followed by + # a non-space) is a plain scalar string "?x" like "-x" and ":x" are -- only + # "?" alone or "? " (followed by space/EOL) opens a complex mapping key and + # is illegal in a value position. Blanket-rejecting every leading "?" would + # over-reject a token PyYAML accepts verbatim; the narrower check below + # (mirroring "-" and ":") handles exactly the illegal bare-indicator forms. if value[0] in ("-", "?", ":") and (len(value) == 1 or value[1] in (" ", "\t")): # A bare block indicator, not a plain scalar: '-'/'- ' opens a sequence # entry, '?'/'? ' a complex mapping key, ':'/': ' a mapping value -- all @@ -1151,7 +1179,21 @@ def _token_via_lines(): # whose `name` equals the request -> host/port-bound token), so the fallback # can only ever be MORE conservative than PyYAML, never less. with open(config_path, encoding="utf-8") as handle: - lines = handle.read().splitlines() + raw_text = handle.read() + + # Whole-document, position-independent reject: PyYAML's Reader rejects the + # ENTIRE document (ReaderError) if a forbidden C0/DEL control byte appears + # ANYWHERE in the raw stream -- in a plain scalar, inside single- or + # double-quoted scalars, in a comment, or in a field this recognizer never + # even looks at. Checking the raw text (before splitlines(), which itself + # splits on some of these same control bytes -- e.g. \x0b, \x0c, \x1c-\x1e + # -- and would otherwise obscure them) mirrors that exactly: fail closed + # for the whole document rather than risk emitting a token PyYAML would + # have refused. + if _FORBIDDEN_CONTROL.search(raw_text): + return None + + lines = raw_text.splitlines() config = _safe_parse(lines) if config is _FAIL: diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index c6d4bdb6..1445f924 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -776,4 +776,121 @@ write_fixture "logins: " assert_token "quoted token with inner tab resolves verbatim" "T${TAB}OK" primary git.example +# 18. CONTROL-CHARACTER FAIL-CLOSE (#865 round-9 blocker 1): PyYAML's Reader +# scans the ENTIRE raw document stream (not merely scalar contents, and NOT +# scoped by quoting) for bytes outside its printable set and raises +# ReaderError -- a WHOLE-DOCUMENT reject -- the instant one is found, +# regardless of where it sits: an unrelated field's plain scalar, inside a +# double- or single-quoted scalar, or a comment. Verified empirically against +# real installed PyYAML 6.0.3 (see detect-platform.sh's _FORBIDDEN_CONTROL +# comment): every C0 control byte {0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F} plus DEL +# (0x7F) rejects in ALL THREE contexts (plain / double-quoted / single-quoted); +# only TAB(0x09), LF(0x0A), CR(0x0D) are accepted among the low byte range +# (TAB has its own narrower, position-aware coverage in section 17 above; LF/CR +# are line separators). A prior fallback ONLY guarded tabs and emitted the +# login token from documents PyYAML rejects over an UNRELATED field's control +# byte -- a dangerous fail-open (credential emission from a document PyYAML +# refuses). write_control_char_fixture writes the raw byte directly via +# printf's octal escape (never through a bash string/variable, which cannot +# hold an embedded NUL) so 0x00 is exercised faithfully alongside the rest. +write_control_char_fixture() { + local octal="$1" quote="${2:-}" + { + if [[ -n "$quote" ]]; then + printf 'bad: %sx' "$quote" + # shellcheck disable=SC2059 # deliberate: $octal supplies printf's + # own \NNN octal escape so the raw control byte reaches the file + # directly, never passing through a bash string (which truncates + # at an embedded NUL and so cannot represent byte 0x00 otherwise). + printf "\\${octal}" + printf 'y%s\n' "$quote" + else + printf 'bad: x' + # shellcheck disable=SC2059 # deliberate: $octal supplies printf's + # own \NNN octal escape so the raw control byte reaches the file + # directly, never passing through a bash string (which truncates + # at an embedded NUL and so cannot represent byte 0x00 otherwise). + printf "\\${octal}" + printf 'y\n' + fi + printf 'logins:\n - name: primary\n url: https://git.example\n token: TOK_PLAIN\n' + } > "$FIXTURE_XDG/tea/config.yml" +} + +# Plain (unquoted) unrelated-field placement: the full empirically-confirmed +# forbidden C0/DEL set. +for octal in 000 001 002 003 004 005 006 007 010 013 014 \ + 016 017 020 021 022 023 024 025 026 027 \ + 030 031 032 033 034 035 036 037 177; do + write_control_char_fixture "$octal" + assert_both_fail_closed "control byte \\$octal in unrelated plain field fails closed" primary git.example +done + +# Inside-quote variants (double and single) for the six bytes called out +# explicitly in the round-9 blocker report: 0x00,0x01,0x07,0x0e,0x1f,0x7f. +for octal in 000 001 007 016 037 177; do + write_control_char_fixture "$octal" '"' + assert_both_fail_closed "control byte \\$octal inside double-quoted unrelated field fails closed" primary git.example + write_control_char_fixture "$octal" "'" + assert_both_fail_closed "control byte \\$octal inside single-quoted unrelated field fails closed" primary git.example +done + +# Same forbidden byte inside a comment line -- PyYAML's Reader check is +# stream-wide, so it rejects here too, not merely inside live scalar content. +{ + printf '# note' + printf '\007' + printf 'here\nlogins:\n - name: primary\n url: https://git.example\n token: TOK_PLAIN\n' +} > "$FIXTURE_XDG/tea/config.yml" +assert_both_fail_closed "control byte in a comment line fails closed" primary git.example + +# NOT over-broad: TAB/LF/CR remain accepted where PyYAML already accepts them +# (covered by section 17's tab fixtures and the ordinary newline-delimited +# fixtures used throughout this file), so no additional assertion is needed +# here beyond confirming the forbidden-control guard does not fire on them. + +# 19. UNSIGNED-EXPONENT FLOAT OVER-REJECTION (#865 round-9 blocker 2): PyYAML +# 6.0.3's implicit float resolver requires an EXPLICIT SIGN on the exponent +# ([eE][-+][0-9]+); an unsigned exponent is NOT matched, so PyYAML resolves +# the scalar as a plain STRING, not a float. A prior fallback's float +# recognizer accepted an OPTIONAL sign ([eE][-+]?[0-9]+), over-matching these +# spellings as floats and dropping the token PyYAML would emit verbatim +# (over-rejection). Verified empirically against real PyYAML 6.0.3. +for form in '1.0e10' '+1.0e10' '-1.0e10' '1.0E10' '.5e10' '4.e8'; do + write_fixture "logins: + - name: primary + url: https://git.example + token: ${form} +" + assert_token "unsigned-exponent form '$form' is a PyYAML string, token resolves" "$form" primary git.example +done + +# Parity guard: a genuine SIGNED-exponent float is still typed as a non-string +# float by PyYAML and must still fail closed (not regress into over-acceptance). +write_fixture 'logins: + - name: primary + url: https://git.example + token: 1.0e+10 +' +assert_token "signed-exponent genuine float still fails closed" "" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: 1.0e-10 +' +assert_token "signed-exponent (negative) genuine float still fails closed" "" primary git.example + +# 20. RESIDUAL OVER-REJECTION found via round-9 differential fuzzing (folded into +# this round, not split off): a plain scalar starting with "?" NOT followed by +# whitespace (e.g. "?x") is a valid PyYAML string -- only a bare "?" or "? " +# (question mark followed by space/EOL) opens a complex mapping key and is +# illegal in a value position. A prior blanket-reject set treated EVERY +# leading "?" as illegal, over-rejecting a token PyYAML accepts verbatim. +write_fixture 'logins: + - name: primary + url: https://git.example + token: ?x +' +assert_token "question-mark-not-space is a plain string, token resolves" "?x" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1 From 0905cdc292f2bb71bbc79a8513feb865ddab32ca Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 22 Jul 2026 01:19:27 -0500 Subject: [PATCH 13/15] fix(git): match PyYAML Reader.NON_PRINTABLE and block-context plain rules in tea token fallback (#865) Round-10 auditor blockers in the PyYAML-absent fallback of get_gitea_token_for_login (detect-platform.sh): Blocker 1 (fail-open): the round-9 control-char guard used a hand-rolled C0/DEL subset that missed code points PyYAML's Reader also rejects -- the C1 block (U+0080-0084, U+0086-009F), the surrogate range, and the BMP noncharacters U+FFFE/U+FFFF -- so the fallback still emitted a token from documents PyYAML rejects whole. _FORBIDDEN_CONTROL now uses PyYAML 6.0.3's EXACT Reader.NON_PRINTABLE character class (negated), verified 0-mismatch against real PyYAML across the full BMP + astral range. Blocker 2 (over-reject): the recognizer blanket-rejected any plain scalar containing a flow indicator, but in BLOCK context (where a tea config value always sits) PyYAML treats ',[]{}' as ordinary content and treats '!&*|>#%@`' and quotes as significant only at the first non-space char. The blanket scan dropped tokens PyYAML emits verbatim (internal '!', ',' '[' ']' '{' '}' '#'-not-space-preceded, ':'-not-space-followed). Removed it; the leading-char guard and the ' #' / ': ' / trailing-':' guards keep the fail-open direction shut. Tests: additions-only sections 21 (printable-boundary fail-close for C1 + noncharacters; NEL/U+00A0/astral still resolve) and 22 (internal indicators resolve; leading indicator / ': ' inline map / trailing ':' fail closed). Verified vs real PyYAML 6.0.3 both directions (present as oracle, absent via PYTHONPATH shadow): wide 602 cases 0 fail-open/0 present-mismatch, dense 712 cases 0 fail-open/0 over-reject/0 mismatch. Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 69 +++++++---- .../tools/git/test-gitea-login-resolution.sh | 114 ++++++++++++++++++ 2 files changed, 161 insertions(+), 22 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 7d55ad10..781977c0 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -810,19 +810,27 @@ def _constructible(text): # PyYAML's Reader.check_printable scans the ENTIRE raw input stream (not just -# scalar contents, and NOT scoped by quoting) for bytes outside its printable -# set and raises ReaderError -- a whole-document reject -- the instant one is -# found, no matter where it sits (an unrelated field, a comment, inside or -# outside quotes). Empirically verified against real PyYAML 6.0.3: EVERY C0 -# control byte below plus DEL (0x7F) rejects in plain scalars, inside -# double-quoted scalars, and inside single-quoted scalars alike; only -# TAB(0x09), LF(0x0A), and CR(0x0D) are accepted among the C0 range (LF/CR are -# line separators, handled elsewhere; TAB has its own narrower, position-aware -# guard above since PyYAML accepts it in some contexts). This set therefore -# excludes 0x09/0x0A/0x0D and matches the forbidden C0 set {0x00-0x08, 0x0B, -# 0x0C, 0x0E-0x1F} plus DEL (0x7F). +# scalar contents, and NOT scoped by quoting) for code points outside its +# printable set and raises ReaderError -- a whole-document reject -- the instant +# one is found, no matter where it sits (an unrelated field, a comment, inside or +# outside quotes). To guarantee parity WITHOUT a hand-rolled subset (which missed +# the C1 block and BMP noncharacters), this is PyYAML 6.0.3's EXACT Reader +# printable definition, negated verbatim: +# Reader.NON_PRINTABLE = +# re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-퟿-�' +# '\U00010000-\U0010FFFF]') +# i.e. PRINTABLE = {TAB(0x09), LF(0x0A), CR(0x0D), 0x20-0x7E, NEL(0x85), +# 0xA0-0xD7FF, 0xE000-0xFFFD, 0x10000-0x10FFFF}; EVERYTHING else (all other C0 +# controls, DEL 0x7F, the entire C1 block 0x80-0x84/0x86-0x9F, the surrogate +# range 0xD800-0xDFFF, and the BMP noncharacters 0xFFFE/0xFFFF) is non-printable +# -> whole-document fail closed. Verified empirically against real PyYAML 6.0.3 +# in both directions: the C1 bytes and 0xFFFE/0xFFFF reject; NEL(0x85), 0xA0, and +# astral code points (e.g. U+1F600) are accepted and resolve the token. The prior +# C0/DEL/tab behavior is a strict subset of this. (A surrogate code point cannot +# occur in valid UTF-8, so it also trips the UTF-8 decode on read and fails +# closed; it is included here for exact definitional parity.) _FORBIDDEN_CONTROL = re.compile( - "[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]" + "[^\x09\x0a\x0d\x20-\x7e\x85\xa0-퟿-�\U00010000-\U0010ffff]" ) # Sentinel: "outside the supported subset -> fail closed". Distinct from a @@ -903,8 +911,21 @@ def _scalar(raw): # remain valid plain scalars and fall through. Fail closed on the bare # indicator so the fallback never emits a token PyYAML would refuse. return _FAIL - if any(ch in _FLOW for ch in value): - return _FAIL + # NO blanket internal-indicator reject here. In BLOCK context (where a tea + # config value always sits) PyYAML treats ',[]{}' as ordinary plain-scalar + # content -- they are flow indicators ONLY inside a flow collection -- and + # treats '!&*|>#%@`' plus quotes as significant ONLY at the FIRST non-space + # character of a node (a leading one starts a tag/anchor/alias/block-scalar/ + # comment/directive/reserved/quote), which the leading-char guard above + # (value[0] ...) already rejects. Verified empirically against real PyYAML + # 6.0.3: an INTERNAL '!' ',' '[' ']' '{' '}' '&' '*' '#'(not space-preceded) + # ':'(not space-followed) all resolve as a plain-string token. The only + # internal positions PyYAML treats as significant in block context are ' #' + # (whitespace-preceded '#' -> comment; stripped by the loop above) and ': ' + # or a trailing ':' (-> mapping; rejected by the ": "/endswith(":") guard + # below). A blanket any(ch in _FLOW ...) scan over-rejected those internal + # indicators, dropping a token PyYAML emits verbatim; removing it restores + # parity while the position-specific guards keep the fail-open direction shut. if "\t" in value: # A tab anywhere in a plain scalar (leading, trailing, or embedded) is a # PyYAML scanner error on the whole document -- it accepts tabs ONLY @@ -1182,14 +1203,18 @@ def _token_via_lines(): raw_text = handle.read() # Whole-document, position-independent reject: PyYAML's Reader rejects the - # ENTIRE document (ReaderError) if a forbidden C0/DEL control byte appears - # ANYWHERE in the raw stream -- in a plain scalar, inside single- or - # double-quoted scalars, in a comment, or in a field this recognizer never - # even looks at. Checking the raw text (before splitlines(), which itself - # splits on some of these same control bytes -- e.g. \x0b, \x0c, \x1c-\x1e - # -- and would otherwise obscure them) mirrors that exactly: fail closed - # for the whole document rather than risk emitting a token PyYAML would - # have refused. + # ENTIRE document (ReaderError) if a NON-printable code point (per its exact + # printable definition -- see _FORBIDDEN_CONTROL: C0 controls, DEL, the C1 + # block, surrogates, and 0xFFFE/0xFFFF) appears ANYWHERE in the raw stream -- + # in a plain scalar, inside single- or double-quoted scalars, in a comment, + # or in a field this recognizer never even looks at. Checking the raw text + # (before splitlines(), which itself splits on some of these same code points + # -- e.g. \x0b, \x0c, \x1c-\x1e, and NEL \x85 -- and would otherwise obscure + # them) mirrors that exactly: fail closed for the whole document rather than + # risk emitting a token PyYAML would have refused. (NEL 0x85 is PRINTABLE to + # PyYAML's Reader, so it is NOT in _FORBIDDEN_CONTROL; where PyYAML treats it + # as a line break the fallback's splitlines() agrees, and where PyYAML then + # errors the recognizer's structural guards fail closed identically.) if _FORBIDDEN_CONTROL.search(raw_text): return None diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index 1445f924..2c265dfb 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -893,4 +893,118 @@ write_fixture 'logins: ' assert_token "question-mark-not-space is a plain string, token resolves" "?x" primary git.example +# 21. PRINTABLE-BOUNDARY FAIL-CLOSE (#865 round-10 blocker 1): the round-9 guard +# used a hand-rolled C0/DEL subset that MISSED code points PyYAML's Reader +# also rejects -- the C1 block (U+0080-0084, U+0086-009F) and the BMP +# noncharacters U+FFFE/U+FFFF -- so the fallback still emitted the token from +# documents PyYAML rejects whole (fail-open). The guard now uses PyYAML +# 6.0.3's EXACT Reader.NON_PRINTABLE character class (see detect-platform.sh +# _FORBIDDEN_CONTROL). Verified empirically against real PyYAML 6.0.3: +# PRINTABLE = {0x09,0x0A,0x0D, 0x20-0x7E, 0x85(NEL), 0xA0-0xD7FF, +# 0xE000-0xFFFD, 0x10000-0x10FFFF}; everything else fails the whole document +# closed. write_codepoint_fixture emits a chosen Unicode code point's real +# UTF-8 bytes (via python3, since bash strings cannot faithfully carry many +# of these) into a selectable position, then the login block follows. +write_codepoint_fixture() { + # $1 = hex code point (e.g. 0x80); $2 = position: field|comment|token|nelterm + CP_HEX="$1" CP_POS="$2" python3 - "$FIXTURE_XDG/tea/config.yml" <<'PY' +import sys +cp = int(__import__("os").environ["CP_HEX"], 16) +pos = __import__("os").environ["CP_POS"] +ch = chr(cp) +head = "logins:\n - name: primary\n url: https://git.example\n token: TOK_PLAIN\n" +if pos == "field": + doc = head + "other: x" + ch + "y\n" +elif pos == "comment": + doc = head + "# note x" + ch + "y here\n" +elif pos == "token": + doc = "logins:\n - name: primary\n url: https://git.example\n token: T" + ch + "K\n" +elif pos == "nelterm": + # NEL (U+0085) used as the line terminator throughout: PyYAML treats it as a + # line break (printable, NOT a ReaderError) and resolves the token; the + # fallback's splitlines() splits on NEL identically -> parity, token resolves. + doc = ("logins:" + ch + " - name: primary" + ch + + " url: https://git.example" + ch + " token: TOK_NEL" + ch) +else: + raise SystemExit("bad pos") +with open(sys.argv[1], "w", encoding="utf-8") as f: + f.write(doc) +PY +} + +# C1-block + BMP-noncharacter code points fail the WHOLE document closed in an +# unrelated field and in a comment, exactly as PyYAML's ReaderError does. +for cphex in 0x80 0x81 0x84 0x86 0x9f 0xfffe 0xffff; do + write_codepoint_fixture "$cphex" field + assert_both_fail_closed "code point $cphex in unrelated field fails closed" primary git.example + write_codepoint_fixture "$cphex" comment + assert_both_fail_closed "code point $cphex in a comment fails closed" primary git.example +done + +# NOT over-broad: printable code points PyYAML ACCEPTS must still resolve the +# token in BOTH paths -- NEL(0x85) as a line separator, U+00A0 (NBSP) inside a +# value, and an astral code point (U+1F600) inside the token value. +write_codepoint_fixture 0x85 nelterm +assert_token "NEL (U+0085) line-terminator resolves token" "TOK_NEL" primary git.example +write_codepoint_fixture 0xa0 field +assert_token "U+00A0 in unrelated value still resolves token" "TOK_PLAIN" primary git.example +write_codepoint_fixture 0x1f600 token +assert_token "astral U+1F600 inside token resolves verbatim" "$(printf 'T\360\237\230\200K')" primary git.example + +# 22. INTERNAL-INDICATOR OVER-REJECTION (#865 round-10 blocker 2): the round-9 +# recognizer blanket-rejected any plain scalar CONTAINING a flow indicator +# ([]{}*&!), but in BLOCK context PyYAML treats ',[]{}' as ordinary content +# and treats '!&*#...' as significant ONLY at the FIRST non-space char. So an +# INTERNAL indicator is legal plain-string content and PyYAML emits the token +# verbatim; the fallback dropped it (over-rejection). Verified empirically +# against real PyYAML 6.0.3. The fix removes the blanket internal scan while +# the leading-char guard and the ' #'/': '/trailing-':' guards keep the +# fail-OPEN direction shut. +for tv in 'a!b' 'a,b' 'a[b' 'a]b' 'a{b' 'a}b' 'a&b' 'a*b' 'a[b]c' 'a{b}c' 'a,b,c' 'a#b' 'a:b'; do + write_fixture "logins: + - name: primary + url: https://git.example + token: ${tv} +" + assert_token "internal-indicator token '$tv' resolves verbatim" "$tv" primary git.example +done + +# Fail-OPEN direction stays shut: a LEADING indicator, a ' #' comment tail, an +# internal ': ' (colon-space) inline map, and a trailing ':' each make PyYAML +# resolve NO usable string token (tag/flow/anchor reject or None, comment strip, +# or a mapping), so BOTH must fail closed. (Leading tag/anchor/flow are the +# documented, round-9-approved structural fail-closed class; kept intact here.) +write_fixture 'logins: + - name: primary + url: https://git.example + token: !x +' +assert_both_fail_closed "leading '!' tag token fails closed" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: [a] +' +assert_both_fail_closed "leading '[' flow-seq token fails closed" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: a # trailing comment +' +# ' #' comment tail: PyYAML strips the comment -> token is the string 'a', which +# still resolves. This is the NOT-over-broad boundary partner of the guard. +assert_token "space-hash comment tail strips to plain token" "a" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: a: b +' +assert_both_fail_closed "internal colon-space (inline map) token fails closed" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: ab: +' +assert_both_fail_closed "trailing colon (map indicator) token fails closed" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1 From 2d821324fe5b63a2e0bd9a874e361b6d19ed8c7e Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 22 Jul 2026 02:24:33 -0500 Subject: [PATCH 14/15] fix(git): PyYAML-absent fallback parity for quoted-scalar breaks and tag/anchor properties (#865 round 11) Two residual over-rejections in the PyYAML-absent fallback of get_gitea_token_for_login, both fail-closed today but diverging from real PyYAML 6.0.3 for valid inputs. Neither is a fail-open; the fix loosens ONLY to exact PyYAML parity and keeps failing closed elsewhere. Blocker A - embedded printable line break inside a quoted scalar. The fallback split the raw document with str.splitlines(), which breaks at NEL(U+0085), LS(U+2028) and PS(U+2029) -- all PRINTABLE to PyYAML's Reader. Inside a flow (quoted) scalar PyYAML does not break at these: it line-folds a double/single-quoted scalar (NEL/LF/CR -> one space; LS/PS verbatim), resolving one token, while splitlines() cut mid-quote and failed the whole document closed. Replace splitlines() with _split_logical_lines, a quote-aware splitter that reproduces PyYAML's flow folding (scan_flow_scalar_spaces/breaks) and raises on a col-0 doc marker (---/...) in a folded continuation. The same code points UNQUOTED or in a comment still make PyYAML raise, so those stay closed. Blocker B - leading non-specific tag / anchor property. The recognizer blanket-rejected any scalar starting with '!' or '&'. But '! ' (bang + space) and '&name ' are transparent node properties: PyYAML strips them and applies normal implicit typing, so '! x'/'&a x' resolve the string 'x'. Add _strip_properties, consuming <=1 non-specific tag and <=1 anchor (either order) and recursing on the remainder. '!x' (tag handle), '!foo x', non-string nodes ('! 123'), duplicate properties and explicit tags ('!!str x') still fail closed. Fallback now matches real PyYAML 6.0.3 (oracle) on a two-direction differential fuzz of the full Unicode line-break set x {double, single, plain, comment, token-position} and the full leading tag/anchor space: 833 cases, 0 fail-opens, 0 present-mismatches, 52 endorsed fail-closed over-rejects (blank-line-in-quote \n folds; explicit !!str/verbose tags; property+quoted; anchor-without-space). Wrapper scripts pr-review.sh and issue-comment.sh unchanged (0-line diff). Tests: additions-only sections 23 (quoted-scalar breaks) and 24 (tag/anchor properties). Co-Authored-By: Claude Opus 4.8 --- .../framework/tools/git/detect-platform.sh | 204 +++++++++++++++++- .../tools/git/test-gitea-login-resolution.sh | 116 +++++++++- 2 files changed, 310 insertions(+), 10 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 781977c0..0d41de6e 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -843,6 +843,9 @@ _FAIL = object() # position makes the whole line fail to match -> the caller fails closed. _KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*) *:(?:[ ](.*)|)$") _FLOW = set("[]{}*&!") +# A plain anchor property: '&' then PyYAML's exact anchor charset [0-9A-Za-z-_]+, +# then whatever follows (captured for the caller to require a space or EOL). +_ANCHOR_RE = re.compile(r"^&[0-9A-Za-z\-_]+(.*)$", re.S) class _Bail(Exception): @@ -852,6 +855,46 @@ class _Bail(Exception): pass +def _strip_properties(value): + # Consume leading YAML node properties -- at most ONE non-specific tag + # ('!' + space) and at most ONE plain anchor ('&name' + space), in either + # order -- and return the remaining node text for normal resolution. PyYAML's + # SafeLoader applies these TRANSPARENT properties and then resolves the + # following node with its ordinary implicit typing, so (verified against real + # PyYAML 6.0.3) '! x' -> 'x', '&a 123' -> int 123, '! ' / '&a' -> null, and + # '! &a x' / '&a ! x' -> 'x', exactly as the bare node would resolve. Returns + # _FAIL for every form we do NOT reproduce identically, so the caller fails + # closed: a tag shorthand ('!x' -> ConstructorError), an explicit or verbatim + # tag ('!!str' / '!!int' / '!<...>' / '!foo') whose type coercion we do not + # emulate, a tab separator (PyYAML scanner error), a duplicate tag or anchor + # ('! ! x' / '&a &b x' -> ParserError), or an anchor glued to a non-space. + # The returned remainder (possibly '') is re-resolved by the caller; '' is a + # null node -> fail closed for a token field, matching PyYAML. + seen_tag = False + seen_anchor = False + while value[:1] in ("!", "&"): + if value[0] == "!": + # Non-specific tag: '!' then AT LEAST ONE SPACE. A tab is a PyYAML + # scanner error; '!x' / '!!type' / '!' are typed/short tags. + if seen_tag or len(value) < 2 or value[1] != " ": + return _FAIL + seen_tag = True + value = value[1:].lstrip(" ") + else: # anchor '&name' + m = _ANCHOR_RE.match(value) + if seen_anchor or m is None: + return _FAIL + rest = m.group(1) + if rest == "": + value = "" # bare '&name' with no following node -> null node + elif rest[0] == " ": + value = rest.lstrip(" ") + else: + return _FAIL # '&name' glued to a non-space (e.g. '&a:') -> error + seen_anchor = True + return value + + def _scalar(raw): # Resolve a single flow scalar (quoted or plain) the way PyYAML would for # tea's simple values, or _FAIL when it is outside the supported subset so @@ -894,8 +937,18 @@ def _scalar(raw): value = value.strip(" ") # spaces only; a tab must survive to fail closed if not value: return None + if value[0] in ("!", "&"): + # Leading node property (non-specific tag '! ' / plain anchor '&name '): + # PyYAML applies it to the FOLLOWING node and resolves THAT, so strip the + # transparent forms and re-resolve the remainder EXACTLY (implicit typing + # and all). _strip_properties returns _FAIL for the tag/anchor forms + # PyYAML rejects or type-coerces, which fail closed here. + remainder = _strip_properties(value) + if remainder is _FAIL: + return _FAIL + return _scalar(remainder) if value[0] in _FLOW or value[0] in ("|", ">", "@", "`", '"', "'", "%", ","): - return _FAIL # flow / block-scalar / reserved / directive / anchor / quote + return _FAIL # flow / block-scalar / reserved / directive / quote # NOTE: "?" is deliberately NOT in the blanket-reject set above. Empirically # verified against real PyYAML 6.0.3: "?x" (indicator immediately followed by # a non-space) is a plain scalar string "?x" like "-x" and ":x" are -- only @@ -1189,6 +1242,138 @@ def _token_via_pyyaml(): return None +_BREAKS = "\r\n\x85

" +_VALUE_OPEN_RE = re.compile(r"^\s*(?:-\s+)*[A-Za-z0-9_][A-Za-z0-9_.\-]*:$") +_SEQ_OPEN_RE = re.compile(r"^\s*(?:-\s+)*-$") + + +def _value_open_before(prefix): + # True when a quote appearing at the END of `prefix` (the current logical + # line so far) begins a flow SCALAR value -- the only position where a quote + # opens a quoted scalar whose embedded line breaks PyYAML folds instead of + # splitting. That is: the prefix is empty/all-indent (root or block scalar + # start), or ends with a mapping ': ' or a sequence '- ' indicator that is + # SPACE-separated from the quote. A quote glued to the previous char (e.g. + # 'key:"x') is NOT a value opener (PyYAML needs the space), and a quote mid + # -content is literal -- both fail this test, so their breaks split the line + # exactly as before (fail closed / plain-scalar behavior preserved). + stripped = prefix.rstrip(" ") + if stripped == "": + return True # start of line (after any indentation) -> node value start + if prefix == stripped: + return False # no space before the quote -> not a value opener + return bool(_VALUE_OPEN_RE.match(stripped) or _SEQ_OPEN_RE.match(stripped)) + + +def _fold_quoted_break_run(run): + # Reproduce PyYAML's flow-scalar folding (scan_flow_scalar_spaces + + # scan_flow_scalar_breaks) for the maximal ' \t' + line-break run inside a + # quoted scalar. A run with NO break is literal whitespace (verbatim). With a + # break: surrounding spaces/tabs are dropped; a single \n-class break + # (\n / \r / \r\n / \x85 NEL) folds to ONE space; 
 (LS) / 
 (PS) + # are kept verbatim; each ADDITIONAL break (a blank line) contributes its own + # break char (\n for the \n-class). Verified against real PyYAML 6.0.3 for + # both double- and single-quoted scalars (they fold identically here). + def _lb(s, j): + if s[j] == "\r" and j + 1 < len(s) and s[j + 1] == "\n": + return "\n", 2 + if s[j] in "\r\n\x85": + return "\n", 1 + return s[j], 1 # 
 / 
 preserved verbatim + n = len(run) + j = 0 + while j < n and run[j] in " \t": + j += 1 + if j >= n: + return run # pure whitespace, no break -> literal content + line_break, adv = _lb(run, j) + j += adv + breaks = [] + while True: + while j < n and run[j] in " \t": + j += 1 + if j < n and run[j] in _BREAKS: + b, adv = _lb(run, j) + j += adv + breaks.append(b) + else: + break + out = "" + if line_break != "\n": + out += line_break + elif not breaks: + out += " " + return out + "".join(breaks) + + +def _split_logical_lines(raw_text): + # Split like str.splitlines() EXCEPT a line break INSIDE a value-opening + # quoted scalar does NOT end the line: PyYAML keeps the quoted scalar together + # and folds the break (flow folding above), so we fold it and continue the + # same logical line. Only the break chars that survive the _FORBIDDEN_CONTROL + # gate reach here: \n, \r, \x85 (NEL), 
 (LS), 
 (PS). OUTSIDE + # quotes every one ends the line (matching PyYAML's block-level line breaks + # and str.splitlines), so a NEL/LS/PS used as the document's line-terminator + # style, and an UNQUOTED plain scalar carrying an embedded break, split and + # fail closed exactly as before. A quoted scalar left unterminated at EOF + # stays on the final line and fails closed in _scalar (PyYAML errors too). + out = [] + cur = [] + i = 0 + n = len(raw_text) + quote = None + while i < n: + ch = raw_text[i] + if quote is None: + if ch in _BREAKS: + out.append("".join(cur)) + cur = [] + i += 2 if (ch == "\r" and i + 1 < n and raw_text[i + 1] == "\n") else 1 + continue + if ch in ("'", '"') and _value_open_before("".join(cur)): + quote = ch + cur.append(ch) + i += 1 + continue + cur.append(ch) + i += 1 + continue + # inside a value-opening quoted scalar + if ch == quote: + cur.append(ch) + quote = None + i += 1 + continue + if ch in " \t" or ch in _BREAKS: + j = i + has_break = False + while j < n and (raw_text[j] in " \t" or raw_text[j] in _BREAKS): + if raw_text[j] in _BREAKS: + has_break = True + j += 1 + run = raw_text[i:j] + if has_break: + # A continuation line beginning (at column 0, no indent) with a + # document marker is a PyYAML scanner error even inside a quoted + # scalar; when the run ends on a break the continuation content at + # j is at column 0, so guard it and fail closed. (An INDENTED + # '---' is literal content to PyYAML and folds normally.) + if run[-1] in _BREAKS and raw_text[j:j + 3] in ("---", "...") and ( + j + 3 >= n or raw_text[j + 3] in "\0 \t" + _BREAKS + ): + raise _Bail() + cur.append(_fold_quoted_break_run(run)) + else: + cur.append(run) + i = j + continue + cur.append(ch) + i += 1 + if cur or not out: + out.append("".join(cur)) + return out + + def _token_via_lines(): # Conservative fallback for hosts without PyYAML. It parses config.yml with a # strict recognizer (_safe_parse) of the narrow block-style subset tea writes, @@ -1208,17 +1393,18 @@ def _token_via_lines(): # block, surrogates, and 0xFFFE/0xFFFF) appears ANYWHERE in the raw stream -- # in a plain scalar, inside single- or double-quoted scalars, in a comment, # or in a field this recognizer never even looks at. Checking the raw text - # (before splitlines(), which itself splits on some of these same code points - # -- e.g. \x0b, \x0c, \x1c-\x1e, and NEL \x85 -- and would otherwise obscure - # them) mirrors that exactly: fail closed for the whole document rather than - # risk emitting a token PyYAML would have refused. (NEL 0x85 is PRINTABLE to - # PyYAML's Reader, so it is NOT in _FORBIDDEN_CONTROL; where PyYAML treats it - # as a line break the fallback's splitlines() agrees, and where PyYAML then - # errors the recognizer's structural guards fail closed identically.) + # (before line splitting, which would otherwise split on some of these same + # code points -- e.g. \x0b, \x0c, \x1c-\x1e -- and obscure them) mirrors that + # exactly: fail closed for the whole document. (NEL 0x85, LS 0x2028 and PS + # 0x2029 are PRINTABLE to PyYAML's Reader, so they are NOT in + # _FORBIDDEN_CONTROL; _split_logical_lines below reproduces PyYAML's line-break + # semantics for them -- a structural break outside quotes, a FOLDED break + # inside a quoted scalar -- rather than str.splitlines(), which would wrongly + # split them mid-quote and fail-close a token PyYAML resolves.) if _FORBIDDEN_CONTROL.search(raw_text): return None - lines = raw_text.splitlines() + lines = _split_logical_lines(raw_text) config = _safe_parse(lines) if config is _FAIL: diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index 2c265dfb..c17338fe 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -922,7 +922,9 @@ elif pos == "token": elif pos == "nelterm": # NEL (U+0085) used as the line terminator throughout: PyYAML treats it as a # line break (printable, NOT a ReaderError) and resolves the token; the - # fallback's splitlines() splits on NEL identically -> parity, token resolves. + # fallback's _split_logical_lines splits on NEL identically OUTSIDE a quote + # -> parity, token resolves. (Round 23 covers NEL/LS/PS INSIDE a quote, where + # PyYAML folds rather than breaks and a naive splitlines() would over-split.) doc = ("logins:" + ch + " - name: primary" + ch + " url: https://git.example" + ch + " token: TOK_NEL" + ch) else: @@ -1007,4 +1009,116 @@ write_fixture 'logins: ' assert_both_fail_closed "trailing colon (map indicator) token fails closed" primary git.example +# 23. EMBEDDED LINE-BREAK INSIDE A QUOTED SCALAR (#865 round-11 blocker A): the +# round-10 fallback split the raw document with str.splitlines(), which breaks +# at NEL(U+0085), LS(U+2028) and PS(U+2029) -- code points that are PRINTABLE +# to PyYAML's Reader. Inside a flow (quoted) scalar PyYAML does NOT break at +# these: it LINE-FOLDS a double/single-quoted scalar (NEL/LF/CR -> a single +# space; LS/PS -> the char verbatim), so it resolves ONE token, while +# splitlines() cut the value mid-quote and failed the whole document closed +# (over-rejection). The fallback now uses _split_logical_lines, which +# reproduces PyYAML's flow-folding. Verified empirically vs real PyYAML 6.0.3. +# write_break_fixture emits a chosen break code point in a selectable context. +write_break_fixture() { + # $1 = hex code point of the break; $2 = context: dq|sq|plain|comment|dq2 + CP_HEX="$1" Q_STYLE="$2" python3 - "$FIXTURE_XDG/tea/config.yml" <<'PY' +import sys, os +cp = int(os.environ["CP_HEX"], 16) +q = os.environ["Q_STYLE"] +ch = chr(cp) +head = "logins:\n - name: primary\n url: https://git.example\n token: " +if q == "dq": + doc = head + '"tok' + ch + 'en"' +elif q == "sq": + doc = head + "'tok" + ch + "en'" +elif q == "plain": + doc = head + "tok" + ch + "en" +elif q == "dq2": + # blank line inside a quoted scalar: PyYAML folds a two-break run to a literal + # newline, which the recognizer's key regex cannot carry -> endorsed + # fail-closed over-reject (see assert_fallback_fails_closed below). + doc = head + '"tok' + ch + ch + 'en"' +elif q == "comment": + doc = ("logins:\n - name: primary\n url: https://git.example\n" + " token: TOK_PLAIN\n# c" + ch + "x") +else: + raise SystemExit("bad q") +with open(sys.argv[1], "w", encoding="utf-8") as f: + f.write(doc + "\n") +PY +} + +# NEL folds to a single space inside double- AND single-quoted scalars: the token +# resolves identically in both paths (fallback no longer over-splits). +write_break_fixture 0x85 dq +assert_token "NEL inside double-quote folds to space, token resolves" "tok en" primary git.example +write_break_fixture 0x85 sq +assert_token "NEL inside single-quote folds to space, token resolves" "tok en" primary git.example +# LS(U+2028)/PS(U+2029) are preserved VERBATIM by PyYAML's flow fold (they are +# not \n-class breaks); the fallback must surface them byte-for-byte. +write_break_fixture 0x2028 dq +assert_token "LS inside double-quote is verbatim" "$(printf 'tok\342\200\250en')" primary git.example +write_break_fixture 0x2028 sq +assert_token "LS inside single-quote is verbatim" "$(printf 'tok\342\200\250en')" primary git.example +write_break_fixture 0x2029 dq +assert_token "PS inside double-quote is verbatim" "$(printf 'tok\342\200\251en')" primary git.example + +# Direction-sensitivity: the SAME code points UNQUOTED (a plain scalar) or in a +# COMMENT make PyYAML raise a scanner error, so both paths must fail closed. The +# fold rule applies ONLY inside a quoted scalar. +write_break_fixture 0x85 plain +assert_token "NEL in an unquoted plain scalar fails closed" "" primary git.example +write_break_fixture 0x2028 plain +assert_token "LS in an unquoted plain scalar fails closed" "" primary git.example +write_break_fixture 0x85 comment +assert_token "NEL in a comment fails closed" "" primary git.example + +# Endorsed fail-closed over-reject: a blank line inside a quoted scalar folds to a +# literal newline that the recognizer cannot carry -- PyYAML resolves a +# (newline-bearing) token, the fallback fails closed (strictly safer). +write_break_fixture 0x85 dq2 +assert_fallback_fails_closed "blank-line-in-quote (NEL run) fails closed" primary git.example + +# 24. LEADING NON-SPECIFIC TAG / ANCHOR PROPERTY (#865 round-11 blocker B): the +# round-10 recognizer blanket-rejected any scalar beginning with '!' or '&'. +# But a NON-SPECIFIC tag '! ' (bang + SPACE) and an anchor '&name ' are +# transparent node properties: PyYAML strips them and applies normal implicit +# typing to the node, so '! x'/'&a x' resolve the STRING 'x'. The fallback now +# consumes <=1 non-specific tag and <=1 anchor (either order) and recurses on +# the remainder. Verified empirically vs real PyYAML 6.0.3. +for pair in '! x=x' '&a x=x' '! x y=x y' '! "q"=q' "! 'q'=q" '! &b x=x' '&b ! x=x'; do + tv="${pair%%=*}"; want="${pair#*=}" + write_fixture "logins: + - name: primary + url: https://git.example + token: ${tv} +" + assert_token "property token '$tv' resolves node string" "$want" primary git.example +done + +# Fail-OPEN direction stays shut. '!x' (bang + NON-space) is a tag HANDLE -> +# ConstructorError; '!foo x'/'* a'/'! !x' raise; a property over a NON-string node +# ('! 123'/'! true'/'! null') types non-str -> no usable token. All fail closed in +# BOTH paths. +for tv in '!x' '!foo x' '* a' '! !x' '! 123' '! true' '! null' '&a &b x'; do + write_fixture "logins: + - name: primary + url: https://git.example + token: ${tv} +" + assert_token "non-resolving property token '$tv' fails closed" "" primary git.example +done + +# Endorsed fail-closed over-reject: an EXPLICIT tag ('!!str x', verbose +# '! x') forces a string PyYAML resolves, but the fallback +# recognizes only the transparent non-specific tag and fails closed (safer). +for tv in '!!str x' '! x'; do + write_fixture "logins: + - name: primary + url: https://git.example + token: ${tv} +" + assert_fallback_fails_closed "explicit-tag token '$tv' fails closed" primary git.example +done + echo "Gitea login resolution regression harness passed" -- 2.49.1 From 8ac7e70f0db6458f824833a438fb69a8ea759db5 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 22 Jul 2026 03:07:56 -0500 Subject: [PATCH 15/15] fix(git): fail closed on YAML anchors in tea token fallback (#865 round 12) The PyYAML-absent fallback's _strip_properties consumed a plain anchor ('&name') per-scalar, with no document-scoped registry of declared anchor names. Real PyYAML's Composer rejects a duplicate anchor NAME declared anywhere in the document (ComposerError, whole-document fail), but the fallback would still emit the later token: fail-open (HIGH). Fix: reject every '&'-anchor property unconditionally instead of building a hand-rolled duplicate registry, which guarantees zero fail-open by construction at the cost of a deliberate, endorsed over-reject on a single non-duplicated anchor. The '!' non-specific-tag branch is unchanged ('! x' -> 'x' still resolves). --- .../framework/tools/git/detect-platform.sh | 57 +++++----- .../tools/git/test-gitea-login-resolution.sh | 106 ++++++++++++++++-- 2 files changed, 124 insertions(+), 39 deletions(-) diff --git a/packages/mosaic/framework/tools/git/detect-platform.sh b/packages/mosaic/framework/tools/git/detect-platform.sh index 0d41de6e..25d8cf8f 100755 --- a/packages/mosaic/framework/tools/git/detect-platform.sh +++ b/packages/mosaic/framework/tools/git/detect-platform.sh @@ -843,9 +843,6 @@ _FAIL = object() # position makes the whole line fail to match -> the caller fails closed. _KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*) *:(?:[ ](.*)|)$") _FLOW = set("[]{}*&!") -# A plain anchor property: '&' then PyYAML's exact anchor charset [0-9A-Za-z-_]+, -# then whatever follows (captured for the caller to require a space or EOL). -_ANCHOR_RE = re.compile(r"^&[0-9A-Za-z\-_]+(.*)$", re.S) class _Bail(Exception): @@ -856,22 +853,32 @@ class _Bail(Exception): def _strip_properties(value): - # Consume leading YAML node properties -- at most ONE non-specific tag - # ('!' + space) and at most ONE plain anchor ('&name' + space), in either - # order -- and return the remaining node text for normal resolution. PyYAML's - # SafeLoader applies these TRANSPARENT properties and then resolves the - # following node with its ordinary implicit typing, so (verified against real - # PyYAML 6.0.3) '! x' -> 'x', '&a 123' -> int 123, '! ' / '&a' -> null, and - # '! &a x' / '&a ! x' -> 'x', exactly as the bare node would resolve. Returns - # _FAIL for every form we do NOT reproduce identically, so the caller fails - # closed: a tag shorthand ('!x' -> ConstructorError), an explicit or verbatim - # tag ('!!str' / '!!int' / '!<...>' / '!foo') whose type coercion we do not - # emulate, a tab separator (PyYAML scanner error), a duplicate tag or anchor - # ('! ! x' / '&a &b x' -> ParserError), or an anchor glued to a non-space. - # The returned remainder (possibly '') is re-resolved by the caller; '' is a - # null node -> fail closed for a token field, matching PyYAML. + # Consume a leading YAML node non-specific tag ('!' + space) and return the + # remaining node text for normal resolution. PyYAML's SafeLoader applies + # this TRANSPARENT property and then resolves the following node with its + # ordinary implicit typing, so (verified against real PyYAML 6.0.3) + # '! x' -> 'x', '! ' -> null, exactly as the bare node would resolve. + # Returns _FAIL for every form we do NOT reproduce identically, so the + # caller fails closed: a tag shorthand ('!x' -> ConstructorError), an + # explicit or verbatim tag ('!!str' / '!!int' / '!<...>' / '!foo') whose + # type coercion we do not emulate, a tab separator (PyYAML scanner error), + # a duplicate tag ('! ! x' -> ParserError). + # + # ANCHORS ('&name'): deliberately rejected in FULL, not just duplicates. + # Real PyYAML tracks anchor NAMES in a document-scoped composer registry + # and raises ComposerError ("found duplicate anchor") the instant the + # SAME anchor name is declared on a SECOND node anywhere in the document + # -- including on a totally unrelated node far from the token field. This + # line-oriented fallback has no such document-wide registry (and building + # one reliably, across every node shape this recognizer does not even + # parse, risks missing a scope and re-opening the fail-open). So instead + # of emulating the registry, every '&'-anchor property fails closed here, + # unconditionally. This is a conservative OVER-reject relative to PyYAML: + # a single, non-duplicated '&a x' is valid YAML that real PyYAML resolves + # to 'x', but we refuse it too. That is intentional and safe -- fail + # closed can only cost an emitted token PyYAML would have allowed, never + # emit one PyYAML would reject. seen_tag = False - seen_anchor = False while value[:1] in ("!", "&"): if value[0] == "!": # Non-specific tag: '!' then AT LEAST ONE SPACE. A tab is a PyYAML @@ -880,18 +887,8 @@ def _strip_properties(value): return _FAIL seen_tag = True value = value[1:].lstrip(" ") - else: # anchor '&name' - m = _ANCHOR_RE.match(value) - if seen_anchor or m is None: - return _FAIL - rest = m.group(1) - if rest == "": - value = "" # bare '&name' with no following node -> null node - elif rest[0] == " ": - value = rest.lstrip(" ") - else: - return _FAIL # '&name' glued to a non-space (e.g. '&a:') -> error - seen_anchor = True + else: # anchor '&name' -- always fail closed, see comment above + return _FAIL return value diff --git a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh index c17338fe..5a65f7fd 100755 --- a/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh +++ b/packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh @@ -1079,21 +1079,28 @@ assert_token "NEL in a comment fails closed" "" primary git.example write_break_fixture 0x85 dq2 assert_fallback_fails_closed "blank-line-in-quote (NEL run) fails closed" primary git.example -# 24. LEADING NON-SPECIFIC TAG / ANCHOR PROPERTY (#865 round-11 blocker B): the -# round-10 recognizer blanket-rejected any scalar beginning with '!' or '&'. -# But a NON-SPECIFIC tag '! ' (bang + SPACE) and an anchor '&name ' are -# transparent node properties: PyYAML strips them and applies normal implicit -# typing to the node, so '! x'/'&a x' resolve the STRING 'x'. The fallback now -# consumes <=1 non-specific tag and <=1 anchor (either order) and recurses on -# the remainder. Verified empirically vs real PyYAML 6.0.3. -for pair in '! x=x' '&a x=x' '! x y=x y' '! "q"=q' "! 'q'=q" '! &b x=x' '&b ! x=x'; do +# 24. LEADING NON-SPECIFIC TAG / ANCHOR PROPERTY (#865 round-11 blocker B, revised +# in round 12): the round-10 recognizer blanket-rejected any scalar beginning +# with '!' or '&'. Round 11 taught it to strip a transparent NON-SPECIFIC tag +# ('! ' bang + SPACE) and a transparent plain ANCHOR ('&name ') so '! x' / +# '&a x' resolve the STRING 'x', matching PyYAML. Round 12 discovered that the +# anchor half of that was a HIGH fail-open: PyYAML's Composer tracks anchor +# NAMES in a document-scoped registry and raises ComposerError ("found +# duplicate anchor") the instant the SAME name is declared on a SECOND node +# ANYWHERE in the document (even an unrelated one) -- the fallback's +# per-scalar-only view has no such registry and would emit the later token. +# Round 12's fix: reject EVERY '&'-anchor property, unconditionally. The +# transparent NON-SPECIFIC TAG behavior ('! x' -> 'x') is unchanged and still +# verified below; only the anchor half now fails closed (deliberate +# conservative over-reject, verified safe both ways against real PyYAML 6.0.3). +for pair in '! x=x' '! x y=x y' '! "q"=q' "! 'q'=q"; do tv="${pair%%=*}"; want="${pair#*=}" write_fixture "logins: - name: primary url: https://git.example token: ${tv} " - assert_token "property token '$tv' resolves node string" "$want" primary git.example + assert_token "tag property token '$tv' resolves node string" "$want" primary git.example done # Fail-OPEN direction stays shut. '!x' (bang + NON-space) is a tag HANDLE -> @@ -1109,6 +1116,35 @@ for tv in '!x' '!foo x' '* a' '! !x' '! 123' '! true' '! null' '&a &b x'; do assert_token "non-resolving property token '$tv' fails closed" "" primary git.example done +# Round 12: a SINGLE, non-duplicated '&a x' is valid YAML that real PyYAML +# resolves to the string 'x' (round-11 behavior, and still true of the oracle). +# The fallback now rejects it anyway -- a deliberate, endorsed CONSERVATIVE +# over-reject (see the round-12 comment block above): fail-closed can only cost +# an emitted token PyYAML would have allowed, never emit one PyYAML rejects, and +# a per-scalar recognizer cannot safely prove document-wide anchor-name +# uniqueness. assert_fallback_fails_closed also confirms PyYAML really does +# resolve a token here, proving this is a genuine (safe-direction) divergence +# and not an accidental parity loss. +write_fixture 'logins: + - name: primary + url: https://git.example + token: &a x +' +assert_fallback_fails_closed "round-12: single non-duplicated anchor '&a x' now fails closed (conservative over-reject; PyYAML resolves x)" primary git.example +# Same over-reject for the combined tag+anchor forms round 11 used to resolve. +write_fixture 'logins: + - name: primary + url: https://git.example + token: ! &b x +' +assert_fallback_fails_closed "round-12: '! &b x' (tag+anchor) now fails closed (conservative over-reject)" primary git.example +write_fixture 'logins: + - name: primary + url: https://git.example + token: &b ! x +' +assert_fallback_fails_closed "round-12: '&b ! x' (anchor+tag) now fails closed (conservative over-reject)" primary git.example + # Endorsed fail-closed over-reject: an EXPLICIT tag ('!!str x', verbose # '! x') forces a string PyYAML resolves, but the fallback # recognizes only the transparent non-specific tag and fails closed (safer). @@ -1121,4 +1157,56 @@ for tv in '!!str x' '! x'; do assert_fallback_fails_closed "explicit-tag token '$tv' fails closed" primary git.example done +# 25. #865 round 12 HIGH fail-open closure: DUPLICATE ANCHOR NAME across separate +# nodes. Real PyYAML's Composer tracks anchor names in a DOCUMENT-SCOPED +# registry and raises ComposerError ("found duplicate anchor ... first +# occurrence") the instant the SAME anchor name is declared a second time +# ANYWHERE in the document -- failing the WHOLE document closed, no token, +# regardless of how far the duplicate sits from the logins block. The round-11 +# fallback tracked anchors only WITHIN a single scalar's `_strip_properties` +# call, so it had no visibility into a duplicate declared on an unrelated +# node and would still emit the (later) token: fail-open. The round-12 fix +# (reject every '&'-anchor property, unconditionally -- see section 24 above) +# closes this as a strict superset: since NO anchor is ever accepted, a +# duplicate anchor can never slip through. These cases exercise that +# document-wide duplicate-anchor invariant specifically (as opposed to +# section 24's single-anchor-on-the-token-field cases) and pair each fallback +# assertion with confirmation that real PyYAML also fails closed here (via +# ComposerError), proving this was a genuine fail-open, not a hypothetical. +write_fixture 'first: &same one +second: &same two +logins: + - name: primary + url: https://git.example + token: TOK_DUP_ROOT +' +assert_both_fail_closed "round-12: duplicate anchor name on two unrelated root nodes fails closed" primary git.example + +write_fixture 'outer: + nested: &dup x +dup_root: &dup y +logins: + - name: primary + url: https://git.example + token: TOK_DUP_NESTED +' +assert_both_fail_closed "round-12: duplicate anchor name across a nested node and a root node fails closed" primary git.example + +write_fixture 'first: &dup one +logins: + - name: primary + url: https://git.example + token: &dup TOK_DUP_TOKEN_NODE +' +assert_both_fail_closed "round-12: anchor name declared earlier and repeated on the token-bearing node fails closed" primary git.example + +# Regression guard: the non-specific TAG half of section 24 ('! x' -> 'x') is +# UNCHANGED by the round-12 anchor fix and must still resolve. +write_fixture 'logins: + - name: primary + url: https://git.example + token: ! x +' +assert_token "round-12 regression: '! x' (tag, no anchor) still resolves 'x'" "x" primary git.example + echo "Gitea login resolution regression harness passed" -- 2.49.1