Round-9 auditor blockers, both fixed at root cause in the PyYAML-absent
line-parser fallback of get_gitea_token_for_login:
1. Control-character fail-open (dangerous): PyYAML 6.0.3's Reader rejects
the WHOLE document (ReaderError) if a forbidden C0/DEL control byte
{0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F, 0x7F} appears ANYWHERE in the raw
stream -- plain scalar, inside quotes, or a comment -- regardless of
position, verified empirically against the real installed PyYAML 6.0.3.
The fallback previously only guarded tabs and emitted the login token
from such documents. Added a whole-document _FORBIDDEN_CONTROL scan on
the raw text (before splitlines(), which itself splits on some of the
same bytes) so the fallback fails closed identically to PyYAML.
2. Unsigned-exponent float over-rejection: PyYAML's implicit float
resolver requires an EXPLICIT sign on the exponent; unsigned-exponent
spellings (1.0e10, +1.0e10, -1.0e10, 1.0E10, .5e10, 4.e8) are PyYAML
STRINGS, not floats. The fallback's _IMPLICIT_FLOAT pattern allowed an
optional sign, misclassifying these as floats and dropping the token.
Tightened the regex to match PyYAML's resolver exactly.
Also folds in a residual over-rejection found via this round's wide
differential fuzz (1767 cases, 0 fail-opens / 0 over-rejections after the
fix): a plain scalar starting with "?" not followed by whitespace (e.g.
"?x") is a valid PyYAML string, but the fallback's blanket-reject set
treated every leading "?" as illegal. Narrowed to match the existing
space-aware handling already used for "-" and ":".
Adds regression fixtures to test-gitea-login-resolution.sh covering all
three fixes under both PyYAML-present and forced-ImportError-absent runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PyYAML raises a ScannerError on a tab used anywhere outside a quoted
scalar (leading/trailing/embedded in a plain value, immediately before or
after a key colon, or as indentation) and yields no token, accepting tabs
ONLY inside single/double-quoted scalars. The conservative block-YAML
fallback in get_gitea_token_for_login normalized those tabs away -- via
_scalar's top-level .strip(), _KEY_RE's [ \t] separators, _Parser.__init__'s
body.rstrip(), parse_seq's body[1:].strip(), and the inline-map emptiness
checks -- and still emitted the login token: a credential fail-open in the
dangerous direction (less conservative than PyYAML).
Extend the whole-document "only ever more conservative than PyYAML, never
less" invariant to tab/scanner parity:
- _KEY_RE now uses SPACE-only separators (` *:` / `[ ](.*)`), so a tab in a
key/value separator makes the line fail to match and the caller fails closed.
- Every whitespace-normalization site strips SPACES only (strip(" ")/rstrip(" "))
so a tab survives to a fail-closed guard instead of being silently removed:
_scalar top-level strip, quoted-trailing strip, post-comment strip,
_Parser.__init__ body rstrip, parse_seq item strip, and the three inline
emptiness checks.
- _scalar fails closed on any tab remaining in a plain scalar.
Tabs strictly inside quoted scalars are preserved verbatim (unchanged parity),
matching exactly what PyYAML accepts.
Verified empirically against PyYAML 6.0.3: ScannerError for each rejected
tab position; string-preserved for quoted inner tabs. Differential fuzz with
tab re-included: 0 fail-opens over ~6000 inputs; 0 over-rejection across 220
PyYAML-accepted quoted-tab cases. Adds section 17 to the regression harness
(fail-close fixtures for trailing/leading/embedded/after-colon/indentation
tabs; parity fixtures for double- and single-quoted inner tabs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-8: the PyYAML-absent conservative recognizer proved STRUCTURE and
implicit-resolver TYPE but not CONSTRUCTOR VALIDITY. PyYAML safe_load raises a
constructor ValueError on the WHOLE document when a plain scalar matches a typed
implicit resolver but is not constructible (e.g. bad calendar date 2023-99-99,
empty-radix ints 0b_/0x_), yielding no token; the fallback instead treated such
a scalar as a null-equivalent, ignored the malformed key, and still emitted the
valid login token -- a credential fail-open in the dangerous direction.
_scalar now checks constructibility via _constructible (replicating PyYAML
6.0.3 construct_yaml_int/float/timestamp, stdlib-only): a typed scalar that is
not constructor-valid returns _FAIL, which the parser turns into _Bail so the
WHOLE document fails closed exactly as PyYAML does. int and timestamp resolver
patterns are byte-identical to PyYAML's; the float pattern is a strict superset
whose extras are all float()-constructible (fuzz-verified 0/400k raise), so it
never fails closed where PyYAML would emit.
Also fail closed on plain scalars beginning with an indicator a plain scalar may
not start with: '%' (directive) and ',' (flow), and the conditional block
indicators '-'/'?'/':' when followed by whitespace or end-of-value (bare
sequence/complex-key/value indicators PyYAML rejects), while '-x'/'-1'/'?x'/':x'
remain valid plain strings.
Differential fuzz through the real function: 0 fail-opens across 1499 diverse
non-tab cases; targeted fixtures assert fallback == PyYAML == fail-closed for
2023-99-99, 2023-13-01, bad-hour timestamp, 0b_, 0x_, 0x__, %broken, ',bad',
bare '-'/'- '/'? key', and assert the token STILL resolves for constructible/
string look-alikes (valid date/datetime, 0o_, 4.e8, 0x1f, -x, :x). Both PyYAML
present and forced-absent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold the remaining round-7 audit items into the tea-CLI comment-invocation fix.
Fallback token parser (detect-platform.sh, test-gitea-login-resolution.sh):
Replace the line-by-line scalar fallback with a strict CONSERVATIVE block-YAML
recognizer that reconstructs the same object PyYAML would or fails closed the
instant it meets anything outside the tea-config subset. Closes 5 structural
fail-open classes the old parser missed (nested-shadow logins, block-scalar
shadow, duplicate root key / login name / token field, malformed-after-valid,
extra-document / end-marker). Validated by a 360k-check differential fuzz vs
real PyYAML (0 fail-open) plus explicit forced-PyYAML-absence fixtures.
ITEM 1 (pr-review.sh) current-head TOCTOU: after the exact review-id read-back
succeeds, re-read the live PR head and fail closed if it advanced past the
submitted commit_id, so a review is never reported as covering a superseded tip.
ITEM 2 (pr-review.sh comment action): require the returned resource be a
pull_request (populated pull_request_url); reject a bare issue_url so a plain
issue #N cannot masquerade as a verified PR comment. issue-comment.sh keeps its
broader issue-or-PR acceptance.
ITEM 3a (both wrappers): move the Authorization bearer OUT of curl argv into a
private mode-0600 curl --config file (gitea_write_auth_config), removed on every
exit path, so the token never appears in the process table.
ITEM 3b (pr-review.sh): bind the review body with presence + string-type + exact
equality instead of `(body or "")`, so a non-empty submitted body persisted as
null/missing fails closed.
Tests: add race, plain-issue, argv-capture (no token printed), and null-body
fixtures; broaden temp-leak checks to the auth-config files. Full gate set green
(bash -n, shellcheck -x -S warning, prettier, all 3 REST/resolution suites with
PyYAML and forced-absent, cold TURBO_FORCE turbo 14/14).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-7 blocker-1 residual: the PyYAML-absent line-parser fallback in
detect-platform.sh (_strip_scalar) returned a stringified scalar for
UNQUOTED YAML values that PyYAML's implicit resolver types as a
non-string (int/null/bool/float/timestamp). That bypassed _accept's
isinstance(str) guard and could surface a garbage credential (e.g.
"12345", "null", "true") where the PyYAML path resolves NO token and
fails closed -- violating the module invariant that the fallback is only
ever MORE conservative than PyYAML, never less.
Root cause fix: mirror PyYAML 6.0.3's SafeLoader implicit resolver. An
unquoted plain scalar matching the null/bool/int/float/timestamp forms
now returns None (fail closed); a quoted scalar is always a string and is
accepted verbatim (quote-stripped) as before. Quoted-string handling,
scope-aware attribution, indentation, and inline-comment stripping are
unchanged. The predicate was fuzzed against real PyYAML over ~800k random
tokens with zero fail-open divergences.
Extends the forced-PyYAML-absence parser-equivalence harness with
token: 12345/null/~/yes/true/3.14 (each fails closed identically to
PyYAML) and token: "12345"/'abc' (quoted literals still accepted).
Blockers 2/3/4 (port-bound host, origin+full-path URL pin, review-body
binding) are untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-6 remediation for PR #866 addressing four cross-host exact-diff audit
blockers (REQUEST_CHANGES governs):
Blocker 1 (detect-platform.sh get_gitea_token_for_login line parser): the
PyYAML-absence fallback attributed any `key: value` at any depth to the current
login, so a token from a nested sub-map or a mis-indented line could be selected
where PyYAML fails closed, and inline comments were not stripped. The fallback is
now scope-aware — a field attaches only at the entry's own direct-field
indentation, only list items at the login list's own dash indent open an entry —
and _strip_scalar strips a trailing inline comment like PyYAML. It is therefore
only ever MORE conservative than PyYAML, never less.
Blocker 2 (detect-platform.sh host bind): credential binding compared
parsed.hostname only, dropping the port, so a :9443 login satisfied a portless
host and a matching :8443 login was rejected. Binding now normalizes scheme +
host + effective port (scheme default applied symmetrically) exactly like
gitea_url_matches_host.
Blocker 3 (issue-comment.sh + pr-review.sh read-back URL check): verification
used path.endswith, accepting a look-alike host or a decoy path prefix. It now
pins the returned issue_url/pull_request_url ORIGIN (scheme+host+effective-port)
and FULL path (deployment prefix + exact owner/repo + kind + number). A new
GITEA_WEB_BASE is exported from gitea_resolve_api_for_login for this.
Blocker 4 (pr-review.sh gitea_submit_review_verified): the submitted review body
was not verified, so a finalized/reused pending review id carrying foreign
Content passed. The persisted body is now bound to the exact submitted body.
Tests: added forced-PyYAML-absence parser-equivalence fixtures (nested sub-map,
sibling, mis-indent, inline comment, tab-indent fail-closed, port match/mismatch)
to test-gitea-login-resolution.sh; URL-forgery fail-closed cases
(wrong-host/owner/repo + prefix injection) to both write suites; and a
reused-review-id body-mismatch case to the pr-review suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
