The PyYAML-absent fallback's _strip_properties consumed a plain anchor
('&name') per-scalar, with no document-scoped registry of declared anchor
names. Real PyYAML's Composer rejects a duplicate anchor NAME declared
anywhere in the document (ComposerError, whole-document fail), but the
fallback would still emit the later token: fail-open (HIGH). Fix: reject
every '&'-anchor property unconditionally instead of building a hand-rolled
duplicate registry, which guarantees zero fail-open by construction at the
cost of a deliberate, endorsed over-reject on a single non-duplicated
anchor. The '!' non-specific-tag branch is unchanged ('! x' -> 'x' still
resolves).
Two residual over-rejections in the PyYAML-absent fallback of
get_gitea_token_for_login, both fail-closed today but diverging from
real PyYAML 6.0.3 for valid inputs. Neither is a fail-open; the fix
loosens ONLY to exact PyYAML parity and keeps failing closed elsewhere.
Blocker A - embedded printable line break inside a quoted scalar.
The fallback split the raw document with str.splitlines(), which breaks
at NEL(U+0085), LS(U+2028) and PS(U+2029) -- all PRINTABLE to PyYAML's
Reader. Inside a flow (quoted) scalar PyYAML does not break at these: it
line-folds a double/single-quoted scalar (NEL/LF/CR -> one space; LS/PS
verbatim), resolving one token, while splitlines() cut mid-quote and
failed the whole document closed. Replace splitlines() with
_split_logical_lines, a quote-aware splitter that reproduces PyYAML's
flow folding (scan_flow_scalar_spaces/breaks) and raises on a col-0
doc marker (---/...) in a folded continuation. The same code points
UNQUOTED or in a comment still make PyYAML raise, so those stay closed.
Blocker B - leading non-specific tag / anchor property. The recognizer
blanket-rejected any scalar starting with '!' or '&'. But '! ' (bang +
space) and '&name ' are transparent node properties: PyYAML strips them
and applies normal implicit typing, so '! x'/'&a x' resolve the string
'x'. Add _strip_properties, consuming <=1 non-specific tag and <=1 anchor
(either order) and recursing on the remainder. '!x' (tag handle),
'!foo x', non-string nodes ('! 123'), duplicate properties and explicit
tags ('!!str x') still fail closed.
Fallback now matches real PyYAML 6.0.3 (oracle) on a two-direction
differential fuzz of the full Unicode line-break set x {double, single,
plain, comment, token-position} and the full leading tag/anchor space:
833 cases, 0 fail-opens, 0 present-mismatches, 52 endorsed fail-closed
over-rejects (blank-line-in-quote \n folds; explicit !!str/verbose tags;
property+quoted; anchor-without-space). Wrapper scripts pr-review.sh and
issue-comment.sh unchanged (0-line diff). Tests: additions-only sections
23 (quoted-scalar breaks) and 24 (tag/anchor properties).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-10 auditor blockers in the PyYAML-absent fallback of
get_gitea_token_for_login (detect-platform.sh):
Blocker 1 (fail-open): the round-9 control-char guard used a hand-rolled
C0/DEL subset that missed code points PyYAML's Reader also rejects -- the
C1 block (U+0080-0084, U+0086-009F), the surrogate range, and the BMP
noncharacters U+FFFE/U+FFFF -- so the fallback still emitted a token from
documents PyYAML rejects whole. _FORBIDDEN_CONTROL now uses PyYAML 6.0.3's
EXACT Reader.NON_PRINTABLE character class (negated), verified 0-mismatch
against real PyYAML across the full BMP + astral range.
Blocker 2 (over-reject): the recognizer blanket-rejected any plain scalar
containing a flow indicator, but in BLOCK context (where a tea config
value always sits) PyYAML treats ',[]{}' as ordinary content and treats
'!&*|>#%@`' and quotes as significant only at the first non-space char.
The blanket scan dropped tokens PyYAML emits verbatim (internal '!', ','
'[' ']' '{' '}' '#'-not-space-preceded, ':'-not-space-followed). Removed
it; the leading-char guard and the ' #' / ': ' / trailing-':' guards keep
the fail-open direction shut.
Tests: additions-only sections 21 (printable-boundary fail-close for C1 +
noncharacters; NEL/U+00A0/astral still resolve) and 22 (internal
indicators resolve; leading indicator / ': ' inline map / trailing ':'
fail closed). Verified vs real PyYAML 6.0.3 both directions (present as
oracle, absent via PYTHONPATH shadow): wide 602 cases 0 fail-open/0
present-mismatch, dense 712 cases 0 fail-open/0 over-reject/0 mismatch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-9 auditor blockers, both fixed at root cause in the PyYAML-absent
line-parser fallback of get_gitea_token_for_login:
1. Control-character fail-open (dangerous): PyYAML 6.0.3's Reader rejects
the WHOLE document (ReaderError) if a forbidden C0/DEL control byte
{0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F, 0x7F} appears ANYWHERE in the raw
stream -- plain scalar, inside quotes, or a comment -- regardless of
position, verified empirically against the real installed PyYAML 6.0.3.
The fallback previously only guarded tabs and emitted the login token
from such documents. Added a whole-document _FORBIDDEN_CONTROL scan on
the raw text (before splitlines(), which itself splits on some of the
same bytes) so the fallback fails closed identically to PyYAML.
2. Unsigned-exponent float over-rejection: PyYAML's implicit float
resolver requires an EXPLICIT sign on the exponent; unsigned-exponent
spellings (1.0e10, +1.0e10, -1.0e10, 1.0E10, .5e10, 4.e8) are PyYAML
STRINGS, not floats. The fallback's _IMPLICIT_FLOAT pattern allowed an
optional sign, misclassifying these as floats and dropping the token.
Tightened the regex to match PyYAML's resolver exactly.
Also folds in a residual over-rejection found via this round's wide
differential fuzz (1767 cases, 0 fail-opens / 0 over-rejections after the
fix): a plain scalar starting with "?" not followed by whitespace (e.g.
"?x") is a valid PyYAML string, but the fallback's blanket-reject set
treated every leading "?" as illegal. Narrowed to match the existing
space-aware handling already used for "-" and ":".
Adds regression fixtures to test-gitea-login-resolution.sh covering all
three fixes under both PyYAML-present and forced-ImportError-absent runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PyYAML raises a ScannerError on a tab used anywhere outside a quoted
scalar (leading/trailing/embedded in a plain value, immediately before or
after a key colon, or as indentation) and yields no token, accepting tabs
ONLY inside single/double-quoted scalars. The conservative block-YAML
fallback in get_gitea_token_for_login normalized those tabs away -- via
_scalar's top-level .strip(), _KEY_RE's [ \t] separators, _Parser.__init__'s
body.rstrip(), parse_seq's body[1:].strip(), and the inline-map emptiness
checks -- and still emitted the login token: a credential fail-open in the
dangerous direction (less conservative than PyYAML).
Extend the whole-document "only ever more conservative than PyYAML, never
less" invariant to tab/scanner parity:
- _KEY_RE now uses SPACE-only separators (` *:` / `[ ](.*)`), so a tab in a
key/value separator makes the line fail to match and the caller fails closed.
- Every whitespace-normalization site strips SPACES only (strip(" ")/rstrip(" "))
so a tab survives to a fail-closed guard instead of being silently removed:
_scalar top-level strip, quoted-trailing strip, post-comment strip,
_Parser.__init__ body rstrip, parse_seq item strip, and the three inline
emptiness checks.
- _scalar fails closed on any tab remaining in a plain scalar.
Tabs strictly inside quoted scalars are preserved verbatim (unchanged parity),
matching exactly what PyYAML accepts.
Verified empirically against PyYAML 6.0.3: ScannerError for each rejected
tab position; string-preserved for quoted inner tabs. Differential fuzz with
tab re-included: 0 fail-opens over ~6000 inputs; 0 over-rejection across 220
PyYAML-accepted quoted-tab cases. Adds section 17 to the regression harness
(fail-close fixtures for trailing/leading/embedded/after-colon/indentation
tabs; parity fixtures for double- and single-quoted inner tabs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-8: the PyYAML-absent conservative recognizer proved STRUCTURE and
implicit-resolver TYPE but not CONSTRUCTOR VALIDITY. PyYAML safe_load raises a
constructor ValueError on the WHOLE document when a plain scalar matches a typed
implicit resolver but is not constructible (e.g. bad calendar date 2023-99-99,
empty-radix ints 0b_/0x_), yielding no token; the fallback instead treated such
a scalar as a null-equivalent, ignored the malformed key, and still emitted the
valid login token -- a credential fail-open in the dangerous direction.
_scalar now checks constructibility via _constructible (replicating PyYAML
6.0.3 construct_yaml_int/float/timestamp, stdlib-only): a typed scalar that is
not constructor-valid returns _FAIL, which the parser turns into _Bail so the
WHOLE document fails closed exactly as PyYAML does. int and timestamp resolver
patterns are byte-identical to PyYAML's; the float pattern is a strict superset
whose extras are all float()-constructible (fuzz-verified 0/400k raise), so it
never fails closed where PyYAML would emit.
Also fail closed on plain scalars beginning with an indicator a plain scalar may
not start with: '%' (directive) and ',' (flow), and the conditional block
indicators '-'/'?'/':' when followed by whitespace or end-of-value (bare
sequence/complex-key/value indicators PyYAML rejects), while '-x'/'-1'/'?x'/':x'
remain valid plain strings.
Differential fuzz through the real function: 0 fail-opens across 1499 diverse
non-tab cases; targeted fixtures assert fallback == PyYAML == fail-closed for
2023-99-99, 2023-13-01, bad-hour timestamp, 0b_, 0x_, 0x__, %broken, ',bad',
bare '-'/'- '/'? key', and assert the token STILL resolves for constructible/
string look-alikes (valid date/datetime, 0o_, 4.e8, 0x1f, -x, :x). Both PyYAML
present and forced-absent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold the remaining round-7 audit items into the tea-CLI comment-invocation fix.
Fallback token parser (detect-platform.sh, test-gitea-login-resolution.sh):
Replace the line-by-line scalar fallback with a strict CONSERVATIVE block-YAML
recognizer that reconstructs the same object PyYAML would or fails closed the
instant it meets anything outside the tea-config subset. Closes 5 structural
fail-open classes the old parser missed (nested-shadow logins, block-scalar
shadow, duplicate root key / login name / token field, malformed-after-valid,
extra-document / end-marker). Validated by a 360k-check differential fuzz vs
real PyYAML (0 fail-open) plus explicit forced-PyYAML-absence fixtures.
ITEM 1 (pr-review.sh) current-head TOCTOU: after the exact review-id read-back
succeeds, re-read the live PR head and fail closed if it advanced past the
submitted commit_id, so a review is never reported as covering a superseded tip.
ITEM 2 (pr-review.sh comment action): require the returned resource be a
pull_request (populated pull_request_url); reject a bare issue_url so a plain
issue #N cannot masquerade as a verified PR comment. issue-comment.sh keeps its
broader issue-or-PR acceptance.
ITEM 3a (both wrappers): move the Authorization bearer OUT of curl argv into a
private mode-0600 curl --config file (gitea_write_auth_config), removed on every
exit path, so the token never appears in the process table.
ITEM 3b (pr-review.sh): bind the review body with presence + string-type + exact
equality instead of `(body or "")`, so a non-empty submitted body persisted as
null/missing fails closed.
Tests: add race, plain-issue, argv-capture (no token printed), and null-body
fixtures; broaden temp-leak checks to the auth-config files. Full gate set green
(bash -n, shellcheck -x -S warning, prettier, all 3 REST/resolution suites with
PyYAML and forced-absent, cold TURBO_FORCE turbo 14/14).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-7 blocker-1 residual: the PyYAML-absent line-parser fallback in
detect-platform.sh (_strip_scalar) returned a stringified scalar for
UNQUOTED YAML values that PyYAML's implicit resolver types as a
non-string (int/null/bool/float/timestamp). That bypassed _accept's
isinstance(str) guard and could surface a garbage credential (e.g.
"12345", "null", "true") where the PyYAML path resolves NO token and
fails closed -- violating the module invariant that the fallback is only
ever MORE conservative than PyYAML, never less.
Root cause fix: mirror PyYAML 6.0.3's SafeLoader implicit resolver. An
unquoted plain scalar matching the null/bool/int/float/timestamp forms
now returns None (fail closed); a quoted scalar is always a string and is
accepted verbatim (quote-stripped) as before. Quoted-string handling,
scope-aware attribution, indentation, and inline-comment stripping are
unchanged. The predicate was fuzzed against real PyYAML over ~800k random
tokens with zero fail-open divergences.
Extends the forced-PyYAML-absence parser-equivalence harness with
token: 12345/null/~/yes/true/3.14 (each fails closed identically to
PyYAML) and token: "12345"/'abc' (quoted literals still accepted).
Blockers 2/3/4 (port-bound host, origin+full-path URL pin, review-body
binding) are untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-6 remediation for PR #866 addressing four cross-host exact-diff audit
blockers (REQUEST_CHANGES governs):
Blocker 1 (detect-platform.sh get_gitea_token_for_login line parser): the
PyYAML-absence fallback attributed any `key: value` at any depth to the current
login, so a token from a nested sub-map or a mis-indented line could be selected
where PyYAML fails closed, and inline comments were not stripped. The fallback is
now scope-aware — a field attaches only at the entry's own direct-field
indentation, only list items at the login list's own dash indent open an entry —
and _strip_scalar strips a trailing inline comment like PyYAML. It is therefore
only ever MORE conservative than PyYAML, never less.
Blocker 2 (detect-platform.sh host bind): credential binding compared
parsed.hostname only, dropping the port, so a :9443 login satisfied a portless
host and a matching :8443 login was rejected. Binding now normalizes scheme +
host + effective port (scheme default applied symmetrically) exactly like
gitea_url_matches_host.
Blocker 3 (issue-comment.sh + pr-review.sh read-back URL check): verification
used path.endswith, accepting a look-alike host or a decoy path prefix. It now
pins the returned issue_url/pull_request_url ORIGIN (scheme+host+effective-port)
and FULL path (deployment prefix + exact owner/repo + kind + number). A new
GITEA_WEB_BASE is exported from gitea_resolve_api_for_login for this.
Blocker 4 (pr-review.sh gitea_submit_review_verified): the submitted review body
was not verified, so a finalized/reused pending review id carrying foreign
Content passed. The persisted body is now bound to the exact submitted body.
Tests: added forced-PyYAML-absence parser-equivalence fixtures (nested sub-map,
sibling, mis-indent, inline comment, tab-indent fail-closed, port match/mismatch)
to test-gitea-login-resolution.sh; URL-forgery fail-closed cases
(wrong-host/owner/repo + prefix injection) to both write suites; and a
reused-review-id body-mismatch case to the pr-review suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI-red root cause (classification a: my round-4 change fails in the clean/cold
CI env): get_gitea_token_for_login hard-required PyYAML (`import yaml`), which is
absent on CI's node:24-alpine (python3 without py3-yaml). Round-4's --login
override cases were the first to exercise that path, turning the mosaic package
test (test:framework-shell -> test-pr-review-gitea-comment.sh) RED. Fix: add an
indentation-aware line-parser fallback that resolves the SAME per-name token
PyYAML would from tea's flat `logins:` list; PyYAML stays the fast path. This
also repairs a latent production defect (--login overrides were silently
unusable on any PyYAML-less host).
Auditor blockers folded into the same round-5:
1. issue_url vs pull_request_url shape (correctness): Gitea populates WEB (html)
URLs in issue_url/pull_request_url, not API paths, and a PR-conversation
comment carries pull_request_url (issue_url empty). Verification now accepts
either web shape scoped to the repo slug + number, so a durable write is never
rejected for URL shape. Test stubs now emit the REAL Gitea web shapes.
2. Cross-host credential binding (security): get_gitea_token_for_login now takes
the repo host and requires the matched login's configured URL host to equal
it; an override login configured for a different host FAILS CLOSED instead of
sending a cross-host credential. Regression tests added to both suites.
3. Non-exhaustive enumeration (false-fail): removed the redundant, non-exhaustive
post-verification list enumeration (gitea_fetch_all + confirm_*_enumerable)
from both wrappers; the exact-id GET is authoritative. Pagination cases
dropped; a guard asserts no list enumeration is performed.
4. Trap clobbering / temp-file leak (security/hygiene): removing the nested
enumeration eliminates the RETURN-trap nesting that clobbered caller cleanup;
remaining RETURN traps are single/non-nested and clean up on all exit paths.
Temp-file leak regression tests (success + failure paths) added to both suites.
5. README: corrected the exhaustive-pagination claim and documented host-bound
--login selection.
Preserves every round-2/3/4 fix (explicit --login fail-closed at all write
sites, token->identity attribution seam). Gates: cold `pnpm turbo run test
--filter=@mosaicstack/mosaic` green (14/14); full test-*.sh suite green with AND
without PyYAML; bash -n, shellcheck -x -S warning, prettier --check README clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the tea-based write + boundary/author read-back with a direct Gitea
REST POST that returns the created record's id, and verify that exact record.
BLOCKER 2 (credential ordering): resolve the acting identity, the write token,
and the read-back token from the SAME effective login. A --login override now
selects the credential used for the POST, GET /user, and the GET-by-id
read-back, so an overridden write is verified against the identity that
performed it -- not the host default. Login-name resolution is best-effort and
non-fatal (the override always wins; otherwise fall back to the host
credential), so exotic/ported hosts still resolve a token.
BLOCKER 1+3 (attribution + tautological tests): the write is now
POST /issues/{n}/comments or POST /pulls/{n}/reviews (event + body + commit_id
== PR head), parsing the provider-returned created id. Verification GETs that
exact id and checks author == acting identity and body (comments) or state +
commit_id (reviews). Keying on the created id closes the concurrency window:
a no-op create yields no id and fails closed with no list-scan fallback, and a
concurrent same-identity record has a different id. The review body travels in
the review submit, removing the separate detached comment.
Tests: the curl stub now models a real server with persistent on-disk
review/comment state -- a POST actually creates+persists a record and returns
its id, and the read-back reads that same state (no fabricated record for the
wrapper to find). Adds same-identity no-op-concurrent and author-mismatch
fail-closed cases for both comments and reviews, and >page-1 pagination
coverage for both. README "Durable review provenance" refreshed for the REST
mechanism.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
