fix(git): match PyYAML Reader.NON_PRINTABLE and block-context plain rules in tea token fallback (#865)
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
Round-10 auditor blockers in the PyYAML-absent fallback of
get_gitea_token_for_login (detect-platform.sh):
Blocker 1 (fail-open): the round-9 control-char guard used a hand-rolled
C0/DEL subset that missed code points PyYAML's Reader also rejects -- the
C1 block (U+0080-0084, U+0086-009F), the surrogate range, and the BMP
noncharacters U+FFFE/U+FFFF -- so the fallback still emitted a token from
documents PyYAML rejects whole. _FORBIDDEN_CONTROL now uses PyYAML 6.0.3's
EXACT Reader.NON_PRINTABLE character class (negated), verified 0-mismatch
against real PyYAML across the full BMP + astral range.
Blocker 2 (over-reject): the recognizer blanket-rejected any plain scalar
containing a flow indicator, but in BLOCK context (where a tea config
value always sits) PyYAML treats ',[]{}' as ordinary content and treats
'!&*|>#%@`' and quotes as significant only at the first non-space char.
The blanket scan dropped tokens PyYAML emits verbatim (internal '!', ','
'[' ']' '{' '}' '#'-not-space-preceded, ':'-not-space-followed). Removed
it; the leading-char guard and the ' #' / ': ' / trailing-':' guards keep
the fail-open direction shut.
Tests: additions-only sections 21 (printable-boundary fail-close for C1 +
noncharacters; NEL/U+00A0/astral still resolve) and 22 (internal
indicators resolve; leading indicator / ': ' inline map / trailing ':'
fail closed). Verified vs real PyYAML 6.0.3 both directions (present as
oracle, absent via PYTHONPATH shadow): wide 602 cases 0 fail-open/0
present-mismatch, dense 712 cases 0 fail-open/0 over-reject/0 mismatch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -893,4 +893,118 @@ write_fixture 'logins:
|
||||
'
|
||||
assert_token "question-mark-not-space is a plain string, token resolves" "?x" primary git.example
|
||||
|
||||
# 21. PRINTABLE-BOUNDARY FAIL-CLOSE (#865 round-10 blocker 1): the round-9 guard
|
||||
# used a hand-rolled C0/DEL subset that MISSED code points PyYAML's Reader
|
||||
# also rejects -- the C1 block (U+0080-0084, U+0086-009F) and the BMP
|
||||
# noncharacters U+FFFE/U+FFFF -- so the fallback still emitted the token from
|
||||
# documents PyYAML rejects whole (fail-open). The guard now uses PyYAML
|
||||
# 6.0.3's EXACT Reader.NON_PRINTABLE character class (see detect-platform.sh
|
||||
# _FORBIDDEN_CONTROL). Verified empirically against real PyYAML 6.0.3:
|
||||
# PRINTABLE = {0x09,0x0A,0x0D, 0x20-0x7E, 0x85(NEL), 0xA0-0xD7FF,
|
||||
# 0xE000-0xFFFD, 0x10000-0x10FFFF}; everything else fails the whole document
|
||||
# closed. write_codepoint_fixture emits a chosen Unicode code point's real
|
||||
# UTF-8 bytes (via python3, since bash strings cannot faithfully carry many
|
||||
# of these) into a selectable position, then the login block follows.
|
||||
write_codepoint_fixture() {
|
||||
# $1 = hex code point (e.g. 0x80); $2 = position: field|comment|token|nelterm
|
||||
CP_HEX="$1" CP_POS="$2" python3 - "$FIXTURE_XDG/tea/config.yml" <<'PY'
|
||||
import sys
|
||||
cp = int(__import__("os").environ["CP_HEX"], 16)
|
||||
pos = __import__("os").environ["CP_POS"]
|
||||
ch = chr(cp)
|
||||
head = "logins:\n - name: primary\n url: https://git.example\n token: TOK_PLAIN\n"
|
||||
if pos == "field":
|
||||
doc = head + "other: x" + ch + "y\n"
|
||||
elif pos == "comment":
|
||||
doc = head + "# note x" + ch + "y here\n"
|
||||
elif pos == "token":
|
||||
doc = "logins:\n - name: primary\n url: https://git.example\n token: T" + ch + "K\n"
|
||||
elif pos == "nelterm":
|
||||
# NEL (U+0085) used as the line terminator throughout: PyYAML treats it as a
|
||||
# line break (printable, NOT a ReaderError) and resolves the token; the
|
||||
# fallback's splitlines() splits on NEL identically -> parity, token resolves.
|
||||
doc = ("logins:" + ch + " - name: primary" + ch
|
||||
+ " url: https://git.example" + ch + " token: TOK_NEL" + ch)
|
||||
else:
|
||||
raise SystemExit("bad pos")
|
||||
with open(sys.argv[1], "w", encoding="utf-8") as f:
|
||||
f.write(doc)
|
||||
PY
|
||||
}
|
||||
|
||||
# C1-block + BMP-noncharacter code points fail the WHOLE document closed in an
|
||||
# unrelated field and in a comment, exactly as PyYAML's ReaderError does.
|
||||
for cphex in 0x80 0x81 0x84 0x86 0x9f 0xfffe 0xffff; do
|
||||
write_codepoint_fixture "$cphex" field
|
||||
assert_both_fail_closed "code point $cphex in unrelated field fails closed" primary git.example
|
||||
write_codepoint_fixture "$cphex" comment
|
||||
assert_both_fail_closed "code point $cphex in a comment fails closed" primary git.example
|
||||
done
|
||||
|
||||
# NOT over-broad: printable code points PyYAML ACCEPTS must still resolve the
|
||||
# token in BOTH paths -- NEL(0x85) as a line separator, U+00A0 (NBSP) inside a
|
||||
# value, and an astral code point (U+1F600) inside the token value.
|
||||
write_codepoint_fixture 0x85 nelterm
|
||||
assert_token "NEL (U+0085) line-terminator resolves token" "TOK_NEL" primary git.example
|
||||
write_codepoint_fixture 0xa0 field
|
||||
assert_token "U+00A0 in unrelated value still resolves token" "TOK_PLAIN" primary git.example
|
||||
write_codepoint_fixture 0x1f600 token
|
||||
assert_token "astral U+1F600 inside token resolves verbatim" "$(printf 'T\360\237\230\200K')" primary git.example
|
||||
|
||||
# 22. INTERNAL-INDICATOR OVER-REJECTION (#865 round-10 blocker 2): the round-9
|
||||
# recognizer blanket-rejected any plain scalar CONTAINING a flow indicator
|
||||
# ([]{}*&!), but in BLOCK context PyYAML treats ',[]{}' as ordinary content
|
||||
# and treats '!&*#...' as significant ONLY at the FIRST non-space char. So an
|
||||
# INTERNAL indicator is legal plain-string content and PyYAML emits the token
|
||||
# verbatim; the fallback dropped it (over-rejection). Verified empirically
|
||||
# against real PyYAML 6.0.3. The fix removes the blanket internal scan while
|
||||
# the leading-char guard and the ' #'/': '/trailing-':' guards keep the
|
||||
# fail-OPEN direction shut.
|
||||
for tv in 'a!b' 'a,b' 'a[b' 'a]b' 'a{b' 'a}b' 'a&b' 'a*b' 'a[b]c' 'a{b}c' 'a,b,c' 'a#b' 'a:b'; do
|
||||
write_fixture "logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: ${tv}
|
||||
"
|
||||
assert_token "internal-indicator token '$tv' resolves verbatim" "$tv" primary git.example
|
||||
done
|
||||
|
||||
# Fail-OPEN direction stays shut: a LEADING indicator, a ' #' comment tail, an
|
||||
# internal ': ' (colon-space) inline map, and a trailing ':' each make PyYAML
|
||||
# resolve NO usable string token (tag/flow/anchor reject or None, comment strip,
|
||||
# or a mapping), so BOTH must fail closed. (Leading tag/anchor/flow are the
|
||||
# documented, round-9-approved structural fail-closed class; kept intact here.)
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: !x
|
||||
'
|
||||
assert_both_fail_closed "leading '!' tag token fails closed" primary git.example
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: [a]
|
||||
'
|
||||
assert_both_fail_closed "leading '[' flow-seq token fails closed" primary git.example
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: a # trailing comment
|
||||
'
|
||||
# ' #' comment tail: PyYAML strips the comment -> token is the string 'a', which
|
||||
# still resolves. This is the NOT-over-broad boundary partner of the guard.
|
||||
assert_token "space-hash comment tail strips to plain token" "a" primary git.example
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: a: b
|
||||
'
|
||||
assert_both_fail_closed "internal colon-space (inline map) token fails closed" primary git.example
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: ab:
|
||||
'
|
||||
assert_both_fail_closed "trailing colon (map indicator) token fails closed" primary git.example
|
||||
|
||||
echo "Gitea login resolution regression harness passed"
|
||||
|
||||
Reference in New Issue
Block a user