fix(git): fail closed on tabs the YAML fallback recognizer would swallow (#865)
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
PyYAML raises a ScannerError on a tab used anywhere outside a quoted
scalar (leading/trailing/embedded in a plain value, immediately before or
after a key colon, or as indentation) and yields no token, accepting tabs
ONLY inside single/double-quoted scalars. The conservative block-YAML
fallback in get_gitea_token_for_login normalized those tabs away -- via
_scalar's top-level .strip(), _KEY_RE's [ \t] separators, _Parser.__init__'s
body.rstrip(), parse_seq's body[1:].strip(), and the inline-map emptiness
checks -- and still emitted the login token: a credential fail-open in the
dangerous direction (less conservative than PyYAML).
Extend the whole-document "only ever more conservative than PyYAML, never
less" invariant to tab/scanner parity:
- _KEY_RE now uses SPACE-only separators (` *:` / `[ ](.*)`), so a tab in a
key/value separator makes the line fail to match and the caller fails closed.
- Every whitespace-normalization site strips SPACES only (strip(" ")/rstrip(" "))
so a tab survives to a fail-closed guard instead of being silently removed:
_scalar top-level strip, quoted-trailing strip, post-comment strip,
_Parser.__init__ body rstrip, parse_seq item strip, and the three inline
emptiness checks.
- _scalar fails closed on any tab remaining in a plain scalar.
Tabs strictly inside quoted scalars are preserved verbatim (unchanged parity),
matching exactly what PyYAML accepts.
Verified empirically against PyYAML 6.0.3: ScannerError for each rejected
tab position; string-preserved for quoted inner tabs. Differential fuzz with
tab re-included: 0 fail-opens over ~6000 inputs; 0 over-rejection across 220
PyYAML-accepted quoted-tab cases. Adds section 17 to the regression harness
(fail-close fixtures for trailing/leading/embedded/after-colon/indentation
tabs; parity fixtures for double- and single-quoted inner tabs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -719,4 +719,61 @@ assert_token "4.e8 float look-alike still resolves token" "TOK_PLAIN" primary gi
|
||||
write_bad_key_fixture '0x1f'
|
||||
assert_token "valid hex int unrelated key still resolves token" "TOK_PLAIN" primary git.example
|
||||
|
||||
# 17. TAB / SCANNER PARITY: PyYAML raises a ScannerError on a tab used anywhere
|
||||
# outside a quoted scalar -- leading, trailing, or embedded in a plain value,
|
||||
# immediately after a key colon, before a key colon, or as indentation -- and
|
||||
# yields NO token, accepting tabs ONLY inside single/double-quoted scalars
|
||||
# (where the tab is preserved as string content). A prior fallback swallowed
|
||||
# those tabs (via .strip()/.rstrip() normalization and [ \t] key separators)
|
||||
# and still emitted the login token -- a fail-open in the dangerous direction.
|
||||
# The recognizer now fails CLOSED for the whole document on any tab PyYAML
|
||||
# rejects, while preserving the tabs PyYAML keeps (inside quotes). All tab
|
||||
# positions were verified empirically against PyYAML 6.0.3 (ScannerError for
|
||||
# each rejected position; string-preserved for quoted inner tabs).
|
||||
TAB=$'\t'
|
||||
# Fail-close: a tab in a plain value position (trailing / leading / embedded).
|
||||
write_bad_key_fixture "l4o${TAB}"
|
||||
assert_both_fail_closed "trailing tab in plain value fails closed" primary git.example
|
||||
write_bad_key_fixture "${TAB}9"
|
||||
assert_both_fail_closed "leading tab in plain value fails closed" primary git.example
|
||||
write_bad_key_fixture "a${TAB}b"
|
||||
assert_both_fail_closed "embedded tab in plain value fails closed" primary git.example
|
||||
# Fail-close: a tab immediately after the key colon (no separating space).
|
||||
write_fixture "bad:${TAB}9
|
||||
logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: TOK_PLAIN
|
||||
"
|
||||
assert_both_fail_closed "tab immediately after key colon fails closed" primary git.example
|
||||
# Fail-close: a tab used as indentation (before a sequence dash).
|
||||
write_fixture "logins:
|
||||
${TAB}- name: primary
|
||||
url: https://git.example
|
||||
token: TOK_PLAIN
|
||||
"
|
||||
assert_both_fail_closed "tab used as indentation fails closed" primary git.example
|
||||
# Fail-close: a tab trailing a sequence-mapping field value.
|
||||
write_fixture "logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: TOK_PLAIN${TAB}
|
||||
"
|
||||
assert_both_fail_closed "tab trailing a seq field value fails closed" primary git.example
|
||||
# NOT over-broad: a tab strictly INSIDE a quoted scalar is valid YAML (PyYAML
|
||||
# keeps it as string content), so the document parses and the login token still
|
||||
# resolves in BOTH paths -- double-quoted and single-quoted.
|
||||
write_bad_key_fixture "\"a${TAB}b\""
|
||||
assert_token "tab inside a double-quoted value still resolves token" "TOK_PLAIN" primary git.example
|
||||
write_bad_key_fixture "'a${TAB}b'"
|
||||
assert_token "tab inside a single-quoted value still resolves token" "TOK_PLAIN" primary git.example
|
||||
# ...and a quoted token value carrying an inner tab resolves to the exact string
|
||||
# (tab preserved), identical to PyYAML's construction.
|
||||
write_fixture "logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: \"T${TAB}OK\"
|
||||
"
|
||||
assert_token "quoted token with inner tab resolves verbatim" "T${TAB}OK" primary git.example
|
||||
|
||||
echo "Gitea login resolution regression harness passed"
|
||||
|
||||
Reference in New Issue
Block a user