fix(git): fail closed on tabs the YAML fallback recognizer would swallow (#865)
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

PyYAML raises a ScannerError on a tab used anywhere outside a quoted
scalar (leading/trailing/embedded in a plain value, immediately before or
after a key colon, or as indentation) and yields no token, accepting tabs
ONLY inside single/double-quoted scalars. The conservative block-YAML
fallback in get_gitea_token_for_login normalized those tabs away -- via
_scalar's top-level .strip(), _KEY_RE's [ \t] separators, _Parser.__init__'s
body.rstrip(), parse_seq's body[1:].strip(), and the inline-map emptiness
checks -- and still emitted the login token: a credential fail-open in the
dangerous direction (less conservative than PyYAML).

Extend the whole-document "only ever more conservative than PyYAML, never
less" invariant to tab/scanner parity:
- _KEY_RE now uses SPACE-only separators (` *:` / `[ ](.*)`), so a tab in a
  key/value separator makes the line fail to match and the caller fails closed.
- Every whitespace-normalization site strips SPACES only (strip(" ")/rstrip(" "))
  so a tab survives to a fail-closed guard instead of being silently removed:
  _scalar top-level strip, quoted-trailing strip, post-comment strip,
  _Parser.__init__ body rstrip, parse_seq item strip, and the three inline
  emptiness checks.
- _scalar fails closed on any tab remaining in a plain scalar.
Tabs strictly inside quoted scalars are preserved verbatim (unchanged parity),
matching exactly what PyYAML accepts.

Verified empirically against PyYAML 6.0.3: ScannerError for each rejected
tab position; string-preserved for quoted inner tabs. Differential fuzz with
tab re-included: 0 fail-opens over ~6000 inputs; 0 over-rejection across 220
PyYAML-accepted quoted-tab cases. Adds section 17 to the regression harness
(fail-close fixtures for trailing/leading/embedded/after-colon/indentation
tabs; parity fixtures for double- and single-quoted inner tabs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Hermes Agent
2026-07-22 00:05:39 -05:00
parent 044744339b
commit 6053e1ee4c
2 changed files with 84 additions and 10 deletions

View File

@@ -719,4 +719,61 @@ assert_token "4.e8 float look-alike still resolves token" "TOK_PLAIN" primary gi
write_bad_key_fixture '0x1f'
assert_token "valid hex int unrelated key still resolves token" "TOK_PLAIN" primary git.example
# 17. TAB / SCANNER PARITY: PyYAML raises a ScannerError on a tab used anywhere
# outside a quoted scalar -- leading, trailing, or embedded in a plain value,
# immediately after a key colon, before a key colon, or as indentation -- and
# yields NO token, accepting tabs ONLY inside single/double-quoted scalars
# (where the tab is preserved as string content). A prior fallback swallowed
# those tabs (via .strip()/.rstrip() normalization and [ \t] key separators)
# and still emitted the login token -- a fail-open in the dangerous direction.
# The recognizer now fails CLOSED for the whole document on any tab PyYAML
# rejects, while preserving the tabs PyYAML keeps (inside quotes). All tab
# positions were verified empirically against PyYAML 6.0.3 (ScannerError for
# each rejected position; string-preserved for quoted inner tabs).
TAB=$'\t'
# Fail-close: a tab in a plain value position (trailing / leading / embedded).
write_bad_key_fixture "l4o${TAB}"
assert_both_fail_closed "trailing tab in plain value fails closed" primary git.example
write_bad_key_fixture "${TAB}9"
assert_both_fail_closed "leading tab in plain value fails closed" primary git.example
write_bad_key_fixture "a${TAB}b"
assert_both_fail_closed "embedded tab in plain value fails closed" primary git.example
# Fail-close: a tab immediately after the key colon (no separating space).
write_fixture "bad:${TAB}9
logins:
- name: primary
url: https://git.example
token: TOK_PLAIN
"
assert_both_fail_closed "tab immediately after key colon fails closed" primary git.example
# Fail-close: a tab used as indentation (before a sequence dash).
write_fixture "logins:
${TAB}- name: primary
url: https://git.example
token: TOK_PLAIN
"
assert_both_fail_closed "tab used as indentation fails closed" primary git.example
# Fail-close: a tab trailing a sequence-mapping field value.
write_fixture "logins:
- name: primary
url: https://git.example
token: TOK_PLAIN${TAB}
"
assert_both_fail_closed "tab trailing a seq field value fails closed" primary git.example
# NOT over-broad: a tab strictly INSIDE a quoted scalar is valid YAML (PyYAML
# keeps it as string content), so the document parses and the login token still
# resolves in BOTH paths -- double-quoted and single-quoted.
write_bad_key_fixture "\"a${TAB}b\""
assert_token "tab inside a double-quoted value still resolves token" "TOK_PLAIN" primary git.example
write_bad_key_fixture "'a${TAB}b'"
assert_token "tab inside a single-quoted value still resolves token" "TOK_PLAIN" primary git.example
# ...and a quoted token value carrying an inner tab resolves to the exact string
# (tab preserved), identical to PyYAML's construction.
write_fixture "logins:
- name: primary
url: https://git.example
token: \"T${TAB}OK\"
"
assert_token "quoted token with inner tab resolves verbatim" "T${TAB}OK" primary git.example
echo "Gitea login resolution regression harness passed"