fix(git): fail closed on YAML anchors in tea token fallback (#865 round 12)
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
The PyYAML-absent fallback's _strip_properties consumed a plain anchor
('&name') per-scalar, with no document-scoped registry of declared anchor
names. Real PyYAML's Composer rejects a duplicate anchor NAME declared
anywhere in the document (ComposerError, whole-document fail), but the
fallback would still emit the later token: fail-open (HIGH). Fix: reject
every '&'-anchor property unconditionally instead of building a hand-rolled
duplicate registry, which guarantees zero fail-open by construction at the
cost of a deliberate, endorsed over-reject on a single non-duplicated
anchor. The '!' non-specific-tag branch is unchanged ('! x' -> 'x' still
resolves).
This commit is contained in:
@@ -843,9 +843,6 @@ _FAIL = object()
|
||||
# position makes the whole line fail to match -> the caller fails closed.
|
||||
_KEY_RE = re.compile(r"^([A-Za-z0-9_][A-Za-z0-9_.\-]*) *:(?:[ ](.*)|)$")
|
||||
_FLOW = set("[]{}*&!")
|
||||
# A plain anchor property: '&' then PyYAML's exact anchor charset [0-9A-Za-z-_]+,
|
||||
# then whatever follows (captured for the caller to require a space or EOL).
|
||||
_ANCHOR_RE = re.compile(r"^&[0-9A-Za-z\-_]+(.*)$", re.S)
|
||||
|
||||
|
||||
class _Bail(Exception):
|
||||
@@ -856,22 +853,32 @@ class _Bail(Exception):
|
||||
|
||||
|
||||
def _strip_properties(value):
|
||||
# Consume leading YAML node properties -- at most ONE non-specific tag
|
||||
# ('!' + space) and at most ONE plain anchor ('&name' + space), in either
|
||||
# order -- and return the remaining node text for normal resolution. PyYAML's
|
||||
# SafeLoader applies these TRANSPARENT properties and then resolves the
|
||||
# following node with its ordinary implicit typing, so (verified against real
|
||||
# PyYAML 6.0.3) '! x' -> 'x', '&a 123' -> int 123, '! ' / '&a' -> null, and
|
||||
# '! &a x' / '&a ! x' -> 'x', exactly as the bare node would resolve. Returns
|
||||
# _FAIL for every form we do NOT reproduce identically, so the caller fails
|
||||
# closed: a tag shorthand ('!x' -> ConstructorError), an explicit or verbatim
|
||||
# tag ('!!str' / '!!int' / '!<...>' / '!foo') whose type coercion we do not
|
||||
# emulate, a tab separator (PyYAML scanner error), a duplicate tag or anchor
|
||||
# ('! ! x' / '&a &b x' -> ParserError), or an anchor glued to a non-space.
|
||||
# The returned remainder (possibly '') is re-resolved by the caller; '' is a
|
||||
# null node -> fail closed for a token field, matching PyYAML.
|
||||
# Consume a leading YAML node non-specific tag ('!' + space) and return the
|
||||
# remaining node text for normal resolution. PyYAML's SafeLoader applies
|
||||
# this TRANSPARENT property and then resolves the following node with its
|
||||
# ordinary implicit typing, so (verified against real PyYAML 6.0.3)
|
||||
# '! x' -> 'x', '! ' -> null, exactly as the bare node would resolve.
|
||||
# Returns _FAIL for every form we do NOT reproduce identically, so the
|
||||
# caller fails closed: a tag shorthand ('!x' -> ConstructorError), an
|
||||
# explicit or verbatim tag ('!!str' / '!!int' / '!<...>' / '!foo') whose
|
||||
# type coercion we do not emulate, a tab separator (PyYAML scanner error),
|
||||
# a duplicate tag ('! ! x' -> ParserError).
|
||||
#
|
||||
# ANCHORS ('&name'): deliberately rejected in FULL, not just duplicates.
|
||||
# Real PyYAML tracks anchor NAMES in a document-scoped composer registry
|
||||
# and raises ComposerError ("found duplicate anchor") the instant the
|
||||
# SAME anchor name is declared on a SECOND node anywhere in the document
|
||||
# -- including on a totally unrelated node far from the token field. This
|
||||
# line-oriented fallback has no such document-wide registry (and building
|
||||
# one reliably, across every node shape this recognizer does not even
|
||||
# parse, risks missing a scope and re-opening the fail-open). So instead
|
||||
# of emulating the registry, every '&'-anchor property fails closed here,
|
||||
# unconditionally. This is a conservative OVER-reject relative to PyYAML:
|
||||
# a single, non-duplicated '&a x' is valid YAML that real PyYAML resolves
|
||||
# to 'x', but we refuse it too. That is intentional and safe -- fail
|
||||
# closed can only cost an emitted token PyYAML would have allowed, never
|
||||
# emit one PyYAML would reject.
|
||||
seen_tag = False
|
||||
seen_anchor = False
|
||||
while value[:1] in ("!", "&"):
|
||||
if value[0] == "!":
|
||||
# Non-specific tag: '!' then AT LEAST ONE SPACE. A tab is a PyYAML
|
||||
@@ -880,18 +887,8 @@ def _strip_properties(value):
|
||||
return _FAIL
|
||||
seen_tag = True
|
||||
value = value[1:].lstrip(" ")
|
||||
else: # anchor '&name'
|
||||
m = _ANCHOR_RE.match(value)
|
||||
if seen_anchor or m is None:
|
||||
return _FAIL
|
||||
rest = m.group(1)
|
||||
if rest == "":
|
||||
value = "" # bare '&name' with no following node -> null node
|
||||
elif rest[0] == " ":
|
||||
value = rest.lstrip(" ")
|
||||
else:
|
||||
return _FAIL # '&name' glued to a non-space (e.g. '&a:') -> error
|
||||
seen_anchor = True
|
||||
else: # anchor '&name' -- always fail closed, see comment above
|
||||
return _FAIL
|
||||
return value
|
||||
|
||||
|
||||
|
||||
@@ -1079,21 +1079,28 @@ assert_token "NEL in a comment fails closed" "" primary git.example
|
||||
write_break_fixture 0x85 dq2
|
||||
assert_fallback_fails_closed "blank-line-in-quote (NEL run) fails closed" primary git.example
|
||||
|
||||
# 24. LEADING NON-SPECIFIC TAG / ANCHOR PROPERTY (#865 round-11 blocker B): the
|
||||
# round-10 recognizer blanket-rejected any scalar beginning with '!' or '&'.
|
||||
# But a NON-SPECIFIC tag '! ' (bang + SPACE) and an anchor '&name ' are
|
||||
# transparent node properties: PyYAML strips them and applies normal implicit
|
||||
# typing to the node, so '! x'/'&a x' resolve the STRING 'x'. The fallback now
|
||||
# consumes <=1 non-specific tag and <=1 anchor (either order) and recurses on
|
||||
# the remainder. Verified empirically vs real PyYAML 6.0.3.
|
||||
for pair in '! x=x' '&a x=x' '! x y=x y' '! "q"=q' "! 'q'=q" '! &b x=x' '&b ! x=x'; do
|
||||
# 24. LEADING NON-SPECIFIC TAG / ANCHOR PROPERTY (#865 round-11 blocker B, revised
|
||||
# in round 12): the round-10 recognizer blanket-rejected any scalar beginning
|
||||
# with '!' or '&'. Round 11 taught it to strip a transparent NON-SPECIFIC tag
|
||||
# ('! ' bang + SPACE) and a transparent plain ANCHOR ('&name ') so '! x' /
|
||||
# '&a x' resolve the STRING 'x', matching PyYAML. Round 12 discovered that the
|
||||
# anchor half of that was a HIGH fail-open: PyYAML's Composer tracks anchor
|
||||
# NAMES in a document-scoped registry and raises ComposerError ("found
|
||||
# duplicate anchor") the instant the SAME name is declared on a SECOND node
|
||||
# ANYWHERE in the document (even an unrelated one) -- the fallback's
|
||||
# per-scalar-only view has no such registry and would emit the later token.
|
||||
# Round 12's fix: reject EVERY '&'-anchor property, unconditionally. The
|
||||
# transparent NON-SPECIFIC TAG behavior ('! x' -> 'x') is unchanged and still
|
||||
# verified below; only the anchor half now fails closed (deliberate
|
||||
# conservative over-reject, verified safe both ways against real PyYAML 6.0.3).
|
||||
for pair in '! x=x' '! x y=x y' '! "q"=q' "! 'q'=q"; do
|
||||
tv="${pair%%=*}"; want="${pair#*=}"
|
||||
write_fixture "logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: ${tv}
|
||||
"
|
||||
assert_token "property token '$tv' resolves node string" "$want" primary git.example
|
||||
assert_token "tag property token '$tv' resolves node string" "$want" primary git.example
|
||||
done
|
||||
|
||||
# Fail-OPEN direction stays shut. '!x' (bang + NON-space) is a tag HANDLE ->
|
||||
@@ -1109,6 +1116,35 @@ for tv in '!x' '!foo x' '* a' '! !x' '! 123' '! true' '! null' '&a &b x'; do
|
||||
assert_token "non-resolving property token '$tv' fails closed" "" primary git.example
|
||||
done
|
||||
|
||||
# Round 12: a SINGLE, non-duplicated '&a x' is valid YAML that real PyYAML
|
||||
# resolves to the string 'x' (round-11 behavior, and still true of the oracle).
|
||||
# The fallback now rejects it anyway -- a deliberate, endorsed CONSERVATIVE
|
||||
# over-reject (see the round-12 comment block above): fail-closed can only cost
|
||||
# an emitted token PyYAML would have allowed, never emit one PyYAML rejects, and
|
||||
# a per-scalar recognizer cannot safely prove document-wide anchor-name
|
||||
# uniqueness. assert_fallback_fails_closed also confirms PyYAML really does
|
||||
# resolve a token here, proving this is a genuine (safe-direction) divergence
|
||||
# and not an accidental parity loss.
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: &a x
|
||||
'
|
||||
assert_fallback_fails_closed "round-12: single non-duplicated anchor '&a x' now fails closed (conservative over-reject; PyYAML resolves x)" primary git.example
|
||||
# Same over-reject for the combined tag+anchor forms round 11 used to resolve.
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: ! &b x
|
||||
'
|
||||
assert_fallback_fails_closed "round-12: '! &b x' (tag+anchor) now fails closed (conservative over-reject)" primary git.example
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: &b ! x
|
||||
'
|
||||
assert_fallback_fails_closed "round-12: '&b ! x' (anchor+tag) now fails closed (conservative over-reject)" primary git.example
|
||||
|
||||
# Endorsed fail-closed over-reject: an EXPLICIT tag ('!!str x', verbose
|
||||
# '!<tag:yaml.org,2002:str> x') forces a string PyYAML resolves, but the fallback
|
||||
# recognizes only the transparent non-specific tag and fails closed (safer).
|
||||
@@ -1121,4 +1157,56 @@ for tv in '!!str x' '!<tag:yaml.org,2002:str> x'; do
|
||||
assert_fallback_fails_closed "explicit-tag token '$tv' fails closed" primary git.example
|
||||
done
|
||||
|
||||
# 25. #865 round 12 HIGH fail-open closure: DUPLICATE ANCHOR NAME across separate
|
||||
# nodes. Real PyYAML's Composer tracks anchor names in a DOCUMENT-SCOPED
|
||||
# registry and raises ComposerError ("found duplicate anchor ... first
|
||||
# occurrence") the instant the SAME anchor name is declared a second time
|
||||
# ANYWHERE in the document -- failing the WHOLE document closed, no token,
|
||||
# regardless of how far the duplicate sits from the logins block. The round-11
|
||||
# fallback tracked anchors only WITHIN a single scalar's `_strip_properties`
|
||||
# call, so it had no visibility into a duplicate declared on an unrelated
|
||||
# node and would still emit the (later) token: fail-open. The round-12 fix
|
||||
# (reject every '&'-anchor property, unconditionally -- see section 24 above)
|
||||
# closes this as a strict superset: since NO anchor is ever accepted, a
|
||||
# duplicate anchor can never slip through. These cases exercise that
|
||||
# document-wide duplicate-anchor invariant specifically (as opposed to
|
||||
# section 24's single-anchor-on-the-token-field cases) and pair each fallback
|
||||
# assertion with confirmation that real PyYAML also fails closed here (via
|
||||
# ComposerError), proving this was a genuine fail-open, not a hypothetical.
|
||||
write_fixture 'first: &same one
|
||||
second: &same two
|
||||
logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: TOK_DUP_ROOT
|
||||
'
|
||||
assert_both_fail_closed "round-12: duplicate anchor name on two unrelated root nodes fails closed" primary git.example
|
||||
|
||||
write_fixture 'outer:
|
||||
nested: &dup x
|
||||
dup_root: &dup y
|
||||
logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: TOK_DUP_NESTED
|
||||
'
|
||||
assert_both_fail_closed "round-12: duplicate anchor name across a nested node and a root node fails closed" primary git.example
|
||||
|
||||
write_fixture 'first: &dup one
|
||||
logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: &dup TOK_DUP_TOKEN_NODE
|
||||
'
|
||||
assert_both_fail_closed "round-12: anchor name declared earlier and repeated on the token-bearing node fails closed" primary git.example
|
||||
|
||||
# Regression guard: the non-specific TAG half of section 24 ('! x' -> 'x') is
|
||||
# UNCHANGED by the round-12 anchor fix and must still resolve.
|
||||
write_fixture 'logins:
|
||||
- name: primary
|
||||
url: https://git.example
|
||||
token: ! x
|
||||
'
|
||||
assert_token "round-12 regression: '! x' (tag, no anchor) still resolves 'x'" "x" primary git.example
|
||||
|
||||
echo "Gitea login resolution regression harness passed"
|
||||
|
||||
Reference in New Issue
Block a user