test(wake): de-flake T10 concurrent-enqueue race — deterministic start-barrier + wait-both (#923)
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
T10 ("two concurrent store.sh enqueue get DISTINCT seqs via flock") launched
its two background enqueues with plain `cmd & cmd & wait` — no guarantee the
scheduler ever actually overlapped them, so the flock contention the test
exists to prove was sometimes never really exercised (or the assertion could
race a not-yet-durable second write), producing an intermittent CI-load flake.
Replace the launch with a deterministic start barrier (flock'd gate file, no
sleep): both children signal a ready marker then block on a shared exclusive
lock; the parent busy-polls (not sleeps) until both markers exist, THEN
releases the gate so both children's pending shared-lock waits are granted
together and they race into store.sh's own enqueue lock for real. Assertions
(distinct+gapless {1,2} seqs, order-independent, both durably enqueued) are
unchanged and still run only after both PIDs have exited.
Verified: 30/30 sequential foreground runs pass (previously intermittent).
RED-catch reproduced 5/5: with _wake_lock_acquire temporarily stubbed to a
no-op, T10 reliably aliases (both enqueues get seq '1') and fails loud,
proving the de-flaked test still catches a real aliasing regression;
restored and reconfirmed GREEN. shellcheck 0.11.0 clean.
Closes #923; Part of #892
This commit is contained in:
@@ -387,13 +387,58 @@ if command -v flock >/dev/null 2>&1; then
|
||||
"$STORE" init >/dev/null 2>&1 || true
|
||||
o1="$TMP_ROOT/t10.o1"
|
||||
o2="$TMP_ROOT/t10.o2"
|
||||
# Two enqueues racing on the SAME store. Without the lock both read
|
||||
# observed_seq=0 and allocate 1 (aliasing + a lost write); the lock serializes
|
||||
# the allocate+write so they get 1 and 2.
|
||||
"$STORE" enqueue --class actionable --locators '{"c":1}' >"$o1" 2>/dev/null &
|
||||
# --- deterministic start barrier (#923 de-flake; NOT a sleep) --------------
|
||||
# A plain `cmd & cmd & wait` gives no guarantee the two enqueues are ever
|
||||
# truly in-flight together — the scheduler can run one to completion before
|
||||
# the other is even forked, so the flock contention this test exists to
|
||||
# prove is never really exercised. Force REAL concurrency: both children
|
||||
# block on a shared start gate (an flock'd file — no sleep, no timing
|
||||
# guess) until BOTH have confirmed they are launched and waiting; only then
|
||||
# does the parent release the gate, so both race into store.sh's own
|
||||
# enqueue lock at (as close to) the same instant as the scheduler allows.
|
||||
gate="$TMP_ROOT/t10.gate"
|
||||
r1="$TMP_ROOT/t10.ready1"
|
||||
r2="$TMP_ROOT/t10.ready2"
|
||||
rm -f "$r1" "$r2"
|
||||
exec 6>"$gate"
|
||||
flock -x 6 # gate CLOSED: a shared-locker below blocks here until released.
|
||||
|
||||
(
|
||||
: >"$r1" # "child 1 is launched and about to block on the gate"
|
||||
exec 7<"$gate"
|
||||
flock -s 7 # blocks until the parent drops its exclusive lock
|
||||
flock -u 7
|
||||
exec 7<&-
|
||||
"$STORE" enqueue --class actionable --locators '{"c":1}'
|
||||
) >"$o1" 2>/dev/null &
|
||||
p1=$!
|
||||
"$STORE" enqueue --class actionable --locators '{"c":2}' >"$o2" 2>/dev/null &
|
||||
(
|
||||
: >"$r2"
|
||||
exec 7<"$gate"
|
||||
flock -s 7
|
||||
flock -u 7
|
||||
exec 7<&-
|
||||
"$STORE" enqueue --class actionable --locators '{"c":2}'
|
||||
) >"$o2" 2>/dev/null &
|
||||
p2=$!
|
||||
|
||||
# Busy-poll (NOT a sleep — no timing assumption) until BOTH children have
|
||||
# confirmed they are launched and blocked on the gate, so the contention
|
||||
# window below is genuine rather than incidental.
|
||||
spins=0
|
||||
while [ ! -e "$r1" ] || [ ! -e "$r2" ]; do
|
||||
spins=$((spins + 1))
|
||||
if [ "$spins" -gt 2000000 ]; then
|
||||
fail_msg "T10: start barrier never saw both children launch — cannot construct genuine concurrency"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Release the gate: both children's pending shared-lock acquisitions are
|
||||
# granted together, so they race into store.sh's OWN enqueue lock for real.
|
||||
flock -u 6
|
||||
exec 6>&-
|
||||
|
||||
wait "$p1"; wait "$p2"
|
||||
a="$(tr -d '[:space:]' <"$o1")"
|
||||
b="$(tr -d '[:space:]' <"$o2")"
|
||||
|
||||
Reference in New Issue
Block a user