fix(wake): close fd 9 in the detector's sleep child so a dead detector's lock dies with it
ci/woodpecker/pr/ci Pipeline was successful

detector.sh:502 opens the single-instance lock as `exec 9>"$lock"`. fd 9 is not
close-on-exec, so the `sleep "$interval"` at :532 inherits it. When the detector
process dies while sleeping, the orphaned `sleep` keeps holding the flock for up
to `$interval` (30s by default). A supervisor restarting inside that window hits
FAIL LOUD at :504 naming a holder that no longer exists.

Measured on the live holder via /proc, not inferred: the `sleep` child sits in
hrtimer_nanosleep with the lock file open, and both it and the bash parent appear
as holders. The kernel is behaving correctly — this is inheritance, not release lag.

`sleep "$interval" 9>&-` closes the descriptor in the child only. The parent keeps
fd 9, so the single-instance invariant is unchanged: a second instance is still
refused while a live detector sleeps.

Scope note: this closes the `sleep` case only. fd 9 is inherited by every child,
and two others are unbounded — the operator-supplied WAKE_DETECTOR_SOURCE_CMD
(observed inherited on 2 of 2 invocations per cycle) and the untimed
`sh -c "$WAKE_BEACON_SINK_CMD"` at beacon.sh:262. A hung one holds the lock
indefinitely after the detector is gone. The durable fix is close-on-exec on fd 9
once; that is a separate change and is deliberately not bundled here.

test-wake-detector.sh is intentionally not modified. Its comment at 224-226
attributes the delay to kernel release lag and is wrong, and its 3s retry at
228-234 is dead weight once the leak is gone — but both are test-side edits to a
suite outside this fix's scope, and the retry is what proves the fix: it is red
today and passes on the first probe with the patch.

Refs #966.

Authored-by-agent: mos-dt (sb-it-1-dt)
This commit is contained in:
Jason Woltje
2026-07-31 04:07:22 -05:00
parent 4fb44f6345
commit 60f8caf4d8
@@ -529,7 +529,7 @@ cmd_run() {
echo "detector.sh: WARN — off-host liveness beacon emit failed (see beacon.sh); the off-host absence check remains the authoritative dead-man." >&2
fi
[ "$once" -eq 1 ] && break
sleep "$interval"
sleep "$interval" 9>&-
done
}