mosaicstack/stack
1
0
Fork 0
Code Issues 77 Pull Requests 1 Actions Packages Projects Releases 15 Wiki Activity

fix(lease-broker): de-flake recovery_runtime b2 broker-socket ConnectionRefused race (#849) #851

Merged
jason.woltje merged 1 commits from fix/849-recovery-runtime-fixture-race into main 2026-07-20 06:44:38 +00:00
Conversation 1 Commits 1 Files Changed 1 +20 -11
jason.woltje commented 2026-07-20 06:27:17 +00:00
Owner
Copy Link

Summary

Fixes the environmental per-test broker-socket ConnectionRefused race in recovery_runtime_unittest.py (b2), which surfaced as intermittent RED on descendant-main push CI. Fixture-lifecycle only: the test request() helper now bounds the Unix-socket connect with a 5s deadline retry that catches only ConnectionRefusedError and re-raises the real failure at the deadline.

Closes #849.

Scope

  • Touches only packages/mosaic/src/lease-broker/recovery_runtime_unittest.py (test fixture). Zero production lease-broker / recovery-runtime source changed.
  • b2 security assertion + transport-trust boundary byte-preserved — SHA-256 verified identical at base and head.

Review provenance — three distinct lanes (author != code-reviewer != security-reviewer)

  • CODE APPROVE — RoR sha256 03234bcf295834e7c98f6ade669976456f9bdabfa15b50badb70ac0d415045c8. Detached exact-head review; AC1-6 + C1-5 all PASS; red-first independently reproduced (base helper RED / head GREEN under an injected pre-listen window); 6x stable; 8 sibling unittests pass; Codex approve 0.94 / 0 findings.
  • FOCUSED SECREV — ATTEST-CLEAN — RoR sha256 cdf4c4d4d143dabfdfcf5005182ce5264644e61f12d1513d7a29f0618ddb206c. S1-S5 all PASS; b2 assertion + WI-6 production transport blobs byte-identical at base/head; retry catches only ConnectionRefusedError; finite deadline re-raises real failures (independently probed).

CI note — read before relying on PR-CI

PR-event CI does not execute recovery_runtime_unittest.py (the pull_request pipeline does not invoke it). PR-CI green here is the general merge gate only. The recovery_runtime GREEN datapoint that validates this fix comes from the post-merge descendant-main push pipeline. Do not treat PR-CI green as proof the flake is fixed.

No self-merge; no completion claim. Merge and FCM-M5-001 completion ownership retained by the coordinator.

## Summary Fixes the environmental per-test broker-socket `ConnectionRefused` race in `recovery_runtime_unittest.py` (b2), which surfaced as intermittent RED on descendant-`main` push CI. **Fixture-lifecycle only:** the test `request()` helper now bounds the Unix-socket connect with a 5s deadline retry that catches **only** `ConnectionRefusedError` and re-raises the real failure at the deadline. Closes #849. ## Scope - Touches **only** `packages/mosaic/src/lease-broker/recovery_runtime_unittest.py` (test fixture). Zero production lease-broker / recovery-runtime source changed. - b2 security assertion + transport-trust boundary **byte-preserved** — SHA-256 verified identical at base and head. ## Review provenance — three distinct lanes (author != code-reviewer != security-reviewer) - **CODE APPROVE** — RoR sha256 `03234bcf295834e7c98f6ade669976456f9bdabfa15b50badb70ac0d415045c8`. Detached exact-head review; AC1-6 + C1-5 all PASS; red-first independently reproduced (base helper RED / head GREEN under an injected pre-listen window); 6x stable; 8 sibling unittests pass; Codex approve 0.94 / 0 findings. - **FOCUSED SECREV — ATTEST-CLEAN** — RoR sha256 `cdf4c4d4d143dabfdfcf5005182ce5264644e61f12d1513d7a29f0618ddb206c`. S1-S5 all PASS; b2 assertion + WI-6 production transport blobs byte-identical at base/head; retry catches only `ConnectionRefusedError`; finite deadline re-raises real failures (independently probed). ## CI note — read before relying on PR-CI PR-event CI does **not** execute `recovery_runtime_unittest.py` (the `pull_request` pipeline does not invoke it). PR-CI green here is the **general merge gate only**. The `recovery_runtime` GREEN datapoint that validates this fix comes from the **post-merge descendant-`main` push** pipeline. Do not treat PR-CI green as proof the flake is fixed. No self-merge; no completion claim. Merge and FCM-M5-001 completion ownership retained by the coordinator.
jason.woltje added 1 commit 2026-07-20 06:27:18 +00:00
test(mosaic): wait for broker socket readiness
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
Details
cbacd10a66
jason.woltje commented 2026-07-20 06:28:23 +00:00
Author
Owner
Copy Link

Durable exact-head review binding — do not merge without the coordinator 6-check

Exact head bound: cbacd10a6601b223cb61d5303b4761e0385a54ee

Three distinct review lanes (author != code-reviewer != security-reviewer):

  • CODE APPROVE — RoR sha256 03234bcf295834e7c98f6ade669976456f9bdabfa15b50badb70ac0d415045c8. Detached exact-head; AC1-6 + C1-5 PASS; red-first independently reproduced (base helper RED / head GREEN under an injected pre-listen window); 6x stable; 8 sibling unittests pass; Codex approve 0.94 / 0 findings.
  • FOCUSED SECREV — ATTEST-CLEAN — RoR sha256 cdf4c4d4d143dabfdfcf5005182ce5264644e61f12d1513d7a29f0618ddb206c. S1-S5 PASS; b2 assertion + WI-6 production transport blobs byte-identical at base/head; retry catches only ConnectionRefusedError; finite deadline re-raises real failures (independently probed).

CI distinction: PR-event CI does not run recovery_runtime_unittest.py (proven by prior pipeline #1907). PR-CI green = general merge gate only. The recovery_runtime GREEN datapoint that validates this fix arrives from the post-merge descendant-main push pipeline (the FCM-M5-001 completion gate). This binding does not claim FCM-M5-001 done.

Posted via Gitea REST (the framework pr-review.sh comment path is the subject of #812).

### Durable exact-head review binding — do not merge without the coordinator 6-check **Exact head bound:** `cbacd10a6601b223cb61d5303b4761e0385a54ee` Three distinct review lanes (author != code-reviewer != security-reviewer): - **CODE APPROVE** — RoR sha256 `03234bcf295834e7c98f6ade669976456f9bdabfa15b50badb70ac0d415045c8`. Detached exact-head; AC1-6 + C1-5 PASS; red-first independently reproduced (base helper RED / head GREEN under an injected pre-listen window); 6x stable; 8 sibling unittests pass; Codex approve 0.94 / 0 findings. - **FOCUSED SECREV — ATTEST-CLEAN** — RoR sha256 `cdf4c4d4d143dabfdfcf5005182ce5264644e61f12d1513d7a29f0618ddb206c`. S1-S5 PASS; b2 assertion + WI-6 production transport blobs byte-identical at base/head; retry catches only `ConnectionRefusedError`; finite deadline re-raises real failures (independently probed). **CI distinction:** PR-event CI does **not** run `recovery_runtime_unittest.py` (proven by prior pipeline #1907). PR-CI green = general merge gate only. The `recovery_runtime` GREEN datapoint that validates this fix arrives from the **post-merge descendant-`main` push** pipeline (the FCM-M5-001 completion gate). This binding does not claim FCM-M5-001 done. Posted via Gitea REST (the framework `pr-review.sh` comment path is the subject of #812).
jason.woltje merged commit 77c9a82614 into main 2026-07-20 06:44:38 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
jason.woltje
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#851
Delete Branch "fix/849-recovery-runtime-fixture-race"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?