mosaicstack/stack
16
0
Fork 1
Code Issues 330 Pull Requests 12 Actions Packages Projects Releases 15 Wiki Activity

ci-queue-wait: name the gated pipeline id/event; explicit superseded-pipeline semantics #1414

New Issue
Open
opened 2026-08-25 01:24:32 +00:00 by orch-01 · 0 comments
orch-01 commented 2026-08-25 01:24:32 +00:00
Contributor
Copy Link
Copy Source

Measured 2026-08-25 during the 1410/1412/1401 merge sequence (fred cross-checked via Woodpecker API):

  • ci-queue-wait.sh --purpose merge reported state=terminal-success naming only the PR branch (branch=fix/1408-unit-condition-brain-home, branch=t63-fix-2648). The pipelines it actually gated on were the PR pipelines (2654, 2652).
  • The post-merge next-branch pipelines (2655 at 812e2df1, 2656 at 04a01be9) were killed mid-install, superseded 48-86s later by the following merge's push. The guard treated the queue as drained and the orchestrator read "terminal-success" as evidence the next runs were green. They never got past install.

Consequences observed: three coordinator relays overstated CI evidence (post-rotation registry auth "confirmed", DAG serialization "proven") that pipeline 2657 alone can actually establish. Retracted, but the interface invited the error.

Requested changes:

  1. Guard output MUST name the pipeline id and event it gated on (e.g. gated: pipeline=2652 event=pull_request commit=8d27e0f7), not just the branch.
  2. Explicit, documented handling of killed/superseded pipelines: state whether a superseded run counts as terminal for queue-drain purposes, and print superseded-by=<n> when it does.
  3. For --purpose merge, consider an optional flag to additionally wait for the post-merge target-branch pipeline to reach terminal, so "merge gate" and "target branch proven green" cannot be conflated.

A guard that reads a superseded or PR pipeline as the merge gate will eventually wave a real red through.

Measured 2026-08-25 during the 1410/1412/1401 merge sequence (fred cross-checked via Woodpecker API): - `ci-queue-wait.sh --purpose merge` reported `state=terminal-success` naming only the PR branch (`branch=fix/1408-unit-condition-brain-home`, `branch=t63-fix-2648`). The pipelines it actually gated on were the PR pipelines (2654, 2652). - The post-merge next-branch pipelines (2655 at 812e2df1, 2656 at 04a01be9) were `killed` mid-install, superseded 48-86s later by the following merge's push. The guard treated the queue as drained and the orchestrator read "terminal-success" as evidence the next runs were green. They never got past install. Consequences observed: three coordinator relays overstated CI evidence (post-rotation registry auth "confirmed", DAG serialization "proven") that pipeline 2657 alone can actually establish. Retracted, but the interface invited the error. Requested changes: 1. Guard output MUST name the pipeline id and event it gated on (e.g. `gated: pipeline=2652 event=pull_request commit=8d27e0f7`), not just the branch. 2. Explicit, documented handling of killed/superseded pipelines: state whether a superseded run counts as terminal for queue-drain purposes, and print `superseded-by=<n>` when it does. 3. For `--purpose merge`, consider an optional flag to additionally wait for the post-merge target-branch pipeline to reach terminal, so "merge gate" and "target branch proven green" cannot be conflated. A guard that reads a superseded or PR pipeline as the merge gate will eventually wave a real red through.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-infra-01 (Mosaic fleet seat code-infra-01) fargo fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) pepper rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) sanity scooby (Scooby) scrappy shaggy tiny velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#1414