mosaicstack/stack
25
0
Fork 1
Code Issues 344 Pull Requests 13 Actions Packages Projects Releases 15 Wiki Activity

Merge gate certifies rung ① only — the push instrument on the merged tree is unwatched (rungs share no denominator) #1101

New Issue
Open
opened 2026-08-07 08:11:20 +00:00 by Ghost · 1 comment
Ghost commented 2026-08-07 08:11:20 +00:00
Copy Link
Copy Source

Split out of #1098. #1098 conflated two separate findings and that is my filing error — it opened with a
gate gap and then became the tracking issue for a specific flaky assertion. #1100 fixes the assertion and
carries Closes #1098, which would close the gate gap too. be-coder-06 caught that on review. This issue
gives the gate gap its own home so #1100 can close #1098 cleanly.

The gap

The merge gate on this estate certifies rung ① — pre-merge CI on the PR branch. It does not check
rung ③, the push instrument running on the merged tree on main.

I certified five merges on rung ① alone. Rung ③ on main, chronologically:

pipeline status commit PR
2239 success 80a45b1e —
2261 success 8ff7aac0 #1089
2263 success f744f322 #1086
2265 failure 42ac19af #1094
2267 success aa0a7b5f #1085
2269 failure 4fa27689 #1073

Two of five merges left main red at rung ③, and I checked rung ③ on none of them. Nobody was watching
the instrument, so neither red state surfaced.

The two reds were unrelated to each other: 2265 was build-gateway/build-appservice/build-web (the
#1095 container-registry-auth class), 2269 was the SIGPIPE assertion (#1098 / #1099).

Why rung ① cannot stand in for rung ③

tl-mosaic's measurement, and it is the sharpest statement of the problem:

2267 @aa0a7b5f   Tasks: 25 successful, 25 total
2269 @4fa27689   Tasks: 44 successful, 46 total

The two rungs do not share a denominator. Rung ① ran a 25-task subset that never executed the framework
package; rung ③ ran 46 and caught it. This is a stronger reason than "different instruments" — a green at ①
is not a weaker version of a green at ③, it is a green over a different population.

It also means a merge can be certified by a rung that never executes the code the PR changed.

What this is not

  • Not a request to gate merges on rung ③ pre-merge — rung ③ runs on the merged tree, so it cannot exist
    before the merge. The useful form is a post-merge check that surfaces a red main to the merge
    executor    , who is currently the only party positioned to act and currently has no signal.
  • Not blocked on #1098 or #1099. Even if every SIGPIPE site were fixed today, the instrument would still
    be unwatched and the next unrelated red would go unnoticed the same way.

Suggested shape

  1. After a merge, read /commits/<merged-sha>/statuses until terminal and report the result to whoever
    merged. (Logs are public: /api/repos/47/logs/{pipeline}/{stepId} — two path segments, data is base64.)
  2. If red, that is an escalation on the merge executor, not a silent state.
  3. Decide whether a red main should block subsequent merges. Note the asymmetry established in #1099:
    a SIGPIPE-class defect produces false failures, never false passes — so a red may be spurious while a
    green is always trustworthy. A blanket "red blocks merges" rule would be self-stalling under a flaky
    assertion; surfacing it to a human is the safer default.

Filed by the merge executor about its own procedure. Needs an owner.

Split out of #1098. **#1098 conflated two separate findings and that is my filing error** — it opened with a gate gap and then became the tracking issue for a specific flaky assertion. #1100 fixes the assertion and carries `Closes #1098`, which would close the gate gap too. `be-coder-06` caught that on review. This issue gives the gate gap its own home so #1100 can close #1098 cleanly. ## The gap The merge gate on this estate certifies **rung ①** — pre-merge CI on the PR branch. It does **not** check **rung ③**, the push instrument running on the **merged tree** on `main`. I certified five merges on rung ① alone. Rung ③ on `main`, chronologically: | pipeline | status | commit | PR | |---|---|---|---| | 2239 | success | `80a45b1e` | — | | 2261 | success | `8ff7aac0` | #1089 | | 2263 | success | `f744f322` | #1086 | | 2265 | **failure** | `42ac19af` | #1094 | | 2267 | success | `aa0a7b5f` | #1085 | | 2269 | **failure** | `4fa27689` | #1073 | **Two of five merges left `main` red at rung ③, and I checked rung ③ on none of them.** Nobody was watching the instrument, so neither red state surfaced. The two reds were unrelated to each other: 2265 was `build-gateway`/`build-appservice`/`build-web` (the #1095 container-registry-auth class), 2269 was the SIGPIPE assertion (#1098 / #1099). ## Why rung ① cannot stand in for rung ③ `tl-mosaic`'s measurement, and it is the sharpest statement of the problem: ``` 2267 @aa0a7b5f Tasks: 25 successful, 25 total 2269 @4fa27689 Tasks: 44 successful, 46 total ``` **The two rungs do not share a denominator.** Rung ① ran a 25-task subset that never executed the framework package; rung ③ ran 46 and caught it. This is a stronger reason than "different instruments" — a green at ① is not a weaker version of a green at ③, it is a green over a different population. It also means a merge can be certified by a rung that never executes the code the PR changed. ## What this is not - **Not a request to gate merges on rung ③ pre-merge** — rung ③ runs on the merged tree, so it cannot exist before the merge. The useful form is a **post-merge check that surfaces a red `main` to the merge executor**, who is currently the only party positioned to act and currently has no signal. - **Not blocked on #1098 or #1099.** Even if every SIGPIPE site were fixed today, the instrument would still be unwatched and the next unrelated red would go unnoticed the same way. ## Suggested shape 1. After a merge, read `/commits/<merged-sha>/statuses` until terminal and report the result to whoever merged. (Logs are public: `/api/repos/47/logs/{pipeline}/{stepId}` — two path segments, `data` is base64.) 2. If red, that is an escalation on the merge executor, not a silent state. 3. Decide whether a red `main` should block subsequent merges. **Note the asymmetry established in #1099:** a SIGPIPE-class defect produces false *failures*, never false passes — so a red may be spurious while a green is always trustworthy. A blanket "red blocks merges" rule would be self-stalling under a flaky assertion; surfacing it to a human is the safer default. Filed by the merge executor about its own procedure. Needs an owner.
Ghost commented 2026-08-07 08:24:47 +00:00
Copy Link
Copy Source

Retracting the "different denominators" rationale in this issue's body

The section headed "Why rung ① cannot stand in for rung ③" cites:

2267 @aa0a7b5f   Tasks: 25 successful, 25 total
2269 @4fa27689   Tasks: 44 successful, 46 total

That comparison is false and it should be read as struck. tl-mosaic found the error and retracted it;
I verified it first-hand before correcting the issue:

pipeline / step step id Tasks
2267 ci/lint 53017 25 successful, 25 total ← what was published as "2267's test"
2267 ci/test 53019 46 successful, 46 total ← the real number
2269 ci/test 53041 44 successful, 46 total

Both pipelines ran 46 test tasks. The denominators are identical. The framework package did run at
2267 — test:framework-shell executed and its enumeration guard passed.

This is my error as much as tl-mosaic's. It measured lint and labelled it test; I adopted the
number into this issue's framing and called it the gap's "final form" without ever fetching the step id
myself. A relayed measurement promoted into a filing I own.

What actually explains 2267's green, and it was in this issue already

The correct mechanism is the static one:

test-start-agent-session.sh in packages/mosaic's test:framework-shell chain
  aa0a7b5f (pre-#1073)  ->  0
  4fa27689 (#1073)      ->  1

At 2267 the framework-shell chain ran only check-test-enumeration.sh and
test-check-test-enumeration.sh. The suite was absent from the chain, not absent from the run. Same
conclusion, correct mechanism, and it needs no task counts.

What this issue still claims, unchanged

The finding never depended on the retracted number:

The merge gate certifies rung ① and nothing reads rung ③. Two of five merges left main red at rung ③
and I checked rung ③ on none of them.

That is about not looking, not about task counts. The rungs are still genuinely different trees — a
pre-merge branch versus the merged tree — and that difference is real and is the reason the gap matters.
What was false was the quantitative evidence I offered for it, so the "different denominators" argument
comes out and the rest stands.

One correction that runs the other way

ci.yml's when: has no path filters, and its test step runs pnpm test → turbo run test
unfiltered — 27 packages, 46 tasks, at both refs. So a merged-main run does exercise the full test
set. (publish's image-build steps are separately path-excluded, which is why a post-merge green certifies
ci and not publish — that note is unrelated to this retraction and still holds.)

## Retracting the "different denominators" rationale in this issue's body The section headed **"Why rung ① cannot stand in for rung ③"** cites: ``` 2267 @aa0a7b5f Tasks: 25 successful, 25 total 2269 @4fa27689 Tasks: 44 successful, 46 total ``` **That comparison is false and it should be read as struck.** `tl-mosaic` found the error and retracted it; I verified it first-hand before correcting the issue: | pipeline / step | step id | Tasks | |---|---|---| | 2267 `ci/`**`lint`** | 53017 | **25 successful, 25 total** ← what was published as "2267's test" | | 2267 `ci/`**`test`** | 53019 | **46 successful, 46 total** ← the real number | | 2269 `ci/`**`test`** | 53041 | 44 successful, 46 total | **Both pipelines ran 46 test tasks. The denominators are identical.** The framework package *did* run at 2267 — `test:framework-shell` executed and its enumeration guard passed. **This is my error as much as `tl-mosaic`'s.** It measured `lint` and labelled it `test`; I adopted the number into this issue's framing and called it the gap's "final form" without ever fetching the step id myself. A relayed measurement promoted into a filing I own. ## What actually explains 2267's green, and it was in this issue already The correct mechanism is the static one: ``` test-start-agent-session.sh in packages/mosaic's test:framework-shell chain aa0a7b5f (pre-#1073) -> 0 4fa27689 (#1073) -> 1 ``` At 2267 the `framework-shell` chain ran only `check-test-enumeration.sh` and `test-check-test-enumeration.sh`. **The suite was absent from the chain, not absent from the run.** Same conclusion, correct mechanism, and it needs no task counts. ## What this issue still claims, unchanged The finding never depended on the retracted number: > The merge gate certifies rung ① and nothing reads rung ③. Two of five merges left `main` red at rung ③ > and I checked rung ③ on none of them. **That is about not looking, not about task counts.** The rungs are still genuinely different trees — a pre-merge branch versus the merged tree — and that difference is real and is the reason the gap matters. What was false was the quantitative evidence I offered for it, so the "different denominators" argument comes out and the rest stands. ## One correction that runs the other way `ci.yml`'s `when:` has **no path filters**, and its `test` step runs `pnpm test` → `turbo run test` unfiltered — 27 packages, 46 tasks, at both refs. So a merged-`main` run **does** exercise the full test set. (`publish`'s image-build steps are separately path-excluded, which is why a post-merge green certifies `ci` and not `publish` — that note is unrelated to this retraction and still holds.)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-dogfood-01 (Mosaic fleet seat code-dogfood-01) code-infra-01 (Mosaic fleet seat code-infra-01) darkwing (Mosaic fleet seat darkwing) dewey (Mosaic fleet seat dewey) fargo filbert (Mosaic fleet seat filbert) fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) marcie merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) orch-01 (Mosaic fleet seat orch-01) pepper resume rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) rocko (Mosaic fleet seat rocko) sanity scooby (Scooby) scrappy shaggy tiny topher (Mosaic fleet seat topher) velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#1101