mosaicstack/stack
16
0
Fork 1
Code Issues 330 Pull Requests 12 Actions Packages Projects Releases 15 Wiki Activity

publish-next-npm mutates workspace concurrently with build-gateway kaniko COPY (PR 1405 follow-up): restore races with parallel image build #1411

New Issue
Closed
opened 2026-08-25 00:37:03 +00:00 by code-be-01 · 0 comments
code-be-01 commented 2026-08-25 00:37:03 +00:00
Member
Copy Link
Copy Source

Follow-up to #1404 / PR #1405. Post-merge pipeline 2648 (next f45928c3) failed build-gateway with ERR_PNPM_OUTDATED_LOCKFILE although publish-next-npm completed green and its own log ends with 'workspace manifests restored byte-exact (git diff clean)'.

Measured (step 61027/61028 logs): publish-next-npm and build-gateway both declare depends_on [build, verify] and run CONCURRENTLY in pipeline 2648. The pin transform's in-place mutation window (transform .. publish .. restore) overlaps kaniko's COPY of apps/gateway/package.json: the build-gateway failure dump shows the manifest carrying exact 0.0.3-next.2648 pins while pnpm-lock.yaml records workspace:^. PR 1405's snapshot/restore is correct but insufficient against a concurrent reader — restore only guarantees the END state of publish-next-npm's own step, not mutual exclusion with a parallel step reading mid-window.

Mechanism class: escaped scope, not missing coverage — the write set (apps/packages/plugins manifests) IS snapshotted and restored; the defect is ordering, not set membership. (Tasking's hypothesis that the snapshot set missed apps/ is refuted by the committed find: it walks apps packages plugins from the start.)

Fix: serialize the pipeline DAG — build-gateway (and any other workspace-consuming image build) must depend on publish-next-npm, so kaniko reads the workspace only after restore. The in-step pristine guard stays as the same-step control; the DAG edge is the cross-step control.

Blocks the PR 1401 merge (USC chain). Filed by code-be-01 per T63FIX2648 tasking.

Follow-up to #1404 / PR #1405. Post-merge pipeline 2648 (next f45928c3) failed build-gateway with ERR_PNPM_OUTDATED_LOCKFILE although publish-next-npm completed green and its own log ends with 'workspace manifests restored byte-exact (git diff clean)'. Measured (step 61027/61028 logs): publish-next-npm and build-gateway both declare depends_on [build, verify] and run CONCURRENTLY in pipeline 2648. The pin transform's in-place mutation window (transform .. publish .. restore) overlaps kaniko's COPY of apps/gateway/package.json: the build-gateway failure dump shows the manifest carrying exact 0.0.3-next.2648 pins while pnpm-lock.yaml records workspace:^. PR 1405's snapshot/restore is correct but insufficient against a concurrent reader — restore only guarantees the END state of publish-next-npm's own step, not mutual exclusion with a parallel step reading mid-window. Mechanism class: escaped scope, not missing coverage — the write set (apps/packages/plugins manifests) IS snapshotted and restored; the defect is ordering, not set membership. (Tasking's hypothesis that the snapshot set missed apps/ is refuted by the committed find: it walks apps packages plugins from the start.) Fix: serialize the pipeline DAG — build-gateway (and any other workspace-consuming image build) must depend on publish-next-npm, so kaniko reads the workspace only after restore. The in-step pristine guard stays as the same-step control; the DAG edge is the cross-step control. Blocks the PR 1401 merge (USC chain). Filed by code-be-01 per T63FIX2648 tasking.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
code-be-01 (Mosaic fleet seat code-be-01) code-be-02 (Mosaic fleet seat code-be-02) code-infra-01 (Mosaic fleet seat code-infra-01) fargo fred gate-merge-01 (Mosaic fleet seat gate-merge-01) happy jason.woltje (Jason Woltje) merge-gate ops-01 (Mosaic fleet seat ops-01) ops-02 (Mosaic fleet seat ops-02) ops-03 (Mosaic fleet seat ops-03) ops-ci-01 (Mosaic fleet seat ops-ci-01) ops-deploy-01 (Mosaic fleet seat ops-deploy-01) pepper rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev-security-03 (Mosaic fleet seat rev-security-03) sanity scooby (Scooby) scrappy shaggy tiny velma veronica (Mosaic fleet seat veronica) vision woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#1411