Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
58b971aba3 | ||
|
|
f4fd5967fc | ||
|
|
f65e9ea656 | ||
|
|
f58b3699a6 |
@@ -8,6 +8,7 @@ coverage
|
||||
.env.local
|
||||
*.tsbuildinfo
|
||||
.pnpm-store
|
||||
__pycache__/
|
||||
docs/reports/
|
||||
|
||||
# Step-CA dev password — real file is gitignored; commit only the .example
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
pnpm typecheck && pnpm lint && pnpm format:check
|
||||
pnpm preflight && pnpm typecheck && pnpm lint && pnpm format:check
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
@mosaicstack:registry=https://git.mosaicstack.dev/api/packages/mosaicstack/npm/
|
||||
# Pin the pnpm store to the same path the ci-base image warms (Dockerfile.ci),
|
||||
# so the pipeline `pnpm install --prefer-offline` consumes the baked store
|
||||
# instead of repopulating a fresh one.
|
||||
store-dir=/root/.local/share/pnpm/store
|
||||
# HOME resolves to /root in the ci-base image, preserving its warmed-store path.
|
||||
# Non-root checkouts use their own HOME. Override without editing this file via
|
||||
# NPM_CONFIG_STORE_DIR (pnpm's environment form of the store-dir setting).
|
||||
store-dir=${HOME}/.local/share/pnpm/store
|
||||
|
||||
@@ -201,8 +201,21 @@ git clone [email protected]:mosaicstack/stack.git
|
||||
cd stack
|
||||
|
||||
# Install dependencies. The local tier uses in-process PGlite; leave DATABASE_URL unset.
|
||||
# The pnpm store defaults to $HOME/.local/share/pnpm/store. Override it without
|
||||
# editing the checkout with NPM_CONFIG_STORE_DIR=$HOME/another-store if needed.
|
||||
pnpm install
|
||||
|
||||
# Verify dependencies and generated state before running source-quality gates.
|
||||
# Missing dependencies exit 42; stale/foreign apps/web/.next state exits 43.
|
||||
# The web build certifies its exact standalone symlink manifest; added, removed,
|
||||
# retargeted, or manifest-only-tampered generated links also exit 43. This detects
|
||||
# accidental, independent, stale, and foreign-residue mutation—the class exposed by
|
||||
# a five-month-stale .next that produced 19 phantom TS2307 errors.
|
||||
# It does NOT defend against a same-UID actor that can rewrite both manifest and
|
||||
# marker consistently (CWE-345). RM-59 tracks the required executor/spine-side
|
||||
# trust anchor outside worktree authority.
|
||||
pnpm preflight
|
||||
|
||||
# Optional local queue service only. This does not start PostgreSQL.
|
||||
docker compose up -d valkey
|
||||
|
||||
@@ -230,6 +243,7 @@ Gateway start command until KBN-101-02 makes that state fail closed.
|
||||
### Quality Gates
|
||||
|
||||
```bash
|
||||
pnpm preflight # Checkout/dependency/generated-state validation
|
||||
pnpm typecheck # TypeScript type checking (all packages)
|
||||
pnpm lint # ESLint (all packages)
|
||||
pnpm test # Vitest (all packages)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"version": "0.0.2",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"build": "next build",
|
||||
"build": "node ../../scripts/build-web.mjs",
|
||||
"dev": "next dev",
|
||||
"lint": "eslint src",
|
||||
"typecheck": "tsc --noEmit",
|
||||
|
||||
@@ -13,3 +13,13 @@ Correct for the CI container (runs as root), fatal for EVERY non-root local chec
|
||||
### **D-2 / hygiene — husky `prepare` fails `EPERM` copying into root-owned `.husky/_/`.** Repo working
|
||||
|
||||
tree has root-owned dirs (`.husky/`, repo root) under a non-root agent. Worked around with the intended `HUSKY=0` escape hatch (does NOT disable the existing pre-commit/pre-push hooks).
|
||||
|
||||
<!-- board-roll: 2 entries rolled from BOARD.md -->
|
||||
|
||||
### **D-5 / P-QUEUE-001 + P-CONFORMANCE-001 — KEYSTONE: an inert gate that erased its own evidence.**
|
||||
|
||||
Merged PR #868 (`b79336a8`) shipped a file that FAILS `pnpm format:check` ⇒ the CI format gate did not block. An unrelated later PR (#872) then reformatted that file via its own `lint-staged`, so `main` went green again and nobody learned the gate had failed to fire. Verified blob-level under the repo's own config. **Detection must be per-merge-commit against that commit's own tree** — a "is main green today" check reports all-clear on this exact defect. Binding on RM-02/RM-55. Full chain in `TASKS.md` §1a. NOT quiet-patched, by Mos's ruling: patching the symptom destroys the signal.
|
||||
|
||||
### **D-4 / P-LIFECYCLE + hygiene — a dispatched agent silently IGNORED an in-message context reset.**
|
||||
|
||||
planner-sol was at 64.3%/372k; the brief asked it to reset first; it began work on dirty context anyway. Only an out-of-band `/new` driven by the orchestrator guaranteed clean state. Confirms the postmortem thesis: **instructions are not enforcement.** Reset must be a mechanical pre-dispatch step, not a request.
|
||||
|
||||
+16
-17
@@ -1,6 +1,6 @@
|
||||
# mos-remediation — LIVE BOARD (keep < 8 KB)
|
||||
|
||||
**Phase:** EXECUTING — first PR merged; RM-01 in flight; all 3 decisions ruled.
|
||||
**Phase:** EXECUTING — P0 open. RM-01 MERGED; RM-02 (keystone gate registry) is next.
|
||||
**Updated:** 2026-07-31 (mos-remediation orchestrator; seat active on `mosaic-fleet`).
|
||||
|
||||
## Head
|
||||
@@ -16,14 +16,13 @@
|
||||
|
||||
## In-flight
|
||||
|
||||
| Task | Owner | State |
|
||||
| ------------------------------------------------- | --------------- | ------------------------------------------------------------------ |
|
||||
| PR #1026 docs (mission record + backlog) | mos-remediation | OPEN, retargeted to main, rebased; diff verified docs-only |
|
||||
| PR #1025 hygiene | — | **MERGED** 52414605; rev-974 APPROVE + CI #2158 8/8 terminal-green |
|
||||
| DECISION-1 wire-in point (charter change) | Mos / Jason | ESCALATED — both planners reject the charter's target |
|
||||
| DECISION-2 rollback artifact + availability trade | Jason | ESCALATED |
|
||||
| DECISION-3 RM-03 vs parked PR #1023 ownership | Mos | ESCALATED |
|
||||
| RM-01 reproducible checkout (unblocks everything) | unassigned | READY TO DISPATCH |
|
||||
| Task | Owner | State |
|
||||
| ----------------------------------- | --------------- | ------------------------------------------------------------------------- |
|
||||
| RM-01 reproducible checkout | — | **MERGED** `f58b3699` (PR #1027) — rev-974 APPROVE + CI #2172 8/8 green |
|
||||
| RM-02 gate registry ★keystone | unassigned | **READY** — depends only on RM-01; not held by RM-03 |
|
||||
| RM-03 queue guard (3 defects) | — | HOLD — #1023 SUPERSEDED-PENDING-JASON |
|
||||
| RM-59 close D-19 residual risk | — | BLOCKED by RM-12/RM-21/RM-25 (spine + executor) — tracked edge, not prose |
|
||||
| `remediation/state` snapshot → main | mos-remediation | opening at this mission seam |
|
||||
|
||||
## Fleet seats
|
||||
|
||||
@@ -39,6 +38,14 @@
|
||||
- Freeze: LIFTED for this workstream only.
|
||||
- Git identity: `MOSAIC_GIT_IDENTITY=mos-dt-0` INTERIM. Mos ruled gate-16 HOLDS (author≠reviewer is what
|
||||
gate-16 protects; rev-974 reviews, mos-dt-0 never self-reviews). Dedicated identity TRACKED, Mos provisions.
|
||||
- Capability check (D-11b): before dispatching seat X to provider Y, verify
|
||||
`~/.config/mosaic/secrets/gitea-tokens/gitea-<Y>-<X>.token` exists. Token-file set = authoritative
|
||||
capability registry. Mos owns provisioning; escalate missing pairs to him.
|
||||
- Seat identity (D-11a): token identity AND `git config user.name`/`user.email` must BOTH be set and
|
||||
agree. Exporting `MOSAIC_GIT_IDENTITY` alone does NOT fix commit authorship.
|
||||
- Standing worker-brief doctrine (accreted, mandatory in every brief): don't weaken a RED test to make
|
||||
it pass; if a check is unrunnable as written SAY SO, never silently substitute; `agent-send -f` never
|
||||
`-m`; heavy artifacts off shared `/tmp`.
|
||||
- Remote control: native `/remote-control` NOT wired in this runtime. Path is **Mos-relay**
|
||||
(Jason ↔ mos-claude via Discord ↔ mos-remediation via agent-send). Not a blocker.
|
||||
|
||||
@@ -72,14 +79,6 @@ PR #1025 AC2's fixture `mkdir -p apps/*/venv/lib` creates a literal `apps/*/venv
|
||||
|
||||
Running the required `ci-queue-wait.sh --purpose push` before pushing produced `state=unknown ... exit 0` — the exact defect at `ci-queue-wait.sh:282-288` that PR #1023 is parked on. It also evaluated `branch=main` rather than the branch being pushed. The mission's own required pre-push gate passed me on an indeterminate result. Third independent live instance of the class.
|
||||
|
||||
### **D-5 / P-QUEUE-001 + P-CONFORMANCE-001 — KEYSTONE: an inert gate that erased its own evidence.**
|
||||
|
||||
Merged PR #868 (`b79336a8`) shipped a file that FAILS `pnpm format:check` ⇒ the CI format gate did not block. An unrelated later PR (#872) then reformatted that file via its own `lint-staged`, so `main` went green again and nobody learned the gate had failed to fire. Verified blob-level under the repo's own config. **Detection must be per-merge-commit against that commit's own tree** — a "is main green today" check reports all-clear on this exact defect. Binding on RM-02/RM-55. Full chain in `TASKS.md` §1a. NOT quiet-patched, by Mos's ruling: patching the symptom destroys the signal.
|
||||
|
||||
### **D-4 / P-LIFECYCLE + hygiene — a dispatched agent silently IGNORED an in-message context reset.**
|
||||
|
||||
planner-sol was at 64.3%/372k; the brief asked it to reset first; it began work on dirty context anyway. Only an out-of-band `/new` driven by the orchestrator guaranteed clean state. Confirms the postmortem thesis: **instructions are not enforcement.** Reset must be a mechanical pre-dispatch step, not a request.
|
||||
|
||||
<!-- BOARD-ROLL:END -->
|
||||
|
||||
## Decisions log
|
||||
|
||||
@@ -7,10 +7,15 @@ mechanically until Build 3 (rotation) makes it automatic.
|
||||
|
||||
## On resume (do in order, before any orchestration action)
|
||||
|
||||
1. `cd /src/mosaic-stack` and confirm you are on the remediation working branch.
|
||||
1. `cd /src/mosaic-stack`, then **`git fetch origin remediation/state`**.
|
||||
⚠ **The live board is on the rolling branch `remediation/state`, NOT on `main`.** `main` carries only
|
||||
periodic snapshots, so reading the board from `main` will silently give you a STALE tick. Read the
|
||||
live files at `origin/remediation/state` (e.g. `git show origin/remediation/state:docs/remediation/BOARD.md`),
|
||||
or check that branch out. Every tick is pushed there immediately, so its HEAD is always the newest state.
|
||||
2. Read `docs/remediation/MISSION.md` — the charter (goal, 4 builds, 15 decisions, sequencing, directives).
|
||||
3. Read `docs/remediation/BOARD.md` — the LIVE state: current phase, in-flight tasks, fleet seat assignments,
|
||||
gate status. This is your single source of in-flight truth (kept small).
|
||||
3. Read `docs/remediation/BOARD.md` **at `origin/remediation/state`** — the LIVE state: current phase,
|
||||
in-flight tasks, fleet seat assignments, gate status. Single source of in-flight truth (kept < 8 KB;
|
||||
older entries roll to `BOARD-LEDGER.md` via `board-roll.sh`).
|
||||
4. Read the discussion checkpoint for full rationale if needed:
|
||||
`../jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md` (or the jarvis-brain repo path).
|
||||
5. **Residency attestation (fail-closed):** restate from the reloaded files — (a) the goal in one line, (b) the
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Mosaic Stack Remediation — Mission Charter
|
||||
|
||||
**Owner:** project orchestrator `mos-remediation` (Claude, launched in `/src/mosaic-stack`).
|
||||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** PLANNING (task decomposition).
|
||||
**Origin:** 2026-07-16..31 fleet lifecycle postmortem. **Status:** EXECUTING (planning complete; RM-01 in flight).
|
||||
**HOLD lifted** for this workstream by Jason, 2026-07-31 — "begin full mosaic fleet operation on this."
|
||||
|
||||
## Goal
|
||||
@@ -10,23 +10,104 @@ Convert the 15 accepted postmortem remediation proposals into a working, **dogfo
|
||||
**North star:** anything with a deterministic right answer moves OUT of the LLM into a deterministic
|
||||
gate/program; the LLM handles only genuine judgment.
|
||||
|
||||
### First-class principle — observe the property, not the exit code
|
||||
|
||||
> **No write is done until the requested PROPERTY is observed. A success exit code is not evidence.**
|
||||
>
|
||||
> **Success output is designed to be believed.** That is the whole reason the inert-gate class exists
|
||||
> and why P-WRAPPER-001's tri-state (`verified` / `written-unverified` / `failed`) is not optional. The
|
||||
> failure is not carelessness — a green is _engineered_ to be trusted, so trusting it is the default
|
||||
> behaviour of a competent operator, not a lapse.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31) after the orchestrator committed this exact error: a
|
||||
> `--draft` flag was silently dropped by a wrapper fallback that still exited 0, and the PR was reported
|
||||
> as a draft on the strength of the exit code rather than an observed `draft: true` (D-12). Twelve
|
||||
> failure instances were banked in that session; **three of them were the orchestrator's own.** That
|
||||
> ratio is the point — the mechanism must catch the mechanic too, or it is not a mechanism.
|
||||
>
|
||||
> Operationally: after any write, read back the property you required. Applies to gates, wrappers, PR
|
||||
> flags, commit authorship, file installs, and message delivery alike.
|
||||
|
||||
### First-class principle — pre-registration prevents retrofitting, and nothing else
|
||||
|
||||
> **A pre-registered check set can fail in three distinct ways:**
|
||||
>
|
||||
> | mode | the set is… | found as |
|
||||
> | --------------------------- | ------------------------------------------------- | -------- |
|
||||
> | **WRONG** | a check does not test what it claims | D-8 |
|
||||
> | **INCOMPLETE** | green while a criterion's requirement is untested | D-17 |
|
||||
> | **INTERNALLY INCONSISTENT** | two criteria cannot both hold | D-18 |
|
||||
>
|
||||
> **Pre-registration protects against exactly one thing: retrofitting a check to fit the implementation
|
||||
> it is supposed to judge.** It confers neither correctness, nor coverage, nor consistency. "We
|
||||
> pre-registered the checks" has been treated as though it settled the question — it settles one of
|
||||
> three.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31). All three modes were found on this mission's own **first
|
||||
> delivery**, by the machinery applied to its own work — not by inspection, and not by looking for them.
|
||||
>
|
||||
> **Enforceable form — RM-02's four clauses.** The registry must establish that: (1) each check is
|
||||
> **right** — proven red for its own stated reason before its green counts; (2) the set **covers** —
|
||||
> every criterion bound to a case that actually exercises it; (3) no two criteria **conflict** —
|
||||
> mutual unsatisfiability is a registry defect discoverable by construction; (4) when a criterion's
|
||||
> meaning changes, the registry **retains original text, restatement, and reason**, so evolution stays
|
||||
> auditable. A criterion with no case that can fail for its own reason is unregistered in substance,
|
||||
> however it reads in the manifest.
|
||||
|
||||
### Corollary — never ship an integrity claim dressed as a property
|
||||
|
||||
> A verification artifact that can be forged by whoever it is meant to catch verifies nothing. If a
|
||||
> manifest, marker, ledger, or receipt is writable by the same actor whose behaviour it certifies, it
|
||||
> **certifies the attack.** Such an artifact must sit inside the integrity envelope it belongs to,
|
||||
> publish atomically, and carry a **tamper negative-control observed red** — otherwise its integrity is
|
||||
> a _claim_, not a _property_.
|
||||
>
|
||||
> **If it cannot be made tamper-evident, say so and reconsider the approach.** Laundering foreign
|
||||
> content as certified is the only unacceptable outcome; an honest "this cannot be verified" is always
|
||||
> available and always preferable.
|
||||
|
||||
### First-class principle — when a property cannot exist at the layer it was specified
|
||||
|
||||
> Some required properties are **impossible at the layer that asked for them** — not hard, impossible.
|
||||
> A local check cannot defend against an actor who can rewrite the check itself. When that happens,
|
||||
> there are exactly three honest moves, and all three are mandatory:
|
||||
>
|
||||
> 1. **Implement what the layer _can_ guarantee.** Partial protection against the class it was actually
|
||||
> born from is worth having.
|
||||
> 2. **State the boundary precisely, in BOTH directions.** What it does _not_ defend, **and** beside it
|
||||
> what it _does_. A reader who sees only the negative dismisses the check as worthless; one who sees
|
||||
> only the positive over-trusts it. **Both together is the honest artifact** — either alone misleads.
|
||||
> 3. **Record where the real guarantee will come from — as a TRACKED DEPENDENCY, not prose.** It must
|
||||
> name a task that someone must close. _A documented gap with no owner becomes a permanent gap that
|
||||
> reads as intentional._
|
||||
>
|
||||
> **A written-down gap is acceptable engineering. An implied-fixed gap is this mission's core failure in
|
||||
> a new costume** — a verification artifact that verifies nothing, with a green to prove it.
|
||||
>
|
||||
> Promoted to the charter by Mos (2026-07-31) from D-19. Origin: the RM-01 symlink manifest could not be
|
||||
> made tamper-evident against a same-UID actor (CWE-345), because the manifest and its marker share one
|
||||
> writable tree. The implementing seat **escalated rather than relabelling self-authentication as
|
||||
> tamper-resistance** — the corollary above firing on its first real adversarial test, on the cheapest
|
||||
> seat in the loop. Residual risk bound to **RM-59** (`depends_on: RM-12, RM-21, RM-25`), where the
|
||||
> choke-point executor and spine verify from _outside_ the worktree's authority.
|
||||
|
||||
## Decision record (authoritative, immutable)
|
||||
|
||||
- **15/15 proposals decided: 13 accept, 2 modify (P-AUTHORITY-001, P-INBOX-001), 0 reject.**
|
||||
- Site + `annotations.json`: `jarvis-brain/docs/postmortem-spec/site/` (committed, origin/main).
|
||||
- Discussion checkpoint (rich rationale per proposal): `jarvis-brain/docs/scratchpads/postmortem/REMEDIATION-DISCUSSION-STATE.md`.
|
||||
- Postmortem report: mosaicstack/stack PR #107 (merged 88f4ee04).
|
||||
- MACP wiring scout (verdict c=STRANDED): `/tmp/macp-wiring-investigation.md` (copy into this dir — see TODO).
|
||||
- MACP wiring scout (verdict c=STRANDED): [`MACP-WIRING-SCOUT.md`](./MACP-WIRING-SCOUT.md) (copied into this dir; TODO discharged). Its findings are sound; its _recommended wire-in point_ is superseded by DECISION-1.
|
||||
|
||||
## The plan — 15 proposals collapse to 4 builds + hygiene
|
||||
|
||||
| Build | Absorbs | What it is |
|
||||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** in at `mosaic_orchestrator.py::run_single_task` — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. |
|
||||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||||
| Build | Absorbs | What it is |
|
||||
| ------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| **1. One choke-point service** (mechanical enforcer) | MISSION, STATE, AUDIT, WRAPPER, QUEUE | Deterministic program every task/data mutation flows through. **Wire the stranded `@mosaicstack/macp`** — typed tasks, gate-runner, event ledger, credential binding, tri-state write outcomes. ⚠ **Target CORRECTED 2026-07-31 (DECISION-1, Mos):** a new production Node `TaskExecutor` on the **live** dispatch path (`packages/mosaic` launch + `packages/coord`), which Coord/Forge/live-dispatch submit through. **NOT** `mosaic_orchestrator.py::run_single_task` — that controller is `"enabled": false` and references a dispatcher absent from this checkout; wiring it would strand the executor, reproducing this mission's own disease. The Python rail is **deleted**, not ported. Both planners reached this independently. |
|
||||
| **2. One durable spine + hot path** | (storage under everything) | **PG system-of-record + Redis hot queue** (transactional-outbox). Mission/tasks/state-claims/audit-ledger/comms-inbox all land here. |
|
||||
| **3. Rotation lifecycle** (finish the Mission Control Plane) | LIFECYCLE, CONTRACT, GUIDE, RECOVERY | Coordinator daemon: contract-hash binding, compaction-detected → rotate-not-compact, checkpoint→fresh-session→rehydrate, broker-independent recovery. Deterministic, not an LLM. Reuse `packages/coord`; existing PRD at `docs/mission-control/`. |
|
||||
| **4. Comms service** | AUTHORITY, INBOX (+ versioning roadmap) | Envelope (comms/v1) → sole-path service → PG/Redis → pluggable adapters (tmux→Matrix/Discord/Slack/Telegram). Version the protocol, not participants. |
|
||||
| **+ Hygiene & proof** | FLEET, WORKFLOW, CONFORMANCE | One roster-owned socket/host + stale GC; allowlist auto-sync; the conformance harness that fault-injects the failure classes and proves builds 1–4 hold. |
|
||||
|
||||
## The finding that sets the cost
|
||||
|
||||
@@ -37,6 +118,7 @@ orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retir
|
||||
## Sequencing (skeleton — adversarial decomposition refines this)
|
||||
|
||||
1. **Spine + choke-point service** (builds 1+2) — foundation; unlocks MISSION/STATE/AUDIT/WRAPPER/QUEUE at one integration point.
|
||||
(Per DECISION-1, a P0 phase of provable-gate + activation work precedes this; see `TASKS.md` §3.)
|
||||
2. **Rotation daemon** (build 3) on that spine — the drift fix proper.
|
||||
3. **Comms service** (build 4) — envelope → service → PG/Redis → adapters; retire direct-tmux.
|
||||
4. **Hygiene + conformance** (build 5) — fleet convergence, allowlist sync, dogfood harness.
|
||||
@@ -50,6 +132,11 @@ orphaned context loader, a fail-open bypass. **Work = wire + consolidate + retir
|
||||
(WRAPPER), auto-sync sweep (WORKFLOW), #1018 stale-consumed (INBOX). The fleet is its own test bed.
|
||||
- **Orchestration tracking → DB**, hard cutover ("rip off the bandaid"), NO flat-file interim. jarvis-brain
|
||||
PDA flat-files untouched. Current flat-file tracking runs as-is/unhardened until DB tracking is real, then one clean replace.
|
||||
- ⚠ **QUALIFIED 2026-07-31 (DECISION-2, Mos):** the DB spine **must NOT be a single-point hard-stop.**
|
||||
A broker-independent / degraded mode **and** a rehearsed rollback artifact are **design requirements**
|
||||
(P-RECOVERY-001), binding now on RM-12, RM-13, RM-23, RM-36 and RM-53. This **supersedes** the earlier
|
||||
orchestrator recommendation to pre-commit "no DB ⇒ the fleet stops" — that answer is _not_ on record.
|
||||
Only the specific availability _target_ remains open, queued for Jason; it does **not** block current work.
|
||||
|
||||
## The 15 decisions (one-line; full rationale in the checkpoint)
|
||||
|
||||
|
||||
+526
-11
@@ -93,6 +93,520 @@ and must not be cited as merge evidence. Rely on reviewer clearance + real CI.
|
||||
Three independent live instances in a single session — format gate, agent context reset, queue guard —
|
||||
is the class confirmed, not anecdote.
|
||||
|
||||
### D-20 — the orchestrator's own documentation overclaimed, and a reviewer disproved it empirically
|
||||
|
||||
`rev-974` blocked PR #1027 a second time. **The defect was not in the code — it was in this file**, at
|
||||
D-18's entry, written by the orchestrator.
|
||||
|
||||
Two faults, both mine:
|
||||
|
||||
1. **D-18's AC2 restatement omitted the scope clause** that D-19 later established as mandatory
|
||||
("within an accidental/independent-mutation threat model").
|
||||
2. **D-18 asserted that the tampered-manifest control turns integrity "from a claim into a property."**
|
||||
It does not, and _cannot_. That sentence was written **before** D-19 proved the property impossible
|
||||
at this layer, and was never revised when D-19 landed.
|
||||
|
||||
**The reviewer did not merely read it — it disproved it.** It performed a **same-UID consistent
|
||||
manifest + marker rewrite**, and **preflight passed**. My documented claim was falsified by experiment.
|
||||
Code, README, scratchpad and PR body all stated both threat-model directions correctly; **this file was
|
||||
the only place still overclaiming.**
|
||||
|
||||
**This is two banked findings firing on the orchestrator at once:**
|
||||
|
||||
- **The integrity-claim corollary** — I wrote an integrity _claim_ in the voice of an integrity
|
||||
_property_, in the very document that defines the rule against doing so.
|
||||
- **D-14 (propagation)** — D-19 superseded D-18's assertion. I propagated the consequence into the
|
||||
charter and the delivery conditions, but **not back into D-18 itself.** A ruling that fails to
|
||||
propagate _backwards_ into the finding it supersedes is the same defect as one that fails to
|
||||
propagate forwards, and I did not audit for that direction.
|
||||
|
||||
**Corrected in place**, with the original wording quoted and the empirical disproof recorded, rather
|
||||
than silently rewritten — the same standard demanded of any restated criterion.
|
||||
|
||||
**Requirement on RM-02 (fifth clause).** Documentation asserting a _security or integrity_ property is
|
||||
itself a claim requiring a negative control. Where a document states "X is guaranteed", the registry
|
||||
must hold a case that **fails if X is not guaranteed** — and that case must have been observed red.
|
||||
**Prose is not exempt from the mission's own evidentiary standard**, and prose in the _governing_
|
||||
document least of all: it is the artifact most likely to be quoted as authority long after the code has
|
||||
moved on.
|
||||
|
||||
**Reviewer credit.** rev-974 was briefed that its highest-priority check was "confirm the PR claims no
|
||||
more than it can deliver, and a softened or omitted boundary is a finding even though the code works."
|
||||
It applied that instruction **to the orchestrator's own governing document** and produced an experiment
|
||||
to settle it. That is the review standard this mission is trying to make ordinary.
|
||||
|
||||
### D-19 — an integrity property that cannot exist at the layer it was specified
|
||||
|
||||
Implementing D-18's manifest, the seat + a Codex security review reached **CWE-345**: the symlink
|
||||
manifest and the source-hash marker both live in the **same same-UID writable generated tree**, so an
|
||||
actor with that UID can plant a rogue link, regenerate _both_, retain the fingerprint, and pass. **No
|
||||
local cryptographic construction fixes self-authentication** without a key outside that actor's
|
||||
authority; relocating the marker changes the path, not the authority.
|
||||
|
||||
The seat **escalated rather than describing self-authentication as tamper-resistant** — the explicit
|
||||
failure mode the charter corollary demands. That is the corollary working, on its first real test.
|
||||
|
||||
**Ruling — Option A: scope AC2 to accidental / independent / stale mutation; retain the design.**
|
||||
Rationale, recorded so it can be challenged:
|
||||
|
||||
1. **The undefendable boundary is not the weak link.** An actor with same-UID write can already edit the
|
||||
source, the tests, `scripts/preflight.mjs` itself, and `.husky/*`. If they have that, _nothing_ in the
|
||||
local checkout is trustworthy — hardening the manifest buys no real security while **implying
|
||||
protection that does not exist**, which is worse than the gap.
|
||||
2. **What AC2 is actually for.** These checks exist because a five-month-stale `.next` produced 19
|
||||
phantom `TS2307` errors indistinguishable from real ones (**D-5**). That is staleness, drift and
|
||||
foreign residue — and against that class the design demonstrably works.
|
||||
3. **A real trust anchor arrives later, from this mission's own architecture.** An anchor must live
|
||||
outside the actor's authority; for a fleet running as one user that means a separate service —
|
||||
precisely the **choke-point executor + PG spine** of Builds 1–2, which verify outside the worktree's
|
||||
authority. Hand-rolling key distribution for a local preflight now would duplicate that work badly.
|
||||
4. Option C (structural policy, no manifest) is strictly worse — it cannot detect a **removed** expected link.
|
||||
|
||||
**Option A is acceptable only with honest labelling**, or it becomes the disease it is meant to cure.
|
||||
Conditions (last two added/sharpened by Mos):
|
||||
|
||||
- Threat model stated verbatim in the code **and** the PR; the words _tamper-proof / tamper-evident /
|
||||
secure_ **barred** from that context; the scope carried in AC2's restatement; every control kept
|
||||
RED-first including manifest-only tamper.
|
||||
- **State the boundary in BOTH directions.** Not only what it does _not_ defend (same-UID write; no
|
||||
local construction can) but, beside it, what it **does** defend: accidental / independent / stale /
|
||||
foreign-residue mutation — the **D-5** class it was born from (the five-month `.next` and its 19
|
||||
phantom `TS2307`s). _A reader who sees only the negative dismisses the check as worthless; one who
|
||||
sees only the positive over-trusts it. Both together is the honest artifact._
|
||||
- **The residual risk is a HARD TRACKED DEPENDENCY EDGE, not a comment.** It is **RM-59**, owned by the
|
||||
choke-point executor + spine work (`depends_on: RM-12, RM-21, RM-25`), and the AC2 scope note must
|
||||
cite that id. _"Record where the real guarantee comes from" only holds if the record is a live
|
||||
dependency someone must close._ **A documented gap with no owner becomes a permanent gap that reads
|
||||
as intentional.**
|
||||
|
||||
**The generalizable rule.** When a required property **cannot exist at the layer where it was
|
||||
specified**, the honest moves are: implement what the layer _can_ guarantee, **state the boundary
|
||||
precisely**, and record where the real guarantee will come from. **A known gap that is written down is
|
||||
acceptable; a gap that is implied fixed is not.** Silence here would have shipped a verification
|
||||
artifact that verifies nothing — with a green to prove it.
|
||||
|
||||
### D-18 — two pre-registered criteria were mutually unsatisfiable, discoverable only at implementation
|
||||
|
||||
Implementing D-17's fix surfaced a conflict **between** pre-registered criteria:
|
||||
|
||||
- **AC2** (as written) — reject symlinked generated state.
|
||||
- **AC4** — the canonical `pnpm -w build` succeeds and leaves no residue.
|
||||
|
||||
Verified independently rather than taken on report: `apps/web/next.config.ts:4` sets
|
||||
`output: 'standalone'`, and the built tree contains **42 legitimate pnpm dependency symlinks** under
|
||||
`.next/standalone/node_modules`. A blanket descendant-symlink rejection makes the canonical build fail
|
||||
its own preflight with exit 43. **AC2 read literally is unsatisfiable alongside AC4 under this
|
||||
configuration**, and nothing short of building the tree would have revealed it.
|
||||
|
||||
**Third distinct failure mode of a pre-registered check set**, completing the chain:
|
||||
|
||||
| finding | a pre-registered check set can be… |
|
||||
| ------- | ------------------------------------------------------------------ |
|
||||
| D-8 | **wrong** — a check that does not test what it claims |
|
||||
| D-17 | **incomplete** — green while a criterion's requirement is untested |
|
||||
| D-18 | **internally inconsistent** — two criteria that cannot both hold |
|
||||
|
||||
The implementing seat escalated instead of silently picking a winner. That matters: **quietly resolving
|
||||
a conflict between pre-registered criteria destroys the point of pre-registering them** — the registration
|
||||
exists so that changes of meaning are auditable rather than absorbed.
|
||||
|
||||
**Resolution (orchestrator ruling).** Approved a **build-certified symlink manifest**: `.next` itself is
|
||||
still rejected as a symlink; descendants are rejected unless _exactly_ certified by a manifest the build
|
||||
publishes atomically. Strictly **stronger** than blanket rejection — it also catches a **retargeted**
|
||||
symlink, which blanket rejection cannot distinguish from a legitimate one.
|
||||
|
||||
**AC2 restated (recorded, not absorbed).** _Generated state must reject `.next` itself being a symlink
|
||||
or non-directory, and must reject any descendant symlink not exactly certified by the build manifest —
|
||||
added, removed, retargeted, or manifest-only-tampered all fail with exit 43 — **within an accidental /
|
||||
independent-mutation threat model.**_
|
||||
|
||||
> ⚠ **This entry is superseded in part by [D-19](#d-19). Do not read D-18 standalone.** The scope clause
|
||||
> above is load-bearing: the design **cannot** defend against a same-UID actor, which can rewrite the
|
||||
> manifest and the marker consistently (CWE-345). D-18 was written **before** that impossibility was
|
||||
> established.
|
||||
|
||||
**Hardening required before this counts.** The manifest is itself generated state, so **a manifest
|
||||
writable by whoever plants a rogue symlink certifies the attack** — that is the one way this design
|
||||
fails. It must sit inside the same ownership/fingerprint envelope, published atomically via the existing
|
||||
marker mechanism, with negative controls **observed red first** for: added, removed, retargeted,
|
||||
**manifest-only-tampered**, plus a positive control that the canonical build passes.
|
||||
|
||||
> ⚠ **CORRECTED (D-20).** This paragraph originally ended: _"without it, integrity is a claim rather
|
||||
> than a property."_ **That overclaimed**, by implying the control makes integrity a _property_. It does
|
||||
> not, and cannot. The manifest-only-tamper control detects **independent** mutation of the manifest;
|
||||
> it confers **no authenticity** against an actor who rewrites manifest _and_ marker together.
|
||||
> `rev-974` disproved the original wording empirically — a same-UID consistent manifest+marker rewrite
|
||||
> **passed preflight**. Integrity here remains a scoped **drift-detection** property, never an
|
||||
> authenticity one. See D-19 and the charter principle on properties that cannot exist at their
|
||||
> specified layer.
|
||||
|
||||
**Requirement on RM-02 (fourth clause).** The registry must detect **conflicts between registered
|
||||
criteria**, not only wrongness and coverage. Two criteria that cannot simultaneously hold is a registry
|
||||
defect discoverable by construction — and when a criterion is restated, the registry must retain the
|
||||
original text, the restatement, and the reason, so the evolution stays auditable.
|
||||
|
||||
### D-17 — a pre-registered criterion passed a green suite without being satisfied
|
||||
|
||||
`rev-974` returned **CHANGES REQUESTED** on PR #1027 with one blocking finding, and it is the sharpest
|
||||
instance of the session's theme because it occurred **inside our own verification machinery**.
|
||||
|
||||
**AC2** was pre-registered before any code was written, and explicitly required that **symlinked
|
||||
generated state be rejected**. The implementation does not do it:
|
||||
|
||||
```sh
|
||||
ln -s /etc/hosts apps/web/.next/reviewer-symlink
|
||||
pnpm preflight # → "checkout preflight passed", exit 0
|
||||
# → required: generated-state exit 43
|
||||
```
|
||||
|
||||
The acceptance suite was **21/21 green** throughout. Confirmed independently rather than relayed:
|
||||
`scripts/preflight.mjs:82-92` rejects symlinks on the **source** path; `:28` merely _skips_ symlinked
|
||||
directories rather than rejecting them; and the **generated-state** path at `:141-163` `lstat`s and
|
||||
checks `uid` (ownership) but **never** calls `isSymbolicLink()`. The suite's only symlink cases
|
||||
(`preflight.test.mjs:59`, `:115`) cover the turbo binary and a _source_ file. No generated-state case
|
||||
exists anywhere.
|
||||
|
||||
**So: criterion pre-registered, suite green, requirement unmet.** Nobody was careless — the coverage gap
|
||||
is _invisible from a green_, which is the entire problem.
|
||||
|
||||
**This sharpens D-8 rather than repeating it.** D-8 established that pre-registration does not confer
|
||||
_correctness_ (a check can be wrong when written). D-17 establishes the adjacent failure:
|
||||
**pre-registration does not confer _coverage_** — a suite can be green, and every registered criterion
|
||||
can appear satisfied, while a criterion's actual requirement is untested. The two together mean a
|
||||
registry of checks needs **two** properties, not one: each check must be _right_, and the set must
|
||||
_actually exercise_ what it claims.
|
||||
|
||||
**Requirement on RM-02 (third clause).** The registry must bind each acceptance criterion to the
|
||||
**specific case that exercises it**, and prove that case red before trusting its green. A criterion with
|
||||
no case that can fail for _that criterion's stated reason_ is unregistered in substance however it
|
||||
appears in the manifest. This is mutation testing pointed at the **criterion-to-case mapping**, not
|
||||
merely at the gate.
|
||||
|
||||
**Credit where due:** the reviewer also declined to re-run AC8, stating plainly that the PR carried it
|
||||
forward with no runnable command rather than silently substituting a different boundary test. That is
|
||||
the D-8 clause working a second time, in the same review that produced D-17.
|
||||
|
||||
### D-16 — the local test gate and the CI test gate disagree by environment
|
||||
|
||||
Mos flagged a shape worth chasing: if `pnpm test` exits non-zero on a _pre-existing_ guard, then either
|
||||
`main` is red and merges step around it (the #868 shape again), or CI does not run that path. **Both
|
||||
branches turned out wrong, and the truth is a third thing.** Established by running it, not by asking:
|
||||
|
||||
CI runs **exactly** `pnpm test` (`.woodpecker/ci.yml`, `test` step) — the same command. So the path _is_
|
||||
exercised. Yet:
|
||||
|
||||
| environment | result |
|
||||
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| CI container | `test` step **green** (#2158, #2167) |
|
||||
| this host, clean worktree | **exit 97** — `WAKE-ASSERT INIT ABORT: BASH_LINENO convention violated on bash 5.2.15(1)-release … expected [3 4], probe reported [3 5] (#973)`, after `PASS=18 FAIL=0` |
|
||||
| this host, main checkout | **exit 1** — a _different_, second defect (below) |
|
||||
|
||||
The guard is **environment-dependent**: it aborts on this host's bash and not in CI's container. `git
|
||||
diff origin/main...` confirms PR #1027 touches **zero** files under `packages/mosaic`, so the guard is
|
||||
genuinely pre-existing and unrelated — **f10-coder's report was accurate in every particular**, and
|
||||
`main` is equally affected on this host.
|
||||
|
||||
**So it is not "merges step around a red" — it is worse in one specific way: the local gate and the CI
|
||||
gate do not agree about what passing means.** No agent on this host can obtain a green `pnpm test` at
|
||||
all, on any branch, including `main`. A gate an operator cannot run is a gate that only CI enforces,
|
||||
and a gate only CI enforces cannot be a pre-push gate. This is the hermeticity/portability class
|
||||
already live as #1007 (PR #1024).
|
||||
|
||||
**Second, independent defect found while establishing the above.** In the main checkout the same
|
||||
package fails differently — exit 1 — because a test **scans the working tree** and asserts on files it
|
||||
finds, picking up `apps/coordinator/venv/**` (third-party `site-packages`: `pi = math.pi` in `rich`,
|
||||
`setuptools`, `mypy`). **A test whose result depends on untracked files present in the tree is not
|
||||
hermetic.** This is the _same_ contamination source that made `pnpm format:check` unpassable (D-1/D-7
|
||||
hygiene) — one untracked foreign tree silently breaking two independent gates.
|
||||
|
||||
**Requirements.** RM-01/RM-04: a gate must produce the same verdict on a developer host and in CI, or
|
||||
declare loudly that it cannot run here — never diverge silently. RM-02 registers both as cases: the
|
||||
environment-divergence guard, and a hermeticity control asserting a suite's verdict is unchanged by the
|
||||
presence of untracked directories. Coordinate with #1007/#1024 rather than opening a third lane.
|
||||
|
||||
**Ownership (Mos, 2026-07-31).** The hermeticity fix **is** PR #1024, which sits in **Jason's parked
|
||||
delivery stack** — so, like #1023, its disposition is Jason's. Marked `SUPERSEDED-PENDING-JASON`
|
||||
alongside #1023. D-16 strengthens the urgency but does not transfer ownership: **we do not open a third
|
||||
lane on a parked PR.** The one-line escalation for Jason: _two independent gates
|
||||
(`format:check`, `pnpm test`) were broken by a single untracked directory, and a third
|
||||
(`pnpm test`) disagrees between host and CI — non-hermetic gates make every green host-dependent._
|
||||
|
||||
**Sharpened statement of the class (Mos).** A pre-push gate an operator _cannot run locally_ is a gate
|
||||
only CI enforces — so pointing `.husky/pre-push` at it **misrepresents where the gate lives**. Combined
|
||||
with the shared root cause across two gates, the finding is: **non-hermetic gates make every green
|
||||
host-dependent.** A gate that only appears to pass depending on which host runs it is this mission's
|
||||
exact subject, one meta-level up.
|
||||
|
||||
**#1027 disposition (Mos):** proceeds on **CI-green**. CI is the authoritative gate; the local exit-97
|
||||
is a known host-specific guard abort, irrelevant to the merge decision.
|
||||
|
||||
### D-15 — token scope is not repository permission (a THIRD capability layer)
|
||||
|
||||
`f10-coder` was provisioned with `gitea-mosaicstack-f10-coder.token`, scopes `write:repository` +
|
||||
`write:issue`, and the mint was verified by "repo access returns 200". It then failed to push:
|
||||
|
||||
```
|
||||
remote: error: User permission denied for writing.
|
||||
remote: error: pre-receive hook declined
|
||||
```
|
||||
|
||||
Verified objectively rather than inferred (per the charter principle):
|
||||
|
||||
| probe | result |
|
||||
| ------------------------------------------------------ | ------------------------------------------- |
|
||||
| `GET /repos/mosaicstack/stack/collaborators/f10-coder` | **404** — not a collaborator |
|
||||
| repo permissions as seen by **its own token** | `admin: false`, `push: false`, `pull: true` |
|
||||
|
||||
**Capability has at least three independent layers, and satisfying two proves nothing about the third:**
|
||||
|
||||
1. **Token file exists** → raw-API authentication works (D-11b).
|
||||
2. **`tea` login exists** → tea-dependent wrapper paths work (D-13).
|
||||
3. **Repository permission granted** (collaborator/team membership) → _writes_ are actually authorised.
|
||||
|
||||
A token can carry `write:repository` scope and still be refused, because **scope bounds what a token
|
||||
may attempt; repository permission decides what the user may do.** They are different systems.
|
||||
|
||||
**This is the charter principle failing on the very check meant to confirm capability.** The mint was
|
||||
validated by an HTTP 200 on a _read_. A 200 proves reachability; it does not prove the property that
|
||||
was required, which was **write**. Both the provisioner and I accepted it — the same
|
||||
`written-unverified` treated as `verified` as D-12, one layer up, on a check whose entire purpose was
|
||||
verification.
|
||||
|
||||
**Requirements.** RM-50's pre-dispatch capability check must probe the **effective permission for the
|
||||
operation intended** — for push authority, assert `permissions.push == true` as that seat, not token
|
||||
existence and not a 200 on a read. RM-04's registry reconciliation covers all three layers, with a
|
||||
must-fail control for each. A capability check that cannot fail on a seat lacking write permission is
|
||||
itself an inert gate.
|
||||
|
||||
### D-14 — a ruled decision did not propagate to the authoritative record
|
||||
|
||||
DECISION-1 (the corrected choke-point wire-in target) was ruled by the coordinator and applied to
|
||||
`TASKS.md`. **`MISSION.md` — the charter, the document a cold-starting seat reads first — kept the
|
||||
superseded target for hours.** It was flagged `CONTESTED` in a board note, then the ruling landed and
|
||||
nobody edited the charter. A seat resuming from the charter would have read the _rejected_ target as
|
||||
authoritative and wired the choke point into a disabled rail — the precise failure the ruling existed
|
||||
to prevent.
|
||||
|
||||
Caught by hand, during an unrelated edit. Nothing would have caught it otherwise.
|
||||
|
||||
**This is P-MISSION-001 turned on ourselves.** The mission's own thesis is that convention exists and
|
||||
_enforcement_ is the gap: a decision that lives in a chat ruling and a board note, but not in the
|
||||
source of truth, has not actually been made — it has been _agreed_. The two are different, and the
|
||||
difference is exactly what this mission is about.
|
||||
|
||||
> ⚠ **AMENDED by D-20 — propagation is BIDIRECTIONAL.** As first written, this requirement was read by
|
||||
> both the orchestrator and the coordinator as _forward_ propagation only: a ruling reaches the
|
||||
> documents that state the new rule. **D-20 proved that insufficient.** When D-19 superseded part of
|
||||
> D-18, the consequence propagated forward into the charter and the delivery conditions but **never
|
||||
> backward into D-18 itself**, which went on asserting a withdrawn claim — and a reviewer disproved it
|
||||
> by experiment. **A supersession must update BOTH the documents that render the new rule AND the
|
||||
> finding it retires, with the retired wording quoted rather than deleted.** Backward propagation is
|
||||
> the same defect as forward; neither of us audited that direction until it bit.
|
||||
|
||||
**Requirement (not merely a fix).** A ruled decision must propagate **mechanically** to the
|
||||
authoritative record; it must not depend on someone remembering to edit a second file. Concretely, once
|
||||
mission state is DB-backed (RM-53 / the P-MISSION cutover):
|
||||
|
||||
- a decision is a **record**, not prose duplicated across documents;
|
||||
- documents _render_ decisions rather than restating them, so there is one place to be wrong;
|
||||
- and where duplication is unavoidable, a check asserts the authoritative record and the derived
|
||||
document agree — with a must-fail control proving divergence is detected.
|
||||
|
||||
Until then, the interim rule: **the same commit that records a ruling updates every document that
|
||||
states it — and every finding it supersedes.** Interim rules are exactly what the DB cutover exists to replace.
|
||||
|
||||
**The rule found a second instance within minutes of being written.** Auditing the charter against all
|
||||
rulings to date (rather than waiting to be bitten again) surfaced that **DECISION-2 had also not
|
||||
propagated**: `MISSION.md`'s standing directives still stated the DB hard-cutover with no mention of
|
||||
Mos's binding qualification that _the spine must not be a single-point hard-stop_ (degraded mode +
|
||||
rollback artifact required). A seat reading the charter would have designed toward an availability
|
||||
posture the coordinator had explicitly rejected — and would have found the superseded
|
||||
"no DB ⇒ the fleet stops" recommendation nowhere contradicted. Now corrected in place.
|
||||
|
||||
**Two un-propagated rulings out of two rulings that touched charter text.** The propagation gap is not
|
||||
an oversight that happened once; without a mechanism it is the _default outcome_. That is the argument
|
||||
for making this a requirement rather than a discipline.
|
||||
|
||||
### D-13 — two credential registries that can disagree (why the `--draft` fallback fired at all)
|
||||
|
||||
Diagnosing D-12's root cause surfaced a distinct defect. There are **two parallel credential
|
||||
registries**, and capability in one does not imply capability in the other:
|
||||
|
||||
| registry | contents for identity `mos-dt-0` on `mosaicstack` |
|
||||
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------- |
|
||||
| token files — `~/.config/mosaic/secrets/gitea-tokens/` | `gitea-mosaicstack-mos-dt-0.token` **EXISTS** |
|
||||
| `tea login list` | **NO** `mosaicstack` login for `mos-dt-0` (only `mosaicstack-mos` and `mosaicstack-rev-974`) |
|
||||
|
||||
So `get_gitea_token` succeeds and every raw-API path works, while every **tea-dependent** wrapper path
|
||||
fails its login validation and silently degrades to the API fallback — which is exactly what dropped
|
||||
`--draft`. **tea is not "stale"; the login simply does not exist for that identity.**
|
||||
|
||||
This matters beyond one flag: capability was declared authoritative by the token-file set (D-11b), but
|
||||
that registry does not govern the tea path. A seat can be _fully provisioned_ by the authoritative
|
||||
registry and still lose functionality with no error — only a warning, and only on the degraded path.
|
||||
|
||||
**Requirements.** RM-04 (activation coherence): the two registries must be reconciled — one source of
|
||||
truth, or a startup check asserting they agree, with a must-fail control proving disagreement is
|
||||
detected. RM-50: the pre-dispatch capability check must verify capability for the **path actually
|
||||
used**, not merely token-file presence.
|
||||
|
||||
**Confirmed working despite the gap** (so this is degradation, not outage): pushes, `pr-merge.sh`,
|
||||
PR/issue creation via API fallback, comment posting, and all reads. Impact is confined to
|
||||
tea-only features — `--draft`, `--labels`, `--milestone`.
|
||||
|
||||
**Reconciliation run by Mos (the manual form of RM-04's assert-agreement, done once by hand).** For
|
||||
`git.mosaicstack.dev`, the token-file registry holds **six** seats; `tea` holds logins for **two**:
|
||||
|
||||
| state | seats |
|
||||
| ------------------------------------------------ | -------------------------------------------------------- |
|
||||
| token file present, **no** mosaicstack tea login | `f10-coder`, `jarvis`, `mos-admin`, `mos-dt-0`, `pepper` |
|
||||
| token file present **and** tea login present | `rev-974` (only) |
|
||||
|
||||
**Five of six provisioned seats are silently degraded on tea-only features.** This is _systemic_, not
|
||||
a one-off — which is why the fix is registry reconciliation (RM-04) and not a per-seat mint. Minting
|
||||
one seat would clear a symptom and leave the class live.
|
||||
|
||||
Mos deliberately deferred the mint: it is not on RM-01's critical path, and additively editing shared
|
||||
`tea` config underneath running work is a change he declined to make without cause. Full remediation —
|
||||
mint the five missing logins **and** wire the startup must-fail assertion that _detects_ disagreement —
|
||||
lands as RM-04 at a non-critical seam, or immediately if any seat needs a tea-only feature to progress.
|
||||
|
||||
**Correction of record:** this supersedes D-11(b)'s claim that the token-file set is _the_ authoritative
|
||||
capability registry. It is **necessary but not sufficient**. Capability is **per-path**: the token file
|
||||
governs the raw-API path, the tea login governs the tea path, and the two can disagree silently.
|
||||
|
||||
### D-12 — a requested SAFETY flag was silently degraded, and I did not check
|
||||
|
||||
I created PR #1027 with `pr-create.sh ... -d` (draft) because it carries **partial, unproven work**.
|
||||
`tea` authentication was stale, so the wrapper fell back to its raw-API path — which cannot set draft —
|
||||
and emitted:
|
||||
|
||||
```
|
||||
Warning: API fallback applies title/body/head/base only; labels/milestone/draft require authenticated tea setup.
|
||||
```
|
||||
|
||||
The PR was created **not-draft**. I read the success output, saw the PR number, and moved on. I then
|
||||
reported to the coordinator that the PR was "opened as draft". **It was open, mergeable, and marked
|
||||
ready for ~25 minutes**, protected only by the words "DRAFT" and "do not merge" in its title and body —
|
||||
i.e. by prose a human might read, not by the platform control I asked for. Detected only because a
|
||||
watcher polled `draft:` and the value disagreed with my belief. Corrected by setting the `WIP:` title
|
||||
prefix (Gitea's draft mechanism); `draft: True` verified after.
|
||||
|
||||
**Three distinct failures, and the third is mine:**
|
||||
|
||||
1. **Silent degradation of a safety flag.** The fallback path dropped `--draft` and still exited 0. A
|
||||
fallback that cannot honour a _safety_ argument must fail, not proceed — degrading `--labels` is a
|
||||
nuisance; degrading `--draft` publishes unproven work as ready to merge.
|
||||
2. **The warning went to stderr and nothing consumed it.** It was correct, specific, and ignored — a
|
||||
warning nobody acts on is indistinguishable from no warning.
|
||||
3. **I did not verify the flag took effect.** I checked that the PR existed, not that it had the
|
||||
property I required. This is the mission's own thesis turned on me: **I trusted a success exit code
|
||||
over an observed state**, on exactly the class of tool this mission exists to distrust.
|
||||
|
||||
**Requirements.** RM-02: a wrapper that cannot honour a safety-relevant argument must exit non-zero —
|
||||
registered with a must-fail control asserting `--draft` on a degraded path fails rather than proceeds.
|
||||
RM-24 (tri-state write outcomes): this is precisely `written-unverified` being treated as `verified` —
|
||||
the PR write succeeded, the _requested property_ was never confirmed, and no one looked.
|
||||
|
||||
### D-11 — seat identity did not survive into git, and seat capability is invisible at dispatch
|
||||
|
||||
Two defects, one dispatch (RM-01 → `f10-coder`):
|
||||
|
||||
**(a) Identity drift — P-WRAPPER-001, reproduced on our own delivery.** The seat's commits are
|
||||
authored `mosaic-coder <[email protected]>` — the generic fallback. **You cannot tell from
|
||||
git history which seat did this work.** Recorded, not rewritten: the drift is the evidence.
|
||||
|
||||
> **Mechanism, corrected (Mos).** My original framing here was wrong, and the error was in the brief
|
||||
> before it was in the finding. `MOSAIC_GIT_IDENTITY` resolves the **token** (which per-slot credential
|
||||
> the wrappers act with). The **commit author** comes from `git config user.name` / `user.email`, which
|
||||
> is a **separate setting** — it fell back to the generic value because nothing set it. Exporting the
|
||||
> identity could never have fixed authorship. **My worker brief instructed only the export, so the
|
||||
> seat did exactly what it was told and the commits were still mis-attributed.**
|
||||
>
|
||||
> **The requirement is coherence: token and authorship must agree.** A seat acting with
|
||||
> `gitea-mosaicstack-f10-coder` must also commit as `f10-coder <[email protected]>`.
|
||||
> Either half alone is identity drift — one produces the right credential with the wrong author, the
|
||||
> other the reverse. That coherence _is_ P-WRAPPER-001, and it belongs in seat setup, not in prose
|
||||
> instructions a seat may follow correctly and still end up wrong.
|
||||
|
||||
**(b) Capability opacity.** Nothing at dispatch time revealed that `f10-coder` had no credential for
|
||||
the target provider. Per-slot tokens live at `~/.config/mosaic/secrets/gitea-tokens/`; the seat holds
|
||||
`gitea-usc-f10-coder` but not `gitea-mosaicstack-f10-coder`. This surfaced only when the seat failed
|
||||
**mid-task, after ~$9 and 69% of its context.** The orchestrator (me) selected a seat without any way
|
||||
to check it could act on the target repo — and there was no way to check.
|
||||
|
||||
`get_gitea_token` behaved **correctly**: it refused to fall through and borrow another slot's token,
|
||||
failing loud precisely to protect gate-16 attribution. The tooling was right; the _dispatch-time
|
||||
information_ did not exist.
|
||||
|
||||
**This is P-RECOVERY-001's "honest capability labeling" applied to seats rather than services.** A seat
|
||||
should declare what it can actually do — which providers, which repos, which credentials — and that
|
||||
declaration must be **checkable before dispatch**, not discovered by failure after the budget is spent.
|
||||
|
||||
**Requirements.**
|
||||
|
||||
- **RM-04 (activation coherence)** gains the identity-binding half: seat setup must set **both** the
|
||||
token identity **and** `git config user.name`/`user.email`, coherently. Verified by an
|
||||
exit-asserting test that makes a commit and asserts its author — never assumed from an instruction
|
||||
in a brief.
|
||||
- **RM-50 (roster ownership)** gains per-seat capability declaration plus a **pre-dispatch capability
|
||||
check**. Mos (who owns provisioning) confirms the check is mechanically trivial: **capability is
|
||||
token-file existence.** Before dispatching seat `X` to provider `Y`, test that
|
||||
`~/.config/mosaic/secrets/gitea-tokens/gitea-<Y>-<X>.token` exists; if absent, provision it or pick a
|
||||
provisioned seat. **The token-file set is the authoritative capability registry.** A one-second check
|
||||
would have replaced a mid-task failure that cost ~$9 and 69% of a seat's context.
|
||||
|
||||
### D-10 — the queue guard's failure modes are exactly backwards
|
||||
|
||||
`ci-queue-wait.sh` — a **required** pre-push/pre-merge gate — was observed this session doing both of
|
||||
these:
|
||||
|
||||
- **Fails OPEN on an unknown result.** `state=unknown ⇒ exit 0`, five times, during real pushes and
|
||||
real merges. It also evaluates `branch=main` rather than the branch being acted on.
|
||||
- **Fails CLOSED on credential resolution.** In a worker seat it aborted with
|
||||
`Gitea token not found`, hard-blocking a legitimate push of completed, tested work. The worker
|
||||
correctly stopped (Constitution gate 8). The identical command run from that worker's _own worktree_
|
||||
in another shell succeeded, so the checkout and remote were fine — the difference was the worker's
|
||||
process environment.
|
||||
|
||||
**A gate that waves through work it never checked, and blocks work that is ready, has its failure
|
||||
modes inverted.** Availability failures (cannot reach the provider, cannot resolve a credential)
|
||||
should degrade to a loud, auditable _inability to assert_ — never to a hard stop on delivery, and
|
||||
never to a silent pass. Correctness failures (unknown, malformed, terminal-failure) are what must
|
||||
block.
|
||||
|
||||
This is also the **Pi-brick shape** (P-RECOVERY-001): a gate whose own unavailability prevents the
|
||||
work needed to recover from it.
|
||||
|
||||
**Requirement on RM-03, extending its existing two defects:** the guard must distinguish
|
||||
`CANNOT_ASSERT` (credential/transport/provider unavailable — loud, audited, does not silently pass and
|
||||
does not permanently block) from `ASSERTED_NOT_READY` (a real non-green CI state — blocks). Both are
|
||||
registered R-002 cases with must-fail controls; neither may exit 0 silently.
|
||||
|
||||
### D-9 — the comms path shell-interprets message bodies (injection-shaped, found by accident)
|
||||
|
||||
Sending a status message with `agent-send.sh -m "...`backticks`..."` caused bash to **execute** the
|
||||
backticked text as command substitution. The recipient received a mangled body plus a
|
||||
`No such file or directory` error; the intended sentence never arrived. The message was reported as
|
||||
delivered.
|
||||
|
||||
This is the **same class** as the already-noted `pr-create.sh` backtick-quoting bug (M2 scratchpad):
|
||||
**two tools in the comms path treat a message body as shell input.** A body that can execute on the
|
||||
sender is a _correctness_ bug before it is ever a security one — and note the failure mode: the
|
||||
send reported success while silently transmitting something other than what was written. Silent
|
||||
corruption with a success receipt is precisely the pattern this mission exists to eliminate.
|
||||
|
||||
**Requirement on RM-40 / RM-42 (comms/v1), hardened by Mos.** The envelope must carry its payload
|
||||
**verbatim** and must not be subject to shell interpretation at **any** hop — sender, transport, or
|
||||
adapter. Concretely: **file/stdin transport, never argv interpolation.**
|
||||
|
||||
**Standing interim rule, effective now (Mos).** Until the envelope lands, use `agent-send.sh -f
|
||||
<file>` for any message body containing special characters — **never `-m`**. Passing a file sidesteps
|
||||
argv interpolation entirely. **This rule is mandatory in every worker brief this mission issues**,
|
||||
alongside the D-8 "if a check is unrunnable, say so" clause. Round-trip fidelity (send a body containing backticks, `$(…)`, quotes, and newlines; assert
|
||||
byte-identical receipt) is a required registered test case under RM-02, including a must-fail control
|
||||
proving the assertion can detect corruption.
|
||||
|
||||
### D-8 — a PRE-REGISTERED acceptance check that was not runnable as written
|
||||
|
||||
On PR #1025 the author (me) pre-registered AC2 with the fixture snippet `mkdir -p apps/*/venv/lib`.
|
||||
@@ -297,17 +811,18 @@ spread is itself information, and X1 says we calibrate on real merged PRs.
|
||||
|
||||
### P5 — Retirements, hygiene, conformance
|
||||
|
||||
| id | task | src | depends_on | est (S/O) | tier |
|
||||
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -------------------------- | ---------------- | ------ |
|
||||
| RM-50 | One roster-owned socket/host; quarantine unmanaged; **deterministic reaper for stale sessions AND dead-session disk scratch** (D-7) | O+S+live | RM-04 | 14K / 150K | sonnet |
|
||||
| RM-51 | Auto-sync **allowlist** (never auto-stage unknown paths) + worktree/lease isolation | O+S | RM-02 | 8K / 110K | sonnet |
|
||||
| RM-52 | Retire the Python controller + duplicate MACP islands (3 → 1) | O+S | RM-26, RM-27, RM-25, RM-28 | 14K / 110K | codex |
|
||||
| RM-53 | Flat-file orchestration → DB hard cutover, with rehearsed rollback artifact | O+S | RM-27, RM-30, RM-34, RM-29 | (in S-10) / 200K | opus |
|
||||
| RM-54 | Fleet-wide inert-gate audit against the RM-02 registry | O | RM-02 | — / 120K | sonnet |
|
||||
| RM-55 | **Conformance harness:** fault-inject the live failure classes on real artifacts | O+S | RM-35, RM-41, RM-53 | 18K / 260K | opus |
|
||||
| RM-56 | Retirement proof: CI asserts all three retirements are complete **and stay complete** | O | RM-52, RM-45, RM-53 | — / 90K | codex |
|
||||
| RM-57 | Operator cutover docs + activation proof; map all 15 decisions to evidence | S | RM-04, RM-36, RM-45, RM-55 | 6K / — | codex |
|
||||
| RM-58 | **Mechanical pre-dispatch context reset** — the orchestrator resets a seat out-of-band and verifies it, rather than asking the agent to reset itself | mos-remediation (D-4) | RM-31, RM-50 | 8K | sonnet |
|
||||
| id | task | src | depends_on | est (S/O) | tier |
|
||||
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | -------------------------- | ---------------- | ------ |
|
||||
| RM-50 | One roster-owned socket/host; quarantine unmanaged; **deterministic reaper for stale sessions AND dead-session disk scratch** (D-7) | O+S+live | RM-04 | 14K / 150K | sonnet |
|
||||
| RM-51 | Auto-sync **allowlist** (never auto-stage unknown paths) + worktree/lease isolation | O+S | RM-02 | 8K / 110K | sonnet |
|
||||
| RM-52 | Retire the Python controller + duplicate MACP islands (3 → 1) | O+S | RM-26, RM-27, RM-25, RM-28 | 14K / 110K | codex |
|
||||
| RM-53 | Flat-file orchestration → DB hard cutover, with rehearsed rollback artifact | O+S | RM-27, RM-30, RM-34, RM-29 | (in S-10) / 200K | opus |
|
||||
| RM-54 | Fleet-wide inert-gate audit against the RM-02 registry | O | RM-02 | — / 120K | sonnet |
|
||||
| RM-55 | **Conformance harness:** fault-inject the live failure classes on real artifacts | O+S | RM-35, RM-41, RM-53 | 18K / 260K | opus |
|
||||
| RM-56 | Retirement proof: CI asserts all three retirements are complete **and stay complete** | O | RM-52, RM-45, RM-53 | — / 90K | codex |
|
||||
| RM-57 | Operator cutover docs + activation proof; map all 15 decisions to evidence | S | RM-04, RM-36, RM-45, RM-55 | 6K / — | codex |
|
||||
| RM-59 | **Close the D-19 residual risk** — generated-state verification anchored **outside** the worktree's authority (executor/spine-side attestation), retiring the same-UID self-authentication gap | mos-remediation (D-19) | RM-12, RM-21, RM-25 | 20K | opus |
|
||||
| RM-58 | **Mechanical pre-dispatch context reset** — the orchestrator resets a seat out-of-band and verifies it, rather than asking the agent to reset itself | mos-remediation (D-4) | RM-31, RM-50 | 8K | sonnet |
|
||||
|
||||
**Critical path:** `RM-01 → RM-02 → RM-10 → RM-11 → RM-12 → RM-21 → RM-23 → RM-31 → RM-33 → RM-34 → RM-53 → RM-55`.
|
||||
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# RM-61 — CI contract exemption for #1000 teardown artifact
|
||||
|
||||
**Tracking:** RM-61 / issue #1000
|
||||
|
||||
**Branch:** `fix/rm-61-ci-contract-exemption`
|
||||
**Owner:** `coder-mos1`
|
||||
|
||||
## Objective
|
||||
|
||||
Determine, by red-first provider controls, whether the `ci-postgres` pod-not-found teardown signature discriminates from a real PostgreSQL failure. Only if it discriminates may a named, bounded CI-contract exemption be implemented. The exemption must retire when #1000 is fixed; fixing #1000 is the closure path.
|
||||
|
||||
## Pre-registered kill criterion
|
||||
|
||||
If an injected real `ci-postgres` failure also yields `pods "wp-svc-<ULID>-ci-postgres" not found` as the service's provider-visible failure, the signature does not discriminate. Option B is unsafe; stop exemption implementation and fall to Option A (#1000).
|
||||
|
||||
## Plan
|
||||
|
||||
1. Capture full `-f json` records for the 11 supplied observations and state counts.
|
||||
2. Run one startup-failure control using the real pgvector/PostgreSQL image with an invalid `initdb` argument.
|
||||
3. Run one post-readiness crash control using real PostgreSQL, `pg_isready`, and a deliberate postmaster kill while a DB-dependent probe is active.
|
||||
4. Compare the raw `ci-postgres` service record independently of failures in dependent steps.
|
||||
5. Investigate runner/time/head clustering only as a hypothesis; never encode incidental correlates or retries into policy.
|
||||
6. If and only if the controls discriminate, implement and test the exact exemption, document its two-way boundary, and track retirement at #1000.
|
||||
|
||||
## Budget
|
||||
|
||||
No explicit token cap supplied. Working estimate: 20K–30K tokens. Limit provider controls to the two pre-registered runs; no retries or re-roll policy.
|
||||
|
||||
## Initial evidence
|
||||
|
||||
Historical JSON saved locally under `.evidence/rm-61/` (not for commit). Supplied pipelines: 11 total. Child-step counts: five pipelines with 9 children and six with 10 children. Seven contain the `ci-postgres` pod-not-found failure (#2170, #2175, #2180, #2181, #2182, #2187, #2188); four do not (#2158, #2167, #2184, #2186). Every observed workflow reports `agent_id=44`, so the available JSON does not separate clean and artifact runs by runner. This refutes runner identity as a discriminator in the sampled record.
|
||||
|
||||
## Progress
|
||||
|
||||
- [x] Requirements and kill criterion recorded before control implementation.
|
||||
- [x] Historical full-JSON records captured.
|
||||
- [x] Startup-failure control observed terminal.
|
||||
- [x] Post-readiness crash control observed terminal.
|
||||
- [x] Discrimination verdict recorded: Option B may proceed.
|
||||
- [x] Conditional exemption implementation.
|
||||
|
||||
## Tests / evidence
|
||||
|
||||
### Control 1 — real startup failure
|
||||
|
||||
- Commit: `3931b0e29eb834914f7b17e4db7e221481d436fa`
|
||||
- Pipeline: #2189, exact commit match.
|
||||
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
|
||||
- `ci-postgres`: `state=failure`, `exit_code=1`, `error=null`, with a five-second execution window.
|
||||
- `test`: `state=failure`, `exit_code=1` after the readiness budget expired.
|
||||
- Pipeline/workflow: terminal `failure`.
|
||||
|
||||
This control is red and its service record differs from #1000 (`exit_code=0` plus pod-not-found). It proves the startup-failure direction only. It does not settle the dangerous post-readiness crash/garbage-collection path.
|
||||
|
||||
### Control 2 — real post-readiness crash
|
||||
|
||||
- Commit: `25ac59715a94dd1b52ef42577472eb44ecc4b446`
|
||||
- Pipeline: #2191, exact commit match.
|
||||
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
|
||||
- Service log proves PostgreSQL reached `database system is ready to accept connections`, the test created the arm table, and the service then killed postmaster PID 7.
|
||||
- Test log proves a successful `SELECT 1` followed by `Connection refused`; it exited the pre-registered control code 61.
|
||||
- `ci-postgres`: `state=failure`, `exit_code=137`, `error=null`, with a 203-second execution window.
|
||||
- `test`: `state=failure`, `exit_code=61`.
|
||||
- Pipeline/workflow: terminal `failure`.
|
||||
|
||||
This is the dangerous post-readiness crash path. Its service record is not pod-not-found and therefore differs from #1000 independently of the dependent test failure.
|
||||
|
||||
### Discrimination verdict
|
||||
|
||||
Both real failures are provider-visible as process exits (`exit_code=1` startup; `exit_code=137` crash) with no pod-not-found error. The seven observed #1000 artifacts are provider reconciliation misses (`exit_code=0` plus the exact pod-not-found error). The declared kill criterion did not fire, so Option B may proceed with a matcher requiring the full conjunction. This evidence does **not** prove every future Kubernetes failure is distinguishable; it proves these two concrete real-failure classes remain blocking and bounds the exemption to the observed reconciliation shape.
|
||||
|
||||
### Unit red-first checkpoint
|
||||
|
||||
The nine-case contract harness was written before the verifier. First execution exited 1 because `verify-terminal-green.py` did not exist; no exemption implementation was live. Cases pre-register ordinary green, the exact artifact, both provider controls, near-miss signatures, an independent failure, and a skipped step.
|
||||
|
||||
### Control 2 setup attempt — invalid, excluded from evidence
|
||||
|
||||
- Commit: `9455cd6a2650b2b7e70f746c07933d96e5cb3d20`
|
||||
- Pipeline: #2190, exact commit match.
|
||||
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
|
||||
- Service log: `/bin/sh: 0: -c requires an argument`.
|
||||
- Root cause: Woodpecker service `commands` did not become the third `sh -c` argument. PostgreSQL never started, so this run is **not** the post-readiness crash control and provides no discrimination evidence.
|
||||
- Focused remediation: place the script directly in the third `entrypoint` element and supply `PGPASSWORD` for the marker query. This is a control-fixture correction, not a retry of #1000 and not evidence for either verdict.
|
||||
|
||||
## Implementation evidence
|
||||
|
||||
- `verify-terminal-green.py` consumes only the full JSON/API record; it performs no fetch, retry, or trigger.
|
||||
- Exact #2188 record: exit 0, 10 children, 9 success + 1 named exemption.
|
||||
- Historical set: #2158/#2167/#2184/#2186 pass with no exemption; #2170/#2175/#2182/#2187/#2188 pass with one named exemption; #2180/#2181 remain red because independent failures exist.
|
||||
- Provider controls: #2189 and #2191 both exit 1 under the verifier; neither is exempted.
|
||||
- Unit harness: initial 9/9 cases passed after the red-first checkpoint; review remediation expands this to 12 cases with expected-head match/missing/mismatch coverage.
|
||||
- Test-membership guard: PASS, population 45; 26 enumerated, 19 signed exclusions; all 39 surface paths present.
|
||||
- Python compile: PASS.
|
||||
- `pnpm typecheck`: PASS, 45/45 tasks.
|
||||
- `pnpm lint`: PASS, 25/25 tasks.
|
||||
- `pnpm format:check`: PASS after moving local evidence outside the repository tree.
|
||||
- `test:framework-shell`: RM-61 and all preceding suites passed, then the pre-existing wake assertion aborted with exit 97 because this host's Bash 5.2.15 reports `BASH_LINENO [3 5]` where that suite requires `[3 4]`. RM-61 does not modify the wake suite; the command is not fully runnable on this host as written and no substitute result is claimed.
|
||||
|
||||
## Independent review
|
||||
|
||||
- Review 67 / comment 20403 at exact head `e7b29219e11efd0a19395156ac0b154bec0c3a73`: **REQUEST CHANGES**.
|
||||
- Blocker: the verifier echoed the pipeline commit but did not bind it to the current PR head; mutating only #2188's commit still returned terminal-green.
|
||||
- Remediation: require `--expect-commit <full-40>`, add a pipeline anomaly on missing/mismatched record commits, emit expected and observed values, wire both CI documentation and the merge-gate baseline to pass provider PR head, and add match/missing/mismatch tests.
|
||||
- This binding is not prohibited head-based clustering policy: it proves the evidence belongs to the commit under verdict. Runner/node/time/head correlation remains excluded from the teardown signature itself.
|
||||
- Review 69 later approved the commit-binding remediation at exact head `033b2ffb46674b2c0bcc5197273c109b461f62d9`; pipeline #2193 was 9/9 success. Before merge-gate, an independent adjudicator found that Python treats JSON `false == 0`, allowing a non-integer exit value to match. The prior gate-ready state was withdrawn. The type-strict set distinguishes genuine red-first controls (`false`, `0.0`, which wrongly exempted) from regression guards (`true`, `"0"`, `null`, which already blocked). Remediation requires the decoded type to be exactly `int` and excludes `bool` explicitly.
|
||||
|
||||
## Documentation checklist
|
||||
|
||||
- [x] CI contract documented in the canonical framework CI/CD guide.
|
||||
- [x] Operator command documented in the Woodpecker tool README.
|
||||
- [x] Merge-gate baseline points to the deterministic verifier and named retirement.
|
||||
- [x] Tracking and retirement cite issue #1000.
|
||||
- [x] Both positive and negative guarantee boundaries are stated.
|
||||
- [x] No API/auth/schema/user-facing navigation change; OpenAPI, user guide, and sitemap are not applicable.
|
||||
|
||||
## Risks
|
||||
|
||||
The controls establish discrimination for deterministic startup failure and an armed post-readiness postmaster crash on the current Woodpecker Kubernetes provider. They cannot prove that every future Kubernetes failure mode will preserve a non-zero exit before reconciliation. The exact matcher minimizes that residual risk, and issue #1000 remains the mandatory provider-seam closure and retirement trigger.
|
||||
@@ -0,0 +1,58 @@
|
||||
# RM-01 — Reproducible checkout
|
||||
|
||||
- Task/ref: RM-01 (`docs/remediation/TASKS.md`, internal mission tracking)
|
||||
- Objective: make checkout/install/typecheck hooks fail on code rather than environmental residue, for root CI and non-root seats.
|
||||
- Scope: pnpm store configuration, transactional Husky installation, dependency/generated-state preflight, checkout regression tests, developer documentation.
|
||||
- Constraints: isolated worktree; no skip-switch fixes; no writes under `/root` or `/tmp`; workers do not edit `docs/remediation/TASKS.md`; author does not review or merge.
|
||||
- Acceptance: AC1–AC8 from the orchestrator dispatch/addendum.
|
||||
- Plan:
|
||||
1. Add RED-first tests for missing dependencies, stale/foreign `.next`, and interrupted hook installation.
|
||||
2. Implement environment-overridable HOME-based pnpm store defaults, deterministic preflight, and transactional hook installation.
|
||||
3. Run focused tests, install/build/baseline gates, and explicit AC negative controls.
|
||||
4. Obtain independent review, push after queue guard, open PR, and send evidence to `mos-remediation`.
|
||||
- Budget: orchestrator estimate 6K/60K; no explicit hard token cap. Keep scope to RM-01 and avoid unrelated cleanup.
|
||||
- Risks: 97%-full shared `/tmp`; native dependency install size; root-owned fixtures may require Docker for realistic verification.
|
||||
|
||||
## Progress / evidence
|
||||
|
||||
- Worktree created at `/home/hermes/agent-work/rm-01` from `origin/main` `06e0d403`.
|
||||
- `/tmp` baseline: 28G used, 889M available (97%); worktree and planned store are on `/home`.
|
||||
- Root causes confirmed from source: committed `.npmrc` pins `/root`; `prepare` invokes Husky directly; web typecheck includes generated `.next` types without validating ownership/freshness.
|
||||
|
||||
## Checkpoint evidence (c45e5e19)
|
||||
|
||||
- AC1 IN PROGRESS: non-root `pnpm install --frozen-lockfile --store-dir "$HOME/.local/share/pnpm/store"` exited 0; `pnpm exec turbo run typecheck --force` exited 0 (45/45 uncached). Clean CI-container run not performed.
|
||||
- AC2 DONE: with `node_modules` absent, `pnpm preflight` exited 42 with `MOSAIC_PREFLIGHT_MISSING_DEPS` and `run pnpm install`; after install it exited 0.
|
||||
- AC3 DONE: appending `export const x: number = "s"` to `packages/types/src/index.ts` made `pnpm -w typecheck` exit 2 with TS2322; reverting made it exit 0.
|
||||
- AC4 IN PROGRESS: local `pnpm -w build` exited 0 and `git status --porcelain` showed no generated residue beyond the intended RM-01 source changes. Fresh-clone proof not performed.
|
||||
- AC5 DONE: non-root install exited 0; `pnpm store path` resolved `/home/hermes/.local/share/pnpm/store/v10`; no `/root` write was attempted.
|
||||
- AC6 IN PROGRESS: focused failure/rollback tests passed, but final review found a concurrent-install race. Two installers can both observe `.husky/_` absent; after one installs successfully, the losing install's catch path can quarantine the winner's active hooks and restore stale Git config (`scripts/install-hooks.mjs`, activation/catch transaction). A RED regression is committed after the checkpoint.
|
||||
- AC7 DONE: install/store/worktree were on `/home`; full `pnpm -w build` exited 0; `/tmp` usage changed by 4096 bytes during the build (23,805,173,760 → 23,805,177,856 bytes), not materially.
|
||||
- AC8 DONE for the implemented path: store resolves under `$HOME`; test/quarantine/build state resolves under the worktree; no implemented component requires a writable path outside `$HOME` or the worktree.
|
||||
|
||||
## Continuation evidence
|
||||
|
||||
- AC6 DONE: the committed race reproducer was observed RED (`node --test --test-name-pattern='a competing successful installer is not removed by the losing process' scripts/install-hooks.test.mjs`, exit 1/ENOENT), then passed after cleanup became ownership-safe. The losing installer never removes an active hook set or restores Git configuration it did not activate. `pnpm test:checkout` passes 21/21, exit 0, including the original race and a post-rename peer-replacement regression.
|
||||
- Generated-state remediation: replaced mtime inference with a source/build-input fingerprint, written only after a serialized successful Next build with unchanged inputs. Failed/interrupted/overlapping builds leave no trusted marker. The fingerprint uses Next's own environment loader, covers resolved `NEXT_PUBLIC_*` values, inherited TypeScript configuration, lock/workspace inputs, and rejects symlink inputs.
|
||||
- Baseline: `pnpm typecheck`, `pnpm lint`, and `pnpm format:check` each exit 0. Local `pnpm test` still exits 97 only at the pre-existing Bash `BASH_LINENO` convention guard (#973/#1003), after checkout tests and package tests pass; this is not reported as a green full-suite result.
|
||||
- Automated review remediation: resolved findings for peer-hook ownership, stale/failed build markers, build-input changes, expanded environment inputs, inherited TypeScript config, symlink inputs, and overlapping build serialization. Independent PR review remains assigned to rev-974.
|
||||
- AC1 DONE at `0f706119`: a clean clone created inside `git.mosaicstack.dev/mosaicstack/stack/ci-base:latest` ran the exact acceptance sequence `pnpm install --frozen-lockfile && pnpm -w typecheck`; exit 0 with 45/45 uncached typecheck tasks successful. An earlier bind-mounted clone attempt exited 1 because root in the container rejected the host-owned Git directory; that failed attempt is not counted as evidence.
|
||||
- AC4 DONE at `0f706119`: in that same fresh clone and CI image, `pnpm -w build` completed 25/25 tasks and the immediately following `git status --porcelain` was empty; combined assertion exit 0.
|
||||
- Push BLOCKED after the required queue guard: `git push origin fix/rm-01-reproducible-checkout` was rejected by Gitea with `User permission denied for writing` / `pre-receive hook declined`, despite `MOSAIC_GIT_IDENTITY=f10-coder` resolving username `f10-coder` from the provisioned `gitea-mosaicstack-f10-coder.token`.
|
||||
|
||||
## Review remediation — restated AC2
|
||||
|
||||
- Independent review correctly found that an added symlink under a successfully built `.next` tree passed preflight. The exact reviewer control, `ln -s /etc/hosts apps/web/.next/reviewer-symlink && pnpm preflight`, was observed passing before remediation.
|
||||
- The original blanket symlink wording conflicts with AC4 because canonical Next `output: 'standalone'` emits legitimate pnpm dependency symlinks. The coordinator independently verified 42 such links and approved the operative restatement: `.next` itself must not be a symlink; descendant symlinks must exactly match the successful build's certified manifest.
|
||||
- RED-first controls were observed failing together against the prior implementation (exit 1): `.next` root, added, removed, retargeted, tampered-manifest, and canonical-style certified-link cases. The build now publishes the manifest atomically before the existing source certification commit marker; that marker binds the manifest SHA-256. Missing/partial/modified manifests remain untrusted.
|
||||
- GREEN evidence: the six-case symlink control passes; the exact reviewer-added link exits 43; removing it restores preflight exit 0. The added RED-first build-publication control also proves a symlinked `.next` cannot redirect certification writes outside the checkout. `pnpm test:checkout` passes 23 top-level tests / 29 including subtests. Canonical `pnpm --filter @mosaicstack/web build` and the following `pnpm preflight` both exit 0.
|
||||
- Threat-model ruling: the manifest detects accidental, independent, stale, and foreign-residue mutation—the class exposed by the five-month-stale `.next` that produced 19 phantom TS2307 errors. It does not defend against a same-UID actor able to rewrite both manifest and marker consistently (CWE-345); no local worktree construction can without an external trust anchor. RM-59 tracks the residual: executor/spine-side attestation outside worktree authority, dependent on RM-12, RM-21, and RM-25.
|
||||
- AC8 concrete proof at `df7530ae`: a clean clone ran in `ci-base:latest` with Docker `--read-only`; its only writable mounts were `/workspace` (the worktree) and `/home/ci` (`HOME`, with `NPM_CONFIG_STORE_DIR=/home/ci/store`). `pnpm install --frozen-lockfile && pnpm -w typecheck` exited 0 with 45/45 uncached tasks. This proves the implemented checkout path requires no writable location outside `$HOME` and the worktree. An initial fixture attempt failed only because Git required `/workspace` safe-directory setup; it is not counted as evidence.
|
||||
|
||||
## Handoff
|
||||
|
||||
1. Keep the newly committed RED tests red until implementing: (a) source-fingerprint marker support for valid incremental `.next` output, and (b) ownership-safe concurrent hook activation.
|
||||
2. The latest automated review rejected oldest-generated-file mtime as a false positive for valid incremental Next output. Use a source-content fingerprint marker written only after successful `next build`; do not continue tuning mtimes.
|
||||
3. For Husky, generation in an isolated temporary Git repo avoids mutating real `core.hooksPath` during staging. Preserve that design. Fix the losing concurrent process so it never removes a peer's completed hook set or restores stale config.
|
||||
4. Codex review runs in a read-only sandbox, so its attempts to run the fixture-writing Node tests report opaque test-file failures. The same tests run normally in the worktree.
|
||||
5. Full `pnpm test` is not green on this host: it exits 97 at the pre-existing Bash `BASH_LINENO` convention guard (#1003), after the changed checkout tests and package tests pass. Do not weaken that gate.
|
||||
@@ -0,0 +1,120 @@
|
||||
# RM-03 — CI Queue Guard Repair
|
||||
|
||||
- **Task:** RM-03
|
||||
- **Issue:** #1019
|
||||
- **Branch:** `fix/rm-03-queue-guard`
|
||||
- **Owner:** coder-mos1
|
||||
- **Reviewer:** rev-974 (independent; author != reviewer)
|
||||
- **Started:** 2026-08-01
|
||||
|
||||
## Objective
|
||||
|
||||
Repair the mandatory CI queue guard so it reads provider payloads, blocks asserted non-green CI, distinguishes provider unavailability from a real non-green result, and inspects the branch actually being pushed or merged.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Worktree only: `/home/hermes/agent-work/rm-03`; never mutate `/src/mosaic-stack`.
|
||||
- JSON payload travels through stdin; never argv. Large payload must remain below no ARG_MAX dependency.
|
||||
- TDD is mandatory. Every behavior case must be observed red before implementation.
|
||||
- No bypass flags or hook suppression.
|
||||
- Do not cite the existing guard's green as evidence; D-23 establishes it is zero-information.
|
||||
- Gate-ready is a frozen exact head. Any push after a merge-gate verdict voids that verdict.
|
||||
- No merge: coordinator holds the merge hand pending Jason.
|
||||
|
||||
## Design
|
||||
|
||||
1. Feed JSON to `python3 -c` on stdin, including pending-context rendering.
|
||||
2. Classify valid green as `READY`; pending/failure/no-status/malformed/mixed as `ASSERTED_NOT_READY`; provider/credential/transport inability as `CANNOT_ASSERT`.
|
||||
3. `ASSERTED_NOT_READY` exits nonzero. `CANNOT_ASSERT` emits a loud diagnostic and appends a local JSONL audit record. Push degrades to exit 0; merge holds with distinct retryable exit 75 until provider recovery, then self-clears without manual reset. Inability to write the audit exits nonzero.
|
||||
4. Derive the current branch when `-B` is omitted. The merge wrapper passes the exact PR head branch, repository, and full commit SHA—not its `main` base—so fork PRs cannot resolve against an adjacent base-repository branch.
|
||||
|
||||
## Test matrix
|
||||
|
||||
| Case | Required outcome |
|
||||
| --- | --- |
|
||||
| success | exit 0; terminal-success |
|
||||
| pending | nonzero after bounded timeout |
|
||||
| failure | nonzero |
|
||||
| no-status | nonzero |
|
||||
| malformed | nonzero |
|
||||
| >=150 KiB payload | unchanged classification; never rc126 |
|
||||
| provider unreachable on push | loud audited CANNOT_ASSERT; degraded exit 0 |
|
||||
| provider unreachable on merge | loud audited CANNOT_ASSERT; retryable exit 75/HOLD |
|
||||
| audit unavailable | nonzero |
|
||||
| implicit push branch | provider URL uses checked-out feature branch |
|
||||
| merge wrapper | queue guard receives exact PR head branch/repository/full SHA |
|
||||
|
||||
## RED-first evidence
|
||||
|
||||
Observed against the unmodified `origin/main` implementation before source edits:
|
||||
|
||||
- `bash packages/mosaic/framework/tools/git/test-ci-queue-wait-tristate.sh` → rc 1 with 15 failed assertions.
|
||||
- Success payload was reported `state=unknown`.
|
||||
- Pending, failure, no-status, and malformed payloads each exited 0 and omitted `ASSERTED_NOT_READY`.
|
||||
- The 160 KiB payload produced rc 141 because Python never consumed the pipe; it did not classify success.
|
||||
- Provider-unreachable exited 7 with no `CANNOT_ASSERT` audit record.
|
||||
- Implicit push queried `/branches/main`, not `/branches/fix/rm-03-fixture`.
|
||||
- Audit-unavailable emitted no audit diagnostic.
|
||||
- A credential-resolution hard-block mutant was then run before trusting that added case: `credential-unresolvable` returned rc 1 and omitted `CANNOT_ASSERT`; the matrix returned rc 1 with two named assertion failures.
|
||||
- Review-blocker controls were observed red: structurally invalid `statuses` string and null-entry payloads each exited 0 as `terminal-success`; unsupported-platform discovery exited 1 without diagnostic or audit (seven named assertion failures total).
|
||||
- After the push/merge asymmetry ruling, merge-side provider unavailability was observed red at rc 0; its registered case required distinct retryable rc 75.
|
||||
- Aggregate `state=success` with zero contexts was observed red: it exited 0 as `terminal-success`; the registered case requires `no-status`/nonzero.
|
||||
- Fork/exact-head controls were observed red: `pr-merge.sh` omitted the fork repository and full SHA, and an ignored-arguments mutant re-resolved through `/branches/` instead of the exact fork commit (two named failures).
|
||||
- GitHub check-run-only success/pending/failure were each misclassified as `no-status`; the RED run had five named failures and proved the Checks API was never queried.
|
||||
- The first merge-pin control was unrunnable because one `local` declaration referenced a variable before assignment under `set -u`; this was disclosed and corrected rather than counted. The runnable RED then showed Gitea payload `{"Do":"squash"}` lacked `head_commit_id`; a separate GitHub run showed `gh pr merge 123 --squash` lacked `--match-head-commit`.
|
||||
- A stale-verdict mutant removed the `--expect-head` comparison and was observed red because a moved head reached the provider merge call.
|
||||
- `bash packages/mosaic/framework/tools/git/test-pr-merge-queue-branch.sh` initially returned rc 1; captured call was `--purpose merge -B main -t 900 -i 15`.
|
||||
|
||||
Logs remain untracked under the worktree as `.mosaic-test-work-red-*.log` and will not be committed.
|
||||
|
||||
## Progress
|
||||
|
||||
- [x] Mission, remediation charter, task evidence, board, issue #1019, and superseded PR #1023 read.
|
||||
- [x] Isolated worktree created and identity configured coherently.
|
||||
- [x] Mutant tests authored and observed red.
|
||||
- [x] Implementation green.
|
||||
- [x] Baseline and focused situational gates green; full package suite has an unrelated framework-shell environment abort recorded below.
|
||||
- [ ] Independent review clean (rev-974 requested changes at `44ffa99a`; bypass remediation committed and awaiting re-review).
|
||||
- [ ] PR CI terminal-green at exact head by full step scan.
|
||||
- [ ] Merge-gate verdict issued against frozen head.
|
||||
|
||||
## Scope disposition
|
||||
|
||||
- The five framework guides are consequential documentation: they define the purpose-aware tri-state contract, including audited push degradation and merge HOLD.
|
||||
- The agent templates are consequential because they ship the same queue-guard instructions into newly seeded agent contracts; leaving them binary/stale would contradict the repaired tool.
|
||||
- `pr-merge.sh` is consequential: it must inspect the PR's exact head branch/repository/SHA and enforce the exact-head merge pin.
|
||||
- `pr-metadata.sh` is consequential only as the normalized source of that head branch/repository/SHA. Its diff is limited to exposing those fields on GitHub and Gitea.
|
||||
- `test-pr-merge-gitea-empty-uid.sh` changes because exact-head Gitea merges now always use the API path (the only path that can send `head_commit_id`), superseding the prior tea-empty-identity fallback behavior.
|
||||
|
||||
## Review remediation
|
||||
|
||||
- rev-974 independently proved that the documented `--skip-queue-guard` merge option bypassed an exit-99 guard stub, reached the provider merge payload, printed success, and exited 0 at head `44ffa99a`.
|
||||
- RED-first reproduction was added to `test-pr-merge-head-pin.sh` before the production fix: `FAIL merge-bypass: --skip-queue-guard reached the provider merge path`, suite rc 1. The test-only commit is `241113e6`.
|
||||
- Production remediation `37aae650` removes the option from parsing, usage, help, and examples. Every merge-capable path now invokes the queue guard; `--dry-run` alone omits it and has a regression proving that it exits before provider dispatch and creates no merge payload.
|
||||
- Existing Gitea merge tests now exercise a successful guard response rather than bypassing the guard.
|
||||
|
||||
## Risks / boundaries
|
||||
|
||||
- The local JSONL audit is durable operational evidence but not tamper-resistant against the same UID. RM-03 does not claim otherwise.
|
||||
- Push-side audited exit 0 is an explicit owner ruling (Option B), accepted to avoid bricking recovery work; merge-side CANNOT_ASSERT remains retryable exit 75/HOLD. The automated security reviewer continues to flag the deliberate push availability tradeoff.
|
||||
- Source/deployed-copy equality is owned by RM-02/D-22; this branch changes repository source and its tests only.
|
||||
|
||||
## Test evidence
|
||||
|
||||
Fresh after rescue checkpoint `b7175012`:
|
||||
|
||||
- Focused situational matrix: tri-state, GitHub checks pagination, branch-absent, merge head branch/repository/SHA, exact-head pin, and Gitea exact-head API regressions all passed.
|
||||
- `bash -n` on the three production shell scripts passed.
|
||||
- `shellcheck -x -P packages/mosaic/framework/tools/git ...` on all changed shell scripts passed.
|
||||
- `pnpm typecheck` passed (45/45 Turbo tasks).
|
||||
- `pnpm lint` passed (25/25 Turbo tasks).
|
||||
- `pnpm format:check` passed.
|
||||
- `pnpm --filter @mosaicstack/mosaic test`: Vitest passed 1508/1508 on the confirmation run; framework-shell then aborted at the pre-existing wake coordinate assertion with exit 97: `BASH_LINENO ... probe reported [3 5], expected [3 4] ... (#973)`. This is outside the RM-03 diff and is disclosed rather than substituted or called green.
|
||||
- The prior package-suite attempt had one transient, out-of-diff `install-ordering-guard.spec.ts` failure (1/1508); its isolated rerun passed 19/19 and the confirmation full Vitest run passed 1508/1508.
|
||||
- After bypass remediation: all six focused RM-03 queue/merge regressions passed, including bypass refusal and dry-run non-dispatch; shell syntax and source-aware ShellCheck passed; `pnpm typecheck`, `pnpm lint`, and `pnpm format:check` passed.
|
||||
- Fresh `test:framework-shell` reached and passed every RM-03 test, then again aborted at the unrelated wake coordinate assertion with exit 97; it remains explicitly non-green rather than substituted.
|
||||
- An ad hoc raw Prettier invocation over `.template` and `.sh` files was unrunnable because no parser is registered for those extensions; it was not used as a substitute for canonical `pnpm format:check`.
|
||||
|
||||
## Final evidence
|
||||
|
||||
Pending.
|
||||
+6
-3
@@ -6,11 +6,14 @@
|
||||
"build": "turbo run build",
|
||||
"dev": "turbo run dev",
|
||||
"lint": "turbo run lint",
|
||||
"typecheck": "turbo run typecheck",
|
||||
"test": "turbo run test",
|
||||
"preflight": "node scripts/preflight.mjs",
|
||||
"clean:generated": "node scripts/clean-generated.mjs",
|
||||
"typecheck": "pnpm preflight && turbo run typecheck",
|
||||
"test:checkout": "node --test scripts/*.test.mjs",
|
||||
"test": "pnpm test:checkout && turbo run test",
|
||||
"format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,md}\"",
|
||||
"format:check": "prettier --check \"**/*.{ts,tsx,js,jsx,json,md}\"",
|
||||
"prepare": "husky"
|
||||
"prepare": "node scripts/install-hooks.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@typescript-eslint/eslint-plugin": "^8.0.0",
|
||||
|
||||
@@ -13,7 +13,14 @@ It is a **gate** role: the one and only merge path.
|
||||
2. **Use the wrapped scripts as the ONLY merge path** — the merge-gate merges
|
||||
**exclusively** by calling **`pr-merge.sh`** (the merge action, which carries the
|
||||
authoritative forbidden-path guard) and **`pr-ci-wait.sh`** (to wait for green
|
||||
CI before merging). These two scripts are the _only_ sanctioned merge path.
|
||||
CI before merging). Before issuing a verdict, scan the full JSON/API child-step
|
||||
record (including `clone`) with **`verify-terminal-green.py --expect-commit
|
||||
<current-provider-PR-head>`** and record the equal expected/observed full-40
|
||||
commits, exact step count, anomalies, and named exemptions. Missing or mismatched
|
||||
commit binding is a hard refusal. The verifier's sole interim
|
||||
exemption is `WP-K8S-1000-CI-POSTGRES-TEARDOWN`; it is signature-scoped, tracked
|
||||
by #1000, and retires when #1000 is fixed. These scripts are the _only_
|
||||
sanctioned merge path.
|
||||
3. **Never call the raw API** — the merge-gate **does NOT** call `tea`, the raw
|
||||
Gitea/forge HTTP API, or any other merge mechanism directly. Only `pr-merge.sh`
|
||||
and `pr-ci-wait.sh`.
|
||||
|
||||
@@ -868,6 +868,38 @@ steps:
|
||||
7. **Test on a short-lived non-main branch first** — open a PR and verify quality gates before merging to `main`
|
||||
8. **Verify images appear** in Gitea Packages tab after successful pipeline
|
||||
|
||||
## Terminal-Green Full-Step Contract
|
||||
|
||||
A successful pipeline summary is not sufficient: verification MUST consume the full JSON/API child-step record, including `clone`.
|
||||
|
||||
```bash
|
||||
PR_HEAD=<full-40-hex-provider-head>
|
||||
~/.config/mosaic/tools/woodpecker/pipeline-status.sh \
|
||||
-r mosaicstack/stack -n <pipeline-number> -f json \
|
||||
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py \
|
||||
--expect-commit "$PR_HEAD" -
|
||||
```
|
||||
|
||||
`PR_HEAD` MUST come from the current provider PR metadata and MUST be the full 40-hex head, not a local branch guess. The verifier fails if the argument is missing, malformed, absent from the pipeline record, or differs from that record.
|
||||
|
||||
The verifier reports the expected and observed commits, total step count, state counts, anomalies, and any applied exemption. Exit `0` means the record satisfies the contract; exit `1` means the commit binding or at least one pipeline, workflow, or child-step state blocks terminal-green; exit `2` means the invocation or JSON input could not be verified.
|
||||
|
||||
### Named interim exemption: `WP-K8S-1000-CI-POSTGRES-TEARDOWN`
|
||||
|
||||
Only this exact conjunction is exempted:
|
||||
|
||||
- pipeline and workflow state are `success`;
|
||||
- exactly one non-success child exists;
|
||||
- its name is `ci-postgres` and type is `service`;
|
||||
- its state is `failure`, exit code is the JSON integer `0` (not boolean, float, string, or null); and
|
||||
- its error exactly matches `pods "wp-svc-<ULID>-ci-postgres" not found`.
|
||||
|
||||
Every near miss remains blocking, including non-zero service exits, startup failures, post-readiness crashes, connection errors, image-pull errors, skipped steps, another failed child, malformed pod names, duplicate matches, or a non-success pipeline/workflow.
|
||||
|
||||
**Boundary in both directions:** this exemption recognizes the observed Woodpecker Kubernetes reconciliation miss after an otherwise-successful run. It does not prove that every future PostgreSQL or Kubernetes failure is distinguishable. It does prove, through provider controls, that a deterministic startup failure (`exit_code=1`) and an armed post-readiness postmaster crash (`exit_code=137`, dependent probe `Connection refused`) do not match and remain red.
|
||||
|
||||
**Tracking and retirement:** [mosaicstack/stack#1000](https://git.mosaicstack.dev/mosaicstack/stack/issues/1000) owns the provider-seam fix. This exemption MUST be removed when #1000 is fixed. It is not authority to retry or re-trigger a pipeline, and no per-PR re-roll is part of the contract.
|
||||
|
||||
## Post-Merge CI Monitoring (Hard Rule)
|
||||
|
||||
For source-code delivery, completion is not allowed at "PR opened" stage.
|
||||
@@ -893,14 +925,16 @@ Woodpecker note:
|
||||
Before pushing a branch or merging a PR, guard against overlapping project pipelines:
|
||||
|
||||
```bash
|
||||
~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main
|
||||
~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main
|
||||
~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push
|
||||
~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>
|
||||
```
|
||||
|
||||
Behavior:
|
||||
|
||||
- If pipeline state is running/queued/pending, wait until queue clears.
|
||||
- If timeout or API/auth failure occurs, treat as `blocked`, report exact failed wrapper command, and stop.
|
||||
- If pipeline state is running/queued/pending, wait until queue clears; timeout is `ASSERTED_NOT_READY` and exits nonzero.
|
||||
- Failure, missing status, malformed status, or any other provider-asserted non-green state is `ASSERTED_NOT_READY` and exits nonzero.
|
||||
- Credential, transport, or provider unavailability is `CANNOT_ASSERT`: the guard emits a loud diagnostic and durable JSONL audit record. For push it exits 0 so recovery work is not bricked. For merge it returns distinct retryable exit 75 and holds until provider recovery; rerunning then self-clears without manual reset. This result is never evidence that CI was clear. If the audit cannot be written, the guard exits nonzero.
|
||||
- `pr-merge.sh` resolves and guards the exact PR head repository and full SHA automatically, including fork PRs.
|
||||
|
||||
## Gitea as Unified Platform
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ Merge strategy enforcement (HARD RULE):
|
||||
- PR target for delivery is `main`.
|
||||
- Direct pushes to `main` are prohibited.
|
||||
- Merge to `main` MUST be squash-only.
|
||||
- Use `~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash` (or PowerShell equivalent).
|
||||
- Use `~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash --expect-head {approved_full_sha}` (or PowerShell equivalent).
|
||||
|
||||
## Review Checklist
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ For implementation work, you MUST run this cycle in order:
|
||||
8. `pre-push queue guard` - before pushing, wait for running/queued project pipelines to clear: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. `push` - push immediately after queue guard passes.
|
||||
10. `PR integration` - if external git provider is available, create/update PR to `main` and merge with required strategy via Mosaic wrappers.
|
||||
11. `pre-merge queue guard` - before merging PR, wait for running/queued project pipelines to clear: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge`.
|
||||
11. `pre-merge queue guard` - before merging PR, wait for running/queued project pipelines on the exact PR head to clear: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
12. `CI/pipeline verification` - wait for terminal CI status and require green before completion (`~/.config/mosaic/tools/git/pr-ci-wait.sh` for PR-based workflow).
|
||||
13. `issue closure` - close linked external issue (or close internal `docs/TASKS.md` task ref when provider is unavailable).
|
||||
14. `greenfield situational test` - validate required user flows in a clean environment/startup path (post-merge for trunk workflow changes).
|
||||
@@ -93,8 +93,8 @@ For implementation work, you MUST run this cycle in order:
|
||||
> the gate (AGENTS.md hard gate "Merge authority"). Solo delivery proceeds
|
||||
> without asking.
|
||||
|
||||
1. `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`
|
||||
2. `~/.config/mosaic/tools/git/pr-merge.sh -n <PR_NUMBER> -m squash`
|
||||
1. `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`
|
||||
2. `~/.config/mosaic/tools/git/pr-merge.sh -n <PR_NUMBER> -m squash --expect-head <APPROVED_FULL_SHA>`
|
||||
3. `~/.config/mosaic/tools/git/pr-ci-wait.sh -n <PR_NUMBER>`
|
||||
4. `~/.config/mosaic/tools/git/issue-close.sh -i <ISSUE_NUMBER>` (or close internal `docs/TASKS.md` ref when no provider exists)
|
||||
5. If any step fails: set status `blocked`, report the exact failed wrapper command, and stop.
|
||||
|
||||
@@ -425,11 +425,11 @@ git push
|
||||
and checklist completed (`~/.config/mosaic/templates/docs/DOCUMENTATION-CHECKLIST.md`) when applicable.
|
||||
13. **PR + CI + Issue Closure Gate** (HARD RULE for source-code tasks):
|
||||
- Before merging, run queue guard:
|
||||
`~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`
|
||||
`~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`
|
||||
- Ensure PR exists for the task branch (create/update via wrappers if needed):
|
||||
`~/.config/mosaic/tools/git/pr-create.sh ... -B main`
|
||||
- Merge via wrapper:
|
||||
`~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash`
|
||||
`~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash --expect-head {approved_full_sha}`
|
||||
- Wait for terminal CI status:
|
||||
`~/.config/mosaic/tools/git/pr-ci-wait.sh -n {PR_NUMBER}`
|
||||
- Close linked issue after merge + green CI:
|
||||
@@ -630,7 +630,7 @@ Construct this from the task row and pass to worker via Task tool:
|
||||
|
||||
**MANDATORY:** This ALWAYS includes linting. If the project has a linter configured
|
||||
(ESLint, Biome, ruff, etc.), you MUST run it and fix ALL violations in files you touched.
|
||||
Do NOT leave lint warnings or errors for someone else to clean up. 6. Run REQUIRED situational tests based on changed surfaces (see `~/.config/mosaic/guides/E2E-DELIVERY.md` and `~/.config/mosaic/guides/QA-TESTING.md`). 7. If task is bug fix/security/auth/critical business logic, apply REQUIRED TDD discipline per `~/.config/mosaic/guides/QA-TESTING.md`. 8. If gates or required situational tests fail: Fix and retry. Do NOT report success with failures. 9. Commit: `git commit -m "fix({finding_id}): brief description"` 10. Before push, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main` 11. Push: `git push origin {branch}` 12. Report result as JSON (see format below)
|
||||
Do NOT leave lint warnings or errors for someone else to clean up. 6. Run REQUIRED situational tests based on changed surfaces (see `~/.config/mosaic/guides/E2E-DELIVERY.md` and `~/.config/mosaic/guides/QA-TESTING.md`). 7. If task is bug fix/security/auth/critical business logic, apply REQUIRED TDD discipline per `~/.config/mosaic/guides/QA-TESTING.md`. 8. If gates or required situational tests fail: Fix and retry. Do NOT report success with failures. 9. Commit: `git commit -m "fix({finding_id}): brief description"` 10. Before push, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B {branch}` 11. Push: `git push origin {branch}` 12. Report result as JSON (see format below)
|
||||
|
||||
## Git Scripts
|
||||
|
||||
@@ -638,8 +638,9 @@ For issue/PR/milestone operations, use scripts (NOT raw tea/gh):
|
||||
|
||||
- `~/.config/mosaic/tools/git/issue-view.sh -i {N}`
|
||||
- `~/.config/mosaic/tools/git/pr-create.sh -t "Title" -b "Desc" -B main`
|
||||
- `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`
|
||||
- `~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash`
|
||||
- Push: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B {task_branch}`
|
||||
- Merge: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B {pr_head_branch} -R {pr_head_owner/repo} --sha {pr_head_full_sha}`
|
||||
- `~/.config/mosaic/tools/git/pr-merge.sh -n {PR_NUMBER} -m squash --expect-head {approved_full_sha}`
|
||||
- `~/.config/mosaic/tools/git/pr-ci-wait.sh -n {PR_NUMBER}`
|
||||
- `~/.config/mosaic/tools/git/issue-close.sh -i {N}`
|
||||
|
||||
|
||||
@@ -23,10 +23,12 @@ Mosaic wrappers at `~/.config/mosaic/tools/git/*.sh` handle platform detection a
|
||||
# Milestones
|
||||
~/.config/mosaic/tools/git/milestone-create.sh
|
||||
|
||||
# CI queue guard (required before push/merge)
|
||||
# CI queue guard (required before push/merge; defaults to the checked-out branch)
|
||||
~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge
|
||||
```
|
||||
|
||||
The guard exits nonzero for any provider-asserted non-green, missing, or malformed CI state. If credentials or the provider are unavailable, it emits `CANNOT_ASSERT` and writes a JSONL audit record. Push degrades to exit 0 so recovery work is not bricked; merge holds with retryable exit 75 until the provider recovers, then self-clears without manual reset. Neither outcome is evidence that CI was clear. `pr-merge.sh` automatically inspects the exact PR head repository and full commit SHA rather than its `main` base; this also handles fork PRs without branch-name ambiguity. Pass `--expect-head <approved-full-sha>` to bind a commit-specific review or merge-gate verdict; Gitea uses atomic `head_commit_id` and GitHub uses `--match-head-commit`.
|
||||
|
||||
### Code Review (Codex)
|
||||
|
||||
```bash
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -88,7 +88,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
|
||||
@@ -147,9 +147,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -97,7 +97,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
@@ -198,9 +198,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -101,7 +101,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
@@ -230,9 +230,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
+2
-2
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -87,7 +87,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
|
||||
+2
-2
@@ -146,9 +146,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
+2
-2
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -84,7 +84,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
|
||||
+2
-2
@@ -136,9 +136,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
2. Do NOT ask for routine confirmation before required push/merge/issue-close/release/tag actions.
|
||||
3. Completion is forbidden at PR-open stage.
|
||||
4. Completion requires merged PR to `main` + terminal green CI + linked issue/internal task closed.
|
||||
5. Before push or merge, run queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
5. Before push or merge, run the queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
6. For issue/PR/milestone operations, use Mosaic wrappers first (`~/.config/mosaic/tools/git/*.sh`).
|
||||
7. If any required wrapper command fails: report `blocked` with the exact failed wrapper command and stop.
|
||||
8. Do NOT stop at "PR created" and do NOT ask "should I merge?" for routine flow.
|
||||
@@ -85,7 +85,7 @@ Reference:
|
||||
5. Do not mark implementation complete until PR is merged.
|
||||
6. Do not mark implementation complete until CI/pipeline status is terminal green.
|
||||
7. Close linked issues/tasks only after merge + green CI.
|
||||
8. Before push or merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge -B main`.
|
||||
8. Before push or merge, run the CI queue guard against the push branch or the merge PR's exact head repository/SHA (`ci-queue-wait.sh --help`); `pr-merge.sh` supplies exact merge metadata automatically.
|
||||
|
||||
## Container Release Strategy (When Applicable)
|
||||
|
||||
|
||||
@@ -133,9 +133,9 @@ Do NOT stop at "PR created" and do NOT ask "should I merge?" or "should I close
|
||||
5. Ensure `docs/PRD.md` or `docs/PRD.json` exists and is current before coding.
|
||||
6. Create scratchpad: `docs/scratchpads/{task-id}-{short-name}.md` and include issue/internal ref.
|
||||
7. Update `docs/TASKS.md` status + issue/internal ref before coding.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push -B main`.
|
||||
8. Before push, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push`.
|
||||
9. Open PR to `main` for delivery changes (no direct push to `main`).
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B main`.
|
||||
10. Before merge, run CI queue guard: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose merge -B <PR_HEAD_BRANCH> -R <PR_HEAD_OWNER/REPO> --sha <PR_HEAD_FULL_SHA>`.
|
||||
11. Merge PRs that pass required checks and review gates with squash strategy only.
|
||||
12. Reference issues/internal refs in commits (`Fixes #123`, `Refs #123`, or `Refs TASKS:T1`).
|
||||
13. Close issue/internal task only after testing and documentation gates pass, PR merge is complete, and CI/pipeline status is terminal green.
|
||||
|
||||
@@ -7,7 +7,9 @@ set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "$SCRIPT_DIR/detect-platform.sh"
|
||||
|
||||
BRANCH="main"
|
||||
BRANCH=""
|
||||
TARGET_REPO=""
|
||||
HEAD_SHA=""
|
||||
TIMEOUT_SEC=900
|
||||
INTERVAL_SEC=15
|
||||
PURPOSE="merge"
|
||||
@@ -15,10 +17,12 @@ REQUIRE_STATUS=0
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $(basename "$0") [-B branch] [-t timeout_sec] [-i interval_sec] [--purpose push|merge] [--require-status]
|
||||
Usage: $(basename "$0") [-B branch] [-R owner/repo] [--sha full-40] [-t timeout_sec] [-i interval_sec] [--purpose push|merge] [--require-status]
|
||||
|
||||
Options:
|
||||
-B, --branch BRANCH Branch head to inspect (default: main)
|
||||
-B, --branch BRANCH Branch head to inspect (default: current branch)
|
||||
-R, --repo OWNER/REPO Repository containing the branch (default: origin repo)
|
||||
--sha FULL_SHA Inspect this exact 40-character commit instead of resolving the branch
|
||||
-t, --timeout SECONDS Max wait time in seconds (default: 900)
|
||||
-i, --interval SECONDS Poll interval in seconds (default: 15)
|
||||
--purpose VALUE Log context: push|merge (default: merge)
|
||||
@@ -27,63 +31,65 @@ Options:
|
||||
|
||||
Examples:
|
||||
$(basename "$0")
|
||||
$(basename "$0") --purpose push -B main -t 600 -i 10
|
||||
$(basename "$0") --purpose push -t 600 -i 10
|
||||
EOF
|
||||
}
|
||||
|
||||
# get_remote_host and get_gitea_token are provided by detect-platform.sh
|
||||
|
||||
get_state_from_status_json() {
|
||||
python3 - <<'PY'
|
||||
# Python source comes from -c so the provider payload remains on stdin.
|
||||
# Never move the payload to argv: commit-status responses can exceed ARG_MAX.
|
||||
python3 -c '
|
||||
import json
|
||||
import sys
|
||||
|
||||
try:
|
||||
payload = json.load(sys.stdin)
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("status payload is not an object")
|
||||
except Exception:
|
||||
print("unknown")
|
||||
print("malformed")
|
||||
raise SystemExit(0)
|
||||
|
||||
statuses = payload.get("statuses") or []
|
||||
state = (payload.get("state") or "").lower()
|
||||
raw_statuses = payload.get("statuses", [])
|
||||
raw_state = payload.get("state", "")
|
||||
if not isinstance(raw_statuses, list) or not isinstance(raw_state, str):
|
||||
print("malformed")
|
||||
raise SystemExit(0)
|
||||
statuses = raw_statuses
|
||||
state = raw_state.lower()
|
||||
|
||||
pending_values = {"pending", "queued", "running", "waiting"}
|
||||
failure_values = {"failure", "error", "failed"}
|
||||
success_values = {"success"}
|
||||
|
||||
if state in pending_values:
|
||||
print("pending")
|
||||
raise SystemExit(0)
|
||||
if state in failure_values:
|
||||
print("terminal-failure")
|
||||
raise SystemExit(0)
|
||||
if state in success_values:
|
||||
print("terminal-success")
|
||||
raise SystemExit(0)
|
||||
|
||||
values = []
|
||||
for item in statuses:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
value = (item.get("status") or item.get("state") or "").lower()
|
||||
if value:
|
||||
values.append(value)
|
||||
print("malformed")
|
||||
raise SystemExit(0)
|
||||
raw_value = item.get("status") or item.get("state")
|
||||
if not isinstance(raw_value, str) or not raw_value:
|
||||
print("malformed")
|
||||
raise SystemExit(0)
|
||||
values.append(raw_value.lower())
|
||||
|
||||
if not values and not state:
|
||||
print("no-status")
|
||||
elif any(v in pending_values for v in values):
|
||||
if any(value in pending_values for value in values) or state in pending_values:
|
||||
print("pending")
|
||||
elif any(v in failure_values for v in values):
|
||||
elif any(value in failure_values for value in values) or state in failure_values:
|
||||
print("terminal-failure")
|
||||
elif values and all(v in success_values for v in values):
|
||||
elif values and all(value in success_values for value in values) and state in {"", "success"}:
|
||||
print("terminal-success")
|
||||
elif not values:
|
||||
print("no-status")
|
||||
else:
|
||||
print("unknown")
|
||||
PY
|
||||
'
|
||||
}
|
||||
|
||||
print_pending_contexts() {
|
||||
python3 - <<'PY'
|
||||
python3 -c '
|
||||
import json
|
||||
import sys
|
||||
|
||||
@@ -104,17 +110,61 @@ for item in statuses:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
name = item.get("context") or item.get("name") or "unknown-context"
|
||||
value = (item.get("status") or item.get("state") or "unknown").lower()
|
||||
value = str(item.get("status") or item.get("state") or "unknown").lower()
|
||||
target = item.get("target_url") or item.get("url") or ""
|
||||
if value in pending_values:
|
||||
found = True
|
||||
if target:
|
||||
print(f"[ci-queue-wait] pending: {name}={value} ({target})")
|
||||
else:
|
||||
print(f"[ci-queue-wait] pending: {name}={value}")
|
||||
suffix = f" ({target})" if target else ""
|
||||
print(f"[ci-queue-wait] pending: {name}={value}{suffix}")
|
||||
if not found:
|
||||
print("[ci-queue-wait] no pending contexts")
|
||||
'
|
||||
}
|
||||
|
||||
record_cannot_assert() {
|
||||
local reason="$1"
|
||||
local audit_log="${MOSAIC_CI_QUEUE_AUDIT_LOG:-${XDG_STATE_HOME:-${HOME:-}/.local/state}/mosaic/audit/ci-queue-wait.jsonl}"
|
||||
|
||||
if [[ -z "$audit_log" ]] || ! mkdir -p "$(dirname "$audit_log")"; then
|
||||
echo "Error: CANNOT_ASSERT and audit directory is unavailable; refusing degraded pass." >&2
|
||||
return 70
|
||||
fi
|
||||
|
||||
if ! python3 - "$audit_log" "$reason" "${PLATFORM:-unknown}" "$PURPOSE" "${BRANCH:-unknown}" "${OWNER:-unknown}/${REPO:-unknown}" <<'PY'
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
path, reason, platform, purpose, branch, repo = sys.argv[1:]
|
||||
record = {
|
||||
"timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"outcome": "CANNOT_ASSERT",
|
||||
"reason": reason,
|
||||
"platform": platform,
|
||||
"purpose": purpose,
|
||||
"disposition": "hold" if purpose == "merge" else "degraded-pass",
|
||||
"branch": branch,
|
||||
"repo": repo,
|
||||
}
|
||||
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
|
||||
try:
|
||||
os.write(fd, (json.dumps(record, separators=(",", ":")) + "\n").encode())
|
||||
finally:
|
||||
os.close(fd)
|
||||
PY
|
||||
then
|
||||
echo "Error: CANNOT_ASSERT and audit write failed at ${audit_log}; refusing degraded pass." >&2
|
||||
return 70
|
||||
fi
|
||||
|
||||
if [[ "$PURPOSE" == "merge" ]]; then
|
||||
echo "[ci-queue-wait] CANNOT_ASSERT reason=${reason} purpose=merge branch=${BRANCH:-unknown}; audited=${audit_log}; HOLD (exit 75). Retry after provider recovery; no manual reset is required." >&2
|
||||
return 75
|
||||
fi
|
||||
|
||||
echo "[ci-queue-wait] CANNOT_ASSERT reason=${reason} purpose=push branch=${BRANCH:-unknown}; audited=${audit_log}; push may proceed in degraded mode." >&2
|
||||
return 0
|
||||
}
|
||||
|
||||
github_get_branch_head_sha() {
|
||||
@@ -128,7 +178,87 @@ github_get_commit_status_json() {
|
||||
local owner="$1"
|
||||
local repo="$2"
|
||||
local sha="$3"
|
||||
gh api "repos/${owner}/${repo}/commits/${sha}/status"
|
||||
local work_root status_file checks_file
|
||||
work_root="${AGENT_WORK_ROOT:-${HOME:-}/.cache/mosaic/ci-queue-wait}"
|
||||
mkdir -p "$work_root" || return 1
|
||||
status_file=$(mktemp "$work_root/github-status.XXXXXX") || return 1
|
||||
checks_file=$(mktemp "$work_root/github-checks.XXXXXX") || {
|
||||
rm -f "$status_file"
|
||||
return 1
|
||||
}
|
||||
|
||||
if ! gh api --paginate --slurp "repos/${owner}/${repo}/commits/${sha}/statuses?per_page=100" > "$status_file" ||
|
||||
! gh api --paginate --slurp "repos/${owner}/${repo}/commits/${sha}/check-runs?per_page=100&filter=latest" > "$checks_file"; then
|
||||
rm -f "$status_file" "$checks_file"
|
||||
return 1
|
||||
fi
|
||||
|
||||
python3 - "$status_file" "$checks_file" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as handle:
|
||||
status_pages = json.load(handle)
|
||||
with open(sys.argv[2], encoding="utf-8") as handle:
|
||||
check_pages = json.load(handle)
|
||||
|
||||
if not isinstance(status_pages, list) or not isinstance(check_pages, list):
|
||||
raise SystemExit(1)
|
||||
|
||||
# The statuses endpoint is newest-first and can contain retries for one context.
|
||||
# Keep only the newest entry per context after flattening every page.
|
||||
combined = []
|
||||
seen_contexts = set()
|
||||
for page in status_pages:
|
||||
if not isinstance(page, list):
|
||||
raise SystemExit(1)
|
||||
for status in page:
|
||||
if not isinstance(status, dict):
|
||||
raise SystemExit(1)
|
||||
context = status.get("context")
|
||||
if not isinstance(context, str) or not context or context in seen_contexts:
|
||||
continue
|
||||
seen_contexts.add(context)
|
||||
combined.append(status)
|
||||
|
||||
check_runs = []
|
||||
reported_total = 0
|
||||
for page in check_pages:
|
||||
if not isinstance(page, dict):
|
||||
raise SystemExit(1)
|
||||
page_runs = page.get("check_runs") or []
|
||||
total_count = page.get("total_count")
|
||||
if not isinstance(page_runs, list) or not isinstance(total_count, int):
|
||||
raise SystemExit(1)
|
||||
reported_total = max(reported_total, total_count)
|
||||
check_runs.extend(page_runs)
|
||||
if len(check_runs) < reported_total:
|
||||
raise SystemExit(1)
|
||||
|
||||
for run in check_runs:
|
||||
if not isinstance(run, dict):
|
||||
raise SystemExit(1)
|
||||
status = run.get("status")
|
||||
conclusion = run.get("conclusion")
|
||||
if status != "completed":
|
||||
value = "pending"
|
||||
elif conclusion == "success":
|
||||
value = "success"
|
||||
elif conclusion in {"failure", "cancelled", "timed_out", "action_required", "startup_failure", "stale"}:
|
||||
value = "failure"
|
||||
else:
|
||||
value = "unknown"
|
||||
combined.append({
|
||||
"context": run.get("name") or "github-check",
|
||||
"status": value,
|
||||
"target_url": run.get("html_url") or run.get("details_url") or "",
|
||||
})
|
||||
|
||||
json.dump({"state": "", "statuses": combined}, sys.stdout)
|
||||
PY
|
||||
local status=$?
|
||||
rm -f "$status_file" "$checks_file"
|
||||
return "$status"
|
||||
}
|
||||
|
||||
gitea_get_branch_head_sha() {
|
||||
@@ -174,6 +304,14 @@ while [[ $# -gt 0 ]]; do
|
||||
BRANCH="$2"
|
||||
shift 2
|
||||
;;
|
||||
-R|--repo)
|
||||
TARGET_REPO="$2"
|
||||
shift 2
|
||||
;;
|
||||
--sha)
|
||||
HEAD_SHA="$2"
|
||||
shift 2
|
||||
;;
|
||||
-t|--timeout)
|
||||
TIMEOUT_SEC="$2"
|
||||
shift 2
|
||||
@@ -206,45 +344,89 @@ if ! [[ "$TIMEOUT_SEC" =~ ^[0-9]+$ ]] || ! [[ "$INTERVAL_SEC" =~ ^[0-9]+$ ]]; th
|
||||
echo "Error: timeout and interval must be integer seconds." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -n "$HEAD_SHA" && ! "$HEAD_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then
|
||||
echo "Error: --sha must be a full 40-character hexadecimal commit SHA." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -n "$TARGET_REPO" && ! "$TARGET_REPO" =~ ^[^/[:space:]]+/[^/[:space:]]+$ ]]; then
|
||||
echo "Error: --repo must be OWNER/REPO." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
OWNER=$(get_repo_owner)
|
||||
REPO=$(get_repo_name)
|
||||
detect_platform > /dev/null
|
||||
if [[ "$PURPOSE" != "push" && "$PURPOSE" != "merge" ]]; then
|
||||
echo "Error: --purpose must be push or merge." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
OWNER="unknown"
|
||||
REPO="unknown"
|
||||
PLATFORM="unknown"
|
||||
if ! OWNER=$(get_repo_owner) || [[ -z "$OWNER" ]]; then
|
||||
record_cannot_assert "repository-owner-unresolvable"
|
||||
exit $?
|
||||
fi
|
||||
if ! REPO=$(get_repo_name) || [[ -z "$REPO" ]]; then
|
||||
record_cannot_assert "repository-name-unresolvable"
|
||||
exit $?
|
||||
fi
|
||||
if ! detect_platform > /dev/null; then
|
||||
PLATFORM="${PLATFORM:-unknown}"
|
||||
record_cannot_assert "unsupported-platform"
|
||||
exit $?
|
||||
fi
|
||||
PLATFORM="${PLATFORM:-unknown}"
|
||||
|
||||
if [[ -n "$TARGET_REPO" ]]; then
|
||||
OWNER="${TARGET_REPO%%/*}"
|
||||
REPO="${TARGET_REPO##*/}"
|
||||
fi
|
||||
|
||||
if [[ -z "$BRANCH" ]]; then
|
||||
if ! BRANCH=$(git symbolic-ref --quiet --short HEAD) || [[ -z "$BRANCH" ]]; then
|
||||
record_cannot_assert "current-branch-unresolvable"
|
||||
exit $?
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ "$PLATFORM" == "github" ]]; then
|
||||
if ! command -v gh >/dev/null 2>&1; then
|
||||
echo "Error: gh CLI is required for GitHub CI queue guard." >&2
|
||||
exit 1
|
||||
record_cannot_assert "github-cli-unavailable"
|
||||
exit $?
|
||||
fi
|
||||
HEAD_SHA=$(github_get_branch_head_sha "$OWNER" "$REPO" "$BRANCH")
|
||||
if [[ -z "$HEAD_SHA" ]]; then
|
||||
echo "Error: Could not resolve ${BRANCH} head SHA." >&2
|
||||
exit 1
|
||||
if ! HEAD_SHA=$(github_get_branch_head_sha "$OWNER" "$REPO" "$BRANCH") || [[ -z "$HEAD_SHA" ]]; then
|
||||
record_cannot_assert "branch-head-unavailable"
|
||||
exit $?
|
||||
fi
|
||||
fi
|
||||
echo "[ci-queue-wait] platform=github purpose=${PURPOSE} branch=${BRANCH} sha=${HEAD_SHA}"
|
||||
elif [[ "$PLATFORM" == "gitea" ]]; then
|
||||
HOST=$(get_remote_host) || {
|
||||
echo "Error: Could not determine remote host." >&2
|
||||
exit 1
|
||||
}
|
||||
TOKEN=$(get_gitea_token "$HOST") || {
|
||||
echo "Error: Gitea token not found. Set GITEA_TOKEN or configure ~/.git-credentials." >&2
|
||||
exit 1
|
||||
}
|
||||
HEAD_SHA=$(gitea_get_branch_head_sha "$HOST" "$OWNER/$REPO" "$BRANCH" "$TOKEN")
|
||||
if [[ "$HEAD_SHA" == "__BRANCH_ABSENT__" ]]; then
|
||||
echo "[ci-queue-wait] branch ${BRANCH} not yet on remote — no in-flight pipeline; queue clear."
|
||||
exit 0
|
||||
if ! HOST=$(get_remote_host) || [[ -z "$HOST" ]]; then
|
||||
record_cannot_assert "remote-host-unresolvable"
|
||||
exit $?
|
||||
fi
|
||||
if ! TOKEN=$(get_gitea_token "$HOST") || [[ -z "$TOKEN" ]]; then
|
||||
record_cannot_assert "credential-unresolvable"
|
||||
exit $?
|
||||
fi
|
||||
if [[ -z "$HEAD_SHA" ]]; then
|
||||
echo "Error: Could not resolve ${BRANCH} head SHA." >&2
|
||||
exit 1
|
||||
if ! HEAD_SHA=$(gitea_get_branch_head_sha "$HOST" "$OWNER/$REPO" "$BRANCH" "$TOKEN"); then
|
||||
record_cannot_assert "branch-head-unavailable"
|
||||
exit $?
|
||||
fi
|
||||
if [[ "$HEAD_SHA" == "__BRANCH_ABSENT__" ]]; then
|
||||
echo "[ci-queue-wait] branch ${BRANCH} not yet on remote — no in-flight pipeline; queue clear."
|
||||
exit 0
|
||||
fi
|
||||
if [[ -z "$HEAD_SHA" ]]; then
|
||||
record_cannot_assert "branch-head-unavailable"
|
||||
exit $?
|
||||
fi
|
||||
fi
|
||||
echo "[ci-queue-wait] platform=gitea purpose=${PURPOSE} branch=${BRANCH} sha=${HEAD_SHA}"
|
||||
else
|
||||
echo "Error: Unsupported platform '${PLATFORM}'." >&2
|
||||
exit 1
|
||||
record_cannot_assert "unsupported-platform"
|
||||
exit $?
|
||||
fi
|
||||
|
||||
START_TS=$(date +%s)
|
||||
@@ -253,14 +435,20 @@ DEADLINE_TS=$((START_TS + TIMEOUT_SEC))
|
||||
while true; do
|
||||
NOW_TS=$(date +%s)
|
||||
if (( NOW_TS > DEADLINE_TS )); then
|
||||
echo "Error: Timed out waiting for CI queue to clear on ${BRANCH} after ${TIMEOUT_SEC}s." >&2
|
||||
echo "Error: ASSERTED_NOT_READY state=pending; timed out waiting for CI queue to clear on ${BRANCH} after ${TIMEOUT_SEC}s." >&2
|
||||
exit 124
|
||||
fi
|
||||
|
||||
if [[ "$PLATFORM" == "github" ]]; then
|
||||
STATUS_JSON=$(github_get_commit_status_json "$OWNER" "$REPO" "$HEAD_SHA")
|
||||
if ! STATUS_JSON=$(github_get_commit_status_json "$OWNER" "$REPO" "$HEAD_SHA"); then
|
||||
record_cannot_assert "status-provider-unreachable"
|
||||
exit $?
|
||||
fi
|
||||
else
|
||||
STATUS_JSON=$(gitea_get_commit_status_json "$HOST" "$OWNER/$REPO" "$HEAD_SHA" "$TOKEN")
|
||||
if ! STATUS_JSON=$(gitea_get_commit_status_json "$HOST" "$OWNER/$REPO" "$HEAD_SHA" "$TOKEN"); then
|
||||
record_cannot_assert "status-provider-unreachable"
|
||||
exit $?
|
||||
fi
|
||||
fi
|
||||
|
||||
STATE=$(printf '%s' "$STATUS_JSON" | get_state_from_status_json)
|
||||
@@ -271,21 +459,24 @@ while true; do
|
||||
printf '%s' "$STATUS_JSON" | print_pending_contexts
|
||||
sleep "$INTERVAL_SEC"
|
||||
;;
|
||||
terminal-success)
|
||||
exit 0
|
||||
;;
|
||||
no-status)
|
||||
if [[ "$REQUIRE_STATUS" -eq 1 ]]; then
|
||||
echo "Error: No CI status contexts found for ${BRANCH} while --require-status is set." >&2
|
||||
exit 1
|
||||
echo "Error: ASSERTED_NOT_READY state=no-status; --require-status was set for ${BRANCH}." >&2
|
||||
else
|
||||
echo "Error: ASSERTED_NOT_READY state=no-status purpose=${PURPOSE} branch=${BRANCH}." >&2
|
||||
fi
|
||||
echo "[ci-queue-wait] no status contexts present; proceeding."
|
||||
exit 0
|
||||
exit 3
|
||||
;;
|
||||
terminal-success|terminal-failure|unknown)
|
||||
# Queue guard only blocks on pending/running/queued states.
|
||||
exit 0
|
||||
terminal-failure|malformed|unknown)
|
||||
echo "Error: ASSERTED_NOT_READY state=${STATE} purpose=${PURPOSE} branch=${BRANCH}." >&2
|
||||
exit 3
|
||||
;;
|
||||
*)
|
||||
echo "[ci-queue-wait] unrecognized state '${STATE}', proceeding conservatively."
|
||||
exit 0
|
||||
echo "Error: ASSERTED_NOT_READY unrecognized-state=${STATE} purpose=${PURPOSE} branch=${BRANCH}." >&2
|
||||
exit 3
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
# pr-merge.sh - Merge pull requests on Gitea or GitHub
|
||||
# Usage: pr-merge.sh -n PR_NUMBER [-m squash] [-d] [--skip-queue-guard]
|
||||
# Usage: pr-merge.sh -n PR_NUMBER [-m squash] [-d]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -12,8 +12,8 @@ source "$SCRIPT_DIR/detect-platform.sh"
|
||||
PR_NUMBER=""
|
||||
MERGE_METHOD="squash"
|
||||
DELETE_BRANCH=false
|
||||
SKIP_QUEUE_GUARD=false
|
||||
DRY_RUN=false
|
||||
EXPECT_HEAD=""
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
@@ -25,15 +25,14 @@ Options:
|
||||
-n, --number NUMBER PR number to merge (required)
|
||||
-m, --method METHOD Merge method: squash only (default: squash)
|
||||
-d, --delete-branch Delete the head branch after merge
|
||||
--skip-queue-guard Skip CI queue guard wait before merge
|
||||
--dry-run Run metadata/login preflight without merging
|
||||
--expect-head SHA Refuse unless the PR head matches this full commit SHA
|
||||
-h, --help Show this help message
|
||||
|
||||
Examples:
|
||||
$(basename "$0") -n 42 # Merge PR #42
|
||||
$(basename "$0") -n 42 -m squash # Squash merge
|
||||
$(basename "$0") -n 42 -d # Squash merge and delete branch
|
||||
$(basename "$0") -n 42 --skip-queue-guard # Skip queue guard wait
|
||||
EOF
|
||||
exit "${1:-1}"
|
||||
}
|
||||
@@ -53,15 +52,14 @@ while [[ $# -gt 0 ]]; do
|
||||
DELETE_BRANCH=true
|
||||
shift
|
||||
;;
|
||||
--skip-queue-guard)
|
||||
SKIP_QUEUE_GUARD=true
|
||||
shift
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
SKIP_QUEUE_GUARD=true
|
||||
shift
|
||||
;;
|
||||
--expect-head)
|
||||
EXPECT_HEAD="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage 0
|
||||
;;
|
||||
@@ -86,18 +84,36 @@ if [[ "$MERGE_METHOD" != "squash" ]]; then
|
||||
echo "Error: Mosaic policy enforces squash merge only. Received '$MERGE_METHOD'." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -n "$EXPECT_HEAD" && ! "$EXPECT_HEAD" =~ ^[0-9a-fA-F]{40}$ ]]; then
|
||||
echo "Error: --expect-head must be a full 40-character hexadecimal commit SHA." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PR_METADATA="$("$SCRIPT_DIR/pr-metadata.sh" -n "$PR_NUMBER")"
|
||||
BASE_BRANCH="$(printf '%s' "$PR_METADATA" | python3 -c 'import json, sys; print((json.load(sys.stdin).get("baseRefName") or "").strip())')"
|
||||
HEAD_BRANCH="$(printf '%s' "$PR_METADATA" | python3 -c 'import json, sys; print((json.load(sys.stdin).get("headRefName") or "").strip())')"
|
||||
HEAD_SHA="$(printf '%s' "$PR_METADATA" | python3 -c 'import json, sys; print((json.load(sys.stdin).get("headRefOid") or "").strip())')"
|
||||
HEAD_REPO="$(printf '%s' "$PR_METADATA" | python3 -c 'import json, sys; value=json.load(sys.stdin).get("headRepository") or ""; print((value.get("nameWithOwner") or value.get("full_name") or "") if isinstance(value, dict) else str(value).strip())')"
|
||||
if [[ "$BASE_BRANCH" != "main" ]]; then
|
||||
echo "Error: Mosaic policy allows merges only for PRs targeting 'main' (found '$BASE_BRANCH')." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$SKIP_QUEUE_GUARD" != true ]]; then
|
||||
if [[ -z "$HEAD_BRANCH" || -z "$HEAD_REPO" || ! "$HEAD_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then
|
||||
echo "Error: Could not resolve the PR head branch, repository, and full commit SHA for queue inspection." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -n "$EXPECT_HEAD" && "$HEAD_SHA" != "$EXPECT_HEAD" ]]; then
|
||||
echo "Error: PR head moved: expected $EXPECT_HEAD, found $HEAD_SHA." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$DRY_RUN" != true ]]; then
|
||||
"$SCRIPT_DIR/ci-queue-wait.sh" \
|
||||
--purpose merge \
|
||||
-B "$BASE_BRANCH" \
|
||||
-B "$HEAD_BRANCH" \
|
||||
-R "$HEAD_REPO" \
|
||||
--sha "$HEAD_SHA" \
|
||||
-t "${MOSAIC_CI_QUEUE_TIMEOUT_SEC:-900}" \
|
||||
-i "${MOSAIC_CI_QUEUE_POLL_SEC:-15}"
|
||||
fi
|
||||
@@ -106,31 +122,22 @@ PLATFORM=$(detect_platform)
|
||||
OWNER=$(get_repo_owner)
|
||||
REPO=$(get_repo_name)
|
||||
|
||||
is_known_tea_empty_identity_failure() {
|
||||
local error_file="$1"
|
||||
|
||||
python3 - "$error_file" <<'PY'
|
||||
import re
|
||||
import sys
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8", errors="replace") as handle:
|
||||
error = handle.read()
|
||||
|
||||
known_empty_identity = re.search(
|
||||
r"user does not exist.*\[.*uid:\s*0,\s*name:\s*\]",
|
||||
error,
|
||||
flags=re.IGNORECASE | re.DOTALL,
|
||||
)
|
||||
raise SystemExit(0 if known_empty_identity else 1)
|
||||
PY
|
||||
}
|
||||
|
||||
merge_gitea_with_api() {
|
||||
local host="$1" api_url token basic_auth body_file raw_code payload
|
||||
api_url="https://${host}/api/v1/repos/${OWNER}/${REPO}/pulls/${PR_NUMBER}/merge"
|
||||
mkdir -p "${AGENT_WORK_ROOT:-${HOME:-/tmp}/mosaic/agent-work}"
|
||||
body_file=$(mktemp "${AGENT_WORK_ROOT:-${HOME:-/tmp}/mosaic/agent-work}/pr-merge-api-response.XXXXXX")
|
||||
payload='{"Do":"squash"}'
|
||||
payload=$(python3 - "$HEAD_SHA" "$DELETE_BRANCH" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
head_sha, delete_branch = sys.argv[1:]
|
||||
payload = {"Do": "squash", "head_commit_id": head_sha}
|
||||
if delete_branch == "true":
|
||||
payload["delete_branch_after_merge"] = True
|
||||
print(json.dumps(payload, separators=(",", ":")))
|
||||
PY
|
||||
)
|
||||
|
||||
token=$(get_gitea_token "$host" || true)
|
||||
if [[ -n "$token" ]]; then
|
||||
@@ -202,7 +209,7 @@ fi
|
||||
|
||||
case "$PLATFORM" in
|
||||
github)
|
||||
cmd=(gh pr merge "$PR_NUMBER" --squash)
|
||||
cmd=(gh pr merge "$PR_NUMBER" --squash --match-head-commit "$HEAD_SHA")
|
||||
[[ "$DELETE_BRANCH" == true ]] && cmd+=(--delete-branch)
|
||||
"${cmd[@]}"
|
||||
;;
|
||||
@@ -211,32 +218,9 @@ case "$PLATFORM" in
|
||||
echo "Error: Cannot determine host from origin remote URL" >&2
|
||||
exit 1
|
||||
}
|
||||
TEA_LOGIN="$(get_gitea_login_for_host "$HOST" || true)"
|
||||
|
||||
if [[ -n "$TEA_LOGIN" ]]; then
|
||||
mkdir -p "${AGENT_WORK_ROOT:-${HOME:-/tmp}/mosaic/agent-work}"
|
||||
TEA_ERROR_FILE=$(mktemp "${AGENT_WORK_ROOT:-${HOME:-/tmp}/mosaic/agent-work}/pr-merge-tea-error.XXXXXX")
|
||||
if tea pr merge "$PR_NUMBER" --style squash --repo "$OWNER/$REPO" --login "$TEA_LOGIN" 2> "$TEA_ERROR_FILE"; then
|
||||
rm -f "$TEA_ERROR_FILE"
|
||||
elif is_known_tea_empty_identity_failure "$TEA_ERROR_FILE"; then
|
||||
cat "$TEA_ERROR_FILE" >&2
|
||||
echo "Known tea empty identity failure detected; using authenticated Gitea API merge fallback." >&2
|
||||
rm -f "$TEA_ERROR_FILE"
|
||||
merge_gitea_with_api "$HOST"
|
||||
else
|
||||
cat "$TEA_ERROR_FILE" >&2
|
||||
rm -f "$TEA_ERROR_FILE"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "No tea login configured for $HOST; using authenticated Gitea API merge fallback." >&2
|
||||
merge_gitea_with_api "$HOST"
|
||||
fi
|
||||
|
||||
# Delete branch after merge if requested
|
||||
if [[ "$DELETE_BRANCH" == true ]]; then
|
||||
echo "Note: Branch deletion after merge may need to be done separately with tea" >&2
|
||||
fi
|
||||
# Gitea's API head_commit_id is an atomic compare-and-merge precondition.
|
||||
# tea cannot express it, so exact-head merges use the authenticated API path.
|
||||
merge_gitea_with_api "$HOST"
|
||||
;;
|
||||
*)
|
||||
echo "Error: Could not detect git platform" >&2
|
||||
|
||||
@@ -109,7 +109,7 @@ PY
|
||||
detect_platform > /dev/null
|
||||
|
||||
if [[ "$PLATFORM" == "github" ]]; then
|
||||
METADATA=$(gh pr view "$PR_NUMBER" --json number,title,body,state,author,headRefName,baseRefName,files,labels,assignees,milestone,createdAt,updatedAt,url,isDraft)
|
||||
METADATA=$(gh pr view "$PR_NUMBER" --json number,title,body,state,author,headRefName,headRefOid,headRepository,baseRefName,files,labels,assignees,milestone,createdAt,updatedAt,url,isDraft)
|
||||
write_metadata "$METADATA"
|
||||
elif [[ "$PLATFORM" == "gitea" ]]; then
|
||||
OWNER=$(get_repo_owner)
|
||||
@@ -182,6 +182,25 @@ if isinstance(head_ref, str) and head_ref.startswith('refs/pull/'):
|
||||
data.get('head_ref'),
|
||||
head_ref,
|
||||
)
|
||||
head_sha = first_non_empty(
|
||||
nested(data, 'head', 'sha'),
|
||||
nested(data, 'head', 'id'),
|
||||
data.get('head_sha'),
|
||||
)
|
||||
head_repo = first_non_empty(
|
||||
nested(data, 'head', 'repo', 'full_name'),
|
||||
nested(data, 'head', 'repo', 'name_with_owner'),
|
||||
)
|
||||
if not head_repo:
|
||||
head_repo_owner = first_non_empty(
|
||||
nested(data, 'head', 'repo', 'owner', 'login'),
|
||||
nested(data, 'head', 'repo', 'owner', 'username'),
|
||||
nested(data, 'head', 'repo', 'owner_name'),
|
||||
)
|
||||
head_repo_name = first_non_empty(nested(data, 'head', 'repo', 'name'))
|
||||
if head_repo_owner and head_repo_name:
|
||||
head_repo = f'{head_repo_owner}/{head_repo_name}'
|
||||
|
||||
base_ref = first_non_empty(
|
||||
nested(data, 'base', 'ref'),
|
||||
nested(data, 'base', 'name'),
|
||||
@@ -207,6 +226,8 @@ normalized = {
|
||||
'state': data.get('state'),
|
||||
'author': nested(data, 'user', 'login') or '',
|
||||
'headRefName': head_ref,
|
||||
'headRefOid': head_sha,
|
||||
'headRepository': head_repo,
|
||||
'baseRefName': base_ref,
|
||||
'labels': [l.get('name', '') for l in data.get('labels', []) if isinstance(l, dict)],
|
||||
'assignees': [a.get('login', '') for a in data.get('assignees', []) if isinstance(a, dict)],
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
# Covers:
|
||||
# (a) 404 branch-absent -> exit 0, "queue clear" message.
|
||||
# (b) 200 existing branch + a terminal CI state -> unchanged behavior.
|
||||
# (c) genuine API error (500) -> still fail-closed (nonzero exit).
|
||||
# (c) genuine API error (500) -> loud, audited CANNOT_ASSERT; degraded exit 0.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -62,7 +62,7 @@ case "$mode" in
|
||||
200) code=200; body='{"commit":{"id":"deadbeefcafef00d0123456789abcdef01234567"}}' ;;
|
||||
500) code=500; body='{"message":"internal server error"}' ;;
|
||||
no-status) code=200; body='{}' ;;
|
||||
terminal-success) code=200; body='{"state":"success"}' ;;
|
||||
terminal-success) code=200; body='{"state":"success","statuses":[{"status":"success"}]}' ;;
|
||||
*)
|
||||
echo "curl stub: unknown mode=$mode" >&2
|
||||
exit 2
|
||||
@@ -91,6 +91,7 @@ run_ci_queue_wait() {
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
export GITEA_TOKEN="stub-token"
|
||||
export GITEA_URL="https://git.example.test"
|
||||
export MOSAIC_CI_QUEUE_AUDIT_LOG="$WORK_DIR/audit/ci-queue-wait.jsonl"
|
||||
"$SCRIPT_DIR/ci-queue-wait.sh" -B "$branch" --purpose push -t 5 -i 1
|
||||
)
|
||||
}
|
||||
@@ -131,19 +132,26 @@ elif [[ "$out_b" == *"queue clear"* ]]; then
|
||||
fail=1
|
||||
fi
|
||||
|
||||
# (c) genuine API error (500) -> still fail-closed, exit nonzero.
|
||||
# (c) genuine API error (500) -> CANNOT_ASSERT is loud and audited, but does not brick delivery.
|
||||
set +e
|
||||
out_c=$(MOSAIC_STUB_BRANCH_MODE=500 run_ci_queue_wait "feat/some-branch" 2>&1)
|
||||
status_c=$?
|
||||
set -e
|
||||
if [[ "$status_c" -eq 0 ]]; then
|
||||
echo "FAIL(c): expected a nonzero exit for a genuine 500 API error, got 0" >&2
|
||||
if [[ "$status_c" -ne 0 ]]; then
|
||||
echo "FAIL(c): expected degraded exit 0 for provider unavailability, got $status_c" >&2
|
||||
echo "$out_c" >&2
|
||||
fail=1
|
||||
elif [[ "$out_c" != *"CANNOT_ASSERT"* ]]; then
|
||||
echo "FAIL(c): expected a loud CANNOT_ASSERT diagnostic" >&2
|
||||
echo "$out_c" >&2
|
||||
fail=1
|
||||
elif [[ "$out_c" == *"queue clear"* ]]; then
|
||||
echo "FAIL(c): a genuine API error must not be reported as queue-clear" >&2
|
||||
echo "$out_c" >&2
|
||||
fail=1
|
||||
elif [[ ! -s "$WORK_DIR/audit/ci-queue-wait.jsonl" ]]; then
|
||||
echo "FAIL(c): expected a durable CANNOT_ASSERT audit record" >&2
|
||||
fail=1
|
||||
fi
|
||||
|
||||
if [[ "$fail" -eq 0 ]]; then
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env bash
|
||||
# GitHub Actions uses Checks API check-runs, not only legacy commit statuses.
|
||||
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/ci-queue-wait-github-checks}"
|
||||
REPO_DIR="$WORK_DIR/repo"
|
||||
STUB_DIR="$WORK_DIR/stubs"
|
||||
rm -rf "$WORK_DIR"
|
||||
mkdir -p "$REPO_DIR" "$STUB_DIR"
|
||||
git -C "$REPO_DIR" init -q
|
||||
git -C "$REPO_DIR" checkout -q -b fix/github-checks
|
||||
git -C "$REPO_DIR" remote add origin https://github.com/acme/widgets.git
|
||||
|
||||
cat > "$STUB_DIR/gh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
endpoint=""
|
||||
for arg in "$@"; do
|
||||
[[ "$arg" == repos/* ]] && endpoint="$arg"
|
||||
done
|
||||
printf '%s\n' "$*" >> "${MOSAIC_GH_CALL_LOG:?}"
|
||||
case "$endpoint" in
|
||||
repos/acme/widgets/branches/fix/github-checks)
|
||||
printf '%s\n' '0123456789abcdef0123456789abcdef01234567'
|
||||
;;
|
||||
repos/acme/widgets/commits/*/statuses?per_page=100)
|
||||
printf '%s\n' '[[]]'
|
||||
;;
|
||||
repos/acme/widgets/commits/*/check-runs?per_page=100\&filter=latest)
|
||||
case "${MOSAIC_GH_CHECK_MODE:?}" in
|
||||
success) printf '%s\n' '[{"total_count":1,"check_runs":[{"name":"ci","status":"completed","conclusion":"success"}]}]' ;;
|
||||
pending) printf '%s\n' '[{"total_count":1,"check_runs":[{"name":"ci","status":"in_progress","conclusion":null}]}]' ;;
|
||||
failure) printf '%s\n' '[{"total_count":1,"check_runs":[{"name":"ci","status":"completed","conclusion":"failure"}]}]' ;;
|
||||
late-failure) printf '%s\n' '[{"total_count":2,"check_runs":[{"name":"first-page","status":"completed","conclusion":"success"}]},{"total_count":2,"check_runs":[{"name":"later-page","status":"completed","conclusion":"failure"}]}]' ;;
|
||||
*) exit 2 ;;
|
||||
esac
|
||||
;;
|
||||
*) echo "unexpected gh endpoint: $endpoint" >&2; exit 2 ;;
|
||||
esac
|
||||
SH
|
||||
chmod +x "$STUB_DIR/gh"
|
||||
|
||||
run_guard() {
|
||||
local mode="$1"
|
||||
(
|
||||
cd "$REPO_DIR" || exit
|
||||
export PATH="$STUB_DIR:$PATH"
|
||||
export MOSAIC_GH_CHECK_MODE="$mode"
|
||||
export MOSAIC_GH_CALL_LOG="$WORK_DIR/gh-calls.log"
|
||||
export MOSAIC_CI_QUEUE_AUDIT_LOG="$WORK_DIR/audit.jsonl"
|
||||
"$SCRIPT_DIR/ci-queue-wait.sh" --purpose push -t 0 -i 0
|
||||
)
|
||||
}
|
||||
|
||||
failures=0
|
||||
assert_case() {
|
||||
local mode="$1" expected_rc="$2" expected_state="$3" output rc
|
||||
set +e
|
||||
output=$(run_guard "$mode" 2>&1)
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ "$expected_rc" == zero && "$rc" -ne 0 ]]; then
|
||||
echo "FAIL github-$mode: expected rc=0, got $rc" >&2
|
||||
failures=$((failures + 1))
|
||||
elif [[ "$expected_rc" == nonzero && "$rc" -eq 0 ]]; then
|
||||
echo "FAIL github-$mode: expected rc!=0, got 0" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
if [[ "$output" != *"state=$expected_state"* ]]; then
|
||||
echo "FAIL github-$mode: expected state=$expected_state, got:" >&2
|
||||
printf '%s\n' "$output" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
set -e
|
||||
: > "$WORK_DIR/gh-calls.log"
|
||||
assert_case success zero terminal-success
|
||||
assert_case pending nonzero pending
|
||||
assert_case failure nonzero terminal-failure
|
||||
assert_case late-failure nonzero terminal-failure
|
||||
|
||||
if [[ $(grep -c 'check-runs?per_page=100&filter=latest' "$WORK_DIR/gh-calls.log") -lt 4 ]]; then
|
||||
echo "FAIL: expected every case to query all Checks API pages" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
if [[ "$failures" -ne 0 ]]; then
|
||||
echo "GitHub check-runs regression failed ($failures assertions)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "GitHub check-runs regression passed (4/4 cases, including later-page failure)"
|
||||
@@ -0,0 +1,232 @@
|
||||
#!/usr/bin/env bash
|
||||
# Exit-asserting RM-03 regression harness for ci-queue-wait.sh.
|
||||
# Every case is a process-level assertion: a classifier-only green cannot satisfy it.
|
||||
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/ci-queue-wait-tristate}"
|
||||
REPO_DIR="$WORK_DIR/repo"
|
||||
STUB_DIR="$WORK_DIR/stubs"
|
||||
AUDIT_LOG="$WORK_DIR/audit/ci-queue-wait.jsonl"
|
||||
FEATURE_BRANCH="fix/rm-03-fixture"
|
||||
|
||||
rm -rf "$WORK_DIR"
|
||||
mkdir -p "$REPO_DIR" "$STUB_DIR"
|
||||
git -C "$REPO_DIR" init -q
|
||||
git -C "$REPO_DIR" checkout -q -b "$FEATURE_BRANCH"
|
||||
git -C "$REPO_DIR" remote add origin https://git.example.test/acme/widgets.git
|
||||
|
||||
cat > "$STUB_DIR/curl" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
url=""
|
||||
has_write_out=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
-w) has_write_out=1 ;;
|
||||
http://*|https://*) url="$arg" ;;
|
||||
esac
|
||||
done
|
||||
printf '%s\n' "$url" >> "${MOSAIC_STUB_URL_LOG:?}"
|
||||
|
||||
case "$url" in
|
||||
*/branches/*)
|
||||
if [[ "${MOSAIC_STUB_BRANCH_MODE:-ok}" == "unreachable" ]]; then
|
||||
exit 7
|
||||
fi
|
||||
body='{"commit":{"id":"deadbeefcafef00d0123456789abcdef01234567"}}'
|
||||
if [[ "$has_write_out" -eq 1 ]]; then
|
||||
printf '%s\n200' "$body"
|
||||
else
|
||||
printf '%s' "$body"
|
||||
fi
|
||||
;;
|
||||
*/status)
|
||||
case "${MOSAIC_STUB_STATUS_MODE:?}" in
|
||||
success) printf '%s' '{"state":"success","statuses":[{"status":"success"}]}' ;;
|
||||
pending) printf '%s' '{"state":"pending","statuses":[{"status":"pending","context":"ci/test"}]}' ;;
|
||||
failure) printf '%s' '{"state":"failure","statuses":[{"status":"failure"}]}' ;;
|
||||
no-status) printf '%s' '{"state":"","statuses":[]}' ;;
|
||||
aggregate-success-no-status) printf '%s' '{"state":"success","statuses":[]}' ;;
|
||||
malformed) printf '%s' 'not-json' ;;
|
||||
malformed-statuses-type) printf '%s' '{"state":"success","statuses":"corrupt"}' ;;
|
||||
malformed-status-entry) printf '%s' '{"state":"success","statuses":[null]}' ;;
|
||||
large-success)
|
||||
python3 -c 'import json; print(json.dumps({"state":"success", "statuses":[{"status":"success"}], "padding":"x" * (160 * 1024)}), end="")'
|
||||
;;
|
||||
unreachable) exit 7 ;;
|
||||
*) echo "unknown status mode" >&2; exit 2 ;;
|
||||
esac
|
||||
;;
|
||||
*) echo "unexpected curl URL: $url" >&2; exit 2 ;;
|
||||
esac
|
||||
SH
|
||||
chmod +x "$STUB_DIR/curl"
|
||||
|
||||
run_guard() {
|
||||
local status_mode="$1"
|
||||
local audit_log="${2:-$AUDIT_LOG}"
|
||||
shift 2 || true
|
||||
(
|
||||
cd "$REPO_DIR" || exit
|
||||
export PATH="$STUB_DIR:$PATH"
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
if [[ "$status_mode" == "credential-unresolvable" ]]; then
|
||||
export HOME="$WORK_DIR/empty-home"
|
||||
mkdir -p "$HOME"
|
||||
unset GITEA_TOKEN GITEA_URL MOSAIC_GIT_IDENTITY
|
||||
export MOSAIC_STUB_STATUS_MODE=success
|
||||
else
|
||||
export GITEA_TOKEN=stub-token
|
||||
export GITEA_URL=https://git.example.test
|
||||
export MOSAIC_STUB_STATUS_MODE="$status_mode"
|
||||
fi
|
||||
export MOSAIC_STUB_URL_LOG="$WORK_DIR/urls.log"
|
||||
export MOSAIC_CI_QUEUE_AUDIT_LOG="$audit_log"
|
||||
"$SCRIPT_DIR/ci-queue-wait.sh" --purpose "${MOSAIC_TEST_PURPOSE:-push}" -t 0 -i 0 "$@"
|
||||
)
|
||||
}
|
||||
|
||||
failures=0
|
||||
run_assertion() {
|
||||
local name="$1" expected_rc="$2" status_mode="$3" required_text="$4"
|
||||
local output rc
|
||||
shift 4
|
||||
set +e
|
||||
output=$(run_guard "$status_mode" "$AUDIT_LOG" "$@" 2>&1)
|
||||
rc=$?
|
||||
set -e
|
||||
|
||||
case "$expected_rc" in
|
||||
zero)
|
||||
if [[ "$rc" -ne 0 ]]; then
|
||||
echo "FAIL $name: expected rc=0, got rc=$rc" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
nonzero)
|
||||
if [[ "$rc" -eq 0 ]]; then
|
||||
echo "FAIL $name: expected rc!=0, got rc=0" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
not126)
|
||||
if [[ "$rc" -eq 126 ]]; then
|
||||
echo "FAIL $name: payload transport hit ARG_MAX (rc=126)" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
if [[ "$output" != *"$required_text"* ]]; then
|
||||
echo "FAIL $name: output missing '$required_text' (rc=$rc)" >&2
|
||||
printf '%s\n' "$output" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
set -e
|
||||
: > "$WORK_DIR/urls.log"
|
||||
run_assertion success zero success 'state=terminal-success'
|
||||
run_assertion pending nonzero pending 'ASSERTED_NOT_READY'
|
||||
run_assertion failure nonzero failure 'ASSERTED_NOT_READY'
|
||||
run_assertion no-status nonzero no-status 'ASSERTED_NOT_READY'
|
||||
run_assertion aggregate-success-no-status nonzero aggregate-success-no-status 'ASSERTED_NOT_READY'
|
||||
run_assertion malformed nonzero malformed 'ASSERTED_NOT_READY'
|
||||
run_assertion malformed-statuses-type nonzero malformed-statuses-type 'ASSERTED_NOT_READY'
|
||||
run_assertion malformed-status-entry nonzero malformed-status-entry 'ASSERTED_NOT_READY'
|
||||
run_assertion large-payload not126 large-success 'state=terminal-success'
|
||||
run_assertion credential-unresolvable zero credential-unresolvable 'CANNOT_ASSERT'
|
||||
run_assertion provider-unreachable zero unreachable 'CANNOT_ASSERT'
|
||||
|
||||
if [[ ! -s "$AUDIT_LOG" ]] || ! grep -q '"outcome":"CANNOT_ASSERT"' "$AUDIT_LOG"; then
|
||||
echo "FAIL provider-unreachable-audit: expected durable CANNOT_ASSERT JSONL record" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
# Merge cannot proceed without exact-head evidence. CANNOT_ASSERT is retryable exit 75,
|
||||
# distinct from ASSERTED_NOT_READY (3/124), and still writes its audit record.
|
||||
merge_audit_lines_before=$(wc -l < "$AUDIT_LOG")
|
||||
set +e
|
||||
merge_unreachable_output=$(MOSAIC_TEST_PURPOSE=merge run_guard unreachable "$AUDIT_LOG" 2>&1)
|
||||
merge_unreachable_rc=$?
|
||||
set -e
|
||||
if [[ "$merge_unreachable_rc" -ne 75 ]]; then
|
||||
echo "FAIL merge-provider-unreachable: expected rc=75, got rc=$merge_unreachable_rc" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
if [[ "$merge_unreachable_output" != *"CANNOT_ASSERT"* ]]; then
|
||||
echo "FAIL merge-provider-unreachable: expected loud CANNOT_ASSERT diagnostic" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
merge_audit_lines_after=$(wc -l < "$AUDIT_LOG")
|
||||
if [[ "$merge_audit_lines_after" -le "$merge_audit_lines_before" ]]; then
|
||||
echo "FAIL merge-provider-unreachable: expected an additional audit record" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
# A feature-branch push with no -B must inspect the checked-out feature branch.
|
||||
if ! grep -q "/branches/$FEATURE_BRANCH" "$WORK_DIR/urls.log"; then
|
||||
echo "FAIL implicit-branch: provider was not queried for $FEATURE_BRANCH" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
# Merge callers can pin both a fork repository and the exact reviewed head SHA.
|
||||
exact_sha=0123456789abcdef0123456789abcdef01234567
|
||||
: > "$WORK_DIR/urls.log"
|
||||
run_assertion exact-fork-head zero success 'state=terminal-success' \
|
||||
-B fix/rm-03-fixture -R contributor/widgets-fork --sha "$exact_sha"
|
||||
if ! grep -q "/repos/contributor/widgets-fork/commits/$exact_sha/status" "$WORK_DIR/urls.log"; then
|
||||
echo "FAIL exact-fork-head: status URL did not bind fork repository and exact SHA" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
if grep -q '/branches/' "$WORK_DIR/urls.log"; then
|
||||
echo "FAIL exact-fork-head: explicit SHA must not be re-resolved through a branch" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
# Platform/repository discovery failures use the same audited CANNOT_ASSERT path.
|
||||
audit_lines_before=$(wc -l < "$AUDIT_LOG")
|
||||
git -C "$REPO_DIR" remote set-url origin https://gitlab.com/acme/widgets.git
|
||||
set +e
|
||||
unsupported_output=$(run_guard success "$AUDIT_LOG" 2>&1)
|
||||
unsupported_rc=$?
|
||||
set -e
|
||||
git -C "$REPO_DIR" remote set-url origin https://git.example.test/acme/widgets.git
|
||||
if [[ "$unsupported_rc" -ne 0 ]]; then
|
||||
echo "FAIL unsupported-platform: expected degraded rc=0, got rc=$unsupported_rc" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
if [[ "$unsupported_output" != *"CANNOT_ASSERT"* ]]; then
|
||||
echo "FAIL unsupported-platform: expected loud CANNOT_ASSERT diagnostic" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
audit_lines_after=$(wc -l < "$AUDIT_LOG")
|
||||
if [[ "$audit_lines_after" -le "$audit_lines_before" ]]; then
|
||||
echo "FAIL unsupported-platform: expected an additional audit record" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
# A degraded pass is forbidden if the audit receipt cannot be written.
|
||||
mkdir -p "$WORK_DIR/not-a-directory"
|
||||
printf 'file' > "$WORK_DIR/not-a-directory/parent"
|
||||
set +e
|
||||
audit_failure_output=$(run_guard unreachable "$WORK_DIR/not-a-directory/parent/audit.jsonl" 2>&1)
|
||||
audit_failure_rc=$?
|
||||
set -e
|
||||
if [[ "$audit_failure_rc" -eq 0 ]]; then
|
||||
echo "FAIL audit-unavailable: expected rc!=0, got rc=0" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
if [[ "$audit_failure_output" != *"audit"* ]]; then
|
||||
echo "FAIL audit-unavailable: expected loud audit failure diagnostic" >&2
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
|
||||
if [[ "$failures" -ne 0 ]]; then
|
||||
echo "ci-queue-wait tri-state regression failed ($failures assertions)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "ci-queue-wait tri-state regression passed (all outcome classes)"
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/bin/bash
|
||||
# Regression harness for pr-merge.sh Gitea non-interactive tea empty identity fallback.
|
||||
# Regression harness for pr-merge.sh Gitea exact-head API path and input safety.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -79,15 +79,24 @@ emit_response() {
|
||||
printf '200'
|
||||
fi
|
||||
}
|
||||
if [[ "$args" == *"/api/v1/repos/mosaicstack/stack/commits/0123456789abcdef0123456789abcdef01234567/status"* ]]; then
|
||||
emit_response '{"state":"success","statuses":[{"context":"ci/test","status":"success"}]}'
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$args" == *"/api/v1/repos/mosaicstack/stack/pulls/123"* && "$args" != *"/api/v1/repos/mosaicstack/stack/pulls/123/merge"* ]]; then
|
||||
emit_response '{"number":123,"title":"mock","state":"open","user":{"login":"tester"},"head":{"ref":"feature/mock"},"base":{"ref":"main"},"labels":[],"assignees":[],"html_url":"https://git.mosaicstack.dev/mosaicstack/stack/pulls/123","mergeable":true}'
|
||||
emit_response '{"number":123,"title":"mock","state":"open","user":{"login":"tester"},"head":{"ref":"feature/mock","sha":"0123456789abcdef0123456789abcdef01234567","repo":{"full_name":"mosaicstack/stack"}},"base":{"ref":"main"},"labels":[],"assignees":[],"html_url":"https://git.mosaicstack.dev/mosaicstack/stack/pulls/123","mergeable":true}'
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$args" == *"-X POST"* && "$args" == *"/api/v1/repos/mosaicstack/stack/pulls/123/merge"* ]]; then
|
||||
if [[ "$post_data" != '{"Do":"squash"}' ]]; then
|
||||
echo "unexpected merge payload: $post_data" >&2
|
||||
exit 96
|
||||
fi
|
||||
POST_DATA="$post_data" python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
payload = json.loads(os.environ["POST_DATA"])
|
||||
assert payload == {
|
||||
"Do": "squash",
|
||||
"head_commit_id": "0123456789abcdef0123456789abcdef01234567",
|
||||
}, payload
|
||||
PY
|
||||
emit_response '{"merged":true,"message":"mock merge complete"}'
|
||||
exit 0
|
||||
fi
|
||||
@@ -107,8 +116,8 @@ export GITEA_URL="https://git.mosaicstack.dev"
|
||||
export GITEA_TOKEN="redacted-test-token"
|
||||
|
||||
OUTPUT="$SANDBOX/output.log"
|
||||
if ! "$SCRIPT_DIR/pr-merge.sh" -n 123 -m squash --skip-queue-guard > "$OUTPUT" 2>&1; then
|
||||
echo "Expected pr-merge.sh to recover via Gitea API fallback." >&2
|
||||
if ! "$SCRIPT_DIR/pr-merge.sh" -n 123 -m squash > "$OUTPUT" 2>&1; then
|
||||
echo "Expected pr-merge.sh to use the exact-head Gitea API path." >&2
|
||||
echo "--- output ---" >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$OUTPUT" >&2
|
||||
echo "--- mock log ---" >&2
|
||||
@@ -127,38 +136,6 @@ if grep -q 'redacted-test-token' "$OUTPUT"; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cat > "$MOCK_BIN/tea" <<'EOF'
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
printf 'tea %q ' "$@" >> "$PR_MERGE_TEST_LOG"
|
||||
printf '\n' >> "$PR_MERGE_TEST_LOG"
|
||||
if [[ "$*" == *"login list"* ]]; then
|
||||
echo '[{"name":"git.mosaicstack.dev","url":"https://git.mosaicstack.dev"}]'
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$*" == *"pr merge"* ]]; then
|
||||
echo 'tea network timeout' >&2
|
||||
exit 2
|
||||
fi
|
||||
exit 0
|
||||
EOF
|
||||
chmod +x "$MOCK_BIN/tea"
|
||||
: > "$LOG_FILE"
|
||||
if "$SCRIPT_DIR/pr-merge.sh" -n 123 -m squash --skip-queue-guard > "$OUTPUT" 2>&1; then
|
||||
echo "Expected arbitrary tea failure to remain blocking." >&2
|
||||
exit 1
|
||||
fi
|
||||
if grep -q '/api/v1/repos/mosaicstack/stack/pulls/123/merge' "$LOG_FILE"; then
|
||||
echo "Arbitrary tea failure unexpectedly used Gitea API merge fallback." >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$LOG_FILE" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q 'tea network timeout' "$OUTPUT"; then
|
||||
echo "Expected arbitrary tea error to be preserved in output." >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$OUTPUT" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cat > "$MOCK_BIN/tea" <<'EOF'
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
@@ -177,8 +154,8 @@ EOF
|
||||
chmod +x "$MOCK_BIN/tea"
|
||||
unset GITEA_LOGIN
|
||||
: > "$LOG_FILE"
|
||||
if ! "$SCRIPT_DIR/pr-merge.sh" -n 123 -m squash --skip-queue-guard > "$OUTPUT" 2>&1; then
|
||||
echo "Expected missing tea login to use authenticated Gitea API fallback." >&2
|
||||
if ! "$SCRIPT_DIR/pr-merge.sh" -n 123 -m squash > "$OUTPUT" 2>&1; then
|
||||
echo "Expected the exact-head API path not to depend on a tea login." >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$OUTPUT" >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$LOG_FILE" >&2
|
||||
exit 1
|
||||
@@ -215,7 +192,7 @@ cd "$REPO_DIR"
|
||||
git remote set-url origin https://github.com/mosaicstack/stack.git
|
||||
: > "$LOG_FILE"
|
||||
rm -f "$SENTINEL"
|
||||
if "$SCRIPT_DIR/pr-merge.sh" -n "$INJECTION" -m squash --skip-queue-guard > "$OUTPUT" 2>&1; then
|
||||
if "$SCRIPT_DIR/pr-merge.sh" -n "$INJECTION" -m squash > "$OUTPUT" 2>&1; then
|
||||
echo "Expected GitHub metacharacter PR number to be rejected." >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$OUTPUT" >&2
|
||||
exit 1
|
||||
@@ -240,7 +217,7 @@ git remote set-url origin https://git.mosaicstack.dev/mosaicstack/stack.git
|
||||
export GITEA_LOGIN="git.mosaicstack.dev"
|
||||
: > "$LOG_FILE"
|
||||
rm -f "$SENTINEL"
|
||||
if "$SCRIPT_DIR/pr-merge.sh" -n "$INJECTION" -m squash --skip-queue-guard > "$OUTPUT" 2>&1; then
|
||||
if "$SCRIPT_DIR/pr-merge.sh" -n "$INJECTION" -m squash > "$OUTPUT" 2>&1; then
|
||||
echo "Expected Gitea metacharacter PR number to be rejected." >&2
|
||||
sed 's/redacted-test-token/***REDACTED***/g' "$OUTPUT" >&2
|
||||
exit 1
|
||||
@@ -260,4 +237,4 @@ if ! grep -q 'Invalid PR number' "$OUTPUT"; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "pr-merge.sh Gitea fallback regression passed"
|
||||
echo "pr-merge.sh Gitea exact-head API regression passed"
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env bash
|
||||
# shellcheck disable=SC2030,SC2031 # Provider arms isolate PATH/credentials in subshells.
|
||||
# The commit whose CI was guarded must be the commit the provider atomically merges.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/pr-merge-head-pin}"
|
||||
SHA=0123456789abcdef0123456789abcdef01234567
|
||||
|
||||
make_fixture() {
|
||||
local name="$1" remote="$2"
|
||||
local root="$WORK_DIR/$name"
|
||||
local tools="$root/tools/git"
|
||||
mkdir -p "$tools" "$root/repo"
|
||||
cp "$SCRIPT_DIR/pr-merge.sh" "$tools/pr-merge.sh"
|
||||
cp "$SCRIPT_DIR/detect-platform.sh" "$tools/detect-platform.sh"
|
||||
git -C "$root/repo" init -q
|
||||
git -C "$root/repo" remote add origin "$remote"
|
||||
cat > "$tools/pr-metadata.sh" <<SH
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' '{"baseRefName":"main","headRefName":"fix/pinned","headRefOid":"$SHA","headRepository":"contributor/widgets-fork"}'
|
||||
SH
|
||||
cat > "$tools/ci-queue-wait.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
exit 0
|
||||
SH
|
||||
chmod +x "$tools"/*.sh
|
||||
}
|
||||
|
||||
rm -rf "$WORK_DIR"
|
||||
make_fixture gitea https://git.example.test/acme/widgets.git
|
||||
make_fixture github https://github.com/acme/widgets.git
|
||||
|
||||
cat > "$WORK_DIR/gitea/curl" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
payload=""
|
||||
for ((i=1; i<=$#; i++)); do
|
||||
if [[ "${!i}" == "-d" ]]; then
|
||||
j=$((i + 1))
|
||||
payload="${!j}"
|
||||
fi
|
||||
done
|
||||
printf '%s' "$payload" > "${MOSAIC_MERGE_PAYLOAD_LOG:?}"
|
||||
printf '200'
|
||||
SH
|
||||
chmod +x "$WORK_DIR/gitea/curl"
|
||||
|
||||
set +e
|
||||
(
|
||||
cd "$WORK_DIR/gitea/repo"
|
||||
export PATH="$WORK_DIR/gitea:$PATH"
|
||||
export GITEA_TOKEN=stub-token
|
||||
export GITEA_URL=https://git.example.test
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
export MOSAIC_MERGE_PAYLOAD_LOG="$WORK_DIR/gitea-payload.json"
|
||||
env -u MOSAIC_GIT_IDENTITY "$WORK_DIR/gitea/tools/git/pr-merge.sh" -n 123
|
||||
) >"$WORK_DIR/gitea.out" 2>&1
|
||||
gitea_rc=$?
|
||||
set -e
|
||||
if [[ "$gitea_rc" -ne 0 ]]; then
|
||||
echo "FAIL gitea-pin: merge fixture returned $gitea_rc" >&2
|
||||
cat "$WORK_DIR/gitea.out" >&2
|
||||
exit 1
|
||||
fi
|
||||
python3 - "$WORK_DIR/gitea-payload.json" "$SHA" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
payload = json.load(open(sys.argv[1], encoding="utf-8"))
|
||||
assert set(payload) <= {"Do", "head_commit_id", "delete_branch_after_merge"}, payload
|
||||
assert payload.get("Do") == "squash", payload
|
||||
assert payload.get("head_commit_id") == sys.argv[2], payload
|
||||
PY
|
||||
|
||||
# A merge-gate verdict is commit-bound. A stale expected head must fail before merge.
|
||||
wrong_sha=ffffffffffffffffffffffffffffffffffffffff
|
||||
rm -f "$WORK_DIR/gitea-payload-stale.json"
|
||||
set +e
|
||||
(
|
||||
cd "$WORK_DIR/gitea/repo"
|
||||
export PATH="$WORK_DIR/gitea:$PATH"
|
||||
export GITEA_TOKEN=stub-token
|
||||
export GITEA_URL=https://git.example.test
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
export MOSAIC_MERGE_PAYLOAD_LOG="$WORK_DIR/gitea-payload-stale.json"
|
||||
env -u MOSAIC_GIT_IDENTITY "$WORK_DIR/gitea/tools/git/pr-merge.sh" -n 123 --expect-head "$wrong_sha"
|
||||
) >"$WORK_DIR/gitea-stale.out" 2>&1
|
||||
stale_rc=$?
|
||||
set -e
|
||||
if [[ "$stale_rc" -eq 0 ]] || [[ -e "$WORK_DIR/gitea-payload-stale.json" ]]; then
|
||||
echo "FAIL stale-verdict: moved head was not refused before provider merge" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A merge-capable path cannot bypass the mandatory queue guard. The legacy
|
||||
# --skip-queue-guard option must be rejected before any provider merge call.
|
||||
cat > "$WORK_DIR/gitea/tools/git/ci-queue-wait.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
exit 99
|
||||
SH
|
||||
chmod +x "$WORK_DIR/gitea/tools/git/ci-queue-wait.sh"
|
||||
rm -f "$WORK_DIR/gitea-payload-bypass.json"
|
||||
set +e
|
||||
(
|
||||
cd "$WORK_DIR/gitea/repo"
|
||||
export PATH="$WORK_DIR/gitea:$PATH"
|
||||
export GITEA_TOKEN=stub-token
|
||||
export GITEA_URL=https://git.example.test
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
export MOSAIC_MERGE_PAYLOAD_LOG="$WORK_DIR/gitea-payload-bypass.json"
|
||||
env -u MOSAIC_GIT_IDENTITY "$WORK_DIR/gitea/tools/git/pr-merge.sh" -n 123 --skip-queue-guard
|
||||
) >"$WORK_DIR/gitea-bypass.out" 2>&1
|
||||
bypass_rc=$?
|
||||
set -e
|
||||
if [[ "$bypass_rc" -eq 0 ]] || [[ -e "$WORK_DIR/gitea-payload-bypass.json" ]]; then
|
||||
echo "FAIL merge-bypass: --skip-queue-guard reached the provider merge path" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Dry-run is the only path that may omit the guard because it exits before the
|
||||
# provider merge dispatch. Prove the exit and absence of a merge payload.
|
||||
rm -f "$WORK_DIR/gitea-payload-dry-run.json"
|
||||
(
|
||||
cd "$WORK_DIR/gitea/repo"
|
||||
export PATH="$WORK_DIR/gitea:$PATH"
|
||||
export GITEA_TOKEN=stub-token
|
||||
export GITEA_URL=https://git.example.test
|
||||
export MOSAIC_CREDENTIALS_FILE="$WORK_DIR/no-credentials.json"
|
||||
export MOSAIC_MERGE_PAYLOAD_LOG="$WORK_DIR/gitea-payload-dry-run.json"
|
||||
env -u MOSAIC_GIT_IDENTITY "$WORK_DIR/gitea/tools/git/pr-merge.sh" -n 123 --dry-run
|
||||
) >"$WORK_DIR/gitea-dry-run.out" 2>&1
|
||||
if [[ -e "$WORK_DIR/gitea-payload-dry-run.json" ]]; then
|
||||
echo "FAIL dry-run: non-merging preflight reached the provider merge path" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cat > "$WORK_DIR/github/gh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
printf '%s\n' "$*" > "${MOSAIC_GH_MERGE_LOG:?}"
|
||||
SH
|
||||
chmod +x "$WORK_DIR/github/gh"
|
||||
(
|
||||
cd "$WORK_DIR/github/repo"
|
||||
export PATH="$WORK_DIR/github:$PATH"
|
||||
export MOSAIC_GH_MERGE_LOG="$WORK_DIR/github-call.log"
|
||||
"$WORK_DIR/github/tools/git/pr-merge.sh" -n 123
|
||||
) >"$WORK_DIR/github.out" 2>&1
|
||||
if ! grep -q -- "--match-head-commit $SHA" "$WORK_DIR/github-call.log"; then
|
||||
echo "FAIL github-pin: merge command omitted --match-head-commit $SHA" >&2
|
||||
cat "$WORK_DIR/github-call.log" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "PR merge exact-head pin regression passed (Gitea + GitHub)"
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
# RM-03: pr-merge must guard the PR head branch, not its main base branch.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORK_DIR="${MOSAIC_TEST_WORK_DIR:-$PWD/.mosaic-test-work/pr-merge-queue-branch}"
|
||||
FIXTURE_DIR="$WORK_DIR/tools/git"
|
||||
CALL_LOG="$WORK_DIR/queue-call.log"
|
||||
|
||||
rm -rf "$WORK_DIR"
|
||||
mkdir -p "$FIXTURE_DIR"
|
||||
cp "$SCRIPT_DIR/pr-merge.sh" "$FIXTURE_DIR/pr-merge.sh"
|
||||
cp "$SCRIPT_DIR/detect-platform.sh" "$FIXTURE_DIR/detect-platform.sh"
|
||||
|
||||
cat > "$FIXTURE_DIR/pr-metadata.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' '{"baseRefName":"main","headRefName":"fix/rm-03-fixture","headRefOid":"0123456789abcdef0123456789abcdef01234567","headRepository":"contributor/widgets-fork"}'
|
||||
SH
|
||||
|
||||
cat > "$FIXTURE_DIR/ci-queue-wait.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "$*" > "${MOSAIC_QUEUE_CALL_LOG:?}"
|
||||
exit 42
|
||||
SH
|
||||
chmod +x "$FIXTURE_DIR"/*.sh
|
||||
|
||||
set +e
|
||||
(
|
||||
cd "$WORK_DIR"
|
||||
export MOSAIC_QUEUE_CALL_LOG="$CALL_LOG"
|
||||
"$FIXTURE_DIR/pr-merge.sh" -n 123
|
||||
) >/dev/null 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
|
||||
if [[ "$rc" -ne 42 ]]; then
|
||||
echo "FAIL: expected queue stub rc=42 to propagate, got $rc" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ ! -s "$CALL_LOG" ]]; then
|
||||
echo "FAIL: merge wrapper did not invoke the queue guard" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q -- '-B fix/rm-03-fixture' "$CALL_LOG"; then
|
||||
echo "FAIL: merge queue guard did not receive PR head branch" >&2
|
||||
cat "$CALL_LOG" >&2
|
||||
exit 1
|
||||
fi
|
||||
if grep -q -- '-B main' "$CALL_LOG"; then
|
||||
echo "FAIL: merge queue guard still received the main base branch" >&2
|
||||
cat "$CALL_LOG" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q -- '-R contributor/widgets-fork' "$CALL_LOG"; then
|
||||
echo "FAIL: merge queue guard did not receive the fork head repository" >&2
|
||||
cat "$CALL_LOG" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q -- '--sha 0123456789abcdef0123456789abcdef01234567' "$CALL_LOG"; then
|
||||
echo "FAIL: merge queue guard did not receive the exact PR head SHA" >&2
|
||||
cat "$CALL_LOG" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "pr-merge queue branch/repository/SHA regression passed"
|
||||
@@ -26,12 +26,13 @@ A Woodpecker API token is required. To configure:
|
||||
|
||||
## Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
| --------------------- | -------------------------------------------- |
|
||||
| `pipeline-list.sh` | List recent pipelines for a repo |
|
||||
| `pipeline-status.sh` | Get status of a specific or latest pipeline |
|
||||
| `pipeline-trigger.sh` | Trigger a new pipeline build |
|
||||
| `ci-wait.sh` | Block until pipeline(s) reach terminal state |
|
||||
| Script | Purpose |
|
||||
| -------------------------- | -------------------------------------------------------------- |
|
||||
| `pipeline-list.sh` | List recent pipelines for a repo |
|
||||
| `pipeline-status.sh` | Get status of a specific or latest pipeline |
|
||||
| `pipeline-trigger.sh` | Trigger a new pipeline build |
|
||||
| `ci-wait.sh` | Block until pipeline(s) reach terminal state |
|
||||
| `verify-terminal-green.py` | Verify every JSON/API child step under the bounded CI contract |
|
||||
|
||||
## Common Options
|
||||
|
||||
@@ -59,4 +60,9 @@ A Woodpecker API token is required. To configure:
|
||||
|
||||
# Block until one or more pipelines finish (event-driven CI wait)
|
||||
~/.config/mosaic/tools/woodpecker/ci-wait.sh -r usc/uconnect -n 3917 -n 3918
|
||||
|
||||
# Verify the full JSON child-step record; do not use the text summary for this gate
|
||||
PR_HEAD=<full-40-hex-provider-head>
|
||||
~/.config/mosaic/tools/woodpecker/pipeline-status.sh -r mosaicstack/stack -n 2188 -f json \
|
||||
| ~/.config/mosaic/tools/woodpecker/verify-terminal-green.py --expect-commit "$PR_HEAD" -
|
||||
```
|
||||
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env bash
|
||||
# Red-first contract harness for RM-61 / #1000.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VERIFIER="$SCRIPT_DIR/verify-terminal-green.py"
|
||||
EXPECTED_COMMIT=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
TMP=$(mktemp -d)
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
write_fixture() {
|
||||
local file="$1" pipeline_status="$2" postgres_state="$3" postgres_exit="$4" postgres_error="$5" test_state="$6"
|
||||
python3 - "$file" "$pipeline_status" "$postgres_state" "$postgres_exit" "$postgres_error" "$test_state" <<'PY'
|
||||
import json, sys
|
||||
path, pipeline_status, pg_state, pg_exit, pg_error, test_state = sys.argv[1:]
|
||||
steps = [
|
||||
{"name": "clone", "type": "clone", "state": "success", "exit_code": 0, "error": None},
|
||||
{"name": "ci-postgres", "type": "service", "state": pg_state, "exit_code": int(pg_exit), "error": pg_error or None},
|
||||
{"name": "test", "type": "commands", "state": test_state, "exit_code": 0 if test_state == "success" else 1, "error": None},
|
||||
]
|
||||
json.dump({
|
||||
"number": 9999,
|
||||
"status": pipeline_status,
|
||||
"commit": "a" * 40,
|
||||
"workflows": [{"name": "ci", "state": pipeline_status, "children": steps}],
|
||||
}, open(path, "w"))
|
||||
PY
|
||||
}
|
||||
|
||||
expect_exit() {
|
||||
local expected_exit="$1" label="$2" file="$3" expected_commit="${4:-$EXPECTED_COMMIT}"
|
||||
set +e
|
||||
output=$(python3 "$VERIFIER" --expect-commit "$expected_commit" "$file" 2>&1)
|
||||
actual=$?
|
||||
set -e
|
||||
if [[ "$actual" -ne "$expected_exit" ]]; then
|
||||
printf 'FAIL %s: expected exit %s, got %s\n%s\n' "$label" "$expected_exit" "$actual" "$output" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'PASS %s\n' "$label"
|
||||
printf '%s' "$output"
|
||||
}
|
||||
|
||||
# Ordinary terminal green.
|
||||
write_fixture "$TMP/green.json" success success 0 '' success
|
||||
out=$(expect_exit 0 green "$TMP/green.json")
|
||||
grep -q '"total_steps": 3' <<<"$out"
|
||||
grep -q '"exempted_steps": 0' <<<"$out"
|
||||
|
||||
# Exact, named #1000 teardown artifact: the only permitted non-success child.
|
||||
artifact='pods "wp-svc-01kyxzjhdf6w81swsnbfzh85z9-ci-postgres" not found'
|
||||
write_fixture "$TMP/artifact.json" success failure 0 "$artifact" success
|
||||
out=$(expect_exit 0 exact-artifact "$TMP/artifact.json")
|
||||
grep -q '"exemption_id": "WP-K8S-1000-CI-POSTGRES-TEARDOWN"' <<<"$out"
|
||||
grep -q '"exempted_steps": 1' <<<"$out"
|
||||
|
||||
# Negative controls: both real PostgreSQL failures must remain red.
|
||||
write_fixture "$TMP/startup.json" failure failure 1 '' failure
|
||||
expect_exit 1 startup-failure "$TMP/startup.json" >/dev/null
|
||||
write_fixture "$TMP/crash.json" failure failure 137 '' failure
|
||||
expect_exit 1 post-readiness-crash "$TMP/crash.json" >/dev/null
|
||||
|
||||
# The exemption is signature-scoped, not step-scoped.
|
||||
write_fixture "$TMP/wrong-error.json" success failure 0 'connection refused' success
|
||||
expect_exit 1 other-postgres-error "$TMP/wrong-error.json" >/dev/null
|
||||
write_fixture "$TMP/wrong-pod.json" success failure 0 'pods "other-ci-postgres" not found' success
|
||||
expect_exit 1 wrong-pod-signature "$TMP/wrong-pod.json" >/dev/null
|
||||
write_fixture "$TMP/nonzero-artifact.json" success failure 137 "$artifact" success
|
||||
expect_exit 1 nonzero-with-artifact-text "$TMP/nonzero-artifact.json" >/dev/null
|
||||
|
||||
# JSON booleans and non-integer zero look equal to 0 in Python but are not exit codes.
|
||||
python3 - "$TMP/artifact.json" "$TMP" <<'PY'
|
||||
import json, os, sys
|
||||
record = json.load(open(sys.argv[1]))
|
||||
for label, value in (("false", False), ("true", True), ("float", 0.0), ("string", "0"), ("null", None)):
|
||||
changed = json.loads(json.dumps(record))
|
||||
changed["workflows"][0]["children"][1]["exit_code"] = value
|
||||
json.dump(changed, open(os.path.join(sys.argv[2], f"exit-{label}.json"), "w"))
|
||||
PY
|
||||
for label in false true float string null; do
|
||||
expect_exit 1 "non-integer-exit-$label" "$TMP/exit-$label.json" >/dev/null
|
||||
done
|
||||
|
||||
# Exact artifact cannot mask any independent failure or non-success pipeline.
|
||||
write_fixture "$TMP/artifact-plus-failure.json" failure failure 0 "$artifact" failure
|
||||
expect_exit 1 artifact-plus-real-failure "$TMP/artifact-plus-failure.json" >/dev/null
|
||||
write_fixture "$TMP/skipped.json" success success 0 '' skipped
|
||||
expect_exit 1 skipped-step "$TMP/skipped.json" >/dev/null
|
||||
|
||||
# The scanned pipeline must be bound to an explicit, full PR-head commit.
|
||||
set +e
|
||||
missing_output=$(python3 "$VERIFIER" "$TMP/artifact.json" 2>&1)
|
||||
missing_rc=$?
|
||||
set -e
|
||||
if [[ "$missing_rc" -ne 2 ]] || ! grep -q -- '--expect-commit' <<<"$missing_output"; then
|
||||
printf 'FAIL missing-expected-commit: expected usage exit 2\n%s\n' "$missing_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
expect_exit 1 mismatched-expected-commit "$TMP/artifact.json" bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb >/dev/null
|
||||
|
||||
python3 - "$TMP/artifact.json" "$TMP/missing-record-commit.json" <<'PY'
|
||||
import json, sys
|
||||
record = json.load(open(sys.argv[1]))
|
||||
record.pop("commit")
|
||||
json.dump(record, open(sys.argv[2], "w"))
|
||||
PY
|
||||
expect_exit 1 missing-record-commit "$TMP/missing-record-commit.json" >/dev/null
|
||||
|
||||
printf 'terminal-green contract harness: PASS (17 cases)\n'
|
||||
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Verify Mosaic's full-step Woodpecker terminal-green contract.
|
||||
|
||||
RM-61 permits one named, signature-scoped exception for issue #1000. The
|
||||
exception retires when #1000 is fixed; all other non-success states block.
|
||||
This program consumes the JSON/API record emitted by pipeline-status.sh -f json.
|
||||
It does not fetch, retry, or re-trigger pipelines.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
EXEMPTION_ID = "WP-K8S-1000-CI-POSTGRES-TEARDOWN"
|
||||
EXEMPTION_ISSUE = "https://git.mosaicstack.dev/mosaicstack/stack/issues/1000"
|
||||
POD_NOT_FOUND = re.compile(
|
||||
r'^pods "wp-svc-[0-9a-hjkmnp-tv-z]{26}-ci-postgres" not found$'
|
||||
)
|
||||
|
||||
|
||||
def fail_usage(message: str) -> int:
|
||||
print(f"terminal-green contract input error: {message}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
def load_record(argument: str | None) -> dict[str, Any]:
|
||||
if argument in (None, "-"):
|
||||
value = json.load(sys.stdin)
|
||||
else:
|
||||
with Path(argument).open(encoding="utf-8") as handle:
|
||||
value = json.load(handle)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("pipeline record must be a JSON object")
|
||||
return value
|
||||
|
||||
|
||||
def is_issue_1000_artifact(step: dict[str, Any]) -> bool:
|
||||
error = step.get("error")
|
||||
exit_code = step.get("exit_code")
|
||||
return (
|
||||
step.get("name") == "ci-postgres"
|
||||
and step.get("type") == "service"
|
||||
and step.get("state") == "failure"
|
||||
and type(exit_code) is int
|
||||
and not isinstance(exit_code, bool)
|
||||
and exit_code == 0
|
||||
and isinstance(error, str)
|
||||
and POD_NOT_FOUND.fullmatch(error) is not None
|
||||
)
|
||||
|
||||
|
||||
def verify(record: dict[str, Any], expected_commit: str) -> tuple[int, dict[str, Any]]:
|
||||
anomalies: list[dict[str, Any]] = []
|
||||
candidates: list[dict[str, Any]] = []
|
||||
steps: list[dict[str, Any]] = []
|
||||
|
||||
pipeline_status = record.get("status")
|
||||
actual_commit = record.get("commit")
|
||||
if actual_commit != expected_commit:
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "pipeline",
|
||||
"name": str(record.get("number", "unknown")),
|
||||
"state": pipeline_status,
|
||||
"reason": "pipeline commit does not equal the expected PR head",
|
||||
"expected_commit": expected_commit,
|
||||
"actual_commit": actual_commit,
|
||||
}
|
||||
)
|
||||
if pipeline_status != "success":
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "pipeline",
|
||||
"name": str(record.get("number", "unknown")),
|
||||
"state": pipeline_status,
|
||||
"reason": "pipeline status is not success",
|
||||
}
|
||||
)
|
||||
|
||||
workflows = record.get("workflows")
|
||||
if not isinstance(workflows, list) or not workflows:
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "pipeline",
|
||||
"name": str(record.get("number", "unknown")),
|
||||
"state": pipeline_status,
|
||||
"reason": "workflows are missing or empty",
|
||||
}
|
||||
)
|
||||
workflows = []
|
||||
|
||||
for workflow_index, workflow in enumerate(workflows):
|
||||
if not isinstance(workflow, dict):
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "workflow",
|
||||
"name": str(workflow_index),
|
||||
"state": None,
|
||||
"reason": "workflow is not an object",
|
||||
}
|
||||
)
|
||||
continue
|
||||
workflow_name = str(workflow.get("name", workflow_index))
|
||||
if workflow.get("state") != "success":
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "workflow",
|
||||
"name": workflow_name,
|
||||
"state": workflow.get("state"),
|
||||
"reason": "workflow state is not success",
|
||||
}
|
||||
)
|
||||
children = workflow.get("children")
|
||||
if not isinstance(children, list) or not children:
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "workflow",
|
||||
"name": workflow_name,
|
||||
"state": workflow.get("state"),
|
||||
"reason": "child-step list is missing or empty",
|
||||
}
|
||||
)
|
||||
continue
|
||||
for child_index, child in enumerate(children):
|
||||
if not isinstance(child, dict):
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "step",
|
||||
"name": f"{workflow_name}[{child_index}]",
|
||||
"state": None,
|
||||
"reason": "step is not an object",
|
||||
}
|
||||
)
|
||||
continue
|
||||
steps.append(child)
|
||||
if child.get("state") == "success":
|
||||
continue
|
||||
if is_issue_1000_artifact(child):
|
||||
candidates.append(child)
|
||||
continue
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "step",
|
||||
"name": child.get("name"),
|
||||
"type": child.get("type"),
|
||||
"state": child.get("state"),
|
||||
"exit_code": child.get("exit_code"),
|
||||
"error": child.get("error"),
|
||||
"reason": "non-success step does not match the #1000 teardown signature",
|
||||
}
|
||||
)
|
||||
|
||||
if len(candidates) > 1:
|
||||
anomalies.append(
|
||||
{
|
||||
"scope": "exemption",
|
||||
"name": EXEMPTION_ID,
|
||||
"state": "invalid",
|
||||
"reason": "the #1000 exemption may apply to exactly one step",
|
||||
}
|
||||
)
|
||||
|
||||
exemption_applies = len(candidates) == 1 and not anomalies
|
||||
state_counts = Counter(str(step.get("state", "missing")) for step in steps)
|
||||
result: dict[str, Any] = {
|
||||
"schema_version": "mosaic-terminal-green/v1",
|
||||
"verdict": "terminal-green" if not anomalies else "not-terminal-green",
|
||||
"pipeline_number": record.get("number"),
|
||||
"commit": actual_commit,
|
||||
"expected_commit": expected_commit,
|
||||
"pipeline_status": pipeline_status,
|
||||
"total_steps": len(steps),
|
||||
"state_counts": dict(sorted(state_counts.items())),
|
||||
"exempted_steps": 1 if exemption_applies else 0,
|
||||
"anomalies": anomalies,
|
||||
}
|
||||
if exemption_applies:
|
||||
candidate = candidates[0]
|
||||
result["exemptions"] = [
|
||||
{
|
||||
"exemption_id": EXEMPTION_ID,
|
||||
"step": candidate.get("name"),
|
||||
"signature": candidate.get("error"),
|
||||
"tracking_issue": EXEMPTION_ISSUE,
|
||||
"retires_when": "issue #1000 is fixed",
|
||||
}
|
||||
]
|
||||
else:
|
||||
result["exemptions"] = []
|
||||
|
||||
return (0 if not anomalies else 1), result
|
||||
|
||||
|
||||
def parse_arguments() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="verify the full Woodpecker terminal-green child-step contract"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--expect-commit",
|
||||
required=True,
|
||||
metavar="FULL_SHA",
|
||||
help="full 40-hex PR-head commit that the pipeline record must match",
|
||||
)
|
||||
parser.add_argument("record", nargs="?", default="-", help="pipeline JSON file or -")
|
||||
arguments = parser.parse_args()
|
||||
if re.fullmatch(r"[0-9a-fA-F]{40}", arguments.expect_commit) is None:
|
||||
parser.error("--expect-commit must be a full 40-hex commit")
|
||||
arguments.expect_commit = arguments.expect_commit.lower()
|
||||
return arguments
|
||||
|
||||
|
||||
def main() -> int:
|
||||
arguments = parse_arguments()
|
||||
try:
|
||||
record = load_record(arguments.record)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as error:
|
||||
return fail_usage(str(error))
|
||||
code, result = verify(record, arguments.expect_commit)
|
||||
print(json.dumps(result, indent=2, sort_keys=True))
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -25,7 +25,7 @@
|
||||
"lint": "eslint src",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run --passWithNoTests && pnpm run test:framework-shell",
|
||||
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
"test:framework-shell": "bash framework/tools/quality/scripts/check-test-enumeration.sh && bash framework/tools/quality/scripts/test-check-test-enumeration.sh && python3 src/lease-broker/daemon_deadline_unittest.py && python3 src/lease-broker/normative_fragments_unittest.py && python3 src/lease-broker/receipt_challenge_unittest.py && python3 src/lease-broker/context_recovery_unittest.py && python3 src/lease-broker/recovery_runtime_unittest.py && python3 src/lease-broker/recovery_b1_adversarial_unittest.py && python3 src/lease-broker/framework_skill_portability_unittest.py && python3 src/mutator-gate/runtime_tools_unittest.py && python3 src/mutator-gate/runtime_launch_guard_unittest.py && python3 src/mutator-gate/version_coupling_unittest.py && python3 framework/tools/lease-broker/check-runtime-launches.py --root ../.. && bash framework/tools/codex/test-pr-diff-context.sh && bash framework/tools/qa/test-deps-preflight.sh && bash framework/tools/git/test-pr-review-gitea-comment.sh && bash framework/tools/git/test-pr-review-repo-host-override.sh && bash framework/tools/git/test-ci-queue-wait-branch-absent.sh && bash framework/tools/git/test-ci-queue-wait-tristate.sh && bash framework/tools/git/test-ci-queue-wait-github-checks.sh && bash framework/tools/git/test-pr-merge-queue-branch.sh && bash framework/tools/git/test-pr-merge-head-pin.sh && bash framework/tools/git/test-git-credential-mosaic.sh && bash framework/tools/git/test-gitea-token-identity.sh && bash framework/tools/woodpecker/test-terminal-green-contract.sh && bash framework/tools/_scripts/test-install-ordering-guard.sh && bash framework/tools/tmux/agent-send.test.sh && bash framework/tools/wake/test-wake-store-ack.sh && bash framework/tools/wake/test-wake-store-enqueue-race.sh && bash framework/tools/wake/test-wake-digest-hmac.sh && bash framework/tools/wake/test-wake-digest-quarantine.sh && bash framework/tools/wake/test-wake-detector.sh && bash framework/tools/wake/test-wake-fn-oracle.sh && bash framework/tools/wake/test-wake-reconcile.sh && bash framework/tools/wake/test-wake-beacon.sh && bash framework/tools/wake/test-wake-preimage.sh && bash framework/tools/wake/test-wake-install.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mosaicstack/brain": "workspace:*",
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { spawn } from 'node:child_process';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { lstat, mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import path from 'node:path';
|
||||
|
||||
import { generatedSymlinkManifest, sourceFingerprint } from './preflight.mjs';
|
||||
|
||||
const scriptRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
||||
function run(command, args, options) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(command, args, options);
|
||||
child.once('error', reject);
|
||||
child.once('exit', (code, signal) => {
|
||||
if (code === 0) resolve();
|
||||
else
|
||||
reject(
|
||||
new Error(signal ? `next build terminated by ${signal}` : `next build exited ${code}`),
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
const delay = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||
|
||||
async function requireRealDirectory(target, { allowMissing = false } = {}) {
|
||||
try {
|
||||
const stats = await lstat(target);
|
||||
if (!stats.isDirectory() || stats.isSymbolicLink()) {
|
||||
throw new Error(`${target} must be a real directory, not a symbolic link.`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (allowMissing && error.code === 'ENOENT') return;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function acquireBuildLock(root) {
|
||||
const workRoot = path.join(root, '.mosaic-test-work');
|
||||
const lock = path.join(workRoot, 'web-build.lock');
|
||||
const nonce = randomUUID();
|
||||
const owner = JSON.stringify({ pid: process.pid, nonce });
|
||||
const deadline = Date.now() + 120_000;
|
||||
await mkdir(workRoot, { recursive: true });
|
||||
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
await mkdir(lock);
|
||||
await writeFile(path.join(lock, 'owner.json'), owner, { mode: 0o600 });
|
||||
return async () => {
|
||||
const current = await readFile(path.join(lock, 'owner.json'), 'utf8');
|
||||
if (current !== owner) throw new Error('Web build lock ownership changed before release.');
|
||||
const released = `${lock}.released-${nonce}`;
|
||||
await rename(lock, released);
|
||||
await rm(released, { recursive: true, force: true });
|
||||
};
|
||||
} catch (error) {
|
||||
if (error.code !== 'EEXIST') throw error;
|
||||
let lockOwner;
|
||||
try {
|
||||
lockOwner = JSON.parse(await readFile(path.join(lock, 'owner.json'), 'utf8'));
|
||||
} catch (ownerError) {
|
||||
if (ownerError.code === 'ENOENT') {
|
||||
await delay(25);
|
||||
continue;
|
||||
}
|
||||
throw new Error(`Web build lock is unreadable at ${lock}.`, { cause: ownerError });
|
||||
}
|
||||
try {
|
||||
process.kill(lockOwner.pid, 0);
|
||||
} catch (processError) {
|
||||
if (processError.code !== 'ESRCH') throw processError;
|
||||
const stale = `${lock}.stale-${nonce}`;
|
||||
try {
|
||||
await rename(lock, stale);
|
||||
await rm(stale, { recursive: true, force: true });
|
||||
} catch (renameError) {
|
||||
if (renameError.code !== 'ENOENT') throw renameError;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
await delay(25);
|
||||
}
|
||||
}
|
||||
throw new Error(`Timed out waiting for the web build lock at ${lock}.`);
|
||||
}
|
||||
|
||||
export async function buildWeb({
|
||||
root = scriptRoot,
|
||||
fingerprint = sourceFingerprint,
|
||||
runBuild = async (webDir) =>
|
||||
run(path.join(webDir, 'node_modules', '.bin', 'next'), ['build'], {
|
||||
cwd: webDir,
|
||||
stdio: 'inherit',
|
||||
}),
|
||||
} = {}) {
|
||||
const releaseLock = await acquireBuildLock(root);
|
||||
try {
|
||||
const webDir = path.join(root, 'apps', 'web');
|
||||
const nextDir = path.join(webDir, '.next');
|
||||
const certificationMarker = path.join(nextDir, '.mosaic-source-hash');
|
||||
const symlinkManifest = path.join(nextDir, '.mosaic-symlink-manifest');
|
||||
const certificationTemporary = `${certificationMarker}.${randomUUID()}.tmp`;
|
||||
const manifestTemporary = `${symlinkManifest}.${randomUUID()}.tmp`;
|
||||
const before = await fingerprint(root);
|
||||
|
||||
await requireRealDirectory(nextDir, { allowMissing: true });
|
||||
await Promise.all([
|
||||
rm(certificationMarker, { force: true }),
|
||||
rm(symlinkManifest, { force: true }),
|
||||
]);
|
||||
await runBuild(webDir);
|
||||
await requireRealDirectory(nextDir);
|
||||
|
||||
const after = await fingerprint(root);
|
||||
if (after !== before) {
|
||||
throw new Error(
|
||||
'Web build inputs changed during next build; generated output was not certified.',
|
||||
);
|
||||
}
|
||||
|
||||
const manifestContents = await generatedSymlinkManifest(nextDir);
|
||||
const certificationContents = `${JSON.stringify({
|
||||
version: 1,
|
||||
sourceFingerprint: before,
|
||||
symlinkManifestHash: createHash('sha256').update(manifestContents).digest('hex'),
|
||||
})}\n`;
|
||||
await Promise.all([
|
||||
writeFile(certificationTemporary, certificationContents, { mode: 0o600 }),
|
||||
writeFile(manifestTemporary, manifestContents, { mode: 0o600 }),
|
||||
]);
|
||||
// The certification marker is the commit point. Publishing the manifest first
|
||||
// leaves interrupted builds untrusted because the marker remains absent.
|
||||
await rename(manifestTemporary, symlinkManifest);
|
||||
await rename(certificationTemporary, certificationMarker);
|
||||
} finally {
|
||||
await releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
await buildWeb();
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { access, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import { buildWeb } from './build-web.mjs';
|
||||
|
||||
const fixtureRoot = path.join(process.cwd(), '.mosaic-test-work', `build-web-${process.pid}`);
|
||||
|
||||
async function fixture(name) {
|
||||
const root = path.join(fixtureRoot, name);
|
||||
await mkdir(path.join(root, 'apps', 'web', '.next'), { recursive: true });
|
||||
return root;
|
||||
}
|
||||
|
||||
async function exists(target) {
|
||||
try {
|
||||
await access(target);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('a successful web build atomically publishes its source and symlink certification', async () => {
|
||||
const root = await fixture('success');
|
||||
const marker = path.join(root, 'apps', 'web', '.next', '.mosaic-source-hash');
|
||||
const manifest = path.join(root, 'apps', 'web', '.next', '.mosaic-symlink-manifest');
|
||||
|
||||
await buildWeb({ root, fingerprint: async () => 'certified', runBuild: async () => {} });
|
||||
|
||||
assert.deepEqual(JSON.parse(await readFile(marker, 'utf8')), {
|
||||
version: 1,
|
||||
sourceFingerprint: 'certified',
|
||||
symlinkManifestHash: '8a5a375cea6a55d24bd5f875856da63feba33adbefb15a92a0007719b84bcf11',
|
||||
});
|
||||
assert.equal(await readFile(manifest, 'utf8'), '{"version":1,"links":[]}\n');
|
||||
});
|
||||
|
||||
test('a failed web build leaves no certification marker', async () => {
|
||||
const root = await fixture('failure');
|
||||
const marker = path.join(root, 'apps', 'web', '.next', '.mosaic-source-hash');
|
||||
const manifest = path.join(root, 'apps', 'web', '.next', '.mosaic-symlink-manifest');
|
||||
await writeFile(marker, 'stale\n');
|
||||
await writeFile(manifest, 'stale\n');
|
||||
|
||||
await assert.rejects(
|
||||
buildWeb({
|
||||
root,
|
||||
fingerprint: async () => 'before',
|
||||
runBuild: async () => {
|
||||
throw new Error('build failed');
|
||||
},
|
||||
}),
|
||||
/build failed/,
|
||||
);
|
||||
|
||||
assert.equal(await exists(marker), false);
|
||||
assert.equal(await exists(manifest), false);
|
||||
});
|
||||
|
||||
test('overlapping web builds are serialized while the marker remains absent', async () => {
|
||||
const root = await fixture('overlap');
|
||||
const marker = path.join(root, 'apps', 'web', '.next', '.mosaic-source-hash');
|
||||
const manifest = path.join(root, 'apps', 'web', '.next', '.mosaic-symlink-manifest');
|
||||
await writeFile(marker, 'stale\n');
|
||||
await writeFile(manifest, 'stale\n');
|
||||
let releaseFirst;
|
||||
let secondEntered = false;
|
||||
const firstEntered = new Promise((resolve) => {
|
||||
releaseFirst = resolve;
|
||||
});
|
||||
let markFirstEntered;
|
||||
const firstStarted = new Promise((resolve) => {
|
||||
markFirstEntered = resolve;
|
||||
});
|
||||
|
||||
const first = buildWeb({
|
||||
root,
|
||||
fingerprint: async () => 'certified',
|
||||
runBuild: async () => {
|
||||
markFirstEntered();
|
||||
await firstEntered;
|
||||
},
|
||||
});
|
||||
await firstStarted;
|
||||
const second = buildWeb({
|
||||
root,
|
||||
fingerprint: async () => 'certified',
|
||||
runBuild: async () => {
|
||||
secondEntered = true;
|
||||
},
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 75));
|
||||
assert.equal(secondEntered, false);
|
||||
assert.equal(await exists(marker), false);
|
||||
assert.equal(await exists(manifest), false);
|
||||
|
||||
releaseFirst();
|
||||
await Promise.all([first, second]);
|
||||
assert.equal(secondEntered, true);
|
||||
assert.equal(JSON.parse(await readFile(marker, 'utf8')).sourceFingerprint, 'certified');
|
||||
assert.equal(await readFile(manifest, 'utf8'), '{"version":1,"links":[]}\n');
|
||||
});
|
||||
|
||||
test('a build that replaces .next with a symbolic link cannot publish outside the checkout', async () => {
|
||||
const root = await fixture('symbolic-next');
|
||||
const nextDir = path.join(root, 'apps', 'web', '.next');
|
||||
const outside = path.join(root, 'outside-generated');
|
||||
await mkdir(outside);
|
||||
|
||||
await assert.rejects(
|
||||
buildWeb({
|
||||
root,
|
||||
fingerprint: async () => 'certified',
|
||||
runBuild: async () => {
|
||||
await rm(nextDir, { recursive: true });
|
||||
await symlink(outside, nextDir);
|
||||
},
|
||||
}),
|
||||
/must be a real directory/,
|
||||
);
|
||||
|
||||
assert.equal(await exists(path.join(outside, '.mosaic-source-hash')), false);
|
||||
assert.equal(await exists(path.join(outside, '.mosaic-symlink-manifest')), false);
|
||||
});
|
||||
|
||||
test('inputs changed during a web build are not certified', async () => {
|
||||
const root = await fixture('changed-inputs');
|
||||
const marker = path.join(root, 'apps', 'web', '.next', '.mosaic-source-hash');
|
||||
const manifest = path.join(root, 'apps', 'web', '.next', '.mosaic-symlink-manifest');
|
||||
const fingerprints = ['before', 'after'];
|
||||
|
||||
await assert.rejects(
|
||||
buildWeb({
|
||||
root,
|
||||
fingerprint: async () => fingerprints.shift(),
|
||||
runBuild: async () => {},
|
||||
}),
|
||||
/inputs changed during next build/,
|
||||
);
|
||||
|
||||
assert.equal(await exists(marker), false);
|
||||
assert.equal(await exists(manifest), false);
|
||||
});
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { access, mkdir, rename, rm } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const root = process.cwd();
|
||||
const generated = path.join(root, 'apps', 'web', '.next');
|
||||
const quarantineRoot = path.join(root, '.mosaic-test-work', 'generated-quarantine');
|
||||
|
||||
try {
|
||||
await access(generated);
|
||||
} catch (error) {
|
||||
if (error.code === 'ENOENT') process.exit(0);
|
||||
throw error;
|
||||
}
|
||||
|
||||
await mkdir(quarantineRoot, { recursive: true });
|
||||
const quarantine = path.join(quarantineRoot, `web-next-${Date.now()}-${process.pid}`);
|
||||
try {
|
||||
await rename(generated, quarantine);
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`MOSAIC_GENERATED_CLEAN_FAILED: could not quarantine apps/web/.next. Fix: sudo rm -rf '${generated}', then rerun pnpm preflight`,
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
|
||||
try {
|
||||
await rm(quarantine, { recursive: true, force: true });
|
||||
} catch {
|
||||
console.warn(
|
||||
`Generated state was deactivated but could not be deleted; quarantined at ${quarantine}`,
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { access, lstat, mkdir, readFile, readdir, rename, rm } from 'node:fs/promises';
|
||||
import { execFile, spawn } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import path from 'node:path';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
function run(command, args, options) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(command, args, options);
|
||||
child.once('error', reject);
|
||||
child.once('exit', (code, signal) => {
|
||||
if (code === 0) resolve();
|
||||
else reject(new Error(signal ? `husky terminated by ${signal}` : `husky exited ${code}`));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function pathExists(target) {
|
||||
try {
|
||||
await access(target);
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (error.code === 'ENOENT') return false;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function directorySnapshot(root) {
|
||||
const snapshot = [];
|
||||
async function walk(current) {
|
||||
const children = await readdir(current, { withFileTypes: true });
|
||||
for (const child of children.sort((left, right) => left.name.localeCompare(right.name))) {
|
||||
const target = path.join(current, child.name);
|
||||
const relative = path.relative(root, target);
|
||||
const stats = await lstat(target);
|
||||
if (child.isDirectory()) {
|
||||
snapshot.push([relative, 'directory', stats.mode & 0o777]);
|
||||
await walk(target);
|
||||
} else {
|
||||
snapshot.push([
|
||||
relative,
|
||||
'file',
|
||||
stats.mode & 0o777,
|
||||
(await readFile(target)).toString('base64'),
|
||||
]);
|
||||
}
|
||||
}
|
||||
}
|
||||
await walk(root);
|
||||
return JSON.stringify(snapshot);
|
||||
}
|
||||
|
||||
async function directoriesMatch(left, right) {
|
||||
return (await directorySnapshot(left)) === (await directorySnapshot(right));
|
||||
}
|
||||
|
||||
export async function installHooks({
|
||||
root = process.cwd(),
|
||||
disabled = process.env.HUSKY === '0',
|
||||
quarantineRoot = path.join(root, '.mosaic-test-work', 'husky-quarantine'),
|
||||
runHusky = async (_stagingHooks, stagingRepo) => {
|
||||
await execFileAsync('git', ['init', '--quiet', stagingRepo]);
|
||||
await run(path.join(root, 'node_modules', '.bin', 'husky'), ['.husky'], {
|
||||
cwd: stagingRepo,
|
||||
stdio: 'inherit',
|
||||
});
|
||||
},
|
||||
activateHooks = async () => {
|
||||
await run('git', ['config', 'core.hooksPath', '.husky/_'], { cwd: root, stdio: 'inherit' });
|
||||
},
|
||||
} = {}) {
|
||||
if (disabled) return;
|
||||
|
||||
const huskyDir = path.join(root, '.husky');
|
||||
const active = path.join(huskyDir, '_');
|
||||
const nonce = `${Date.now()}-${process.pid}`;
|
||||
const stagingRepo = path.join(root, '.mosaic-test-work', `husky-stage-${nonce}`);
|
||||
const stagingHooks = path.join(stagingRepo, '.husky');
|
||||
const quarantined = path.join(quarantineRoot, `${path.basename(root)}-${nonce}`);
|
||||
await mkdir(huskyDir, { recursive: true });
|
||||
await mkdir(quarantineRoot, { recursive: true });
|
||||
|
||||
const previousComplete = (await pathExists(active)) && (await pathExists(path.join(active, 'h')));
|
||||
let previousQuarantined = false;
|
||||
try {
|
||||
if ((await pathExists(active)) && !previousComplete) {
|
||||
await rename(active, quarantined);
|
||||
previousQuarantined = true;
|
||||
}
|
||||
await mkdir(stagingRepo, { recursive: true });
|
||||
await runHusky(stagingHooks, stagingRepo);
|
||||
const staged = path.join(stagingHooks, '_');
|
||||
if (!(await pathExists(path.join(staged, 'h')))) {
|
||||
throw new Error('husky did not produce its required h shim');
|
||||
}
|
||||
if (previousComplete) {
|
||||
if (!(await directoriesMatch(active, staged))) {
|
||||
throw new Error('existing complete hook set differs from the installed Husky version');
|
||||
}
|
||||
await rm(stagingRepo, { recursive: true, force: true });
|
||||
} else {
|
||||
await rename(staged, active);
|
||||
await rm(stagingRepo, { recursive: true, force: true });
|
||||
}
|
||||
await activateHooks();
|
||||
if (previousQuarantined) {
|
||||
try {
|
||||
await rm(quarantined, { recursive: true, force: true });
|
||||
} catch {
|
||||
console.warn(
|
||||
`Previous hook state was deactivated but remains quarantined at ${quarantined}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
const cleanupFailures = [];
|
||||
try {
|
||||
if (await pathExists(stagingRepo)) {
|
||||
await rename(stagingRepo, `${quarantined}-staging`);
|
||||
}
|
||||
} catch (cleanupError) {
|
||||
cleanupFailures.push(`staging hooks: ${cleanupError.message}`);
|
||||
}
|
||||
const cleanup =
|
||||
cleanupFailures.length === 0
|
||||
? 'No partial hook set was activated.'
|
||||
: `Automatic cleanup was incomplete (${cleanupFailures.join('; ')}).`;
|
||||
throw new Error(
|
||||
`Hook installation failed: ${error.message}. ${cleanup} Fix: rm -rf .husky/_ && git config core.hooksPath .husky/_ && pnpm install --frozen-lockfile`,
|
||||
{ cause: error },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
try {
|
||||
await installHooks();
|
||||
} catch (error) {
|
||||
console.error(error.message);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,208 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { access, mkdir, readFile, readdir, rm, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import { installHooks } from './install-hooks.mjs';
|
||||
|
||||
const fixtureRoot = path.join(process.cwd(), '.mosaic-test-work', `hooks-${process.pid}`);
|
||||
const quarantineRoot = path.join(fixtureRoot, 'quarantine');
|
||||
|
||||
async function fixture(name) {
|
||||
const root = path.join(fixtureRoot, name);
|
||||
await mkdir(path.join(root, '.husky'), { recursive: true });
|
||||
return root;
|
||||
}
|
||||
|
||||
async function exists(target) {
|
||||
try {
|
||||
await access(target);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('an interrupted install quarantines the partial active hook set and fails loudly', async () => {
|
||||
const root = await fixture('interrupted');
|
||||
let restoredHooksPath = 'not-called';
|
||||
|
||||
await assert.rejects(
|
||||
installHooks({
|
||||
root,
|
||||
quarantineRoot,
|
||||
runHusky: async (stagingHooks) => {
|
||||
await mkdir(path.join(stagingHooks, '_'), { recursive: true });
|
||||
await writeFile(path.join(stagingHooks, '_', 'h'), 'partial');
|
||||
throw new Error('simulated interruption');
|
||||
},
|
||||
activateHooks: async () => {},
|
||||
readHooksPath: async () => null,
|
||||
restoreHooksPath: async (value) => {
|
||||
restoredHooksPath = value;
|
||||
},
|
||||
}),
|
||||
(error) => {
|
||||
assert.match(error.message, /Hook installation failed/);
|
||||
assert.match(error.message, /pnpm install --frozen-lockfile/);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
|
||||
assert.equal(await exists(path.join(root, '.husky', '_')), false);
|
||||
assert.equal(restoredHooksPath, 'not-called');
|
||||
const quarantined = await readdir(quarantineRoot);
|
||||
assert.equal(quarantined.length, 1);
|
||||
});
|
||||
|
||||
test('a failed replacement restores a previously complete active hook set', async () => {
|
||||
const root = await fixture('rollback');
|
||||
const activeShim = path.join(root, '.husky', '_', 'h');
|
||||
await mkdir(path.dirname(activeShim), { recursive: true });
|
||||
await writeFile(activeShim, 'previous-complete');
|
||||
let previousRemainedActiveDuringStaging = false;
|
||||
|
||||
await assert.rejects(
|
||||
installHooks({
|
||||
root,
|
||||
quarantineRoot: path.join(fixtureRoot, 'rollback-quarantine'),
|
||||
runHusky: async () => {
|
||||
previousRemainedActiveDuringStaging =
|
||||
(await readFile(activeShim, 'utf8')) === 'previous-complete';
|
||||
throw new Error('simulated replacement failure');
|
||||
},
|
||||
activateHooks: async () => {},
|
||||
readHooksPath: async () => '.husky/_',
|
||||
restoreHooksPath: async () => {},
|
||||
}),
|
||||
/Hook installation failed/,
|
||||
);
|
||||
|
||||
assert.equal(previousRemainedActiveDuringStaging, true);
|
||||
assert.equal(await readFile(activeShim, 'utf8'), 'previous-complete');
|
||||
});
|
||||
|
||||
test('a mismatched complete hook set fails loudly instead of reporting a stale install as current', async () => {
|
||||
const root = await fixture('mismatch');
|
||||
const activeShim = path.join(root, '.husky', '_', 'h');
|
||||
await mkdir(path.dirname(activeShim), { recursive: true });
|
||||
await writeFile(activeShim, 'old-complete');
|
||||
|
||||
await assert.rejects(
|
||||
installHooks({
|
||||
root,
|
||||
quarantineRoot: path.join(fixtureRoot, 'mismatch-quarantine'),
|
||||
runHusky: async (stagingHooks) => {
|
||||
await mkdir(path.join(stagingHooks, '_'), { recursive: true });
|
||||
await writeFile(path.join(stagingHooks, '_', 'h'), 'new-complete');
|
||||
},
|
||||
activateHooks: async () => {},
|
||||
readHooksPath: async () => '.husky/_',
|
||||
restoreHooksPath: async () => {},
|
||||
}),
|
||||
/Hook installation failed.*pnpm install --frozen-lockfile/,
|
||||
);
|
||||
|
||||
assert.equal(await readFile(activeShim, 'utf8'), 'old-complete');
|
||||
});
|
||||
|
||||
test('a competing successful installer is not removed by the losing process', async () => {
|
||||
const root = await fixture('concurrent');
|
||||
const activeShim = path.join(root, '.husky', '_', 'h');
|
||||
let restored = false;
|
||||
|
||||
await assert.rejects(
|
||||
installHooks({
|
||||
root,
|
||||
quarantineRoot: path.join(fixtureRoot, 'concurrent-quarantine'),
|
||||
runHusky: async (stagingHooks) => {
|
||||
await mkdir(path.join(stagingHooks, '_'), { recursive: true });
|
||||
await writeFile(path.join(stagingHooks, '_', 'h'), 'ours');
|
||||
await mkdir(path.dirname(activeShim), { recursive: true });
|
||||
await writeFile(activeShim, 'peer');
|
||||
},
|
||||
activateHooks: async () => {},
|
||||
readHooksPath: async () => null,
|
||||
restoreHooksPath: async () => {
|
||||
restored = true;
|
||||
},
|
||||
}),
|
||||
/Hook installation failed/,
|
||||
);
|
||||
|
||||
assert.equal(await readFile(activeShim, 'utf8'), 'peer');
|
||||
assert.equal(restored, false);
|
||||
});
|
||||
|
||||
test("a competing installer that replaces this installer's active set is preserved", async () => {
|
||||
const root = await fixture('concurrent-after-rename');
|
||||
const active = path.join(root, '.husky', '_');
|
||||
const activeShim = path.join(active, 'h');
|
||||
let restored = false;
|
||||
|
||||
await assert.rejects(
|
||||
installHooks({
|
||||
root,
|
||||
quarantineRoot: path.join(fixtureRoot, 'concurrent-after-rename-quarantine'),
|
||||
runHusky: async (stagingHooks) => {
|
||||
await mkdir(path.join(stagingHooks, '_'), { recursive: true });
|
||||
await writeFile(path.join(stagingHooks, '_', 'h'), 'ours');
|
||||
},
|
||||
activateHooks: async () => {
|
||||
await rm(active, { recursive: true, force: true });
|
||||
await mkdir(active, { recursive: true });
|
||||
await writeFile(activeShim, 'peer');
|
||||
throw new Error('our activation lost to peer');
|
||||
},
|
||||
readHooksPath: async () => null,
|
||||
restoreHooksPath: async () => {
|
||||
restored = true;
|
||||
},
|
||||
}),
|
||||
/Hook installation failed/,
|
||||
);
|
||||
|
||||
assert.equal(await readFile(activeShim, 'utf8'), 'peer');
|
||||
assert.equal(restored, false);
|
||||
});
|
||||
|
||||
test('an explicit interactive HUSKY=0 opt-out preserves existing hooks without running installer', async () => {
|
||||
const root = await fixture('disabled');
|
||||
const activeShim = path.join(root, '.husky', '_', 'h');
|
||||
await mkdir(path.dirname(activeShim), { recursive: true });
|
||||
await writeFile(activeShim, 'preserved');
|
||||
let ran = false;
|
||||
|
||||
await installHooks({
|
||||
root,
|
||||
disabled: true,
|
||||
runHusky: async () => {
|
||||
ran = true;
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(ran, false);
|
||||
assert.equal(await readFile(activeShim, 'utf8'), 'preserved');
|
||||
});
|
||||
|
||||
test('a successful install leaves a complete active hook set', async () => {
|
||||
const root = await fixture('success');
|
||||
|
||||
await installHooks({
|
||||
root,
|
||||
quarantineRoot,
|
||||
runHusky: async (stagingHooks) => {
|
||||
await mkdir(path.join(stagingHooks, '_'), { recursive: true });
|
||||
await writeFile(path.join(stagingHooks, '_', 'h'), 'complete');
|
||||
},
|
||||
activateHooks: async () => {},
|
||||
readHooksPath: async () => null,
|
||||
restoreHooksPath: async () => {},
|
||||
});
|
||||
|
||||
assert.equal(await exists(path.join(root, '.husky', '_', 'h')), true);
|
||||
});
|
||||
@@ -0,0 +1,254 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { constants } from 'node:fs';
|
||||
import { access, lstat, readFile, readdir, readlink } from 'node:fs/promises';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import path from 'node:path';
|
||||
|
||||
export const MISSING_DEPS_EXIT = 42;
|
||||
export const GENERATED_STATE_EXIT = 43;
|
||||
|
||||
const scriptRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
||||
async function entries(root) {
|
||||
const result = [];
|
||||
async function walk(current) {
|
||||
let children;
|
||||
try {
|
||||
children = await readdir(current, { withFileTypes: true });
|
||||
} catch (error) {
|
||||
if (error.code === 'ENOENT') return;
|
||||
throw error;
|
||||
}
|
||||
for (const child of children) {
|
||||
const target = path.join(current, child.name);
|
||||
result.push(target);
|
||||
if (child.isDirectory() && !child.isSymbolicLink()) await walk(target);
|
||||
}
|
||||
}
|
||||
await walk(root);
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function generatedSymlinkManifest(nextDir) {
|
||||
const links = [];
|
||||
for (const target of (await entries(nextDir)).sort()) {
|
||||
const stats = await lstat(target);
|
||||
if (!stats.isSymbolicLink()) continue;
|
||||
links.push({
|
||||
path: path.relative(nextDir, target).split(path.sep).join('/'),
|
||||
target: await readlink(target),
|
||||
});
|
||||
}
|
||||
return `${JSON.stringify({ version: 1, links })}\n`;
|
||||
}
|
||||
|
||||
const webSourceRoots = (root) => [
|
||||
path.join(root, 'apps', 'web', 'src'),
|
||||
path.join(root, 'apps', 'web', 'public'),
|
||||
path.join(root, 'apps', 'web', 'next-env.d.ts'),
|
||||
path.join(root, 'apps', 'web', 'next.config.ts'),
|
||||
path.join(root, 'apps', 'web', 'postcss.config.mjs'),
|
||||
path.join(root, 'apps', 'web', 'package.json'),
|
||||
path.join(root, 'apps', 'web', 'tsconfig.json'),
|
||||
path.join(root, 'packages', 'design-tokens', 'src'),
|
||||
path.join(root, 'packages', 'design-tokens', 'package.json'),
|
||||
path.join(root, 'packages', 'design-tokens', 'tsconfig.json'),
|
||||
path.join(root, 'package.json'),
|
||||
path.join(root, 'tsconfig.base.json'),
|
||||
path.join(root, 'pnpm-lock.yaml'),
|
||||
path.join(root, 'pnpm-workspace.yaml'),
|
||||
path.join(root, 'turbo.json'),
|
||||
];
|
||||
|
||||
// next.config.ts currently reads no server-only environment. Add any future
|
||||
// server-side build inputs here; all resolved NEXT_PUBLIC_* inputs are automatic.
|
||||
const serverBuildEnvironmentKeys = [];
|
||||
|
||||
function publicBuildEnvironment(root) {
|
||||
const webDir = path.join(root, 'apps', 'web');
|
||||
const requireFromWeb = createRequire(path.join(scriptRoot, 'apps', 'web', 'package.json'));
|
||||
const requireFromNext = createRequire(requireFromWeb.resolve('next/package.json'));
|
||||
const { loadEnvConfig, resetEnv, updateInitialEnv } = requireFromNext('@next/env');
|
||||
const originalEnvironment = { ...process.env };
|
||||
updateInitialEnv(originalEnvironment);
|
||||
try {
|
||||
const { combinedEnv } = loadEnvConfig(webDir, false, { info() {}, error() {} }, true);
|
||||
return Object.fromEntries(
|
||||
Object.entries(combinedEnv).filter(
|
||||
([key, value]) =>
|
||||
value !== undefined &&
|
||||
(key.startsWith('NEXT_PUBLIC_') || serverBuildEnvironmentKeys.includes(key)),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
resetEnv();
|
||||
}
|
||||
}
|
||||
|
||||
export async function sourceFingerprint(root = process.cwd()) {
|
||||
const files = [];
|
||||
for (const sourceRoot of webSourceRoots(root)) {
|
||||
try {
|
||||
const stats = await lstat(sourceRoot);
|
||||
if (stats.isSymbolicLink()) {
|
||||
throw new Error(
|
||||
`Web build input must not be a symbolic link: ${path.relative(root, sourceRoot)}`,
|
||||
);
|
||||
}
|
||||
if (stats.isFile()) files.push(sourceRoot);
|
||||
if (stats.isDirectory()) {
|
||||
for (const target of await entries(sourceRoot)) {
|
||||
const targetStats = await lstat(target);
|
||||
if (targetStats.isSymbolicLink()) {
|
||||
throw new Error(
|
||||
`Web build input must not be a symbolic link: ${path.relative(root, target)}`,
|
||||
);
|
||||
}
|
||||
if (targetStats.isFile()) files.push(target);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
}
|
||||
|
||||
const digest = createHash('sha256');
|
||||
for (const [key, value] of Object.entries(publicBuildEnvironment(root)).sort()) {
|
||||
digest.update(`env:${key}\0${value.length}\0${value}\0`);
|
||||
}
|
||||
for (const target of files.sort()) {
|
||||
const contents = await readFile(target);
|
||||
digest.update(path.relative(root, target).split(path.sep).join('/'));
|
||||
digest.update('\0');
|
||||
digest.update(String(contents.length));
|
||||
digest.update('\0');
|
||||
digest.update(contents);
|
||||
digest.update('\0');
|
||||
}
|
||||
return digest.digest('hex');
|
||||
}
|
||||
|
||||
export async function runPreflight({ root = process.cwd(), uid = process.getuid?.() } = {}) {
|
||||
const binDir = path.join(root, 'node_modules', '.bin');
|
||||
const requiredBinaries = ['eslint', 'husky', 'prettier', 'tsc', 'turbo', 'vitest'];
|
||||
const missingBinaries = [];
|
||||
for (const binary of requiredBinaries) {
|
||||
try {
|
||||
await access(path.join(binDir, binary), constants.X_OK);
|
||||
} catch {
|
||||
missingBinaries.push(binary);
|
||||
}
|
||||
}
|
||||
if (missingBinaries.length > 0) {
|
||||
return {
|
||||
code: MISSING_DEPS_EXIT,
|
||||
message: `MOSAIC_PREFLIGHT_MISSING_DEPS: dependency installation is missing ${missingBinaries.join(', ')}; run pnpm install --frozen-lockfile`,
|
||||
};
|
||||
}
|
||||
|
||||
const buildLock = path.join(root, '.mosaic-test-work', 'web-build.lock');
|
||||
try {
|
||||
await lstat(buildLock);
|
||||
return {
|
||||
code: GENERATED_STATE_EXIT,
|
||||
message: `MOSAIC_PREFLIGHT_GENERATED_STATE: web build is in progress or interrupted at ${buildLock}; wait for it to finish or rerun pnpm build to recover the stale lock`,
|
||||
};
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
|
||||
const nextDir = path.join(root, 'apps', 'web', '.next');
|
||||
let generated = [];
|
||||
try {
|
||||
const nextStats = await lstat(nextDir);
|
||||
if (!nextStats.isDirectory() || nextStats.isSymbolicLink()) {
|
||||
return {
|
||||
code: GENERATED_STATE_EXIT,
|
||||
message:
|
||||
'MOSAIC_PREFLIGHT_GENERATED_STATE: apps/web/.next must be a real directory, not a symbolic link, and is not trustworthy; run pnpm clean:generated, then rerun the gate',
|
||||
};
|
||||
}
|
||||
generated = [nextDir, ...(await entries(nextDir))];
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
|
||||
if (generated.length > 0) {
|
||||
const foreign = [];
|
||||
for (const target of generated) {
|
||||
const stats = await lstat(target);
|
||||
if (uid !== undefined && stats.uid !== uid) foreign.push(path.relative(root, target));
|
||||
}
|
||||
|
||||
// Detects accidental, independent, stale, and foreign-residue mutation of
|
||||
// generated state: the class this check was born from was a five-month-stale
|
||||
// .next whose validator referenced deleted pages and produced 19 phantom TS2307
|
||||
// errors indistinguishable from real type errors.
|
||||
//
|
||||
// Does NOT defend against an actor with same-UID write access to the generated
|
||||
// tree, which can regenerate both the manifest and marker consistently
|
||||
// (CWE-345). No local construction can, absent a trust anchor outside that
|
||||
// actor's authority. RM-59 tracks executor/spine-side attestation.
|
||||
let certification = null;
|
||||
let certifiedManifest = null;
|
||||
try {
|
||||
const [certificationContents, manifestContents] = await Promise.all([
|
||||
readFile(path.join(nextDir, '.mosaic-source-hash'), 'utf8'),
|
||||
readFile(path.join(nextDir, '.mosaic-symlink-manifest'), 'utf8'),
|
||||
]);
|
||||
try {
|
||||
const parsed = JSON.parse(certificationContents);
|
||||
if (
|
||||
parsed.version === 1 &&
|
||||
typeof parsed.sourceFingerprint === 'string' &&
|
||||
typeof parsed.symlinkManifestHash === 'string'
|
||||
) {
|
||||
certification = parsed;
|
||||
certifiedManifest = manifestContents;
|
||||
}
|
||||
} catch {
|
||||
// Invalid certification is handled as untrusted generated state below.
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') throw error;
|
||||
}
|
||||
const stale = certification?.sourceFingerprint !== (await sourceFingerprint(root));
|
||||
const actualManifest = await generatedSymlinkManifest(nextDir);
|
||||
const certifiedManifestHash =
|
||||
certifiedManifest === null
|
||||
? null
|
||||
: createHash('sha256').update(certifiedManifest).digest('hex');
|
||||
const changedSymlinks =
|
||||
certification?.symlinkManifestHash !== certifiedManifestHash ||
|
||||
certifiedManifest !== actualManifest;
|
||||
if (foreign.length > 0 || stale || changedSymlinks) {
|
||||
const reasons = [
|
||||
foreign.length > 0 ? `foreign-owned paths: ${foreign.slice(0, 3).join(', ')}` : '',
|
||||
stale ? 'generated source fingerprint does not match web source/configuration' : '',
|
||||
changedSymlinks
|
||||
? 'generated symbolic-link manifest does not match the certified build'
|
||||
: '',
|
||||
].filter(Boolean);
|
||||
return {
|
||||
code: GENERATED_STATE_EXIT,
|
||||
message: `MOSAIC_PREFLIGHT_GENERATED_STATE: apps/web/.next is not trustworthy (${reasons.join('; ')}); run pnpm clean:generated, then rerun the gate`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { code: 0, message: 'checkout preflight passed' };
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const result = await runPreflight();
|
||||
const stream = result.code === 0 ? process.stdout : process.stderr;
|
||||
stream.write(`${result.message}\n`);
|
||||
process.exitCode = result.code;
|
||||
}
|
||||
|
||||
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
await main();
|
||||
}
|
||||
@@ -0,0 +1,274 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { chmod, mkdir, rm, symlink, utimes, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import { runPreflight, sourceFingerprint } from './preflight.mjs';
|
||||
|
||||
const fixtureRoot = path.join(process.cwd(), '.mosaic-test-work', `preflight-${process.pid}`);
|
||||
|
||||
const requiredBins = ['eslint', 'husky', 'prettier', 'tsc', 'turbo', 'vitest'];
|
||||
|
||||
async function fixture(name) {
|
||||
const root = path.join(fixtureRoot, name);
|
||||
await mkdir(path.join(root, 'apps', 'web', 'src', 'app'), { recursive: true });
|
||||
await writeFile(path.join(root, 'apps', 'web', 'src', 'app', 'page.tsx'), 'export default 1;\n');
|
||||
return root;
|
||||
}
|
||||
|
||||
async function installRequiredBins(root) {
|
||||
const binDir = path.join(root, 'node_modules', '.bin');
|
||||
await mkdir(binDir, { recursive: true });
|
||||
await Promise.all(
|
||||
requiredBins.map(async (name) => {
|
||||
const target = path.join(binDir, name);
|
||||
await writeFile(target, '');
|
||||
await chmod(target, 0o755);
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
async function certifyGeneratedState(root, links = []) {
|
||||
const nextDir = path.join(root, 'apps', 'web', '.next');
|
||||
await mkdir(nextDir, { recursive: true });
|
||||
const manifest = `${JSON.stringify({ version: 1, links })}\n`;
|
||||
const manifestHash = createHash('sha256').update(manifest).digest('hex');
|
||||
await writeFile(path.join(nextDir, '.mosaic-symlink-manifest'), manifest);
|
||||
await writeFile(
|
||||
path.join(nextDir, '.mosaic-source-hash'),
|
||||
`${JSON.stringify({
|
||||
version: 1,
|
||||
sourceFingerprint: await sourceFingerprint(root),
|
||||
symlinkManifestHash: manifestHash,
|
||||
})}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
await rm(fixtureRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('missing dependencies have a dedicated exit code and install remediation', async () => {
|
||||
const root = await fixture('missing-deps');
|
||||
const result = await runPreflight({ root });
|
||||
|
||||
assert.equal(result.code, 42);
|
||||
assert.match(result.message, /MOSAIC_PREFLIGHT_MISSING_DEPS/);
|
||||
assert.match(result.message, /run pnpm install/i);
|
||||
});
|
||||
|
||||
test('a partial dependency install keeps the dedicated missing-deps result', async () => {
|
||||
const root = await fixture('partial-deps');
|
||||
await mkdir(path.join(root, 'node_modules', '.bin'), { recursive: true });
|
||||
await writeFile(path.join(root, 'node_modules', '.bin', 'tsc'), '', { mode: 0o755 });
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 42);
|
||||
assert.match(result.message, /turbo/);
|
||||
});
|
||||
|
||||
test('a dangling required dependency shim keeps the dedicated missing-deps result', async () => {
|
||||
const root = await fixture('dangling-deps');
|
||||
await installRequiredBins(root);
|
||||
const turbo = path.join(root, 'node_modules', '.bin', 'turbo');
|
||||
await rm(turbo);
|
||||
await symlink(path.join(root, 'node_modules', 'missing-turbo'), turbo);
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 42);
|
||||
assert.match(result.message, /turbo/);
|
||||
});
|
||||
|
||||
test('installed dependencies pass when generated state is absent', async () => {
|
||||
const root = await fixture('clean');
|
||||
await installRequiredBins(root);
|
||||
|
||||
assert.deepEqual(await runPreflight({ root }), { code: 0, message: 'checkout preflight passed' });
|
||||
});
|
||||
|
||||
test('foreign-owned generated Next state is identified separately from source errors', async () => {
|
||||
const root = await fixture('foreign-next');
|
||||
await installRequiredBins(root);
|
||||
const generated = path.join(root, 'apps', 'web', '.next', 'types', 'validator.ts');
|
||||
await mkdir(path.dirname(generated), { recursive: true });
|
||||
await writeFile(generated, 'generated output');
|
||||
|
||||
const result = await runPreflight({ root, uid: (process.getuid?.() ?? 0) + 1 });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /MOSAIC_PREFLIGHT_GENERATED_STATE/);
|
||||
assert.match(result.message, /foreign-owned/);
|
||||
});
|
||||
|
||||
test('a generated marker mismatch is identified separately from source errors', async () => {
|
||||
const root = await fixture('stale-next');
|
||||
await installRequiredBins(root);
|
||||
const generated = path.join(root, 'apps', 'web', '.next', 'types', 'validator.ts');
|
||||
await mkdir(path.dirname(generated), { recursive: true });
|
||||
await writeFile(generated, 'stale generated output');
|
||||
await writeFile(path.join(root, 'apps', 'web', '.next', '.mosaic-source-hash'), 'old-source');
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /MOSAIC_PREFLIGHT_GENERATED_STATE/);
|
||||
assert.match(result.message, /apps\/web\/\.next/);
|
||||
assert.match(result.message, /pnpm clean:generated/);
|
||||
});
|
||||
|
||||
test('generated-state symbolic links are accepted only when exactly build-certified', async (t) => {
|
||||
await t.test('apps/web/.next itself is rejected when it is a symbolic link', async () => {
|
||||
const root = await fixture('symbolic-next-root');
|
||||
await installRequiredBins(root);
|
||||
await writeFile(path.join(root, 'outside-generated'), 'not a Next build\n');
|
||||
await symlink(path.join(root, 'outside-generated'), path.join(root, 'apps', 'web', '.next'));
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /MOSAIC_PREFLIGHT_GENERATED_STATE/);
|
||||
assert.match(result.message, /symbolic link/);
|
||||
});
|
||||
|
||||
await t.test('apps/web/.next is rejected when it is not a directory', async () => {
|
||||
const root = await fixture('non-directory-next-root');
|
||||
await installRequiredBins(root);
|
||||
await writeFile(path.join(root, 'apps', 'web', '.next'), 'not a Next build\n');
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /MOSAIC_PREFLIGHT_GENERATED_STATE/);
|
||||
assert.match(result.message, /real directory/);
|
||||
});
|
||||
|
||||
await t.test('an added descendant symlink is rejected', async () => {
|
||||
const root = await fixture('symbolic-next-added');
|
||||
await installRequiredBins(root);
|
||||
await certifyGeneratedState(root);
|
||||
await symlink('/etc/hosts', path.join(root, 'apps', 'web', '.next', 'reviewer-symlink'));
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /symbolic-link manifest/);
|
||||
});
|
||||
|
||||
await t.test('a removed certified descendant symlink is rejected', async () => {
|
||||
const root = await fixture('symbolic-next-removed');
|
||||
await installRequiredBins(root);
|
||||
const link = path.join(root, 'apps', 'web', '.next', 'dependency-link');
|
||||
await mkdir(path.dirname(link), { recursive: true });
|
||||
await symlink('../dependency-one', link);
|
||||
await certifyGeneratedState(root, [{ path: 'dependency-link', target: '../dependency-one' }]);
|
||||
await rm(link);
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /symbolic-link manifest/);
|
||||
});
|
||||
|
||||
await t.test('a retargeted certified descendant symlink is rejected', async () => {
|
||||
const root = await fixture('symbolic-next-retargeted');
|
||||
await installRequiredBins(root);
|
||||
const link = path.join(root, 'apps', 'web', '.next', 'dependency-link');
|
||||
await mkdir(path.dirname(link), { recursive: true });
|
||||
await symlink('../dependency-one', link);
|
||||
await certifyGeneratedState(root, [{ path: 'dependency-link', target: '../dependency-one' }]);
|
||||
await rm(link);
|
||||
await symlink('../dependency-two', link);
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /symbolic-link manifest/);
|
||||
});
|
||||
|
||||
await t.test('a manifest edited to whitelist a rogue symlink is rejected', async () => {
|
||||
const root = await fixture('symbolic-next-tampered-manifest');
|
||||
await installRequiredBins(root);
|
||||
await certifyGeneratedState(root);
|
||||
const nextDir = path.join(root, 'apps', 'web', '.next');
|
||||
await symlink('/etc/hosts', path.join(nextDir, 'reviewer-symlink'));
|
||||
await writeFile(
|
||||
path.join(nextDir, '.mosaic-symlink-manifest'),
|
||||
`${JSON.stringify({
|
||||
version: 1,
|
||||
links: [{ path: 'reviewer-symlink', target: '/etc/hosts' }],
|
||||
})}\n`,
|
||||
);
|
||||
|
||||
const result = await runPreflight({ root });
|
||||
assert.equal(result.code, 43);
|
||||
assert.match(result.message, /symbolic-link manifest/);
|
||||
});
|
||||
|
||||
await t.test('unchanged canonical-style descendant symlinks are accepted', async () => {
|
||||
const root = await fixture('symbolic-next-certified');
|
||||
await installRequiredBins(root);
|
||||
const link = path.join(
|
||||
root,
|
||||
'apps',
|
||||
'web',
|
||||
'.next',
|
||||
'standalone',
|
||||
'node_modules',
|
||||
'dependency',
|
||||
);
|
||||
await mkdir(path.dirname(link), { recursive: true });
|
||||
await symlink('../.pnpm/dependency', link);
|
||||
await certifyGeneratedState(root, [
|
||||
{ path: 'standalone/node_modules/dependency', target: '../.pnpm/dependency' },
|
||||
]);
|
||||
|
||||
assert.deepEqual(await runPreflight({ root }), {
|
||||
code: 0,
|
||||
message: 'checkout preflight passed',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
test('the source fingerprint includes inherited TypeScript configuration', async () => {
|
||||
const root = await fixture('inherited-typescript-config');
|
||||
const config = path.join(root, 'tsconfig.base.json');
|
||||
await writeFile(config, '{"compilerOptions":{"strict":true}}\n');
|
||||
const first = await sourceFingerprint(root);
|
||||
await writeFile(config, '{"compilerOptions":{"strict":false}}\n');
|
||||
const second = await sourceFingerprint(root);
|
||||
|
||||
assert.notEqual(first, second);
|
||||
});
|
||||
|
||||
test('the source fingerprint rejects symbolic-link build inputs', async () => {
|
||||
const root = await fixture('symbolic-source');
|
||||
await writeFile(path.join(root, 'outside.ts'), 'export default 1;\n');
|
||||
await symlink(path.join(root, 'outside.ts'), path.join(root, 'apps', 'web', 'src', 'linked.ts'));
|
||||
|
||||
await assert.rejects(sourceFingerprint(root), /must not be a symbolic link/);
|
||||
});
|
||||
|
||||
test('the source fingerprint includes expanded public web build environment', async () => {
|
||||
const root = await fixture('public-build-environment');
|
||||
const envFile = path.join(root, 'apps', 'web', '.env.production');
|
||||
await writeFile(
|
||||
envFile,
|
||||
'RM01_GATEWAY_URL=https://one.example\nNEXT_PUBLIC_RM01_URL=$RM01_GATEWAY_URL\n',
|
||||
);
|
||||
const first = await sourceFingerprint(root);
|
||||
await writeFile(
|
||||
envFile,
|
||||
'RM01_GATEWAY_URL=https://two.example\nNEXT_PUBLIC_RM01_URL=$RM01_GATEWAY_URL\n',
|
||||
);
|
||||
const second = await sourceFingerprint(root);
|
||||
|
||||
assert.notEqual(first, second);
|
||||
});
|
||||
|
||||
test('a matching generation marker accepts incremental output with mixed mtimes', async () => {
|
||||
const root = await fixture('incremental-next');
|
||||
await installRequiredBins(root);
|
||||
const generated = path.join(root, 'apps', 'web', '.next', 'types', 'validator.ts');
|
||||
await mkdir(path.dirname(generated), { recursive: true });
|
||||
await writeFile(generated, 'unchanged generated output');
|
||||
await utimes(generated, new Date('2020-01-01T00:00:00Z'), new Date('2020-01-01T00:00:00Z'));
|
||||
const fresh = path.join(root, 'apps', 'web', '.next', 'types', 'routes.ts');
|
||||
await writeFile(fresh, 'fresh generated output');
|
||||
await certifyGeneratedState(root);
|
||||
|
||||
assert.deepEqual(await runPreflight({ root }), { code: 0, message: 'checkout preflight passed' });
|
||||
});
|
||||
Reference in New Issue
Block a user