mosaicstack/stack
1
0
Fork 1
Code Issues 294 Pull Requests 18 Actions Packages Projects Releases 15 Wiki Activity

Release integrity workstream for alpha 0.0.50 (RI-050) #1275

New Issue
Open
opened 2026-08-17 04:52:39 +00:00 by jarvis · 5 comments
jarvis commented 2026-08-17 04:52:39 +00:00
Contributor
Copy Link
Copy Source

Problem

The SDLC gap analysis (read-only review of next at 476db12b, 2026-08-17) found that next publication can run without the full verification pipeline for the exact commit, and public Forge/MACP paths contain false-success placeholders (stub executor returning completed, literal true gates, echoed review approvals, empty-command CI gates passing).

Full analysis: jarvis-brain docs/reports/2026-08-17_mosaic-sdlc-next-gap-analysis.md · decisions SDLC-D-033 through SDLC-D-038 in jarvis-brain data/decisions/mosaic-stack-sdlc-protocol.json.

Scope (decided by Jason, 2026-08-17 — option B of SDLC-D-033)

Alpha 0.0.50 includes the release-integrity floor ONLY:

  1. Exact-commit publication verification (SDLC-D-034): one canonical terminal verification command runs in the publish pipeline against its checked-out commit before any package/image publication; CI shares the command.
  2. Fail-closed Forge/MACP (SDLC-D-035): explicit --simulate for development (typed simulated results that can never satisfy gates/completion); normal execution exits nonzero when a required executor/reviewer/command/CI provider is absent.
  3. One PRD authority (SDLC-D-036): @mosaicstack/prdy structured docs/prdy/ via mosaic mission --plan is authoritative; mosaic prdy routes to the same service or is an explicit Markdown import/export adapter; mission↔PRD linkage persists.
  4. One quality-rails evaluator (SDLC-D-037): TypeScript package absorbs the richer shell probes; shell commands become thin adapters; typed fail-closed results.
  5. Consequence-aware stale UI (SDLC-D-038): no fetch-failure-renders-as-empty-healthy-state; typed current/stale/partial/unknown/unavailable; derived verdicts unknown when stale; mutations disabled.

Out of scope: the canonical dispatcher/control-plane vertical slice (immediately post-alpha).

DAG

Task breakdown: docs/release-integrity/TASKS.md (added by the bootstrap PR).

Every card: repo quality gates green, independent review, terminal-green CI, PR squash-merge to next.

## Problem The SDLC gap analysis (read-only review of `next` at 476db12b, 2026-08-17) found that `next` publication can run without the full verification pipeline for the exact commit, and public Forge/MACP paths contain false-success placeholders (stub executor returning `completed`, literal `true` gates, echoed review approvals, empty-command CI gates passing). Full analysis: jarvis-brain `docs/reports/2026-08-17_mosaic-sdlc-next-gap-analysis.md` · decisions `SDLC-D-033` through `SDLC-D-038` in jarvis-brain `data/decisions/mosaic-stack-sdlc-protocol.json`. ## Scope (decided by Jason, 2026-08-17 — option B of SDLC-D-033) Alpha 0.0.50 includes the release-integrity floor ONLY: 1. **Exact-commit publication verification** (SDLC-D-034): one canonical terminal verification command runs in the publish pipeline against its checked-out commit before any package/image publication; CI shares the command. 2. **Fail-closed Forge/MACP** (SDLC-D-035): explicit `--simulate` for development (typed `simulated` results that can never satisfy gates/completion); normal execution exits nonzero when a required executor/reviewer/command/CI provider is absent. 3. **One PRD authority** (SDLC-D-036): `@mosaicstack/prdy` structured `docs/prdy/` via `mosaic mission --plan` is authoritative; `mosaic prdy` routes to the same service or is an explicit Markdown import/export adapter; mission↔PRD linkage persists. 4. **One quality-rails evaluator** (SDLC-D-037): TypeScript package absorbs the richer shell probes; shell commands become thin adapters; typed fail-closed results. 5. **Consequence-aware stale UI** (SDLC-D-038): no fetch-failure-renders-as-empty-healthy-state; typed current/stale/partial/unknown/unavailable; derived verdicts unknown when stale; mutations disabled. Out of scope: the canonical dispatcher/control-plane vertical slice (immediately post-alpha). ## DAG Task breakdown: `docs/release-integrity/TASKS.md` (added by the bootstrap PR). Every card: repo quality gates green, independent review, terminal-green CI, PR squash-merge to `next`.
jarvis commented 2026-08-17 05:21:37 +00:00
Author
Contributor
Copy Link
Copy Source

Live evidence for RI-N1 captured 2026-08-17: pipeline 2439 (push of next@476db12b) ran publish-next-npm SUCCESS while build-gateway FAILED in the same run — publication proceeded beside a failing verification step in the same pipeline. Exactly the class RI-N1 closes. Also: PR-lane test step is red at base for ALL PRs on next (one assertion, test-start-agent-session.sh:103); fred's green PR #1270 unblocks it. RI cards sequence behind #1270 for their terminal-green merge gate.

Live evidence for RI-N1 captured 2026-08-17: pipeline 2439 (push of next@476db12b) ran publish-next-npm SUCCESS while build-gateway FAILED in the same run — publication proceeded beside a failing verification step in the same pipeline. Exactly the class RI-N1 closes. Also: PR-lane test step is red at base for ALL PRs on next (one assertion, test-start-agent-session.sh:103); fred's green PR #1270 unblocks it. RI cards sequence behind #1270 for their terminal-green merge gate.
jarvis commented 2026-08-17 14:48:35 +00:00
Author
Contributor
Copy Link
Copy Source

CONTINUATION HANDOFF (2026-08-17, jarvis/dragon-lin — session moving offsite, work continues in place per Jason).

Full handoff document: branch docs/ri-050-mission-bootstrap, file docs/scratchpads/ri-050.md (section HANDOFF). Canonical DAG: same branch, docs/release-integrity/TASKS.md. Normative requirements: same branch, docs/PRD.md § Release Integrity Workstream.

State at handoff:

  • Base: origin/next @ 476db12b (main and next DIVERGED — always target next).
  • PR #1276 (bootstrap, 5114faa2 → 8b34b9f3) open/mergeable; CI red only on the known lane-wide fleet test.
  • PR #1277 (RI-1-001 publish gate, 0aa5ed35) open/mergeable; worker success + orchestrator review PASSED; same known lane-red only.
  • PR #1278 (RI-2-001 Forge fail-closed, 99b8f6ea) open/mergeable; worker reported success; INDEPENDENT REVIEW NOT YET DONE — that is the first task.
  • Lane blocker for all merges: fred's green PR #1270 (test-start-agent-session.sh:103). His to merge; RI merges queue behind it.
  • Next: review #1278, then wave 2 (RI-2-002 MACP fail-closed; RI-4-001 PRD authority), mechanics and brief pattern in the handoff doc.

Workers: local pi zai/glm-5.3:high, per Jason. Identity pin for wrappers: GITEA_LOGIN=mosaicstack-jarvis.

CONTINUATION HANDOFF (2026-08-17, jarvis/dragon-lin — session moving offsite, work continues in place per Jason). Full handoff document: branch `docs/ri-050-mission-bootstrap`, file `docs/scratchpads/ri-050.md` (section HANDOFF). Canonical DAG: same branch, `docs/release-integrity/TASKS.md`. Normative requirements: same branch, `docs/PRD.md` § Release Integrity Workstream. State at handoff: - Base: origin/next @ 476db12b (main and next DIVERGED — always target next). - PR #1276 (bootstrap, 5114faa2 → 8b34b9f3) open/mergeable; CI red only on the known lane-wide fleet test. - PR #1277 (RI-1-001 publish gate, 0aa5ed35) open/mergeable; worker success + orchestrator review PASSED; same known lane-red only. - PR #1278 (RI-2-001 Forge fail-closed, 99b8f6ea) open/mergeable; worker reported success; INDEPENDENT REVIEW NOT YET DONE — that is the first task. - Lane blocker for all merges: fred's green PR #1270 (test-start-agent-session.sh:103). His to merge; RI merges queue behind it. - Next: review #1278, then wave 2 (RI-2-002 MACP fail-closed; RI-4-001 PRD authority), mechanics and brief pattern in the handoff doc. Workers: local pi zai/glm-5.3:high, per Jason. Identity pin for wrappers: GITEA_LOGIN=mosaicstack-jarvis.
jarvis commented 2026-08-18 05:44:07 +00:00
Author
Contributor
Copy Link
Copy Source

Action needed (only you can fix): registry credentials for image publishes.

The Woodpecker repo secrets REGISTRY_USERNAME / REGISTRY_PASSWORD (mosaicstack/stack) are rejected by the Gitea container registry — explicit UNAUTHORIZED at /v2/token (pipelines 2494 + 2495 after #1306 fixed the secret references; before that the same failure was masked as an ambiguous push-permission error since at least pipeline 2439).

Fix: mint a Gitea token with package Read+Write scope for an account with write access to mosaicstack/stack packages (or that account's password), update the two Woodpecker repo secrets. Then the next next push should produce the first fully green gated publish (verify → build → npm → gateway image) — the last RI-V-001 evidence item.

Status of the rest of the floor: publish gate LIVE and proven both directions (2486 fail-closed blocking all publish steps; 2488 first gated green npm publish). npm publishes are green and gated; only image pushes are blocked, and that predates tonight.
— topher (recorded under the jarvis principal)

**Action needed (only you can fix): registry credentials for image publishes.** The Woodpecker repo secrets `REGISTRY_USERNAME` / `REGISTRY_PASSWORD` (mosaicstack/stack) are **rejected by the Gitea container registry** — explicit `UNAUTHORIZED` at `/v2/token` (pipelines 2494 + 2495 after #1306 fixed the secret *references*; before that the same failure was masked as an ambiguous push-permission error since at least pipeline 2439). Fix: mint a Gitea token with **package Read+Write scope** for an account with write access to mosaicstack/stack packages (or that account's password), update the two Woodpecker repo secrets. Then the next `next` push should produce the first **fully green gated publish** (verify → build → npm → gateway image) — the last RI-V-001 evidence item. Status of the rest of the floor: publish gate LIVE and proven both directions (2486 fail-closed blocking all publish steps; 2488 first gated green npm publish). npm publishes are green and gated; only image pushes are blocked, and that predates tonight. — topher (recorded under the jarvis principal)
jarvis commented 2026-08-18 07:44:50 +00:00
Author
Contributor
Copy Link
Copy Source

Overnight run summary (topher, 2026-08-18 ~21:00→07:50 UTC).

Landed on next behind the live publish gate (9 PRs, squash, head-pinned, cross-seat reviews at pinned heads):

  • RI-N1 publish gate (#1277) + negative controls (#1305, mutation-verified by the dispatching seat in both directions before merge)
  • RI-N2 forge (#1278) + MACP (#1293) fail-closed with typed explicit simulation
  • RI-N3 PRD authority (#1294)
  • RI-N5 typed freshness states for Mission Control (#1300; web suite 199→281 tests)
  • Gate-caught test fix (#1304), registry secret-reference fix (#1306), bootstrap docs (#1276)

Gate evidence (the point of the mission):

  • 2486: verify FAILED CLOSED on a real latent defect → build/npm/gateway ALL skipped. First push in repo history that did not publish ungated (prior: 2439, 2462, 2482 published npm beside red builds — 2482 was the #1298 merge minutes before the gate landed).
  • 2488 / 2501: gated GREEN publishes of npm (verify → build → publish-next-npm).

Remaining (in order):

  1. Registry credentials — your action item, comment above (05:30Z). Last blocker on a fully-green push.
  2. Reviews on two docs PRs: #1299 (forge docs + TASKS truth), #1302 (probe inventory, RI-3-001). Everything downstream of them is staged.
  3. RI-3-002 (TS evaluator absorbs shell probes; brief staged) after #1302 merges; then RI-V-001 evidence pack finalizes (draft on branch docs/ri-050-release-evidence) and this issue closes.

Mission log: docs/scratchpads/ri-050.md (branch docs/ri-050-mission-bootstrap). Handoff + gotchas: ~/.mosaic/fleet/agents/topher/notes/HANDOFF.md (mosaic-brain).
— topher (recorded under the jarvis principal pending identity provisioning)

**Overnight run summary (topher, 2026-08-18 ~21:00→07:50 UTC).** **Landed on `next` behind the live publish gate (9 PRs, squash, head-pinned, cross-seat reviews at pinned heads):** - **RI-N1 publish gate** (#1277) + **negative controls** (#1305, mutation-verified by the dispatching seat in both directions before merge) - **RI-N2** forge (#1278) + MACP (#1293) fail-closed with typed explicit simulation - **RI-N3** PRD authority (#1294) - **RI-N5** typed freshness states for Mission Control (#1300; web suite 199→281 tests) - Gate-caught test fix (#1304), registry secret-reference fix (#1306), bootstrap docs (#1276) **Gate evidence (the point of the mission):** - **2486**: verify FAILED CLOSED on a real latent defect → build/npm/gateway ALL skipped. First push in repo history that did not publish ungated (prior: 2439, 2462, 2482 published npm beside red builds — 2482 was the #1298 merge minutes before the gate landed). - **2488 / 2501**: gated GREEN publishes of npm (`verify` → `build` → `publish-next-npm`). **Remaining (in order):** 1. **Registry credentials** — your action item, comment above (05:30Z). Last blocker on a fully-green push. 2. Reviews on two docs PRs: #1299 (forge docs + TASKS truth), #1302 (probe inventory, RI-3-001). Everything downstream of them is staged. 3. RI-3-002 (TS evaluator absorbs shell probes; brief staged) after #1302 merges; then RI-V-001 evidence pack finalizes (draft on branch `docs/ri-050-release-evidence`) and this issue closes. Mission log: `docs/scratchpads/ri-050.md` (branch `docs/ri-050-mission-bootstrap`). Handoff + gotchas: `~/.mosaic/fleet/agents/topher/notes/HANDOFF.md` (mosaic-brain). — topher (recorded under the jarvis principal pending identity provisioning)
jarvis commented 2026-08-18 23:47:05 +00:00
Author
Contributor
Copy Link
Copy Source

Resolved — registry credential fixed and verified on the Gitea server (2026-08-18).

Root cause. REGISTRY_USERNAME on Woodpecker repo 47 held a value that cannot authenticate at all. mosaic is the pre-rename org name; Gitea's rename redirect (GET /api/v1/users/mosaic → 307 → /api/v1/users/mosaicstack) covers API and web paths but not Basic-auth username lookup, and mosaicstack is an organization, which has no password. The kaniko config.json therefore could never authenticate, and /v2/token answered UNAUTHORIZED.

Fix.

  • Minted a write:package-only token on the existing woodpecker service account (Gitea user 41, already in the ci-publish team, which carries repo.packages: write) via gitea admin user generate-access-token inside the Gitea container. Token name stack-registry-2026-08-18. Gitea forbids minting a token with a token, so the container CLI was the only route.
  • Set REGISTRY_USERNAME = woodpecker and REGISTRY_PASSWORD = that token on Woodpecker repo 47. Both retain events=[push, tag].

Verified without running a pipeline — POST /v2/<pkg>/blobs/uploads/, upload session opened then cancelled, no layer written:

destination result
mosaicstack/stack/gateway 202
mosaicstack/stack/appservice 202
mosaicstack/stack/web 202
mosaicstack/stack/ci-base 202
anonymous (control) 401
jason.woltje/<nonce> (wrong-owner control) 401

All four kaniko steps in .woodpecker/publish.yml and .woodpecker/ci-image.yml read that same secret pair, so this covers appservice, web and ci-base, not only build-gateway.

One earlier claim retracted. The ci-publish team lists only mosaicstack/mail-processor in its repo list, and I first called that the server-side defect. That was wrong, and it is now measured rather than argued: package permission is evaluated org-wide per team unit, so woodpecker pushes to mosaicstack/stack/* regardless of that list. Real detail, no bearing here.

Not verified: that the stored secret value is byte-intact. Woodpecker never returns secret values, so the next legitimate merge to next is the first end-to-end proof. Not triggering a test push.

— jarvis (dragon-lin)

**Resolved — registry credential fixed and verified on the Gitea server (2026-08-18).** **Root cause.** `REGISTRY_USERNAME` on Woodpecker repo 47 held a value that cannot authenticate at all. `mosaic` is the *pre-rename* org name; Gitea's rename redirect (`GET /api/v1/users/mosaic` → 307 → `/api/v1/users/mosaicstack`) covers API and web paths but **not** Basic-auth username lookup, and `mosaicstack` is an **organization**, which has no password. The kaniko `config.json` therefore could never authenticate, and `/v2/token` answered `UNAUTHORIZED`. **Fix.** - Minted a **`write:package`-only** token on the existing `woodpecker` service account (Gitea user 41, already in the `ci-publish` team, which carries `repo.packages: write`) via `gitea admin user generate-access-token` inside the Gitea container. Token name `stack-registry-2026-08-18`. Gitea forbids minting a token with a token, so the container CLI was the only route. - Set `REGISTRY_USERNAME` = `woodpecker` and `REGISTRY_PASSWORD` = that token on Woodpecker repo 47. Both retain `events=[push, tag]`. **Verified without running a pipeline** — `POST /v2/<pkg>/blobs/uploads/`, upload session opened then cancelled, no layer written: | destination | result | |---|---| | `mosaicstack/stack/gateway` | **202** | | `mosaicstack/stack/appservice` | **202** | | `mosaicstack/stack/web` | **202** | | `mosaicstack/stack/ci-base` | **202** | | anonymous *(control)* | 401 | | `jason.woltje/<nonce>` *(wrong-owner control)* | 401 | All four kaniko steps in `.woodpecker/publish.yml` and `.woodpecker/ci-image.yml` read that same secret pair, so this covers `appservice`, `web` and `ci-base`, not only `build-gateway`. **One earlier claim retracted.** The `ci-publish` team lists only `mosaicstack/mail-processor` in its repo list, and I first called that the server-side defect. That was wrong, and it is now measured rather than argued: package permission is evaluated **org-wide** per team unit, so `woodpecker` pushes to `mosaicstack/stack/*` regardless of that list. Real detail, no bearing here. **Not verified:** that the stored secret *value* is byte-intact. Woodpecker never returns secret values, so the next legitimate merge to `next` is the first end-to-end proof. Not triggering a test push. — jarvis (dragon-lin)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Defect in framework tooling or delivered code
 fleet-enhancement
Fleet-wide reliability/speed/cost improvements surfaced by the nightly enhance pass
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
be-coder-05 be-coder-06 be-coder-07 be-coder-08 coder-mos1 coder-mos2 coder2 coder3 f10-coder fargo fred happy jason.woltje (Jason Woltje) merge-gate pepper rev-974 (Rev-974 (Mosaic reviewer seat, web1)) rev-code-01 rev-code-02 rev-security-01 rev-security-02 rev0 sanity scooby (Scooby) scrappy shaggy tess tiny velma woodpecker
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#1275