From 8e9b05de8d4d0288ac6d56a271756626594baf06 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Fri, 20 Nov 2026 07:32:51 -0600 Subject: [PATCH 1/4] feat(release): pi-unpin S5: package.json gives pi the range >=0.85.1 and the lock still resolves 0.85.1; the image tag is mosaic-poc-agent:-r, built by scripts/image-tag.mjs from package-lock.json for common.sh, mosaic-task.mjs, the gate's lane-image.mjs, test-task.sh and test-auth.sh, never from the package.json string; scripts/pi-fit-image.mjs runs the fit check on the pi the lock resolves, as npm ci installed it, before every image build (build_image in common.sh for build.sh, release.sh package and verify.sh, and the gate's prepareImage), so an unfit locked pi fails the build; it runs on the host because docker build's seccomp refuses bwrap; agent-host-dev.sh records pi --version for its banner and no longer refuses on version; docs match (#1873, lane pi-unpin-s5) --- Containerfile | 7 +- LAYERS.md | 4 +- README.md | 5 +- adapters/README.md | 5 +- package-lock.json | 2 +- package.json | 2 +- .../cli/tests/gate-lane-image-base.test.mjs | 8 +- packages/cli/tests/gate-lane-image.test.mjs | 10 +- packages/cli/tests/pi-unpin-s5.test.mjs | 290 ++++++++++++++++++ packages/mosaic/README.md | 2 +- scripts/agent-host-dev.sh | 8 +- scripts/build.sh | 2 +- scripts/common.sh | 20 +- scripts/image-tag.mjs | 54 ++++ scripts/mosaic-task.mjs | 9 +- scripts/pi-fit-image.mjs | 70 +++++ scripts/release.sh | 2 +- scripts/test-auth.sh | 3 +- scripts/test-task.sh | 3 +- scripts/testing/lane-image.mjs | 19 +- scripts/verify.sh | 2 +- 21 files changed, 490 insertions(+), 37 deletions(-) create mode 100644 packages/cli/tests/pi-unpin-s5.test.mjs create mode 100755 scripts/image-tag.mjs create mode 100755 scripts/pi-fit-image.mjs diff --git a/Containerfile b/Containerfile index 3f79cc204..ce17a2372 100644 --- a/Containerfile +++ b/Containerfile @@ -15,8 +15,11 @@ RUN apt-get update \ # uid/gid 1000, which matches the host user that owns the runtime # state directory mounted at /var/lib/mosaic. It is reused as-is. -# Pinned Pi install: package.json pins the exact version and -# package-lock.json is installed with npm ci. No unversioned installs. +# Pi install: package.json gives pi a range, not a pin (#1873), and npm ci +# installs the version package-lock.json resolves. No unversioned installs. +# The build entry points run scripts/pi-fit-image.mjs on the host first, so a +# lock that resolves an unfit pi fails the build (the fit check needs bwrap, +# which docker build's seccomp profile refuses, so it is not a RUN step). WORKDIR /opt/app COPY package.json package-lock.json ./ RUN npm ci --ignore-scripts diff --git a/LAYERS.md b/LAYERS.md index 7cc72bca6..f0b22c3c3 100644 --- a/LAYERS.md +++ b/LAYERS.md @@ -6,8 +6,8 @@ not implemented (see BRIEF.md, "Explicit exclusions"). ## L0 — Implemented: container returns MOSAIC_HELLO_OK -One image (`mosaic-poc-agent:0.84.4`, built on digest-pinned `node:24.22.2-bookworm-slim`, non-root, -pinned Pi) runs one Pi agent one-shot. Four immutable local contract files are +One image (`mosaic-poc-agent:-r`, the pi version `package-lock.json` resolves; built on digest-pinned `node:24.22.2-bookworm-slim`, non-root, +pi from the lock, not pinned, and fit-checked before the build) runs one Pi agent one-shot. Four immutable local contract files are loaded in fixed order into the generated system prompt (`/var/lib/mosaic/private/system-prompt.md`, private to that container). One real model request is sent noninteractively; the response must equal `MOSAIC_HELLO_OK` exactly or the diff --git a/README.md b/README.md index 051cadc94..3b65a5a9c 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,10 @@ explicit managed-name refusals with migration compatibility. ## How it works 1. `scripts/build.sh` builds the release image (`mosaic-poc-agent:-r`, - tag derived from `RELEASE` + the pinned Pi version) with Docker Compose. + tag derived from `RELEASE` + the pi version `package-lock.json` resolves, by + `scripts/image-tag.mjs`) with Docker Compose. `package.json` gives pi a range, + never a pin. Before the build, `scripts/pi-fit-image.mjs` runs the fit check + on the pi the lock resolves, so a lock that resolves an unfit pi fails it. 2. On each run, `/opt/mosaic/src/load-contracts.sh` reads the four contract files in fixed order (CONSTITUTION, STANDARDS, SOUL, USER), joins them with clear separators, and writes `/var/lib/mosaic/private/system-prompt.md` in the diff --git a/adapters/README.md b/adapters/README.md index 6faf72d30..40771f38a 100644 --- a/adapters/README.md +++ b/adapters/README.md @@ -52,7 +52,7 @@ Optional, adapter-specific (documented per adapter): ## Included adapters -- `pi` — the pinned `@earendil-works/pi-coding-agent` CLI in noninteractive +- `pi` — the `@earendil-works/pi-coding-agent` CLI (not pinned; #1873) in noninteractive print mode (`-p`), ambient discovery disabled, stdin detached. - `mock` — deterministic echo of `MOSAIC_MOCK_RESPONSE`. Test-only: never use it where a real model response is required. @@ -121,7 +121,8 @@ doesn't exist stops the adapter with exit 2. The fit check runs the headless form of this argv, with two skill directories, one `-e` file, no tools and no session, as its last `-p` run (`workerShape`, `packages/harness/README.md`); the TUI shape the launch uses is not run. The fit check runs -before bus sessions only; worker containers run the pi pinned in the image. +before bus sessions, and on the pi `package-lock.json` resolves before an +image build (`scripts/pi-fit-image.mjs`); worker containers run that pi. ### A role's skills (row 64) diff --git a/package-lock.json b/package-lock.json index bd8028aa8..7196afacb 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,7 @@ "version": "0.0.3", "license": "UNLICENSED", "dependencies": { - "@earendil-works/pi-coding-agent": "0.85.1", + "@earendil-works/pi-coding-agent": ">=0.85.1", "oauth4webapi": "3.8.8", "yaml": "2.9.0" } diff --git a/package.json b/package.json index ede39d66f..3e5ce0c15 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "description": "Minimal Mosaic Stack container proof of concept: one Pi agent, four local contract files, one real model request returning MOSAIC_HELLO_OK.", "license": "UNLICENSED", "dependencies": { - "@earendil-works/pi-coding-agent": "0.85.1", + "@earendil-works/pi-coding-agent": ">=0.85.1", "oauth4webapi": "3.8.8", "yaml": "2.9.0" } diff --git a/packages/cli/tests/gate-lane-image-base.test.mjs b/packages/cli/tests/gate-lane-image-base.test.mjs index 1fac7ddb8..b50958157 100644 --- a/packages/cli/tests/gate-lane-image-base.test.mjs +++ b/packages/cli/tests/gate-lane-image-base.test.mjs @@ -30,9 +30,10 @@ function fixture(t) { const actual = { Id: 'sha256:' + 'a'.repeat(64), Os: 'linux', Architecture: 'amd64' }; function base(root) { return /^FROM (\S+)$/m.exec(readFileSync(join(root, 'Containerfile'), 'utf8'))[1]; } function receipt(root) { - const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf8')); + // #1873 S5: the tag carries the pi version the lock resolves, never the package.json range. + const locked = JSON.parse(readFileSync(join(root, 'package-lock.json'), 'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version; const release = readFileSync(join(root, 'RELEASE'), 'utf8').trim(); - return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' }; + return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${locked}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' }; } test('image receipt records the actual pinned FROM rather than a stale mutable tag', t => { @@ -47,7 +48,8 @@ test('image receipt records the actual pinned FROM rather than a stale mutable t if (args[2] === 'node:24-bookworm-slim') return JSON.stringify({ RepoDigests: ['node@sha256:' + 'b'.repeat(64)] }); return JSON.stringify(actual); }; - const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke); + // The fixture has no node_modules, so a stand-in takes the image fit step's place (#1873 S5). + const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke, () => {}); assert.equal(result.baseDigest, selected); assert.equal(calls.some(args => args[2] === 'node:24-bookworm-slim'), false); assert.equal(calls.filter(args => args[0] === 'compose').length, 1); diff --git a/packages/cli/tests/gate-lane-image.test.mjs b/packages/cli/tests/gate-lane-image.test.mjs index 7cff43714..10c341d95 100644 --- a/packages/cli/tests/gate-lane-image.test.mjs +++ b/packages/cli/tests/gate-lane-image.test.mjs @@ -5,6 +5,9 @@ const image=await import('../../../scripts/testing/lane-image.mjs').catch(e=>{if const docker=await import('../../../scripts/testing/lane-docker.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};}); function fixtureBase(root){return /^FROM (\S+)$/m.exec(readFileSync(join(root,'Containerfile'),'utf8'))[1];} +// #1873 S5: prepareImage runs the image fit step before the build; these fixtures have no node_modules, +// so they pass a stand-in step that records that it ran. +const fitStandIn=calls=>checkout=>calls.push(['fit',checkout]); const processingInputs=['packages/processing-control/src/state.mjs','packages/processing-control/src/entrypoint.mjs']; function copyProcessingInputs(root){ mkdirSync(join(root,'packages/processing-control/src'),{recursive:true}); @@ -38,7 +41,8 @@ test('exclusive image preparation records actual image ID and build wall time an writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='compose')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/); - const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke);assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build')); + const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build')); + assert.ok(calls.findIndex(x=>x[0]==='fit')x.join(' ')==='compose build'),'the image fit step runs before the build'); }); test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{ const project='mg197-a-abcdef123456',id='123456789abc';let removed=false,label='foreign'; @@ -74,11 +78,11 @@ test('a tag changed between supervised steps fails the next boundary and suppres test('pinned-base rebuilds with identical files have distinct image IDs; cold and warm counters stay separate',t=>{ const {root}=imageFixture(t);let builds=0,id='sha256:'+'a'.repeat(64); const fake=args=>{if(args[0]==='compose'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});}; - const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake); + const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake,fitStandIn([])); assert.equal(cold.builds,1);assert.ok(cold.buildWallMs>=0);assert.equal(builds,1); for(let i=0;i<3;i++)image.verifyImage(cold,root,()=>({Id:id,Os:'linux',Architecture:'amd64'})); assert.equal(builds,1,'warm verification builds zero images'); - id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake); + id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake,fitStandIn([])); assert.equal(rebuilt.inputDigest,cold.inputDigest);assert.equal(rebuilt.builds,1);assert.equal(builds,2); assert.throws(()=>image.verifyImage(cold,root,()=>({Id:rebuilt.imageId,Os:'linux',Architecture:'amd64'})),/image ID/); }); diff --git a/packages/cli/tests/pi-unpin-s5.test.mjs b/packages/cli/tests/pi-unpin-s5.test.mjs new file mode 100644 index 000000000..54e7a3101 --- /dev/null +++ b/packages/cli/tests/pi-unpin-s5.test.mjs @@ -0,0 +1,290 @@ +// #1873 pi-unpin S5: package.json gives pi a range, the image tag comes from the version +// package-lock.json resolves, an image fit step runs the fit check on the locked pi before the +// image build, and the host launcher records pi's version without comparing it to anything. +// The image-building scripts (build.sh, release.sh, verify.sh, test-task.sh, test-auth.sh) need +// Docker or an instance config, so their tag and build logic is tested here through the shell +// functions and modules they call, against a stand-in lock. +import test from "node:test"; +import assert from "node:assert/strict"; +import { chmodSync, cpSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync, existsSync } from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { tmpdir } from "node:os"; +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { sandboxProblem } from "../../seat/src/sandbox.mjs"; +import { piStandIn } from "../../harness/tests/pi-stand-in.mjs"; + +const REPO = resolve(fileURLToPath(new URL("../../../", import.meta.url))); +const missing = (e) => { if (e.code !== "ERR_MODULE_NOT_FOUND") throw e; return {}; }; +const tagMod = await import("../../../scripts/image-tag.mjs").catch(missing); +const fitMod = await import("../../../scripts/pi-fit-image.mjs").catch(missing); +const image = await import("../../../scripts/testing/lane-image.mjs").catch(missing); +const SKIP = sandboxProblem() ? { skip: "no working bwrap here" } : {}; +const NAME = "@earendil-works/pi-coding-agent"; +const KEY = `node_modules/${NAME}`; +const RANGE = ">=0.85.1"; +const NODE_BIN = dirname(process.execPath); +const PATH = `${NODE_BIN}:/usr/bin:/bin`; + +const entry = (version) => ({ version, resolved: `https://registry.npmjs.org/${NAME}/-/pi-coding-agent-${version}.tgz`, integrity: `sha512-fixture${version}` }); +const lockOf = (version) => ({ name: "fixture", lockfileVersion: 3, requires: true, packages: { "": { name: "fixture", dependencies: { [NAME]: RANGE } }, ...(version ? { [KEY]: entry(version) } : {}) } }); + +// A checkout with package.json's range, a lock that resolves `version`, and RELEASE. +function checkout(t, { version = "9.9.9", release = "0.0.13" } = {}) { + const root = mkdtempSync(join(tmpdir(), "s5-")); + t.after(() => rmSync(root, { recursive: true, force: true })); + writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } })); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version))); + writeFileSync(join(root, "RELEASE"), `${release}\n`); + return root; +} + +// npm ci's effect in a checkout: the package, its .bin link and node_modules/.package-lock.json. +function install(root, knobs = {}, { version = "9.9.9", hidden = entry(version) } = {}) { + const pkg = join(root, KEY); + mkdirSync(join(pkg, "dist", "bundle"), { recursive: true }); + writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: NAME, version, bin: { pi: "dist/bundle/cli.js" } })); + writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn({ version, ...knobs }), { mode: 0o755 }); + mkdirSync(join(root, "node_modules", ".bin"), { recursive: true }); + symlinkSync(`../${NAME}/dist/bundle/cli.js`, join(root, "node_modules", ".bin", "pi")); + writeFileSync(join(root, "node_modules", ".package-lock.json"), JSON.stringify({ name: "fixture", lockfileVersion: 3, packages: hidden ? { [KEY]: hidden } : {} })); + return pkg; +} + +const noFit = () => { throw new Error("the fit check must not run when the locked pi can't be told"); }; + +test("pi-unpin S5: a stand-in lock at 9.9.9 gives mosaic-poc-agent:9.9.9-r, and the package.json range never appears in the tag", t => { + assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag"); + const root = checkout(t); + const tag = tagMod.imageTag(root); + assert.equal(tag, "mosaic-poc-agent:9.9.9-r0.0.13"); + assert.ok(!tag.includes(RANGE) && !tag.includes("0.85.1"), "the tag carries the lock's version, not the range"); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), root], { encoding: "utf8", env: { PATH } }); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(cli.stdout, "mosaic-poc-agent:9.9.9-r0.0.13\n"); + // The repository itself: its lock resolves 0.85.1 under the range. + assert.equal(JSON.parse(readFileSync(join(REPO, "package.json"), "utf8")).dependencies[NAME], RANGE); + assert.equal(tagMod.imageTag(REPO), `mosaic-poc-agent:${JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version}-r${readFileSync(join(REPO, "RELEASE"), "utf8").trim()}`); +}); + +test("pi-unpin S5: a lock with no pi entry, a range for a version, or a bad RELEASE refuses the tag, with no path in the line", t => { + assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag"); + const none = checkout(t, { version: null }); + assert.throws(() => tagMod.imageTag(none), { message: `image tag: package-lock.json has no entry for ${NAME}` }); + const ranged = checkout(t); + writeFileSync(join(ranged, "package-lock.json"), JSON.stringify({ lockfileVersion: 3, packages: { [KEY]: { ...entry("9.9.9"), version: RANGE } } })); + assert.throws(() => tagMod.imageTag(ranged), { message: `image tag: package-lock.json's ${NAME} version isn't a plain version` }); + const gone = checkout(t); + rmSync(join(gone, "package-lock.json")); + assert.throws(() => tagMod.imageTag(gone), { message: "image tag: package-lock.json can't be read" }); + assert.throws(() => tagMod.imageTag(checkout(t, { release: "next" })), { message: "image tag: RELEASE must be a version like 0.0.13" }); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), none], { encoding: "utf8", env: { PATH } }); + assert.equal(cli.status, 1); + assert.equal(cli.stdout, ""); + assert.ok(!cli.stderr.includes(none), "a refusal line carries no host path"); +}); + +// scripts/common.sh as the image-building scripts source it, in a stand-in checkout. +function shellCheckout(t, opts) { + const root = checkout(t, opts); + mkdirSync(join(root, "scripts")); + for (const f of ["common.sh", "image-tag.mjs"]) cpSync(join(REPO, "scripts", f), join(root, "scripts", f)); + return root; +} +const sh = (root, script, env = {}) => spawnSync("bash", ["-c", `source scripts/common.sh && ${script}`], { cwd: root, encoding: "utf8", env: { PATH, ...env } }); + +test("pi-unpin S5: load_release (build.sh, release.sh, verify.sh, test-release.sh) exports the lock's tag and refuses a lock it can't read", t => { + const root = shellCheckout(t); + const r = sh(root, 'load_release && printf "%s|%s" "$MOSAIC_RELEASE" "$MOSAIC_IMAGE_TAG"'); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stdout, "0.0.13|mosaic-poc-agent:9.9.9-r0.0.13"); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); + const bad = sh(root, 'load_release && printf "reached"'); + assert.notEqual(bad.status, 0); + assert.equal(bad.stdout, "", "a lock with no pi entry stops the script"); + assert.match(bad.stderr, /image tag: package-lock\.json has no entry/); +}); + +test("pi-unpin S5: build_image runs the image fit step first, and an unfit pi never reaches docker compose build", t => { + const root = shellCheckout(t); + mkdirSync(join(root, "bin")); + const log = join(root, "docker.log"); + writeFileSync(join(root, "bin", "docker"), `#!/bin/sh\nprintf '%s\\n' "$*" >> "${log}"\n`, { mode: 0o755 }); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `import { appendFileSync } from "node:fs";\nif (process.env.FIT === "unfit") { process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1); }\nappendFileSync(${JSON.stringify(log)}, "fit\\n");\n`); + const env = { PATH: `${join(root, "bin")}:${PATH}` }; + const unfit = sh(root, "build_image; echo after", { ...env, FIT: "unfit" }); + assert.notEqual(unfit.status, 0); + assert.equal(unfit.stdout, "", "the calling script stops at the fit step"); + assert.match(unfit.stderr, /lacks --offline/); + assert.equal(existsSync(log), false, "docker is never called for an unfit pi"); + const fit = sh(root, "build_image", env); + assert.equal(fit.status, 0, fit.stderr); + assert.equal(readFileSync(log, "utf8"), "fit\ncompose build\n"); +}); + +test("pi-unpin S5: no script builds the image tag from package.json, and every image build goes through build_image", () => { + const read = (f) => readFileSync(join(REPO, f), "utf8"); + const scripts = ["scripts/common.sh", "scripts/build.sh", "scripts/release.sh", "scripts/verify.sh", "scripts/test-task.sh", "scripts/test-auth.sh", "scripts/agent-host-dev.sh", "scripts/mosaic-task.mjs", "scripts/testing/lane-image.mjs"]; + for (const f of scripts) { + assert.doesNotMatch(read(f), /dependencies\s*(\?\.)?\[\s*['"]@earendil-works\/pi-coding-agent['"]\s*\]/, `${f} reads pi's version from package.json`); + } + for (const f of ["scripts/build.sh", "scripts/release.sh", "scripts/verify.sh"]) { + assert.match(read(f), /^\s*build_image\b/m, `${f} builds through build_image`); + assert.doesNotMatch(read(f), /docker compose build/, `${f} builds without the image fit step`); + } + for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /node scripts\/image-tag\.mjs/, `${f} takes the tag from scripts/image-tag.mjs`); + assert.match(read("scripts/mosaic-task.mjs"), /MOSAIC_IMAGE_TAG = tag;/); +}); + +// The gate's image inputs, with a lock at `version` and package.json's range. +function imageCheckout(t, version = "9.9.9") { + const root = mkdtempSync("/mnt/storage/scratch/s5i-"); + t.after(() => rmSync(root, { recursive: true, force: true })); + cpSync(join(REPO, "Containerfile"), join(root, "Containerfile")); + writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } })); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version))); + writeFileSync(join(root, "RELEASE"), "0.0.1\n"); + mkdirSync(join(root, "packages/processing-control/src"), { recursive: true }); + for (const f of ["state.mjs", "entrypoint.mjs"]) cpSync(join(REPO, "packages/processing-control/src", f), join(root, "packages/processing-control/src", f)); + for (const d of ["contracts", "src", "adapters"]) { mkdirSync(join(root, d)); writeFileSync(join(root, d, "one"), "same"); } + return root; +} +const base = (root) => /^FROM (\S+)$/m.exec(readFileSync(join(root, "Containerfile"), "utf8"))[1]; + +test("pi-unpin S5: verifyImage accepts the lock-derived tag when package.json holds a range, and refuses a tag the lock doesn't give", t => { + assert.equal(typeof image.verifyImage, "function"); + const root = imageCheckout(t); + const id = "sha256:" + "a".repeat(64); + const receipt = { inputDigest: image.imageFingerprint(root), imageId: id, tag: "mosaic-poc-agent:9.9.9-r0.0.1", baseDigest: base(root), platform: "linux/amd64" }; + const inspect = () => ({ Id: id, Os: "linux", Architecture: "amd64" }); + assert.equal(image.verifyImage(receipt, root, inspect).tag, "mosaic-poc-agent:9.9.9-r0.0.1"); + assert.throws(() => image.verifyImage({ ...receipt, tag: "mosaic-poc-agent:0.85.1-r0.0.1" }, root, inspect), /image tag differs from release identity/); + assert.throws(() => image.verifyImage({ ...receipt, tag: `mosaic-poc-agent:${RANGE}-r0.0.1` }, root, inspect), /refused/); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); + assert.throws(() => image.verifyImage({ ...receipt, inputDigest: image.imageFingerprint(root) }, root, inspect), /image tag: package-lock\.json has no entry/); +}); + +test("pi-unpin S5: prepareImage tags from the lock, runs the image fit step before compose build, and a refusing step stops the build", t => { + assert.equal(typeof image.prepareImage, "function"); + assert.equal(typeof image.imageFitStep, "function", "lane-image.mjs exports the image fit step"); + const root = imageCheckout(t); + const calls = []; + const invoke = (args) => { calls.push(args.join(" ")); return args[0] === "compose" ? "" : JSON.stringify({ Id: "sha256:" + "a".repeat(64), Os: "linux", Architecture: "amd64" }); }; + const receipt = image.prepareImage(root, { grant: "fixture", tempRoot: root }, invoke, (c) => calls.push(`fit ${c === root}`)); + assert.equal(receipt.tag, "mosaic-poc-agent:9.9.9-r0.0.1"); + assert.deepEqual(calls, ["fit true", "compose build", "image inspect mosaic-poc-agent:9.9.9-r0.0.1 --format {{json .}}"]); + const stopped = []; + assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a), () => { throw new Error("unfit"); }), /unfit/); + assert.deepEqual(stopped, [], "no build after a refusing fit step"); + // The default step runs the checkout's scripts/pi-fit-image.mjs and refuses with its last line. + mkdirSync(join(root, "scripts")); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1);\n`); + assert.throws(() => image.imageFitStep(root), { message: "gate-lane: refused: image fit step refused: pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline" }); + assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a)), /image fit step refused/); + assert.deepEqual(stopped, []); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), "process.exit(0);\n"); + assert.equal(image.imageFitStep(root), undefined); +}); + +test("pi-unpin S5: the image fit step fails on a locked stand-in pi that lacks a flag, and passes one at another version that has them all", SKIP, async t => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const unfit = checkout(t); + install(unfit, { dropFlags: ["--offline"] }); + const r = await fitMod.imageFit(unfit); + assert.deepEqual(r, { ok: false, message: "pi 9.9.9 (the lock's pi) lacks --offline" }); + const ok = checkout(t); + install(ok); + const pass = await fitMod.imageFit(ok); + assert.equal(pass.ok, true, pass.message); + assert.equal(pass.version, "9.9.9"); + assert.equal(pass.locked, "9.9.9"); +}); + +test("pi-unpin S5: the image fit step passes on the installed pi the repository's lock resolves", SKIP, async () => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const locked = JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version; + const r = await fitMod.imageFit(REPO); + assert.equal(r.ok, true, r.message); + assert.equal(r.locked, locked); + assert.equal(r.version, locked); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/pi-fit-image.mjs")], { encoding: "utf8", env: { PATH, HOME: process.env.HOME, TMPDIR: tmpdir() }, timeout: 120000 }); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(cli.stdout, `pi-fit-image: pi ${locked} (the lock's pi) passed the fit check\n`); +}); + +test("pi-unpin S5: the image fit step can't tell, and refuses, when node_modules doesn't hold the lock's pi", async t => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const lead = "pi (the lock's pi) can't be checked: "; + const npmCi = "run npm ci --ignore-scripts in the repository first"; + const cases = [ + ["no install", () => {}, `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["no hidden lock entry", (root) => install(root, {}, { hidden: null }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["another integrity", (root) => install(root, {}, { hidden: { ...entry("9.9.9"), integrity: "sha512-other" } }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["another version installed", (root) => install(root, {}, { version: "9.9.8", hidden: entry("9.9.9") }), `the installed pi's package.json doesn't carry the lock's version: ${npmCi}`], + ["a lock with no pi", (root) => { install(root); writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); }, `image tag: package-lock.json has no entry for ${NAME}`], + ["a .bin/pi that isn't executable", (root) => { install(root); chmodSync(join(root, KEY, "dist", "bundle", "cli.js"), 0o644); }, `the lookup refused the lock's pi: ${npmCi}`], + ["a .bin/pi in another copy of the package", (root) => { + install(root); + const other = join(root, "other", NAME); + mkdirSync(join(other, "dist", "bundle"), { recursive: true }); + writeFileSync(join(other, "package.json"), JSON.stringify({ name: NAME, version: "9.9.9", bin: { pi: "dist/bundle/cli.js" } })); + writeFileSync(join(other, "dist", "bundle", "cli.js"), piStandIn({ version: "9.9.9" }), { mode: 0o755 }); + rmSync(join(root, "node_modules", ".bin", "pi")); + symlinkSync(join(other, "dist", "bundle", "cli.js"), join(root, "node_modules", ".bin", "pi")); + }, `node_modules/.bin/pi isn't the lock's pi package: ${npmCi}`], + ]; + for (const [name, plant, problem] of cases) { + const root = checkout(t); + plant(root); + const r = await fitMod.imageFit(root, { fit: noFit }); + assert.deepEqual(r, { ok: false, message: lead + problem }, name); + assert.ok(!r.message.includes(root) && !r.message.includes(realpathSync(root)), `${name}: no host path in the line`); + } + // An answer the fit check doesn't give reads as can't tell. + const root = checkout(t); + install(root); + assert.deepEqual(await fitMod.imageFit(root, { fit: async () => undefined }), { ok: false, message: `${lead}the check gave no answer` }); + // The fit check gets the found pi under the step's own label, with the module's thinking levels. + let seen; + const ok = await fitMod.imageFit(root, { fit: async (pi, opts) => { seen = { pi, opts }; return { ok: true, version: "9.9.9" }; } }); + assert.equal(ok.ok, true); + assert.equal(ok.locked, "9.9.9"); + assert.equal(seen.pi.label, "the lock's pi"); + assert.equal(seen.pi.pkg, realpathSync(join(root, KEY))); + assert.equal("locked" in seen.pi, false); + assert.ok(Array.isArray(seen.opts.levels) && seen.opts.levels.includes("high")); +}); + +// scripts/agent-host-dev.sh --check in a checkout with the range in package.json and a stand-in pi. +function hostCheckout(t, piScript) { + const root = mkdtempSync(join(tmpdir(), "s5h-")); + t.after(() => rmSync(root, { recursive: true, force: true })); + for (const d of ["scripts", "agents/fixture", "contracts", "user", "node_modules/.bin", "extensions/goal"]) mkdirSync(join(root, d), { recursive: true }); + cpSync(join(REPO, "scripts/agent-host-dev.sh"), join(root, "scripts/agent-host-dev.sh")); + writeFileSync(join(root, "scripts/common.sh"), 'load_config() { MOSAIC_DATA_ROOT="$PWD"; MOSAIC_PROVIDER=test; MOSAIC_MODEL=test-model; MOSAIC_ADAPTER=pi; }\n'); + writeFileSync(join(root, "package.json"), JSON.stringify({ dependencies: { [NAME]: RANGE } })); + for (const f of ["agents/fixture/SOUL.md", "agents/fixture/CONTEXT.md", "contracts/CONSTITUTION.md", "contracts/STANDARDS.md", "user/USER.md", "AGENTS.md", "extensions/goal/index.ts"]) writeFileSync(join(root, f), `fixture ${f}\n`); + for (const s of ["ms-tools", "ms-file-read", "ms-file-write", "ms-conductor", "ms-communications", "ms-watch", "ms-sdlc", "ms-proactive-agent", "ms-goal", "ms-unslop"]) { + mkdirSync(join(root, "skills", s), { recursive: true }); + writeFileSync(join(root, "skills", s, "SKILL.md"), `fixture ${s}\n`); + } + writeFileSync(join(root, "node_modules/.bin/pi"), piScript, { mode: 0o755 }); + return root; +} +const hostCheck = (root) => spawnSync("bash", [join(root, "scripts/agent-host-dev.sh"), "fixture", "--check"], { cwd: root, encoding: "utf8", env: { PATH, HOME: root }, timeout: 20000 }); + +test("pi-unpin S5: scripts/agent-host-dev.sh doesn't refuse pi for its version; it records the version pi prints", t => { + const root = hostCheckout(t, "#!/bin/sh\n[ \"$1\" = --version ] && { echo 9.9.9; exit 0; }\nexit 1\n"); + const r = hostCheck(root); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /fixture: native host TUI \| Pi 9\.9\.9 \| test\/test-model/); + assert.match(r.stdout, /configuration checks passed/); + // A pi whose --version fails or prints nothing still starts; the banner says so. + for (const script of ["#!/bin/sh\nexit 3\n", "#!/bin/sh\nexit 0\n"]) { + const quiet = hostCheckout(t, script); + const q = hostCheck(quiet); + assert.equal(q.status, 0, q.stdout + q.stderr); + assert.match(q.stdout, /\| Pi version unknown \|/); + } + assert.doesNotMatch(readFileSync(join(REPO, "scripts/agent-host-dev.sh"), "utf8"), /package\.json pins|\$PIN\b/); +}); diff --git a/packages/mosaic/README.md b/packages/mosaic/README.md index 70a194d45..2a9dfac2f 100644 --- a/packages/mosaic/README.md +++ b/packages/mosaic/README.md @@ -36,4 +36,4 @@ Tests can supply `refresh: { mode, timeoutMs }`. Modes are rotate, unchanged, fa Limits: store locks are in-process, not a production distributed lock. Credential-store commit and filesystem publication are not crash-atomic together. A post-publication failure records uncertainty, retains the generation and rolls back the in-memory draft; this simulator has no external token issuer to reconcile. A real refresh backend would need a separate reconciliation protocol before reuse. Filesystem ownership does not defend against arbitrary same-UID tampering or establish hardlink provenance. -Exact Pi 0.85.1 isolation is static-source evidence only. Its model-catalog network flag does NOT suppress OAuth refresh networking. Never substitute real Pi in these tests, even with expired synthetic OAuth tokens. Final whole-increment review is required before publication. +The Pi isolation described here was read from the 0.85.1 source (pi is not pinned; #1873) and is static-source evidence only. Its model-catalog network flag does NOT suppress OAuth refresh networking. Never substitute real Pi in these tests, even with expired synthetic OAuth tokens. Final whole-increment review is required before publication. diff --git a/scripts/agent-host-dev.sh b/scripts/agent-host-dev.sh index 5ea108150..20df43ea1 100755 --- a/scripts/agent-host-dev.sh +++ b/scripts/agent-host-dev.sh @@ -66,9 +66,9 @@ fi USER_FILE="${USER_FILE:-$MOSAIC_DATA_ROOT/user/USER.md}" PI="$REPO/node_modules/.bin/pi" [ -x "$PI" ] || fail 'run npm ci --ignore-scripts --no-audit --no-fund in the repository first' -PIN="$(node -p "require('./package.json').dependencies['@earendil-works/pi-coding-agent']")" -PI_VERSION="$("$PI" --version)" -[ "$PI_VERSION" = "$PIN" ] || fail "local Pi is $PI_VERSION, package.json pins $PIN; run npm ci in $REPO" +# #1873 S5: no pin. The version pi prints is recorded for the banner, never compared. +PI_VERSION="$("$PI" --version 2>/dev/null | head -n 1)" || true +[ -n "$PI_VERSION" ] || PI_VERSION="version unknown" command -v flock >/dev/null || fail 'flock is required' CONTEXT_FILES=("$CONSTITUTION" "$REPO/contracts/STANDARDS.md" "$SOUL" "$USER_FILE" "$REPO/AGENTS.md" "$REPO/agents/$AGENT/CONTEXT.md") for file in "${CONTEXT_FILES[@]}"; do @@ -83,7 +83,7 @@ for skill in "${SKILLS[@]}"; do done [ -s "$REPO/extensions/goal/index.ts" ] || fail 'missing canonical goal extension' STATE="$REPO/.pi/state/$AGENT" -echo "$AGENT: native host TUI | Pi $PIN | $PROVIDER/$MODEL${THINKING:+:$THINKING}" +echo "$AGENT: native host TUI | Pi $PI_VERSION | $PROVIDER/$MODEL${THINKING:+:$THINKING}" echo "$AGENT: workspace $REPO" echo "$AGENT: SOUL=$SOUL | CONSTITUTION=$CONSTITUTION | USER=$USER_FILE" echo "$AGENT: sessions $STATE/sessions | skills: ${SKILLS[*]}" diff --git a/scripts/build.sh b/scripts/build.sh index 275989c56..99f5adc90 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -10,4 +10,4 @@ load_release bootstrap_runtime_dir -docker compose build +build_image diff --git a/scripts/common.sh b/scripts/common.sh index 87797b250..897908bef 100755 --- a/scripts/common.sh +++ b/scripts/common.sh @@ -26,17 +26,29 @@ load_config() { # Resolve the release identity: RELEASE is the single source of the # release version (stays 0.0.X until declared stable); the image tag -# derives from it plus the pinned pi dependency version. +# derives from it plus the pi version package-lock.json resolves (#1873 S5: +# package.json gives pi a range, never a pin, and the tag never carries it). +# scripts/image-tag.mjs builds the tag and refuses a lock it can't read. load_release() { - local release pi_version + local release tag release="$(tr -d '[:space:]' < RELEASE)" if ! printf '%s' "$release" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$'; then echo "common: RELEASE must be a semver-ish version, got: '$release'" >&2 exit 1 fi - pi_version="$(node -p "require('./package.json').dependencies['@earendil-works/pi-coding-agent']")" + tag="$(node scripts/image-tag.mjs .)" || exit 1 export MOSAIC_RELEASE="$release" - export MOSAIC_IMAGE_TAG="mosaic-poc-agent:${pi_version}-r${release}" + export MOSAIC_IMAGE_TAG="$tag" +} + +# Build the agent image (#1873 S5). The image fit step runs first: it runs the +# fit check on the pi package-lock.json resolves, as npm ci installed it here, +# and refuses with one line when that pi is unfit or the check can't tell, so a +# lock that resolves an unfit pi fails the build. scripts/pi-fit-image.mjs +# says why the step runs on the host and not inside the image build. +build_image() { + node scripts/pi-fit-image.mjs || exit 1 + docker compose build } # Ensure the configured runtime data directory exists and carries this diff --git a/scripts/image-tag.mjs b/scripts/image-tag.mjs new file mode 100755 index 000000000..7e3d4e685 --- /dev/null +++ b/scripts/image-tag.mjs @@ -0,0 +1,54 @@ +#!/usr/bin/env node +// #1873 pi-unpin S5: the agent image tag. package.json gives pi a range (no pin); the version the +// image installs is the one package-lock.json resolves, since the Containerfile runs npm ci. So the +// tag is built from the lock's entry for pi and RELEASE, never from the package.json string: +// mosaic-poc-agent:-r. A lock or RELEASE this can't read refuses. +// +// node scripts/image-tag.mjs [checkout] prints the tag (checkout: default this repository) +import { readFileSync, realpathSync } from "node:fs"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +export const PI_PACKAGE = "@earendil-works/pi-coding-agent"; +export const PI_LOCK_KEY = `node_modules/${PI_PACKAGE}`; +const VERSION = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; +const RELEASE = /^\d+\.\d+\.\d+$/; + +const fail = (message) => { throw new Error(`image tag: ${message}`); }; +const readJson = (path, name) => { + let text; + try { text = readFileSync(path, "utf8"); } catch { fail(`${name} can't be read`); } + try { return JSON.parse(text); } catch { fail(`${name} isn't valid JSON`); } +}; + +// The lock's entry for pi: { version, resolved, integrity }. Refuses a lock with no entry, or one +// whose version isn't a plain semver. +export function lockedPi(root) { + const lock = readJson(join(root, "package-lock.json"), "package-lock.json"); + const entry = lock?.packages?.[PI_LOCK_KEY]; + if (!entry || typeof entry !== "object") fail(`package-lock.json has no entry for ${PI_PACKAGE}`); + if (typeof entry.version !== "string" || !VERSION.test(entry.version)) fail(`package-lock.json's ${PI_PACKAGE} version isn't a plain version`); + return { version: entry.version, resolved: entry.resolved ?? null, integrity: entry.integrity ?? null }; +} + +export function release(root) { + let text; + try { text = readFileSync(join(root, "RELEASE"), "utf8"); } catch { fail("RELEASE can't be read"); } + const value = text.replace(/\s/g, ""); + if (!RELEASE.test(value)) fail("RELEASE must be a version like 0.0.13"); + return value; +} + +export const imageTag = (root) => `mosaic-poc-agent:${lockedPi(root).version}-r${release(root)}`; + +const self = fileURLToPath(import.meta.url); +let main = false; +try { main = realpathSync(process.argv[1] ?? "") === realpathSync(self); } catch {} +if (main) { + try { + process.stdout.write(`${imageTag(process.argv[2] ?? join(self, "..", ".."))}\n`); + } catch (e) { + process.stderr.write(`${e.message}\n`); + process.exit(1); + } +} diff --git a/scripts/mosaic-task.mjs b/scripts/mosaic-task.mjs index 624221c0c..725bdf942 100755 --- a/scripts/mosaic-task.mjs +++ b/scripts/mosaic-task.mjs @@ -35,6 +35,7 @@ import process from "node:process"; import { randomBytes } from "node:crypto"; import { spawnSync } from "node:child_process"; import { fileURLToPath } from "node:url"; +import { imageTag } from "./image-tag.mjs"; import { BusinessError, validateRoleDocument } from "../packages/organization/src/index.mjs"; import { RunsError, isRunId, listRunRecords, readRunRecord } from "../packages/runs/src/index.mjs"; import { initializeControl, assertProcessing } from "../packages/processing-control/src/state.mjs"; @@ -339,6 +340,10 @@ function runTask(taskFile, options = {}) { catch (e) { fail(3, e.code?.startsWith('processing-') ? e.code : 'processing-state-invalid'); } const task = validateTask(readJsonFile(taskFile, "task file"), path.resolve(taskFile)); + // #1873 S5: the image tag carries the pi version package-lock.json resolves, never the + // package.json range. Read before the run directory exists, so a lock it can't read leaves none. + let tag; + try { tag = imageTag(PROJECT_ROOT); } catch (e) { fail(4, e.message); } const runId = `r-${utcStamp()}-${randomBytes(3).toString("hex")}`; const runDir = path.join(runsRoot(resolved), runId); @@ -367,9 +372,7 @@ function runTask(taskFile, options = {}) { // (Names here are the compose interpolation consumers, not PI_*.) spawnEnv.MOSAIC_PROVIDER = resolved.execution.provider; spawnEnv.MOSAIC_MODEL = resolved.execution.model; - const release = fs.readFileSync(path.join(PROJECT_ROOT, "RELEASE"), "utf8").trim(); - const piVersion = JSON.parse(fs.readFileSync(path.join(PROJECT_ROOT, "package.json"), "utf8")).dependencies["@earendil-works/pi-coding-agent"]; - spawnEnv.MOSAIC_IMAGE_TAG = `mosaic-poc-agent:${piVersion}-r${release}`; + spawnEnv.MOSAIC_IMAGE_TAG = tag; // Workspace (M5): create only the selected host source. The container // launcher gives it the fixed /workspace target and mounts no parent. diff --git a/scripts/pi-fit-image.mjs b/scripts/pi-fit-image.mjs new file mode 100755 index 000000000..1cfe8693a --- /dev/null +++ b/scripts/pi-fit-image.mjs @@ -0,0 +1,70 @@ +#!/usr/bin/env node +// #1873 pi-unpin S5: the image fit step. The agent image installs the pi package-lock.json resolves +// (npm ci), and nothing pins its version. This step runs the fit check (packages/harness/src/ +// pi-fit.mjs, as sessions run it) on that pi before the image is built, so a lock that resolves an +// unfit pi fails the build. scripts/build.sh, scripts/release.sh package, scripts/verify.sh and the +// gate's image preparation run it first. +// +// It runs on the host, not as a Containerfile RUN step: the fit check runs pi inside bwrap, and +// docker build's default seccomp profile refuses the namespaces bwrap needs. The host copy stands for +// the image's: npm ci installs both from the same lock entry, and this step refuses unless the +// checkout's node_modules holds exactly that entry (version, resolved and integrity, as npm recorded +// them in node_modules/.package-lock.json, and the installed package.json's version). +// +// node scripts/pi-fit-image.mjs exit 0: passed; exit 1: one line on stderr, unfit or can't tell +import { readFileSync, realpathSync } from "node:fs"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { lockedPi, PI_LOCK_KEY } from "./image-tag.mjs"; +import { locate } from "../packages/plugins/harness/pi/binary.mjs"; +import { levels } from "../packages/plugins/harness/pi/index.mjs"; +import { piFit } from "../packages/harness/src/pi-fit.mjs"; + +// The name refusal lines use for this pi. Never a path. +export const LABEL = "the lock's pi"; +const RUN_NPM_CI = "run npm ci --ignore-scripts in the repository first"; + +const readJson = (path) => { try { return JSON.parse(readFileSync(path, "utf8")); } catch { return null; } }; +const real = (path) => { try { return realpathSync(path); } catch { return null; } }; + +// The lookup's answer for the pi npm installed from the lock in root, relabelled, or +// { bin: null, problem }. lookup: binary.mjs's locate (tests pass a stand-in). +export function lockedFound(root, { lookup = locate } = {}) { + let locked; + try { locked = lockedPi(root); } catch (e) { return { bin: null, problem: e.message }; } + const installed = readJson(join(root, "node_modules", ".package-lock.json"))?.packages?.[PI_LOCK_KEY]; + if (!installed || installed.version !== locked.version || installed.resolved !== locked.resolved || installed.integrity !== locked.integrity) { + return { bin: null, problem: `the installed pi isn't the one package-lock.json resolves: ${RUN_NPM_CI}` }; + } + const dir = join(root, PI_LOCK_KEY); + if (readJson(join(dir, "package.json"))?.version !== locked.version) { + return { bin: null, problem: `the installed pi's package.json doesn't carry the lock's version: ${RUN_NPM_CI}` }; + } + const found = lookup({ MOSAIC_PI_BIN: join(root, "node_modules", ".bin", "pi") }); + if (!found?.bin) return { bin: null, problem: `the lookup refused the lock's pi: ${RUN_NPM_CI}` }; + if (found.pkg !== real(dir)) return { bin: null, problem: `node_modules/.bin/pi isn't the lock's pi package: ${RUN_NPM_CI}` }; + return { ...found, label: LABEL, locked: locked.version }; +} + +// The fit check's answer for root's locked pi: { ok: true, version, ... } or { ok: false, message }. +// fit: pi-fit.mjs's piFit (tests may pass a stand-in). +export async function imageFit(root, { lookup = locate, fit = piFit } = {}) { + const found = lockedFound(root, { lookup }); + if (!found.bin) return { ok: false, message: `pi (${LABEL}) can't be checked: ${found.problem}` }; + const { locked, ...pi } = found; + const result = await fit(pi, { levels: levels() }); + return result?.ok === true ? { ...result, locked } : { ok: false, message: typeof result?.message === "string" ? result.message : `pi (${LABEL}) can't be checked: the check gave no answer` }; +} + +const self = fileURLToPath(import.meta.url); +let main = false; +try { main = realpathSync(process.argv[1] ?? "") === realpathSync(self); } catch {} +if (main) { + const result = await imageFit(join(self, "..", "..")); + if (result.ok) { + process.stdout.write(`pi-fit-image: pi ${result.locked} (${LABEL}) passed the fit check\n`); + } else { + process.stderr.write(`pi-fit-image: ${result.message.split("\n")[0]}\n`); + process.exit(1); + } +} diff --git a/scripts/release.sh b/scripts/release.sh index 5c05375cf..c321a3312 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -119,7 +119,7 @@ console.log("active: " + p.release + " (" + p.imageTag + ") since " + p.activ } cmd_package() { - docker compose build + build_image append_log "package" "$MOSAIC_RELEASE" "$MOSAIC_IMAGE_TAG" echo "release: packaged $MOSAIC_IMAGE_TAG" } diff --git a/scripts/test-auth.sh b/scripts/test-auth.sh index e9580f8c5..f1bf6436d 100755 --- a/scripts/test-auth.sh +++ b/scripts/test-auth.sh @@ -36,7 +36,8 @@ CFG="MOSAIC_CONFIG=$SANDBOX/mock-config.json" # release pointer so agent.sh's ensure stays on the no-op fast path REL="$(tr -d '[:space:]' < RELEASE)" -printf '{"pointerVersion":1,"release":"%s","imageTag":"mosaic-poc-agent:0.85.1-r%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$REL" > "$SANDBOX/data/state/active.json" +TAG="$(node scripts/image-tag.mjs)" # #1873 S5: the tag the lock gives, never a pinned string +printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" # fixtures: a default-harness credential and two mosaic-managed accounts printf '{"anthropic":{"type":"oauth","access":"%s"},"zai":{"type":"api_key","key":"%s"}}' "$TOKEN" "$SECRET" > "$SANDBOX/pi/auth.json" diff --git a/scripts/test-task.sh b/scripts/test-task.sh index bbc9942fb..7b86a405c 100755 --- a/scripts/test-task.sh +++ b/scripts/test-task.sh @@ -266,7 +266,8 @@ EOF EOF REL="$(tr -d '[:space:]' < RELEASE)" mkdir -p "$SANDBOX/data/state" - printf '{"pointerVersion":1,"release":"%s","imageTag":"mosaic-poc-agent:0.85.1-r%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$REL" > "$SANDBOX/data/state/active.json" + TAG="$(node scripts/image-tag.mjs)" # #1873 S5: the tag the lock gives, never a pinned string + printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" scripts/skill.sh install ms-tools >/dev/null 2>&1 scripts/skill.sh activate ms-tools >/dev/null 2>&1 env MOSAIC_CONFIG="$SANDBOX/mock-config.json" MOSAIC_MOCK_RESPONSE=MOCKED \ diff --git a/scripts/testing/lane-image.mjs b/scripts/testing/lane-image.mjs index 303759e6f..b545fa612 100644 --- a/scripts/testing/lane-image.mjs +++ b/scripts/testing/lane-image.mjs @@ -3,6 +3,8 @@ import { readFileSync, readdirSync, lstatSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { execFileSync } from 'node:child_process'; import { refuse } from './lane-state.mjs'; +import { imageTag } from '../image-tag.mjs'; +const lockTag=checkout=>{try{return imageTag(checkout);}catch(e){refuse(e.message);}}; // This is the declared build input, not an observation of a running binary. function pinnedBase(checkout){ const lines=readFileSync(join(checkout,'Containerfile'),'utf8').split('\n').filter(line=>/^\s*FROM(?:\s|$)/i.test(line)); @@ -31,19 +33,26 @@ export function verifyImage(receipt,checkout,inspect=inspectImage){ const actual=inspect(receipt.tag); if(actual.Id!==receipt.imageId)refuse('pinned image ID differs'); if(`${actual.Os}/${actual.Architecture}`!==receipt.platform)refuse('image platform differs'); - const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim();const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8')); - if(pkg.dependencies?.['@earendil-works/pi-coding-agent']&&receipt.tag!==`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`)refuse('image tag differs from release identity'); + // #1873 S5: the tag carries the pi version package-lock.json resolves, never the package.json range. + if(receipt.tag!==lockTag(checkout))refuse('image tag differs from release identity'); return receipt; } // Called by the CLI only while both lanes and the image lock are exclusive. -export function prepareImage(checkout,{grant,tempRoot,out},invoke){ +// #1873 S5: the image fit step. The checkout's own scripts/pi-fit-image.mjs runs the fit check on the +// pi package-lock.json resolves, as npm ci installed it in the checkout, before the build. It exits +// non-zero with one line when that pi is unfit or the check can't tell. +export function imageFitStep(checkout){ + try{execFileSync(process.execPath,[join(checkout,'scripts/pi-fit-image.mjs')],{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.TMPDIR?{TMPDIR:process.env.TMPDIR}:{})}});} + catch(e){const line=String(e.stderr??'').trim().split('\n').pop();refuse(`image fit step refused: ${line||'it could not run'}`);} +} +export function prepareImage(checkout,{grant,tempRoot,out},invoke,fit=imageFitStep){ if(!grant)refuse('missing image preparation grant'); const baseDigest=pinnedBase(checkout); const inputDigest=imageFingerprint(checkout); - const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8')); const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim(); if(!/^\d+\.\d+\.\d+$/.test(release))refuse('invalid image release'); - const tag=`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`; + const tag=lockTag(checkout); + fit(checkout); const run=invoke??((args)=>execFileSync('docker',args,{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.DOCKER_HOST?{DOCKER_HOST:process.env.DOCKER_HOST}:{}),COMPOSE_PROJECT_NAME:'mg197-image-prepare',MOSAIC_IMAGE_TAG:tag,MOSAIC_DATA_ROOT:join(tempRoot,'data'),MOSAIC_PROVIDER:'fixture',MOSAIC_MODEL:'fixture',PI_AUTH_FILE:'/dev/null'}})); const start=performance.now();let buildOutput;try{buildOutput=run(['compose','build']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start; if(inputDigest!==imageFingerprint(checkout))refuse('image inputs changed while building'); diff --git a/scripts/verify.sh b/scripts/verify.sh index ffddc89b9..8798cea1e 100755 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -37,7 +37,7 @@ ensure_release_aligned if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then echo "verify: image $IMAGE not found, building..." >&2 bootstrap_runtime_dir - docker compose build + build_image fi # 2. Run the agent request (stdout only = model response). -- 2.54.0 From 5210522e13d461c3fe91e18fd2aed5137e2e6b7c Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Fri, 20 Nov 2026 07:57:53 -0600 Subject: [PATCH 2/4] fix(release): pi-unpin S5 review round: compose.yaml loses its build section and sets pull_policy: never, so a compose run (hello.sh, agent.sh, the task runner) never builds or pulls the agent image past the fit step and refuses a missing one; build_image runs docker build -f Containerfile, refuses without MOSAIC_IMAGE_TAG, and with the gate's prepareImage is the only builder; test-task.sh and test-auth.sh stop when the tag can't be read; the gate test's order check no longer passes on a missing call; README and pi-fit-image.mjs say the fit check runs on the host's node, which can differ from the image's (#1873, lane pi-unpin-s5) --- README.md | 10 +++++--- compose.yaml | 8 +++--- .../cli/tests/gate-lane-image-base.test.mjs | 4 +-- packages/cli/tests/gate-lane-image.test.mjs | 8 +++--- packages/cli/tests/pi-unpin-s5.test.mjs | 25 +++++++++++++------ scripts/common.sh | 8 +++++- scripts/pi-fit-image.mjs | 2 ++ scripts/test-auth.sh | 2 +- scripts/test-task.sh | 2 +- scripts/testing/lane-image.mjs | 5 ++-- 10 files changed, 49 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 3b65a5a9c..00f21222e 100644 --- a/README.md +++ b/README.md @@ -194,9 +194,13 @@ explicit managed-name refusals with migration compatibility. 1. `scripts/build.sh` builds the release image (`mosaic-poc-agent:-r`, tag derived from `RELEASE` + the pi version `package-lock.json` resolves, by - `scripts/image-tag.mjs`) with Docker Compose. `package.json` gives pi a range, - never a pin. Before the build, `scripts/pi-fit-image.mjs` runs the fit check - on the pi the lock resolves, so a lock that resolves an unfit pi fails it. + `scripts/image-tag.mjs`) with `docker build -f Containerfile`. `package.json` + gives pi a range, never a pin. Before the build, `scripts/pi-fit-image.mjs` + runs the fit check on the pi the lock resolves, so a lock that resolves an + unfit pi fails it. The check runs on the host's node, which can differ from + the image's (the Containerfile's `FROM`). `compose.yaml` has no build section + and `pull_policy: never`, so a compose run never builds or pulls the image; + it refuses a missing one. 2. On each run, `/opt/mosaic/src/load-contracts.sh` reads the four contract files in fixed order (CONSTITUTION, STANDARDS, SOUL, USER), joins them with clear separators, and writes `/var/lib/mosaic/private/system-prompt.md` in the diff --git a/compose.yaml b/compose.yaml index ec8021551..83e520b43 100644 --- a/compose.yaml +++ b/compose.yaml @@ -1,8 +1,10 @@ services: mosaic-agent: - build: - context: . - dockerfile: Containerfile + # #1873 S5: no build section, and never pulled. The image is built only by + # build_image (scripts/common.sh) or the gate's prepareImage, which run the + # image fit step first, so a missing image refuses here instead of compose + # building one that skipped the fit check. + pull_policy: never image: ${MOSAIC_IMAGE_TAG:?MOSAIC_IMAGE_TAG must be set by scripts/load_release (run via scripts/*.sh)} user: "1000:1000" environment: diff --git a/packages/cli/tests/gate-lane-image-base.test.mjs b/packages/cli/tests/gate-lane-image-base.test.mjs index b50958157..86a5ddffd 100644 --- a/packages/cli/tests/gate-lane-image-base.test.mjs +++ b/packages/cli/tests/gate-lane-image-base.test.mjs @@ -44,7 +44,7 @@ test('image receipt records the actual pinned FROM rather than a stale mutable t const calls = []; const invoke = args => { calls.push(args); - if (args[0] === 'compose') return ''; + if (args[0] === 'build') return ''; if (args[2] === 'node:24-bookworm-slim') return JSON.stringify({ RepoDigests: ['node@sha256:' + 'b'.repeat(64)] }); return JSON.stringify(actual); }; @@ -52,7 +52,7 @@ test('image receipt records the actual pinned FROM rather than a stale mutable t const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke, () => {}); assert.equal(result.baseDigest, selected); assert.equal(calls.some(args => args[2] === 'node:24-bookworm-slim'), false); - assert.equal(calls.filter(args => args[0] === 'compose').length, 1); + assert.equal(calls.filter(args => args[0] === 'build').length, 1); }); test('a well-formed stale base receipt refuses before any image inspection', t => { diff --git a/packages/cli/tests/gate-lane-image.test.mjs b/packages/cli/tests/gate-lane-image.test.mjs index 10c341d95..9305a101b 100644 --- a/packages/cli/tests/gate-lane-image.test.mjs +++ b/packages/cli/tests/gate-lane-image.test.mjs @@ -39,10 +39,10 @@ test('exclusive image preparation records actual image ID and build wall time an for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f)); copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); - const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='compose')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; + const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='build')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/); - const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build')); - assert.ok(calls.findIndex(x=>x[0]==='fit')x.join(' ')==='compose build'),'the image fit step runs before the build'); + const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='build -f Containerfile -t mosaic-poc-agent:0.85.1-r0.0.1 .'),'a plain docker build of the Containerfile under the lock tag (compose.yaml has no build section)'); + const fitAt=calls.findIndex(x=>x[0]==='fit');assert.ok(fitAt>=0&&fitAtx[0]==='build'),'the image fit step runs, before the build'); }); test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{ const project='mg197-a-abcdef123456',id='123456789abc';let removed=false,label='foreign'; @@ -77,7 +77,7 @@ test('a tag changed between supervised steps fails the next boundary and suppres }); test('pinned-base rebuilds with identical files have distinct image IDs; cold and warm counters stay separate',t=>{ const {root}=imageFixture(t);let builds=0,id='sha256:'+'a'.repeat(64); - const fake=args=>{if(args[0]==='compose'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});}; + const fake=args=>{if(args[0]==='build'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});}; const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake,fitStandIn([])); assert.equal(cold.builds,1);assert.ok(cold.buildWallMs>=0);assert.equal(builds,1); for(let i=0;i<3;i++)image.verifyImage(cold,root,()=>({Id:id,Os:'linux',Architecture:'amd64'})); diff --git a/packages/cli/tests/pi-unpin-s5.test.mjs b/packages/cli/tests/pi-unpin-s5.test.mjs index 54e7a3101..3a15a2a74 100644 --- a/packages/cli/tests/pi-unpin-s5.test.mjs +++ b/packages/cli/tests/pi-unpin-s5.test.mjs @@ -105,21 +105,25 @@ test("pi-unpin S5: load_release (build.sh, release.sh, verify.sh, test-release.s assert.match(bad.stderr, /image tag: package-lock\.json has no entry/); }); -test("pi-unpin S5: build_image runs the image fit step first, and an unfit pi never reaches docker compose build", t => { +test("pi-unpin S5: build_image runs the image fit step first, and an unfit pi never reaches the image build", t => { const root = shellCheckout(t); mkdirSync(join(root, "bin")); const log = join(root, "docker.log"); writeFileSync(join(root, "bin", "docker"), `#!/bin/sh\nprintf '%s\\n' "$*" >> "${log}"\n`, { mode: 0o755 }); writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `import { appendFileSync } from "node:fs";\nif (process.env.FIT === "unfit") { process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1); }\nappendFileSync(${JSON.stringify(log)}, "fit\\n");\n`); const env = { PATH: `${join(root, "bin")}:${PATH}` }; - const unfit = sh(root, "build_image; echo after", { ...env, FIT: "unfit" }); + const unfit = sh(root, "load_release && build_image; echo after", { ...env, FIT: "unfit" }); assert.notEqual(unfit.status, 0); assert.equal(unfit.stdout, "", "the calling script stops at the fit step"); assert.match(unfit.stderr, /lacks --offline/); assert.equal(existsSync(log), false, "docker is never called for an unfit pi"); - const fit = sh(root, "build_image", env); + const fit = sh(root, "load_release && build_image", env); assert.equal(fit.status, 0, fit.stderr); - assert.equal(readFileSync(log, "utf8"), "fit\ncompose build\n"); + assert.equal(readFileSync(log, "utf8"), "fit\nbuild -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.13 .\n"); + // Without load_release there is no tag to build under. + const untagged = spawnSync("bash", ["-c", "source scripts/common.sh && build_image"], { cwd: root, encoding: "utf8", env }); + assert.notEqual(untagged.status, 0); + assert.match(untagged.stderr, /build_image needs MOSAIC_IMAGE_TAG from load_release/); }); test("pi-unpin S5: no script builds the image tag from package.json, and every image build goes through build_image", () => { @@ -130,8 +134,13 @@ test("pi-unpin S5: no script builds the image tag from package.json, and every i } for (const f of ["scripts/build.sh", "scripts/release.sh", "scripts/verify.sh"]) { assert.match(read(f), /^\s*build_image\b/m, `${f} builds through build_image`); - assert.doesNotMatch(read(f), /docker compose build/, `${f} builds without the image fit step`); + assert.doesNotMatch(read(f), /docker (compose )?build/, `${f} builds without the image fit step`); } + // compose.yaml can't build or pull the agent image, so a missing image refuses at compose run + // instead of being built without the fit step. + const compose = read("compose.yaml"); + assert.doesNotMatch(compose, /^\s*build:/m, "compose.yaml has no build section"); + assert.match(compose, /^\s*pull_policy: never$/m); for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /node scripts\/image-tag\.mjs/, `${f} takes the tag from scripts/image-tag.mjs`); assert.match(read("scripts/mosaic-task.mjs"), /MOSAIC_IMAGE_TAG = tag;/); }); @@ -164,15 +173,15 @@ test("pi-unpin S5: verifyImage accepts the lock-derived tag when package.json ho assert.throws(() => image.verifyImage({ ...receipt, inputDigest: image.imageFingerprint(root) }, root, inspect), /image tag: package-lock\.json has no entry/); }); -test("pi-unpin S5: prepareImage tags from the lock, runs the image fit step before compose build, and a refusing step stops the build", t => { +test("pi-unpin S5: prepareImage tags from the lock, runs the image fit step before the build, and a refusing step stops the build", t => { assert.equal(typeof image.prepareImage, "function"); assert.equal(typeof image.imageFitStep, "function", "lane-image.mjs exports the image fit step"); const root = imageCheckout(t); const calls = []; - const invoke = (args) => { calls.push(args.join(" ")); return args[0] === "compose" ? "" : JSON.stringify({ Id: "sha256:" + "a".repeat(64), Os: "linux", Architecture: "amd64" }); }; + const invoke = (args) => { calls.push(args.join(" ")); return args[0] === "build" ? "" : JSON.stringify({ Id: "sha256:" + "a".repeat(64), Os: "linux", Architecture: "amd64" }); }; const receipt = image.prepareImage(root, { grant: "fixture", tempRoot: root }, invoke, (c) => calls.push(`fit ${c === root}`)); assert.equal(receipt.tag, "mosaic-poc-agent:9.9.9-r0.0.1"); - assert.deepEqual(calls, ["fit true", "compose build", "image inspect mosaic-poc-agent:9.9.9-r0.0.1 --format {{json .}}"]); + assert.deepEqual(calls, ["fit true", "build -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.1 .", "image inspect mosaic-poc-agent:9.9.9-r0.0.1 --format {{json .}}"]); const stopped = []; assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a), () => { throw new Error("unfit"); }), /unfit/); assert.deepEqual(stopped, [], "no build after a refusing fit step"); diff --git a/scripts/common.sh b/scripts/common.sh index 897908bef..f92736ed1 100755 --- a/scripts/common.sh +++ b/scripts/common.sh @@ -46,9 +46,15 @@ load_release() { # and refuses with one line when that pi is unfit or the check can't tell, so a # lock that resolves an unfit pi fails the build. scripts/pi-fit-image.mjs # says why the step runs on the host and not inside the image build. +# compose.yaml has no build section, so this is the only way the scripts build +# the image; compose run refuses a missing one. Call load_release first. build_image() { + if [ -z "${MOSAIC_IMAGE_TAG:-}" ]; then + echo "common: build_image needs MOSAIC_IMAGE_TAG from load_release" >&2 + exit 1 + fi node scripts/pi-fit-image.mjs || exit 1 - docker compose build + docker build -f Containerfile -t "$MOSAIC_IMAGE_TAG" . } # Ensure the configured runtime data directory exists and carries this diff --git a/scripts/pi-fit-image.mjs b/scripts/pi-fit-image.mjs index 1cfe8693a..21a1078bc 100755 --- a/scripts/pi-fit-image.mjs +++ b/scripts/pi-fit-image.mjs @@ -10,6 +10,8 @@ // the image's: npm ci installs both from the same lock entry, and this step refuses unless the // checkout's node_modules holds exactly that entry (version, resolved and integrity, as npm recorded // them in node_modules/.package-lock.json, and the installed package.json's version). +// It runs pi on the host's node, which can be a different version from the image's (the +// Containerfile's FROM). Running it on the image's node is left for later. // // node scripts/pi-fit-image.mjs exit 0: passed; exit 1: one line on stderr, unfit or can't tell import { readFileSync, realpathSync } from "node:fs"; diff --git a/scripts/test-auth.sh b/scripts/test-auth.sh index f1bf6436d..48c2c2ea0 100755 --- a/scripts/test-auth.sh +++ b/scripts/test-auth.sh @@ -36,7 +36,7 @@ CFG="MOSAIC_CONFIG=$SANDBOX/mock-config.json" # release pointer so agent.sh's ensure stays on the no-op fast path REL="$(tr -d '[:space:]' < RELEASE)" -TAG="$(node scripts/image-tag.mjs)" # #1873 S5: the tag the lock gives, never a pinned string +TAG="$(node scripts/image-tag.mjs)" || exit 1 # #1873 S5: the tag the lock gives, never a pinned string printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" # fixtures: a default-harness credential and two mosaic-managed accounts diff --git a/scripts/test-task.sh b/scripts/test-task.sh index 7b86a405c..06fb5b401 100755 --- a/scripts/test-task.sh +++ b/scripts/test-task.sh @@ -266,7 +266,7 @@ EOF EOF REL="$(tr -d '[:space:]' < RELEASE)" mkdir -p "$SANDBOX/data/state" - TAG="$(node scripts/image-tag.mjs)" # #1873 S5: the tag the lock gives, never a pinned string + TAG="$(node scripts/image-tag.mjs)" || exit 1 # #1873 S5: the tag the lock gives, never a pinned string printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" scripts/skill.sh install ms-tools >/dev/null 2>&1 scripts/skill.sh activate ms-tools >/dev/null 2>&1 diff --git a/scripts/testing/lane-image.mjs b/scripts/testing/lane-image.mjs index b545fa612..1d4571f16 100644 --- a/scripts/testing/lane-image.mjs +++ b/scripts/testing/lane-image.mjs @@ -40,7 +40,8 @@ export function verifyImage(receipt,checkout,inspect=inspectImage){ // Called by the CLI only while both lanes and the image lock are exclusive. // #1873 S5: the image fit step. The checkout's own scripts/pi-fit-image.mjs runs the fit check on the // pi package-lock.json resolves, as npm ci installed it in the checkout, before the build. It exits -// non-zero with one line when that pi is unfit or the check can't tell. +// non-zero with one line when that pi is unfit or the check can't tell. compose.yaml has no build +// section, so the build below is a plain docker build of the Containerfile under the lock's tag. export function imageFitStep(checkout){ try{execFileSync(process.execPath,[join(checkout,'scripts/pi-fit-image.mjs')],{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.TMPDIR?{TMPDIR:process.env.TMPDIR}:{})}});} catch(e){const line=String(e.stderr??'').trim().split('\n').pop();refuse(`image fit step refused: ${line||'it could not run'}`);} @@ -54,7 +55,7 @@ export function prepareImage(checkout,{grant,tempRoot,out},invoke,fit=imageFitSt const tag=lockTag(checkout); fit(checkout); const run=invoke??((args)=>execFileSync('docker',args,{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.DOCKER_HOST?{DOCKER_HOST:process.env.DOCKER_HOST}:{}),COMPOSE_PROJECT_NAME:'mg197-image-prepare',MOSAIC_IMAGE_TAG:tag,MOSAIC_DATA_ROOT:join(tempRoot,'data'),MOSAIC_PROVIDER:'fixture',MOSAIC_MODEL:'fixture',PI_AUTH_FILE:'/dev/null'}})); - const start=performance.now();let buildOutput;try{buildOutput=run(['compose','build']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start; + const start=performance.now();let buildOutput;try{buildOutput=run(['build','-f','Containerfile','-t',tag,'.']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start; if(inputDigest!==imageFingerprint(checkout))refuse('image inputs changed while building'); const actual=JSON.parse(run(['image','inspect',tag,'--format','{{json .}}'])); const receipt={inputDigest,imageId:actual.Id,tag,baseDigest,platform:`${actual.Os}/${actual.Architecture}`,grant,builds:1,buildWallMs,at:new Date().toISOString()}; -- 2.54.0 From 9d02ab4a69caee8634e5081c69fecd9e92bc64a9 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Fri, 20 Nov 2026 08:08:20 -0600 Subject: [PATCH 3/4] test(release): pi-unpin S5 re-check round: gate-lane-image.test.mjs reads the expected tag from the copied lock instead of 0.85.1, so it follows the lock to a newer pi; pi-unpin-s5.test.mjs requires test-task.sh and test-auth.sh to stop when image-tag.mjs refuses; build.sh's header and docs/TOOLS.md say the tag comes from the lock, build.sh runs the fit step, and build.sh comes before test-task.sh since compose no longer builds a missing image (#1873, lane pi-unpin-s5) --- docs/TOOLS.md | 4 ++-- packages/cli/tests/gate-lane-image.test.mjs | 9 ++++++--- packages/cli/tests/pi-unpin-s5.test.mjs | 3 ++- scripts/build.sh | 3 ++- 4 files changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/TOOLS.md b/docs/TOOLS.md index f924deee4..436fc896f 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -16,7 +16,7 @@ and changes nothing. | Command | Purpose | Notes | |---|---|---| | `scripts/bootstrap.sh [--instance NAME]` | Create `~/.config/mosaic-dev/config.json` if absent; with `--instance NAME`, `~/.config/mosaic-NAME/config.json` and the claimed root `~/.mosaic-NAME` | Idempotent; existing config validated, never rewritten. See [Stack instances](#stack-instances---instance-name-row-190) | -| `scripts/build.sh` | Build the release image | Tag derived from `RELEASE` + pinned pi version | +| `scripts/build.sh` | Build the release image | Tag `mosaic-poc-agent:-r`, pi being the version `package-lock.json` resolves (`scripts/image-tag.mjs`); runs the image fit step (`scripts/pi-fit-image.mjs`) first. compose runs never build or pull the image, so run this before anything that needs a new tag | | `scripts/hello.sh` | One-shot startup request | Prints model response on stdout | | `scripts/verify.sh` | Full gated test | Exit 0 only on exact `MOSAIC_HELLO_OK`; `EXPECTED_MARKER` overrides for negative drills | @@ -65,8 +65,8 @@ addition to its full repository gate; it was not run by this source repair: ```bash node --test --test-concurrency=1 packages/processing-control/tests/containers.test.mjs packages/processing-control/tests/coordinator.test.mjs packages/processing-control/tests/effects.test.mjs packages/processing-control/tests/mount.test.mjs packages/processing-control/tests/runtimes.test.mjs packages/processing-control/tests/state.test.mjs packages/processing-control/tests/stop-run.test.mjs packages/processing-control/tests/task.test.mjs node --test scripts/test-darkwing-launch.mjs -scripts/test-task.sh scripts/build.sh +scripts/test-task.sh ``` ## Agent (interactive TUI) diff --git a/packages/cli/tests/gate-lane-image.test.mjs b/packages/cli/tests/gate-lane-image.test.mjs index 9305a101b..8c9e39c3d 100644 --- a/packages/cli/tests/gate-lane-image.test.mjs +++ b/packages/cli/tests/gate-lane-image.test.mjs @@ -8,6 +8,9 @@ function fixtureBase(root){return /^FROM (\S+)$/m.exec(readFileSync(join(root,'C // #1873 S5: prepareImage runs the image fit step before the build; these fixtures have no node_modules, // so they pass a stand-in step that records that it ran. const fitStandIn=calls=>checkout=>calls.push(['fit',checkout]); +// The tag the fixture's copied lock gives (RELEASE 0.0.1), read here from the lock, so these tests +// follow the lock when it resolves a newer pi. +const lockTagOf=root=>'mosaic-poc-agent:'+JSON.parse(readFileSync(join(root,'package-lock.json'),'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version+'-r0.0.1'; const processingInputs=['packages/processing-control/src/state.mjs','packages/processing-control/src/entrypoint.mjs']; function copyProcessingInputs(root){ mkdirSync(join(root,'packages/processing-control/src'),{recursive:true}); @@ -21,7 +24,7 @@ test('a changed COPY input or actual image ID refuses compatible image sharing', copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n'); for(const d of ['contracts','src','adapters']){mkdirSync(join(root,d));writeFileSync(join(root,d,'one'),'same');} - const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'}; + const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'}; const inspect=()=>({Id:id,Os:'linux',Architecture:'amd64'}); assert.equal(image.verifyImage(receipt,root,inspect).imageId,id); assert.throws(()=>image.verifyImage(receipt,root,()=>({...inspect(),Id:'sha256:'+'c'.repeat(64)})),/image ID/); @@ -41,7 +44,7 @@ test('exclusive image preparation records actual image ID and build wall time an writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='build')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/); - const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='build -f Containerfile -t mosaic-poc-agent:0.85.1-r0.0.1 .'),'a plain docker build of the Containerfile under the lock tag (compose.yaml has no build section)'); + const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='build -f Containerfile -t '+lockTagOf(root)+' .'),'a plain docker build of the Containerfile under the lock tag (compose.yaml has no build section)'); const fitAt=calls.findIndex(x=>x[0]==='fit');assert.ok(fitAt>=0&&fitAtx[0]==='build'),'the image fit step runs, before the build'); }); test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{ @@ -56,7 +59,7 @@ function imageFixture(t){ for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f)); copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); - const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'}; + const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'}; return {root,receipt}; } test('missing receipt and missing image refuse without permitting a gate',t=>{ diff --git a/packages/cli/tests/pi-unpin-s5.test.mjs b/packages/cli/tests/pi-unpin-s5.test.mjs index 3a15a2a74..e2eccd1b3 100644 --- a/packages/cli/tests/pi-unpin-s5.test.mjs +++ b/packages/cli/tests/pi-unpin-s5.test.mjs @@ -141,7 +141,8 @@ test("pi-unpin S5: no script builds the image tag from package.json, and every i const compose = read("compose.yaml"); assert.doesNotMatch(compose, /^\s*build:/m, "compose.yaml has no build section"); assert.match(compose, /^\s*pull_policy: never$/m); - for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /node scripts\/image-tag\.mjs/, `${f} takes the tag from scripts/image-tag.mjs`); + // The tag comes from scripts/image-tag.mjs, and a refusal stops the script instead of running on an empty tag. + for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /^\s*TAG="\$\(node scripts\/image-tag\.mjs\)" \|\| exit 1(\s|$)/m, `${f} takes the tag from scripts/image-tag.mjs and stops when it can't`); assert.match(read("scripts/mosaic-task.mjs"), /MOSAIC_IMAGE_TAG = tag;/); }); diff --git a/scripts/build.sh b/scripts/build.sh index 99f5adc90..c274d25e0 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# Build the mosaic-agent container image using Docker Compose. +# Build the mosaic-agent container image through build_image in common.sh: the image fit step, +# then the build. compose.yaml has no build section, so this is how the image gets built. set -euo pipefail cd "$(dirname "$0")/.." # shellcheck source=common.sh -- 2.54.0 From 9287e2fac7ac52642f9961d52acba4c38caecf63 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Fri, 20 Nov 2026 09:12:32 -0600 Subject: [PATCH 4/4] docs(agents): AGENTS.md "Version pin" becomes "Harness versions": no harness binary is pinned, package.json holds a floor range and the lock records what npm ci installs, the fit check and the host timer stand where the pin stood, and image tags derive from RELEASE and the lock (#1873, lane pi-unpin-s5) --- AGENTS.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e29901feb..5ed3c84ec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -222,8 +222,18 @@ Compacted, restarted, or new? Nothing that matters is lost: this file + `git log --oneline -10` + the suites reconstruct the full state. **Never guess** — verify with the suites; the run records and logs hold the receipts. -## Version pin +## Harness versions -`@earendil-works/pi-coding-agent` is pinned exactly (see `package.json` / -`RELEASE`); never install unversioned. Release identity: `RELEASE` file -(0.0.X until declared stable); image tags derive from it. +Harness binaries are not pinned (Jason, 2026-11-19). pi runs from +`MOSAIC_PI_BIN`, else the first `pi` on PATH, else the repository's +`node_modules`. `package.json` holds a floor range, and `package-lock.json` +records the version `npm ci` installs and the worker image carries. Nothing +refuses a pi for its version. Before a session or conversation starts, a +fit check proves the flags and behaviours Mosaic needs and refuses with a +plain line when one is missing. The bus host repeats the check on a timer +and when a binary changes. The version and a digest of the files that +ran are recorded in the claim and the run record. A change under a live +claim still refuses. Never install unversioned: a lock change is a +reviewed commit that passes the fit check and the suites. Release +identity: `RELEASE` file (0.0.X until declared stable). Image tags derive +from it and the lock's resolved pi version. -- 2.54.0