diff --git a/AGENTS.md b/AGENTS.md index e29901feb..5ed3c84ec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -222,8 +222,18 @@ Compacted, restarted, or new? Nothing that matters is lost: this file + `git log --oneline -10` + the suites reconstruct the full state. **Never guess** — verify with the suites; the run records and logs hold the receipts. -## Version pin +## Harness versions -`@earendil-works/pi-coding-agent` is pinned exactly (see `package.json` / -`RELEASE`); never install unversioned. Release identity: `RELEASE` file -(0.0.X until declared stable); image tags derive from it. +Harness binaries are not pinned (Jason, 2026-11-19). pi runs from +`MOSAIC_PI_BIN`, else the first `pi` on PATH, else the repository's +`node_modules`. `package.json` holds a floor range, and `package-lock.json` +records the version `npm ci` installs and the worker image carries. Nothing +refuses a pi for its version. Before a session or conversation starts, a +fit check proves the flags and behaviours Mosaic needs and refuses with a +plain line when one is missing. The bus host repeats the check on a timer +and when a binary changes. The version and a digest of the files that +ran are recorded in the claim and the run record. A change under a live +claim still refuses. Never install unversioned: a lock change is a +reviewed commit that passes the fit check and the suites. Release +identity: `RELEASE` file (0.0.X until declared stable). Image tags derive +from it and the lock's resolved pi version. diff --git a/Containerfile b/Containerfile index 3f79cc204..ce17a2372 100644 --- a/Containerfile +++ b/Containerfile @@ -15,8 +15,11 @@ RUN apt-get update \ # uid/gid 1000, which matches the host user that owns the runtime # state directory mounted at /var/lib/mosaic. It is reused as-is. -# Pinned Pi install: package.json pins the exact version and -# package-lock.json is installed with npm ci. No unversioned installs. +# Pi install: package.json gives pi a range, not a pin (#1873), and npm ci +# installs the version package-lock.json resolves. No unversioned installs. +# The build entry points run scripts/pi-fit-image.mjs on the host first, so a +# lock that resolves an unfit pi fails the build (the fit check needs bwrap, +# which docker build's seccomp profile refuses, so it is not a RUN step). WORKDIR /opt/app COPY package.json package-lock.json ./ RUN npm ci --ignore-scripts diff --git a/LAYERS.md b/LAYERS.md index 7cc72bca6..f0b22c3c3 100644 --- a/LAYERS.md +++ b/LAYERS.md @@ -6,8 +6,8 @@ not implemented (see BRIEF.md, "Explicit exclusions"). ## L0 — Implemented: container returns MOSAIC_HELLO_OK -One image (`mosaic-poc-agent:0.84.4`, built on digest-pinned `node:24.22.2-bookworm-slim`, non-root, -pinned Pi) runs one Pi agent one-shot. Four immutable local contract files are +One image (`mosaic-poc-agent:-r`, the pi version `package-lock.json` resolves; built on digest-pinned `node:24.22.2-bookworm-slim`, non-root, +pi from the lock, not pinned, and fit-checked before the build) runs one Pi agent one-shot. Four immutable local contract files are loaded in fixed order into the generated system prompt (`/var/lib/mosaic/private/system-prompt.md`, private to that container). One real model request is sent noninteractively; the response must equal `MOSAIC_HELLO_OK` exactly or the diff --git a/README.md b/README.md index 051cadc94..00f21222e 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,14 @@ explicit managed-name refusals with migration compatibility. ## How it works 1. `scripts/build.sh` builds the release image (`mosaic-poc-agent:-r`, - tag derived from `RELEASE` + the pinned Pi version) with Docker Compose. + tag derived from `RELEASE` + the pi version `package-lock.json` resolves, by + `scripts/image-tag.mjs`) with `docker build -f Containerfile`. `package.json` + gives pi a range, never a pin. Before the build, `scripts/pi-fit-image.mjs` + runs the fit check on the pi the lock resolves, so a lock that resolves an + unfit pi fails it. The check runs on the host's node, which can differ from + the image's (the Containerfile's `FROM`). `compose.yaml` has no build section + and `pull_policy: never`, so a compose run never builds or pulls the image; + it refuses a missing one. 2. On each run, `/opt/mosaic/src/load-contracts.sh` reads the four contract files in fixed order (CONSTITUTION, STANDARDS, SOUL, USER), joins them with clear separators, and writes `/var/lib/mosaic/private/system-prompt.md` in the diff --git a/adapters/README.md b/adapters/README.md index 6faf72d30..40771f38a 100644 --- a/adapters/README.md +++ b/adapters/README.md @@ -52,7 +52,7 @@ Optional, adapter-specific (documented per adapter): ## Included adapters -- `pi` — the pinned `@earendil-works/pi-coding-agent` CLI in noninteractive +- `pi` — the `@earendil-works/pi-coding-agent` CLI (not pinned; #1873) in noninteractive print mode (`-p`), ambient discovery disabled, stdin detached. - `mock` — deterministic echo of `MOSAIC_MOCK_RESPONSE`. Test-only: never use it where a real model response is required. @@ -121,7 +121,8 @@ doesn't exist stops the adapter with exit 2. The fit check runs the headless form of this argv, with two skill directories, one `-e` file, no tools and no session, as its last `-p` run (`workerShape`, `packages/harness/README.md`); the TUI shape the launch uses is not run. The fit check runs -before bus sessions only; worker containers run the pi pinned in the image. +before bus sessions, and on the pi `package-lock.json` resolves before an +image build (`scripts/pi-fit-image.mjs`); worker containers run that pi. ### A role's skills (row 64) diff --git a/compose.yaml b/compose.yaml index ec8021551..83e520b43 100644 --- a/compose.yaml +++ b/compose.yaml @@ -1,8 +1,10 @@ services: mosaic-agent: - build: - context: . - dockerfile: Containerfile + # #1873 S5: no build section, and never pulled. The image is built only by + # build_image (scripts/common.sh) or the gate's prepareImage, which run the + # image fit step first, so a missing image refuses here instead of compose + # building one that skipped the fit check. + pull_policy: never image: ${MOSAIC_IMAGE_TAG:?MOSAIC_IMAGE_TAG must be set by scripts/load_release (run via scripts/*.sh)} user: "1000:1000" environment: diff --git a/docs/TOOLS.md b/docs/TOOLS.md index f924deee4..436fc896f 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -16,7 +16,7 @@ and changes nothing. | Command | Purpose | Notes | |---|---|---| | `scripts/bootstrap.sh [--instance NAME]` | Create `~/.config/mosaic-dev/config.json` if absent; with `--instance NAME`, `~/.config/mosaic-NAME/config.json` and the claimed root `~/.mosaic-NAME` | Idempotent; existing config validated, never rewritten. See [Stack instances](#stack-instances---instance-name-row-190) | -| `scripts/build.sh` | Build the release image | Tag derived from `RELEASE` + pinned pi version | +| `scripts/build.sh` | Build the release image | Tag `mosaic-poc-agent:-r`, pi being the version `package-lock.json` resolves (`scripts/image-tag.mjs`); runs the image fit step (`scripts/pi-fit-image.mjs`) first. compose runs never build or pull the image, so run this before anything that needs a new tag | | `scripts/hello.sh` | One-shot startup request | Prints model response on stdout | | `scripts/verify.sh` | Full gated test | Exit 0 only on exact `MOSAIC_HELLO_OK`; `EXPECTED_MARKER` overrides for negative drills | @@ -65,8 +65,8 @@ addition to its full repository gate; it was not run by this source repair: ```bash node --test --test-concurrency=1 packages/processing-control/tests/containers.test.mjs packages/processing-control/tests/coordinator.test.mjs packages/processing-control/tests/effects.test.mjs packages/processing-control/tests/mount.test.mjs packages/processing-control/tests/runtimes.test.mjs packages/processing-control/tests/state.test.mjs packages/processing-control/tests/stop-run.test.mjs packages/processing-control/tests/task.test.mjs node --test scripts/test-darkwing-launch.mjs -scripts/test-task.sh scripts/build.sh +scripts/test-task.sh ``` ## Agent (interactive TUI) diff --git a/package-lock.json b/package-lock.json index bd8028aa8..7196afacb 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,7 @@ "version": "0.0.3", "license": "UNLICENSED", "dependencies": { - "@earendil-works/pi-coding-agent": "0.85.1", + "@earendil-works/pi-coding-agent": ">=0.85.1", "oauth4webapi": "3.8.8", "yaml": "2.9.0" } diff --git a/package.json b/package.json index ede39d66f..3e5ce0c15 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "description": "Minimal Mosaic Stack container proof of concept: one Pi agent, four local contract files, one real model request returning MOSAIC_HELLO_OK.", "license": "UNLICENSED", "dependencies": { - "@earendil-works/pi-coding-agent": "0.85.1", + "@earendil-works/pi-coding-agent": ">=0.85.1", "oauth4webapi": "3.8.8", "yaml": "2.9.0" } diff --git a/packages/cli/tests/gate-lane-image-base.test.mjs b/packages/cli/tests/gate-lane-image-base.test.mjs index 1fac7ddb8..86a5ddffd 100644 --- a/packages/cli/tests/gate-lane-image-base.test.mjs +++ b/packages/cli/tests/gate-lane-image-base.test.mjs @@ -30,9 +30,10 @@ function fixture(t) { const actual = { Id: 'sha256:' + 'a'.repeat(64), Os: 'linux', Architecture: 'amd64' }; function base(root) { return /^FROM (\S+)$/m.exec(readFileSync(join(root, 'Containerfile'), 'utf8'))[1]; } function receipt(root) { - const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf8')); + // #1873 S5: the tag carries the pi version the lock resolves, never the package.json range. + const locked = JSON.parse(readFileSync(join(root, 'package-lock.json'), 'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version; const release = readFileSync(join(root, 'RELEASE'), 'utf8').trim(); - return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' }; + return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${locked}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' }; } test('image receipt records the actual pinned FROM rather than a stale mutable tag', t => { @@ -43,14 +44,15 @@ test('image receipt records the actual pinned FROM rather than a stale mutable t const calls = []; const invoke = args => { calls.push(args); - if (args[0] === 'compose') return ''; + if (args[0] === 'build') return ''; if (args[2] === 'node:24-bookworm-slim') return JSON.stringify({ RepoDigests: ['node@sha256:' + 'b'.repeat(64)] }); return JSON.stringify(actual); }; - const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke); + // The fixture has no node_modules, so a stand-in takes the image fit step's place (#1873 S5). + const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke, () => {}); assert.equal(result.baseDigest, selected); assert.equal(calls.some(args => args[2] === 'node:24-bookworm-slim'), false); - assert.equal(calls.filter(args => args[0] === 'compose').length, 1); + assert.equal(calls.filter(args => args[0] === 'build').length, 1); }); test('a well-formed stale base receipt refuses before any image inspection', t => { diff --git a/packages/cli/tests/gate-lane-image.test.mjs b/packages/cli/tests/gate-lane-image.test.mjs index 7cff43714..8c9e39c3d 100644 --- a/packages/cli/tests/gate-lane-image.test.mjs +++ b/packages/cli/tests/gate-lane-image.test.mjs @@ -5,6 +5,12 @@ const image=await import('../../../scripts/testing/lane-image.mjs').catch(e=>{if const docker=await import('../../../scripts/testing/lane-docker.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};}); function fixtureBase(root){return /^FROM (\S+)$/m.exec(readFileSync(join(root,'Containerfile'),'utf8'))[1];} +// #1873 S5: prepareImage runs the image fit step before the build; these fixtures have no node_modules, +// so they pass a stand-in step that records that it ran. +const fitStandIn=calls=>checkout=>calls.push(['fit',checkout]); +// The tag the fixture's copied lock gives (RELEASE 0.0.1), read here from the lock, so these tests +// follow the lock when it resolves a newer pi. +const lockTagOf=root=>'mosaic-poc-agent:'+JSON.parse(readFileSync(join(root,'package-lock.json'),'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version+'-r0.0.1'; const processingInputs=['packages/processing-control/src/state.mjs','packages/processing-control/src/entrypoint.mjs']; function copyProcessingInputs(root){ mkdirSync(join(root,'packages/processing-control/src'),{recursive:true}); @@ -18,7 +24,7 @@ test('a changed COPY input or actual image ID refuses compatible image sharing', copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n'); for(const d of ['contracts','src','adapters']){mkdirSync(join(root,d));writeFileSync(join(root,d,'one'),'same');} - const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'}; + const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'}; const inspect=()=>({Id:id,Os:'linux',Architecture:'amd64'}); assert.equal(image.verifyImage(receipt,root,inspect).imageId,id); assert.throws(()=>image.verifyImage(receipt,root,()=>({...inspect(),Id:'sha256:'+'c'.repeat(64)})),/image ID/); @@ -36,9 +42,10 @@ test('exclusive image preparation records actual image ID and build wall time an for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f)); copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); - const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='compose')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; + const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='build')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});}; assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/); - const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke);assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build')); + const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='build -f Containerfile -t '+lockTagOf(root)+' .'),'a plain docker build of the Containerfile under the lock tag (compose.yaml has no build section)'); + const fitAt=calls.findIndex(x=>x[0]==='fit');assert.ok(fitAt>=0&&fitAtx[0]==='build'),'the image fit step runs, before the build'); }); test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{ const project='mg197-a-abcdef123456',id='123456789abc';let removed=false,label='foreign'; @@ -52,7 +59,7 @@ function imageFixture(t){ for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f)); copyProcessingInputs(root); writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d)); - const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'}; + const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'}; return {root,receipt}; } test('missing receipt and missing image refuse without permitting a gate',t=>{ @@ -73,12 +80,12 @@ test('a tag changed between supervised steps fails the next boundary and suppres }); test('pinned-base rebuilds with identical files have distinct image IDs; cold and warm counters stay separate',t=>{ const {root}=imageFixture(t);let builds=0,id='sha256:'+'a'.repeat(64); - const fake=args=>{if(args[0]==='compose'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});}; - const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake); + const fake=args=>{if(args[0]==='build'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});}; + const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake,fitStandIn([])); assert.equal(cold.builds,1);assert.ok(cold.buildWallMs>=0);assert.equal(builds,1); for(let i=0;i<3;i++)image.verifyImage(cold,root,()=>({Id:id,Os:'linux',Architecture:'amd64'})); assert.equal(builds,1,'warm verification builds zero images'); - id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake); + id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake,fitStandIn([])); assert.equal(rebuilt.inputDigest,cold.inputDigest);assert.equal(rebuilt.builds,1);assert.equal(builds,2); assert.throws(()=>image.verifyImage(cold,root,()=>({Id:rebuilt.imageId,Os:'linux',Architecture:'amd64'})),/image ID/); }); diff --git a/packages/cli/tests/pi-unpin-s5.test.mjs b/packages/cli/tests/pi-unpin-s5.test.mjs new file mode 100644 index 000000000..e2eccd1b3 --- /dev/null +++ b/packages/cli/tests/pi-unpin-s5.test.mjs @@ -0,0 +1,300 @@ +// #1873 pi-unpin S5: package.json gives pi a range, the image tag comes from the version +// package-lock.json resolves, an image fit step runs the fit check on the locked pi before the +// image build, and the host launcher records pi's version without comparing it to anything. +// The image-building scripts (build.sh, release.sh, verify.sh, test-task.sh, test-auth.sh) need +// Docker or an instance config, so their tag and build logic is tested here through the shell +// functions and modules they call, against a stand-in lock. +import test from "node:test"; +import assert from "node:assert/strict"; +import { chmodSync, cpSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync, existsSync } from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { tmpdir } from "node:os"; +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { sandboxProblem } from "../../seat/src/sandbox.mjs"; +import { piStandIn } from "../../harness/tests/pi-stand-in.mjs"; + +const REPO = resolve(fileURLToPath(new URL("../../../", import.meta.url))); +const missing = (e) => { if (e.code !== "ERR_MODULE_NOT_FOUND") throw e; return {}; }; +const tagMod = await import("../../../scripts/image-tag.mjs").catch(missing); +const fitMod = await import("../../../scripts/pi-fit-image.mjs").catch(missing); +const image = await import("../../../scripts/testing/lane-image.mjs").catch(missing); +const SKIP = sandboxProblem() ? { skip: "no working bwrap here" } : {}; +const NAME = "@earendil-works/pi-coding-agent"; +const KEY = `node_modules/${NAME}`; +const RANGE = ">=0.85.1"; +const NODE_BIN = dirname(process.execPath); +const PATH = `${NODE_BIN}:/usr/bin:/bin`; + +const entry = (version) => ({ version, resolved: `https://registry.npmjs.org/${NAME}/-/pi-coding-agent-${version}.tgz`, integrity: `sha512-fixture${version}` }); +const lockOf = (version) => ({ name: "fixture", lockfileVersion: 3, requires: true, packages: { "": { name: "fixture", dependencies: { [NAME]: RANGE } }, ...(version ? { [KEY]: entry(version) } : {}) } }); + +// A checkout with package.json's range, a lock that resolves `version`, and RELEASE. +function checkout(t, { version = "9.9.9", release = "0.0.13" } = {}) { + const root = mkdtempSync(join(tmpdir(), "s5-")); + t.after(() => rmSync(root, { recursive: true, force: true })); + writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } })); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version))); + writeFileSync(join(root, "RELEASE"), `${release}\n`); + return root; +} + +// npm ci's effect in a checkout: the package, its .bin link and node_modules/.package-lock.json. +function install(root, knobs = {}, { version = "9.9.9", hidden = entry(version) } = {}) { + const pkg = join(root, KEY); + mkdirSync(join(pkg, "dist", "bundle"), { recursive: true }); + writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: NAME, version, bin: { pi: "dist/bundle/cli.js" } })); + writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn({ version, ...knobs }), { mode: 0o755 }); + mkdirSync(join(root, "node_modules", ".bin"), { recursive: true }); + symlinkSync(`../${NAME}/dist/bundle/cli.js`, join(root, "node_modules", ".bin", "pi")); + writeFileSync(join(root, "node_modules", ".package-lock.json"), JSON.stringify({ name: "fixture", lockfileVersion: 3, packages: hidden ? { [KEY]: hidden } : {} })); + return pkg; +} + +const noFit = () => { throw new Error("the fit check must not run when the locked pi can't be told"); }; + +test("pi-unpin S5: a stand-in lock at 9.9.9 gives mosaic-poc-agent:9.9.9-r, and the package.json range never appears in the tag", t => { + assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag"); + const root = checkout(t); + const tag = tagMod.imageTag(root); + assert.equal(tag, "mosaic-poc-agent:9.9.9-r0.0.13"); + assert.ok(!tag.includes(RANGE) && !tag.includes("0.85.1"), "the tag carries the lock's version, not the range"); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), root], { encoding: "utf8", env: { PATH } }); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(cli.stdout, "mosaic-poc-agent:9.9.9-r0.0.13\n"); + // The repository itself: its lock resolves 0.85.1 under the range. + assert.equal(JSON.parse(readFileSync(join(REPO, "package.json"), "utf8")).dependencies[NAME], RANGE); + assert.equal(tagMod.imageTag(REPO), `mosaic-poc-agent:${JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version}-r${readFileSync(join(REPO, "RELEASE"), "utf8").trim()}`); +}); + +test("pi-unpin S5: a lock with no pi entry, a range for a version, or a bad RELEASE refuses the tag, with no path in the line", t => { + assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag"); + const none = checkout(t, { version: null }); + assert.throws(() => tagMod.imageTag(none), { message: `image tag: package-lock.json has no entry for ${NAME}` }); + const ranged = checkout(t); + writeFileSync(join(ranged, "package-lock.json"), JSON.stringify({ lockfileVersion: 3, packages: { [KEY]: { ...entry("9.9.9"), version: RANGE } } })); + assert.throws(() => tagMod.imageTag(ranged), { message: `image tag: package-lock.json's ${NAME} version isn't a plain version` }); + const gone = checkout(t); + rmSync(join(gone, "package-lock.json")); + assert.throws(() => tagMod.imageTag(gone), { message: "image tag: package-lock.json can't be read" }); + assert.throws(() => tagMod.imageTag(checkout(t, { release: "next" })), { message: "image tag: RELEASE must be a version like 0.0.13" }); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), none], { encoding: "utf8", env: { PATH } }); + assert.equal(cli.status, 1); + assert.equal(cli.stdout, ""); + assert.ok(!cli.stderr.includes(none), "a refusal line carries no host path"); +}); + +// scripts/common.sh as the image-building scripts source it, in a stand-in checkout. +function shellCheckout(t, opts) { + const root = checkout(t, opts); + mkdirSync(join(root, "scripts")); + for (const f of ["common.sh", "image-tag.mjs"]) cpSync(join(REPO, "scripts", f), join(root, "scripts", f)); + return root; +} +const sh = (root, script, env = {}) => spawnSync("bash", ["-c", `source scripts/common.sh && ${script}`], { cwd: root, encoding: "utf8", env: { PATH, ...env } }); + +test("pi-unpin S5: load_release (build.sh, release.sh, verify.sh, test-release.sh) exports the lock's tag and refuses a lock it can't read", t => { + const root = shellCheckout(t); + const r = sh(root, 'load_release && printf "%s|%s" "$MOSAIC_RELEASE" "$MOSAIC_IMAGE_TAG"'); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stdout, "0.0.13|mosaic-poc-agent:9.9.9-r0.0.13"); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); + const bad = sh(root, 'load_release && printf "reached"'); + assert.notEqual(bad.status, 0); + assert.equal(bad.stdout, "", "a lock with no pi entry stops the script"); + assert.match(bad.stderr, /image tag: package-lock\.json has no entry/); +}); + +test("pi-unpin S5: build_image runs the image fit step first, and an unfit pi never reaches the image build", t => { + const root = shellCheckout(t); + mkdirSync(join(root, "bin")); + const log = join(root, "docker.log"); + writeFileSync(join(root, "bin", "docker"), `#!/bin/sh\nprintf '%s\\n' "$*" >> "${log}"\n`, { mode: 0o755 }); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `import { appendFileSync } from "node:fs";\nif (process.env.FIT === "unfit") { process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1); }\nappendFileSync(${JSON.stringify(log)}, "fit\\n");\n`); + const env = { PATH: `${join(root, "bin")}:${PATH}` }; + const unfit = sh(root, "load_release && build_image; echo after", { ...env, FIT: "unfit" }); + assert.notEqual(unfit.status, 0); + assert.equal(unfit.stdout, "", "the calling script stops at the fit step"); + assert.match(unfit.stderr, /lacks --offline/); + assert.equal(existsSync(log), false, "docker is never called for an unfit pi"); + const fit = sh(root, "load_release && build_image", env); + assert.equal(fit.status, 0, fit.stderr); + assert.equal(readFileSync(log, "utf8"), "fit\nbuild -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.13 .\n"); + // Without load_release there is no tag to build under. + const untagged = spawnSync("bash", ["-c", "source scripts/common.sh && build_image"], { cwd: root, encoding: "utf8", env }); + assert.notEqual(untagged.status, 0); + assert.match(untagged.stderr, /build_image needs MOSAIC_IMAGE_TAG from load_release/); +}); + +test("pi-unpin S5: no script builds the image tag from package.json, and every image build goes through build_image", () => { + const read = (f) => readFileSync(join(REPO, f), "utf8"); + const scripts = ["scripts/common.sh", "scripts/build.sh", "scripts/release.sh", "scripts/verify.sh", "scripts/test-task.sh", "scripts/test-auth.sh", "scripts/agent-host-dev.sh", "scripts/mosaic-task.mjs", "scripts/testing/lane-image.mjs"]; + for (const f of scripts) { + assert.doesNotMatch(read(f), /dependencies\s*(\?\.)?\[\s*['"]@earendil-works\/pi-coding-agent['"]\s*\]/, `${f} reads pi's version from package.json`); + } + for (const f of ["scripts/build.sh", "scripts/release.sh", "scripts/verify.sh"]) { + assert.match(read(f), /^\s*build_image\b/m, `${f} builds through build_image`); + assert.doesNotMatch(read(f), /docker (compose )?build/, `${f} builds without the image fit step`); + } + // compose.yaml can't build or pull the agent image, so a missing image refuses at compose run + // instead of being built without the fit step. + const compose = read("compose.yaml"); + assert.doesNotMatch(compose, /^\s*build:/m, "compose.yaml has no build section"); + assert.match(compose, /^\s*pull_policy: never$/m); + // The tag comes from scripts/image-tag.mjs, and a refusal stops the script instead of running on an empty tag. + for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /^\s*TAG="\$\(node scripts\/image-tag\.mjs\)" \|\| exit 1(\s|$)/m, `${f} takes the tag from scripts/image-tag.mjs and stops when it can't`); + assert.match(read("scripts/mosaic-task.mjs"), /MOSAIC_IMAGE_TAG = tag;/); +}); + +// The gate's image inputs, with a lock at `version` and package.json's range. +function imageCheckout(t, version = "9.9.9") { + const root = mkdtempSync("/mnt/storage/scratch/s5i-"); + t.after(() => rmSync(root, { recursive: true, force: true })); + cpSync(join(REPO, "Containerfile"), join(root, "Containerfile")); + writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } })); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version))); + writeFileSync(join(root, "RELEASE"), "0.0.1\n"); + mkdirSync(join(root, "packages/processing-control/src"), { recursive: true }); + for (const f of ["state.mjs", "entrypoint.mjs"]) cpSync(join(REPO, "packages/processing-control/src", f), join(root, "packages/processing-control/src", f)); + for (const d of ["contracts", "src", "adapters"]) { mkdirSync(join(root, d)); writeFileSync(join(root, d, "one"), "same"); } + return root; +} +const base = (root) => /^FROM (\S+)$/m.exec(readFileSync(join(root, "Containerfile"), "utf8"))[1]; + +test("pi-unpin S5: verifyImage accepts the lock-derived tag when package.json holds a range, and refuses a tag the lock doesn't give", t => { + assert.equal(typeof image.verifyImage, "function"); + const root = imageCheckout(t); + const id = "sha256:" + "a".repeat(64); + const receipt = { inputDigest: image.imageFingerprint(root), imageId: id, tag: "mosaic-poc-agent:9.9.9-r0.0.1", baseDigest: base(root), platform: "linux/amd64" }; + const inspect = () => ({ Id: id, Os: "linux", Architecture: "amd64" }); + assert.equal(image.verifyImage(receipt, root, inspect).tag, "mosaic-poc-agent:9.9.9-r0.0.1"); + assert.throws(() => image.verifyImage({ ...receipt, tag: "mosaic-poc-agent:0.85.1-r0.0.1" }, root, inspect), /image tag differs from release identity/); + assert.throws(() => image.verifyImage({ ...receipt, tag: `mosaic-poc-agent:${RANGE}-r0.0.1` }, root, inspect), /refused/); + writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); + assert.throws(() => image.verifyImage({ ...receipt, inputDigest: image.imageFingerprint(root) }, root, inspect), /image tag: package-lock\.json has no entry/); +}); + +test("pi-unpin S5: prepareImage tags from the lock, runs the image fit step before the build, and a refusing step stops the build", t => { + assert.equal(typeof image.prepareImage, "function"); + assert.equal(typeof image.imageFitStep, "function", "lane-image.mjs exports the image fit step"); + const root = imageCheckout(t); + const calls = []; + const invoke = (args) => { calls.push(args.join(" ")); return args[0] === "build" ? "" : JSON.stringify({ Id: "sha256:" + "a".repeat(64), Os: "linux", Architecture: "amd64" }); }; + const receipt = image.prepareImage(root, { grant: "fixture", tempRoot: root }, invoke, (c) => calls.push(`fit ${c === root}`)); + assert.equal(receipt.tag, "mosaic-poc-agent:9.9.9-r0.0.1"); + assert.deepEqual(calls, ["fit true", "build -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.1 .", "image inspect mosaic-poc-agent:9.9.9-r0.0.1 --format {{json .}}"]); + const stopped = []; + assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a), () => { throw new Error("unfit"); }), /unfit/); + assert.deepEqual(stopped, [], "no build after a refusing fit step"); + // The default step runs the checkout's scripts/pi-fit-image.mjs and refuses with its last line. + mkdirSync(join(root, "scripts")); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1);\n`); + assert.throws(() => image.imageFitStep(root), { message: "gate-lane: refused: image fit step refused: pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline" }); + assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a)), /image fit step refused/); + assert.deepEqual(stopped, []); + writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), "process.exit(0);\n"); + assert.equal(image.imageFitStep(root), undefined); +}); + +test("pi-unpin S5: the image fit step fails on a locked stand-in pi that lacks a flag, and passes one at another version that has them all", SKIP, async t => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const unfit = checkout(t); + install(unfit, { dropFlags: ["--offline"] }); + const r = await fitMod.imageFit(unfit); + assert.deepEqual(r, { ok: false, message: "pi 9.9.9 (the lock's pi) lacks --offline" }); + const ok = checkout(t); + install(ok); + const pass = await fitMod.imageFit(ok); + assert.equal(pass.ok, true, pass.message); + assert.equal(pass.version, "9.9.9"); + assert.equal(pass.locked, "9.9.9"); +}); + +test("pi-unpin S5: the image fit step passes on the installed pi the repository's lock resolves", SKIP, async () => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const locked = JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version; + const r = await fitMod.imageFit(REPO); + assert.equal(r.ok, true, r.message); + assert.equal(r.locked, locked); + assert.equal(r.version, locked); + const cli = spawnSync(process.execPath, [join(REPO, "scripts/pi-fit-image.mjs")], { encoding: "utf8", env: { PATH, HOME: process.env.HOME, TMPDIR: tmpdir() }, timeout: 120000 }); + assert.equal(cli.status, 0, cli.stderr); + assert.equal(cli.stdout, `pi-fit-image: pi ${locked} (the lock's pi) passed the fit check\n`); +}); + +test("pi-unpin S5: the image fit step can't tell, and refuses, when node_modules doesn't hold the lock's pi", async t => { + assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit"); + const lead = "pi (the lock's pi) can't be checked: "; + const npmCi = "run npm ci --ignore-scripts in the repository first"; + const cases = [ + ["no install", () => {}, `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["no hidden lock entry", (root) => install(root, {}, { hidden: null }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["another integrity", (root) => install(root, {}, { hidden: { ...entry("9.9.9"), integrity: "sha512-other" } }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`], + ["another version installed", (root) => install(root, {}, { version: "9.9.8", hidden: entry("9.9.9") }), `the installed pi's package.json doesn't carry the lock's version: ${npmCi}`], + ["a lock with no pi", (root) => { install(root); writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); }, `image tag: package-lock.json has no entry for ${NAME}`], + ["a .bin/pi that isn't executable", (root) => { install(root); chmodSync(join(root, KEY, "dist", "bundle", "cli.js"), 0o644); }, `the lookup refused the lock's pi: ${npmCi}`], + ["a .bin/pi in another copy of the package", (root) => { + install(root); + const other = join(root, "other", NAME); + mkdirSync(join(other, "dist", "bundle"), { recursive: true }); + writeFileSync(join(other, "package.json"), JSON.stringify({ name: NAME, version: "9.9.9", bin: { pi: "dist/bundle/cli.js" } })); + writeFileSync(join(other, "dist", "bundle", "cli.js"), piStandIn({ version: "9.9.9" }), { mode: 0o755 }); + rmSync(join(root, "node_modules", ".bin", "pi")); + symlinkSync(join(other, "dist", "bundle", "cli.js"), join(root, "node_modules", ".bin", "pi")); + }, `node_modules/.bin/pi isn't the lock's pi package: ${npmCi}`], + ]; + for (const [name, plant, problem] of cases) { + const root = checkout(t); + plant(root); + const r = await fitMod.imageFit(root, { fit: noFit }); + assert.deepEqual(r, { ok: false, message: lead + problem }, name); + assert.ok(!r.message.includes(root) && !r.message.includes(realpathSync(root)), `${name}: no host path in the line`); + } + // An answer the fit check doesn't give reads as can't tell. + const root = checkout(t); + install(root); + assert.deepEqual(await fitMod.imageFit(root, { fit: async () => undefined }), { ok: false, message: `${lead}the check gave no answer` }); + // The fit check gets the found pi under the step's own label, with the module's thinking levels. + let seen; + const ok = await fitMod.imageFit(root, { fit: async (pi, opts) => { seen = { pi, opts }; return { ok: true, version: "9.9.9" }; } }); + assert.equal(ok.ok, true); + assert.equal(ok.locked, "9.9.9"); + assert.equal(seen.pi.label, "the lock's pi"); + assert.equal(seen.pi.pkg, realpathSync(join(root, KEY))); + assert.equal("locked" in seen.pi, false); + assert.ok(Array.isArray(seen.opts.levels) && seen.opts.levels.includes("high")); +}); + +// scripts/agent-host-dev.sh --check in a checkout with the range in package.json and a stand-in pi. +function hostCheckout(t, piScript) { + const root = mkdtempSync(join(tmpdir(), "s5h-")); + t.after(() => rmSync(root, { recursive: true, force: true })); + for (const d of ["scripts", "agents/fixture", "contracts", "user", "node_modules/.bin", "extensions/goal"]) mkdirSync(join(root, d), { recursive: true }); + cpSync(join(REPO, "scripts/agent-host-dev.sh"), join(root, "scripts/agent-host-dev.sh")); + writeFileSync(join(root, "scripts/common.sh"), 'load_config() { MOSAIC_DATA_ROOT="$PWD"; MOSAIC_PROVIDER=test; MOSAIC_MODEL=test-model; MOSAIC_ADAPTER=pi; }\n'); + writeFileSync(join(root, "package.json"), JSON.stringify({ dependencies: { [NAME]: RANGE } })); + for (const f of ["agents/fixture/SOUL.md", "agents/fixture/CONTEXT.md", "contracts/CONSTITUTION.md", "contracts/STANDARDS.md", "user/USER.md", "AGENTS.md", "extensions/goal/index.ts"]) writeFileSync(join(root, f), `fixture ${f}\n`); + for (const s of ["ms-tools", "ms-file-read", "ms-file-write", "ms-conductor", "ms-communications", "ms-watch", "ms-sdlc", "ms-proactive-agent", "ms-goal", "ms-unslop"]) { + mkdirSync(join(root, "skills", s), { recursive: true }); + writeFileSync(join(root, "skills", s, "SKILL.md"), `fixture ${s}\n`); + } + writeFileSync(join(root, "node_modules/.bin/pi"), piScript, { mode: 0o755 }); + return root; +} +const hostCheck = (root) => spawnSync("bash", [join(root, "scripts/agent-host-dev.sh"), "fixture", "--check"], { cwd: root, encoding: "utf8", env: { PATH, HOME: root }, timeout: 20000 }); + +test("pi-unpin S5: scripts/agent-host-dev.sh doesn't refuse pi for its version; it records the version pi prints", t => { + const root = hostCheckout(t, "#!/bin/sh\n[ \"$1\" = --version ] && { echo 9.9.9; exit 0; }\nexit 1\n"); + const r = hostCheck(root); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /fixture: native host TUI \| Pi 9\.9\.9 \| test\/test-model/); + assert.match(r.stdout, /configuration checks passed/); + // A pi whose --version fails or prints nothing still starts; the banner says so. + for (const script of ["#!/bin/sh\nexit 3\n", "#!/bin/sh\nexit 0\n"]) { + const quiet = hostCheckout(t, script); + const q = hostCheck(quiet); + assert.equal(q.status, 0, q.stdout + q.stderr); + assert.match(q.stdout, /\| Pi version unknown \|/); + } + assert.doesNotMatch(readFileSync(join(REPO, "scripts/agent-host-dev.sh"), "utf8"), /package\.json pins|\$PIN\b/); +}); diff --git a/packages/mosaic/README.md b/packages/mosaic/README.md index 70a194d45..2a9dfac2f 100644 --- a/packages/mosaic/README.md +++ b/packages/mosaic/README.md @@ -36,4 +36,4 @@ Tests can supply `refresh: { mode, timeoutMs }`. Modes are rotate, unchanged, fa Limits: store locks are in-process, not a production distributed lock. Credential-store commit and filesystem publication are not crash-atomic together. A post-publication failure records uncertainty, retains the generation and rolls back the in-memory draft; this simulator has no external token issuer to reconcile. A real refresh backend would need a separate reconciliation protocol before reuse. Filesystem ownership does not defend against arbitrary same-UID tampering or establish hardlink provenance. -Exact Pi 0.85.1 isolation is static-source evidence only. Its model-catalog network flag does NOT suppress OAuth refresh networking. Never substitute real Pi in these tests, even with expired synthetic OAuth tokens. Final whole-increment review is required before publication. +The Pi isolation described here was read from the 0.85.1 source (pi is not pinned; #1873) and is static-source evidence only. Its model-catalog network flag does NOT suppress OAuth refresh networking. Never substitute real Pi in these tests, even with expired synthetic OAuth tokens. Final whole-increment review is required before publication. diff --git a/scripts/agent-host-dev.sh b/scripts/agent-host-dev.sh index 5ea108150..20df43ea1 100755 --- a/scripts/agent-host-dev.sh +++ b/scripts/agent-host-dev.sh @@ -66,9 +66,9 @@ fi USER_FILE="${USER_FILE:-$MOSAIC_DATA_ROOT/user/USER.md}" PI="$REPO/node_modules/.bin/pi" [ -x "$PI" ] || fail 'run npm ci --ignore-scripts --no-audit --no-fund in the repository first' -PIN="$(node -p "require('./package.json').dependencies['@earendil-works/pi-coding-agent']")" -PI_VERSION="$("$PI" --version)" -[ "$PI_VERSION" = "$PIN" ] || fail "local Pi is $PI_VERSION, package.json pins $PIN; run npm ci in $REPO" +# #1873 S5: no pin. The version pi prints is recorded for the banner, never compared. +PI_VERSION="$("$PI" --version 2>/dev/null | head -n 1)" || true +[ -n "$PI_VERSION" ] || PI_VERSION="version unknown" command -v flock >/dev/null || fail 'flock is required' CONTEXT_FILES=("$CONSTITUTION" "$REPO/contracts/STANDARDS.md" "$SOUL" "$USER_FILE" "$REPO/AGENTS.md" "$REPO/agents/$AGENT/CONTEXT.md") for file in "${CONTEXT_FILES[@]}"; do @@ -83,7 +83,7 @@ for skill in "${SKILLS[@]}"; do done [ -s "$REPO/extensions/goal/index.ts" ] || fail 'missing canonical goal extension' STATE="$REPO/.pi/state/$AGENT" -echo "$AGENT: native host TUI | Pi $PIN | $PROVIDER/$MODEL${THINKING:+:$THINKING}" +echo "$AGENT: native host TUI | Pi $PI_VERSION | $PROVIDER/$MODEL${THINKING:+:$THINKING}" echo "$AGENT: workspace $REPO" echo "$AGENT: SOUL=$SOUL | CONSTITUTION=$CONSTITUTION | USER=$USER_FILE" echo "$AGENT: sessions $STATE/sessions | skills: ${SKILLS[*]}" diff --git a/scripts/build.sh b/scripts/build.sh index 275989c56..c274d25e0 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# Build the mosaic-agent container image using Docker Compose. +# Build the mosaic-agent container image through build_image in common.sh: the image fit step, +# then the build. compose.yaml has no build section, so this is how the image gets built. set -euo pipefail cd "$(dirname "$0")/.." # shellcheck source=common.sh @@ -10,4 +11,4 @@ load_release bootstrap_runtime_dir -docker compose build +build_image diff --git a/scripts/common.sh b/scripts/common.sh index 87797b250..f92736ed1 100755 --- a/scripts/common.sh +++ b/scripts/common.sh @@ -26,17 +26,35 @@ load_config() { # Resolve the release identity: RELEASE is the single source of the # release version (stays 0.0.X until declared stable); the image tag -# derives from it plus the pinned pi dependency version. +# derives from it plus the pi version package-lock.json resolves (#1873 S5: +# package.json gives pi a range, never a pin, and the tag never carries it). +# scripts/image-tag.mjs builds the tag and refuses a lock it can't read. load_release() { - local release pi_version + local release tag release="$(tr -d '[:space:]' < RELEASE)" if ! printf '%s' "$release" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$'; then echo "common: RELEASE must be a semver-ish version, got: '$release'" >&2 exit 1 fi - pi_version="$(node -p "require('./package.json').dependencies['@earendil-works/pi-coding-agent']")" + tag="$(node scripts/image-tag.mjs .)" || exit 1 export MOSAIC_RELEASE="$release" - export MOSAIC_IMAGE_TAG="mosaic-poc-agent:${pi_version}-r${release}" + export MOSAIC_IMAGE_TAG="$tag" +} + +# Build the agent image (#1873 S5). The image fit step runs first: it runs the +# fit check on the pi package-lock.json resolves, as npm ci installed it here, +# and refuses with one line when that pi is unfit or the check can't tell, so a +# lock that resolves an unfit pi fails the build. scripts/pi-fit-image.mjs +# says why the step runs on the host and not inside the image build. +# compose.yaml has no build section, so this is the only way the scripts build +# the image; compose run refuses a missing one. Call load_release first. +build_image() { + if [ -z "${MOSAIC_IMAGE_TAG:-}" ]; then + echo "common: build_image needs MOSAIC_IMAGE_TAG from load_release" >&2 + exit 1 + fi + node scripts/pi-fit-image.mjs || exit 1 + docker build -f Containerfile -t "$MOSAIC_IMAGE_TAG" . } # Ensure the configured runtime data directory exists and carries this diff --git a/scripts/image-tag.mjs b/scripts/image-tag.mjs new file mode 100755 index 000000000..7e3d4e685 --- /dev/null +++ b/scripts/image-tag.mjs @@ -0,0 +1,54 @@ +#!/usr/bin/env node +// #1873 pi-unpin S5: the agent image tag. package.json gives pi a range (no pin); the version the +// image installs is the one package-lock.json resolves, since the Containerfile runs npm ci. So the +// tag is built from the lock's entry for pi and RELEASE, never from the package.json string: +// mosaic-poc-agent:-r. A lock or RELEASE this can't read refuses. +// +// node scripts/image-tag.mjs [checkout] prints the tag (checkout: default this repository) +import { readFileSync, realpathSync } from "node:fs"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +export const PI_PACKAGE = "@earendil-works/pi-coding-agent"; +export const PI_LOCK_KEY = `node_modules/${PI_PACKAGE}`; +const VERSION = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; +const RELEASE = /^\d+\.\d+\.\d+$/; + +const fail = (message) => { throw new Error(`image tag: ${message}`); }; +const readJson = (path, name) => { + let text; + try { text = readFileSync(path, "utf8"); } catch { fail(`${name} can't be read`); } + try { return JSON.parse(text); } catch { fail(`${name} isn't valid JSON`); } +}; + +// The lock's entry for pi: { version, resolved, integrity }. Refuses a lock with no entry, or one +// whose version isn't a plain semver. +export function lockedPi(root) { + const lock = readJson(join(root, "package-lock.json"), "package-lock.json"); + const entry = lock?.packages?.[PI_LOCK_KEY]; + if (!entry || typeof entry !== "object") fail(`package-lock.json has no entry for ${PI_PACKAGE}`); + if (typeof entry.version !== "string" || !VERSION.test(entry.version)) fail(`package-lock.json's ${PI_PACKAGE} version isn't a plain version`); + return { version: entry.version, resolved: entry.resolved ?? null, integrity: entry.integrity ?? null }; +} + +export function release(root) { + let text; + try { text = readFileSync(join(root, "RELEASE"), "utf8"); } catch { fail("RELEASE can't be read"); } + const value = text.replace(/\s/g, ""); + if (!RELEASE.test(value)) fail("RELEASE must be a version like 0.0.13"); + return value; +} + +export const imageTag = (root) => `mosaic-poc-agent:${lockedPi(root).version}-r${release(root)}`; + +const self = fileURLToPath(import.meta.url); +let main = false; +try { main = realpathSync(process.argv[1] ?? "") === realpathSync(self); } catch {} +if (main) { + try { + process.stdout.write(`${imageTag(process.argv[2] ?? join(self, "..", ".."))}\n`); + } catch (e) { + process.stderr.write(`${e.message}\n`); + process.exit(1); + } +} diff --git a/scripts/mosaic-task.mjs b/scripts/mosaic-task.mjs index 624221c0c..725bdf942 100755 --- a/scripts/mosaic-task.mjs +++ b/scripts/mosaic-task.mjs @@ -35,6 +35,7 @@ import process from "node:process"; import { randomBytes } from "node:crypto"; import { spawnSync } from "node:child_process"; import { fileURLToPath } from "node:url"; +import { imageTag } from "./image-tag.mjs"; import { BusinessError, validateRoleDocument } from "../packages/organization/src/index.mjs"; import { RunsError, isRunId, listRunRecords, readRunRecord } from "../packages/runs/src/index.mjs"; import { initializeControl, assertProcessing } from "../packages/processing-control/src/state.mjs"; @@ -339,6 +340,10 @@ function runTask(taskFile, options = {}) { catch (e) { fail(3, e.code?.startsWith('processing-') ? e.code : 'processing-state-invalid'); } const task = validateTask(readJsonFile(taskFile, "task file"), path.resolve(taskFile)); + // #1873 S5: the image tag carries the pi version package-lock.json resolves, never the + // package.json range. Read before the run directory exists, so a lock it can't read leaves none. + let tag; + try { tag = imageTag(PROJECT_ROOT); } catch (e) { fail(4, e.message); } const runId = `r-${utcStamp()}-${randomBytes(3).toString("hex")}`; const runDir = path.join(runsRoot(resolved), runId); @@ -367,9 +372,7 @@ function runTask(taskFile, options = {}) { // (Names here are the compose interpolation consumers, not PI_*.) spawnEnv.MOSAIC_PROVIDER = resolved.execution.provider; spawnEnv.MOSAIC_MODEL = resolved.execution.model; - const release = fs.readFileSync(path.join(PROJECT_ROOT, "RELEASE"), "utf8").trim(); - const piVersion = JSON.parse(fs.readFileSync(path.join(PROJECT_ROOT, "package.json"), "utf8")).dependencies["@earendil-works/pi-coding-agent"]; - spawnEnv.MOSAIC_IMAGE_TAG = `mosaic-poc-agent:${piVersion}-r${release}`; + spawnEnv.MOSAIC_IMAGE_TAG = tag; // Workspace (M5): create only the selected host source. The container // launcher gives it the fixed /workspace target and mounts no parent. diff --git a/scripts/pi-fit-image.mjs b/scripts/pi-fit-image.mjs new file mode 100755 index 000000000..21a1078bc --- /dev/null +++ b/scripts/pi-fit-image.mjs @@ -0,0 +1,72 @@ +#!/usr/bin/env node +// #1873 pi-unpin S5: the image fit step. The agent image installs the pi package-lock.json resolves +// (npm ci), and nothing pins its version. This step runs the fit check (packages/harness/src/ +// pi-fit.mjs, as sessions run it) on that pi before the image is built, so a lock that resolves an +// unfit pi fails the build. scripts/build.sh, scripts/release.sh package, scripts/verify.sh and the +// gate's image preparation run it first. +// +// It runs on the host, not as a Containerfile RUN step: the fit check runs pi inside bwrap, and +// docker build's default seccomp profile refuses the namespaces bwrap needs. The host copy stands for +// the image's: npm ci installs both from the same lock entry, and this step refuses unless the +// checkout's node_modules holds exactly that entry (version, resolved and integrity, as npm recorded +// them in node_modules/.package-lock.json, and the installed package.json's version). +// It runs pi on the host's node, which can be a different version from the image's (the +// Containerfile's FROM). Running it on the image's node is left for later. +// +// node scripts/pi-fit-image.mjs exit 0: passed; exit 1: one line on stderr, unfit or can't tell +import { readFileSync, realpathSync } from "node:fs"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { lockedPi, PI_LOCK_KEY } from "./image-tag.mjs"; +import { locate } from "../packages/plugins/harness/pi/binary.mjs"; +import { levels } from "../packages/plugins/harness/pi/index.mjs"; +import { piFit } from "../packages/harness/src/pi-fit.mjs"; + +// The name refusal lines use for this pi. Never a path. +export const LABEL = "the lock's pi"; +const RUN_NPM_CI = "run npm ci --ignore-scripts in the repository first"; + +const readJson = (path) => { try { return JSON.parse(readFileSync(path, "utf8")); } catch { return null; } }; +const real = (path) => { try { return realpathSync(path); } catch { return null; } }; + +// The lookup's answer for the pi npm installed from the lock in root, relabelled, or +// { bin: null, problem }. lookup: binary.mjs's locate (tests pass a stand-in). +export function lockedFound(root, { lookup = locate } = {}) { + let locked; + try { locked = lockedPi(root); } catch (e) { return { bin: null, problem: e.message }; } + const installed = readJson(join(root, "node_modules", ".package-lock.json"))?.packages?.[PI_LOCK_KEY]; + if (!installed || installed.version !== locked.version || installed.resolved !== locked.resolved || installed.integrity !== locked.integrity) { + return { bin: null, problem: `the installed pi isn't the one package-lock.json resolves: ${RUN_NPM_CI}` }; + } + const dir = join(root, PI_LOCK_KEY); + if (readJson(join(dir, "package.json"))?.version !== locked.version) { + return { bin: null, problem: `the installed pi's package.json doesn't carry the lock's version: ${RUN_NPM_CI}` }; + } + const found = lookup({ MOSAIC_PI_BIN: join(root, "node_modules", ".bin", "pi") }); + if (!found?.bin) return { bin: null, problem: `the lookup refused the lock's pi: ${RUN_NPM_CI}` }; + if (found.pkg !== real(dir)) return { bin: null, problem: `node_modules/.bin/pi isn't the lock's pi package: ${RUN_NPM_CI}` }; + return { ...found, label: LABEL, locked: locked.version }; +} + +// The fit check's answer for root's locked pi: { ok: true, version, ... } or { ok: false, message }. +// fit: pi-fit.mjs's piFit (tests may pass a stand-in). +export async function imageFit(root, { lookup = locate, fit = piFit } = {}) { + const found = lockedFound(root, { lookup }); + if (!found.bin) return { ok: false, message: `pi (${LABEL}) can't be checked: ${found.problem}` }; + const { locked, ...pi } = found; + const result = await fit(pi, { levels: levels() }); + return result?.ok === true ? { ...result, locked } : { ok: false, message: typeof result?.message === "string" ? result.message : `pi (${LABEL}) can't be checked: the check gave no answer` }; +} + +const self = fileURLToPath(import.meta.url); +let main = false; +try { main = realpathSync(process.argv[1] ?? "") === realpathSync(self); } catch {} +if (main) { + const result = await imageFit(join(self, "..", "..")); + if (result.ok) { + process.stdout.write(`pi-fit-image: pi ${result.locked} (${LABEL}) passed the fit check\n`); + } else { + process.stderr.write(`pi-fit-image: ${result.message.split("\n")[0]}\n`); + process.exit(1); + } +} diff --git a/scripts/release.sh b/scripts/release.sh index 5c05375cf..c321a3312 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -119,7 +119,7 @@ console.log("active: " + p.release + " (" + p.imageTag + ") since " + p.activ } cmd_package() { - docker compose build + build_image append_log "package" "$MOSAIC_RELEASE" "$MOSAIC_IMAGE_TAG" echo "release: packaged $MOSAIC_IMAGE_TAG" } diff --git a/scripts/test-auth.sh b/scripts/test-auth.sh index e9580f8c5..48c2c2ea0 100755 --- a/scripts/test-auth.sh +++ b/scripts/test-auth.sh @@ -36,7 +36,8 @@ CFG="MOSAIC_CONFIG=$SANDBOX/mock-config.json" # release pointer so agent.sh's ensure stays on the no-op fast path REL="$(tr -d '[:space:]' < RELEASE)" -printf '{"pointerVersion":1,"release":"%s","imageTag":"mosaic-poc-agent:0.85.1-r%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$REL" > "$SANDBOX/data/state/active.json" +TAG="$(node scripts/image-tag.mjs)" || exit 1 # #1873 S5: the tag the lock gives, never a pinned string +printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" # fixtures: a default-harness credential and two mosaic-managed accounts printf '{"anthropic":{"type":"oauth","access":"%s"},"zai":{"type":"api_key","key":"%s"}}' "$TOKEN" "$SECRET" > "$SANDBOX/pi/auth.json" diff --git a/scripts/test-task.sh b/scripts/test-task.sh index bbc9942fb..06fb5b401 100755 --- a/scripts/test-task.sh +++ b/scripts/test-task.sh @@ -266,7 +266,8 @@ EOF EOF REL="$(tr -d '[:space:]' < RELEASE)" mkdir -p "$SANDBOX/data/state" - printf '{"pointerVersion":1,"release":"%s","imageTag":"mosaic-poc-agent:0.85.1-r%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$REL" > "$SANDBOX/data/state/active.json" + TAG="$(node scripts/image-tag.mjs)" || exit 1 # #1873 S5: the tag the lock gives, never a pinned string + printf '{"pointerVersion":1,"release":"%s","imageTag":"%s","activatedAt":"2026-01-01T00:00:00Z"}\n' "$REL" "$TAG" > "$SANDBOX/data/state/active.json" scripts/skill.sh install ms-tools >/dev/null 2>&1 scripts/skill.sh activate ms-tools >/dev/null 2>&1 env MOSAIC_CONFIG="$SANDBOX/mock-config.json" MOSAIC_MOCK_RESPONSE=MOCKED \ diff --git a/scripts/testing/lane-image.mjs b/scripts/testing/lane-image.mjs index 303759e6f..1d4571f16 100644 --- a/scripts/testing/lane-image.mjs +++ b/scripts/testing/lane-image.mjs @@ -3,6 +3,8 @@ import { readFileSync, readdirSync, lstatSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { execFileSync } from 'node:child_process'; import { refuse } from './lane-state.mjs'; +import { imageTag } from '../image-tag.mjs'; +const lockTag=checkout=>{try{return imageTag(checkout);}catch(e){refuse(e.message);}}; // This is the declared build input, not an observation of a running binary. function pinnedBase(checkout){ const lines=readFileSync(join(checkout,'Containerfile'),'utf8').split('\n').filter(line=>/^\s*FROM(?:\s|$)/i.test(line)); @@ -31,21 +33,29 @@ export function verifyImage(receipt,checkout,inspect=inspectImage){ const actual=inspect(receipt.tag); if(actual.Id!==receipt.imageId)refuse('pinned image ID differs'); if(`${actual.Os}/${actual.Architecture}`!==receipt.platform)refuse('image platform differs'); - const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim();const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8')); - if(pkg.dependencies?.['@earendil-works/pi-coding-agent']&&receipt.tag!==`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`)refuse('image tag differs from release identity'); + // #1873 S5: the tag carries the pi version package-lock.json resolves, never the package.json range. + if(receipt.tag!==lockTag(checkout))refuse('image tag differs from release identity'); return receipt; } // Called by the CLI only while both lanes and the image lock are exclusive. -export function prepareImage(checkout,{grant,tempRoot,out},invoke){ +// #1873 S5: the image fit step. The checkout's own scripts/pi-fit-image.mjs runs the fit check on the +// pi package-lock.json resolves, as npm ci installed it in the checkout, before the build. It exits +// non-zero with one line when that pi is unfit or the check can't tell. compose.yaml has no build +// section, so the build below is a plain docker build of the Containerfile under the lock's tag. +export function imageFitStep(checkout){ + try{execFileSync(process.execPath,[join(checkout,'scripts/pi-fit-image.mjs')],{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.TMPDIR?{TMPDIR:process.env.TMPDIR}:{})}});} + catch(e){const line=String(e.stderr??'').trim().split('\n').pop();refuse(`image fit step refused: ${line||'it could not run'}`);} +} +export function prepareImage(checkout,{grant,tempRoot,out},invoke,fit=imageFitStep){ if(!grant)refuse('missing image preparation grant'); const baseDigest=pinnedBase(checkout); const inputDigest=imageFingerprint(checkout); - const pkg=JSON.parse(readFileSync(join(checkout,'package.json'),'utf8')); const release=readFileSync(join(checkout,'RELEASE'),'utf8').trim(); if(!/^\d+\.\d+\.\d+$/.test(release))refuse('invalid image release'); - const tag=`mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`; + const tag=lockTag(checkout); + fit(checkout); const run=invoke??((args)=>execFileSync('docker',args,{cwd:checkout,encoding:'utf8',stdio:['ignore','pipe','pipe'],env:{PATH:process.env.PATH,HOME:process.env.HOME,...(process.env.DOCKER_HOST?{DOCKER_HOST:process.env.DOCKER_HOST}:{}),COMPOSE_PROJECT_NAME:'mg197-image-prepare',MOSAIC_IMAGE_TAG:tag,MOSAIC_DATA_ROOT:join(tempRoot,'data'),MOSAIC_PROVIDER:'fixture',MOSAIC_MODEL:'fixture',PI_AUTH_FILE:'/dev/null'}})); - const start=performance.now();let buildOutput;try{buildOutput=run(['compose','build']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start; + const start=performance.now();let buildOutput;try{buildOutput=run(['build','-f','Containerfile','-t',tag,'.']);}catch(e){if(out)writeFileSync(join(out,'image-build.log'),String(e.stdout??'')+String(e.stderr??''),{mode:0o600,flag:'wx'});throw e;}if(out)writeFileSync(join(out,'image-build.log'),buildOutput,{mode:0o600,flag:'wx'});const buildWallMs=performance.now()-start; if(inputDigest!==imageFingerprint(checkout))refuse('image inputs changed while building'); const actual=JSON.parse(run(['image','inspect',tag,'--format','{{json .}}'])); const receipt={inputDigest,imageId:actual.Id,tag,baseDigest,platform:`${actual.Os}/${actual.Architecture}`,grant,builds:1,buildWallMs,at:new Date().toISOString()}; diff --git a/scripts/verify.sh b/scripts/verify.sh index ffddc89b9..8798cea1e 100755 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -37,7 +37,7 @@ ensure_release_aligned if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then echo "verify: image $IMAGE not found, building..." >&2 bootstrap_runtime_dir - docker compose build + build_image fi # 2. Run the agent request (stdout only = model response).