diff --git a/BUILD-LOG.md b/BUILD-LOG.md index 8cd73593a..fed78f4d7 100644 --- a/BUILD-LOG.md +++ b/BUILD-LOG.md @@ -6001,3 +6001,5 @@ Owner directed static-only fallback after actual nativepriority drift: no new pr 2026-11-03 | GPT-6.1 Sol implementation sub-agent | #797 / Gitea #1839 lifecycle Task 1 after | Added host-only createRequestStore/createRequestLifecycle, strict CAS transition reduction, byte-preserving intake from existing user-origin human.input/message records, immutable mode/association plan bindings, latest-evidence effect reservations, and broker-derived authorization for request.open/get/projects/propose. Autonomous/guided/existing are enumerated selections; existing stages a filtered software-project picker and a separate exact-association confirmation. PM preparation uses existing message.send authority and rechecks its live claim after awaits; only the owner resolves the user-routed nonblocking project.bootstrap decisions, with null task/requirement refs. No resource creation, PRD activation, scoped tasks, role launch or artifact execution. Missing preparation/admission/bootstrap/attachment remains an explicit setup wait; oversize full resource displays retain source bytes and wait at the existing decision size limit. Added five append-only lifecycle tables, immutable new 3n→3o migration, strict source/digest/version checks and fixture-only offline CLI/old-engine refusal evidence. Existing migrations, approved PRD, role JSON, Console code/verbs, queue/tracker and live config/service/secrets untouched. Minimal collateral: older migration test expectations advance to 3o; role Markdown names the newly gated action; stack-release restore counting includes all five new timed tables. RED intake 0/12; further binding RED 15/16, origin/association RED 21/23, setup/display RED 23/25 and receipt/path RED 23/25. Final intake 26/26; required intake/schema/store/hard-waits 41/41; full bus 1433/1433; full business 296/296; bounded CLI restore/accounting 7/7; sandbox import closure 1/1, all zero failures/skips. Prior full bus 1389 pass/30 fail comprised 27 changed-schema/closed-vocabulary expectations and three managed-launch failures traced to harness-binary-missing; restored existing pinned dependencies with npm ci --ignore-scripts, lockfile unchanged, then full bus 1425/1425 and final 1433/1433. Prior business 295/296 required the two existing hard-wait prose lists to match the closed vocabulary. Test fixture setup mistakes (tester switch, halted-before-launch and incomplete library mapping) were corrected without weakening assertions. Retained evidence: /mnt/storage/scratch/tmp/mosaic-797-task1-logs/; report: /mnt/storage/scratch/tmp/mosaic-797-lifecycle-task-1-report.md. Controller owns independent review and integration; local commit only, no push/merge/PR/deployment. 2026-11-04T00:20:48Z | Lead (Claude Opus 5.5, T3 c5c1e8ba) | #797 / Gitea #1839 lifecycle Task 1 landing | Review R1 (GPT-6.1 Sol): Needs fixes, I1 PM self-citation reachability, I2 existing route lost after setup waits, I3 stale association confirmed after async admission. I1 deferred to tracker #799 as a security follow-up (MVP rule). Fix round 1 (Sonnet 5.5 xhigh) bec358328: RequestState.route from selection.staged; staleCode recheck before admission and inside the recording transaction for propose and resume; 12 regressions that fail on 3b071cac7. Gates on bec358328: intake 38/38, full bus 1445/1445, business 296/296, zero skips (/mnt/storage/scratch/tmp/nlR5yC). Review R2 (GPT-6.1 Sol): I2 resolved; I3 partly resolved, an in-process change in the continuation after the final awaited inventory read still reaches proposal or confirmation (repro /mnt/storage/scratch/tmp/C5CJ6q). Lead decision: land 3b071cac7+bec358328 without a second round; residual is tracker #800, and the Task 2 executor must re-verify the association at execution. Tradeoff: confirmation records consent, not freshness, until #800 lands. Plan doc RequestState gains route. + +2026-11-04T01:50:11Z | Claude Sonnet 5.5 implementation sub-agent, t3/request-lifecycle-bootstrap | #797 / Gitea #1839 lifecycle Task 2 after | Protected project bootstrap that becomes usable at runtime, on 138171ca0. Organization writer: addProject (private exact-add path under the existing lock and receipt discipline, input and serialized bytes both compared against the plan, replacement/removal/library/unrelated-key changes and a mismatched receipt refused, stale revision refused), hold (sync check under the lock), findAddition; write still refuses every project change (organization-projects-protected). New bootstrap executor journals reserve, seed, working, mapping, records, intent, chat and activate as separate receipts bootstrap::; the code seed is published before the mapping and records init runs under the published mapping; interrupted effects are compared by deterministic path, stored commit and exact bytes and retained as bootstrap-uncertain, never removed or rewound. Original displayed intent is saved with writeHumanFile as the decision owner, writeRecord still cannot write spec/. Tracker: the product file omits tracker.project, the board lives in the immutable plan and admission projection, trackerFor skips reserved products and a fresh boot verifies product IDs. Runtime: activateManagedProject reloads only a receipt-backed additive root inside /products after revalidating the full business, prior mappings, credentials, bots, libraries and roles; broker admission and the launcher (via host IPC request.admission) refuse a pending product until activation; restart rebuilds readiness from receipts. initializeWorking (host-only, seat/working.mjs) seeds the working repository under the existing repo lock with no work ref or push authority. #800: the bound organization revision and association are re-verified synchronously inside organization.hold at the start of execution; a mismatch records setup wait request-plan-stale and performs no effect (regression: an_organization_change_after_confirmation_waits_without_effect and four sibling stale cases). Schema change was needed and is append-only: migration 3o to 3p widens the request_events kind CHECK with bootstrap.stale and bootstrap.completed (old table renamed away, new table created under its final name, rows and seq copied, guards recreated); 13 migration-pin tests moved 3o to 3p the way Task 1 did for 3n to 3o, with a new 3o_to_3p test (rows and seq kept, guards intact, the 3o engine refuses the migrated database). RED: project-bootstrap suite failed on writer.hold missing; bus suite failed 9/10 on the kind CHECK before 3p. Corrections: the first full bus run exposed the CHECK; a test expectation for a replayed revision named the revision guard when the append-only trigger fires first (test fixed, schema unchanged); the first-boot test opened a second broker before closing the first (writer-locked, fixture fixed). GREEN: brief command 18/18, kept-green trio 49/49, business 304/304 (was 296), bus 1456/1456 (was 1445), project+launcher 80/80, stack-release 225/225, related cli suites 422/422, seat 223/223, zero skips. Logs /mnt/storage/scratch/tmp/BKItrk/. Not executed on any running instance; local commit only, no push/merge/PR/deployment. diff --git a/docs/SESSIONS.md b/docs/SESSIONS.md index eb322870c..842d54129 100644 --- a/docs/SESSIONS.md +++ b/docs/SESSIONS.md @@ -1992,3 +1992,5 @@ are never rewritten or removed; corrections are new entries. 2026-11-03 | GPT-6.1 Sol implementation sub-agent, prepared t3/0c734dbd | #797 / Gitea #1839 lifecycle Task 1 | Durable pre-task intake, trusted origin/provenance, revision/plan-bound user selections, host-injected preparation/admission foundation and explicit setup waits; strict append-only 3n→3o offline migration with no old migration edits. Intake 26/26, required checks 41/41, full bus 1433/1433, business 296/296, bounded CLI restore/accounting 7/7 and sandbox import 1/1, zero skips; initial RED and schema/dependency/fixture failures retained or described in BUILD-LOG/report. Existing pinned dependencies restored via lockfile only. Controller owns queue/tracker, independent review and integration; tested local commit only; no subsequent lifecycle task or live/authority/Console changes. 2026-11-04 | Lead (Claude Opus 5.5, T3 c5c1e8ba) | #797 lifecycle Task 1 review, fix round and landing | R1 needs fixes, fix bec358328, R2 I2 resolved, I3 residual to #800, I1 to #799; landed on refactor + +2026-11-04 | Claude Sonnet 5.5 implementation sub-agent, t3/request-lifecycle-bootstrap | #797 / Gitea #1839 lifecycle Task 2 | Protected project bootstrap: exact-add organization writer, journaled bootstrap executor (reserve, seed, working repo, mapping, records, original intent as owner, chat, activate), runtime activation of a receipt-backed additive product root, host request.admission gate for the launcher, #800 stale re-verify inside the organization lock (request-plan-stale), append-only 3o to 3p migration. Business 304/304, bus 1456/1456, brief 18/18, project+launcher 80/80, stack-release 225/225, zero skips; local commit only, no push. diff --git a/packages/bus/schema.sql b/packages/bus/schema.sql index 37f7e609e..0a10100f1 100644 --- a/packages/bus/schema.sql +++ b/packages/bus/schema.sql @@ -535,7 +535,7 @@ CREATE TABLE request_events ( business TEXT NOT NULL, request TEXT NOT NULL REFERENCES events(id), revision INTEGER NOT NULL CHECK (revision >= 1), - kind TEXT NOT NULL CHECK (kind IN ('opened','selection.proposed','selection.staged','selection.confirmed','setup.wait','cancelled')), + kind TEXT NOT NULL CHECK (kind IN ('opened','selection.proposed','selection.staged','selection.confirmed','setup.wait','bootstrap.stale','bootstrap.completed','cancelled')), actor_role TEXT, actor_run TEXT, at TEXT NOT NULL, body TEXT NOT NULL CHECK (json_valid(body) AND json_type(body) = 'object'), diff --git a/packages/bus/src/broker.mjs b/packages/bus/src/broker.mjs index 8cd26df89..ae951de7d 100644 --- a/packages/bus/src/broker.mjs +++ b/packages/bus/src/broker.mjs @@ -640,6 +640,26 @@ export class Broker { if (next.human !== this.#businesses[name].human) fail('invalid-business'); this.#businesses[name] = next; } + // Lifecycle Task 2 (#797): the chat project a bootstrapped product lists under. Host-only, so no verb + // reaches it. State is 'free', 'associated' (this exact id, name and product) or 'conflict' (anything else + // holding the id or the name), which lets a retry insert the row once. + chatProjectState({ business, project, name: label, id: pid }) { + if (!Object.hasOwn(this.#businesses, business)) fail('unknown-business'); + const byId = this.#store.get('SELECT name,repo FROM projects WHERE id=? AND business=?', pid, business); + if (byId) return byId.name === label && byId.repo === project ? 'associated' : 'conflict'; + if (this.#store.get('SELECT 1 FROM projects WHERE id=?', pid)) return 'conflict'; + return this.#store.get('SELECT 1 FROM projects WHERE business=? AND name=?', business, label) ? 'conflict' : 'free'; + } + associateChat({ business, project, name: label, id: pid }) { + if (!name(label)) fail('invalid-request'); + identifier(project); + const state = this.chatProjectState({ business, project, name: label, id: pid }); + if (state === 'conflict') fail('project-exists'); + if (state === 'free') + this.#store.run('INSERT INTO projects(id,at,business,name,repo,created_by) VALUES(?,?,?,?,?,?)', + pid, this.#now(), business, label, project, this.#business(business).human); + return { id: pid }; + } #processingState() { try { const state = this.#processing.read(); diff --git a/packages/bus/src/migrations/3o-3p.sql b/packages/bus/src/migrations/3o-3p.sql new file mode 100644 index 000000000..155b3a265 --- /dev/null +++ b/packages/bus/src/migrations/3o-3p.sql @@ -0,0 +1,50 @@ +-- #797 / #1839 Task 2: the request journal learns two more kinds of event, bootstrap.stale (a confirmed plan no +-- longer matched the world, nothing was bound) and bootstrap.completed (every resource of a confirmed new-project +-- plan passed). request_events keeps its rows, its seq values and its guards; only the kind list widens. The guards +-- that read it, request_events_origin, request_events_revision and request_effects_binding, are dropped first so +-- the rename rewrites nothing, then created again as schema.sql has them. The copy runs with the guards gone, in +-- seq order, inside the migration's one transaction. +DROP TRIGGER request_events_origin; +DROP TRIGGER request_events_revision; +DROP TRIGGER request_effects_binding; +DROP TRIGGER request_events_no_update; +DROP TRIGGER request_events_no_delete; +DROP TRIGGER request_events_no_replace; +DROP INDEX request_events_request; +ALTER TABLE request_events RENAME TO request_events_3o; +CREATE TABLE request_events ( + seq INTEGER PRIMARY KEY AUTOINCREMENT, + business TEXT NOT NULL, + request TEXT NOT NULL REFERENCES events(id), + revision INTEGER NOT NULL CHECK (revision >= 1), + kind TEXT NOT NULL CHECK (kind IN ('opened','selection.proposed','selection.staged','selection.confirmed','setup.wait','bootstrap.stale','bootstrap.completed','cancelled')), + actor_role TEXT, actor_run TEXT, + at TEXT NOT NULL, + body TEXT NOT NULL CHECK (json_valid(body) AND json_type(body) = 'object'), + CHECK ((actor_role IS NULL) = (actor_run IS NULL)), + UNIQUE (business,request,revision) +) STRICT; +INSERT INTO request_events(seq,business,request,revision,kind,actor_role,actor_run,at,body) + SELECT seq,business,request,revision,kind,actor_role,actor_run,at,body FROM request_events_3o ORDER BY seq; +DELETE FROM sqlite_sequence WHERE name='request_events'; +INSERT INTO sqlite_sequence(name,seq) SELECT 'request_events', seq FROM sqlite_sequence WHERE name='request_events_3o'; +DROP TABLE request_events_3o; +CREATE INDEX request_events_request ON request_events (business,request,seq); +CREATE TRIGGER request_events_origin BEFORE INSERT ON request_events + WHEN NOT EXISTS (SELECT 1 FROM events e JOIN messages m ON m.id=e.subject AND m.id=json_extract(e.body,'$.message') + WHERE e.id=NEW.request AND e.business=NEW.business AND m.business=e.business AND e.kind='human.input' + AND e.actor_role IS NULL AND e.actor_run IS NULL AND json_extract(e.body,'$.kind')='instruction' AND m.from_role='human') + BEGIN SELECT RAISE(ABORT, 'request-origin-refused'); END; +CREATE TRIGGER request_events_revision BEFORE INSERT ON request_events + WHEN NEW.revision <> 1 + coalesce((SELECT max(revision) FROM request_events WHERE business=NEW.business AND request=NEW.request),0) + OR (NEW.revision=1) <> (NEW.kind='opened') + BEGIN SELECT RAISE(ABORT, 'request-revision-conflict'); END; +CREATE TRIGGER request_effects_binding BEFORE INSERT ON request_effects + WHEN NOT EXISTS (SELECT 1 FROM request_events WHERE business=NEW.business AND request=NEW.request) + OR EXISTS (SELECT 1 FROM request_effects WHERE business=NEW.business AND request=NEW.request AND op=NEW.op AND (digest<>NEW.digest OR kind<>NEW.kind)) + OR (NEW.stage<>'prepared' AND NOT EXISTS (SELECT 1 FROM request_effects WHERE business=NEW.business AND request=NEW.request AND op=NEW.op AND stage='prepared')) + OR EXISTS (SELECT 1 FROM request_effects WHERE business=NEW.business AND request=NEW.request AND op=NEW.op AND stage IN ('committed','refused')) + BEGIN SELECT RAISE(ABORT, 'request-effect-conflict'); END; +CREATE TRIGGER request_events_no_update BEFORE UPDATE ON request_events BEGIN SELECT RAISE(ABORT, 'request_events is append-only'); END; +CREATE TRIGGER request_events_no_delete BEFORE DELETE ON request_events BEGIN SELECT RAISE(ABORT, 'request_events is append-only'); END; +CREATE TRIGGER request_events_no_replace BEFORE INSERT ON request_events WHEN EXISTS (SELECT 1 FROM request_events WHERE seq=NEW.seq OR (business=NEW.business AND request=NEW.request AND revision=NEW.revision)) BEGIN SELECT RAISE(ABORT, 'request_events is append-only'); END; diff --git a/packages/bus/src/process.mjs b/packages/bus/src/process.mjs index 955a147c9..224317c31 100644 --- a/packages/bus/src/process.mjs +++ b/packages/bus/src/process.mjs @@ -37,6 +37,7 @@ const LAUNCH_OPS = new Set([ 'sessionClearUncertain', 'sessionClearRetire', 'sessionClearStart', + 'request.admission', ]); function launchOp(m) { if (m.op === 'identity') return runtime.identity(m.cap); @@ -79,6 +80,9 @@ function launchOp(m) { if (m.op === 'sessionClearUncertain') return runtime.sessionClearUncertain(m.record); if (m.op === 'sessionClearRetire') return runtime.sessionClearRetire(m.record); if (m.op === 'sessionClearStart') return runtime.sessionClearStart(m.record); + // Lifecycle Task 2 (#797): may the launcher clone this project's code checkout. Scoped to the one + // business and project named; a malformed ask answers launch-op-refused, never ready. + if (m.op === 'request.admission') return runtime.admission({ business: m.business, project: m.project }); if (m.op === 'credentialStatus') return runtime.credentialStatus(m.business, m.role); if (m.op === 'modelPreflight') return runtime.modelPreflight(m); if (m.op === 'modelStall') return runtime.modelStall(m); diff --git a/packages/bus/src/request-bootstrap.mjs b/packages/bus/src/request-bootstrap.mjs new file mode 100644 index 000000000..cf44e3134 --- /dev/null +++ b/packages/bus/src/request-bootstrap.mjs @@ -0,0 +1,268 @@ +// Lifecycle Task 2 (#797 / Gitea #1839): the host executor behind a confirmed new-project selection. +// +// prepareBootstrap builds the immutable resource plan the owner sees. executeBootstrap consumes exactly that +// plan's binding and walks the steps below, each under its own journal receipt (`bootstrap::`), +// so a restart re-drives it from the receipts and the on-disk evidence without doing any step twice: +// +// reserve re-verify the plan against the world (#800), then reserve the product paths and a marker +// seed clone the source commit, drop its remote, commit the one project-file change +// working seed the working repository from that commit +// mapping publish the one organization mapping through the private exact-add writer +// records initialize the records repository under that mapping +// intent save the original displayed intent as the owner's edit of spec/north-star.md +// chat associate the chat project +// activate admit the project to the running broker, then mark the product complete +// +// Every inspection compares deterministic paths, the stored commits and exact bytes. What it cannot prove +// is kept as it is and reported as `bootstrap-uncertain`; nothing unrelated is removed and no commit is rewound. +import { createHash } from 'node:crypto'; +import { execFileSync } from 'node:child_process'; +import { lstatSync, mkdirSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync } from 'node:fs'; +import { dirname, join, sep } from 'node:path'; +import { digest, refuse, validatePlan } from './request-store.mjs'; +import { gitEnv, runGit } from '../../seat/src/checkout.mjs'; +import { initializeWorking } from '../../seat/src/working.mjs'; +import { initProject, writeHumanFile } from '../../project/src/index.mjs'; +import { SAFE, identity } from '../../project/src/git.mjs'; +import { loadProject } from '../../business/src/project.mjs'; +import { canonicalJson } from '../../business/src/util.mjs'; +import { managedPaths, markerBytes, markerFile, readMarker } from '../../business/src/project-bootstrap.mjs'; + +const INTENT_PATH = 'spec/north-star.md'; +const LAYOUT_SUBJECT = 'Initialize project layout v1'; +const exists = path => { try { return lstatSync(path); } catch (e) { if (e.code === 'ENOENT') return null; throw e; } }; +const within = (parent, path) => path === parent || path.startsWith(parent + sep); +const overlaps = (a, b) => within(a, b) || within(b, a); +const plain = x => x !== null && typeof x === 'object' && !Array.isArray(x); + +// The checked-out commit of a repository root, read synchronously so the stale check has no await in it. +function headOf(root) { + try { + return execFileSync('git', [...SAFE, '-C', root, 'rev-parse', '--verify', 'HEAD^{commit}'], + { env: { ...gitEnv(), GIT_CEILING_DIRECTORIES: dirname(root) }, encoding: 'utf8', timeout: 10000, stdio: ['ignore', 'pipe', 'pipe'] }).trim(); + } catch { return null; } +} + +// The chat project id is a function of the plan, so a retry computes the same one. +export function chatProjectId(request, planDigest) { + const h = createHash('sha256').update(`chat-project:${request}:${planDigest}`).digest('hex'); + return `${h.slice(0, 8)}-${h.slice(8, 12)}-4${h.slice(13, 16)}-8${h.slice(17, 20)}-${h.slice(20, 32)}`; +} + +// prepareBootstrap({ business, request, sourceProject, project, name, mode, dataRoot, organizationRevision, tracker, staff, intent }) +// Resolves the validated, immutable plan for one mode. `business` is the organization document; the plan names +// the source commit, the two managed roots and the original intent exactly as the owner will confirm them. +export async function prepareBootstrap({ business, request, sourceProject, project, name, mode, dataRoot, organizationRevision, tracker, staff, intent }) { + const source = business?.projects?.[sourceProject]; + if (!plain(source) || source.kind === 'library' || typeof source.root !== 'string') refuse('request-project-refused'); + const commit = headOf(source.root); + if (!commit) refuse('request-project-refused'); + const factory = loadProject(source.root); + // The plan carries variables only, so a project file with role overrides cannot be reproduced from it. + if (Object.keys(factory.roles).length) refuse('request-project-refused'); + const vars = { ...factory.vars }; + delete vars['tracker.project']; + const { root, repo } = managedPaths({ dataRoot, business: business.id, project }); + return validatePlan({ version: 1, business: business.id, request, project, name, chat_project: null, mode, organization_revision: organizationRevision, + source: { project: sourceProject, commit, remote: source.remote ?? null, forks: [] }, code_root: root, records_root: repo, + tracker: { root: tracker.root, project: tracker.project }, staff: { ...staff }, + intent: { text: intent.text, sha256: digest(intent.text), source_message: intent.source_message, generated: false }, + project_vars: JSON.parse(JSON.stringify(vars)), existing: false }); +} + +class Uncertain extends Error {} + +// executeBootstrap({ binding, owner, journal, services }) -> { project, chat_project, spec_commit, organization_revision } +// services: dataRoot, organization (the writer), chat { state, associate }, activate (synchronous), barrier (tests). +// Throws RequestError('request-plan-stale') before any effect when the world no longer matches the plan; any other +// failure leaves the receipts as they are for the next run. +export async function executeBootstrap({ binding, owner, journal, services }) { + const plan = validatePlan(structuredClone(binding?.plan)); + if (plan.existing || binding.digest !== digest(plan)) refuse('invalid-request-plan'); + const { organization: writer, dataRoot } = services; + const { business, project, request } = plan; + const planDigest = binding.digest, base = name => `bootstrap:${planDigest}:${name}`; + const here = managedPaths({ dataRoot, business, project }); + const product = dirname(here.root), marker = markerFile({ dataRoot, business, project }); + const chatId = chatProjectId(request, planDigest), human = () => writer.read(business).document.human; + const uncertain = () => { throw new Uncertain('bootstrap-uncertain'); }; + const markerOwns = () => { + const m = readMarker(marker); + return m !== null && m.version === 1 && m.project === project && m.request === request && m.plan_digest === planDigest; + }; + + const git = (cwd, args, extra = {}, file = false) => runGit([...SAFE, ...(file ? ['-c', 'protocol.file.allow=always'] : []), ...(cwd ? ['-C', cwd] : []), ...args], + { ...gitEnv(), ...(cwd ? { GIT_CEILING_DIRECTORIES: dirname(cwd) } : {}), ...extra }); + const out = async (cwd, args, extra, file) => { const r = await git(cwd, args, extra, file); if (!r.ok) refuse('bootstrap-git-failed'); return r.out.trim(); }; + const tryOut = async (cwd, args) => { const r = await git(cwd, args); return r.ok ? r.out.trim() : null; }; + + // One receipt per step. A committed receipt replays its result; anything else re-inspects the evidence. + async function step(name, run, { after = true } = {}) { + const op = base(name), last = journal.effects(business, request).filter(r => r.op === op).at(-1); + if (last?.stage === 'committed') return last.result; + if (last?.stage === 'refused') refuse(last.code ?? 'bootstrap-refused'); + const header = { version: 1, business, request, op, kind: 'bootstrap', digest: planDigest }; + journal.prepareEffect({ ...header, stage: 'prepared', result: null, code: null }); + let result; + try { result = await run({ recovering: last !== undefined }); } + catch (e) { + if (!(e instanceof Uncertain)) throw e; + journal.finishEffect({ ...header, stage: 'uncertain', result: null, code: 'bootstrap-uncertain' }); + refuse('bootstrap-uncertain'); + } + if (after) await services.barrier?.(name); + journal.finishEffect({ ...header, stage: 'committed', result, code: null }); + return result; + } + const done = name => journal.effects(business, request).filter(r => r.op === base(name) && r.stage === 'committed').at(-1)?.result ?? uncertain(); + + // 1. reserve. Everything the plan assumed is checked in one synchronous step, inside the organization lock, so + // no write lands between the check and the reservation. A mismatch is a stale plan and has no effect. + const reserve = ({ recovering }) => { + if (recovering && exists(marker)) { if (!markerOwns()) uncertain(); return { state: 'reserved' }; } + if (recovering && (exists(product) || exists(here.repo))) uncertain(); + const stale = () => refuse('request-plan-stale'); + return writer.hold(business, ({ revision, document }) => { + const source = document.projects?.[plan.source.project]; + if (revision !== plan.organization_revision || document.human !== owner || document.id !== business) stale(); + if (!plain(source) || source.kind === 'library' || (source.remote ?? null) !== plan.source.remote) stale(); + if (Object.hasOwn(document.projects, project)) stale(); + if (plan.code_root !== here.root || plan.records_root !== here.repo || overlaps(here.root, here.repo)) stale(); + for (const declared of Object.values(document.projects)) + for (const path of [declared?.root, declared?.repo]) if (typeof path === 'string' && (overlaps(path, here.root) || overlaps(path, here.repo))) stale(); + if (services.chat.state({ business, project, name: plan.name, id: chatId }) !== 'free') stale(); + for (const target of [product, here.repo]) { + if (!within(dataRoot, target) || target === dataRoot) stale(); + for (let path = target; path !== dataRoot; path = dirname(path)) { + const st = exists(path); + if (st === null) continue; + if (path === target || st.isSymbolicLink() || !st.isDirectory()) stale(); + } + } + if (headOf(source.root) !== plan.source.commit) stale(); + mkdirSync(product, { recursive: true, mode: 0o700 }); + mkdirSync(here.repo, { recursive: true, mode: 0o700 }); + writeFileSync(marker, markerBytes({ state: 'reserved', request, plan_digest: planDigest, project }), { flag: 'wx', mode: 0o600 }); + return { state: 'reserved' }; + }); + }; + + // 2. seed. The checkout is the source commit plus one commit that changes only the project file. + const seedBytes = JSON.stringify({ projectVersion: 1, id: project, vars: plan.project_vars }, null, 2) + '\n'; + const seedFile = join(here.root, '.mosaic/project.json'); + async function seeded() { + const code = here.root, head = await out(code, ['rev-parse', 'HEAD']), remotes = await out(code, ['remote']); + const status = await out(code, ['status', '--porcelain=v1', '--untracked-files=all', '--ignored=matching']); + if (!['', 'origin'].includes(remotes)) uncertain(); + if (head === plan.source.commit) { + // Cloned, or seeded but not yet committed. The only change allowed is the exact project file. + const onlyFile = status === '' || (/^( M|M ) \.mosaic\/project\.json$/.test(status) && readFileSync(seedFile, 'utf8') === seedBytes); + if (!onlyFile) uncertain(); + if (remotes === 'origin') await out(code, ['remote', 'remove', 'origin']); + if (status === '') writeFileSync(seedFile, seedBytes); + await out(code, ['add', '--', '.mosaic/project.json']); + await out(code, ['commit', '-q', '-m', `Bootstrap ${project} from ${plan.source.project}`], identity(human())); + return { commit: await out(code, ['rev-parse', 'HEAD']), base: plan.source.commit }; + } + const names = await tryOut(code, ['diff', '--name-status', `${head}~1`, head]); + if (await tryOut(code, ['rev-parse', `${head}~1`]) === plan.source.commit && names === 'M\t.mosaic/project.json' && + status === '' && remotes === '' && readFileSync(seedFile, 'utf8') === seedBytes) return { commit: head, base: plan.source.commit }; + return uncertain(); + } + const seed = async () => { + const code = exists(here.root), tmp = join(product, 'code.tmp'); + if (code) { if (!code.isDirectory() || code.isSymbolicLink() || !markerOwns()) uncertain(); return seeded(); } + const document = writer.read(business).document, source = document.projects?.[plan.source.project]; + if (!markerOwns() || !plain(source) || source.kind === 'library' || (source.remote ?? null) !== plan.source.remote) uncertain(); + if (exists(tmp)) rmSync(tmp, { recursive: true, force: true }); + await out(null, ['clone', '-q', '--no-hardlinks', '--template=', '--', source.root, tmp], {}, true); + if (await out(tmp, ['rev-parse', 'HEAD']) !== plan.source.commit) refuse('request-plan-stale'); + await out(tmp, ['remote', 'remove', 'origin']); + renameSync(tmp, here.root); + return seeded(); + }; + + // 3. working. The working repository's main follows the checkout's HEAD, which is the seed commit. + const working = async () => { + const seedCommit = done('seed').commit, w = await initializeWorking({ dataRoot, business, project, root: here.root }); + if (w.commit !== seedCommit) uncertain(); + return { commit: w.commit }; + }; + + // 4. mapping. Only the private exact-add writer can publish it; a revision that moved needs a fresh preview. + const entry = { root: here.root, repo: here.repo }; + const receipt = { request, plan_digest: planDigest, project, data_root: dataRoot }; + const mapping = () => { + const found = writer.findAddition(business, { request, plan_digest: planDigest, project }); + const now = writer.read(business); + if (found) { + if (canonicalJson(now.document.projects[project]) !== canonicalJson(entry)) uncertain(); + return { organization_revision: found.after }; + } + if (Object.hasOwn(now.document.projects, project)) uncertain(); + try { writer.addProject(business, { revision: plan.organization_revision, project, entry, receipt }, now.document.human); } + catch (e) { if (e?.code === 'revision-conflict') refuse('request-plan-stale'); throw e; } + return { organization_revision: writer.findAddition(business, { request, plan_digest: planDigest, project })?.after ?? uncertain() }; + }; + + // 5. records. The layout commit, by the owner, in the empty directory the reservation made. + const context = () => ({ business: writer.read(business).document, dataRoot, project }); + const records = async () => { + const repo = here.repo, st = exists(repo); + if (!st || !st.isDirectory() || st.isSymbolicLink()) uncertain(); + if (readdirSync(repo).length === 0) { const made = await initProject(context(), { author: human() }); return { commit: made.commit }; } + const count = await tryOut(repo, ['rev-list', '--count', 'HEAD']); + const head = await tryOut(repo, ['log', '-1', '--format=%H|%an|%s', 'HEAD']); + if (count !== '1' || head === null) uncertain(); + const [commit, author, subject] = head.split('|'); + if (author !== human() || subject !== LAYOUT_SUBJECT || await tryOut(repo, ['status', '--porcelain=v1', '--untracked-files=all']) !== '') uncertain(); + return { commit }; + }; + + // 6. intent. The original displayed text, saved as the owner through the record writer. + const intent = async () => { + const repo = here.repo, init = done('records').commit; + const count = await tryOut(repo, ['rev-list', '--count', 'HEAD']), head = await tryOut(repo, ['rev-parse', 'HEAD']); + if (count === '1' && head === init) { + const saved = await writeHumanFile({ path: INTENT_PATH, content: plan.intent.text, base: init, ...context() }); + return { commit: saved.commit }; + } + const line = await tryOut(repo, ['log', '-1', '--format=%P|%an|%s', 'HEAD']); + if (count !== '2' || line === null) uncertain(); + const [parent, author, subject] = line.split('|'); + if (parent !== init || author !== human() || subject !== `Edit ${INTENT_PATH}` || + await tryOut(repo, ['status', '--porcelain=v1', '--untracked-files=all']) !== '' || + readFileSync(join(repo, INTENT_PATH), 'utf8') !== plan.intent.text) uncertain(); + return { commit: head }; + }; + + // 7. chat. The association is one row; a row that is not this plan's is a conflict, never replaced. + const chat = () => { + const args = { business, project, name: plan.name, id: chatId }, state = services.chat.state(args); + if (state === 'conflict') uncertain(); + if (state === 'free') services.chat.associate(args); + return { chat_project: chatId }; + }; + + // 8. activate. Synchronous from the activation to the receipt, so the broker and the marker agree. + const activate = async () => { + await services.barrier?.('activate'); + const mapped = done('mapping'); + services.activate({ business, project, receipt, organization_revision: mapped.organization_revision }); + const complete = markerBytes({ state: 'complete', request, plan_digest: planDigest, project }); + writeFileSync(`${marker}.tmp`, complete, { mode: 0o600 }); + renameSync(`${marker}.tmp`, marker); + return { project, organization_revision: mapped.organization_revision }; + }; + + await services.barrier?.('begin'); + await step('reserve', reserve); + await step('seed', seed); + await step('working', working); + await step('mapping', mapping); + await step('records', records); + const saved = await step('intent', intent); + const associated = await step('chat', chat); + const active = await step('activate', activate, { after: false }); + return { project, chat_project: associated.chat_project, spec_commit: saved.commit, organization_revision: active.organization_revision }; +} diff --git a/packages/bus/src/request-lifecycle.mjs b/packages/bus/src/request-lifecycle.mjs index ab04b732d..cc261face 100644 --- a/packages/bus/src/request-lifecycle.mjs +++ b/packages/bus/src/request-lifecycle.mjs @@ -176,10 +176,26 @@ export function createRequestLifecycle({ journal, services }) { }); }); } + // Runs the confirmed bootstrap plan through the host executor. The executor re-verifies the bound + // organization revision and association before any effect (#800); a stale plan reverts to selection so + // the owner gets a fresh preview, and anything else waits as uncertain until recovery drives it again. + async function runBootstrap(state) { + if (!services.bootstrap || state.phase !== 'bootstrap' || !state.plan_digest) return state; + const binding = state.selection?.bindings.find(b => b.digest === state.plan_digest); + if (!binding || binding.plan.existing) return state; + let done; + try { done = await services.bootstrap({ binding: structuredClone(binding), state: structuredClone(state) }); } + catch (e) { + if (e?.code === 'request-plan-stale') return append(state, 'bootstrap.stale', { digest: binding.digest, code: 'request-plan-stale' }); + return wait(state, 'bootstrap-uncertain'); + } + return append(state, 'bootstrap.completed', { digest: binding.digest, project: done.project, chat_project: done.chat_project, + spec_commit: done.spec_commit, organization_revision: done.organization_revision }); + } async function resume(business, request) { return serial(business, request, async () => { const state = stateOf(business, request); - if (state.wait?.kind !== 'decision') return state; + if (state.wait?.kind !== 'decision') return runBootstrap(state); const close = services.readDecision(business, state.selection.decision); if (!close || close.op === 'seen') return state; if (close.op === 'withdrawn' || close.op === 'expired') return append(state, 'cancelled', { decision: state.selection.decision }); @@ -199,11 +215,12 @@ export function createRequestLifecycle({ journal, services }) { // Admission is the last slow call. Read the association once more and confirm only if it still // matches, in the same synchronous step as the append. const rows = await inventory(); - return journal.transaction(tx => { + const confirmed = journal.transaction(tx => { const late = staleCode(p, state, planArgs, planSource, rows); if (late) return wait(state, late, null, {}, tx.appendRequest); return append(state, 'selection.confirmed', { decision: binding.decision, choice: binding.choice, digest: binding.digest }, null, tx.appendRequest); }); + return runBootstrap(confirmed); }); } return Object.freeze({ open, get, projects, propose, resume, diff --git a/packages/bus/src/request-store.mjs b/packages/bus/src/request-store.mjs index 2e1219686..5ea2acae5 100644 --- a/packages/bus/src/request-store.mjs +++ b/packages/bus/src/request-store.mjs @@ -96,6 +96,17 @@ function reduce(state, e) { s.project = p.project; s.chat_project = p.chat_project; s.mode = p.mode; s.mode_revision = 1; s.plan_digest = binding.digest; s.phase = p.existing ? 'discovery' : 'bootstrap'; s.wait = { kind: 'setup', code: p.existing ? 'request-attachment-unavailable' : 'request-bootstrap-unavailable', subject: s.request, resume_phase: s.phase, observed_revision: s.revision }; + } else if (e.kind === 'bootstrap.stale') { + // The confirmed plan no longer matches the world (#800). Nothing was bound; the owner gets a fresh preview. + exact(b, ['digest','code'], [], code); + if (s.phase !== 'bootstrap' || s.plan_digest !== b.digest || b.code !== 'request-plan-stale') refuse(code); + s.project = null; s.chat_project = null; s.mode = null; s.mode_revision = 0; s.plan_digest = null; s.selection = null; s.phase = 'selection'; + s.wait = { kind: 'setup', code: b.code, subject: s.request, resume_phase: 'selection', observed_revision: s.revision }; + } else if (e.kind === 'bootstrap.completed') { + exact(b, ['digest','project','chat_project','spec_commit','organization_revision'], [], code); + if (s.phase !== 'bootstrap' || s.plan_digest !== b.digest || b.project !== s.project || !id(b.chat_project) || !oid(b.spec_commit) || !text(b.organization_revision, 256)) refuse(code); + s.chat_project = b.chat_project; s.spec_commit = b.spec_commit; s.phase = 'discovery'; + s.wait = { kind: 'setup', code: 'request-discovery-unavailable', subject: s.request, resume_phase: 'discovery', observed_revision: s.revision }; } else if (e.kind === 'cancelled') { exact(b, ['decision'], [], code); if (s.wait?.kind !== 'decision' || s.selection?.decision !== b.decision) refuse(code); @@ -139,6 +150,16 @@ export function createRequestStore(store) { if (!d || d.action !== 'project.bootstrap' || d.route_to !== 'human' || d.blocking !== 0 || d.task_ref !== null || d.requirement_ref !== null || d.raised_by_role !== state.pm || store.get("SELECT 1 FROM decision_events WHERE decision=? AND op IN ('resolved','withdrawn','expired')", d.id)) refuse('invalid-request-event'); } + // A bootstrap outcome follows only from the receipts it names, so the journal cannot claim a project + // that was not activated, or retreat from one whose mapping already landed. + if (e.kind === 'bootstrap.completed' || e.kind === 'bootstrap.stale') { + const op = step => store.get("SELECT result FROM request_effects WHERE business=? AND request=? AND op=? AND kind='bootstrap' AND stage='committed'", + e.business, e.request, `bootstrap:${e.body.digest}:${step}`); + if (e.kind === 'bootstrap.completed') { + const done = op('activate'); + if (!done || JSON.parse(done.result).project !== e.body.project) refuse('invalid-request-event'); + } else if (op('mapping')) refuse('invalid-request-event'); + } store.run('INSERT INTO request_events(business,request,revision,kind,actor_role,actor_run,at,body) VALUES(?,?,?,?,?,?,?,?)', e.business, e.request, next.revision, e.kind, e.actor.role, e.actor.run, new Date().toISOString(), JSON.stringify(e.body)); return next; @@ -168,6 +189,17 @@ export function createRequestStore(store) { return Object.freeze({ readRequest, appendRequest: e => store.transaction(() => appendInTransaction(e)), prepareEffect: r => receipt(r, true), finishEffect: r => receipt(r, false), transaction: fn => store.transaction(() => fn({ appendRequest: appendInTransaction, readRequest })), + // Receipts read back, parsed. Hosts use these to resume a bootstrap and to rebuild readiness. + effects: (business, request) => { + if (!key(business) || !id(request)) refuse('invalid-request'); + return store.all('SELECT version,business,request,op,kind,digest,stage,result,code FROM request_effects WHERE business=? AND request=? ORDER BY seq', business, request) + .map(x => ({ ...x, result: x.result === null ? null : JSON.parse(x.result) })); + }, + committed: (business, kind) => { + if (!key(business)) refuse('invalid-request'); + return store.all("SELECT request,op,digest,result FROM request_effects WHERE business=? AND kind=? AND stage='committed' ORDER BY seq", business, kind) + .map(x => ({ ...x, result: x.result === null ? null : JSON.parse(x.result) })); + }, listRequests: () => store.all('SELECT DISTINCT business,request FROM request_events ORDER BY business,request'), proposal: (business, request, op) => { const row = store.get("SELECT body FROM request_events WHERE business=? AND request=? AND json_extract(body,'$.op')=? ORDER BY seq DESC LIMIT 1", business, request, op); diff --git a/packages/bus/src/runtime.mjs b/packages/bus/src/runtime.mjs index 44eb24bb1..0440df5cf 100644 --- a/packages/bus/src/runtime.mjs +++ b/packages/bus/src/runtime.mjs @@ -4,6 +4,8 @@ import { fileURLToPath } from 'node:url'; import { Store } from './store.mjs'; import { createRequestStore } from './request-store.mjs'; import { createRequestLifecycle } from './request-lifecycle.mjs'; +import { executeBootstrap } from './request-bootstrap.mjs'; +import { managedPaths, managedState } from '../../business/src/project-bootstrap.mjs'; import * as projectModule from '../../project/src/index.mjs'; import * as libraryModule from '../../library/src/index.mjs'; import { Broker, BusError, LIBRARY_FETCH_FIX } from './broker.mjs'; @@ -275,6 +277,22 @@ export async function startBroker({ const modelRuns = new Map(); let store, credentials, server, adapter, processing, syncTimer = null, syncPass = null, accessTimer = null; try { + // Lifecycle Task 2 (#797): a product whose mapping is published but not yet activated is not part of + // this process. The request journal says which are activated; every other managed entry waits for the + // bootstrap executor's activateManagedProject, so the broker, the Credentials roots and the boot copy + // never see a half-built product. `businesses` becomes this process's own copy. + store = new Store(dataRoot); + const requestJournal = createRequestStore(store); + const activatedProjects = business => new Set(requestJournal.committed(business, 'bootstrap') + .filter(r => r.op.endsWith(':activate') && typeof r.result?.project === 'string').map(r => r.result.project)); + const pendingProject = (business, project, entry, activated) => entry?.kind !== 'library' && !activated.has(project) + && managedState({ dataRoot, business, project, entry }) !== 'unmanaged'; + const withoutPending = (business, b, admit = []) => { + const activated = new Set([...activatedProjects(business), ...admit]); + const projects = Object.fromEntries(Object.entries(b.projects ?? {}).filter(([project, entry]) => !pendingProject(business, project, entry, activated))); + return Object.keys(projects).length === Object.keys(b.projects ?? {}).length ? b : { ...b, projects }; + }; + businesses = Object.fromEntries(Object.entries(businesses).map(([id, b]) => [id, withoutPending(id, b)])); const proxyReceipt = entry => process.stderr.write(`model-proxy: ${JSON.stringify(entry)}\n`); const references = {}; for (const [business, b] of Object.entries(businesses)) { @@ -289,7 +307,6 @@ export async function startBroker({ ]; // Row 199 (#1697, design D1): the per-agent store is /secrets/agents. const storeRoot = organization ? join(organization.dir, 'secrets', 'agents') : null; - store = new Store(dataRoot); let requestLifecycle; // Store's writer lock admits one bus per data root before it holds the management client. if (proxy) { @@ -479,7 +496,7 @@ export async function startBroker({ const b = loadBusiness(id, { dir: organization.dir, rolesDir: organization.rolesDir }); const resolved = {}; for (const instance of Object.keys(b.roles)) resolved[instance] = resolveInstance({ system: organization.system, business: b, instance }); - const next = busBusiness(b, resolved); + const next = withoutPending(id, busBusiness(b, resolved)); if (!sameFixed(booted.get(id), next)) { reloadLog({ event: 'reload-needs-restart', business: id }); return false; @@ -491,8 +508,60 @@ export async function startBroker({ return false; } } + // Lifecycle Task 2 (#797): the only way a project joins a running bus. The bootstrap executor calls it + // synchronously after every earlier step passed. It admits exactly the receipt-backed addition under the + // organization lock: the file must still be this business plus that entry, with the credentials, bots, + // libraries, roles and prior mappings the bus booted with. Anything else leaves the product pending. + const writer = organization ? organizationWriter({ dir: organization.dir, rolesDir: organization.rolesDir }) : null; + const known = new Map(Object.entries(businesses).map(([id, b]) => [id, canonicalJson(b.projects ?? {})])); + function activateManagedProject({ business: id, project, receipt, organization_revision }) { + if (!writer || !booted.has(id) || !Object.hasOwn(businesses, id)) throw new BusError('unknown-business'); + if (Object.hasOwn(businesses[id].projects ?? {}, project)) return; // already admitted: activation is idempotent + const refuse = code => { throw new BusError(code); }; + writer.hold(id, () => { + const found = writer.findAddition(id, receipt); + if (!found || found.after !== organization_revision) refuse('bootstrap-receipt-mismatch'); + const b = loadBusiness(id, { dir: organization.dir, rolesDir: organization.rolesDir }); + const resolved = {}; + for (const instance of Object.keys(b.roles)) resolved[instance] = resolveInstance({ system: organization.system, business: b, instance }); + const full = busBusiness(b, resolved); + const entry = full.projects?.[project]; + const here = managedPaths({ dataRoot, business: id, project }); + if (!entry || entry.kind === 'library' || entry.root !== here.root || entry.repo !== here.repo + || Object.keys(entry).some(k => !['root', 'repo', 'remote'].includes(k))) refuse('bootstrap-mapping-mismatch'); + const next = withoutPending(id, full, [project]); + const { [project]: _added, ...before } = next.projects; + if (canonicalJson(before) !== known.get(id)) refuse('bootstrap-mapping-mismatch'); + const expected = { ...booted.get(id), roots: canonicalJson([...JSON.parse(booted.get(id).roots), entry.root].sort()) }; + if (!sameFixed(expected, next)) refuse('bootstrap-restart-required'); + broker.reloadBusiness(id, next); + protectedRoots.push(entry.root); + businesses[id] = next; + booted.set(id, expected); + known.set(id, canonicalJson(next.projects)); + }); + } + // Host IPC request.admission: may a project's code checkout be cloned and its tools used. A project the + // broker does not hold is never ready, and a declared one that is not yet held says it is pending. + function admission({ business: id, project }) { + const no = code => ({ ready: false, revision: null, code }); + if (typeof id !== 'string' || typeof project !== 'string' || !Object.hasOwn(businesses, id)) return no('project-unknown'); + let revision = null, declared = false; + try { + if (writer) { + const now = writer.read(id); + revision = now.revision; declared = Object.hasOwn(now.document.projects ?? {}, project); + } + } catch { return no('organization-unreadable'); } + if (Object.hasOwn(businesses[id].projects ?? {}, project)) return { ready: true, revision, code: null }; + return no(declared ? 'project-pending' : 'project-unknown'); + } broker = new Broker({ - requestLifecycle: services => (requestLifecycle = createRequestLifecycle({ journal: createRequestStore(store), services: { ...requestServices, ...services } })), + requestLifecycle: services => (requestLifecycle = createRequestLifecycle({ journal: requestJournal, services: { ...requestServices, ...services, + // The new-project executor needs the organization writer; without one a confirmed selection stays at its setup wait. + ...(writer ? { bootstrap: ({ binding, state }) => processing.work(() => executeBootstrap({ binding, owner: state.owner, journal: requestJournal, services: { + dataRoot, organization: writer, activate: activateManagedProject, barrier: requestServices.barrier, + chat: { state: args => broker.chatProjectState(args), associate: args => broker.associateChat(args) } } })) } : {}) } })), // FED R3: the access file is read per user request; a refusal leaves the user no grant. access: access ? () => loadAccess(access.file, access.context) : null, // FED R4 (#1801, design 3.7 trigger 1): a request stats the file and reads it only when this @@ -887,6 +956,7 @@ export async function startBroker({ return { broker, requestLifecycle, + admission, credentials, // P2 consumes this inside the bus only. No process IPC/verb returns it. get modelProxy() { return modelProxyClient; }, diff --git a/packages/bus/src/store.mjs b/packages/bus/src/store.mjs index 1aaa5033f..c74b70522 100644 --- a/packages/bus/src/store.mjs +++ b/packages/bus/src/store.mjs @@ -13,8 +13,8 @@ import { join, isAbsolute } from 'node:path'; import { createHash } from 'node:crypto'; const ddl = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8'); -export const SCHEMA_SHA256 = '5a055f1687704043439d28c5ed2f1b2131b3c3d1af6dfdb1aa4f3cdc01dc01d6'; -export const SCHEMA_VERSION = '3o'; +export const SCHEMA_SHA256 = 'ae56fbdcb5ea9df3a15a47c7978f5c22516cf1c85dd9a1c9885a63bf7f80d252'; +export const SCHEMA_VERSION = '3p'; // Ordered, named offline steps. New schema changes append a step and keep prior DDL pinned. const legacyDDL = readFileSync(new URL('./migrations/3b.sql', import.meta.url), 'utf8'); const LEGACY_SHA256 = '179ffe356d4ff19a49b5ebad39b6c6bfd7771deb8e1b7c55b5e746f039d69e65'; @@ -87,6 +87,12 @@ const migrations = [{ from: '3n', to: '3o', sql: readFileSync(new URL('./migrations/3n-3o.sql', import.meta.url), 'utf8'), +}, { + // #797 / #1839 Task 2: the request journal's bootstrap.stale and bootstrap.completed kinds. request_events + // is rebuilt with the wider kind list and keeps every row, seq and guard. + from: '3o', + to: '3p', + sql: readFileSync(new URL('./migrations/3o-3p.sql', import.meta.url), 'utf8'), }]; const timedTables = [ 'events', diff --git a/packages/bus/tests/account-migration.test.mjs b/packages/bus/tests/account-migration.test.mjs index 2862617ac..ccb96b5c6 100644 --- a/packages/bus/tests/account-migration.test.mjs +++ b/packages/bus/tests/account-migration.test.mjs @@ -89,6 +89,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) { copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql')); copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql')); copyFileSync(here('../src/migrations/3n-3o.sql'), join(moduleRoot, 'src/migrations/3n-3o.sql')); + copyFileSync(here('../src/migrations/3o-3p.sql'), join(moduleRoot, 'src/migrations/3o-3p.sql')); return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store; } @@ -96,7 +97,7 @@ test('REQ-LIMIT-4 3h copy migrates to 3i: every row keeps its seq, the sequence const { root, path } = source3h(t), before = snapshot(path), was = master(path); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); // Every table's rows as they were, usage_records with its seq values and sqlite_sequence with 50. assert.deepEqual(migrated3l(snapshot(path, before.columns).rows), before.rows); assert.deepEqual(before.rows.usage_records.map(r => r.seq), [1, 2, 3]); @@ -146,7 +147,7 @@ test('REQ-LIMIT-4 a 3h→3i step that fails partway leaves the store at 3h with assert.deepEqual(snapshot(path, before.columns).rows, before.rows); // Still a whole 3h store: the real step migrates it. new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(migrated3l(snapshot(path, before.columns).rows), before.rows); }); @@ -158,14 +159,14 @@ test('REQ-LIMIT-4 the 3h engine refuses a migrated store, migrating or not, and schema: ddl3h, store: (source) => source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3h).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3h';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3h';") .replace(/\}, \{\n {2}\/\/ Row 126 \(#1620\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'), }); const pinned = readFileSync(join(root, 'engine-3h/src/store.mjs'), 'utf8'); assert.ok(pinned.includes("export const SCHEMA_VERSION = '3h';") && !pinned.includes('3h-3i'), 'the 3h engine has no 3h→3i step'); assert.throws(() => new Store3h(root), /schema-mismatch/); assert.throws(() => new Store3h(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(snapshot(path, after.columns).rows, after.rows); }); diff --git a/packages/bus/tests/chief-of-staff.test.mjs b/packages/bus/tests/chief-of-staff.test.mjs index 05c1f6680..8e5f22fb3 100644 --- a/packages/bus/tests/chief-of-staff.test.mjs +++ b/packages/bus/tests/chief-of-staff.test.mjs @@ -614,7 +614,7 @@ async function database3c(t) { let source = readFileSync(here('../src/store.mjs'), 'utf8'); const pinned = source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3c).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3c';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3c';") .replace(/\}, \{\n {2}from: '3c',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'); assert.notEqual(pinned, source); assert.ok(!pinned.includes('3c-3d'), 'the 3c engine has no 3c→3d step'); @@ -638,7 +638,7 @@ test('REQ-ROLE-2, REQ-COS-1: the 3c→3d migration lets the chief of staff end a assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); t.after(() => store.close()); - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}'); assert.equal(store.get("SELECT holder_run FROM role_claims WHERE role='coder'").holder_run, 'coder-run'); const b = new Broker({ store, businesses: demo(), organization: null }); diff --git a/packages/bus/tests/comment-migration.test.mjs b/packages/bus/tests/comment-migration.test.mjs index 1d10ecc6e..85b42a244 100644 --- a/packages/bus/tests/comment-migration.test.mjs +++ b/packages/bus/tests/comment-migration.test.mjs @@ -33,7 +33,7 @@ test('explicit 3b copy migration preserves evidence and opens as 3g; normal star const original = readFileSync(source); assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); assert.equal(store.get("SELECT body FROM events WHERE id='preserved'").body, '{"fixture":true}'); assert.equal(store.get('SELECT count(*) n FROM task_comment_cursors').n, 0); const epoch = store.get("SELECT value FROM meta WHERE key='comment_cursor_epoch'").value; @@ -75,15 +75,15 @@ test('named migration registry chains 3b through 3g to a fixture-only future ver const sha = createHash('sha256').update(futureDDL).digest('hex'); let source = readFileSync(new URL('../src/store.mjs', import.meta.url), 'utf8'); source = source.replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${sha}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3p';") - .replace('}];\nconst timedTables', `}, { from: '3o', to: '3p', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`); + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3q';") + .replace('}];\nconst timedTables', `}, { from: '3p', to: '3q', sql: ${JSON.stringify(futureSQL)} }];\nconst timedTables`); writeFileSync(join(moduleRoot, 'src/store.mjs'), source); writeFileSync(join(moduleRoot, 'schema.sql'), futureDDL); - for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql', '3j-3k.sql', '3k-3l.sql', '3l-3m.sql', '3m-3n.sql', '3n-3o.sql']) + for (const name of ['3b.sql', '3b-3c.sql', '3c-3d.sql', '3d-3e.sql', '3e-3f.sql', '3f-3g.sql', '3g-3h.sql', '3h-3i.sql', '3i-3j.sql', '3j-3k.sql', '3k-3l.sql', '3l-3m.sql', '3m-3n.sql', '3n-3o.sql', '3o-3p.sql']) copyFileSync(new URL('../src/migrations/' + name, import.meta.url), join(moduleRoot, 'src/migrations', name)); const { Store: FutureStore } = await import(pathToFileURL(join(moduleRoot, 'src/store.mjs'))); const future = new FutureStore(root, { migrateSchema: true }); - assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); + assert.equal(future.get("SELECT value FROM meta WHERE key='schema_version'").value, '3q'); assert.equal(future.get("SELECT name FROM sqlite_master WHERE name='future_fixture'").name, 'future_fixture'); assert.equal(future.get("SELECT count(*) n FROM task_comment_cursors").n, 0); assert.equal(future.get("SELECT id FROM events").id, 'preserved'); @@ -109,7 +109,7 @@ test('offline command migrates only the supplied fixture root and reports usage const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname; const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' }); assert.equal(migrated.status, 0, migrated.stderr); - assert.match(migrated.stdout, /bus schema 3o ready/); + assert.match(migrated.stdout, /bus schema 3p ready/); new Store(root).close(); const usage = spawnSync(process.execPath, [command], { encoding: 'utf8' }); assert.equal(usage.status, 2); diff --git a/packages/bus/tests/gitea-migration.test.mjs b/packages/bus/tests/gitea-migration.test.mjs index ea7b55718..f54a3b553 100644 --- a/packages/bus/tests/gitea-migration.test.mjs +++ b/packages/bus/tests/gitea-migration.test.mjs @@ -79,6 +79,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) { copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql')); copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql')); copyFileSync(here('../src/migrations/3n-3o.sql'), join(moduleRoot, 'src/migrations/3n-3o.sql')); + copyFileSync(here('../src/migrations/3o-3p.sql'), join(moduleRoot, 'src/migrations/3o-3p.sql')); return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store; } function fresh(t) { @@ -100,7 +101,7 @@ test('3j copy migrates to 3k: every row kept, and sqlite_master gains only the t const { root, path } = source3j(t), before = rows(path), was = master(path); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(migrated3l(rows(path)), before); const now = master(path); assert.deepEqual(now.filter(r => !NEW.includes(r.name) && !later3l(r)), was.filter(r => !later3l(r))); @@ -126,7 +127,7 @@ test('a 3j→3k step that fails partway leaves the store at 3j with every row an assert.deepEqual(master(path), was); assert.deepEqual(rows(path), before); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); }); test('the 3j engine refuses a migrated store, migrating or not, and leaves it as it was', async t => { @@ -137,14 +138,14 @@ test('the 3j engine refuses a migrated store, migrating or not, and leaves it as schema: ddl3j, store: (source) => source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3j).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3j';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3j';") .replace(/\}, \{\n {2}\/\/ Row 236 \(#1742[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'), }); const pinned = readFileSync(join(root, 'engine-3j/src/store.mjs'), 'utf8'); assert.ok(pinned.includes("export const SCHEMA_VERSION = '3j';") && !pinned.includes('3j-3k'), 'the 3j engine has no 3j→3k step'); assert.throws(() => new Store3j(root), /schema-mismatch/); assert.throws(() => new Store3j(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(rows(path), after); }); diff --git a/packages/bus/tests/grant-migration.test.mjs b/packages/bus/tests/grant-migration.test.mjs index 0680bc8ee..2cea63494 100644 --- a/packages/bus/tests/grant-migration.test.mjs +++ b/packages/bus/tests/grant-migration.test.mjs @@ -82,6 +82,7 @@ async function engine(root, name, { store, schema, step = (s) => s }) { copyFileSync(here('../src/migrations/3l-3m.sql'), join(moduleRoot, 'src/migrations/3l-3m.sql')); copyFileSync(here('../src/migrations/3m-3n.sql'), join(moduleRoot, 'src/migrations/3m-3n.sql')); copyFileSync(here('../src/migrations/3n-3o.sql'), join(moduleRoot, 'src/migrations/3n-3o.sql')); + copyFileSync(here('../src/migrations/3o-3p.sql'), join(moduleRoot, 'src/migrations/3o-3p.sql')); return (await import(pathToFileURL(join(moduleRoot, 'src/store.mjs')))).Store; } function fresh(t) { @@ -105,7 +106,7 @@ test('3i copy migrates to 3j: every row kept, and sqlite_master gains only the t const { root, path } = source3i(t), before = rows(path), was = master(path); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(migrated3l(rows(path)), before); const now = master(path).filter(r => !LATER.includes(r.name)); assert.deepEqual(now.filter(r => !NEW.includes(r.name) && !later3l(r)), was.filter(r => !later3l(r))); @@ -130,7 +131,7 @@ test('a 3i→3j step that fails partway leaves the store at 3i with every row an assert.deepEqual(master(path), was); assert.deepEqual(rows(path), before); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); }); test('the 3i engine refuses a migrated store, migrating or not, and leaves it as it was', async t => { @@ -141,14 +142,14 @@ test('the 3i engine refuses a migrated store, migrating or not, and leaves it as schema: ddl3i, store: (source) => source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3i).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3i';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3i';") .replace(/\}, \{\n {2}\/\/ Row 234 \(#1740\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'), }); const pinned = readFileSync(join(root, 'engine-3i/src/store.mjs'), 'utf8'); assert.ok(pinned.includes("export const SCHEMA_VERSION = '3i';") && !pinned.includes('3i-3j'), 'the 3i engine has no 3i→3j step'); assert.throws(() => new Store3i(root), /schema-mismatch/); assert.throws(() => new Store3i(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(rows(path), after); }); diff --git a/packages/bus/tests/name-migration.test.mjs b/packages/bus/tests/name-migration.test.mjs index 921c4722d..cf7267683 100644 --- a/packages/bus/tests/name-migration.test.mjs +++ b/packages/bus/tests/name-migration.test.mjs @@ -65,7 +65,7 @@ test('3f copy migrates to 3g preserving every old column/row, with an empty inst assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); try { - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null }); assert.equal(store.get('SELECT count(*) AS n FROM instance_names').n, 0); assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: 'docs/spec.md', spec_commit: 'c'.repeat(40) }); diff --git a/packages/bus/tests/request-bootstrap.test.mjs b/packages/bus/tests/request-bootstrap.test.mjs new file mode 100644 index 000000000..97a46069d --- /dev/null +++ b/packages/bus/tests/request-bootstrap.test.mjs @@ -0,0 +1,203 @@ +// Lifecycle Task 2 (#797 / Gitea #1839): a confirmed new-project selection becomes a usable project. +// Real startBroker over a real organization file, a real git factory repo and a fixture data root. +// The test barrier stands in for a kill: it throws after a step's effect and before its receipt. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { execFileSync } from 'node:child_process'; +import { existsSync, lstatSync, mkdirSync, readFileSync, readdirSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { startBroker } from '../src/runtime.mjs'; +import { prepareBootstrap } from '../src/request-bootstrap.mjs'; +import { bootConfig, loadSystem } from '../../cli/src/config.mjs'; +import { fixture, tmp } from '../../cli/tests/helpers.mjs'; +import { organizationWriter } from '../../business/src/organization.mjs'; +import { loadBusiness } from '../../business/src/business.mjs'; +import { workingRepo } from '../../seat/src/working.mjs'; + +const INTENT = 'Build a single-file timer.\nKeep these exact bytes.\n'; +const GIT_ENV = { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null', GIT_AUTHOR_NAME: 'Factory', GIT_AUTHOR_EMAIL: 'f@example.test', + GIT_COMMITTER_NAME: 'Factory', GIT_COMMITTER_EMAIL: 'f@example.test' }; +const git = (cwd, ...args) => execFileSync('git', args, { cwd, env: GIT_ENV, encoding: 'utf8' }).trim(); +const PROJECT_FILE = { projectVersion: 1, id: 'stack', vars: { 'tracker.project': 1, 'git.workingBranch': 'main' } }; + +// The factory: the declared `stack` project is a git repository with a project file. +function factory(root) { + const dir = join(root, 'project'); + mkdirSync(join(dir, '.mosaic'), { recursive: true }); + writeFileSync(join(dir, '.mosaic/project.json'), JSON.stringify(PROJECT_FILE, null, 2) + '\n'); + writeFileSync(join(dir, 'README.md'), 'factory\n'); + git(dir, 'init', '-q', '-b', 'main'); git(dir, 'add', '-A'); git(dir, 'commit', '-q', '-m', 'factory'); + return dir; +} + +async function setup(t, { barrier } = {}) { + const root = tmp(t, 'bootstrap-'), f = fixture(root, 'demo'), source = factory(root); + const hooks = { barrier }; + const system = loadSystem({ env: f.env }); + const writer = organizationWriter({ dir: join(root, 'config'), rolesDir: f.roles }); + const plans = (dataRoot, ready = () => true) => ({ + preparePlans: (state, args, from) => Promise.all(['autonomous', 'guided'].map(mode => prepareBootstrap({ + business: writer.read('demo').document, request: state.request, sourceProject: args.source_project, project: args.project, name: args.name, + mode, dataRoot, organizationRevision: writer.read('demo').revision, tracker: { root: 1, project: args.tracker_project }, + staff: { pm: state.pm, technical_reviewer: 'cto', coder: 'coder', reviewer: 'reviewer', chief: 'chief' }, + intent: { text: from.text, source_message: from.source_message } }))), + admitPlan: plan => plan.organization_revision === writer.read('demo').revision && ready() ? { ready: true } : { ready: false, code: 'organization-stale' }, + barrier: step => hooks.barrier?.(step), + }); + const boot = async () => { + const warnings = []; + const { trackers, ...config } = JSON.parse(JSON.stringify(bootConfig({ system, businessId: 'demo', env: f.env, warn: w => warnings.push(w) }))); + const bus = await startBroker({ ...config, requestServices: plans(f.dataRoot) }); + t.after(() => bus.close().catch(() => {})); + return { bus, config, trackers, warnings }; + }; + const first = await boot(); + const caps = { human: first.bus.broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true }) }; + for (const role of ['pm', 'cto', 'coder']) { + caps[role] = first.bus.broker.bindLaunch({ business: 'demo', role, run: role + '-run', harness: 'pi' }); + first.bus.broker.request(caps[role], { verb: 'role.claim' }); + } + const call = (who, verb, args = {}) => first.bus.broker.request(caps[who], { verb, args }); + const sent = call('human', 'message.send', { to: 'pm', body: INTENT }); + const opened = await call('pm', 'request.open', { request: sent.request }); + const proposed = await call('pm', 'request.propose', { request: sent.request, revision: opened.revision, op: 'plan-1', source_project: 'stack', + project: 'timer', name: 'Timer', tracker_project: 2 }); + const choose = async (choice = 'guided') => { + call('human', 'decision.resolve', { id: proposed.selection.decision, choice }); + return first.bus.requestLifecycle.resume('demo', sent.request); + }; + const paths = { code: join(f.dataRoot, 'products/demo/timer/code'), records: join(f.dataRoot, 'projects/demo/timer'), product: join(f.dataRoot, 'products/demo/timer') }; + const changes = () => { try { return readFileSync(writer.logFile('demo'), 'utf8').trim().split('\n').map(l => JSON.parse(l)); } catch { return []; } }; + const additions = () => changes().filter(c => c.state === 'committed' && c.detail?.verb === 'project.add'); + return { root, f, system, source, writer, hooks, boot, bus: first.bus, first, call, caps, request: sent.request, proposed, choose, paths, changes, additions, + rows: () => first.bus.broker.request(caps.human, { verb: 'project.list', args: {} }) }; +} + +test('bootstrap_connects_all_resources', async t => { + const f = await setup(t), factoryHead = git(f.source, 'rev-parse', 'HEAD'), orgBefore = f.writer.read('demo'); + const done = await f.choose('guided'); + assert.equal(done.phase, 'discovery'); assert.equal(done.project, 'timer'); assert.equal(done.mode, 'guided'); + assert.equal(done.wait.code, 'request-discovery-unavailable'); + // The organization file: exactly one added mapping, everything else byte-for-byte the same document. + const org = f.writer.read('demo'); + assert.deepEqual(org.document.projects.timer, { root: f.paths.code, repo: f.paths.records }); + assert.deepEqual(org.document.projects.stack, orgBefore.document.projects.stack); + const { projects: a, ...restNow } = org.document, { projects: b, ...restBefore } = orgBefore.document; + assert.deepEqual(restNow, restBefore); + assert.equal(f.additions().length, 1); + assert.equal(f.additions()[0].detail.plan_digest, done.plan_digest); + assert.equal(f.additions()[0].after, org.revision); + // The code checkout: the factory commit plus one seed commit that writes only the project file. + assert.equal(git(f.paths.code, 'rev-parse', 'HEAD~1'), factoryHead); + assert.equal(git(f.paths.code, 'diff', '--name-status', 'HEAD~1', 'HEAD'), 'M\t.mosaic/project.json'); + assert.deepEqual(JSON.parse(readFileSync(join(f.paths.code, '.mosaic/project.json'), 'utf8')), { projectVersion: 1, id: 'timer', vars: { 'git.workingBranch': 'main' } }); + assert.equal(git(f.paths.code, 'remote'), ''); + assert.equal(git(f.paths.code, 'status', '--porcelain'), ''); + // The records repository: layout commit, then the original intent as the owner's edit. + assert.equal(git(f.paths.records, 'rev-list', '--count', 'HEAD'), '2'); + assert.equal(git(f.paths.records, 'log', '-1', '--format=%an|%s'), 'jason|Edit spec/north-star.md'); + assert.equal(git(f.paths.records, 'rev-parse', 'HEAD'), done.spec_commit); + assert.equal(readFileSync(join(f.paths.records, 'spec/north-star.md'), 'utf8'), INTENT); + // The chat association and the working repository. + const chat = f.rows().find(p => p.name === 'Timer'); + assert.equal(chat.repo, 'timer'); assert.equal(chat.id, done.chat_project); + assert.equal(f.rows().filter(p => p.name === 'Timer').length, 1); + assert.equal(git(workingRepo(f.f.dataRoot, 'demo', 'timer'), 'rev-parse', 'refs/heads/main'), git(f.paths.code, 'rev-parse', 'HEAD')); + // The single tracker root stays; the selected board lives in the immutable plan and the admission projection. + const plan = done.selection.bindings.find(x => x.digest === done.plan_digest).plan; + assert.deepEqual(plan.tracker, { root: 1, project: 2 }); + assert.deepEqual(await f.first.bus.admission({ business: 'demo', project: 'timer' }), { ready: true, revision: org.revision, code: null }); + // The factory is untouched and the broker serves the new project without a restart. + assert.equal(git(f.source, 'rev-parse', 'HEAD'), factoryHead); assert.equal(git(f.source, 'status', '--porcelain'), ''); + assert.ok(f.call('human', 'project.create', { name: 'Side', repo: 'timer' }).id); + // The next launch: a fresh boot lists the project, verifies it and still names the one tracker project. + await f.first.bus.close(); + const next = await f.boot(); + assert.deepEqual(next.config.businesses.demo.projects.timer, { root: f.paths.code, repo: f.paths.records }); + assert.equal(next.trackers.demo.project, 1); + assert.deepEqual(next.warnings, []); + assert.equal(loadBusiness('demo', { dir: join(f.root, 'config'), rolesDir: f.f.roles }).projects.timer.root, f.paths.code); +}); + +test('a_product_is_not_admitted_until_every_step_passes', async t => { + let seen; + const f = await setup(t, { barrier: async step => { if (step === 'intent') seen = { admission: await f.first.bus.admission({ business: 'demo', project: 'timer' }) }; } }); + await f.choose(); + assert.equal(seen.admission.ready, false); assert.equal(typeof seen.admission.code, 'string'); + assert.equal((await f.first.bus.admission({ business: 'demo', project: 'timer' })).ready, true); + assert.deepEqual(await f.first.bus.admission({ business: 'demo', project: 'stack' }), { ready: true, revision: f.writer.read('demo').revision, code: null }); + assert.deepEqual(await f.first.bus.admission({ business: 'demo', project: 'absent' }), { ready: false, revision: null, code: 'project-unknown' }); + assert.deepEqual(await f.first.bus.admission({ business: 'other', project: 'timer' }), { ready: false, revision: null, code: 'project-unknown' }); +}); + +// Lifecycle #800: an association or organization change can land after the last awaited read. +function staleCase(name, change, { keep } = {}) { + test(name, async t => { + let armed = true; + const f = await setup(t, { barrier: step => { if (armed && step === 'begin') { armed = false; change(f); } } }); + const orgBefore = readFileSync(join(f.root, 'config/businesses/demo.json')); + const out = await f.choose(); + assert.equal(out.wait.code, 'request-plan-stale'); assert.equal(out.phase, 'selection'); assert.equal(out.project, null); + assert.equal(out.plan_digest, null); assert.equal(out.selection, null); + assert.equal(f.additions().length, 0); + assert.equal(existsSync(f.paths.records), false); + assert.equal(existsSync(join(f.paths.code, '.mosaic')), false); + keep?.(f, orgBefore); + // Nothing is retained that a fresh preview could trip over, and a re-run does not resume the old plan. + assert.equal((await f.first.bus.requestLifecycle.resume('demo', f.request)).wait.code, 'request-plan-stale'); + }); +} +staleCase('an_organization_change_after_confirmation_waits_without_effect', f => { + const now = f.writer.read('demo'), next = structuredClone(now.document); next.vars['tracker.pollSeconds'] = 61; + f.writer.write('demo', next, { revision: now.revision, actor: 'jason' }); +}, { keep: f => assert.equal(existsSync(f.paths.product), false) }); +staleCase('a_changed_source_commit_waits_without_effect', f => { + writeFileSync(join(f.source, 'README.md'), 'changed\n'); git(f.source, 'commit', '-qam', 'moved'); +}, { keep: f => assert.equal(existsSync(f.paths.product), false) }); +staleCase('preexisting_unrelated_data_is_left_alone_and_the_plan_waits', f => { + mkdirSync(f.paths.code, { recursive: true }); writeFileSync(join(f.paths.code, 'keep.txt'), 'mine\n'); +}, { keep: f => assert.equal(readFileSync(join(f.paths.code, 'keep.txt'), 'utf8'), 'mine\n') }); +staleCase('a_symlink_ancestor_is_refused_and_its_target_is_untouched', f => { + const outside = join(f.root, 'outside'); mkdirSync(outside); symlinkSync(outside, join(f.f.dataRoot, 'products')); +}, { keep: f => { assert.deepEqual(readdirSync(join(f.root, 'outside')), []); assert.ok(lstatSync(join(f.f.dataRoot, 'products')).isSymbolicLink()); } }); +staleCase('a_path_overlap_with_a_declared_project_waits_without_effect', f => { + const now = f.writer.read('demo'), next = structuredClone(now.document); + next.projects.stack = { ...next.projects.stack, repo: join(f.paths.code, 'inner') }; + writeFileSync(join(f.root, 'config/businesses/demo.json'), JSON.stringify(next, null, 2) + '\n'); +}, { keep: f => assert.equal(existsSync(f.paths.product), false) }); + +// Kill (the barrier throws after the effect and before its receipt), reopen, recover exactly once. +for (const step of ['mapping', 'intent']) { + test(`kill_after_${step}_then_reopen_recovers_the_same_project_exactly_once`, async t => { + let armed = true; + const f = await setup(t, { barrier: s => { if (armed && s === step) { armed = false; throw new Error('killed'); } } }); + const factoryHead = git(f.source, 'rev-parse', 'HEAD'); + const stopped = await f.choose(); + assert.equal(stopped.phase, 'bootstrap'); assert.equal(stopped.wait.code, 'bootstrap-uncertain'); + assert.equal(f.writer.read('demo').document.projects.timer !== undefined, true); + assert.equal((await f.first.bus.admission({ business: 'demo', project: 'timer' })).ready, false); + const mapped = f.writer.read('demo').revision; + await f.first.bus.close(); + const next = await f.boot(); + const human = next.bus.broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true }); + const done = await next.bus.broker.request(human, { verb: 'request.get', args: { request: f.request } }); + assert.equal(done.phase, 'discovery'); + if (step === 'mapping') assert.equal(f.writer.read('demo').revision, mapped, 'recovery adds no second mapping'); + assert.equal(f.additions().length, 1); + assert.equal(git(f.paths.records, 'rev-list', '--count', 'HEAD'), '2'); + assert.equal(git(f.paths.records, 'rev-parse', 'HEAD'), done.spec_commit); + assert.equal(git(f.paths.code, 'rev-list', '--count', 'HEAD'), '2'); + assert.equal(git(f.paths.code, 'rev-parse', 'HEAD~1'), factoryHead); + assert.equal(git(f.source, 'rev-parse', 'HEAD'), factoryHead); assert.equal(git(f.source, 'status', '--porcelain'), ''); + assert.equal(next.bus.broker.request(human, { verb: 'project.list', args: {} }).filter(p => p.name === 'Timer').length, 1); + assert.equal((await next.bus.admission({ business: 'demo', project: 'timer' })).ready, true); + assert.equal(next.warnings.length, 0); + }); +} + +test('ordinary_organization_writers_still_refuse_every_project_change', async t => { + const f = await setup(t), now = f.writer.read('demo'), next = structuredClone(now.document); + next.projects.timer = { root: f.paths.code, repo: f.paths.records }; + assert.throws(() => f.writer.write('demo', next, { revision: now.revision, actor: 'jason' }), { code: 'organization-projects-protected' }); + assert.equal(f.writer.read('demo').revision, now.revision); +}); diff --git a/packages/bus/tests/request-intake.test.mjs b/packages/bus/tests/request-intake.test.mjs index 7342b6d34..0fc2a0f10 100644 --- a/packages/bus/tests/request-intake.test.mjs +++ b/packages/bus/tests/request-intake.test.mjs @@ -250,7 +250,8 @@ test('plan_preparation_rejects_unknown_keys_changed_intent_tracker_modes_and_sym test('3n_to_3o_is_explicit_offline_preserves_rows_and_old_engine_refuses_new_schema', t => { const root = mkdtempSync(join(tmpdir(), 'request-migration-')); t.after(() => rmSync(root, { recursive: true, force: true })); mkdirSync(join(root, 'bus'), { mode: 0o700 }); - const path = join(root, 'bus/bus.sqlite'), currentDDL = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8'); + // Task 2's 3o→3p step widens only request_events.kind, so the 3o schema is the current one with the old list. + const path = join(root, 'bus/bus.sqlite'), currentDDL = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8').replace("'setup.wait','bootstrap.stale','bootstrap.completed','cancelled'", "'setup.wait','cancelled'"); const newDDL = readFileSync(new URL('../src/migrations/3n-3o.sql', import.meta.url), 'utf8'); assert.ok(currentDDL.endsWith(newDDL)); const oldDDL = currentDDL.slice(0, -newDDL.length); const { schemaDigest, SCHEMA_VERSION } = awaitImportStore; @@ -262,7 +263,7 @@ test('3n_to_3o_is_explicit_offline_preserves_rows_and_old_engine_refuses_new_sch db.close(); chmodSync(path, 0o600); const before = readFileSync(path); assert.throws(() => new Store(root), /schema-mismatch/); assert.deepEqual(readFileSync(path), before); const cli = spawnSync(process.execPath, [new URL('../src/migrate-schema.mjs', import.meta.url).pathname, root], { encoding: 'utf8' }); - assert.equal(cli.status, 0, cli.stderr); assert.equal(cli.stdout, 'bus schema 3o ready\n'); assert.equal(SCHEMA_VERSION, '3o'); + assert.equal(cli.status, 0, cli.stderr); assert.equal(cli.stdout, 'bus schema 3p ready\n'); assert.equal(SCHEMA_VERSION, '3p'); const store = new Store(root); assert.equal(store.get("SELECT id FROM events").id, 'preserved'); for (const table of ['request_events','request_effects','request_snapshots','request_snapshot_events','request_project_events']) assert.equal(store.get(`SELECT count(*) n FROM ${table}`).n, 0); store.close(); assert.equal(spawnSync(process.execPath, [new URL('../src/migrate-schema.mjs', import.meta.url).pathname, root]).status, 0); @@ -278,6 +279,54 @@ test('3n_to_3o_is_explicit_offline_preserves_rows_and_old_engine_refuses_new_sch }); const awaitImportStore = await import('../src/store.mjs'); +test('3o_to_3p_widens_only_the_event_kinds_and_keeps_rows_seq_and_guards', t => { + const root = mkdtempSync(join(tmpdir(), 'request-migration-3p-')); + t.after(() => rmSync(root, { recursive: true, force: true })); mkdirSync(join(root, 'bus'), { mode: 0o700 }); + const path = join(root, 'bus/bus.sqlite'); + // The 3o schema is the current one with the narrower kind list. + const ddl3o = readFileSync(new URL('../schema.sql', import.meta.url), 'utf8').replace("'setup.wait','bootstrap.stale','bootstrap.completed','cancelled'", "'setup.wait','cancelled'"); + const { schemaDigest } = awaitImportStore, AT = '2026-11-03T00:00:00.000Z'; + const db = new DatabaseSync(path); db.exec(ddl3o); + db.prepare('INSERT INTO meta VALUES(?,?)').run('schema_version', '3o'); + db.prepare('INSERT INTO meta VALUES(?,?)').run('schema_digest', schemaDigest(db)); + db.prepare('INSERT INTO meta VALUES(?,?)').run('comment_cursor_epoch', AT); + db.prepare("INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,body) VALUES('m1',?,'demo','human','human','pm','info','build a timer')").run(AT); + db.prepare("INSERT INTO events(id,at,business,kind,subject,body) VALUES('r1',?,'demo','human.input','m1',?)").run(AT, JSON.stringify({ message: 'm1', kind: 'instruction' })); + const add = (revision, kind) => db.prepare("INSERT INTO request_events(business,request,revision,kind,at,body) VALUES('demo','r1',?,?,?,'{}')").run(revision, kind, AT); + add(1, 'opened'); add(2, 'setup.wait'); + assert.throws(() => add(3, 'bootstrap.stale'), /CHECK constraint failed/, 'the 3o schema refuses the new kind'); + db.prepare("INSERT INTO request_effects(at,version,business,request,op,kind,digest,stage) VALUES(?,1,'demo','r1','bootstrap:x:reserve','bootstrap',?,'prepared')").run(AT, 'a'.repeat(64)); + const before = db.prepare('SELECT * FROM request_events ORDER BY seq').all(), effects = db.prepare('SELECT * FROM request_effects ORDER BY seq').all(); + db.close(); chmodSync(path, 0o600); + assert.throws(() => new Store(root), /schema-mismatch/); + const cli = spawnSync(process.execPath, [new URL('../src/migrate-schema.mjs', import.meta.url).pathname, root], { encoding: 'utf8' }); + assert.equal(cli.status, 0, cli.stderr); assert.equal(cli.stdout, 'bus schema 3p ready\n'); + const store = new Store(root); + assert.deepEqual(store.all('SELECT * FROM request_events ORDER BY seq'), before, 'every row keeps its seq and bytes'); + assert.deepEqual(store.all('SELECT * FROM request_effects ORDER BY seq'), effects); + const next = (revision, kind) => store.run("INSERT INTO request_events(business,request,revision,kind,at,body) VALUES('demo','r1',?,?,?,'{}')", revision, kind, AT); + next(3, 'bootstrap.stale'); + assert.equal(store.get('SELECT max(seq) n FROM request_events').n, 3, 'the sequence carries on from the old table'); + assert.throws(() => next(3, 'bootstrap.completed'), /append-only/, 'a replayed revision is refused'); + assert.throws(() => next(5, 'selection.proposed'), /request-revision-conflict/, 'the revision guard is back'); + assert.throws(() => store.run("INSERT INTO request_events(business,request,revision,kind,at,body) VALUES('demo','nope',1,'opened',?,'{}')", AT), /request-origin-refused/, 'the origin guard is back'); + assert.throws(() => store.run("INSERT INTO request_events(business,request,revision,kind,at,body) VALUES('demo','r1',4,'surprise',?,'{}')", AT), /CHECK constraint failed/); + assert.throws(() => store.run("UPDATE request_events SET kind='opened' WHERE seq=1"), /append-only/); + assert.throws(() => store.run('DELETE FROM request_events'), /append-only/); + assert.throws(() => store.run("INSERT INTO request_effects(at,version,business,request,op,kind,digest,stage) VALUES(?,1,'demo','r1','bootstrap:x:reserve','delivery',?,'prepared')", AT, 'a'.repeat(64)), /request-effect-conflict/, 'the effect guard is back'); + store.close(); + assert.equal(spawnSync(process.execPath, [new URL('../src/migrate-schema.mjs', import.meta.url).pathname, root]).status, 0, 'a second run is a no-op'); + // The Task 1 engine (3o) refuses the migrated fixture and does not change it. + const migrated = readFileSync(path), moduleRoot = join(root, 'old-engine'); mkdirSync(join(moduleRoot, 'src/migrations'), { recursive: true }); + writeFileSync(join(moduleRoot, 'schema.sql'), ddl3o); + const oldSource = spawnSync('git', ['show', '138171ca0:packages/bus/src/store.mjs'], { encoding: 'utf8' }); + assert.equal(oldSource.status, 0); writeFileSync(join(moduleRoot, 'src/store.mjs'), oldSource.stdout); + for (const m of ['3b','3b-3c','3c-3d','3d-3e','3e-3f','3f-3g','3g-3h','3h-3i','3i-3j','3j-3k','3k-3l','3l-3m','3m-3n','3n-3o']) + writeFileSync(join(moduleRoot, 'src/migrations', m + '.sql'), readFileSync(new URL('../src/migrations/' + m + '.sql', import.meta.url))); + const old = spawnSync(process.execPath, ['--input-type=module', '-e', `import {Store} from ${JSON.stringify(join(moduleRoot, 'src/store.mjs'))}; try { new Store(${JSON.stringify(root)}); process.exit(9); } catch(e) { if(e.message !== 'schema-mismatch') throw e; }`], { encoding: 'utf8' }); + assert.equal(old.status, 0, old.stderr); assert.deepEqual(readFileSync(path), migrated); +}); + test('real_runtime_socket_loads_host_lifecycle_and_recovery_keeps_intake_without_capabilities', async t => { const root = mkdtempSync(join(tmpdir(), 'request-runtime-')); let runtime; diff --git a/packages/bus/tests/session-clear.test.mjs b/packages/bus/tests/session-clear.test.mjs index 34ca4dbe4..edd519096 100644 --- a/packages/bus/tests/session-clear.test.mjs +++ b/packages/bus/tests/session-clear.test.mjs @@ -530,7 +530,7 @@ test('T4 schema: a 3m store migrates to 3n, session_ops and message_cancels are const before = tables(path); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); const after = tables(path); for (const name of ['session_ops', 'message_cancels', 'request_events', 'request_effects', 'request_snapshots', 'request_snapshot_events', 'request_project_events']) { assert.deepEqual(after[name], [], name); @@ -571,7 +571,7 @@ async function engine3m(root) { const source = readFileSync(here('../src/store.mjs'), 'utf8'); const edited = source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3m).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3m';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3m';") .replace(/\}, \{\n {2}\/\/ Row 271 \(#1776\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'); assert.ok(edited.includes("export const SCHEMA_VERSION = '3m';") && edited.includes('3l-3m') && !edited.includes('3m-3n'), 'the 3m engine has no 3m→3n step'); writeFileSync(join(moduleRoot, 'src/store.mjs'), edited); @@ -588,7 +588,7 @@ test('T4 schema: the 3m engine refuses a 3n store, migrating or not, and leaves const Store3m = await engine3m(root); assert.throws(() => new Store3m(root), /schema-mismatch/); assert.throws(() => new Store3m(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(tables(path), before); // The same engine opens a fresh 3m store, so the refusal is the version's and not a broken copy. diff --git a/packages/bus/tests/sessions.test.mjs b/packages/bus/tests/sessions.test.mjs index 563723535..c9030b402 100644 --- a/packages/bus/tests/sessions.test.mjs +++ b/packages/bus/tests/sessions.test.mjs @@ -374,7 +374,7 @@ test('T7: a 3k store migrates to 3l, each thread\'s default session has the thre const { root, path } = source3k(t); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); // The migrated schema is the fresh one, object for object. const freshRoot = mkdtempSync(join(tmpdir(), 'sessions-fresh-')); t.after(() => rmSync(freshRoot, { recursive: true, force: true })); @@ -430,7 +430,7 @@ async function engine3k(root) { const source = readFileSync(here('../src/store.mjs'), 'utf8'); const edited = source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3k).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3k';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3k';") .replace(/\}, \{\n {2}\/\/ Row 264a \(#1769\)[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'); assert.ok(edited.includes("export const SCHEMA_VERSION = '3k';") && edited.includes('3j-3k') && !edited.includes('3k-3l'), 'the 3k engine has no 3k→3l step'); writeFileSync(join(moduleRoot, 'src/store.mjs'), edited); @@ -447,7 +447,7 @@ test('T7 D9: the 3k engine refuses a 3l store, migrating or not, and leaves it a const Store3k = await engine3k(root); assert.throws(() => new Store3k(root), /schema-mismatch/); assert.throws(() => new Store3k(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(rows(path), before); // The same engine opens a fresh 3k store, so the refusal is the version's and not a broken copy. diff --git a/packages/bus/tests/spec-migration.test.mjs b/packages/bus/tests/spec-migration.test.mjs index 9967f7783..7d4de1f00 100644 --- a/packages/bus/tests/spec-migration.test.mjs +++ b/packages/bus/tests/spec-migration.test.mjs @@ -63,7 +63,7 @@ test('3e copy migrates through 3f to 3g preserving every old column/row and addi assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); try { - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); assert.deepEqual({ ...store.get('SELECT spec_path,spec_commit FROM decisions') }, { spec_path: null, spec_commit: null }); assert.equal(store.get('PRAGMA quick_check').quick_check, 'ok'); assert.deepEqual(store.all('PRAGMA foreign_key_check'), []); diff --git a/packages/bus/tests/thread-migration.test.mjs b/packages/bus/tests/thread-migration.test.mjs index d0b35794a..95096fef6 100644 --- a/packages/bus/tests/thread-migration.test.mjs +++ b/packages/bus/tests/thread-migration.test.mjs @@ -49,7 +49,7 @@ async function database3d(t) { const source = readFileSync(here('../src/store.mjs'), 'utf8'); const pinned = source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3d).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3d';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3d';") .replace(/\}, \{\n {2}from: '3d',[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'); assert.ok(!pinned.includes('3d-3e'), 'the 3d engine has no 3d→3e step'); writeFileSync(join(moduleRoot, 'src/store.mjs'), pinned); @@ -101,7 +101,7 @@ test('REQ-THREAD-1 REQ-ORG-2: the 3d→3e migration adds the thread tables and l assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); t.after(() => store.close()); - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); for (const table of ['projects', 'threads', 'thread_claims', 'message_threads', 'thread_opens']) assert.equal(store.get(`SELECT count(*) n FROM ${table}`).n, 0, table); // Parked: the undelivered ones are counted per business and role, and none is changed or given a thread. @@ -161,5 +161,5 @@ test('REQ-THREAD-1: the offline command prints the kept-message counts before th const command = new URL('../src/migrate-schema.mjs', import.meta.url).pathname; const migrated = spawnSync(process.execPath, [command, root], { encoding: 'utf8' }); assert.equal(migrated.status, 0, migrated.stderr); - assert.equal(migrated.stdout, [...REPORT, 'bus schema 3o ready', ''].join('\n')); + assert.equal(migrated.stdout, [...REPORT, 'bus schema 3p ready', ''].join('\n')); }); diff --git a/packages/bus/tests/thread-shares.test.mjs b/packages/bus/tests/thread-shares.test.mjs index 769a1a19e..0afe46120 100644 --- a/packages/bus/tests/thread-shares.test.mjs +++ b/packages/bus/tests/thread-shares.test.mjs @@ -403,7 +403,7 @@ test('a 3l store migrates to 3m: thread_shares is new and empty, and every other const before = tables(path); assert.throws(() => new Store(root), /schema-mismatch/); new Store(root, { migrateSchema: true }).close(); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); const after = tables(path); assert.deepEqual(after.thread_shares, []); delete after.thread_shares; @@ -439,7 +439,7 @@ async function engine3l(root) { const source = readFileSync(here('../src/store.mjs'), 'utf8'); const edited = source .replace(/export const SCHEMA_SHA256 = '[a-f0-9]+';/, `export const SCHEMA_SHA256 = '${createHash('sha256').update(ddl3l).digest('hex')}';`) - .replace("export const SCHEMA_VERSION = '3o';", "export const SCHEMA_VERSION = '3l';") + .replace("export const SCHEMA_VERSION = '3p';", "export const SCHEMA_VERSION = '3l';") .replace(/\}, \{\n {2}\/\/ Row 206 \(193b2[\s\S]*?\}\];\nconst timedTables/, '}];\nconst timedTables'); assert.ok(edited.includes("export const SCHEMA_VERSION = '3l';") && edited.includes('3k-3l') && !edited.includes('3l-3m'), 'the 3l engine has no 3l→3m step'); writeFileSync(join(moduleRoot, 'src/store.mjs'), edited); @@ -456,7 +456,7 @@ test('the 3l engine refuses a 3m store, migrating or not, and leaves it as it wa const Store3l = await engine3l(root); assert.throws(() => new Store3l(root), /schema-mismatch/); assert.throws(() => new Store3l(root, { migrateSchema: true }), /schema-mismatch/); - assert.equal(version(path), '3o'); + assert.equal(version(path), '3p'); assert.deepEqual(master(path), was); assert.deepEqual(tables(path), before); // The same engine opens a fresh 3l store, so the refusal is the version's and not a broken copy. diff --git a/packages/bus/tests/usage-migration.test.mjs b/packages/bus/tests/usage-migration.test.mjs index 7aba25d9e..bd7801060 100644 --- a/packages/bus/tests/usage-migration.test.mjs +++ b/packages/bus/tests/usage-migration.test.mjs @@ -68,7 +68,7 @@ test('REQ-LIMIT-4 3g copy migrates through 3h, 3i, 3j and 3k to 3l preserving ev assert.throws(() => new Store(root), /schema-mismatch/); const store = new Store(root, { migrateSchema: true }); try { - assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3o'); + assert.equal(store.get("SELECT value FROM meta WHERE key='schema_version'").value, '3p'); assert.deepEqual({ ...store.get('SELECT to_name FROM messages') }, { to_name: null }); assert.equal(store.get('SELECT count(*) AS n FROM usage_records').n, 0); assert.equal(store.get('SELECT name FROM instance_names').name, 'Builder'); diff --git a/packages/business/src/organization.mjs b/packages/business/src/organization.mjs index 2acb92e41..cf11c2857 100644 --- a/packages/business/src/organization.mjs +++ b/packages/business/src/organization.mjs @@ -10,6 +10,7 @@ import { businessFilePath, validateBusinessDocument } from "./business.mjs"; import { requireId, sha256, canonicalJson } from "./util.mjs"; import { isDailyModelQuota } from "../../plugins/src/model-quota.mjs"; import { checkReplyLimits } from "../../library/src/limits.mjs"; +import { ADDITION_VERB, additionProblem } from "./project-bootstrap.mjs"; export class OrganizationError extends Error { constructor(code, message = code) { @@ -244,7 +245,10 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) { // after it returns carries revoked: true, the lock's release included (duckworth-0165 C2), and the // bus names it so the human reads that the tokens may already be revoked. beforeWrite must not // write this business itself: it would find its own lock and refuse organization-busy. - function write(id, document, { revision, actor, detail, context, beforeWrite = null }) { + // The one commit path. `guard(old, document)` is what a caller may do to `projects`: write() keeps + // them whole (C2-04 M9) and addProject() allows one exact addition. `document` may be a function of + // the old document, for a writer that builds what it publishes from what it read under the lock. + function commit(id, document, { revision, actor, detail, context, beforeWrite = null, guard }) { requireId(actor, "authenticated actor"); requireId(id, "organization id"); let revoked = false; @@ -256,10 +260,8 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) { const before = sha256(oldBytes); if (revision !== before) fail("revision-conflict", "organization revision changed; read again"); const old = JSON.parse(oldBytes); - // C2-04 M9: every organization setter preserves the whole project - // mapping, including library playgrounds and hybrid universes. - if (canonicalJson(document?.projects) !== canonicalJson(old.projects)) - fail('organization-projects-protected'); + if (typeof document === "function") document = document(old); + guard(old, document); replyLimitsFence(old, document, actor); modelQuotaFence(old, document, actor); usageFence(old, document, actor); @@ -268,8 +270,7 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) { // Check and log exactly what lands on disk, for every setter: a value JSON // can't carry (a Date, a Map) serializes as something else (row 131, sage-0314). const written = JSON.parse(bytes); - if (canonicalJson(written.projects) !== canonicalJson(old.projects)) - fail('organization-projects-protected'); + guard(old, written); replyLimitsFence(old, written, actor); modelQuotaFence(old, written, actor); usageFence(old, written, actor); @@ -311,6 +312,70 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) { } } + // C2-04 M9: every organization setter preserves the whole project mapping, including library + // playgrounds and hybrid universes. + const projectsProtected = (old, document) => { + if (canonicalJson(document?.projects) !== canonicalJson(old.projects)) fail("organization-projects-protected"); + }; + function write(id, document, options) { + return commit(id, document, { ...options, guard: projectsProtected }); + } + + // Lifecycle Task 2: the only writer that adds a project mapping, and only the exact bootstrap + // addition: one new key under the managed product layout, a receipt naming the request and plan, and + // no other change. It compares both the document it built and the bytes it will publish. A stale + // revision refuses revision-conflict: the owner confirms a fresh preview, never a silent rebase. + function addProject(id, { revision, project, entry, receipt }, actor) { + const guard = (old, document) => { + const problem = additionProblem({ old, next: document, project, entry, receipt }); + if (problem) fail("organization-project-add-refused", problem); + }; + return commit(id, old => ({ ...old, projects: { ...old.projects, [project]: entry } }), { + revision, actor, guard, + detail: { verb: ADDITION_VERB, request: receipt?.request, plan_digest: receipt?.plan_digest, project }, + }); + } + + // Runs `fn({ revision, document })` under the writer lock, after recovery, and nothing else may + // write this business meanwhile. `fn` must be synchronous: the lock is a directory held for its + // duration, and an awaited call would leave a half-checked world to whoever finds the lock gone. + function hold(id, fn) { + requireId(id, "organization id"); + return withLock(id, () => { + recover(id); + const { document, revision } = read(id); + const out = fn({ document, revision }); + if (out !== null && typeof out === "object" && typeof out.then === "function") { + out.catch?.(() => {}); + fail("organization-hold-sync", "a held check must be synchronous"); + } + return out; + }); + } + + // The committed addition of this request and plan, if there is one. A prepared receipt whose + // `after` is the file's current hash counts: the rename landed and the commit line did not. + function findAddition(id, { request, plan_digest, project }) { + let text; + try { + text = regular(logFile(id)); + } catch (e) { + if (e.code === "ENOENT") return null; + throw e; + } + if (text && !text.endsWith("\n")) fail("organization-uncertain", "torn organization receipt"); + const last = new Map(); + for (const e of text.trim() ? text.trim().split("\n").map(JSON.parse) : []) last.set(e.transaction, e); + const hash = sha256(regular(businessFilePath(id, dir))); + for (const e of last.values()) { + const d = e.detail; + if (d?.verb !== ADDITION_VERB || d.request !== request || d.plan_digest !== plan_digest || d.project !== project) continue; + if (e.state === "committed" || (e.state === "prepared" && e.after === hash)) + return { after: e.after, before: e.before, transaction: e.transaction }; + } + return null; + } + // Row 199: a receipt with no document change, for a credential verb that changes only the agents' // store (a revoke or delete of store references). Committed at once; there is nothing to recover. function record(id, { actor, detail }) { @@ -328,5 +393,5 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) { }); } - return { logFile, read, readRaw, write, record, recover: id => withLock(id, () => recover(id)) }; + return { logFile, read, readRaw, write, addProject, hold, findAddition, record, recover: id => withLock(id, () => recover(id)) }; } diff --git a/packages/business/src/project-bootstrap.mjs b/packages/business/src/project-bootstrap.mjs new file mode 100644 index 000000000..cdb09b82a --- /dev/null +++ b/packages/business/src/project-bootstrap.mjs @@ -0,0 +1,74 @@ +// Lifecycle Task 2 (#797 / Gitea #1839): what an exact project addition looks like. The organization +// writer's addProject calls additionProblem on the document it is about to publish, and again on the +// bytes it will write. The bootstrap executor names the paths it reserves with managedPaths, so the +// writer and the executor cannot disagree about the layout. +import { lstatSync, readFileSync } from "node:fs"; +import { isAbsolute, join, normalize } from "node:path"; +import { canonicalJson } from "./util.mjs"; +import { ID_PATTERN } from "./vocabulary.mjs"; + +export const ADDITION_VERB = "project.add"; +const RECEIPT_KEYS = ["data_root", "plan_digest", "project", "request"]; +const ENTRY_KEYS = ["remote", "repo", "root"]; +const plain = x => x !== null && typeof x === "object" && !Array.isArray(x); +const REQUEST_ID = /^[a-zA-Z0-9][a-zA-Z0-9_.:/-]*$/; + +// A managed product's code checkout and its records repository, from the data root alone. +export function managedPaths({ dataRoot, business, project }) { + return { root: join(dataRoot, "products", business, project, "code"), repo: join(dataRoot, "projects", business, project) }; +} + +// null when `next` is `old` plus exactly the planned `projects[project]` entry, else the reason. +export function additionProblem({ old, next, project, entry, receipt }) { + if (typeof project !== "string" || !ID_PATTERN.test(project)) return "project key is not an id"; + if (!plain(receipt) || Object.keys(receipt).sort().join() !== RECEIPT_KEYS.join()) return "receipt has the wrong keys"; + if (typeof receipt.request !== "string" || receipt.request.length > 160 || !REQUEST_ID.test(receipt.request)) return "receipt request is not an id"; + if (typeof receipt.plan_digest !== "string" || !/^[a-f0-9]{64}$/.test(receipt.plan_digest)) return "receipt plan digest is not a digest"; + if (receipt.project !== project) return "receipt names another project"; + if (typeof receipt.data_root !== "string" || !isAbsolute(receipt.data_root) || normalize(receipt.data_root) !== receipt.data_root) return "receipt data root is not absolute"; + if (!plain(entry) || Object.keys(entry).some(k => !ENTRY_KEYS.includes(k))) return "entry holds more than root, repo and remote"; + if (typeof entry.root !== "string" || typeof entry.repo !== "string" || ("remote" in entry && typeof entry.remote !== "string")) return "entry root and repo are required strings"; + if (!plain(old?.projects) || !plain(next?.projects)) return "business has no project mapping"; + const expected = managedPaths({ dataRoot: receipt.data_root, business: old.id, project }); + if (entry.root !== expected.root || entry.repo !== expected.repo) return "entry is not the managed product layout"; + if (Object.hasOwn(old.projects, project)) return "the business already declares this project"; + if (canonicalJson(next.projects) !== canonicalJson({ ...old.projects, [project]: entry })) return "mapping differs from the old mapping plus the entry"; + const { projects: _next, ...rest } = next, { projects: _old, ...was } = old; + if (canonicalJson(rest) !== canonicalJson(was)) return "a key outside projects changed"; + return null; +} + +// The product marker, /products///bootstrap.json. The bootstrap executor writes it +// `reserved` before the mapping exists and `complete` once the product is activated, so a process that cannot +// read the request journal (the launcher and the boot config) can tell a half-built product from a finished one. +const MARKER_STATES = ["reserved", "complete"]; +export const markerFile = ({ dataRoot, business, project }) => join(dataRoot, "products", business, project, "bootstrap.json"); +export function readMarker(file) { + let bytes; + try { + if (!lstatSync(file).isFile()) return { state: "invalid" }; + bytes = readFileSync(file, "utf8"); + } catch (e) { + if (e.code === "ENOENT") return null; + return { state: "invalid" }; + } + try { + const m = JSON.parse(bytes); + const keys = Object.keys(m ?? {}).sort().join(); + if (plain(m) && keys === "plan_digest,project,request,state,version" && m.version === 1 && MARKER_STATES.includes(m.state) && + typeof m.project === "string" && typeof m.request === "string" && /^[a-f0-9]{64}$/.test(m.plan_digest)) return m; + } catch { /* falls through */ } + return { state: "invalid" }; +} +export const markerBytes = ({ state, request, plan_digest, project }) => + JSON.stringify({ version: 1, state, request, plan_digest, project }, null, 2) + "\n"; + +// 'unmanaged' for any entry that is not at the managed layout (or has no marker, so it is checked like any +// declared project), 'complete' for a finished product, 'reserved' for everything else at the layout. +export function managedState({ dataRoot, business, project, entry }) { + const here = managedPaths({ dataRoot, business, project }); + if (!plain(entry) || entry.root !== here.root || entry.repo !== here.repo) return "unmanaged"; + const marker = readMarker(markerFile({ dataRoot, business, project })); + if (marker === null) return "unmanaged"; + return marker.state === "complete" && marker.project === project ? "complete" : "reserved"; +} diff --git a/packages/business/tests/project-bootstrap.test.mjs b/packages/business/tests/project-bootstrap.test.mjs new file mode 100644 index 000000000..6f364b0d0 --- /dev/null +++ b/packages/business/tests/project-bootstrap.test.mjs @@ -0,0 +1,142 @@ +// Lifecycle Task 2 (#797 / Gitea #1839): the one narrow organization-writer method that adds a +// bootstrapped project's mapping. Every ordinary writer keeps refusing project changes. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { organizationWriter } from '../src/organization.mjs'; +import { businessDoc, rolesCopy, tmp, writeJson } from './helpers.mjs'; +import { mappingEntry } from '../../library/tests/helpers.mjs'; + +const DIGEST = 'a'.repeat(64); + +function fixture(t, { barrier } = {}) { + const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true })); + const doc = businessDoc(root); + doc.projects = { books: mappingEntry(), ...doc.projects }; + doc.roles.chief.credentials = { gitea: { ...doc.roles.coder.credentials.gitea } }; + const dir = join(root, 'config'), file = writeJson(join(dir, 'businesses/acme.json'), doc); + const dataRoot = join(root, 'data'); + const writer = organizationWriter({ dir, rolesDir: rolesCopy(root), ...(barrier ? { barrier: step => barrier(step) } : {}) }); + const entry = { root: join(dataRoot, 'products/acme/timer/code'), repo: join(dataRoot, 'projects/acme/timer') }; + const receipt = { request: 'req-1', plan_digest: DIGEST, project: 'timer', data_root: dataRoot }; + const add = (over = {}) => writer.addProject('acme', { revision: writer.read('acme').revision, project: 'timer', entry, receipt, ...over }, 'jason'); + return { root, dataRoot, file, writer, entry, receipt, add, log: writer.logFile('acme') }; +} +const bytes = f => ({ file: readFileSync(f.file), log: (() => { try { return readFileSync(f.log); } catch { return null; } })() }); +const untouched = (f, before) => { + assert.deepEqual(readFileSync(f.file), before.file); + assert.deepEqual((() => { try { return readFileSync(f.log); } catch { return null; } })(), before.log); +}; + +test('add_project_publishes_exactly_one_mapping_with_a_matching_receipt', t => { + const f = fixture(t), old = f.writer.read('acme'); + const out = f.add(); + const now = f.writer.read('acme'); + assert.equal(out.revision, now.revision); + assert.deepEqual(Object.keys(now.document.projects).sort(), ['books', 'stack', 'timer']); + assert.deepEqual(now.document.projects.timer, f.entry); + assert.deepEqual(now.document.projects.stack, old.document.projects.stack); + assert.deepEqual(now.document.projects.books, old.document.projects.books); + const rest = structuredClone(now.document), was = structuredClone(old.document); + delete rest.projects; delete was.projects; + assert.deepEqual(rest, was, 'nothing outside projects changed'); + const last = JSON.parse(readFileSync(f.log, 'utf8').trim().split('\n').at(-1)); + assert.equal(last.state, 'committed'); assert.equal(last.actor, 'jason'); assert.equal(last.before, old.revision); assert.equal(last.after, now.revision); + assert.deepEqual(last.changes.map(c => c.path), ['projects.timer']); + assert.deepEqual(last.detail, { verb: 'project.add', request: 'req-1', plan_digest: DIGEST, project: 'timer' }); + assert.deepEqual(f.writer.findAddition('acme', { request: 'req-1', plan_digest: DIGEST, project: 'timer' }), + { after: now.revision, before: old.revision, transaction: last.transaction }); + assert.equal(f.writer.findAddition('acme', { request: 'req-1', plan_digest: 'b'.repeat(64), project: 'timer' }), null); + assert.equal(f.writer.findAddition('acme', { request: 'req-2', plan_digest: DIGEST, project: 'timer' }), null); +}); + +test('add_project_refuses_everything_but_the_exact_addition_and_changes_no_byte', t => { + const f = fixture(t), before = bytes(f); + const refuse = (what, over, code = 'organization-project-add-refused') => { + assert.throws(() => f.add(over), { code }, what); untouched(f, before); + }; + refuse('replacement', { project: 'stack', receipt: { ...f.receipt, project: 'stack' }, entry: { ...f.entry } }); + refuse('library name', { project: 'books', receipt: { ...f.receipt, project: 'books' } }); + refuse('library entry shape', { entry: { ...mappingEntry() } }); + refuse('extra entry key', { entry: { ...f.entry, kind: 'library' } }); + refuse('unrelated entry key', { entry: { ...f.entry, tracker: { project: 3 } } }); + refuse('entry root off the product layout', { entry: { ...f.entry, root: join(f.root, 'elsewhere') } }); + refuse('entry repo off the records layout', { entry: { ...f.entry, repo: join(f.root, 'elsewhere') } }); + refuse('entry in another business area', { entry: { root: join(f.dataRoot, 'products/other/timer/code'), repo: join(f.dataRoot, 'projects/other/timer') } }); + refuse('receipt names another project', { receipt: { ...f.receipt, project: 'other' } }); + refuse('receipt data root differs', { receipt: { ...f.receipt, data_root: join(f.root, 'other-data') } }); + refuse('receipt digest is not a digest', { receipt: { ...f.receipt, plan_digest: 'nope' } }); + refuse('receipt carries extra keys', { receipt: { ...f.receipt, extra: 1 } }); + refuse('receipt lacks the request', { receipt: { project: 'timer', plan_digest: DIGEST, data_root: f.dataRoot } }); + refuse('relative project key', { project: '../timer', receipt: { ...f.receipt, project: '../timer' } }); + assert.throws(() => f.add({ revision: 'stale' }), { code: 'revision-conflict' }); untouched(f, before); + assert.throws(() => f.writer.addProject('acme', { revision: f.writer.read('acme').revision, project: 'timer', entry: f.entry, receipt: f.receipt }, ''), /authenticated actor/); + untouched(f, before); +}); + +test('add_project_checks_what_lands_on_disk_not_only_the_input', t => { + const f = fixture(t), before = bytes(f); + const entry = { ...f.entry }; + Object.defineProperty(entry, 'toJSON', { value: () => ({ ...f.entry, root: join(f.root, 'serialized-elsewhere') }) }); + assert.throws(() => f.add({ entry }), { code: 'organization-project-add-refused' }); + untouched(f, before); +}); + +test('add_project_twice_is_refused_and_the_first_receipt_stays_findable', t => { + const f = fixture(t); f.add(); + const after = bytes(f); + assert.throws(() => f.add(), { code: 'organization-project-add-refused' }); + untouched(f, after); + assert.ok(f.writer.findAddition('acme', { request: 'req-1', plan_digest: DIGEST, project: 'timer' })); +}); + +test('ordinary_writers_still_refuse_any_project_change_even_the_exact_addition', t => { + const f = fixture(t), before = bytes(f); + const change = fn => { + const next = structuredClone(f.writer.read('acme').document); fn(next); + assert.throws(() => f.writer.write('acme', next, { revision: f.writer.read('acme').revision, actor: 'jason' }), { code: 'organization-projects-protected' }); + untouched(f, before); + }; + change(next => { next.projects.timer = f.entry; }); + change(next => { delete next.projects.stack; }); + change(next => { next.projects.stack.root = join(f.root, 'moved'); }); + // The record verb writes no document, so it cannot add one either. + assert.equal(typeof f.writer.record, 'function'); + assert.equal(Object.hasOwn(f.writer, 'commit'), false, 'the shared commit is private'); +}); + +test('hold_runs_the_check_under_the_writer_lock_after_recovery', t => { + const f = fixture(t), seen = []; + const out = f.writer.hold('acme', ({ revision, document }) => { + seen.push(revision, document.id); + assert.throws(() => f.add(), { code: 'organization-busy' }, 'a writer inside the hold finds the lock'); + return 'checked'; + }); + assert.equal(out, 'checked'); assert.deepEqual(seen, [f.writer.read('acme').revision, 'acme']); + assert.ok(f.add(), 'the lock is released afterwards'); + assert.throws(() => f.writer.hold('acme', async () => {}), { code: 'organization-hold-sync' }, 'a hold callback must be synchronous'); +}); + +test('a_crash_before_publication_leaves_the_file_and_retry_recovers_the_prepared_receipt', t => { + let armed = true; + const f = fixture(t, { barrier: step => { if (armed && step === 'prepared') { armed = false; throw new Error('killed'); } } }); + const before = readFileSync(f.file); + assert.throws(() => f.add(), /killed/); + assert.deepEqual(readFileSync(f.file), before); + assert.equal(f.writer.findAddition('acme', { request: 'req-1', plan_digest: DIGEST, project: 'timer' }), null); + f.add(); + assert.ok(f.writer.findAddition('acme', { request: 'req-1', plan_digest: DIGEST, project: 'timer' })); + const states = readFileSync(f.log, 'utf8').trim().split('\n').map(l => JSON.parse(l).state); + assert.deepEqual(states, ['prepared', 'aborted', 'prepared', 'committed']); +}); + +test('a_crash_after_the_rename_still_finds_the_addition_from_the_prepared_receipt', t => { + let armed = true; + const f = fixture(t, { barrier: step => { if (armed && step === 'renamed') { armed = false; throw new Error('killed'); } } }); + assert.throws(() => f.add(), /killed/); + assert.deepEqual(f.writer.read('acme').document.projects.timer, f.entry); + const found = f.writer.findAddition('acme', { request: 'req-1', plan_digest: DIGEST, project: 'timer' }); + assert.equal(found.after, f.writer.read('acme').revision); + assert.throws(() => f.add(), { code: 'organization-project-add-refused' }); +}); diff --git a/packages/cli/src/config.mjs b/packages/cli/src/config.mjs index 11394df6f..782567e44 100644 --- a/packages/cli/src/config.mjs +++ b/packages/cli/src/config.mjs @@ -7,6 +7,7 @@ import { spawnSync } from "node:child_process"; import { existsSync, lstatSync } from "node:fs"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; +import { managedState } from "../../business/src/project-bootstrap.mjs"; import { BusinessError, accessFilePath, configDir, loadAccess, loadBusiness, loadProject, projectFilePath, resolveInstance, systemVars } from "../../business/src/index.mjs"; import { busBusiness } from "../../bus/src/business.mjs"; import { BusError } from "../../bus/src/broker.mjs"; @@ -53,8 +54,11 @@ function business(fn) { // entry comes from the one declared project whose file sets it. No // tracker.baseUrl, or no project naming a tracker project, means no entry // and no task verbs; two projects naming one each refuse, since the boot -// shape holds one tracker project per business. -function trackerFor(system, b, warn) { +// shape holds one tracker project per business. A product the request lifecycle is still building +// (Task 2, #797) holds the factory project file until its seed commit lands, so a reserved entry is +// skipped; a finished one is checked like any other and omits tracker.project, so the root board stays +// the one tracker project and no second adapter is made for a product. +function trackerFor(system, b, warn, dataRoot) { if (!b.tracker) return null; const first = Object.keys(b.roles)[0]; const base = resolveInstance({ system, business: b, instance: first }).vars; @@ -62,6 +66,7 @@ function trackerFor(system, b, warn) { const named = []; for (const [id, entry] of Object.entries(b.projects)) { if (entry.kind === 'library') continue; + if (dataRoot && managedState({ dataRoot, business: b.id, project: id, entry }) === 'reserved') continue; if (!existsSync(projectFilePath(entry.root))) continue; const project = loadProject(entry.root); if (project.id !== id) throw new CliError(`project file ${project.file} has id ${project.id}, but business ${b.id} declares it as ${id}`, 3); @@ -136,7 +141,7 @@ export function bootConfig({ system, businessId, env = process.env, warn = () => launches: [], readers: [b.id], }; - const tracker = trackerFor(vars, b, warn); + const tracker = trackerFor(vars, b, warn, system.dataRoot); if (tracker) config.trackers = { [b.id]: tracker }; const access = accessFor(system, env, warn); if (access) config.access = access; diff --git a/packages/cli/src/launcher.mjs b/packages/cli/src/launcher.mjs index abc383c43..ec602ce4c 100644 --- a/packages/cli/src/launcher.mjs +++ b/packages/cli/src/launcher.mjs @@ -146,6 +146,7 @@ import { dirname, join, relative } from "node:path"; import { configDir, loadBusiness, resolveInstance, systemVars } from "../../business/src/index.mjs"; import { THREAD_DEFINITIONS } from "../../business/src/vocabulary.mjs"; import { limitHit } from "../../business/src/launch.mjs"; +import { managedState } from "../../business/src/project-bootstrap.mjs"; import { BundleError, buildBundle, hasFallback, readSkills, sessionModel } from "../../harness/src/bundle.mjs"; import { createLoader } from "../../plugins/src/loader.mjs"; import { @@ -564,11 +565,25 @@ export function createLauncher({ async function startLocked({ instance, launchedBy, resolved, model, account, thread, session, human, admissionGeneration, store, credentials, workspace, workspaceDest, lock }) { // What the session sees: a legacy session's workspace at the old instance path. const inSandbox = workspaceDest ?? workspace; + // Lifecycle Task 2 (#797): a product the request lifecycle built under /products is cloned only + // once the bus has admitted it. The bus answers from its receipts, so a reserved or half-built product + // (a mapping published before the seed and records finished) is left out and the launch goes on without + // it. An ordinary mapping is never asked about. + const codeProjects = {}; + for (const [project, entry] of Object.entries(b.projects)) { + if (entry.kind === 'library') continue; + if (managedState({ dataRoot, business: b.id, project, entry }) !== 'unmanaged') { + let answer = null; + try { answer = await host.op({ op: 'request.admission', business: b.id, project }); } catch (e) { answer = { ready: false, code: e.code ?? 'admission-unavailable' }; } + if (answer?.ready !== true) { record({ event: "checkout-skipped", instance, project, reason: answer?.code ?? 'project-pending' }); continue; } + } + codeProjects[project] = entry; + } let checkouts; try { checkouts = await withProcessing(dataRoot, () => { admission(admissionGeneration); - return ensureCheckouts({ workspace, projects: Object.fromEntries(Object.entries(b.projects).filter(([, entry]) => entry.kind !== 'library')), env }); + return ensureCheckouts({ workspace, projects: codeProjects, env }); }); } catch (e) { if (e instanceof CheckoutError) throw new Refusal("checkout-failed", `${e.project}: ${e.message}`); diff --git a/packages/seat/src/working.mjs b/packages/seat/src/working.mjs index 5fe928530..7254e89a6 100644 --- a/packages/seat/src/working.mjs +++ b/packages/seat/src/working.mjs @@ -261,6 +261,24 @@ export function push({ dataRoot, business, project, root, taskRef, branch, bundl }); } +// await initializeWorking({ dataRoot, business, project, root, env, limits }) +// Host-only (lifecycle Task 2, #797): makes the project's working repository and seeds main from +// the root's checked-out commit, under the same lock and through the same private seed a push +// uses. It makes no work ref, takes no bundle and has no remote to push to. A second call is the +// same seed: it moves nothing the first call left. Resolves { repo, commit } with main's commit. +export function initializeWorking({ dataRoot, business, project, root, env = process.env, limits = GIT_LIMITS }) { + if (!PROJECT.test(project ?? "") || !PROJECT.test(business ?? "")) refuse("invalid-request", "not an id"); + const repo = workingRepo(dataRoot, business, project); + const g = bound(env, limits); + return locked(repo, async () => { + await ensure(repo, g); + await seed(repo, root, g); + const commit = await tip(repo, "refs/heads/main", g); + if (!commit) refuse("working-repo-invalid", "main has no commit after the seed"); + return { repo, commit }; + }); +} + // await review({ dataRoot, business, project, root, taskRef, commit, have, out, env, limits }) // Writes a bundle of `commit` to `out`, with one ref, refs/mosaic/review. // `have` lists commits the reviewer's clone holds; those the repository