Compare commits

..
Author SHA1 Message Date
jason.woltje 8827905ba0 fix(org): restore names bus recover for a stale lock; receipt taken at mount; restore-then-dashboard test (#1873)
Build review P3-1, P3-3 and P3-4.
2026-11-11 20:14:46 -06:00
jason.woltje a559957085 feat(org): restore starts the bus; pages follow restart and quick setup (#1873)
P1-4: org restore starts the bus as mosaic bus start ID --dev-local-owner
--detach would, and its receipt names the bus. Where that start is refused
(deployed or stamped instance, agent run) restore refuses before changing
anything; --no-bus keeps the old restore. The Console Restore action
restores and starts the bus with one receipt, and offers Restart bus when
the start fails after a good restore.

P1-5: Quick Setup refreshes the sidebar organization list, and the
organization dashboard redraws on the config event Restart bus already
dispatches, so the bus line and System types follow without a reload.
2026-11-11 20:03:35 -06:00
jason.woltje 6d0e9f89be fix(bus): sweep ungranted copy slots at start, clean a failed stage (#1873)
Security review P2: a hard crash after the stage left the staged token
on disk. sweepCopySlots runs at start, before Credentials loads, and
removes each copy service slot whose binding holds no grant, plus any
staged temp file. It never touches a role's slots, a slot a loaded
reference names, or an organization whose document cannot be read. A
stage write that fails removes its temp file, and its slot when the
rename has already happened.

Build review P3s: push is held to the binding's approved level, not the
token's. Tests now cover the boot-copy pre-checks, the format check
running before the stage, and stageCopy's reference refusal.
2026-11-11 17:13:04 -06:00
jason.woltje d5aa963893 feat(bus): copy credential issue CB2, Gitea verified live (#1873)
An owner-issued Gitea token for an agent copy is staged inactive in the
self-only store, proved live against the copy's Gitea (token scopes by
semantic comparison with all and admin refused, the copy's dedicated
per-org/copy/service account, every reachable repository, expiry), then
committed under the writer's generation, revision, decision, copyId,
incarnation and definition-pin recheck. The grant record holds only an
opaque grant id and rotate date. Vikunja refuses
credential-scope-unverified. Copies still cannot launch (CB3).
2026-11-11 17:13:04 -06:00
24 changed files with 2038 additions and 59 deletions

No files matched your search

+1 -1
View File
@@ -335,7 +335,7 @@ For local development on a named instance only, `mosaic bus start BUSINESS --dev
`mosaic console restart` and `mosaic org reload ORG` are the Console's Settings controls (F3) from a terminal (U1, #1873). Each sends one request to the running Console named in the instance's console state, as the page sends it (the Console's own Host and Origin, `Sec-Fetch-Site: same-origin`, a JSON `{}` body), so the Console applies the same owner, `--send` and origin checks and writes the same action trail, `<dataRoot>/console/actions.jsonl`. Both refuse inside an agent run (exit 3) before reading anything, and refuse with exit 3 when no Console runs. `console restart` works only for a Console started in the foreground by `mosaic console start`, or by `mosaic console start --dev-local-owner --detach`, which leaves a detached supervisor (`packages/cli/src/console-supervisor.mjs`, its own session) running the Console as the foreground start does; its receipt adds the supervisor's `supervisor` pid, and its later warnings go to `console/supervisor-*.log` under the data root. `mosaic console stop` stops that Console, and its supervisor then exits. `mosaic console start` refuses inside an agent run (exit 3), with or without `--dev-local-owner` or `--detach`, checking the starting process's own environment as well as the one given for the Console (agent-guard, #1873). A detached start whose supervisor does not reply within 60 seconds signals the supervisor's process group, then kills it, and clears `console/start.lock` only when the lock names that supervisor (pid and start time) and it is gone. Any other Console answers `not-supervised`. `console restart` waits up to 30 seconds for the relaunched Console to answer and prints `{"outcome":"restarted","from":OLD,"boot":NEW}`. `org reload ORG` restarts a running bus or starts a stopped one and prints the Console's done receipt. A request that left but got no answer is reported as outcome unknown (exit 1) and is never retried; check `mosaic console status`, `mosaic bus status ORG` and the trail. `mosaic bus restart ORG` remains the trail-less host verb.
The bus and Console lifecycle verbs refuse inside an agent run (agent-guard, #1873): `mosaic bus start`, `bus stop`, `bus restart` and `bus recover`, and `mosaic console start`, `console stop` and `console restart`, and `mosaic org archive` and `org delete`, which stop the organization's bus. Each exits 3 with `refused: this runs only from a human shell, outside any agent run (found MARKER in the environment)` before it reads anything. It checks the environment it is given and its own process environment, so a sanitized environment does not hide a marker. A usage error (a missing or extra argument, `--pm` with `--no-launch`, or a bad `--dev-insecure-bind`/`--dev-allow-host`) still comes first (exit 4). `mosaic bus status` and `mosaic console status` stay open to agents. The Console's Restart bus and Restart Console controls keep working: the Console starts its bus host with its own environment, which carries no marker, and Restart Console relaunches the Console under its supervisor without going through the CLI. The markers are the ones `packages/cli/src/transport.mjs` lists. The check is cooperative: a process that drops the markers from its environment is not stopped by it.
The bus and Console lifecycle verbs refuse inside an agent run (agent-guard, #1873): `mosaic bus start`, `bus stop`, `bus restart` and `bus recover`, and `mosaic console start`, `console stop` and `console restart`, and `mosaic org archive` and `org delete`, which stop the organization's bus, and `mosaic org restore`, which starts it (`org restore --no-bus` starts nothing and is not refused). Each exits 3 with `refused: this runs only from a human shell, outside any agent run (found MARKER in the environment)` before it reads anything. It checks the environment it is given and its own process environment, so a sanitized environment does not hide a marker. A usage error (a missing or extra argument, `--pm` with `--no-launch`, or a bad `--dev-insecure-bind`/`--dev-allow-host`) still comes first (exit 4). `mosaic bus status` and `mosaic console status` stay open to agents. The Console's Restart bus and Restart Console controls keep working: the Console starts its bus host with its own environment, which carries no marker, and Restart Console relaunches the Console under its supervisor without going through the CLI. The markers are the ones `packages/cli/src/transport.mjs` lists. The check is cooperative: a process that drops the markers from its environment is not stopped by it.
The instance processing switch is separate from per-business launches off/on.
`mosaic halt` durably stops managed processing across every business in the
+30 -1
View File
@@ -642,6 +642,7 @@ export class Broker {
// Row 236 (#436b): giteaGrant and giteaKeys from gitea-grant.mjs, or null, and the remote halves of
// section 5's restart rows and of revokes a restart cut short, for reconcileGitea.
#giteaGrant;
#copyGrant;
#giteaKeys;
#giteaUndo = [];
// Row 281 (#1786, d6 section 3b): each tracked Gitea promise and the business/agent it acts for, or
@@ -667,7 +668,7 @@ export class Broker {
#processing;
#taskTracker;
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
constructor({ store, businesses, pluginService = null, secretCheck = () => {}, credentialStatus = () => [], organization = null, hires = null, autonomy = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, proxyInit = null, proxyFenceClear = null, proxyPage = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, grantScope = null, roleFile = null, giteaGrant = null, giteaKeys = null, access = null, accessStamp = null, compactions = null, requestLifecycle = null, requirementsHost = null, workspaceRead = null, taskTracker = null }) {
constructor({ store, businesses, pluginService = null, secretCheck = () => {}, credentialStatus = () => [], organization = null, hires = null, autonomy = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, proxyInit = null, proxyFenceClear = null, proxyPage = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, grantScope = null, roleFile = null, giteaGrant = null, copyGrant = null, giteaKeys = null, access = null, accessStamp = null, compactions = null, requestLifecycle = null, requirementsHost = null, workspaceRead = null, taskTracker = null }) {
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
// Row 194 b15 (sage-2501): the function-argument checks come before anything reads the store.
if (harnessOn !== null && typeof harnessOn !== 'function') throw new TypeError('harnessOn must be a function');
@@ -701,6 +702,7 @@ export class Broker {
this.#hostGrant = hostGrant;
this.#hostKeys = hostKeys;
this.#giteaGrant = giteaGrant;
this.#copyGrant = copyGrant;
this.#giteaKeys = giteaKeys;
this.#access = access;
this.#accessStamp = accessStamp;
@@ -6369,6 +6371,11 @@ export class Broker {
if (a.token !== undefined) string(a.token, 16384);
if (a.date !== undefined) string(a.date, 10);
if (a.reason !== undefined) string(a.reason, 500);
// #1873 CB2: an issue for a copy this bus booted with goes to the copy verifier (copy-grant.mjs).
// The boot's copies view leaves out a name the file's own roles hold; a copy that needs no
// credential joins the roles but is still a copy. Any other unknown name answers credential-unknown.
const booted = this.#business(s.business);
if (verb === 'credentials.issue' && this.#copyGrant && booted.copies && Object.hasOwn(booted.copies, a.agent)) return this.#issueCopyGrant(s, a);
if (!Object.hasOwn(this.#business(s.business).roles, a.agent)) fail('credential-unknown');
// A running agent keeps its tokens until its runs end: delete waits for every claim to end.
if (verb === 'credentials.delete' && (this.#holder(s.business, a.agent)?.op === 'claim' || this.#threadHolders(s.business, a.agent).length))
@@ -6399,6 +6406,28 @@ export class Broker {
}
return result;
}
// #1873 CB2: owner only, as every binding decision (design Q1): not the tester, a reader or an
// org.hire member. The recheck runs after the remote work and again under the writer's locks.
#copyOwner(s) {
if (s.tester || s.reader || !s.human) fail(s.tester ? 'tester-refused' : 'human-required');
if (s.hireMembership ? s.hireMembership.owner !== true : s.human !== this.#business(s.business).human) fail('human-required');
}
#issueCopyGrant(s, a) {
this.#copyOwner(s);
const passed = (e) => e instanceof BusError && (e.code.startsWith('credential-') || ['copy-unavailable', 'human-required',
'definition-holds-credential', 'definition-scan-unreadable'].includes(e.code));
return this.#copyGrant(s.business, a, { actor: s.human, owner: true, recheck: () => this.#copyOwner(s) })
.then((result) => {
this.#secretCheck(result);
if (JSON.stringify(result).includes(a.token)) fail('credential-leak');
return result;
}, (e) => {
if (passed(e)) throw e;
if (['revision-conflict', 'decision-stale'].includes(e?.code) || e?.code?.startsWith('organization-')) fail(e.code);
if (e?.name === 'BusinessError') fail('invalid-request');
throw new BusError('outcome-unknown');
});
}
async #autonomyConsole(s,verb,a) {
if(verb==='autonomy.get.console') {
keys(a,[]);
+233
View File
@@ -0,0 +1,233 @@
// #1873 CB2 (design section 5 steps 4 to 6): credentials.issue for an agent copy's bound service.
// Gitea is the only service with a verifier in this slice. A copy token is accepted only after the
// Gitea server it belongs to proves the whole grant live: the token's scopes, the account it belongs
// to (the copy's own dedicated account, never an administrator or another copy's or agent's), every
// repository that account can reach and the token's expiry, against the owner-approved binding. A
// Vikunja binding refuses credential-scope-unverified: that is the shipped behaviour.
//
// Order (no lock is held across the remote work):
// 1. checks that need nothing written: owner, service, date, token form, definition scan, the
// record and the boot snapshot, the bound projects' remotes and the approved scopes;
// 2. stage: the writer deactivates the record (pending, the service's grant fields gone), then the
// token goes to the copy's slot in the self-only store;
// 3. the slot is read back by the identity the stage wrote, and Gitea is asked;
// 4. commit: the writer rechecks registry incarnation, document revision, binding generation,
// copyId, decision and definition pin under its locks; its beforeWrite reads the slot back by
// identity again and runs the caller's session recheck. Only then do the grant metadata and the
// receipt land.
// Any refusal after the stage removes the slot it wrote and leaves the record pending. A lost reply
// or a crash leaves the record pending too; nothing resolves a pending record, and recovery is a
// new verified issue. A token a crash left staged goes at the next start (sweepCopySlots). Nothing here logs, and no result, receipt or refusal holds the token, the
// slot's path, a request or anything derived from the token.
import { lstatSync, readdirSync, unlinkSync } from 'node:fs';
import { join } from 'node:path';
import { BusError } from './broker.mjs';
import { Credentials, assertNotHeld, day, privateDir } from './credentials.mjs';
import { send, versionAtLeast, REPO_PAGE, REPO_PAGES } from './gitea-grant.mjs';
import { scanOrganizations } from './definition-scan.mjs';
import { parseGiteaUrl, GITEA_SCOPE_CATEGORIES, ID_PATTERN, SERVICES } from '../../organization/src/vocabulary.mjs';
import { copyBotLogin, scopesWithin } from '../../organization/src/copy-bindings.mjs';
const deny = (code, reason) => {
throw new BusError(code, reason);
};
const unverified = (reason) => deny('credential-scope-unverified', reason);
const DAY_MS = 86400000;
// A copy credential's rotation date is after today and at most this many days ahead.
export const MAX_ROTATE_DAYS = 366;
const object = (x) => x !== null && typeof x === 'object' && !Array.isArray(x);
const positive = (n) => Number.isSafeInteger(n) && n > 0;
const lower = (s) => (typeof s === 'string' ? s.toLowerCase() : null);
// The scopes a Gitea token reports, as {category: level}. Gitea writes write:X as implying read:X and
// may list both; the higher level wins. "all", any admin category, a category Gitea added that this
// bus doesn't know, or anything that doesn't parse returns null: such a token is broader than any
// binding can approve.
export function tokenLevels(scopes) {
if (!Array.isArray(scopes) || scopes.length === 0) return null;
const out = {};
for (const scope of scopes) {
const m = typeof scope === 'string' ? /^(read|write):([a-z]+)$/.exec(scope) : null;
if (!m || !GITEA_SCOPE_CATEGORIES.includes(m[2])) return null;
if (out[m[2]] !== 'write') out[m[2]] = m[1];
}
return out;
}
const asList = (levels) => Object.entries(levels).map(([c, l]) => `${l}:${c}`).sort();
// The approved Gitea repositories of a copy's binding: one https origin, lower-cased "owner/repo"
// names, from each bound project's remote. A project with no Gitea remote, two remotes that disagree
// or two servers cannot be proved.
export function boundRepositories(doc, projects) {
let origin = null;
const repos = new Set();
for (const project of projects ?? []) {
const entry = doc.projects?.[project];
if (!object(entry)) unverified('remote');
const remotes = [entry.remote, entry.git?.mode === 'remote' ? entry.git.remote : undefined].filter((r) => r !== undefined);
if (remotes.length === 0 || remotes.some((r) => r !== remotes[0])) unverified('remote');
const parsed = parseGiteaUrl(remotes[0], { repo: true });
if (!parsed.url) unverified('remote');
if (origin !== null && parsed.url.origin !== origin) unverified('remote');
origin = parsed.url.origin;
repos.add(`${parsed.owner}/${parsed.repo}`.toLowerCase());
}
if (origin === null) unverified('remote');
return { origin, repos };
}
// Live introspection of one Gitea token against an approved binding. Returns { userId, tokenId,
// login } or throws the refusal. `approved` is the binding's Gitea scopes; `repos` the lower-cased
// names it may reach; `login` the account this copy's grant must belong to; `rotateBy` the date the
// owner gave. Every call is GET with the token; a missing or ambiguous answer is unverified.
export async function verifyGiteaToken({ transport = send, url, ca, timeoutMs, token, approved, repos, login, rotateBy }) {
const call = (path) => transport({ url, method: 'GET', path, auth: { token }, ca, timeoutMs });
const version = await call('/api/v1/version');
if (version.status !== 200 || !object(version.json) || !versionAtLeast(version.json.version)) unverified('version');
const info = await call('/api/v1/token');
if (info.status === 401 || info.status === 403 || info.status === 404) unverified('token');
if (info.status !== 200 || !object(info.json) || !positive(info.json.id) || !object(info.json.user) || !positive(info.json.user.id)
|| typeof info.json.user.login !== 'string') unverified('token');
if (lower(info.json.user.login) !== lower(login)) deny('credential-identity-refused', login);
const levels = tokenLevels(info.json.scopes);
if (!levels || !scopesWithin('gitea', asList(levels), approved)) deny('credential-scope-exceeded');
if (!levels.user) unverified('scopes');
// An expiry Gitea reports must not come before the owner's rotation date ends.
if (info.json.expires_at !== undefined && info.json.expires_at !== null) {
const at = typeof info.json.expires_at === 'string' ? Date.parse(info.json.expires_at) : NaN;
if (Number.isNaN(at)) unverified('token');
if (at < Date.parse(rotateBy + 'T00:00:00Z') + DAY_MS) deny('credential-date');
}
const user = await call('/api/v1/user');
if (user.status !== 200 || !object(user.json) || typeof user.json.is_admin !== 'boolean') unverified('user');
if (user.json.id !== info.json.user.id || lower(user.json.login) !== lower(login) || user.json.is_admin !== false)
deny('credential-identity-refused', login);
// Every repository the account can reach: owned, collaborator and team access. One unrelated
// repository, an admin permission, or push where the binding approved read refuses. Push is the
// account's right, so it is held to what the binding approved, not to the token's own level.
const write = tokenLevels(approved)?.repository === 'write';
const seen = new Map();
let total = null;
for (let page = 1; ; page++) {
if (page > REPO_PAGES) unverified('repositories');
const r = await call(`/api/v1/user/repos?limit=${REPO_PAGE}&page=${page}`);
if (r.status !== 200 || !Array.isArray(r.json) || r.total === null || (total !== null && r.total !== total)) unverified('repositories');
total = r.total;
if (total > REPO_PAGE * REPO_PAGES) unverified('repositories');
for (const repo of r.json) {
if (!object(repo) || typeof repo.full_name !== 'string' || !object(repo.permissions)) unverified('repositories');
const name = repo.full_name.toLowerCase();
if (!repos.has(name)) deny('credential-scope-exceeded');
const p = repo.permissions;
if (p.admin !== false || typeof p.push !== 'boolean' || p.pull !== true || (p.push && !write)) deny('credential-scope-exceeded');
seen.set(name, true);
}
if (seen.size >= total || r.json.length === 0) break;
}
if (seen.size !== total) unverified('repositories');
for (const name of repos) if (!seen.has(name)) unverified('repositories');
// An organization scope reaches organizations' own settings: the account must belong to none.
if (levels.organization) {
const orgs = await call(`/api/v1/user/orgs?limit=${REPO_PAGE}&page=1`);
if (orgs.status !== 200 || !Array.isArray(orgs.json)) unverified('organizations');
if (orgs.json.length) deny('credential-scope-exceeded');
}
return { userId: info.json.user.id, tokenId: info.json.id, login };
}
// The issue for one copy's service. `copies` is the bus's boot snapshot ({ org, copies,
// definitions }); `writer` the organization writer; `credentials` the bus's store. `hooks` are the
// tests' crash points (afterStage, afterRemote, beforeCommit), never set at runtime.
export function copyGrant({ credentials, writer, copies = null, organizations = null, transport = send, ca = undefined, timeoutMs = undefined, now = () => new Date(), hooks = {} }) {
return async (business, args, { actor, owner = false, recheck = () => {} } = {}) => {
const { agent: instance, service, token, date } = args;
if (owner !== true || actor === 'tester') deny('human-required');
if (service === 'vikunja') unverified('service');
if (service !== 'gitea') deny('credential-unknown');
const today = now().toISOString().slice(0, 10);
if (!day(date) || date <= today || Date.parse(date) - Date.parse(today) > MAX_ROTATE_DAYS * DAY_MS) deny('credential-date');
const value = Credentials.checkFormat(service, token);
const { document: doc, revision } = writer.read(business);
scanOrganizations({ organizations: organizations?.() ?? [business], own: { id: business, document: doc }, readRaw: (b) => writer.readRaw(b),
boot: copies?.definitions ? { org: copies.org, definitions: copies.definitions } : null }, (v) => assertNotHeld(v, [value]));
const record = doc.copyBindings?.[instance];
if (!object(record) || record.state === 'unbound' || !object(record.services?.[service])) deny('credential-unbound');
const copy = copies?.org === business && Object.hasOwn(copies.copies ?? {}, instance) ? copies.copies[instance] : null;
if (!copy || copy.state === 'refused' || Object.hasOwn(doc.roles ?? {}, instance)) deny('copy-unavailable');
if (copy.level !== record.source?.level || copy.name !== record.source?.name || (copy.project ?? null) !== (record.source?.project ?? null)
|| copy.definition !== record.definition?.ref) deny('credential-binding-changed');
const approved = record.services[service];
if (!Array.isArray(approved.scopes) || !approved.scopes.some((s) => /^(read|write):user$/.test(s))) unverified('scopes');
const { origin, repos } = boundRepositories(doc, approved.resources?.projects);
const login = copyBotLogin(record.orgIncarnation, record.copyId, service);
if (!login) deny('credential-binding-changed');
const op = { instance, service, copyId: record.copyId, generation: record.generation, decision: record.decision, incarnation: record.orgIncarnation };
const definitions = copies?.org === business ? copies.definitions ?? {} : {};
const staged = writer.stageCopyGrant(business, op, { revision, actor, owner, definitions });
const slot = business + '/' + instance;
let identity = null;
try {
identity = credentials.stageCopy(slot, service, value.token);
await hooks.afterStage?.();
// An inode number can be reused after an unlink, so the read-back must also be the staged token.
const read = credentials.readCopy(slot, service, identity);
if (read.token !== value.token) deny('credential-file');
const grant = await verifyGiteaToken({ transport, url: origin, ca, timeoutMs, token: read.token, approved: approved.scopes, repos, login, rotateBy: date });
await hooks.afterRemote?.();
await recheck();
const grantId = `gitea:${grant.userId}:${grant.tokenId}`;
const after = { ...op, generation: staged.generation };
const committed = writer.commitCopyGrant(business, after, {
grantId, rotateBy: date, revision: staged.revision, actor, owner, definitions,
beforeWrite: () => {
hooks.beforeCommit?.();
const again = credentials.readCopy(slot, service, identity);
if (again.token !== read.token) deny('credential-file');
recheck();
},
});
return { agent: instance, service, verb: 'issue', date, state: committed.state, account: login, generation: committed.generation };
} catch (e) {
if (identity) {
try {
credentials.unlinkCopy(slot, service, identity);
} catch {}
}
throw e;
}
};
}
// Security review P2: at start, before Credentials loads and before any issue can run, each staged
// file of a copy's service whose binding holds no grant for that service goes, and so does any
// interrupted write's temp file. An issue killed after its stage leaves the record pending and the
// token on disk; nothing resolves a pending record and no reference loads a copy slot, so the file
// only keeps a secret nothing will use. `documents` is [business, validated organization document]
// for each loaded business (null when unreadable: nothing of it is touched). Only names the document
// binds as copies are looked at, never a name its own roles hold or a slot a loaded reference names.
// A store dir that isn't private refuses credential-file, as Credentials would. Returns
// [{ instance, service }], never a path.
export function sweepCopySlots(storeRoot, documents, references = {}) {
const removed = [];
if (!storeRoot || !privateDir(storeRoot, { missing: true })) return removed;
for (const [business, doc] of documents) {
if (!object(doc) || !object(doc.copyBindings) || !ID_PATTERN.test(business)) continue;
const dir = join(storeRoot, business);
if (!privateDir(dir, { missing: true })) continue;
for (const [copy, record] of Object.entries(doc.copyBindings)) {
if (!ID_PATTERN.test(copy) || Object.hasOwn(doc.roles ?? {}, copy)) continue;
const copyDir = join(dir, copy);
if (!privateDir(copyDir, { missing: true })) continue;
const instance = business + '/' + copy;
for (const name of readdirSync(copyDir)) {
const service = name.endsWith('.tmp') ? name.slice(0, -4) : name;
if (!SERVICES.includes(service) || references[instance]?.[service] !== undefined) continue;
if (name === service && typeof record?.services?.[service]?.grantId === 'string') continue;
if (lstatSync(join(copyDir, name)).isDirectory()) continue;
unlinkSync(join(copyDir, name));
removed.push({ instance, service });
}
}
}
return removed;
}
+103 -17
View File
@@ -6,7 +6,7 @@ import { dirname, isAbsolute, join, relative, resolve } from 'node:path';
import { createHash } from 'node:crypto';
import { isDeepStrictEqual } from 'node:util';
import { BusError } from './broker.mjs';
import { credentialKind, isHostKind, ID_PATTERN } from '../../organization/src/vocabulary.mjs';
import { credentialKind, isHostKind, ID_PATTERN, SERVICES } from '../../organization/src/vocabulary.mjs';
const deny = (code) => {
throw new BusError(code);
};
@@ -90,7 +90,7 @@ export function parseHostKey(text) {
// Credentials.assertClean's test: credential-leak when the value's JSON text holds an entry's token
// or one of its retiring model keys. org-defscan (#1873): issue's check runs it on the value it is
// about to hold.
function assertNotHeld(value, entries) {
export function assertNotHeld(value, entries) {
let encoded;
try {
encoded = JSON.stringify(value);
@@ -479,6 +479,18 @@ export class Credentials {
const value = Credentials.#format(service, token);
check((v) => assertNotHeld(v, [value]));
this.#invalidateModelProof(instance, service);
this.#writeSlot(instance, service, service === 'model' ? JSON.stringify(value.model) + '\n' : value.key ? value.token : value.token + '\n');
const file = this.#storePath(instance, service);
commit();
this.#refs.set(instance + ':' + service, { ref: { service, store: true, file, [service === 'vikunja' ? 'expires' : 'rotateBy']: date }, date });
this.#tokens.set(instance + ':' + service, { ...value, date });
this.#revoked.delete(instance + ':' + service);
this.#held.delete(instance + ':' + service);
return this.status().find((e) => e.instance === instance && e.service === service);
}
// The store slot write issue and stageCopy share: tmp (O_EXCL, O_NOFOLLOW, 0600), fsync, rename
// over the slot, fsync the dir. Returns the identity of the file it wrote.
#writeSlot(instance, service, text, { undo = false } = {}) {
const file = this.#storePath(instance, service);
const agentDir = dirname(file);
mkdirSync(this.#storeRoot, { recursive: true, mode: 0o700 });
@@ -493,27 +505,101 @@ export class Credentials {
} catch (e) {
if (e.code !== 'ENOENT') deny('credential-file');
}
let fd;
// A write that fails leaves no token behind: the temp file this call made goes, and with `undo`
// (stageCopy) so does the slot it renamed into place.
let fd, identity;
try {
fd = openSync(tmp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
writeSync(fd, service === 'model' ? JSON.stringify(value.model) + '\n' : value.key ? value.token : value.token + '\n');
fsyncSync(fd);
try {
fd = openSync(tmp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
const st = fstatSync(fd);
identity = { dev: st.dev, ino: st.ino };
writeSync(fd, text);
fsyncSync(fd);
} finally {
if (fd !== undefined) closeSync(fd);
}
renameSync(tmp, file);
} catch (e) {
if (identity) {
try {
const st = lstatSync(tmp);
if (st.isFile() && st.dev === identity.dev && st.ino === identity.ino) unlinkSync(tmp);
} catch {}
}
throw e;
}
try {
const dirFd = openSync(agentDir, constants.O_RDONLY);
try {
fsyncSync(dirFd);
} finally {
closeSync(dirFd);
}
} catch (e) {
if (undo) {
try {
this.unlinkCopy(instance, service, identity);
} catch {}
}
throw e;
}
return identity;
}
// #1873 CB2 (design section 5 step 4): a copy's service slot, <store>/<business>/<copy>/<service>.
// The bus stages it while the copy's binding is inactive, and afterwards reads back or removes only
// the file with the identity (dev, ino) it wrote. Memory and references never hold a copy's token
// in this slice; launch admission (CB3) reads the slot. A name a loaded reference uses refuses.
static checkFormat(service, token) {
return Credentials.#format(service, token);
}
stageCopy(instance, service, token) {
if (this.#refs.has(instance + ':' + service) || !SERVICES.includes(service)) deny('credential-reference');
const value = Credentials.#format(service, token);
return this.#writeSlot(instance, service, value.token + '\n', { undo: true });
}
// The staged token, read back with the store's checks plus one link only and the exact identity the
// stage wrote: a symlink, a hard link, a replaced or a stale file refuses credential-file.
readCopy(instance, service, identity) {
const file = this.#storePath(instance, service);
privateDir(this.#storeRoot);
privateDir(dirname(dirname(file)));
privateDir(dirname(file));
const same = (st) => st.dev === identity?.dev && st.ino === identity?.ino;
let original;
try {
original = lstatSync(file);
} catch {
deny('credential-file');
}
if (original.isSymbolicLink() || !original.isFile() || original.nlink !== 1 || !same(original)) deny('credential-file');
let fd, text;
try {
fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW);
const st = fstatSync(fd);
if (!st.isFile() || st.uid !== process.getuid() || (st.mode & 0o777) !== 0o600 || st.size > 16384 || st.nlink !== 1 || !same(st))
deny('credential-file');
text = readFileSync(fd, 'utf8');
} catch (e) {
if (e instanceof BusError) throw e;
deny('credential-file');
} finally {
if (fd !== undefined) closeSync(fd);
}
renameSync(tmp, file);
const dirFd = openSync(agentDir, constants.O_RDONLY);
return Credentials.#format(service, text);
}
// Removes the slot only while it is still the file the stage wrote; another file is left alone.
unlinkCopy(instance, service, identity) {
const file = this.#storePath(instance, service);
let st;
try {
fsyncSync(dirFd);
} finally {
closeSync(dirFd);
st = lstatSync(file);
} catch (e) {
if (e.code === 'ENOENT') return false;
deny('credential-file');
}
commit();
this.#refs.set(instance + ':' + service, { ref: { service, store: true, file, [service === 'vikunja' ? 'expires' : 'rotateBy']: date }, date });
this.#tokens.set(instance + ':' + service, { ...value, date });
this.#revoked.delete(instance + ':' + service);
this.#held.delete(instance + ':' + service);
return this.status().find((e) => e.instance === instance && e.service === service);
if (!st.isFile() || st.dev !== identity?.dev || st.ino !== identity?.ino) return false;
unlinkSync(file);
return true;
}
// Row 236 (#436b section 5, settle forward): memory takes the store file as it is now, with the
// constructor's checks, and `date` becomes the entry's date. A held entry refuses.
+4
View File
@@ -22,6 +22,10 @@ group("organization-bindings-protected", "R1/R8: copy bindings change only throu
group("credential-binding-changed credential-role-changed credential-definition-unpinned", "R1/R8: binding approval pinned to the current copy, definition and generation", "Read the copy's current binding and definition, then ask the chief to raise a fresh binding decision for the owner; an old approval never carries over.");
group("credential-scope-exceeded", "R1/R8: effective access is the role intersected with the approval", "Narrow the requested scopes and projects to what the copy's role and admitted projects allow, or change the role through its own decision first.");
group("credential-unbound", "R1/R8: launch needs an approved, verified binding", "Ask the chief to raise a binding decision for the missing service; the owner approves it. A pending binding is not usable until verification exists.");
// #1873 CB2: a copy's service credential is accepted only after live verification at the service.
group("credential-scope-unverified", "R1/R8: a copy credential is accepted only when the service proves its whole grant", "Gitea: approve read:user in the copy's binding (its role must declare it) so the account's repositories can be listed, give every bound project a Gitea remote on one server, and retry when the server answers. Vikunja has no verifier yet; its copy bindings stay unusable.");
group("credential-identity-refused", "R1/R8: one dedicated service account per organization, copy and service", "Create the account the refusal names on the copy's Gitea server, not an administrator, with access to the bound projects' repositories only, and issue a token from that account. Never reuse another copy's or agent's account or token.");
group("credential-date credential-format", "R1/R8: well-formed credentials with a future rotation date", "Supply the token on standard input alone, and a rotation date after today in YYYY-MM-DD form, no more than a year ahead for a copy credential.");
group("copy-unavailable copy-refused", "R3/R8: one active, unambiguous copy identity", "Read the organization's copies. Ask the owner to resolve the definition collision or restore the copy's source before raising a binding decision.");
group("copy-stale", "R3: root policy changed since this bus booted", "Ask the owner to restart the organization bus to load the approved root policy, then launch a fresh copy.");
group("root-policy-unreadable", "R3: root policy evidence must be readable", "Ask the owner to inspect root agent receipt metadata and restart the bus after repairing the read failure.");
+20
View File
@@ -46,6 +46,7 @@ import { ROLE_RIGHTS } from '../../organization/src/org-rights.mjs';
import { getProjectRemote } from '../../organization/src/project-remotes.mjs';
import { credentialVerbs } from './credential-verbs.mjs';
import { loadWithCopies } from '../../organization/src/agent-copies.mjs';
import { copyGrant, sweepCopySlots } from './copy-grant.mjs';
import { loadRole } from '../../organization/src/role.mjs';
import { resolveInstance } from '../../organization/src/resolve.mjs';
import { canonicalJson } from '../../organization/src/util.mjs';
@@ -403,6 +404,20 @@ export async function startBroker({
// Row 236 (#1742, #436b r5 section 5): each gitea grant record gives its agent a store reference to
// gitea, and `held` names the files memory must not load until reconcile or a revoke.
const { held } = storeRoot ? giteaStart(store, storeRoot, businesses, references) : { held: [] };
// #1873 CB2 (security review P2): a copy slot an issue staged before a crash, whose binding holds
// no grant for it, goes before Credentials loads and before any issue can run.
if (storeRoot) {
const reader = organizationWriter({ dir: organization.dir, rolesDir: organization.rolesDir, dataRoot: instanceRoot ?? dataRoot });
const documents = Object.keys(businesses).map((id) => {
try {
return [id, reader.read(id).document];
} catch {
return [id, null];
}
});
for (const entry of sweepCopySlots(storeRoot, documents, references))
process.stderr.write(`copies: ${JSON.stringify({ event: 'copy-slot-swept', ...entry })}\n`);
}
// #1886 (F3 defect l): a store reference with no file yet reads as no key (design D3a), but Credentials
// refuses a missing store root, and only issue() made it. Start makes it the same way issue() does when
// any reference names the store. A root that exists with loose modes or as a symlink still refuses.
@@ -792,6 +807,11 @@ export async function startBroker({
// Row 236 (#1742): the Gitea grant path. The admin slot is <configDir>/secrets/grants/<business>/gitea.
giteaGrant: organization ? giteaGrant({ configDir: organization.dir }) : null,
giteaKeys: storeRoot ? giteaKeys({ credentials, storeRoot }) : null,
// #1873 CB2: credentials.issue for an agent copy's bound Gitea service, verified live at the server.
copyGrant: organization && storeRoot
? copyGrant({ credentials, writer: organizationWriter({ dir: organization.dir, rolesDir: organization.rolesDir, dataRoot: instanceRoot ?? dataRoot }),
copies, organizations: () => Object.keys(businesses) })
: null,
usage,
limits,
store,
@@ -0,0 +1,227 @@
// #1873 CB2 live acceptance: credentials.issue for an agent copy's Gitea service against a scratch
// gitea/gitea:1.27.1, a throwaway container on 127.0.0.1 over TLS with a scratch CA, removed after.
// The test never pulls an image; without Docker or the image it skips. Every token here is a scratch
// token this container made; none is printed, and nothing here reaches git.mosaicstack.dev or any
// real service.
import test from 'node:test';
import assert from 'node:assert/strict';
import { createHash, randomBytes } from 'node:crypto';
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { request } from 'node:https';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { spawnSync } from 'node:child_process';
import { Broker, BusError } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { Credentials } from '../src/credentials.mjs';
import { copyGrant } from '../src/copy-grant.mjs';
import { hireDecisions } from '../../organization/src/hire-decisions.mjs';
import { organizationWriter } from '../../organization/src/organization.mjs';
import { loadBusiness } from '../../organization/src/organization-config.mjs';
import { checkRootAgents } from '../../organization/src/agent-levels.mjs';
import { materializeCopies, copiesView } from '../../organization/src/agent-copies.mjs';
import { activateOrg, reserveOrg } from '../../organization/src/org-registry.mjs';
import { copyBotLogin } from '../../organization/src/copy-bindings.mjs';
import { businessDoc, rolesCopy, writeJson } from '../../organization/tests/helpers.mjs';
import { scratchCa } from '../../seat/tests/fake-gitea.mjs';
const IMAGE = 'gitea/gitea:1.27.1';
const SUBJECT = { provider: 'test', iss: 'https://issuer.test', sub: 'owner' };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const ahead = (days) => new Date(Date.now() + days * 86400000).toISOString().slice(0, 10);
const sha = (s) => createHash('sha256').update(s).digest('hex');
const docker = (...args) => spawnSync('docker', args, { encoding: 'utf8', timeout: 120000 });
const skip = docker('image', 'inspect', IMAGE).status === 0 ? false : `no Docker or no local ${IMAGE}; this test never pulls`;
const freePort = () =>
new Promise((resolve, reject) => {
const s = createServer();
s.once('error', reject);
s.listen(0, '127.0.0.1', () => {
const { port } = s.address();
s.close(() => resolve(port));
});
});
// One HTTPS call to the scratch container, trusting only the scratch CA.
function call(url, ca, method, path, { auth = null, body = null } = {}) {
return new Promise((resolve) => {
const data = body ? JSON.stringify(body) : null;
const headers = { accept: 'application/json', ...(data ? { 'content-type': 'application/json' } : {}), ...(auth ? { authorization: auth } : {}) };
const req = request(url + path, { method, ca, headers, timeout: 10000 }, (res) => {
let text = '';
res.on('data', (c) => (text += c));
res.on('end', () => {
let json = null;
try {
json = text ? JSON.parse(text) : null;
} catch {}
resolve({ status: res.statusCode, json });
});
});
req.on('timeout', () => req.destroy());
req.on('error', () => resolve({ status: 0, json: null }));
req.end(data ?? undefined);
});
}
test('CB2-D: on scratch 1.27.1 an owner-issued copy token is proved live: equal and narrower bind; broader, all, admin, foreign, unrelated and opaque refuse', { skip, timeout: 300000 }, async (t) => {
// Capture the child's complete TAP stdout/stderr before any of it reaches this runner, so the parent
// can search for the actual minted values without putting one in an assertion.
if (!process.env.CB2_COPY_CAPTURE) {
const capture = mkdtempSync(join(tmpdir(), 'cb2cap-'));
t.after(() => rmSync(capture, { recursive: true, force: true }));
const file = join(capture, 'minted');
const childEnv = { ...process.env, CB2_COPY_CAPTURE: file };
delete childEnv.NODE_TEST_CONTEXT;
const child = spawnSync(process.execPath, ['--test', '--test-name-pattern=^CB2-D:', fileURLToPath(import.meta.url)], {
encoding: 'utf8', timeout: 280000, env: childEnv,
});
const values = existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : [];
const output = (child.stdout ?? '') + (child.stderr ?? '');
assert.ok(values.every((value) => !output.includes(value) && !output.includes(sha(value))), 'complete TAP stdout/stderr contain no minted token');
t.diagnostic(output);
t.diagnostic(`minted ${values.length}, hits 0 (complete TAP stdout/stderr)`);
assert.equal(child.status, 0, 'Docker child passed');
assert.equal(values.length, 6, 'all six scratch tokens were minted: the acceptance actually ran');
return;
}
const dir = mkdtempSync(join(tmpdir(), 'cb2d-'));
t.after(() => rmSync(dir, { recursive: true, force: true }));
const tlsDir = join(dir, 'tls'), tls = scratchCa(tlsDir);
// The container's git user reads the scratch key; it is a throwaway key for this run only.
chmodSync(tlsDir, 0o755);
chmodSync(join(tlsDir, 'server.key'), 0o644);
const ca = readFileSync(tls.ca, 'utf8'), port = await freePort();
const url = `https://127.0.0.1:${port}`, name = 'cb2-' + randomBytes(6).toString('hex');
t.after(() => docker('rm', '-f', name));
const started = docker('run', '-d', '--rm', '--name', name, '-p', `127.0.0.1:${port}:3000`, '-v', `${tlsDir}:/certs:ro`,
'-e', 'GITEA__security__INSTALL_LOCK=true', '-e', 'GITEA__database__DB_TYPE=sqlite3',
'-e', 'GITEA__service__DISABLE_REGISTRATION=true', '-e', 'GITEA__server__PROTOCOL=https',
'-e', 'GITEA__server__CERT_FILE=/certs/server.pem', '-e', 'GITEA__server__KEY_FILE=/certs/server.key',
'-e', `GITEA__server__ROOT_URL=${url}/`, IMAGE);
assert.equal(started.status, 0, 'scratch container starts');
const end = Date.now() + 120000;
while ((await call(url, ca, 'GET', '/api/v1/version')).status !== 200) {
assert.ok(Date.now() < end, 'scratch Gitea answers in 120 seconds');
await sleep(500);
}
const gitea = (...args) => docker('exec', '-u', 'git', name, 'gitea', ...args);
const minted = [];
const createUser = (user, admin = false) => {
const r = gitea('admin', 'user', 'create', '--username', user, '--email', `${user}@example.test`,
'--password', randomBytes(16).toString('base64url'), '--must-change-password=false', ...(admin ? ['--admin'] : []));
assert.equal(r.status, 0, `create scratch ${user}`);
};
const mint = (user, scopes) => {
const r = gitea('admin', 'user', 'generate-access-token', '--username', user, '--token-name', 'cb2-' + randomBytes(4).toString('hex'),
'--scopes', scopes.join(','), '--raw');
assert.equal(r.status, 0, `mint scratch ${user}`);
const value = r.stdout.trim();
assert.ok(/^[0-9a-f]{40}$/.test(value), 'minted token shape');
minted.push(value);
writeFileSync(process.env.CB2_COPY_CAPTURE, JSON.stringify(minted), { mode: 0o600 });
return value;
};
createUser('scratch-admin', true);
createUser('owner');
const admin = 'token ' + mint('scratch-admin', ['write:admin', 'write:repository', 'write:user']);
for (const repo of ['stack', 'other']) {
const made = await call(url, ca, 'POST', '/api/v1/admin/users/owner/repos', { auth: admin, body: { name: repo, auto_init: true, private: true } });
assert.equal(made.status, 201, `create owner/${repo}`);
}
// The organization: one root agent "rev" whose copy binding approves write:repository and
// read:user on project stack, whose remote is the scratch owner/stack.
const root = mkdtempSync(join(tmpdir(), 'cb2o-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const rolesDir = rolesCopy(root);
const doc = { ...businessDoc(root, 'demo'), plugins: { harness: ['pi'] }, vars: { 'bus.tester': true } };
doc.projects.stack.remote = `${url}/owner/stack`;
const file = writeJson(join(root, 'businesses/demo.json'), doc);
const dataRoot = join(root, 'data');
mkdirSync(dataRoot, { mode: 0o700 });
const ctx = { configDir: root, dataRoot, instance: 'test' };
await reserveOrg(ctx, 'demo');
await activateOrg(ctx, 'demo');
const snapshot = materializeCopies({ orgId: 'demo', doc, root: { state: 'ok', ...checkRootAgents({ agentsVersion: 1, agents: { rev: { definition: 'reviewer' } } }, { rolesDir }) }, rolesDir });
const writer = organizationWriter({ dir: root, rolesDir }), store = new Store(dataRoot);
t.after(() => store.close());
const service = hireDecisions({ writer, rolesDir, dataRoot, configDir: root, assertClean: () => {}, copies: (id) => (snapshot.org === id ? snapshot : null) });
const loaded = () => {
const d = loadBusiness('demo', { dir: root, rolesDir });
// The boot's copies view, as the runtime's loadWithCopies gives the broker.
return { ...d, roles: Object.fromEntries(Object.entries(d.roles).map(([n, e]) => [n, { ...d.definitions[e.definition], ...e }])), copies: copiesView(snapshot.copies) };
};
const storeRoot = join(root, 'secrets', 'agents');
mkdirSync(storeRoot, { recursive: true, mode: 0o700 });
chmodSync(join(root, 'secrets'), 0o700);
const credentials = new Credentials({ references: {}, storeRoot, repoRoots: [], dataRoot: '/nonexistent-data' });
const issue = copyGrant({ credentials, writer, copies: snapshot, organizations: () => ['demo'], ca, timeoutMs: 10000 });
const b = new Broker({
store, businesses: { demo: loaded() },
hires: { ...service, apply: async (...args) => { const r = await service.apply(...args); b.reloadBusiness('demo', loaded()); return r; } },
credentialVerbs: () => { throw new BusError('credential-unknown'); }, copyGrant: (...a) => issue(...a),
});
const consoleCap = b.bindConsole({ business: 'demo', human: 'jason', subject: SUBJECT });
const human = b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
const request_ = (cap, verb, args) => Promise.resolve().then(() => b.request(cap, { verb, args }));
await request_(consoleCap, 'hire.start.console', { op: 'change', level: 'root', name: 'rev', reason: 'Bind the copy',
binding: { services: { gitea: { scopes: ['write:repository', 'read:user'], projects: ['stack'] } } } });
const rec = () => JSON.parse(readFileSync(file, 'utf8')).copyBindings.rev;
assert.equal(rec().state, 'pending');
const slot = join(storeRoot, 'demo', 'rev', 'gitea');
const issueToken = (token) => request_(human, 'credentials.issue', { agent: 'rev', service: 'gitea', date: ahead(30), token });
// The copy's dedicated account: a read collaborator on owner/stack only.
const login = copyBotLogin(rec().orgIncarnation, rec().copyId, 'gitea');
createUser(login);
const collaborate = (repo, permission) => call(url, ca, 'PUT', `/api/v1/repos/owner/${repo}/collaborators/${login}`, { auth: admin, body: { permission } });
assert.equal((await collaborate('stack', 'read')).status, 204, 'bot is a read collaborator on owner/stack');
const refused = async (token, code, label) => {
await assert.rejects(issueToken(token), { code }, label);
assert.equal(rec().state, 'pending', label);
assert.equal(rec().services.gitea.grantId, undefined, label);
assert.ok(!existsSync(slot), `${label}: a refused token leaves no slot`);
};
// Equal: the approved scopes exactly.
const equal = mint(login, ['write:repository', 'read:user']);
const self = await call(url, ca, 'GET', '/api/v1/token', { auth: 'token ' + equal });
assert.equal(self.status, 200);
assert.equal(self.json.user.login, login, 'Gitea 1.27.1 names the token owner on /api/v1/token');
const one = await issueToken(equal);
assert.equal(one.state, 'bound');
assert.equal(one.account, login);
assert.deepEqual([rec().state, rec().services.gitea.grantId, rec().services.gitea.grantGeneration], ['bound', `gitea:${self.json.user.id}:${self.json.id}`, 1]);
assert.equal(statSync(slot).mode & 0o777, 0o600);
assert.equal(readFileSync(slot, 'utf8'), equal + '\n');
// Narrower: read within the approved write.
const narrow = mint(login, ['read:repository', 'read:user']);
await issueToken(narrow);
assert.deepEqual([rec().state, rec().services.gitea.grantGeneration], ['bound', 2]);
assert.equal(readFileSync(slot, 'utf8'), narrow + '\n');
// Broader in one category, and "all": exceeded; a failed re-issue deactivates the binding.
await refused(mint(login, ['write:repository', 'write:user']), 'credential-scope-exceeded', 'write:user beyond read:user');
await refused(mint(login, ['all']), 'credential-scope-exceeded', 'all');
// The administrator's own token, inside the approved scopes: not the copy's account.
await refused(mint('scratch-admin', ['read:repository', 'read:user']), 'credential-identity-refused', 'admin account');
// A token Gitea never issued.
await refused(randomBytes(20).toString('hex'), 'credential-scope-unverified', 'opaque token');
// An unrelated repository the account can also reach.
assert.equal((await collaborate('other', 'read')).status, 204);
await refused(narrow, 'credential-scope-exceeded', 'unrelated repository');
// Nothing kept names a token, its digest, the slot path or a request.
const kept = [file, join(root, 'businesses/.history/demo/changes.jsonl')].filter(existsSync).map((p) => readFileSync(p, 'utf8')).join('\n')
+ ['events', 'decisions'].map((tb) => JSON.stringify(store.all(`SELECT * FROM ${tb}`))).join('\n');
for (const value of minted) assert.ok(!kept.includes(value) && !kept.includes(sha(value)) && !kept.includes(sha(value + '\n')), 'kept text holds no token');
for (const bad of [slot, storeRoot, '/api/v1', 'authorization']) assert.ok(!kept.includes(bad), 'kept text holds no path or request');
t.diagnostic(`minted ${minted.length}`);
});
+558
View File
@@ -0,0 +1,558 @@
// #1873 CB2 (design section 5 steps 4 to 6, section 8 row CB2): credentials.issue for an agent copy's
// bound Gitea service, verified live against a fake Gitea before any grant metadata lands. The
// organization writer's half is packages/organization/tests/copy-grant-writer.test.mjs; the real
// server acceptance is copy-grant-docker.test.mjs.
import test from 'node:test';
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, renameSync, rmSync, statSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { Broker, BusError } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { Credentials } from '../src/credentials.mjs';
import { REFUSALS } from '../src/refusals.mjs';
import { copyGrant, tokenLevels, verifyGiteaToken, boundRepositories, MAX_ROTATE_DAYS, sweepCopySlots } from '../src/copy-grant.mjs';
import { fakeGiteaApi } from './fake-gitea-api.mjs';
import { hireDecisions } from '../../organization/src/hire-decisions.mjs';
import { organizationWriter } from '../../organization/src/organization.mjs';
import { loadBusiness } from '../../organization/src/organization-config.mjs';
import { checkRootAgents } from '../../organization/src/agent-levels.mjs';
import { materializeCopies, copiesView } from '../../organization/src/agent-copies.mjs';
import { activateOrg, reserveOrg } from '../../organization/src/org-registry.mjs';
import { copyBotLogin } from '../../organization/src/copy-bindings.mjs';
import { businessDoc, rolesCopy, writeJson } from '../../organization/tests/helpers.mjs';
const SUBJECT = { provider: 'test', iss: 'https://issuer.test', sub: 'owner' };
const READ = { gitea: { scopes: ['read:repository', 'read:user'], projects: ['stack'] } };
const ahead = (days) => new Date(Date.now() + days * 86400000).toISOString().slice(0, 10);
const present = (p) => { try { lstatSync(p); return true; } catch { return false; } };
const sha = (s) => createHash('sha256').update(s).digest('hex');
async function fixture(t, { services = READ, edit = (d) => d } = {}) {
const root = mkdtempSync(join(tmpdir(), 'cb2-')), rolesDir = rolesCopy(root);
const api = await fakeGiteaApi(t, root);
const base = { ...businessDoc(root, 'demo'), plugins: { harness: ['pi'] }, vars: { 'bus.tester': true } };
base.projects.stack.agents = { checker: { definition: 'reviewer' } };
base.projects.stack.remote = `${api.url}/owner/stack`;
const doc = edit(base);
const file = writeJson(join(root, 'businesses/demo.json'), doc);
const dataRoot = join(root, 'data'); mkdirSync(dataRoot, { mode: 0o700 });
const ctx = { configDir: root, dataRoot, instance: 'test' };
await reserveOrg(ctx, 'demo'); await activateOrg(ctx, 'demo');
const snapshot = materializeCopies({ orgId: 'demo', doc, root: { state: 'ok', ...checkRootAgents({ agentsVersion: 1, agents: { rev: { definition: 'reviewer' } } }, { rolesDir }) }, rolesDir });
const writer = organizationWriter({ dir: root, rolesDir }), store = new Store(dataRoot);
const service = hireDecisions({ writer, rolesDir, dataRoot, configDir: root, assertClean: () => {}, copies: (id) => snapshot.org === id ? snapshot : null });
const loaded = () => { const d = loadBusiness('demo', { dir: root, rolesDir }); return { ...d, roles: Object.fromEntries(Object.entries(d.roles).map(([n, e]) => [n, { ...d.definitions[e.definition], ...e }])), copies: copiesView((opts.copies ?? snapshot).copies) }; };
const storeRoot = join(root, 'secrets', 'agents');
mkdirSync(storeRoot, { recursive: true, mode: 0o700 }); chmodSync(join(root, 'secrets'), 0o700);
const credentials = new Credentials({ references: {}, storeRoot, repoRoots: [], dataRoot: '/nonexistent-data' });
const hooks = {};
const opts = { timeoutMs: 3000 };
const issue = (...a) => copyGrant({ credentials, writer, copies: opts.copies ?? snapshot, organizations: () => ['demo'], ca: api.ca, timeoutMs: opts.timeoutMs, hooks })(...a);
const direct = (args, ctx) => issue('demo', args, ctx);
const b = new Broker({ store, businesses: { demo: loaded() }, hires: { ...service, apply: async (...args) => { const r = await service.apply(...args); b.reloadBusiness('demo', loaded()); return r; } },
credentialVerbs: () => { throw new BusError('credential-unknown'); }, copyGrant: (...a) => issue(...a) });
const owner = b.bindConsole({ business: 'demo', human: 'jason', subject: SUBJECT });
const human = b.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
const chief = b.bindLaunch({ business: 'demo', role: 'chief', run: 'chief-run', harness: 'pi' });
b.request(chief, { verb: 'role.claim' });
const call = (cap, verb, args = {}) => Promise.resolve().then(() => b.request(cap, { verb, args }));
const bind = (name = 'rev', svc = services, extra = {}) => call(owner, 'hire.start.console', { op: 'change', level: 'root', name, reason: 'Bind the copy', binding: { services: svc }, ...extra });
await bind();
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
const rec = (name = 'rev') => JSON.parse(readFileSync(file, 'utf8')).copyBindings?.[name];
const repo = api.addRepo('owner', 'stack');
// The copy's dedicated account, a read collaborator on owner/stack, and one token from it.
const bot = (name = 'rev', { perm = 'read', admin = false } = {}) => {
const r = rec(name);
const login = copyBotLogin(r.orgIncarnation, r.copyId, 'gitea');
const u = api.users.get(login) ?? api.addUser(login, { is_admin: admin }).user;
repo.collaborators.push(login); repo.permissions[login] = perm;
return u;
};
const token = (u, scopes = ['read:repository', 'read:user'], o) => api.addToken(u.login, `t${api.issued.length}`, scopes, o);
const slot = (name = 'rev') => join(storeRoot, 'demo', name, 'gitea');
const issueAs = (cap, args) => call(cap, 'credentials.issue', { agent: 'rev', service: 'gitea', date: ahead(30), ...args });
return { root, file, api, b, store, writer, credentials, hooks, opts, direct, owner, human, chief, call, bind, rec, bot, token, slot, repo, storeRoot, issueAs, loaded, snapshot };
}
// Everything the bus or the organization keeps after an issue: the document, the receipts, the store.
function kept(f) {
const files = [f.file, join(f.root, 'businesses/.history/demo/changes.jsonl')].filter(existsSync).map((p) => readFileSync(p, 'utf8'));
const rows = ['events', 'decisions'].map((tb) => JSON.stringify(f.store.all(`SELECT * FROM ${tb}`)));
return files.join('\n') + rows.join('\n');
}
test('CB2 an owner-issued copy token is verified live, staged and bound; nothing kept names the token or its path', async (t) => {
const f = await fixture(t);
assert.equal(f.rec().state, 'pending');
const u = f.bot();
const tok = f.token(u);
const date = ahead(30);
const result = await f.issueAs(f.human, { token: tok.sha1, date });
assert.deepEqual(result, { agent: 'rev', service: 'gitea', verb: 'issue', date, state: 'bound', account: u.login, generation: result.generation });
const r = f.rec();
assert.equal(r.state, 'bound');
assert.deepEqual(r.services.gitea, { slot: 'self', scopes: ['read:repository', 'read:user'], resources: { projects: ['stack'] }, rotateBy: date, grantId: `gitea:${u.id}:${tok.id}`, grantGeneration: 1 });
assert.equal(statSync(f.slot()).mode & 0o777, 0o600);
assert.equal(readFileSync(f.slot(), 'utf8'), tok.sha1 + '\n');
// Only GETs went to Gitea, every one as the copy's account.
assert.ok(f.api.requests.length > 0);
for (const req of f.api.requests) {
assert.equal(req.method, 'GET');
if (req.path !== '/api/v1/version') assert.equal(req.as, u.login);
}
const text = kept(f) + JSON.stringify(result);
for (const bad of [tok.sha1, sha(tok.sha1), sha(tok.sha1 + '\n'), f.slot(), f.storeRoot, '/api/v1', 'authorization']) assert.ok(!text.includes(bad), 'kept text holds a secret, path or request');
// Copies still cannot launch in CB2: the snapshot does not resolve a gitea-only copy (CB3 owns launch).
});
test('CB2 narrower and equal scopes are accepted; any broader token is exceeded and leaves the record inactive', async (t) => {
const f = await fixture(t, { services: { gitea: { scopes: ['write:repository', 'read:user'], projects: ['stack'] } } });
const u = f.bot('rev', { perm: 'write' });
for (const scopes of [['write:repository', 'write:user'], ['all'], ['write:admin', 'read:user'], ['read:repository', 'read:user', 'read:issue'], ['read:user', 'read:misc']]) {
const tok = f.token(u, scopes);
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-exceeded' }, scopes.join());
assert.equal(f.rec().state, 'pending', scopes.join());
assert.equal(f.rec().services.gitea.grantId, undefined);
assert.ok(!existsSync(f.slot()), 'a refused token leaves no slot');
}
// Equal.
await f.issueAs(f.human, { token: f.token(u, ['write:repository', 'read:user']).sha1 });
assert.equal(f.rec().state, 'bound');
// Narrower (read within the approved write) re-issues; the grant generation counts up.
f.repo.permissions[u.login] = 'read';
const narrow = f.token(u, ['read:repository', 'read:user']);
await f.issueAs(f.human, { token: narrow.sha1 });
assert.deepEqual([f.rec().state, f.rec().services.gitea.grantGeneration, f.rec().services.gitea.grantId], ['bound', 2, `gitea:${u.id}:${narrow.id}`]);
// A failed re-issue after bound deactivates the binding and removes its slot.
await assert.rejects(f.issueAs(f.human, { token: f.token(u, ['all']).sha1 }), { code: 'credential-scope-exceeded' });
assert.deepEqual([f.rec().state, f.rec().services.gitea.grantId, existsSync(f.slot())], ['pending', undefined, false]);
});
test('CB2 read is narrower than write within a category; categories are independent; push needs write', async (t) => {
assert.deepEqual(tokenLevels(['read:repository', 'write:repository', 'read:user']), { repository: 'write', user: 'read' });
assert.deepEqual(tokenLevels(['write:issue', 'read:issue']), { issue: 'write' });
for (const bad of [['all'], ['write:admin'], ['read:admin'], ['read:sudo'], ['repository'], [], null, [7]]) assert.equal(tokenLevels(bad), null, JSON.stringify(bad));
const f = await fixture(t);
// The binding approved read:repository; the account can push, so the token reaches more than read.
const u = f.bot('rev', { perm: 'write' });
await assert.rejects(f.issueAs(f.human, { token: f.token(u).sha1 }), { code: 'credential-scope-exceeded' });
// write:issue is another category: read:repository never approves it.
f.repo.permissions[u.login] = 'read';
await assert.rejects(f.issueAs(f.human, { token: f.token(u, ['read:repository', 'read:user', 'write:issue']).sha1 }), { code: 'credential-scope-exceeded' });
await f.issueAs(f.human, { token: f.token(u).sha1 });
assert.equal(f.rec().state, 'bound');
});
test('CB2 push is held to the binding, not the token: a narrower token binds for a push account on a write binding', async () => {
const push = { '/api/v1/user/repos': { status: 200, json: [{ full_name: 'owner/stack', permissions: { admin: false, push: true, pull: true } }], total: 1 } };
await assert.rejects(V(scripted(push).transport), { code: 'credential-scope-exceeded' });
const write = { approved: ['write:repository', 'read:user'] };
assert.deepEqual(await V(scripted(push).transport, write), { userId: 5, tokenId: 9, login: 'bot' });
const writeToken = { '/api/v1/token': { status: 200, json: { id: 9, scopes: ['write:repository', 'read:user'], user: { id: 5, login: 'bot' } } } };
assert.deepEqual(await V(scripted({ ...push, ...writeToken }).transport, write), { userId: 5, tokenId: 9, login: 'bot' });
await assert.rejects(V(scripted({ ...push, ...writeToken }).transport), { code: 'credential-scope-exceeded' });
});
test('CB2 the boot copy must still match the record: a missing, refused or changed copy refuses before anything is staged or sent', async (t) => {
const f = await fixture(t);
const tok = f.token(f.bot());
const rev = f.snapshot.copies.rev;
const cases = [
[{ ...f.snapshot, copies: {} }, 'copy-unavailable'],
[{ ...f.snapshot, org: 'other' }, 'copy-unavailable'],
[{ ...f.snapshot, copies: { rev: { ...rev, state: 'refused' } } }, 'copy-unavailable'],
[{ ...f.snapshot, copies: { rev: { ...rev, level: 'project', project: 'stack' } } }, 'credential-binding-changed'],
[{ ...f.snapshot, copies: { rev: { ...rev, name: 'other' } } }, 'credential-binding-changed'],
[{ ...f.snapshot, copies: { rev: { ...rev, definition: 'coder' } } }, 'credential-binding-changed'],
];
const before = kept(f), sent = f.api.requests.length;
for (const [copies, code] of cases) {
f.opts.copies = copies;
await assert.rejects(f.direct({ agent: 'rev', service: 'gitea', token: tok.sha1, date: ahead(30) }, { actor: 'jason', owner: true }), { code }, JSON.stringify(copies.copies?.rev ?? copies.org));
}
f.opts.copies = null;
// No record for the name, or a name the organization's own roles hold.
await assert.rejects(f.direct({ agent: 'nobody', service: 'gitea', token: tok.sha1, date: ahead(30) }, { actor: 'jason', owner: true }), { code: 'credential-unbound' });
f.opts.copies = { ...f.snapshot, copies: { ...f.snapshot.copies, chief: { ...rev, name: 'chief' } } };
const { document } = f.writer.read('demo');
const shadow = { ...f.writer, read: () => ({ ...f.writer.read('demo'), document: { ...document, copyBindings: { ...document.copyBindings, chief: document.copyBindings.rev } } }) };
await assert.rejects(copyGrant({ credentials: f.credentials, writer: shadow, copies: f.opts.copies, organizations: () => ['demo'] })('demo', { agent: 'chief', service: 'gitea', token: tok.sha1, date: ahead(30) }, { actor: 'jason', owner: true }), { code: 'copy-unavailable' });
f.opts.copies = null;
assert.deepEqual([kept(f) === before, f.api.requests.length, present(join(f.storeRoot, 'demo'))], [true, sent, false]);
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
});
test('CB2 stageCopy refuses a name a loaded reference holds and an unknown service, leaving the file alone', async (t) => {
const root = mkdtempSync(join(tmpdir(), 'cb2-ref-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const storeRoot = join(root, 'agents');
mkdirSync(join(storeRoot, 'demo', 'rev'), { recursive: true, mode: 0o700 }); chmodSync(storeRoot, 0o700); chmodSync(join(storeRoot, 'demo'), 0o700);
const file = join(storeRoot, 'demo', 'rev', 'gitea');
writeFileSync(file, 'r'.repeat(40) + '\n', { mode: 0o600 });
const credentials = new Credentials({ references: { 'demo/rev': { gitea: { service: 'gitea', store: true, file, rotateBy: ahead(30) } } }, storeRoot, repoRoots: [], dataRoot: '/nonexistent-data' });
assert.throws(() => credentials.stageCopy('demo/rev', 'gitea', 'a'.repeat(40)), { code: 'credential-reference' });
assert.equal(readFileSync(file, 'utf8'), 'r'.repeat(40) + '\n');
assert.throws(() => credentials.stageCopy('demo/other', 'model', 'a'.repeat(40)), { code: 'credential-reference' });
assert.equal(present(join(storeRoot, 'demo', 'other')), false);
});
test('CB2 a Vikunja binding refuses credential-scope-unverified before anything is staged or sent', async (t) => {
const vik = { tasks: ['read_one'] };
const f = await fixture(t, { services: { gitea: READ.gitea, vikunja: { scopes: vik, projects: ['stack'] } } });
const before = readFileSync(f.file, 'utf8');
await assert.rejects(f.issueAs(f.human, { service: 'vikunja', token: 'v'.repeat(40) }), { code: 'credential-scope-unverified' });
assert.equal(readFileSync(f.file, 'utf8'), before);
assert.equal(f.api.requests.length, 0);
assert.ok(!existsSync(join(f.storeRoot, 'demo', 'rev')));
// With a Gitea grant the record still waits on Vikunja: pending, never bound.
await f.issueAs(f.human, { token: f.token(f.bot()).sha1 });
assert.equal(f.rec().state, 'pending');
assert.ok(f.rec().services.gitea.grantId);
assert.ok(REFUSALS['credential-scope-unverified'].nextStep.includes('Vikunja'));
});
test('CB2 unrelated repositories, admin rights or a missing bound repository refuse', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
const other = f.api.addRepo('someone', 'private', { collaborators: [u.login] });
other.permissions[u.login] = 'read';
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-exceeded' });
other.collaborators.length = 0;
const own = f.api.addRepo(u.login, 'scratch');
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-exceeded' });
f.api.repos.splice(f.api.repos.indexOf(own), 1);
f.repo.collaborators.length = 0;
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-unverified' });
assert.equal(f.rec().state, 'pending');
f.repo.collaborators.push(u.login);
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
});
// A scripted Gitea for the verifier's own branches: paging, organizations and malformed answers.
function scripted(over = {}) {
const base = {
'/api/v1/version': { status: 200, json: { version: '1.27.1' } },
'/api/v1/token': { status: 200, json: { id: 9, scopes: ['read:repository', 'read:user'], user: { id: 5, login: 'bot' } } },
'/api/v1/user': { status: 200, json: { id: 5, login: 'bot', is_admin: false } },
'/api/v1/user/repos': { status: 200, json: [{ full_name: 'owner/stack', permissions: { admin: false, push: false, pull: true } }], total: 1 },
'/api/v1/user/orgs': { status: 200, json: [], total: 0 },
...over,
};
const seen = [];
const transport = async ({ path, method }) => {
seen.push(`${method} ${path}`);
const key = path.split('?')[0];
const a = base[key];
return typeof a === 'function' ? a(path) : { total: null, ...a };
};
return { transport, seen };
}
const V = (transport, extra = {}) => verifyGiteaToken({ transport, url: 'https://127.0.0.1:1', token: 'x'.repeat(40), approved: ['read:repository', 'read:user', 'read:organization'], repos: new Set(['owner/stack']), login: 'bot', rotateBy: ahead(30), ...extra });
test('CB2 verifier: paging must agree, organizations are refused with an organization scope', async () => {
assert.deepEqual(await V(scripted().transport), { userId: 5, tokenId: 9, login: 'bot' });
const pages = (p1, p2) => (path) => (path.includes('page=1') ? p1 : p2);
const repo = (n) => ({ full_name: `owner/${n}`, permissions: { admin: false, push: false, pull: true } });
const two = new Set(['owner/stack', 'owner/b']);
const many = Array.from({ length: 50 }, (_, i) => repo(`r${i}`));
const fifty = new Set(many.map((r) => r.full_name.toLowerCase()));
// A total that changes between pages, a page that comes back short, and no total at all.
await assert.rejects(V(scripted({ '/api/v1/user/repos': pages({ status: 200, json: many, total: 51 }, { status: 200, json: [repo('x')], total: 52 }) }).transport, { repos: fifty }), { code: 'credential-scope-unverified' });
await assert.rejects(V(scripted({ '/api/v1/user/repos': pages({ status: 200, json: many, total: 52 }, { status: 200, json: [], total: 52 }) }).transport, { repos: fifty }), { code: 'credential-scope-unverified' });
await assert.rejects(V(scripted({ '/api/v1/user/repos': { status: 200, json: [repo('stack')], total: null } }).transport), { code: 'credential-scope-unverified' });
await assert.rejects(V(scripted({ '/api/v1/user/repos': { status: 200, json: [repo('stack'), repo('b')], total: 2 } }).transport), { code: 'credential-scope-exceeded' });
assert.ok(await V(scripted({ '/api/v1/user/repos': { status: 200, json: [repo('stack'), repo('b')], total: 2 } }).transport, { repos: two }));
await assert.rejects(V(scripted({ '/api/v1/user/repos': { status: 200, json: [{ full_name: 'owner/stack', permissions: { push: false, pull: true } }], total: 1 } }).transport), { code: 'credential-scope-exceeded' });
await assert.rejects(V(scripted({ '/api/v1/user/repos': { status: 200, json: [{ full_name: 'owner/stack' }], total: 1 } }).transport), { code: 'credential-scope-unverified' });
// Organizations: only asked when the token has an organization level.
const plain = scripted({ '/api/v1/user/orgs': { status: 200, json: [{ id: 1, username: 'acme' }], total: 1 } });
assert.ok(await V(plain.transport));
assert.ok(!plain.seen.some((s) => s.includes('/user/orgs')));
const orgTok = { '/api/v1/token': { status: 200, json: { id: 9, scopes: ['read:organization', 'read:repository', 'read:user'], user: { id: 5, login: 'bot' } } } };
assert.ok(await V(scripted(orgTok).transport));
await assert.rejects(V(scripted({ ...orgTok, '/api/v1/user/orgs': { status: 200, json: [{ id: 1, username: 'acme' }], total: 1 } }).transport), { code: 'credential-scope-exceeded' });
await assert.rejects(V(scripted({ ...orgTok, '/api/v1/user/orgs': { status: 500, json: null } }).transport), { code: 'credential-scope-unverified' });
// The bound projects' remotes: one https Gitea origin, agreeing remotes.
const doc = (projects) => ({ projects });
assert.deepEqual([...boundRepositories(doc({ a: { remote: 'https://127.0.0.1:9/Owner/Stack' } }), ['a']).repos], ['owner/stack']);
for (const [d, ps] of [[doc({ a: { remote: 'http://127.0.0.1:9/o/r' } }), ['a']], [doc({ a: {} }), ['a']], [doc({}), ['a']], [doc({ a: { remote: 'https://127.0.0.1:9/o/r' }, b: { remote: 'https://127.0.0.2:9/o/r' } }), ['a', 'b']], [doc({}), []],
[doc({ a: { remote: 'https://127.0.0.1:9/o/r', git: { mode: 'remote', remote: 'https://127.0.0.1:9/o/s' } } }), ['a']]])
assert.throws(() => boundRepositories(d, ps), { code: 'credential-scope-unverified' });
});
test('CB2 a foreign account, an administrator or another copy\'s token is identity-refused', async (t) => {
const f = await fixture(t);
const u = f.bot();
const foreign = f.api.addUser('someone-else').user;
f.repo.collaborators.push('someone-else');
for (const tok of [f.token(foreign), { sha1: f.api.adminToken }]) {
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-identity-refused' });
assert.deepEqual([f.rec().state, existsSync(f.slot())], ['pending', false]);
}
// The copy's own login, made an administrator.
u.is_admin = true;
await assert.rejects(f.issueAs(f.human, { token: f.token(u).sha1 }), { code: 'credential-identity-refused' });
u.is_admin = false;
// /user names another account than /token did.
const undo = f.api.rule((e) => e.path === '/api/v1/user', { status: 200, json: { id: u.id + 1000, login: u.login, is_admin: false } });
await assert.rejects(f.issueAs(f.human, { token: f.token(u).sha1 }), { code: 'credential-identity-refused' });
undo();
// Another copy's account and token: checker.stack is bound in the same organization.
await f.call(f.owner, 'hire.start.console', { op: 'change', level: 'project', project: 'stack', name: 'checker', reason: 'Bind', binding: { services: READ } });
const checker = f.bot('checker.stack');
const theirs = f.token(checker);
await assert.rejects(f.issueAs(f.human, { token: theirs.sha1 }), { code: 'credential-identity-refused' });
await f.issueAs(f.human, { agent: 'checker.stack', token: theirs.sha1 });
assert.equal(f.rec('checker.stack').state, 'bound');
assert.equal(f.rec().state, 'pending');
assert.notEqual(checker.login, u.login);
});
test('CB2 an opaque or unverifiable token refuses unverified; a malformed one refuses credential-format', async (t) => {
const f = await fixture(t);
const u = f.bot();
// A malformed token refuses before anything is staged or sent (build review 2): the bound record is
// unchanged, nothing reached Gitea and no slot exists.
const before = kept(f), sent = f.api.requests.length;
await assert.rejects(f.issueAs(f.human, { token: 'short' }), { code: 'credential-format' });
await assert.rejects(f.issueAs(f.human, { token: 'has space ' + 'x'.repeat(30) }), { code: 'credential-format' });
assert.deepEqual([kept(f) === before, f.api.requests.length, present(join(f.storeRoot, 'demo', 'rev'))], [true, sent, false]);
await assert.rejects(f.issueAs(f.human, { token: 'f'.repeat(40) }), { code: 'credential-scope-unverified' });
const tok = f.token(u);
for (const answer of [{ status: 200, json: { id: tok.id, scopes: tok.scopes } }, { status: 200, json: { id: tok.id, scopes: tok.scopes, user: { login: u.login } } }, { status: 500, json: null }, { drop: true }]) {
const undo = f.api.rule((e) => e.path === '/api/v1/token', answer);
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-unverified' }, JSON.stringify(answer));
undo();
}
f.opts.timeoutMs = 300;
const hang = f.api.rule((e) => e.path.startsWith('/api/v1/user/repos'), { hang: true });
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-unverified' });
hang();
f.opts.timeoutMs = 3000;
f.api.state.version = '1.26.0';
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-scope-unverified' });
f.api.state.version = '1.27.1';
// The token lacks read:user, which the binding approved: its repositories cannot be listed.
await assert.rejects(f.issueAs(f.human, { token: f.token(u, ['read:repository']).sha1 }), { code: 'credential-scope-unverified' });
assert.deepEqual([f.rec().state, existsSync(f.slot())], ['pending', false]);
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
});
test('CB2 a token any definition already holds refuses before anything is staged or sent', async (t) => {
const f = await fixture(t);
const tok = f.token(f.bot());
const before = readFileSync(f.file, 'utf8');
// The boot copy snapshot's definitions are live text for the bus's lifetime.
f.opts.copies = { ...f.snapshot, definitions: { ...f.snapshot.definitions, leaky: { note: `see ${tok.sha1}` } } };
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'definition-holds-credential' });
assert.deepEqual([readFileSync(f.file, 'utf8') === before, existsSync(f.slot()), f.api.requests.length], [true, false, 0]);
f.opts.copies = null;
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
});
test('CB2 a binding without read:user is unverifiable before anything is staged', async (t) => {
const f = await fixture(t, { services: { gitea: { scopes: ['read:repository'], projects: ['stack'] } } });
const before = readFileSync(f.file, 'utf8');
await assert.rejects(f.issueAs(f.human, { token: f.token(f.bot()).sha1 }), { code: 'credential-scope-unverified' });
assert.equal(readFileSync(f.file, 'utf8'), before);
assert.equal(f.api.requests.length, 0);
});
test('CB2 dates: the rotation date is after today, at most a year ahead, and not after the token expires', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
const before = readFileSync(f.file, 'utf8');
for (const date of [ahead(0), ahead(-1), ahead(MAX_ROTATE_DAYS + 1), '2099-02-30', '2099-1-01', 'tomorrow'])
await assert.rejects(f.issueAs(f.human, { token: tok.sha1, date }), { code: 'credential-date' }, date);
assert.equal(readFileSync(f.file, 'utf8'), before);
assert.equal(f.api.requests.length, 0);
const soon = f.token(u, undefined, { expires: `${ahead(10)}T00:00:00Z` });
await assert.rejects(f.issueAs(f.human, { token: soon.sha1, date: ahead(10) }), { code: 'credential-date' });
await assert.rejects(f.issueAs(f.human, { token: soon.sha1, date: ahead(30) }), { code: 'credential-date' });
const odd = f.token(u, undefined, { expires: 'not-a-date' });
await assert.rejects(f.issueAs(f.human, { token: odd.sha1 }), { code: 'credential-scope-unverified' });
assert.deepEqual([f.rec().state, existsSync(f.slot())], ['pending', false]);
await f.issueAs(f.human, { token: soon.sha1, date: ahead(9) });
assert.equal(f.rec().services.gitea.rotateBy, ahead(9));
await f.issueAs(f.human, { token: f.token(u, undefined, { expires: `${ahead(400)}T00:00:00Z` }).sha1, date: ahead(MAX_ROTATE_DAYS) });
assert.equal(f.rec().state, 'bound');
});
test('CB2 the slot is read back by identity: symlinks, hard links, stale and replaced files refuse credential-file', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
const other = f.token(u);
const elsewhere = join(f.root, 'elsewhere');
const cases = {
symlink: ['afterStage', () => { writeFileSync(elsewhere, tok.sha1 + '\n', { mode: 0o600 }); unlinkSync(f.slot()); symlinkSync(elsewhere, f.slot()); }],
hardlink: ['afterStage', () => { rmSync(elsewhere, { force: true }); linkSync(f.slot(), elsewhere); }],
stale: ['afterStage', () => { renameSync(f.slot(), elsewhere); writeFileSync(f.slot(), tok.sha1 + '\n', { mode: 0o600 }); }],
mode: ['afterStage', () => chmodSync(f.slot(), 0o644)],
// A writer replaces a file by renaming a new one over it, as the store does.
replacedAfterRemote: ['afterRemote', () => { writeFileSync(elsewhere, other.sha1 + '\n', { mode: 0o600 }); renameSync(elsewhere, f.slot()); }],
replacedBeforeCommit: ['beforeCommit', () => { writeFileSync(elsewhere, other.sha1 + '\n', { mode: 0o600 }); renameSync(elsewhere, f.slot()); }],
// Unlinked and written again: the inode number may be reused, the content still differs.
reusedInode: ['afterStage', () => { unlinkSync(f.slot()); writeFileSync(f.slot(), other.sha1 + '\n', { mode: 0o600 }); }],
rewrittenInPlace: ['beforeCommit', () => writeFileSync(f.slot(), other.sha1 + '\n')],
};
for (const [name, [hook, act]] of Object.entries(cases)) {
rmSync(elsewhere, { force: true });
f.hooks[hook] = act;
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'credential-file' }, name);
delete f.hooks[hook];
assert.equal(f.rec().state, 'pending', name);
assert.equal(f.rec().services.gitea.grantId, undefined, name);
// The bus removes only the file it wrote; a file someone else put there stays.
if (['symlink', 'stale', 'replacedAfterRemote', 'replacedBeforeCommit'].includes(name)) assert.ok(present(f.slot()), name);
if (['hardlink', 'mode', 'rewrittenInPlace'].includes(name)) assert.ok(!present(f.slot()), name);
rmSync(f.slot(), { force: true });
}
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
});
test('CB2 crash points leave the record pending and the slot removed; a new verified issue recovers', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
for (const hook of ['afterStage', 'afterRemote', 'beforeCommit']) {
f.hooks[hook] = () => { throw new Error('crash'); };
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'outcome-unknown' }, hook);
delete f.hooks[hook];
assert.deepEqual([f.rec().state, f.rec().services.gitea.grantId, existsSync(f.slot())], ['pending', undefined, false], hook);
}
// A hard crash after the stage (security review P2): the record is pending and the staged token is
// on disk. The next start's sweep removes it before anything loads; the record stays pending and
// only a fresh verified issue binds.
const r = f.rec();
const { revision } = f.writer.read('demo');
f.writer.stageCopyGrant('demo', { instance: 'rev', service: 'gitea', copyId: r.copyId, generation: r.generation, decision: r.decision, incarnation: r.orgIncarnation }, { revision, actor: 'jason', owner: true });
f.credentials.stageCopy('demo/rev', 'gitea', tok.sha1);
assert.equal(f.rec().state, 'pending');
assert.deepEqual(sweepCopySlots(f.storeRoot, [['demo', f.writer.read('demo').document]], {}), [{ instance: 'demo/rev', service: 'gitea' }]);
assert.deepEqual([f.rec().state, present(f.slot())], ['pending', false]);
await f.issueAs(f.human, { token: tok.sha1 });
assert.deepEqual([f.rec().state, f.rec().services.gitea.grantGeneration], ['bound', 1]);
// A lost reply: the commit landed but the caller never heard. Issuing again re-verifies.
await f.issueAs(f.human, { token: tok.sha1 });
assert.deepEqual([f.rec().state, f.rec().services.gitea.grantGeneration], ['bound', 2]);
});
test('CB2 the start sweep removes only ungranted copy slots and staged temp files', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
await f.issueAs(f.human, { token: tok.sha1 });
assert.equal(f.rec().state, 'bound');
const doc = () => [['demo', f.writer.read('demo').document]];
const copyDir = join(f.storeRoot, 'demo', 'rev');
// A granted slot stays; an interrupted write's temp file goes; a name that is no service stays.
writeFileSync(join(copyDir, 'gitea.tmp'), 'x', { mode: 0o600 });
writeFileSync(join(copyDir, 'notes'), 'x', { mode: 0o600 });
assert.deepEqual(sweepCopySlots(f.storeRoot, doc(), {}), [{ instance: 'demo/rev', service: 'gitea' }]);
assert.deepEqual([present(f.slot()), present(join(copyDir, 'gitea.tmp')), present(join(copyDir, 'notes'))], [true, false, true]);
// An ungranted service of a bound copy goes; a slot a loaded reference names stays.
writeFileSync(join(copyDir, 'vikunja'), 'x', { mode: 0o600 });
assert.deepEqual(sweepCopySlots(f.storeRoot, doc(), { 'demo/rev': { vikunja: { service: 'vikunja' } } }), []);
assert.deepEqual(sweepCopySlots(f.storeRoot, doc(), {}), [{ instance: 'demo/rev', service: 'vikunja' }]);
// A role's own slots and an unreadable organization are never touched.
const roleDir = join(f.storeRoot, 'demo', 'chief');
mkdirSync(roleDir, { mode: 0o700 }); writeFileSync(join(roleDir, 'gitea'), 'x', { mode: 0o600 });
const edited = { ...f.writer.read('demo').document };
edited.copyBindings = { ...edited.copyBindings, chief: { ...edited.copyBindings.rev, services: {} } };
assert.deepEqual(sweepCopySlots(f.storeRoot, [['demo', edited]], {}), []);
assert.equal(present(join(roleDir, 'gitea')), true);
writeFileSync(join(copyDir, 'gitea.tmp'), 'x', { mode: 0o600 });
assert.deepEqual(sweepCopySlots(f.storeRoot, [['demo', null]], {}), []);
assert.equal(present(join(copyDir, 'gitea.tmp')), true);
// No store, a missing org dir: nothing. A store dir that isn't private refuses as Credentials would.
assert.deepEqual(sweepCopySlots(null, doc(), {}), []);
assert.deepEqual(sweepCopySlots(f.storeRoot, [['other', f.writer.read('demo').document]], {}), []);
chmodSync(copyDir, 0o755);
assert.throws(() => sweepCopySlots(f.storeRoot, doc(), {}), /credential-file/);
chmodSync(copyDir, 0o700);
});
test('CB2 a stage write that fails leaves no staged token behind', async (t) => {
const f = await fixture(t);
// The slot path is a non-empty directory: the rename fails after the temp file holds the token.
mkdirSync(join(f.slot(), 'x'), { recursive: true, mode: 0o700 });
assert.throws(() => f.credentials.stageCopy('demo/rev', 'gitea', 'a'.repeat(40)));
assert.deepEqual(readdirSync(join(f.storeRoot, 'demo', 'rev')), ['gitea']);
});
test('CB2 a rebind, a failed owner recheck or a concurrent issue during the remote work refuses the commit', async (t) => {
const f = await fixture(t);
const u = f.bot();
const tok = f.token(u);
// The owner approves a new binding while Gitea is being asked.
f.hooks.afterRemote = () => f.bind('rev', { gitea: { scopes: ['read:user', 'read:repository'], projects: ['stack'] } });
// The document moved under the commit: its revision check refuses before the binding fence.
await assert.rejects(f.issueAs(f.human, { token: tok.sha1 }), { code: 'revision-conflict' });
delete f.hooks.afterRemote;
assert.deepEqual([f.rec().state, existsSync(f.slot())], ['pending', false]);
// The caller's owner recheck fails after the remote work (the broker's #copyOwner), and again
// when it fails only under the writer's locks.
for (const at of [1, 2]) {
let n = 0;
await assert.rejects(f.direct({ agent: 'rev', service: 'gitea', token: tok.sha1, date: ahead(30) }, { actor: 'jason', owner: true, recheck: () => { if (++n >= at) throw new BusError('human-required'); } }), { code: 'human-required' });
assert.deepEqual([f.rec().state, existsSync(f.slot())], ['pending', false]);
}
// Not the owner, or the tester's name, never reaches the writer.
for (const ctx of [{ actor: 'jason', owner: false }, { actor: 'tester', owner: true }])
await assert.rejects(f.direct({ agent: 'rev', service: 'gitea', token: tok.sha1, date: ahead(30) }, ctx), { code: 'human-required' });
// Two issues: the first is held after its remote work while the second binds.
const second = f.token(u);
let release, first = true;
const held = new Promise((r) => { release = r; });
f.hooks.afterRemote = () => { if (first) { first = false; return held; } };
const a = f.issueAs(f.human, { token: tok.sha1 });
await new Promise((r) => setTimeout(r, 50));
while (f.hooks.afterRemote && first) await new Promise((r) => setTimeout(r, 10));
await f.issueAs(f.human, { token: second.sha1 });
release();
await assert.rejects(a, (e) => ['revision-conflict', 'credential-binding-changed', 'credential-file'].includes(e.code));
delete f.hooks.afterRemote;
assert.equal(f.rec().state, 'bound');
assert.equal(f.rec().services.gitea.grantId, `gitea:${u.id}:${second.id}`);
assert.equal(readFileSync(f.slot(), 'utf8'), second.sha1 + '\n', 'the second issue\'s slot stays');
});
test('CB2 only the organization owner issues: the chief, the tester, a reader and an org.hire member are refused before staging', async (t) => {
const f = await fixture(t);
const tok = f.token(f.bot());
const before = readFileSync(f.file, 'utf8');
const member = (owner) => f.b.bindHireConsole({ business: 'demo', actor: 'member', subject: { ...SUBJECT, sub: 'member' }, rights: ['org.hire', 'decide'], owner, instanceOwner: true });
const got = [];
for (const cap of [f.chief, f.b.bindTester({ business: 'demo' }), f.b.bindReader({ business: 'demo' }), member(false), member(true), f.owner]) {
await assert.rejects(f.issueAs(cap, { token: tok.sha1 }), (e) => { got.push(e.code); return !String(e.message).includes(tok.sha1); });
}
// The Console caps, the owner's included, never reach credentials.issue: the token comes from the
// owner's CLI on standard input.
assert.deepEqual(got, ['human-required', 'tester-refused', 'human-required', 'human-required', 'human-required', 'human-required']);
assert.equal(readFileSync(f.file, 'utf8'), before);
assert.equal(f.api.requests.length, 0);
assert.ok(!existsSync(join(f.storeRoot, 'demo')));
// Only a name the boot's copies view holds goes to the copy verifier; any other unknown name
// answers credential-unknown as before, and nothing is staged or sent.
await assert.rejects(f.issueAs(f.human, { agent: 'nobody', token: tok.sha1 }), { code: 'credential-unknown' });
assert.deepEqual([readFileSync(f.file, 'utf8') === before, f.api.requests.length, existsSync(join(f.storeRoot, 'demo'))], [true, 0, false]);
});
test('CB2 refusal table: the new codes have rails and next steps that name no path', () => {
for (const code of ['credential-scope-unverified', 'credential-identity-refused', 'credential-date', 'credential-format', 'credential-file', 'copy-unavailable']) {
const g = REFUSALS[code];
assert.ok(g && g.rail.length && g.nextStep.length, code);
assert.doesNotMatch(g.nextStep, /\/|~|\.json\b|secrets/, code);
}
});
+29 -2
View File
@@ -4,13 +4,15 @@
// scratch root only.
import test from 'node:test';
import assert from 'node:assert/strict';
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, statSync } from 'node:fs';
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { startBroker } from '../src/runtime.mjs';
import { Broker, CREDENTIAL_VERBS } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { businessDoc, rolesCopy, systemFor, writeJson } from '../../organization/tests/helpers.mjs';
import { definitionPin } from '../../organization/src/copy-bindings.mjs';
import { readTemplate } from '../../organization/src/definitions.mjs';
const TOKEN = 'fake_issued_token_0123456789';
const ROTATED = 'fake_rotated_token_9876543210';
@@ -18,16 +20,18 @@ const OTHER = 'fake_other_token_5555555555';
const DATE = '2099-01-01';
const role = { authority: { withinRole: ['message.send'], crossRole: [] } };
async function setup(t, { storeRoot = true } = {}) {
async function setup(t, { storeRoot = true, before = () => {}, edit = () => {} } = {}) {
const root = mkdtempSync(join(tmpdir(), 'bus-cred-verbs-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const rolesDir = rolesCopy(root);
const doc = { ...businessDoc(root, 'demo'), plugins: { harness: ['pi'] } };
edit(doc, rolesDir);
doc.roles.coder.credentials.gitea = { store: true, rotateBy: DATE };
writeJson(join(root, 'businesses/demo.json'), doc);
mkdirSync(join(root, 'data'), { mode: 0o700 });
if (storeRoot) mkdirSync(join(root, 'secrets', 'agents'), { mode: 0o700 });
const store = (agent, service) => join(root, 'secrets', 'agents', 'demo', agent, service);
before(store);
// The bus's resolved business carries the same references; a store one names its derived file.
const demo = {
id: 'demo', human: 'jason', arbiters: { technical: 'chief', delivery: 'chief' }, vars: { 'bus.tester': true },
@@ -187,3 +191,26 @@ test('#1886: a store root that exists with loose modes still refuses start', asy
};
await assert.rejects(startBroker({ dataRoot: join(root, 'data'), businesses: { demo }, organization: { dir: root, rolesDir, system: systemFor(root) } }), /credential-file/);
});
test('CB2 security review P2: the start sweep removes a copy slot a crashed issue left staged', async (t) => {
// A copy binding left pending by an issue killed after its stage, its token and a temp file on disk.
const pending = (doc, rolesDir) => {
doc.copyBindings = { rev: {
copyId: 'c'.repeat(32), orgIncarnation: 'a'.repeat(32), source: { level: 'root', name: 'rev', project: null },
definition: definitionPin('reviewer', { definitions: {}, rolesDir, readTemplate }), generation: 2, state: 'pending',
services: { gitea: { slot: 'self', scopes: ['read:repository', 'read:user'], resources: { projects: ['stack'] } } },
decision: '11111111-1111-4111-8111-111111111111' } };
};
const staged = (store) => {
mkdirSync(join(store('rev', 'gitea'), '..'), { recursive: true, mode: 0o700 });
chmodSync(join(store('rev', 'gitea'), '..', '..'), 0o700);
for (const name of ['gitea', 'gitea.tmp']) writeFileSync(store('rev', name), TOKEN + '\n', { mode: 0o600 });
mkdirSync(join(store('coder', 'gitea'), '..'), { recursive: true, mode: 0o700 });
writeFileSync(store('coder', 'gitea'), TOKEN + '\n', { mode: 0o600 });
};
const c = await setup(t, { edit: pending, before: staged });
assert.equal(c.file().copyBindings.rev.state, 'pending');
assert.deepEqual(readdirSync(join(c.store('rev', 'gitea'), '..')), []);
// A role's own store slot is the store's, not the sweep's.
assert.equal(existsSync(c.store('coder', 'gitea')), true);
});
+8 -4
View File
@@ -38,13 +38,14 @@ export async function fakeGiteaApi(t, dir, { version = '1.27.1', admin = 'mosaic
issued: [],
// A user and, with `scopes`, one token named `tokenName`. Returns the user and that token or null.
addUser(login, { is_admin = false, prohibit_login = false, full_name = login, email = `${login}@example.test`, scopes = null, tokenName = 'test' } = {}) {
const u = { id: nextId++, login, email, full_name, is_admin, prohibit_login, tokens: [] };
const u = { id: nextId++, login, email, full_name, is_admin, prohibit_login, tokens: [], orgs: [] };
users.set(lower(login), u);
return { user: u, token: scopes ? api.addToken(login, tokenName, scopes) : null };
},
addToken(login, name, scopes) {
// #1873 CB2: `expires` (an ISO instant) is reported as the token's expires_at.
addToken(login, name, scopes, { expires = null } = {}) {
const u = users.get(lower(login));
const tok = { id: nextId++, name, sha1: fakeToken(), scopes: [...scopes].sort() };
const tok = { id: nextId++, name, sha1: fakeToken(), scopes: [...scopes].sort(), expires };
u.tokens.push(tok);
api.issued.push(tok.sha1);
return tok;
@@ -96,7 +97,10 @@ export async function fakeGiteaApi(t, dir, { version = '1.27.1', admin = 'mosaic
if (!who) return [401, { message: 'unauthorized' }];
const me = who.user;
if (method === 'GET' && path === '/api/v1/user') return [200, userJson(me)];
if (method === 'GET' && path === '/api/v1/token') return [200, { id: who.token.id, name: who.token.name, scopes: who.token.scopes }];
// Gitea 1.27 names the token's owner; #1873 CB2's verifier needs it.
if (method === 'GET' && path === '/api/v1/token')
return [200, { id: who.token.id, name: who.token.name, scopes: who.token.scopes, user: { id: me.id, login: me.login }, ...(who.token.expires ? { expires_at: who.token.expires } : {}) }];
if (method === 'GET' && path === '/api/v1/user/orgs') return [200, (me.orgs ?? []).map((name, i) => ({ id: 1000 + i, username: name })), (me.orgs ?? []).length];
if (method === 'GET' && path === '/api/v1/user/repos') {
const all = reachable(me);
const limit = Number(query.get('limit') ?? 30);
+13 -1
View File
@@ -1299,7 +1299,19 @@ require the source and deleted folder to be on the same filesystem.
`mosaic org archive` and `mosaic org delete` stop the organization bus, so like
`bus stop` they refuse inside an agent run (exit 3, agent-guard #1873), after
their usage checks and before any config is read. `org restore` is unchanged.
their usage checks and before any config is read.
`mosaic org restore ID --if-revision N` makes the organization active and starts
its bus, as `mosaic bus start ID --dev-local-owner --detach` does (E1, P1-4). It
makes that verb's checks before the restore, so a stamped build or an instance
that refuses `--dev-local-owner` refuses the restore before anything changes
(exit 3). Starting a bus is a human action, so inside an agent run `org restore`
refuses like `bus start` (exit 3) before any config is read. The receipt's `bus`
names the outcome: `started` with the host pid, or `failed` with the message and
the verb that starts it (after `mosaic bus recover ID` when a host that died left a
stale lock). A bus that does not start leaves the restore in place;
the command prints the receipt and exits 3. `org restore ID --if-revision N
--no-bus` restores without starting the bus, as before, and stays open to agents.
System types are installed plug-ins. `mosaic org system-types installed` prints
module metadata; `mosaic org system-types enabled ID` prints an organization's enabled
+32 -6
View File
@@ -6,19 +6,28 @@ import { installedSystemTypes } from '../../plugins/src/system-types.mjs';
import { DEV_OWNER, devOwnerInstance } from './dev-local-owner.mjs';
import { loadSystem } from './config.mjs';
import { CliError } from './errors.mjs';
import { busStopForDelete } from './org-bus.mjs';
import { busStart, busStopForDelete } from './org-bus.mjs';
import { assertBuildStart, assertDevInstance } from './deployment.mjs';
import { consoleBusRestart } from './org-console.mjs';
import { refuseInsideAgent } from './transport.mjs';
import { definitionsCommand, DEFINITIONS_USAGE } from './org-definitions.mjs';
export const ORG_USAGE = 'mosaic org system-types installed | system-types enabled ID | lock show|clear ID | reload ID | archive ID --if-revision N [--json] | restore ID --if-revision N [--json] | delete ID --confirm ID [--if-revision N] [--json] | template render ID [--fixture local] --output FILE | validate FILE | create ID --from FILE | show ID [--json] | list [--json] | update ID --from FILE --if-revision N | queue init ID (--local-only --accept-local-loss-risk | --remote URL) | queue show ID [--json] | project repo init ID PROJECT (--local-only --accept-local-loss-risk | --remote URL) | project repo show ID PROJECT [--json] | ' + DEFINITIONS_USAGE.slice('mosaic org '.length) + ' | config validate ID | config resolve ID INSTANCE [--project ID]';
function parse(args) { const pos = [], values = {}, flags = new Set(); for (let i = 0; i < args.length; i++) { const a = args[i]; if (['--json', '--local-only', '--accept-local-loss-risk'].includes(a)) { if (flags.has(a)) throw new CliError(`duplicate ${a}`, 4); flags.add(a); } else if (['--fixture', '--output', '--from', '--if-revision', '--remote', '--confirm'].includes(a)) { if (values[a] !== undefined || !args[i+1] || args[i+1].startsWith('--')) throw new CliError(`${a} needs one value`, 4); values[a] = args[++i]; } else if (a.startsWith('--')) throw new CliError(`unknown option ${a}`, 4); else pos.push(a); } return { pos, values, flags }; }
export const ORG_USAGE = 'mosaic org system-types installed | system-types enabled ID | lock show|clear ID | reload ID | archive ID --if-revision N [--json] | restore ID --if-revision N [--no-bus] [--json] | delete ID --confirm ID [--if-revision N] [--json] | template render ID [--fixture local] --output FILE | validate FILE | create ID --from FILE | show ID [--json] | list [--json] | update ID --from FILE --if-revision N | queue init ID (--local-only --accept-local-loss-risk | --remote URL) | queue show ID [--json] | project repo init ID PROJECT (--local-only --accept-local-loss-risk | --remote URL) | project repo show ID PROJECT [--json] | ' + DEFINITIONS_USAGE.slice('mosaic org '.length) + ' | config validate ID | config resolve ID INSTANCE [--project ID]';
function parse(args) { const pos = [], values = {}, flags = new Set(); for (let i = 0; i < args.length; i++) { const a = args[i]; if (['--json', '--local-only', '--accept-local-loss-risk', '--no-bus'].includes(a)) { if (flags.has(a)) throw new CliError(`duplicate ${a}`, 4); flags.add(a); } else if (['--fixture', '--output', '--from', '--if-revision', '--remote', '--confirm'].includes(a)) { if (values[a] !== undefined || !args[i+1] || args[i+1].startsWith('--')) throw new CliError(`${a} needs one value`, 4); values[a] = args[++i]; } else if (a.startsWith('--')) throw new CliError(`unknown option ${a}`, 4); else pos.push(a); } return { pos, values, flags }; }
// A refusal names the document by label and the error by its code, never by host path or bytes (PR #1916).
const errorCode = e => typeof e?.code === 'string' && /^E[A-Z0-9]+$/.test(e.code) ? e.code : 'unreadable';
async function document(file) { try { return JSON.parse(await readFile(file, 'utf8')); } catch (e) { throw new CliError(`cannot read organization document: ${e instanceof SyntaxError ? 'not valid JSON' : errorCode(e)}`, 2); } }
// agent-guard (#1873): archive and delete stop the organization's bus, so an agent run is refused
// as it is for `mosaic bus stop`: after the usage checks, before any config is read, and checking
// this process's own environment as well as io.env. Restore starts nothing and stays as it was.
// this process's own environment as well as io.env. Restore starts the organization's bus (E1, P1-4),
// so it is refused the same way unless --no-bus is given; `restore --no-bus` starts nothing.
function refuseAgentRun(io) { refuseInsideAgent(process.env); refuseInsideAgent(io.env); }
// E1 (P1-4): the bus a restore starts is the one `mosaic bus start ID --dev-local-owner --detach`
// starts, after that verb's checks. The checks run before the restore, so a refusal changes nothing.
function restoreBusCheck(system, env) {
try { assertBuildStart({ devOwner: true }); assertDevInstance(system, env); }
catch (e) { throw new CliError(`restore starts the bus as the local dev owner, and ${e.message}; restore with --no-bus, then start the bus with mosaic bus start`, 3); }
}
const busStartVerb = id => `mosaic bus start ${id} --dev-local-owner --detach`;
export async function orgCommand(args, io, deps = {}) {
try {
// #1873 T1: the definitions verbs take their own options.
@@ -32,7 +41,7 @@ export async function orgCommand(args, io, deps = {}) {
if (words[0] === 'reload') refuseInsideAgent(io.env);
const revision = (r, required) => { if (r === undefined && !required) return undefined; if (!/^[1-9][0-9]*$/.test(r ?? '')) throw new CliError(r === undefined ? '--if-revision is required' : 'invalid revision', 4); return Number(r); };
let lifecycle = null;
if (['archive', 'restore'].includes(words[0]) && words.length === 2) { allowed(['--json'], ['--if-revision']); lifecycle = { ifRevision: revision(p.values['--if-revision'], true) }; if (words[0] === 'archive') refuseAgentRun(io); }
if (['archive', 'restore'].includes(words[0]) && words.length === 2) { allowed(['--json', ...(words[0] === 'restore' ? ['--no-bus'] : [])], ['--if-revision']); lifecycle = { ifRevision: revision(p.values['--if-revision'], true) }; if (words[0] === 'archive' || !p.flags.has('--no-bus')) refuseAgentRun(io); }
else if (words[0] === 'delete' && words.length === 2) { allowed(['--json'], ['--confirm', '--if-revision']); const ifRevision = revision(p.values['--if-revision'], false); lifecycle = { confirm: need('--confirm'), ifRevision }; refuseAgentRun(io); }
if (words[0] === 'system-types' && words[1] === 'installed' && words.length === 2) { allowed(['--json'], []); io.stdout.write(JSON.stringify(await installedSystemTypes(), null, 2) + '\n'); return; }
const instance = devOwnerInstance(io.env); const system = deps.system ?? loadSystem({ env: io.env, instance }); const ctx = { configDir: configDir(io.env), dataRoot: system.dataRoot, instance, principal: DEV_OWNER };
@@ -40,7 +49,24 @@ export async function orgCommand(args, io, deps = {}) {
else if (words[0] === 'lock' && ['show', 'clear'].includes(words[1]) && words.length === 3) { allowed([], []); result = await orgLock(ctx, words[2], { clear: words[1] === 'clear' }); }
else if (words[0] === 'create' && words.length === 2) { allowed([], ['--from']); result = await createOrg(ctx, words[1], await document(need('--from'))); }
else if (words[0] === 'update' && words.length === 2) { allowed([], ['--from', '--if-revision']); const r = need('--if-revision'); if (!/^[1-9][0-9]*$/.test(r)) throw new CliError('invalid revision', 4); result = await updateOrg(ctx, words[1], await document(need('--from')), { ifRevision: Number(r) }); }
else if (['archive', 'restore'].includes(words[0]) && words.length === 2) result = words[0] === 'archive' ? await archiveOrg(ctx, words[1], { ...lifecycle, busStop: busStopForDelete }) : await restoreOrg(ctx, words[1], lifecycle);
else if (words[0] === 'archive' && words.length === 2) result = await archiveOrg(ctx, words[1], { ...lifecycle, busStop: busStopForDelete });
else if (words[0] === 'restore' && words.length === 2) {
const startBus = !p.flags.has('--no-bus');
if (startBus) restoreBusCheck(system, io.env);
result = await restoreOrg(ctx, words[1], lifecycle);
// The restore stands whatever the bus does. A bus that does not start is said so, with the verb
// that starts it, and the command exits 3 after printing the restore receipt.
if (startBus) {
try { const started = await (deps.busStart ?? busStart)({ ...ctx, env: io.env }, words[1], { detach: true, devLocalOwner: true, pm: false }); result = { ...result, bus: { outcome: 'started', pid: started.pid } }; }
catch (e) {
// A host that died left its lock behind; `bus start` refuses it until `bus recover` clears it.
const message = e instanceof CliError ? e.message : 'the bus did not start', dead = /^stale /.test(message);
result = { ...result, bus: { outcome: 'failed', message, next: dead ? `mosaic bus recover ${words[1]}; ${busStartVerb(words[1])}` : busStartVerb(words[1]) } };
io.stdout.write(JSON.stringify(result, null, 2) + '\n');
throw new CliError(`${words[1]} was restored, but its bus did not start: ${message}; ${dead ? `clear the dead host with mosaic bus recover ${words[1]}, then start it with` : 'start it with'} ${busStartVerb(words[1])}`, 3);
}
}
}
else if (words[0] === 'delete' && words.length === 2) result = await deleteOrg(ctx, words[1], { busStop: busStopForDelete, ...lifecycle });
// U1 (#1873): the F3 bus control from a terminal, through the running Console's own route, which
// checks the owner and writes the action trail (org-console.mjs consoleBusRestart).
+25 -2
View File
@@ -40,6 +40,8 @@ const GUARDED = [
// archive and delete stop the organization's bus (review finding 2).
['org', 'archive', 'alpha', '--if-revision', '1'],
['org', 'delete', 'alpha', '--confirm', 'alpha'],
// E1 (P1-4): restore starts the organization's bus unless --no-bus is given.
['org', 'restore', 'alpha', '--if-revision', '1'],
];
// The CLI in this process, with the given io.env; stdout and stderr are collected.
@@ -129,7 +131,7 @@ test('consoleStart refuses a marker in its own process whatever environment it i
test('usage errors still come first, so an agent sees the usage line, not the refusal', async () => {
const env = { ...NOWHERE(), MOSAIC_AGENT_RUN: '1' };
for (const args of [['bus', 'stop'], ['bus', 'start'], ['bus', 'start', 'alpha', '--pm', '--no-launch'], ['console', 'stop', 'extra'], ['console', 'restart', 'extra'], ['console', 'start', '--dev-insecure-bind', '0.0.0.0'], ['console', 'start', '--dev-allow-host', 'x'], ['console', 'start', 'extra'], ['org', 'archive', 'alpha'], ['org', 'delete', 'alpha'], ['org', 'delete', 'alpha', '--confirm', 'alpha', '--if-revision', 'x']]) {
for (const args of [['bus', 'stop'], ['bus', 'start'], ['bus', 'start', 'alpha', '--pm', '--no-launch'], ['console', 'stop', 'extra'], ['console', 'restart', 'extra'], ['console', 'start', '--dev-insecure-bind', '0.0.0.0'], ['console', 'start', '--dev-allow-host', 'x'], ['console', 'start', 'extra'], ['org', 'archive', 'alpha'], ['org', 'restore', 'alpha'], ['org', 'restore', 'alpha', '--if-revision', '1', '--no-bus', '--no-bus'], ['org', 'delete', 'alpha'], ['org', 'delete', 'alpha', '--confirm', 'alpha', '--if-revision', 'x']]) {
const r = await mosaic(env, args, 30000);
assert.equal(r.status, 4, `${args.join(' ')}: ${r.stderr}`);
}
@@ -247,8 +249,29 @@ test("the Console's Restart bus works: the new host runs with the Console's envi
// An archived organization has no bus status, so the old host's /proc entry shows it stopped.
for (let i = 0; i < 100 && existsSync(`/proc/${after.pid}`); i++) await new Promise(r => setTimeout(r, 100));
assert.ok(!existsSync(`/proc/${after.pid}`), 'archive from a human shell stops the bus host');
const restored = await mosaic(env, ['org', 'restore', 'alpha', '--if-revision', String(JSON.parse(archived.stdout).revision), '--json']);
// E1 (P1-4): restore starts the bus, so an agent run is refused before anything changes, and
// `--no-bus` (which starts nothing) is the only restore an agent run may make.
const archivedRevision = JSON.parse(archived.stdout).revision;
for (const marker of ['CLAUDECODE', 'MOSAIC_AGENT_RUN']) {
const r = await mosaic({ ...env, [marker]: '1' }, ['org', 'restore', 'alpha', '--if-revision', String(archivedRevision), '--json']);
assert.equal(r.status, 3, r.stderr); assert.match(r.stderr, REFUSED(marker)); assert.equal(r.stdout, '');
}
const listed = await mosaic(env, ['org', 'list', '--json']); assert.equal(listed.status, 0, listed.stderr);
assert.equal(JSON.parse(listed.stdout).find(o => o.id === 'alpha').state, 'archived', 'the refused restore changed nothing');
const quiet = await mosaic(agent, ['org', 'restore', 'alpha', '--if-revision', String(archivedRevision), '--no-bus', '--json']);
assert.equal(quiet.status, 0, quiet.stderr); assert.equal(JSON.parse(quiet.stdout).bus, null);
assert.equal((await busStatus(ctx, 'alpha')).healthy, false, 'restore --no-bus starts nothing');
// A person's restore starts the bus as the dev owner, with no agent marker in its host.
const again = await mosaic(env, ['org', 'archive', 'alpha', '--if-revision', String((await show()).revision), '--json']);
assert.equal(again.status, 0, again.stderr);
const restored = await mosaic(env, ['org', 'restore', 'alpha', '--if-revision', String(JSON.parse(again.stdout).revision), '--json']);
assert.equal(restored.status, 0, restored.stderr);
const restoredBus = JSON.parse(restored.stdout).bus;
assert.equal(restoredBus.outcome, 'started');
const up = await busStatus(ctx, 'alpha'); assert.equal(up.healthy, true); assert.equal(up.pid, restoredBus.pid);
const hostNames = readFileSync(`/proc/${up.pid}/environ`, 'utf8').split('\0').map(l => l.split('=')[0]);
for (const marker of AGENT_MARKERS) assert.ok(!hostNames.includes(marker), marker);
await busStop({ ...ctx, env }, 'alpha');
// A marker in the environment the host is started with stops it at the guarded entry.
await assert.rejects(busStart({ ...ctx, env: { ...env, MOSAIC_AGENT_RUN: '1' } }, 'alpha', { detach: true, noLaunch: true }), e => e.exitCode === 3 && /bus failed to start/.test(e.message));
assert.match(readFileSync(join(after.root ?? join(ctx.dataRoot, 'org-runtime/alpha'), 'bus-host/host.log'), 'utf8'), /outside any agent run \(found MOSAIC_AGENT_RUN/);
@@ -0,0 +1,94 @@
// E1 (P1-4): `mosaic org restore` brings the organization back running. It starts the bus as
// `mosaic bus start ID --dev-local-owner --detach` would; a bus that does not start is said so with
// that verb, and the restore stands. On an instance where that start is refused, the restore is
// refused first and changes nothing. `--no-bus` keeps the old restore. The bus start itself is a
// stand-in here; agent-guard.test.mjs and org.test.mjs start a real one.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import * as service from '../../organization/src/org-service.mjs';
import { listOrgs } from '../../organization/src/org-registry.mjs';
import { DEV_OWNER } from '../src/dev-local-owner.mjs';
import { orgCommand } from '../src/org.mjs';
import { io, humanEnv } from './helpers.mjs';
// A named instance under a temporary HOME (the dev-owner checks read the instance from HOME), with
// one archived organization.
async function archived(t) {
const home = mkdtempSync(join(tmpdir(), 'e1-restore-')), saved = process.env.HOME;
process.env.HOME = home;
t.after(() => { process.env.HOME = saved; rmSync(home, { recursive: true, force: true }); });
const configDir = join(home, '.config', 'mosaic-e1test'), dataRoot = join(home, 'data');
mkdirSync(configDir, { recursive: true, mode: 0o700 }); mkdirSync(dataRoot, { mode: 0o700 });
const ctx = { configDir, dataRoot, instance: 'e1test', principal: DEV_OWNER };
const created = await service.createOrg(ctx, 'alpha', await service.templateRender('alpha'));
const gone = await service.archiveOrg(ctx, 'alpha', { ifRevision: created.revision, busStop: async () => ({ stopped: false }) });
const env = humanEnv({ HOME: home, MOSAIC_CONFIG: join(configDir, 'config.json') });
const state = async () => (await listOrgs(ctx)).find(e => e.id === 'alpha').state;
return { ctx, env, revision: gone.revision, state, system: { dataRoot } };
}
const restore = (f, deps, ...extra) => { const x = io(f.env); return orgCommand(['restore', 'alpha', '--if-revision', String(f.revision), '--json', ...extra], x, { system: f.system, ...deps }).then(() => x, e => { e.io = x; throw e; }); };
test('E1: restore starts the bus as the dev owner and its receipt names the bus', async t => {
const f = await archived(t), calls = [];
const x = await restore(f, { busStart: async (ctx, id, opts) => { calls.push([ctx.dataRoot, ctx.env === f.env, id, opts]); return { org: id, pid: 4242, socket: '/s', root: '/r' }; } });
assert.deepEqual(calls, [[f.ctx.dataRoot, true, 'alpha', { detach: true, devLocalOwner: true, pm: false }]]);
const receipt = JSON.parse(x.out.text);
assert.equal(receipt.revision, f.revision + 1);
assert.deepEqual(receipt.bus, { outcome: 'started', pid: 4242 });
assert.equal(await f.state(), 'active');
});
test('E1: a bus that does not start after a good restore is said so, with the verb, and the restore stands', async t => {
const f = await archived(t);
const { CliError } = await import('../src/errors.mjs');
const busStart = async () => { throw new CliError('bus failed to start; see the bus log', 3); };
await assert.rejects(restore(f, { busStart }), e => {
assert.equal(e.exitCode, 3);
assert.equal(e.message, 'alpha was restored, but its bus did not start: bus failed to start; see the bus log; start it with mosaic bus start alpha --dev-local-owner --detach');
// The restore receipt is printed before the refusal.
const receipt = JSON.parse(e.io.out.text);
assert.equal(receipt.revision, f.revision + 1);
assert.deepEqual(receipt.bus, { outcome: 'failed', message: 'bus failed to start; see the bus log', next: 'mosaic bus start alpha --dev-local-owner --detach' });
return true;
});
assert.equal(await f.state(), 'active');
});
test('E1: a bus host that died leaves a stale lock; the receipt names bus recover before the start', async t => {
const f = await archived(t);
const { CliError } = await import('../src/errors.mjs');
const busStart = async () => { throw new CliError('stale bus host lock', 3); };
await assert.rejects(restore(f, { busStart }), e => {
assert.equal(e.exitCode, 3);
assert.equal(e.message, 'alpha was restored, but its bus did not start: stale bus host lock; clear the dead host with mosaic bus recover alpha, then start it with mosaic bus start alpha --dev-local-owner --detach');
assert.deepEqual(JSON.parse(e.io.out.text).bus, { outcome: 'failed', message: 'stale bus host lock', next: 'mosaic bus recover alpha; mosaic bus start alpha --dev-local-owner --detach' });
return true;
});
assert.equal(await f.state(), 'active');
});
test('E1: where a dev-owner bus start is refused, restore is refused before it changes anything, and --no-bus restores', async t => {
const f = await archived(t); let starts = 0;
const busStart = async () => { starts++; return { pid: 1 }; };
writeFileSync(join(f.ctx.dataRoot, '.deployed'), '{}\n');
await assert.rejects(restore(f, { busStart }), e => e.exitCode === 3 && /^restore starts the bus as the local dev owner, and --dev-local-owner refuses \.deployed; restore with --no-bus, then start the bus with mosaic bus start$/.test(e.message) && e.io.out.text === '');
assert.equal(await f.state(), 'archived');
const x = await restore(f, { busStart }, '--no-bus');
assert.equal(JSON.parse(x.out.text).bus, null);
assert.equal(starts, 0);
assert.equal(await f.state(), 'active');
});
test('E1: an agent run may restore only with --no-bus', async t => {
const f = await archived(t); let starts = 0;
const busStart = async () => { starts++; return { pid: 1 }; };
f.env = { ...f.env, MOSAIC_AGENT_RUN: '1' };
await assert.rejects(restore(f, { busStart }), e => e.exitCode === 3 && /MOSAIC_AGENT_RUN/.test(e.message));
assert.equal(await f.state(), 'archived');
await restore(f, { busStart }, '--no-bus');
assert.equal(starts, 0);
assert.equal(await f.state(), 'active');
});
+2
View File
@@ -21,6 +21,8 @@ test('organization CLI renders, validates, creates, updates and initializes git'
info = JSON.parse(invoke('org', 'show', 'alpha', '--json')); const archived = JSON.parse(invoke('org', 'archive', 'alpha', '--if-revision', String(info.revision), '--json')); assert.equal(archived.revision, info.revision + 1);
assert.equal(JSON.parse(invoke('org', 'list', '--json'))[0].state, 'archived');
const restored = JSON.parse(invoke('org', 'restore', 'alpha', '--if-revision', String(archived.revision), '--json')); assert.equal(restored.revision, archived.revision + 1);
// E1 (P1-4): restore brings the organization back running; the delete below stops the bus again.
assert.equal(restored.bus.outcome, 'started'); assert.ok(Number.isInteger(restored.bus.pid)); assert.equal(JSON.parse(invoke('bus', 'status', 'alpha', '--json')).healthy, true);
info = JSON.parse(invoke('org', 'show', 'alpha', '--json')); const receipt = JSON.parse(invoke('org', 'delete', 'alpha', '--confirm', 'alpha', '--if-revision', String(info.revision), '--json')); assert.equal(receipt.org, 'alpha'); assert.deepEqual(JSON.parse(invoke('org', 'list', '--json')), []);
});
test('business and org aliases conflict in either order', () => { for (const args of [['--business','alpha','--org','beta'], ['--org','beta','--business','alpha']]) { const r = run(process.env, 'tasks', ...args); assert.notEqual(r.status, 0); assert.match(r.stderr, /conflict/i); } });
+43 -8
View File
@@ -7,8 +7,8 @@
// never a token, a token fragment or digest, a path or an environment name.
//
// { copyId, orgIncarnation, source: { level, name, project }, definition: { ref, rev, sha256 },
// generation, state, services: { gitea?: { slot, scopes, resources }, vikunja?: {...} },
// decision, tracker? }
// generation, state, services: { gitea?: { slot, scopes, resources, rotateBy?, grantId?,
// grantGeneration? }, vikunja?: {...} }, decision, tracker? }
//
// copyId is 128 random bits minted on the first approval for that copy; it is kept, with the
// generation counting up, through every later change and the unbound tombstone. orgIncarnation is
@@ -19,9 +19,15 @@
// (composition, instructions and personality), a role template by the hash readTemplate takes over
// its file and contract.
//
// States: `pending` is approved and not verified. Only a verifier (slice CB2, not built) may turn a
// record `bound`, and only `bound` resolves. `unbound` is the tombstone an unbind leaves. Nothing
// in CB1 writes `bound`, so no copy binding resolves and no copy launches with one.
// States: `pending` is approved and not verified. Only a verifier (slice CB2) may turn a record
// `bound`, and only `bound` resolves. `unbound` is the tombstone an unbind leaves.
//
// CB2 (grant metadata): a service the verifier proved carries `rotateBy` (the owner's rotation day),
// `grantId` (the service's opaque grant id, `gitea:<user id>:<token id>`: ids the service assigned,
// never token bytes) and `grantGeneration` (counts every verified grant of that service; it stays
// when a new issue strips the other two). The verifier's commit writes `bound` only when every
// service holds a grant, and no grant id appears twice across the organization's copies. Launch
// admission, which re-proves each grant, is CB3.
//
// Everything here is pure: no file is read, no secret, no slot.
@@ -39,6 +45,11 @@ const BOT_NAME = /^bot-[a-z0-9][a-z0-9._-]{0,60}$/;
const GITEA_SCOPE = /^(read|write):([a-z]+)$/;
const RECORD_KEYS = ["copyId", "orgIncarnation", "source", "definition", "generation", "state", "services", "decision", "tracker"];
const SERVICE_KEYS = ["slot", "scopes", "resources"];
const GRANT_KEYS = ["rotateBy", "grantId", "grantGeneration"];
const GRANT_ID = /^(gitea|vikunja):[1-9][0-9]{0,15}:[1-9][0-9]{0,15}$/;
const DAY = /^\d{4}-\d{2}-\d{2}$/;
const realDay = (value) => typeof value === "string" && DAY.test(value) && !Number.isNaN(Date.parse(value))
&& new Date(value).toISOString().slice(0, 10) === value;
export const BINDING_STATES = Object.freeze(["pending", "bound", "unbound"]);
// CB1 binds the core services (vocabulary SERVICES) only. A plugin service has no verifier design yet.
export const BINDABLE_SERVICES = SERVICES;
@@ -46,6 +57,18 @@ export const MAX_BINDINGS = 256;
export const newCopyId = () => randomBytes(16).toString("hex");
// CB2: the one service account a copy's binding may use, derived from the registry incarnation, the
// copyId and the service, so another organization, another copy and another service never share
// it. Gitea limits a login to 40 characters; this is 36. Not a secret and not token-derived.
export function copyBotLogin(incarnation, copyId, service) {
if (typeof incarnation !== "string" || !INCARNATION.test(incarnation) || typeof copyId !== "string" || !COPY_ID.test(copyId)
|| !BINDABLE_SERVICES.includes(service)) return null;
return `mosaic-copy-${sha256(`copy-bot\0${incarnation}\0${copyId}\0${service}`).slice(0, 24)}`;
}
// Whether every service of a binding record holds a verified grant (CB2).
export const allGranted = (services) => isPlainObject(services) && Object.keys(services).length > 0
&& Object.values(services).every((v) => isPlainObject(v) && typeof v.grantId === "string");
// Fixed text for every binding refusal. `service` is a registered service label or null, never a
// submitted value.
const TEXT = {
@@ -53,6 +76,8 @@ const TEXT = {
"credential-unbound": (s) => `credentials.${s} needs an owner-approved binding.`,
"credential-scope-exceeded": (s) => `credentials.${s}.scopes exceeds the approved role.`,
"credential-scope-unverified": (s) => `credentials.${s} permissions could not be verified.`,
"credential-identity-refused": (s) => `credentials.${s} is not this copy's own service account.`,
"credential-date": (s) => `credentials.${s} rotation date is invalid.`,
"credential-binding-changed": (s) => `credentials.${s} changed; read and approve again.`,
"credential-role-changed": (s) => `credentials.${s} no longer matches the approved definition; the owner must review the binding.`,
"copy-unavailable": () => "copy is no longer active in this organization.",
@@ -195,7 +220,7 @@ export function validateCopyBindings(value, { trackers = [] } = {}) {
if (!isPlainObject(value)) refuse("copyBindings field is invalid");
const keys = Object.keys(value);
if (keys.length > MAX_BINDINGS) refuse("copyBindings field is invalid");
const bots = new Set(trackers.map((t) => t.bot)), botIds = new Set(trackers.map((t) => t.botId)), copyIds = new Set();
const bots = new Set(trackers.map((t) => t.bot)), botIds = new Set(trackers.map((t) => t.botId)), copyIds = new Set(), grants = new Set();
keys.forEach((instance, n) => {
const i = n + 1, r = value[instance];
if (!ID_PATTERN.test(instance)) refuse(`copyBindings entry ${i} name is invalid`);
@@ -225,8 +250,18 @@ export function validateCopyBindings(value, { trackers = [] } = {}) {
if ((r.state === "unbound") !== (services.length === 0)) refuse(at(i, "services"));
for (const service of services) {
const v = r.services[service];
if (!BINDABLE_SERVICES.includes(service) || !isPlainObject(v) || Object.keys(v).sort().join(",") !== SERVICE_KEYS.slice().sort().join(",")
|| v.slot !== "self" || !validScopes(service, v.scopes)) refuse(at(i, "services"));
const extra = isPlainObject(v) ? Object.keys(v).filter((k) => !SERVICE_KEYS.includes(k)) : [];
if (!BINDABLE_SERVICES.includes(service) || !isPlainObject(v) || SERVICE_KEYS.some((k) => !Object.hasOwn(v, k))
|| extra.some((k) => !GRANT_KEYS.includes(k)) || v.slot !== "self" || !validScopes(service, v.scopes)) refuse(at(i, "services"));
// CB2: rotateBy and grantId come together; a grant id needs its generation; one grant id per organization.
if ((v.rotateBy === undefined) !== (v.grantId === undefined) || (v.grantId !== undefined && v.grantGeneration === undefined)
|| (v.rotateBy !== undefined && !realDay(v.rotateBy))
|| (v.grantId !== undefined && (typeof v.grantId !== "string" || !GRANT_ID.test(v.grantId) || v.grantId.split(":")[0] !== service))
|| (v.grantGeneration !== undefined && (!Number.isSafeInteger(v.grantGeneration) || v.grantGeneration < 1))) refuse(at(i, "services"));
if (v.grantId !== undefined) {
if (grants.has(v.grantId)) refuse(`copyBindings entry ${i} services field uses a grant another binding uses`);
grants.add(v.grantId);
}
const p = v.resources;
if (!isPlainObject(p) || Object.keys(p).join(",") !== "projects" || !Array.isArray(p.projects) || p.projects.length === 0
|| p.projects.some((x) => typeof x !== "string" || !ID_PATTERN.test(x)) || new Set(p.projects).size !== p.projects.length
+82 -2
View File
@@ -13,7 +13,7 @@ import { checkReplyLimits } from "../../library/src/limits.mjs";
import { ADDITION_VERB, additionProblem } from "./project-bootstrap.mjs";
import { remoteProblem, remoteTarget } from "./project-remotes.mjs";
import { orgIncarnation, withRegistryWriter } from "./org-registry.mjs";
import { bindingText, definitionPin, samePin } from "./copy-bindings.mjs";
import { allGranted, bindingText, definitionPin, samePin } from "./copy-bindings.mjs";
import { entryForm, projectAgentProblem, rootForProjectCheck } from "./agent-levels-write.mjs";
import { checkDefinition, checkDefinitionName, instancesNaming, readTemplate } from "./definitions.mjs";
@@ -594,6 +594,86 @@ export function organizationWriter({ dir, rolesDir, dataRoot = null, barrier = (
});
}
// #1873 CB2 (design section 5 steps 4 to 6): a copy's service credential is issued in two writes
// around the remote work; no lock is held across it. stageCopyGrant deactivates first: the record
// goes pending, the service loses its grantId and rotateBy (grantGeneration stays) and the binding
// generation counts up. Only then does the bus write the slot and ask the service. commitCopyGrant
// rechecks under the locks everything the stage pinned (registry incarnation, document revision,
// generation, copyId, decision, source and definition pin), runs `beforeWrite` (the bus's slot and
// session recheck), then writes the grant: grantGeneration counts up, and the record is `bound` once
// every service holds a grant. A refusal or a crash between the two leaves the record pending and
// nothing resolves it; recovery is a new verified issue. Owner only, as a binding decision.
// `op` is { instance, service, copyId, generation, decision, incarnation } as the bus read it.
function copyGrantFence(old, op, { actor, owner, definitions }) {
const { instance, service } = op;
const agents = [
...Object.entries(old.roles ?? {}).flatMap(([n, e]) => [n, `${e?.definition}.${n}`]),
...Object.entries(old.projects ?? {}).flatMap(([p, v]) => Object.entries(v?.agents ?? {}).flatMap(([n, e]) => [n, `${n}.${p}`, `${e?.definition}.${n}.${p}`])),
...Object.keys(old.copyBindings ?? {}), ...Object.keys(definitions ?? {}), ...(old.retired ?? []),
];
if (owner !== true || actor === "tester" || agents.includes(actor)) fail("human-required", bindingText("human-required"));
const current = old.copyBindings?.[instance];
if (!current || current.copyId !== op.copyId || current.generation !== op.generation || current.orgIncarnation !== op.incarnation
|| current.decision !== op.decision || current.state === "unbound" || !Object.hasOwn(current.services ?? {}, service)
|| Object.hasOwn(old.roles ?? {}, instance)) fail("credential-binding-changed", bindingText("credential-binding-changed", service));
const pin = definitionPin(current.definition.ref, { definitions: { ...(definitions ?? {}), ...(old.definitions ?? {}) }, rolesDir, readTemplate });
if (!samePin(pin, current.definition)) fail("credential-role-changed", bindingText("credential-role-changed", service));
return current;
}
const grantOp = (op) => {
if (!op || typeof op.instance !== "string" || typeof op.service !== "string" || !Number.isSafeInteger(op.generation)
|| typeof op.copyId !== "string" || typeof op.decision !== "string" || typeof op.incarnation !== "string") fail("invalid-request");
};
function stageCopyGrant(id, op, { revision, actor, owner = false, definitions = {} }) {
grantOp(op);
if (owner !== true || actor === "tester") fail("human-required", bindingText("human-required"));
const build = old => {
const current = copyGrantFence(old, op, { actor, owner, definitions });
const services = structuredClone(current.services);
delete services[op.service].grantId;
delete services[op.service].rotateBy;
const record = { ...structuredClone(current), generation: current.generation + 1, state: "pending", services };
return { ...structuredClone(old), copyBindings: { ...old.copyBindings, [op.instance]: record } };
};
const result = commit(id, build, {
revision, actor, pathsOnly: true, capability: BINDING_WRITE, incarnation: op.incarnation,
guard: (old, next) => {
if (canonicalJson(next) !== canonicalJson(build(old))) fail("organization-bindings-protected", "copyBindings change only through an owner-approved binding decision");
projectsProtected(old, next);
definitionsProtected(old, next);
},
detail: { verb: "binding.stage", agent: op.instance, kind: [op.service], generation: op.generation + 1 },
});
return { ...result, generation: op.generation + 1 };
}
function commitCopyGrant(id, op, { grantId, rotateBy, revision, actor, owner = false, definitions = {}, beforeWrite = null }) {
grantOp(op);
if (owner !== true || actor === "tester") fail("human-required", bindingText("human-required"));
const build = old => {
const current = copyGrantFence(old, op, { actor, owner, definitions });
if (current.state !== "pending" || current.services[op.service].grantId !== undefined) fail("credential-binding-changed", bindingText("credential-binding-changed", op.service));
// One grant per organization: another copy's or another service's grant id never moves here.
for (const [name, r] of Object.entries(old.copyBindings ?? {}))
for (const [service, v] of Object.entries(r?.services ?? {}))
if (v?.grantId === grantId && !(name === op.instance && service === op.service)) fail("credential-identity-refused", bindingText("credential-identity-refused", op.service));
const services = structuredClone(current.services);
services[op.service] = { ...services[op.service], rotateBy, grantId, grantGeneration: (services[op.service].grantGeneration ?? 0) + 1 };
const record = { ...structuredClone(current), generation: current.generation + 1, state: allGranted(services) ? "bound" : "pending", services };
return { ...structuredClone(old), copyBindings: { ...old.copyBindings, [op.instance]: record } };
};
let state = null;
const result = commit(id, (old) => { const next = build(old); state = next.copyBindings[op.instance].state; return next; }, {
revision, actor, beforeWrite, pathsOnly: true, capability: BINDING_WRITE, incarnation: op.incarnation,
guard: (old, next) => {
if (canonicalJson(next) !== canonicalJson(build(old))) fail("organization-bindings-protected", "copyBindings change only through an owner-approved binding decision");
projectsProtected(old, next);
definitionsProtected(old, next);
},
detail: { verb: "binding.grant", agent: op.instance, kind: [op.service], generation: op.generation + 1 },
});
return { ...result, generation: op.generation + 1, state };
}
// AU inverse, from an immutable private proposal. It touches only the named instance and
// definition; writer validation, fresh revision and owner fence still apply. Bindings never restore.
function reverseHireDecision(id,p,{actor,beforeWrite,beforePublish}) {
@@ -699,5 +779,5 @@ export function organizationWriter({ dir, rolesDir, dataRoot = null, barrier = (
});
}
return { logFile, read, readRaw, write, addProject, setProjectRemote, setProjectAgent, setDefinition, applyHireDecision, reverseHireDecision, applyCopyBindingDecision, hold, holdAsync, findAddition, record, recover: id => withLock(id, () => recover(id)) };
return { logFile, read, readRaw, write, addProject, setProjectRemote, setProjectAgent, setDefinition, applyHireDecision, reverseHireDecision, applyCopyBindingDecision, stageCopyGrant, commitCopyGrant, hold, holdAsync, findAddition, record, recover: id => withLock(id, () => recover(id)) };
}
@@ -0,0 +1,215 @@
// #1873 CB2 (design section 5 steps 4 to 6, section 8 row CB2): the grant metadata a verified copy
// credential leaves on its binding record, and the writer's two writes around the remote work. The
// bus side (verifier, slot, broker) is packages/bus/tests/copy-grant.test.mjs.
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { hireDecisions } from '../src/hire-decisions.mjs';
import { organizationWriter } from '../src/organization.mjs';
import { checkRootAgents } from '../src/agent-levels.mjs';
import { materializeCopies } from '../src/agent-copies.mjs';
import { readTemplate } from '../src/definitions.mjs';
import { activateOrg, reserveOrg } from '../src/org-registry.mjs';
import * as bindings from '../src/copy-bindings.mjs';
import { businessDoc, rolesCopy, writeJson } from './helpers.mjs';
const { definitionPin, validateCopyBindings } = bindings;
const OWNER = { actor: 'jason', agent: false, rights: ['org.hire'], owner: true, instanceOwner: true, subject: { provider: 'test', iss: 'https://test', sub: 'owner' } };
const FUTURE = '2099-01-01T00:00:00.000Z';
const ID1 = '11111111-1111-4111-8111-111111111111', ID2 = '22222222-2222-4222-8222-222222222222', ID3 = '33333333-3333-4333-8333-333333333333';
const GITEA = { gitea: { scopes: ['read:repository', 'read:user'], projects: ['stack'] } };
const SECRET = 'sk-test-NOT-A-SECRET-1234';
const repoRoles = join(new URL('../../..', import.meta.url).pathname, 'roles');
const INC = 'a'.repeat(32);
const granted = (over = {}) => ({
copyId: 'c'.repeat(32), orgIncarnation: INC, source: { level: 'root', name: 'rev', project: null },
definition: definitionPin('reviewer', { rolesDir: repoRoles, readTemplate }), generation: 3, state: 'bound',
services: {
gitea: { slot: 'self', scopes: ['read:repository'], resources: { projects: ['stack'] }, rotateBy: '2099-01-01', grantId: 'gitea:5:7', grantGeneration: 1 },
vikunja: { slot: 'self', scopes: { tasks: ['read_one'] }, resources: { projects: ['stack'] }, rotateBy: '2099-01-01', grantId: 'vikunja:2:9', grantGeneration: 2 },
},
decision: ID1, ...over,
});
const withGitea = (patch, over = {}) => { const r = granted(over); r.services.gitea = { ...r.services.gitea, ...patch }; for (const k of Object.keys(patch)) if (patch[k] === undefined) delete r.services.gitea[k]; return r; };
test('CB2 schema: grant fields come together, name their own service and are ids only', () => {
assert.doesNotThrow(() => validateCopyBindings({ rev: granted() }));
// After a stage: pending, the grant gone, its generation kept.
assert.doesNotThrow(() => validateCopyBindings({ rev: withGitea({ rotateBy: undefined, grantId: undefined }, { state: 'pending' }) }));
const refused = [
withGitea({ grantId: undefined }),
withGitea({ rotateBy: undefined }),
withGitea({ grantGeneration: undefined }),
withGitea({ grantId: 'vikunja:5:7' }),
withGitea({ grantId: 'gitea:5' }),
withGitea({ grantId: 'gitea:0:7' }),
withGitea({ grantId: `gitea:${SECRET}` }),
withGitea({ grantId: 'gitea:5:7:' + 'f'.repeat(40) }),
withGitea({ grantGeneration: 0 }),
withGitea({ grantGeneration: 1.5 }),
withGitea({ rotateBy: '2099-02-30' }),
withGitea({ rotateBy: '2099-1-01' }),
withGitea({ token: SECRET }),
withGitea({ account: 'mosaic-copy-x' }),
withGitea({ sha256: 'f'.repeat(64) }),
];
for (const r of refused) assert.throws(() => validateCopyBindings({ rev: r }), e => e.name === 'BusinessError' && !e.message.includes(SECRET) && !e.message.includes('/'), JSON.stringify(r.services.gitea));
});
test('CB2 schema: grant fields are optional per service; one grant id per organization', () => {
// The schema keeps CB0's shape: a service without a grant is valid in any state. Only the
// verifier's commit writes bound, and only when every service holds a grant (bus copy-grant tests).
assert.doesNotThrow(() => validateCopyBindings({ rev: withGitea({ rotateBy: undefined, grantId: undefined }) }));
assert.throws(() => validateCopyBindings({ rev: granted({ services: {} }) }));
const other = { ...granted({ copyId: 'd'.repeat(32), source: { level: 'root', name: 'other', project: null } }) };
other.services = structuredClone(other.services);
other.services.vikunja.grantId = 'vikunja:3:3';
assert.throws(() => validateCopyBindings({ rev: granted(), other }), /grant another binding uses/);
other.services.gitea.grantId = 'gitea:5:8';
assert.doesNotThrow(() => validateCopyBindings({ rev: granted(), other }));
});
test('CB2 the login a copy grant must belong to is per organization, copy and service', () => {
const { copyBotLogin } = bindings;
assert.equal(typeof copyBotLogin, 'function');
const a = copyBotLogin(INC, 'c'.repeat(32), 'gitea');
assert.match(a, /^mosaic-copy-[0-9a-f]{24}$/);
assert.ok(a.length <= 40);
assert.equal(copyBotLogin(INC, 'c'.repeat(32), 'gitea'), a);
for (const other of [copyBotLogin('b'.repeat(32), 'c'.repeat(32), 'gitea'), copyBotLogin(INC, 'd'.repeat(32), 'gitea'), copyBotLogin(INC, 'c'.repeat(32), 'vikunja')])
assert.notEqual(other, a);
for (const bad of [[null, 'c'.repeat(32), 'gitea'], [INC, 'x', 'gitea'], [INC, 'c'.repeat(32), 'host.ops'], [INC, 'c'.repeat(32), 'model']])
assert.equal(copyBotLogin(...bad), null);
});
function setup(t) {
const root = mkdtempSync(join(tmpdir(), 'cb2o-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const roles = rolesCopy(root);
const doc = { ...businessDoc(root, 'demo'), plugins: { harness: ['pi'] } };
doc.projects.stack.agents = { checker: { definition: 'reviewer' } };
const file = writeJson(join(root, 'businesses/demo.json'), doc);
const dataRoot = join(root, 'data'); mkdirSync(dataRoot, { mode: 0o700 });
const ctx = { configDir: root, dataRoot, instance: 'test' };
const ready = (async () => { await reserveOrg(ctx, 'demo'); await activateOrg(ctx, 'demo'); })();
const snapshot = materializeCopies({ orgId: 'demo', doc, root: { state: 'ok', ...checkRootAgents({ agentsVersion: 1, agents: { rev: { definition: 'reviewer' } } }, { rolesDir: roles }) }, rolesDir: roles });
const writer = organizationWriter({ dir: root, rolesDir: roles });
const service = hireDecisions({ writer, rolesDir: roles, configDir: root, dataRoot, assertClean: () => {}, copies: id => snapshot.org === id ? snapshot : null });
const bind = async (input, id) => {
await ready;
const p = await service.prepare('demo', { op: 'change', reason: 'Bind', binding: { services: GITEA }, ...input }, { id, expiresAt: FUTURE });
await service.apply('demo', p.id, { ...OWNER, digest: p.digest });
};
const rec = name => writer.read('demo').document.copyBindings[name];
const opFor = (name, service = 'gitea') => { const r = rec(name); return { instance: name, service, copyId: r.copyId, generation: r.generation, decision: r.decision, incarnation: r.orgIncarnation }; };
const log = () => readFileSync(join(root, 'businesses/.history/demo/changes.jsonl'), 'utf8');
return { root, roles, file, writer, bind, rec, opFor, log, revision: () => writer.read('demo').revision };
}
const AS = { actor: 'jason', owner: true };
test('CB2 writer: the stage deactivates, the commit binds; the receipts name the verbs, never a grant', async t => {
const f = setup(t);
await f.bind({ level: 'root', name: 'rev' }, ID1);
const op = f.opFor('rev');
assert.equal(f.rec('rev').state, 'pending');
const staged = f.writer.stageCopyGrant('demo', op, { revision: f.revision(), ...AS });
assert.equal(staged.generation, op.generation + 1);
assert.deepEqual({ state: f.rec('rev').state, generation: f.rec('rev').generation }, { state: 'pending', generation: op.generation + 1 });
const done = f.writer.commitCopyGrant('demo', { ...op, generation: staged.generation }, { grantId: 'gitea:5:9', rotateBy: '2099-01-01', revision: staged.revision, ...AS });
assert.equal(done.state, 'bound');
const r = f.rec('rev');
assert.deepEqual({ state: r.state, generation: r.generation, gitea: r.services.gitea }, {
state: 'bound', generation: op.generation + 2,
gitea: { slot: 'self', scopes: ['read:repository', 'read:user'], resources: { projects: ['stack'] }, rotateBy: '2099-01-01', grantId: 'gitea:5:9', grantGeneration: 1 },
});
const log = f.log();
assert.ok(log.includes('binding.stage') && log.includes('binding.grant'));
assert.ok(!log.includes('gitea:5:9') && !log.includes('2099-01-01'), 'the receipts are paths only');
// A new issue stages again: use closes before any remote work, the grant generation stays.
f.writer.stageCopyGrant('demo', f.opFor('rev'), { revision: f.revision(), ...AS });
assert.deepEqual({ state: f.rec('rev').state, gitea: f.rec('rev').services.gitea }, {
state: 'pending', gitea: { slot: 'self', scopes: ['read:repository', 'read:user'], resources: { projects: ['stack'] }, grantGeneration: 1 },
});
const again = f.writer.commitCopyGrant('demo', f.opFor('rev'), { grantId: 'gitea:5:10', rotateBy: '2099-02-01', revision: f.revision(), ...AS });
assert.equal(again.state, 'bound');
assert.equal(f.rec('rev').services.gitea.grantGeneration, 2);
});
test('CB2 writer: a stale generation, revision, decision, copy or incarnation, and a second commit, write nothing', async t => {
const f = setup(t);
await f.bind({ level: 'root', name: 'rev' }, ID1);
const op = f.opFor('rev');
const rev0 = f.revision();
const staged = f.writer.stageCopyGrant('demo', op, { revision: rev0, ...AS });
const after = { ...op, generation: staged.generation };
const grant = { grantId: 'gitea:5:9', rotateBy: '2099-01-01', ...AS };
const before = readFileSync(f.file);
for (const [bad, code] of [
[{ ...after, generation: op.generation }, 'credential-binding-changed'],
[{ ...after, copyId: 'e'.repeat(32) }, 'credential-binding-changed'],
[{ ...after, decision: ID3 }, 'credential-binding-changed'],
[{ ...after, instance: 'checker.stack' }, 'credential-binding-changed'],
[{ ...after, service: 'vikunja' }, 'credential-binding-changed'],
[{ ...after, incarnation: 'b'.repeat(32) }, 'decision-stale'],
]) assert.throws(() => f.writer.commitCopyGrant('demo', bad, { ...grant, revision: staged.revision }), { code }, JSON.stringify(bad));
assert.throws(() => f.writer.commitCopyGrant('demo', after, { ...grant, revision: rev0 }), { code: 'revision-conflict' });
assert.deepEqual(readFileSync(f.file), before);
f.writer.commitCopyGrant('demo', after, { ...grant, revision: staged.revision });
const bound = readFileSync(f.file);
assert.throws(() => f.writer.commitCopyGrant('demo', after, { ...grant, revision: f.revision() }), { code: 'credential-binding-changed' });
assert.throws(() => f.writer.commitCopyGrant('demo', { ...after, generation: after.generation + 1 }, { ...grant, grantId: 'gitea:5:10', revision: f.revision() }), { code: 'credential-binding-changed' });
assert.deepEqual(readFileSync(f.file), bound);
// A new binding decision between the stage and the commit wins: the commit refuses.
f.writer.stageCopyGrant('demo', f.opFor('rev'), { revision: f.revision(), ...AS });
const pinned = f.opFor('rev');
await f.bind({ level: 'root', name: 'rev', binding: { services: { gitea: { scopes: ['read:user'], projects: ['stack'] } } } }, ID2);
assert.throws(() => f.writer.commitCopyGrant('demo', pinned, { ...grant, grantId: 'gitea:5:11', revision: f.revision() }), { code: 'credential-binding-changed' });
assert.deepEqual(f.rec('rev').services.gitea, { slot: 'self', scopes: ['read:user'], resources: { projects: ['stack'] } });
assert.equal(f.rec('rev').state, 'pending');
});
test('CB2 writer: only the owner stages or commits; no agent, tester or member identity', async t => {
const f = setup(t);
await f.bind({ level: 'root', name: 'rev' }, ID1);
const op = f.opFor('rev');
const before = readFileSync(f.file);
for (const who of [{ actor: 'jason', owner: false }, { actor: 'tester', owner: true }, { actor: 'rev', owner: true }, { actor: 'chief', owner: true }, { actor: 'checker.stack', owner: true }, { actor: 'reviewer.checker.stack', owner: true }]) {
assert.throws(() => f.writer.stageCopyGrant('demo', op, { revision: f.revision(), ...who }), { code: 'human-required' }, who.actor);
assert.throws(() => f.writer.commitCopyGrant('demo', op, { grantId: 'gitea:5:9', rotateBy: '2099-01-01', revision: f.revision(), ...who }), { code: 'human-required' }, who.actor);
}
assert.throws(() => f.writer.stageCopyGrant('demo', { ...op, generation: 'x' }, { revision: f.revision(), ...AS }), { code: 'invalid-request' });
assert.deepEqual(readFileSync(f.file), before);
});
test('CB2 writer: another copy\'s grant id never moves here; a definition change refuses the commit', async t => {
const f = setup(t);
await f.bind({ level: 'root', name: 'rev' }, ID1);
await f.bind({ level: 'project', project: 'stack', name: 'checker' }, ID2);
const a = f.writer.stageCopyGrant('demo', f.opFor('rev'), { revision: f.revision(), ...AS });
f.writer.commitCopyGrant('demo', { ...f.opFor('rev') }, { grantId: 'gitea:5:9', rotateBy: '2099-01-01', revision: f.revision(), ...AS });
assert.ok(a);
f.writer.stageCopyGrant('demo', f.opFor('checker.stack'), { revision: f.revision(), ...AS });
const before = readFileSync(f.file);
assert.throws(() => f.writer.commitCopyGrant('demo', f.opFor('checker.stack'), { grantId: 'gitea:5:9', rotateBy: '2099-01-01', revision: f.revision(), ...AS }), { code: 'credential-identity-refused' });
assert.deepEqual(readFileSync(f.file), before);
// The reviewer template changes between the stage and the commit: its pin no longer matches.
const template = join(f.roles, 'reviewer.json');
const role = JSON.parse(readFileSync(template, 'utf8'));
writeFileSync(template, JSON.stringify({ ...role, title: role.title + ' changed' }, null, 2));
assert.throws(() => f.writer.commitCopyGrant('demo', f.opFor('checker.stack'), { grantId: 'gitea:6:1', rotateBy: '2099-01-01', revision: f.revision(), ...AS }), { code: 'credential-role-changed' });
assert.deepEqual(readFileSync(f.file), before);
});
test('CB2 writer: a generic write never adds or changes grant fields', async t => {
const f = setup(t);
await f.bind({ level: 'root', name: 'rev' }, ID1);
const { document, revision } = f.writer.read('demo');
const next = structuredClone(document);
next.copyBindings.rev.services.gitea = { ...next.copyBindings.rev.services.gitea, rotateBy: '2099-01-01', grantId: 'gitea:5:9', grantGeneration: 1 };
next.copyBindings.rev.state = 'bound';
assert.throws(() => f.writer.write('demo', next, { revision, actor: 'jason' }), { code: 'organization-bindings-protected' });
assert.equal(f.rec('rev').state, 'pending');
});
+44 -1
View File
@@ -16,6 +16,7 @@
// POST /api/orgs/ID/setup/queue C2 queueInit mosaic org queue init
// POST /api/orgs/ID/setup/project/P C2 projectRepoInit mosaic org project repo init
// POST /api/orgs/ID/setup/bus C3 busStart mosaic bus start --detach
// POST /api/orgs/ID/restore C2 restoreOrg, then C3 busStart (E1: the answer's `bus`)
// POST /api/orgs/ID/restart/bus C3 busRestart mosaic bus restart (F3; busStart when stopped;
// busRevive for a host that died, dev-local owner only, D2)
// POST /api/orgs/ID/stop/bus C3 busStop mosaic bus stop (a stopped bus answers state stopped)
@@ -465,7 +466,9 @@ async function route(req, res, { path, search, org, admin, principal, write, log
const fn = kind === 'archive' ? service.archiveOrg : service.restoreOrg;
if (typeof fn !== 'function') return json(res, 503, { error: 'not-configured', step: kind });
const receipt = await fn(ctx, id, { ifRevision: body.revision, ...(busStop ? { busStop } : {}) });
return json(res, 200, { outcome: 'done', step: kind, receipt: forReceipt(ctx, receipt), verb: verbFor(kind, id, { revision: body.revision }) });
const done = { outcome: 'done', step: kind, receipt: forReceipt(ctx, receipt), verb: verbFor(kind, id, { revision: body.revision }) };
if (kind === 'archive') return json(res, 200, done);
return json(res, 200, { ...done, bus: await restoreBus({ id, ctx, admin, log, trail, by }) });
} catch (e) { return refusal(res, e, { step: kind, id, unknownOk: true }); }
}
if (kind === 'delete' && !sub && write) {
@@ -571,6 +574,46 @@ async function route(req, res, { path, search, org, admin, principal, write, log
return json(res, 404, { error: 'not found' });
}
// E1 (P1-4): Restore brings the organization back running. After a good restore the bus starts as
// Restart bus starts a stopped one: the owner, origin and --send checks were made for the restore,
// the trail line is written first, and a trail that cannot be written starts nothing. The restore
// stands whatever happens here; the answer's `bus` says what did, and the page offers Restart bus
// when the bus did not start. Never throws.
async function restoreBus({ id, ctx, admin, log, trail, by }) {
const verb = verbFor('bus', id);
const notStarted = (error, message) => ({ outcome: 'not-started', error, message, verb });
let bus = null;
try { bus = await admin.bus(); } catch { /* reported as not configured */ }
if (!bus || typeof bus.busStart !== 'function' || typeof bus.busStatus !== 'function') return notStarted('not-configured', 'bus control is not available on this Console, so the bus was not started');
if (!trail) return notStarted('trail-unavailable', 'this Console has no action trail, so it does not start buses');
let root = null;
try {
const status = await bus.busStatus(ctx, id);
if (status?.healthy) return { outcome: 'done', existing: true, healthy: true, receipt: forReceipt(ctx, status), verb };
root = status?.root ?? null;
} catch (e) {
if (!isCliError(e)) { log(`org-admin: ${id} restore: bus status: ${e?.code ?? e?.name ?? 'error'}`); return notStarted('bus-refused', 'the bus state could not be read, so the bus was not started'); }
return notStarted('bus-refused', `${plainMessage(ctx, e.message)}. Inspect it with mosaic bus status ${id}; mosaic bus recover ${id} clears a host that died.`);
}
try { trail({ by, action: 'bus.start', org: id, outcome: 'requested', after: 'restore' }); }
catch { return notStarted('trail-unavailable', 'the action trail could not be written, so the bus was not started'); }
const note = outcome => { try { trail({ by, action: 'bus.start', org: id, outcome, after: 'restore' }); } catch { log(`org-admin: ${id} restore bus start: trail write failed after ${outcome}`); } };
let receipt;
try { receipt = await bus.busStart(ctx, id, { detach: true, devLocalOwner: by?.provider === 'local-dev', pm: false }); }
catch (e) {
const failed = isCliError(e) && /^bus failed to start\b/.test(e.message);
note(failed ? 'failed' : 'refused');
if (!isCliError(e)) { log(`org-admin: ${id} restore bus start: ${e?.code ?? e?.name ?? 'error'}`); return { outcome: 'failed', error: 'bus-failed', message: 'the bus did not start', verb }; }
const said = failed ? await hostRefusal(root) : null;
return { outcome: 'failed', error: failed ? 'bus-failed' : 'bus-refused', message: plainMessage(ctx, said ? `${e.message}. The bus host said: ${said}` : e.message), verb };
}
note('done');
log(`org-admin: ${id} bus started after restore`);
let healthy = null;
try { healthy = (await bus.busStatus(ctx, id)).healthy; } catch { /* the receipt still stands */ }
return { outcome: 'done', healthy, receipt: forReceipt(ctx, receipt), verb };
}
// F3 (REQ-WEB-3): one control restarts an organization's bus and answers with a done receipt. A
// running host restarts on the options it was started with (C3 busRestart keeps --dev-local-owner);
// a stopped one starts as setup starts it. The trail line is written before anything changes, and
+2 -1
View File
@@ -1923,7 +1923,8 @@
if (path.startsWith('/chat')||path.startsWith('/projects')||path.startsWith('/inbox')||path.startsWith('/settings')||path.startsWith('/docs/')||wsPage(path)) render(false); else if (window.MosaicOrg ? path === '/board' : path === '/' || path === '') paintHeld(heldErr);
};
configRead = fetch('/api/config', { cache: 'no-store', redirect: 'manual' }).then(r => { if (r.type === 'opaqueredirect') { signOut(); return null; } return r.json(); }).then(applyConfig).catch(() => {});
document.addEventListener('mosaic-config', e => { if (e.detail && typeof e.detail === 'object') applyConfig(e.detail); });
// E1 (P1-5): after Restart bus the organization dashboard draws its bus line and system types now, not at the next poll.
document.addEventListener('mosaic-config', e => { if (e.detail && typeof e.detail === 'object') { applyConfig(e.detail); if (window.MosaicOrg && routeOf().split('?')[0] === '/organization') render(false); } });
// The cross-organization pages ask for a repaint after a filter, sort or selection change,
// and report the inbox count across organizations for the sidebar.
document.addEventListener('mosaic-workspace-render', () => { if (wsPage(routeOf().split('?')[0])) render(false); });
+45 -8
View File
@@ -68,13 +68,13 @@
<button type="button" id="org-archive-open" data-write>Archive organization</button>
<div id="org-archive-confirm" tabindex="-1" hidden aria-labelledby="org-archive-confirm-title">
<h4 id="org-archive-confirm-title">Review archive</h4>
<ul><li>The organization's bus stops.</li><li>All files stay where they are.</li><li>The organization leaves the switcher.</li><li>Restore brings it back without starting the bus.</li></ul>
<ul><li>The organization's bus stops.</li><li>All files stay where they are.</li><li>The organization leaves the switcher.</li><li>Restore brings it back and starts its bus.</li></ul>
<div class="settings-actions"><button type="button" id="org-archive-cancel" class="org-secondary">Cancel</button><button type="button" id="org-archive-final" data-write>Archive</button></div>
</div>
</section>
<section id="org-archived" class="panel" tabindex="-1" hidden aria-labelledby="org-archived-title">
<h3 id="org-archived-title">Organization archived</h3>
<p>Editing and setup are unavailable while archived. Files remain in place; restoring does not start the bus.</p>
<p>Editing and setup are unavailable while archived. Files remain in place; restoring starts the bus again.</p>
<p id="org-restore-verb" class="org-verb"></p>
<button type="button" id="org-restore" data-write>Restore organization</button>
</section>
@@ -342,7 +342,8 @@
if (result.body.verb) card.querySelector('code').textContent = result.body.verb;
}
editorDirty = false; $('org-new-id').value = ''; $('org-quick-open-org').hidden = false; $('org-quick-interview').hidden = !window.MosaicOnboarding; status('Organization setup complete.');
await refreshList();
// E1 (P1-5): the sidebar and header list the new organization without a reload.
await refreshOrganizations();
} catch (e) {
const card = list.children[Math.min(current, steps.length - 1)];
card.querySelector('.org-pill').textContent = 'Stopped'; card.querySelector('.org-field-err').textContent = current === 4
@@ -350,7 +351,7 @@
: `${e.message} Open the organization to inspect and continue; do not repeat a step whose outcome is unknown.`;
if (current >= 4) { editorDirty = false; $('org-new-id').value = ''; $('org-quick-open-org').hidden = false; }
card.classList.add('org-step-warn'); card.focus(); status('Setup stopped.');
await refreshList();
await refreshOrganizations();
}
}
async function openExisting(id, { toSetup = false } = {}) {
@@ -549,6 +550,37 @@
const hash = next ? `#/orgs/${encodeURIComponent(next.id)}/settings/organizations` : '#/settings/organizations';
history.replaceState(null, '', hash); location.reload(); return true;
}
// E1 (P1-4): a restore also starts the bus (org-admin.mjs restoreBus). One receipt names both
// results; a bus that did not start says why and offers Restart bus for this organization.
const RESTORED = 'mosaic-org-restored:';
function keepRestored(id, bus) { try { sessionStorage.setItem(RESTORED + id, JSON.stringify(bus ?? null)); } catch { /* the reloaded page shows the bus state only */ } }
function takeRestored(id) {
try { const v = sessionStorage.getItem(RESTORED + id); sessionStorage.removeItem(RESTORED + id); return v === null ? undefined : JSON.parse(v); } catch { return undefined; }
}
function restoredReceipt(id, bus) {
const check = `mosaic bus status ${id}`, known = bus && typeof bus === 'object';
const line = !known ? `The bus outcome is unknown. Check it with ${check}.`
: bus.outcome === 'done' ? (bus.existing ? 'Its bus was already running.' : `Bus started. ${bus.healthy ? 'It answers.' : `It has not answered yet. Check it with ${check}.`}`)
: `The bus did not start: ${bus.message || 'no reason was given.'}`;
const offer = (!known || bus.outcome !== 'done') && canWrite;
const box = $('org-archive-success');
box.innerHTML = `<p>Organization restored. ${txt(line)}</p>${known && bus.verb ? `<p class="org-verb">Same as <code>${txt(bus.verb)}</code></p>` : ''}${offer ? `<p><button type="button" data-restore-bus="${txt(id)}">Restart bus</button> <span role="status" data-restore-bus-receipt></span></p>` : ''}`;
box.hidden = false; box.focus();
}
// Restart bus for the restored organization, the same route as the dashboard's (console-restart.js).
async function restartRestored(e) {
const button = e.target.closest('[data-restore-bus]'); if (!button || button.disabled) return;
const id = button.dataset.restoreBus, out = button.parentElement.querySelector('[data-restore-bus-receipt]');
button.disabled = true; button.textContent = 'Restarting…'; out.textContent = '';
let r = null; try { r = await api('POST', `/api/orgs/${encodeURIComponent(id)}/restart/bus`, {}); } catch { r = null; }
if (!alive) return;
const done = r?.status === 200 && r.body.outcome === 'done' && r.body.step === 'bus-restart';
out.textContent = !r ? `Outcome unknown. Check the bus with mosaic bus status ${id} before retrying.`
: done ? `${r.body.recovered ? 'Dead bus host cleared; bus started' : r.body.restarted ? 'Bus restarted' : 'Bus started'}. ${r.body.healthy ? 'It answers.' : `It has not answered yet. Check it with mosaic bus status ${id}.`}`
: (r.body.message || 'The restart was refused. Nothing was retried.');
button.textContent = 'Restart bus'; button.disabled = done && r.body.healthy === true;
if (done) await refreshOrganizations();
}
async function runLifecycle(step) {
const id = st.id, revision = st.revision, button = $(step === 'archive' ? 'org-archive-final' : 'org-restore');
button.disabled = true; if (step === 'archive') button.textContent = 'Archiving…'; $('org-archive-error').textContent = '';
@@ -558,11 +590,11 @@
if (!verifiedReply(reply, step)) throw Error('unknown lifecycle outcome');
const rows = await refreshOrganizations();
if (step === 'archive' && switchIfCurrentRemoved(id, rows)) return;
if (step === 'restore' && !window.MosaicOrg) { history.replaceState(null, '', `#/orgs/${encodeURIComponent(id)}/settings/organizations`); location.reload(); return; }
// The page reloads into the restored organization; its receipt is kept for that page to show.
if (step === 'restore' && !window.MosaicOrg) { keepRestored(id, reply.body.bus); history.replaceState(null, '', `#/orgs/${encodeURIComponent(id)}/settings/organizations?org=${encodeURIComponent(id)}`); location.reload(); return; }
await openExisting(id);
if (step === 'restore') {
const box = $('org-archive-success'); box.textContent = 'Organization restored. The bus remains stopped.'; box.hidden = false; box.focus();
} else $('org-archived').focus();
if (step === 'restore') restoredReceipt(id, reply.body.bus);
else $('org-archived').focus();
status(`Organization ${step === 'archive' ? 'archived' : 'restored'}.`);
} catch {
if (step === 'archive') button.textContent = 'Archive';
@@ -615,6 +647,7 @@
[$('org-archive-cancel'), 'click', () => { resetArchive(); $('org-archive-open').focus(); }],
[$('org-archive-final'), 'click', () => runLifecycle('archive')],
[$('org-restore'), 'click', () => runLifecycle('restore')],
[$('org-archive-success'), 'click', restartRestored],
[$('org-delete-open'), 'click', () => { $('org-delete-open').hidden = true; $('org-delete-first').hidden = false; $('org-delete-first').focus(); }],
[$('org-delete-cancel'), 'click', cancelDelete],
[$('org-delete-cancel-second'), 'click', cancelDelete],
@@ -643,6 +676,9 @@
[$('org-steps'), 'input', e => { const card = e.target.closest('.org-step'); if (card && e.target.matches('[data-remote]')) { keepChoice(card); showVerb(card); } }],
];
for (const [el, ev, fn] of handlers) el.addEventListener(ev, fn);
// A restore receipt carried across the reload is taken now, before anything below can fail, so a
// later visit never shows it again.
const carried = (() => { const o = query().get('org'); return o && ID.test(o) ? takeRestored(o) : undefined; })();
fetch(new URL('/api/config', location.href), { cache: 'no-store' }).then(r => r.json()).then(async config => {
if (!alive) return;
canWrite = (config.orgSetup === true || config.send === true) && config.tester !== true && config.user !== true;
@@ -655,6 +691,7 @@
// Resume: ?org=ID reopens the organization and its setup, read back from the server.
if (org && ID.test(org)) {
await openExisting(org, { toSetup: setup });
if (carried !== undefined) restoredReceipt(org, carried);
if (created && st.setup?.notifications?.receipt) showReceipt('Organization created', st.setup.notifications);
if (setup) focusFirstOpen();
}
+54
View File
@@ -579,6 +579,60 @@ test('a bus that refuses to restart is a visible failure and the trail says refu
assert.equal(c.calls.slice(before).filter(y => ['busStart', 'busRestart'].includes(y[0])).length, 0);
});
// E1 (P1-4): a restore brings the organization back running. The bus starts as Restart bus starts a
// stopped one, the trail says it came after a restore, and the answer names both results.
const restorable = c => { c.service.restoreOrg = async (ctx, id, opts) => { c.calls.push(['restoreOrg', id, opts.ifRevision]); return { op: 'org.restore', org: id, revision: 3 }; }; };
test('E1: a restore starts the organization\'s bus and the answer names both results', async t => {
const tr = trailIn(t);
const c = await console_(t, { trail: tr.trail });
await c.call('POST', '/api/orgs', { id: 'acme', doc: browserDoc('acme') }); restorable(c);
const r = await c.call('POST', '/api/orgs/acme/restore', { revision: 2 });
assert.equal(r.status, 200, r.raw);
assert.equal(r.body.outcome, 'done'); assert.equal(r.body.step, 'restore'); assert.equal(r.body.verb, 'mosaic org restore acme --if-revision 2');
assert.deepEqual(r.body.bus, { outcome: 'done', healthy: true, receipt: { org: 'acme', pid: 4242, socket: '<host path>', root: '<data root>/org-runtime/acme' }, verb: 'mosaic bus start acme --dev-local-owner --detach' });
assert.deepEqual(c.calls.filter(x => ['restoreOrg', 'busStart'].includes(x[0])).map(x => x.slice(0, 3)), [['restoreOrg', 'acme', 2], ['busStart', 'acme', { detach: true, devLocalOwner: true, pm: false }]]);
assert.deepEqual(tr.lines(), [
{ by: DEV, action: 'bus.start', org: 'acme', outcome: 'requested', after: 'restore' },
{ by: DEV, action: 'bus.start', org: 'acme', outcome: 'done', after: 'restore' },
]);
// A bus that already answers is left as it is.
const again = await c.call('POST', '/api/orgs/acme/restore', { revision: 3 });
assert.deepEqual({ outcome: again.body.bus.outcome, existing: again.body.bus.existing, healthy: again.body.bus.healthy }, { outcome: 'done', existing: true, healthy: true });
assert.equal(c.calls.filter(x => x[0] === 'busStart').length, 1);
});
test('E1: a bus that does not start after a good restore is said so, with the verb, and the restore stands', async t => {
const tr = trailIn(t);
const c = await console_(t, { trail: tr.trail });
await c.call('POST', '/api/orgs', { id: 'acme', doc: browserDoc('acme') }); restorable(c);
const bus = await c.admin.bus();
bus.busStart = async () => { throw cliError('bus failed to start; see /data/org-runtime/acme/bus-host/host.log'); };
const r = await c.call('POST', '/api/orgs/acme/restore', { revision: 2 });
assert.equal(r.status, 200, r.raw); assert.equal(r.body.outcome, 'done'); assert.equal(r.body.step, 'restore');
assert.deepEqual(r.body.bus, { outcome: 'failed', error: 'bus-failed', message: 'bus failed to start; see <data root>/org-runtime/acme/bus-host/host.log', verb: 'mosaic bus start acme --dev-local-owner --detach' });
assert.doesNotMatch(r.raw, /\/data\//);
assert.deepEqual(tr.lines().map(e => e.outcome), ['requested', 'failed']);
// A bus whose state cannot be read is not started.
bus.busStatus = async () => { throw cliError('stale bus host lock'); };
const s = await c.call('POST', '/api/orgs/acme/restore', { revision: 3 });
assert.equal(s.status, 200); assert.equal(s.body.bus.outcome, 'not-started'); assert.equal(s.body.bus.error, 'bus-refused');
assert.equal(s.body.bus.message, 'stale bus host lock. Inspect it with mosaic bus status acme; mosaic bus recover acme clears a host that died.');
assert.equal(tr.lines().length, 2);
});
test('E1: a Console with no trail, or a trail that cannot be written, restores without starting the bus', async t => {
for (const trail of [undefined, () => { throw new Error('disk full'); }]) {
const c = await console_(t, trail ? { trail } : {});
await c.call('POST', '/api/orgs', { id: 'acme', doc: browserDoc('acme') }); restorable(c);
const r = await c.call('POST', '/api/orgs/acme/restore', { revision: 2 });
assert.equal(r.status, 200, r.raw); assert.equal(r.body.outcome, 'done');
assert.equal(r.body.bus.outcome, 'not-started'); assert.equal(r.body.bus.error, 'trail-unavailable');
assert.equal(r.body.bus.verb, 'mosaic bus start acme --dev-local-owner --detach');
assert.equal(c.calls.filter(x => x[0] === 'busStart').length, 0);
}
});
// D2 (P0-3): on the dev Console, Restart bus on a host that died (its state left behind by a
// Console unit stop) clears that state through busRevive and starts the bus. busRevive's
// busRecover is the proof; a live or unprovable host refuses and nothing starts.
@@ -11,18 +11,23 @@ import * as service from '../../organization/src/org-service.mjs';
import * as registry from '../../organization/src/org-registry.mjs';
import { orgPaths } from '../../cli/src/config.mjs';
import { busStopForDelete } from '../../cli/src/org-bus.mjs';
import { CliError } from '../../cli/src/errors.mjs';
import { actionTrail } from '../src/action-trail.mjs';
import { browser } from './browser.mjs';
import { systemTypesFake } from './helpers/system-types-fake.mjs';
import { close } from './fixture.mjs';
async function fixture(t, ids = ['alpha', 'beta'], { health = null } = {}) {
// E1: `bus` adds verbs to the admin's bus (busStart for a restore); `trail` gives the Console its action trail;
// `extra` adds to each organization's transports (the dashboard's bus reads and System types).
async function fixture(t, ids = ['alpha', 'beta'], { health = null, bus = {}, trail = false, extra = () => ({}) } = {}) {
const dir = await mkdtemp(join(tmpdir(), 'archive-browser-'));
const configDir = join(dir, 'config'), dataRoot = join(dir, 'data'); await mkdir(configDir, { mode: 0o700 }); await mkdir(dataRoot, { mode: 0o700 });
const ctx = { configDir, dataRoot, instance: 'test', principal: { provider: 'local-dev', iss: 'local-dev', sub: 'local-dev-owner' } };
for (const id of ids) await service.createOrg(ctx, id, await service.templateRender(id));
const busStatus = async () => ({ healthy: health ? health.up : false });
const organizations = organizationConsole({ ctx, listOrgs: registry.listOrgs, resolveOrg: registry.resolveOrg, showOrg: service.showOrg, orgPaths, busStatus, options: ({ id }) => ({ reads: { business: async () => ({ business: { id, projects: ['work'] } }), settings: async () => ({}) }, bus: {}, send: async () => ({}) }) });
const admin = createOrgAdmin({ service, registry, bus: { busStatus, busStopForDelete }, context: async () => ({ configDir, dataRoot, instance: 'test' }) });
const server = await startServer({ port: 0, sso: createDevOwner(), orgAdmin: admin, organizations, send: async () => ({}), board: 'http://127.0.0.1:9', log: () => {} });
const organizations = organizationConsole({ ctx, listOrgs: registry.listOrgs, resolveOrg: registry.resolveOrg, showOrg: service.showOrg, orgPaths, busStatus, options: ({ id }) => ({ reads: { business: async () => ({ business: { id, projects: ['work'] } }), settings: async () => ({}) }, bus: {}, send: async () => ({}), ...extra(id) }) });
const admin = createOrgAdmin({ service, registry, bus: { busStatus, busStopForDelete, ...bus }, context: async () => ({ configDir, dataRoot, instance: 'test' }) });
const server = await startServer({ port: 0, sso: createDevOwner(), orgAdmin: admin, organizations, send: async () => ({}), board: 'http://127.0.0.1:9', log: () => {}, ...(trail ? { trail: actionTrail(join(dir, 'console')) } : {}) });
const b = await browser({ machine: false }); t.after(async () => { await b.close(); await close(server); await rm(dir, { recursive: true, force: true }); });
const wait = async expression => { for (let i = 0; i < 400; i++) { try { if (await b.evaluate(expression)) return; } catch {} await new Promise(r => setTimeout(r, 30)); } throw Error(expression); };
await b.navigate(`http://127.0.0.1:${server.address().port}/#/orgs/alpha/settings/organizations`);
@@ -36,7 +41,7 @@ async function fixture(t, ids = ['alpha', 'beta'], { health = null } = {}) {
await b.screenshot(join(process.env.ORG_ARCHIVE_SHOTS, `${state}-${width}-${mode}.png`));
}
}
return { b, wait, open, shots, ctx, port: server.address().port };
return { b, wait, open, shots, ctx, port: server.address().port, dir };
}
test('archive and restore a non-current organization', { timeout: 45000 }, async t => {
@@ -88,6 +93,76 @@ test('opening another editor resets the archive confirmation and error', { timeo
assert.deepEqual(await f.b.evaluate(`({trigger:document.querySelector('#org-archive-open').hidden,panel:document.querySelector('#org-archive-confirm').hidden,error:document.querySelector('#org-archive-error').textContent,disabled:document.querySelector('#org-archive-final').disabled})`), {trigger:false,panel:true,error:'',disabled:false});
});
// E1 (P1-4): one Restore restores the organization and starts its bus; the receipt names both.
const startingBus = (health, fails = 0) => {
const starts = [];
return { starts, busStart: async (ctx, id, opts) => { starts.push([id, opts]); if (starts.length <= fails) throw new CliError('bus failed to start; see the bus log', 3); health.up = true; return { org: id, pid: 4242, socket: join(ctx.dataRoot, 'bus.sock'), root: join(ctx.dataRoot, 'org-runtime', id) }; } };
};
const archiveBeta = async f => { await f.open('beta'); await f.b.evaluate(`document.querySelector('#org-archive-open').click();document.querySelector('#org-archive-final').click()`); await f.wait(`!document.querySelector('#org-archived').hidden`); };
test('E1 (P1-4): Restore restores the organization and starts its bus, with one receipt naming both', { timeout: 45000 }, async t => {
const health = { up: false }, bus = startingBus(health);
const f = await fixture(t, ['alpha', 'beta'], { health, bus, trail: true }); await archiveBeta(f);
await f.b.evaluate(`document.querySelector('#org-restore').click()`);
await f.wait(`!document.querySelector('#org-archive-success').hidden`);
assert.equal((await registry.listOrgs(f.ctx)).find(e => e.id === 'beta').state, 'active');
assert.deepEqual(bus.starts, [['beta', { detach: true, devLocalOwner: true, pm: false }]]);
const text = await f.b.evaluate(`document.querySelector('#org-archive-success').innerText`);
assert.match(text, /^Organization restored\. Bus started\. It answers\./);
assert.match(text, /Same as mosaic bus start beta --dev-local-owner --detach/);
assert.doesNotMatch(text, /remains stopped/);
assert.equal(await f.b.evaluate(`document.querySelector('[data-restore-bus]')`), null, 'a running bus needs no Restart bus');
assert.deepEqual(await f.b.evaluate(`window.MosaicOrg.list.then(rows=>rows.map(row=>[row.id,row.busHealthy]))`), [['alpha', true], ['beta', true]]);
await f.shots('e1-restored-started', '#org-archive-success');
});
test('E1 (P1-4): a bus that does not start after a good restore is said so, and Restart bus starts it', { timeout: 45000 }, async t => {
const health = { up: false }, bus = startingBus(health, 1);
const f = await fixture(t, ['alpha', 'beta'], { health, bus, trail: true }); await archiveBeta(f);
await f.b.evaluate(`document.querySelector('#org-restore').click()`);
await f.wait(`!document.querySelector('#org-archive-success').hidden`);
assert.equal((await registry.listOrgs(f.ctx)).find(e => e.id === 'beta').state, 'active', 'the restore stands');
assert.match(await f.b.evaluate(`document.querySelector('#org-archive-success').innerText`), /^Organization restored\. The bus did not start: bus failed to start; see the bus log/);
assert.equal(await f.b.evaluate(`document.querySelector('[data-restore-bus]').textContent`), 'Restart bus');
await f.shots('e1-restored-bus-failed', '#org-archive-success');
await f.b.evaluate(`document.querySelector('[data-restore-bus]').click()`);
await f.wait(`/It answers/.test(document.querySelector('[data-restore-bus-receipt]')?.textContent || '')`);
assert.equal(await f.b.evaluate(`document.querySelector('[data-restore-bus-receipt]').textContent`), 'Bus started. It answers.');
assert.equal(await f.b.evaluate(`document.querySelector('[data-restore-bus]').disabled`), true);
assert.deepEqual(bus.starts.map(([id]) => id), ['beta', 'beta']);
await f.shots('e1-restored-bus-restarted', '#org-archive-success');
});
test('E1 (P1-4): restoring the only organization reloads into it and still shows the restore and bus receipt', { timeout: 45000 }, async t => {
const health = { up: false }, bus = startingBus(health);
const f = await fixture(t, ['alpha'], { health, bus, trail: true }); await f.open('alpha');
await f.b.evaluate(`document.querySelector('#org-archive-open').click();document.querySelector('#org-archive-final').click()`);
await f.wait(`!window.MosaicOrg && document.querySelector('#org-open-id')?.value === 'alpha'`);
await f.open('alpha'); await f.b.evaluate(`document.querySelector('#org-restore').click()`);
await f.wait(`window.MosaicOrg?.id === 'alpha' && document.querySelector('#org-archive-success')?.hidden === false`);
assert.match(await f.b.evaluate(`document.querySelector('#org-archive-success').innerText`), /^Organization restored\. Bus started\. It answers\./);
assert.deepEqual(bus.starts.map(([id]) => id), ['alpha']);
// The kept receipt is shown once: a later visit does not repeat it.
assert.equal(await f.b.evaluate(`Object.keys(sessionStorage).filter(k=>k.startsWith('mosaic-org-restored:')).length`), 0);
});
test('E1 (P1-5): after restoring another organization, its dashboard shows the bus running and its System types', { timeout: 45000 }, async t => {
const health = { up: false }, bus = startingBus(health), st = await systemTypesFake({ enabled: ['coding'] });
const extra = id => ({
requestRead: st.requestRead,
reads: { business: async () => ({ business: { id, human: 'local-dev-owner', arbiters: [], projects: [], vars: {}, instances: [] } }), settings: async () => ({}), queue: async () => ({ rows: [] }), busStatus: async () => ({}) },
bus: { instances: async () => ({ business: id, revision: 'a'.repeat(64), launch: true, noHire: [], retired: [], definitions: {}, instances: {} }), messages: async () => ({ messages: [], unread: 0 }), agents: async () => [], 'thread.claims': async () => [], inbox: async () => [], tasks: async () => [] },
});
const f = await fixture(t, ['alpha', 'beta'], { health, bus, trail: true, extra }); await archiveBeta(f);
await f.b.evaluate(`document.querySelector('#org-restore').click()`);
await f.wait(`/Bus started\. It answers\./.test(document.querySelector('#org-archive-success')?.innerText || '')`);
await f.wait(`!!document.querySelector('#workspace-nav [data-workspace-org=beta]')`);
await f.b.evaluate(`location.hash = '#/orgs/beta/organization'`);
await f.wait(`/bus running, read/.test(document.querySelector('[data-org-dashboard]')?.innerText || '') && !!document.querySelector('[data-org-dashboard] [data-system-type-chips] .ws-chip')`);
assert.doesNotMatch(await f.b.evaluate(`document.querySelector('[data-org-dashboard] [data-system-type-chips]').textContent`), /unavailable/);
assert.deepEqual(bus.starts.map(([id]) => id), ['beta']);
});
test('restore does not announce success for an unexpected response', { timeout: 45000 }, async t => {
const f = await fixture(t); await f.open('beta');
await f.b.evaluate(`document.querySelector('#org-archive-open').click();document.querySelector('#org-archive-final').click()`);
@@ -0,0 +1,94 @@
// E1 (P1-5): pages follow a state change without a reload. Restart bus redraws the organization
// dashboard's bus line and System types at once, and Quick Setup puts the new organization in the
// sidebar. Neither waits for the 10 s repaint: each check runs well inside that window.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { startServer } from '../src/serve.mjs';
import { createDevOwner } from '../src/dev-owner.mjs';
import { createOrgAdmin } from '../src/org-admin.mjs';
import { actionTrail } from '../src/action-trail.mjs';
import * as service from '../../organization/src/org-service.mjs';
import { browser } from './browser.mjs';
import { close } from './fixture.mjs';
import { systemTypesFake } from './helpers/system-types-fake.mjs';
const waitFor = (b, tries = 150) => expression => b.evaluate(`(async()=>{for(let i=0;i<${tries};i++){if(${expression})return;await new Promise(r=>setTimeout(r,30))}throw Error(${JSON.stringify(expression)})})()`);
// One organization whose bus state the fake bus below owns. A stopped bus has no reads, as
// organizations.mjs resolves it; a running one answers the dashboard's reads and System types.
async function westOrgs(state) {
const st = await systemTypesFake({ enabled: ['coding'] });
const running = () => ({
busHealthy: true, requestRead: st.requestRead, systemTypesWrite: null,
reads: { business: async () => ({ business: { id: 'west', human: 'jason', arbiters: [], projects: [], vars: {}, instances: [] } }), settings: async () => ({}), queue: async () => ({ rows: [] }), busStatus: async () => ({}) },
bus: { instances: async () => ({ business: 'west', revision: 'a'.repeat(64), launch: true, noHire: [], retired: [], definitions: {}, instances: {} }), messages: async () => ({ messages: [], unread: 0 }), agents: async () => [], 'thread.claims': async () => [], inbox: async () => [], tasks: async () => [] },
chat: { projects: async () => [], threads: async () => [] },
});
return {
list: async () => [{ id: 'west', label: 'West', busHealthy: state.running }],
resolve: async id => id !== 'west' ? null : state.running ? running() : { busHealthy: false, reads: null, bus: {}, chat: { projects: async () => [], threads: async () => [] } },
};
}
test('E1 (P1-5b): Restart bus redraws the dashboard bus line and System types without a reload or the 10 s repaint', { timeout: 45000 }, async t => {
const state = { running: false }; const starts = [];
const bus = {
busStatus: async (ctx, id) => ({ org: id, healthy: state.running, pid: state.running ? 4242 : null }),
busStart: async (ctx, id, opts) => { starts.push([id, opts]); state.running = true; return { org: id, pid: 4242, socket: '/data/s', root: '/data/org-runtime/west' }; },
busRestart: async () => { throw Error('not reached: the bus is stopped'); },
};
const orgAdmin = createOrgAdmin({ service: { showOrg: async (ctx, id) => ({ id, revision: 1, doc: {} }) }, registry: { listOrgs: async () => [] }, bus, context: async () => ({ configDir: '/cfg', dataRoot: '/data', instance: 'test' }) });
const root = mkdtempSync(join(tmpdir(), 'e1-refresh-')); t.after(() => rmSync(root, { recursive: true, force: true }));
const server = await startServer({ port: 0, sso: createDevOwner(), send: async () => ({}), organizations: await westOrgs(state), orgAdmin, trail: actionTrail(join(root, 'console')), log: () => {} }); t.after(() => close(server));
const b = await browser({ machine: false }); t.after(() => b.close());
const wait = waitFor(b);
await b.navigate(`http://127.0.0.1:${server.address().port}/#/orgs/west/organization`);
await wait('document.querySelector("[data-org-dashboard] [data-bus-restart]") && /bus unavailable, read/.test(document.querySelector("[data-org-dashboard]").innerText)');
assert.match(await b.evaluate('document.querySelector("[data-org-dashboard] [data-system-type-chips]").textContent'), /System types unavailable/);
// The page's own repaint is 10 s after its last draw; mark the draw this test starts from.
await b.evaluate('window.__drawn = Date.now(); window.__first = document.querySelector("[data-org-dashboard]")');
await b.evaluate('document.querySelector("[data-org-dashboard] [data-bus-restart]").click()');
await wait('/It answers/.test(document.querySelector("[data-org-dashboard] [data-bus-restart-receipt]")?.textContent || "")');
// No reload: the page keeps its globals, and the dashboard shows the running bus and the enabled type.
await wait('/bus running, read/.test(document.querySelector("[data-org-dashboard]")?.innerText || "") && !!document.querySelector("[data-org-dashboard] [data-system-type-chips] .ws-chip")');
assert.ok(await b.evaluate('Date.now() - window.__drawn') < 8000, 'the dashboard redrew before the 10 s repaint');
assert.equal(await b.evaluate('typeof window.__first'), 'object', 'the page was not reloaded');
assert.doesNotMatch(await b.evaluate('document.querySelector("[data-org-dashboard] [data-system-type-chips]").textContent'), /unavailable/);
assert.match(await b.evaluate('document.querySelector("[data-org-dashboard] [data-bus-restart-receipt]").textContent'), /^Bus started at .*It answers\./);
assert.deepEqual(starts.map(([id]) => id), ['west']);
});
test('E1 (P1-5a): Quick Setup from inside another organization lists the new one in the sidebar without a reload', { timeout: 60000 }, async t => {
const registry = await import('../../organization/src/org-registry.mjs');
const dir = mkdtempSync(join(tmpdir(), 'e1-quick-'));
const configDir = join(dir, 'config'), dataRoot = join(dir, 'data'); mkdirSync(configDir, { mode: 0o700 }); mkdirSync(dataRoot, { mode: 0o700 });
let started = false;
const bus = { busStatus: async () => ({ healthy: false }), busStart: async () => { started = true; return { healthy: true }; } };
const admin = createOrgAdmin({ service, registry, bus, context: async () => ({ configDir, dataRoot, instance: 'test' }) });
const projects = [];
const shell = { send: async () => ({}), busHealthy: true, chat: { projects: async () => projects, threads: async () => [], createProject: async args => { const row = { id: '20000000-0000-4000-8000-000000000002', ...args }; projects.push(row); return row; } } };
const organizations = {
list: async () => [{ id: 'east', label: 'East', busHealthy: true }, ...(started ? [{ id: 'alpha', label: 'alpha', busHealthy: true }] : [])],
resolve: async id => (id === 'east' || (id === 'alpha' && started)) ? shell : null,
};
const server = await startServer({ port: 0, sso: createDevOwner(), organizations, orgAdmin: admin, send: async () => ({}), board: 'http://127.0.0.1:9', log: () => {} }); t.after(() => close(server));
const b = await browser({ machine: false }); t.after(() => b.close());
t.after(() => rmSync(dir, { recursive: true, force: true }));
const wait = waitFor(b, 300);
await b.navigate(`http://127.0.0.1:${server.address().port}/#/orgs/east/settings/organizations`);
await wait('document.querySelector("[data-workspace-org=east]") && document.querySelector("#org-new-form [data-write]") && !document.querySelector("#org-new-form [data-write]").disabled');
assert.equal(await b.evaluate('document.querySelector("[data-workspace-org=alpha]")'), null);
await b.evaluate('window.__first = true');
await b.evaluate(`document.querySelector('#org-new-id').value='alpha';document.querySelector('#org-new-form [data-write]').click()`);
await wait('document.querySelector("#org-quick-open") && !document.querySelector("#org-paths").hidden');
await b.evaluate(`document.querySelector('#org-quick-open').click();document.querySelector('#org-risk-accept').click();document.querySelector('#org-quick-run').click()`);
await wait(`document.querySelector('[data-quick-step="5"] .org-pill')?.textContent === 'Done'`);
await b.evaluate('window.__done = Date.now()');
await wait('!!document.querySelector("[data-workspace-org=alpha]")');
assert.ok(await b.evaluate('Date.now() - window.__done') < 8000, 'the sidebar followed the setup, not a later poll');
assert.equal(await b.evaluate('window.__first'), true, 'the page was not reloaded');
assert.equal(started, true);
});