Compare commits

...
52 changed files with 2785 additions and 17 deletions

No files matched your search

+1
View File
@@ -5176,3 +5176,4 @@ Before: managed project/thread verbs had no Console conversation area. After: fi
2026-10-20 05:10:34 UTC | Row 178 #1676 round 2 verified | Before: C1 render refusals lacked killing tests and row 181 base was pending. Rebased onto 931651f2d, kept all COUNTED/count code and tests, appended T178 tail; added independent owner/collision/absolute/charset/header and empty-loopback proof tests. Four files singly 156/156, all 24 mutants killed, metadata and sandbox fixture probes pass; scratch sweep empty. Packet agents/rocko/work/queue-178/r2/, candidate 9300a670, inventory d0dcde7b (59/59), scan 58 files no findings; in-review revs 2166-2167, expected no-token request failure. No live actions or commits.
2026-10-21T01:28:05Z | Darkwing | row 232 #1734 installer I1 build, round 1 in review | Before: stack-release.sh managed only user units (systemctl --user), and stack-copy.sh had no bundle copy. After: --units external (adopt, activate, rollback --to, status, confirm, revert, gate --since, AWAITING-UNITS, stale-start, external-pm, --plan --expect-from) and stack-copy check --units external, copy --bundle FILE SHA, wrapper ID --create; user mode byte-identical to the base by test. First build 0af0c73c6 on 00d4cc1f6; rebased onto row 200 (cb5fdb064) as a0ffabc3f on darkwing/row232-r1; --sso refused under external (D9), row 200 config_checks run under external (D10). Gate on Sage's slot (sage-1618): cli 561/0, every scripts/test-*.sh green; 17/17 mutants killed. Results commit 84f1a91e6 (pushed, seat branch only). Manifest 7a3b4ea1 (9 paths), queue revs 2595-2596, request comment 29825, reviewer Filbert. Packet agents/darkwing/work/queue-232/BUILD.md.
2026-10-21T04:50:44Z | Darkwing | row 233 #1735 installer I3 build, round 1 in review | Before: root had no upgrade driver for root-owned units and no host run sheet; stack-release.sh had no rehearse --backup last and no external plan refusals. After: scripts/stack-upgrade.sh upgrade and rollback (r6 A1-A5, A2f, R1-R4) with 20 tests, docs/guides/host-run-sheet.md, TOOLS.md lines, and on Sage's rulings stack-release.sh --backup last, BACKUP_STORE and the --plan refusals open-activation and migrate-required (3 tests). Base e65a3ab4d (row 232's landing). Gate on Sage's slot (sage-1704) on d99166ade: cli 616/0, every scripts/test-*.sh green, 1201/0 summed by awk; m1-m8 8/8 killed. Correction: the process sweep after the gate found an orphaned sleep 60 from the lock test's flock holder (N1); 08db6ac3f fixes it test only, Sage ruled the gate stands (sage-1707), rerun c0 20/0 and m1 killed with 0 left (sage-1708). Tip 1a1dc80ca (pushed, seat branch only). Manifest 464ce9eb (7 paths), queue revs 2644 and 2647, request comment 29901, reviewer Filbert. Packet agents/darkwing/work/queue-233/BUILD.md.
2026-10-21T09:00Z | Herb | row 215 C2-04 implementation started | Before: business files reject library entries and root consumers assume code projects; the organization writer can change all project mappings. Plan: approved manifest 2522025af on C2-01 a2c72ac95, Filbert reply to herb-0056, scope sage-1848/1852/1860. Focused slot sage-1869 observed expected red mapping and M9 cases, then 14/0 green for new T1/T2 cases, no skips; clear herb-0062 with empty test/fixture process scan. After that clear, T3/T4 tests and host-only mirror/bus implementation continue under sage-1874, unrun while Sage holds the slot. Full singles, suites and implementation review remain pending. No source commit, push or live change.
+53
View File
@@ -0,0 +1,53 @@
# Row 212 build notes, C2-01 packages/library (#1713, Vikunja #416)
## Rounds
- Round 1: bdf18272a on gosalyn/c2-01, base 32b54ac3b. Webby approved it in Gitea comment 29704.
- Round 2: d3c7bbf839d9c3bf50f154b18e7e4472fbd005ed on gosalyn/c2-01-r2, rebased onto 9b5362018 after row 232 drifted three paths (sage-1714). Webby approved it at merge scope, rev 2666 against comment 29921 (webby-0052).
- Round 3: this commit on gosalyn/c2-01-r3, one commit on the row 234 landing c771a36f4 (sage-1736, sage-1779, mos-1160). Row 234 changed scripts/mosaic, so round 3 review covers that one file.
## Round 2 merge
range-diff 32b54ac3b..bdf18272a against 9b5362018..d3c7bbf83 shows one change, the context line where upstream added `credentials` to the scripts/mosaic case line. Patch-id moved from 8f80dfa3 to 175f45c6 for that reason only. All three drifted hunks keep both sides:
- scripts/stack-copy.sh COPY_PATHS reads `packages/queue/src packages/library/src`. Row 232's bundle, --create, --units and instance code is unchanged.
- scripts/mosaic runs the `library survey` exec block before the case line, and the case line keeps `credentials`.
- packages/cli/tests/stack-copy.test.mjs adds `packages/library/src/index.mjs` to the copy file list.
## Round 2 singles, evidence for the 19 paths other than scripts/mosaic
Run on d3c7bbf83 in Sage's slot (sage-1723) from 2026-10-21T05:00:09Z to 05:11:29Z. The HEAD and a clean tree were checked first. The counts are an awk sum over the 23 logs: 481 passed, 0 failed, 0 skipped, 0 cancelled, and every file exited rc=0.
| Package | File | Pass |
|---|---|---|
| library | links | 6 |
| library | mapping | 7 |
| library | reader | 11 |
| library | render | 11 |
| library | survey | 4 |
| cli | app | 30 |
| cli | case2-unit | 30 |
| cli | credentials | 5 |
| cli | host | 42 |
| cli | instances | 5 |
| cli | project | 4 |
| cli | stack-copy | 16 |
| cli | stack-instances | 23 |
| cli | stack-release-external | 25 |
| cli | stack-release | 192 |
| cli | tester-reads | 6 |
| cli | tester-threads | 5 |
| cli | usage | 2 |
| queue | dispatch | 4 |
| runs | task-cli | 9 |
| webui | pinned-copy | 10 |
| webui | service | 31 |
| webui | views | 3 |
The file list comes from the library tests plus every test that names scripts/mosaic or stack-copy.sh, by basename and by join() path. No 64-hex pin in the tests names a merged path.
## Round 3 scripts/mosaic
The only path both row 234 and this row change is scripts/mosaic. Row 234 added `hosts` to the cli case line and changed nothing else in the file. Round 3 takes the landed blob cce1fdee4 and adds this row's six-line `library survey` block before the case line by hand, so `hosts` stays. The mode stays 100755. The other 19 paths are byte-identical to d3c7bbf83. Against d3c7bbf83 the only difference is the `hosts` context line.
Round 3 runs only the tests that read scripts/mosaic, in Sage's slot. Their counts go to Sage in T3 and onto #1713, since this file can't hold them and stay one commit. Those tests are library survey, cli app, case2-unit, credentials, hosts, host, instances, project, stack-copy, stack-release, stack-release-external, stack-upgrade, tester-reads, tester-threads and usage, queue dispatch, runs task-cli, webui pinned-copy, service and views, plus scripts/test-queue.sh. hosts.test.mjs is new with row 234.
+271
View File
@@ -0,0 +1,271 @@
# Row 215 build record
Owner: Herb. Reviewer: Filbert. Claim: herb-215-start, rev 2753.
Brief: docs/plans/2026-10-20_case2-rows.md, C2-04, pin
16847e94ae1768881711318fe81cbb5e6c66b971. Design: Gosalyn r11,
sections 1.1, 1.3, 2.1, M4, M9 and 7.2.
Base: a2c72ac952df94ecab7e391d5042f49b7f88b816, row 212's reviewed
candidate, under Q177. Branch: herb/row215. Rebuild on the combined 212+197
landing when Sage names it. Do not replay or revert landed changes.
## Scope rulings (sage-1848)
1. Add packages/bus/src/runtime.mjs for library adapter injection and the
library skip in protectedRoots. The skip has its own consumer test. Change
nothing else there.
2. Add packages/library/src/mirror.mjs, named mirror rather than store. It
creates a bare mirror without a remote, reads at the served commit and
reads sync state. Export one state filename constant, and export this
module through packages/library/src/index.mjs. Row 222 imports the constant.
3. Add packages/bus/src/server.mjs only to await library.file and
library.status, with socket tests proving their replies are results, not {}.
4. HTTP exposure is deferred. Row 225 owns the library.status route;
C2-11b owns library pages. Leave serve.mjs alone. webui/src/cli.mjs gets
only callback wiring next to the project-file callbacks on both listeners.
5. Reuse existing refusal codes where possible.
6. Build on a2c72ac95 until Sage supplies the 212 landing SHA.
Additional ruling, sage-1852: library-state-invalid is approved for malformed,
unsafe or mismatched state, including a missing file while the served ref
exists. The version-1 shared contract below is approved. Row 222 also owns
runtime.mjs for its sync and timer wiring, recorded in that row's manifest.
Additional rulings, sage-1860, after Filbert's CHANGES on 6a508804f:
- B1 adds packages/cli/src/config.mjs and cli/tests/trackers-boot.test.mjs.
trackerFor skips library entries. A real bus boot with tracker.baseUrl and
a library entry succeeds and gets no tracker from that entry.
- B2 refuses a role publish grant naming a library project in the loader.
- B3 readers use the sync lock. mirror.mjs exports withLibraryLock, with a
5000 ms wait bound and library-sync-busy on timeout. Sync locks only the
served-ref move and state publication, never the fetch.
- N3, git-socket doPush/recordPush accepting a library ID, is a later row.
Sage records it in DEFERRED.md when 215 lands. No fix here.
## Shared contract with row 222
Agreed in herb-0048, megavolt-0081/0082 and sage-1852. This section is the
contract Filbert reviews and Megavolt cites for Dewey. It is not a sync writer.
Module: packages/library/src/mirror.mjs. Export
`SYNC_STATE_FILE = 'mosaic-library-sync.json'`. The full path is
`join(mirror, SYNC_STATE_FILE)`, inside the bare repository. Only sync.mjs
writes it, using a sibling temp file, fsync, rename, and directory fsync.
Version 1 has exactly these keys:
```json
{
"version": 1,
"branch": "main",
"served": null,
"lastFetch": null,
"lastError": null,
"diverged": null,
"problems": []
}
```
- branch is the configured mapping branch and must match the reader's branch.
- served is null or a full lowercase 40-hex commit ID. refs/heads/served is
authority. The state value must equal that ref, which must name a commit.
- lastFetch is null or a canonical ISO UTC timestamp.
- lastError is null or exactly {code, at}. code matches [a-z-]{1,64}; at is
canonical ISO UTC. It never holds stderr, prose or a credential value.
- diverged is null or exactly {commit, at}, with the same commit/time rules.
- problems uses C2-01's exact {code, path, message, line?}. code is R1-R7,
message is that code's fixed RULES text, line is a positive safe integer
when present. No file body or arbitrary message is accepted.
- An absent file and absent served ref return the empty object above with
the configured branch. An absent file with an existing served ref refuses.
- Unknown keys, bad types, unsafe filesystem metadata, invalid JSON and a
mismatch refuse library-state-invalid. Reads use a bounded regular owned
private file without following links. The read never repairs state.
- refs/remotes/library/<branch> is not duplicated in this file. Row 222 may
read it when it needs the fetched tip.
Exports consumed by row 222:
| Export | Contract |
| --- | --- |
| mirrorPath({business, project, dataRoot}) | Require a library entry; return projectRepo's checked path |
| ensureMirror(context) | Promise of the mirror path; race-safe idempotent bare init; no remote or fetch |
| servedCommit(mirror) | Promise of full commit ID or null; refs/heads/served only |
| readSyncState(mirror, {branch}) | Promise of checked version-1 state; compare with served ref |
| withLibraryLock(mirror, operation) | Await the same exclusive lock as readers, then await operation({readSyncState}); always release |
The lock is a stable private regular file, mosaic-library-sync.lock inside
the mirror. It is never unlinked. A kernel flock is held by the caller's
file descriptor. Acquisition waits at most 5000 ms and refuses
library-sync-busy on timeout. Unsafe lock metadata refuses library-state-invalid.
There is no fs.flock API. The exact protocol is: open the stable file with
O_NOFOLLOW, check regular/private/owned/single-link metadata, pass its fd as
child FD 3 to `flock --exclusive --wait 5 --conflict-exit-code 75 3` using
asynchronous spawn, await
that child's exit 0, then run the callback while the parent still holds its
original descriptor. Both descriptors share the open file description, so
the lock survives acquisition-child exit. Finally close the parent's fd.
Process death also closes it. No holder child stays alive in the callback,
no stale file is deleted, and unrelated operation children inherit only 0-2.
Acquisition never uses spawnSync: the bus must serve other calls while a
library reader waits (megavolt-0087).
A scratch-only probe on node v26.8.1 at 2026-10-21T08:34:00Z confirmed this:
acquisition child exited 0; a second flock -n returned 1 while the parent fd
stayed open; after closeSync it returned 0. The probe removed its own root.
This was a design probe, not a package test, suite, Docker command or proof.
readSyncState takes that lock around reading the served ref and state file.
The callback-scoped readSyncState({branch}) supplied by withLibraryLock reads
under the already-held lock and does not reacquire it. The helper awaits
asynchronous callbacks and releases in finally. A callback-scoped reader
used after its callback ends refuses; it cannot escape its lock lifetime.
Row 222 fetches without this lock, then calls withLibraryLock across the
served-ref move and atomic state write. Its tests hold publication between
those steps and prove a concurrent reader waits, then sees the new consistent
pair, including first sync. Row 215 tests that publication shape and timeout.
A file read releases the lock after snapshotting a consistent served SHA,
then reads that immutable commit. Row 222's separate outer sync lock spans
fetch to prevent two sync operations interleaving (megavolt-0086).
Host adapter exports for this row:
- libraryStatus(context): return {project, ...state}, with no repository path
or content. Do not fetch or rebuild on this read.
- readLibraryFile({...context, path}): read a mapped, readable file at the
served commit, returning {project, path, commit, content}. No caller-picked
commit, working-tree fallback, fetch or write. C2-01's R1/R2/R7 exclusions
apply. A never-synced mirror refuses library-commit for a file read.
Row 222 owns syncCommand, sync, writeSyncState, its outer sync lock, the timer
and state publication. Row 215 owns the shared publication/read lock per
sage-1860. Sync rebuilds problems with existing readLibrary({repo, commit,
mapping}). It does not need a second rebuild function from this row.
## Root consumer audit and reviewer notes
On a2c72ac95, ran exactly the sage-1860 audit:
`git grep -n 'entry\.root\|\.root)' -- 'packages/*/src/*.mjs'`.
It has 14 hits. runtime.mjs:95, business/cli.mjs:61,72,78,118,
instance-settings.mjs:142-143, resolve.mjs:44, cli/config.mjs:62-63 and
project/paths.mjs:42 all get the library guard in this row. business.mjs:233
and :242 are code-project validation after the library dispatch, not an
unguarded consumer. queue.mjs:1144 compares queue genesis roots and never
iterates business.projects; it is safe and unchanged. A second search for
projectFilePath/business.projects/b.projects found no additional root reader.
ensureCheckouts destructures root rather than spelling .root; its guard and
the launcher's filter remain in T1. Bare projectFilePath/loadProject remain
strict and unchanged.
Filbert N1 is handled at the existing owned paths.mjs boundary: projectRepo
defaults to record-project use and refuses a library entry with project-invalid.
Only mirrorPath's internal call opts into allowLibrary. Thus ordinary human
and tester project init cannot occupy an uninitialized library mirror path.
No record API or bus argument exposes that internal option.
Filbert N2: ensureMirror failure refuses bus boot. No socket becomes ready.
The runtime closes resources it opened, following its existing failure path.
A T4 test uses an unsafe/pre-existing mirror and proves the refusal, rather
than serving a half-configured library adapter.
## Verification record
Planning stage. No product code, package test, suite, image build, live operation
or push for row 215 has happened. Product implementation awaits Filbert's
manifest approval. Sage owns suite and full-gate grants. Row 197's proof
commit is separate and is not part of this row's changes.
## Implementation ledger
Plan: agents/herb/work/queue-215/MANIFEST.md at 2522025af, approved by
Filbert in the reply to herb-0056. Execution is inline by Herb under the
executing-plans and test-driven-development skills. No agent delegation.
Pre-flight: T1's validated mapping feeds T2's protected document and T3's
mirror context; T3's lock/state helpers feed T4's injected host adapter.
The shared contract is pinned above, with no interface conflict found.
Sage-1867 overrides COMMON's singles allowance: every test and executable
probe needs an explicit slot grant. No further probe/test has run since
that reminder. Read-only inspection and test authoring continue while the
slot is held. The first red execution is pending; product code is unchanged.
Filbert's approved-manifest build notes: ensureMirror creates the lock in
its staged directory before publication; a missing lock in an existing
mirror refuses library-state-invalid. The top-level metadata allowlist
permits only normal bare Git entries plus the state and stable lock files.
Tests cover an unknown entry. The host needs /usr/bin/flock; ENOENT yields
a handled library-state-invalid refusal, never an unhandled child error.
Lock acquisition uses /usr/bin/flock directly and asynchronous spawn.
T1: consumer and loader regression tests authored, not run. T2: direct M9
write and serialization regression tests authored, not run. No completion
or passing claim is made for either task.
Update, sage-1869 focused slot, 2026-10-21T08:47:48Z to 08:51:07Z:
the busy check was CLEAR before the first test. T1's mapping regression
failed 0/1 on unsupported kind as expected, then passed 2/0 with the publish
refusal included. Other focused greens: business CLI 1/0, resolver 1/0,
instance settings 1/0, seat checkout 1/0, project paths 1/0, sandboxed launcher
1/0, and trackers-boot whole 2/0. T2's direct writer case failed 0/1 on the
missing refusal, then its whole new file passed 4/0. Green total 14/0 with
no skips or cancellations. Logs and counts: granted-t1/. The final /proc
scan found no test process or fixture cwd/argv reference; its private root
was removed. Clear sent as herb-0062. Existing files' other cases have not
been run yet, so T1/T2's full-file acceptance is still pending.
Sage-1874 directs continued authoring and implementation while the landing
gate/proof holds the slot. T3/T4 tests were written first. Their initial
red runs cannot execute during that hold; implementation proceeds under
that owner direction, with no claim of a witnessed red or green for T3/T4.
Their baseline red and candidate checks need a later explicit grant.
Implementation ruling: initialization uses GNU /usr/bin/mv --no-clobber
--no-target-directory for its bounded staging rename, rather than Node's
renameSync, which could replace an empty directory made by a competing
actor. This preserves pre-existing directories and needs no new package on
the Linux/coreutils bus host. The host also needs /usr/bin/flock. Neither
command has been executed for T3/T4 yet. Publication errors fail closed.
Sync-state reads are capped at 8 MiB, including a checked extra byte; an
oversized or concurrently changed file refuses library-state-invalid.
T5 static dependency audit found two partial pinned-copy fixtures that copy
business/src but not its newly imported library mapping validator:
cli/tests/stack-release.test.mjs:1772 and stack-release-fixture.mjs:309 on
the base. Each now includes packages/library/src in the existing copy list.
This changes fixture closure only, no assertion or timeout. The complete
stack-release file and each test file importing that shared fixture require
a later granted single-file run. They are outside the four-file T3/T4 slot.
Sage-1879 places the four-file T3/T4 focused run after their 212+197 proof,
before Gizmoduck, and requires their explicit clear before starting. No run
has occurred after herb-0062. Row 197's N7 base-image-tag finding is left to
Sage's DEFERRED entry; no row 197 source change is made here.
Focused T3/T4 grant sage-1882, 2026-10-21 09:13-09:18Z: busy check
CLEAR. Final whole-file results: mirror 16/0, bus library 6/0, WebUI
library-cli 1/0, project-runtime 3/0, all with zero skips/cancellations.
Syntax checks passed for the four files and five changed host modules.
Logs, all intermediate failures, counts and empty /proc cleanup scan are
in granted-t34/. Baseline a2c72ac95: three focused bus cases 0/3
(unknown-verb), CLI 0/1 (missing callbacks). Mirror baseline 0/1 is an
import refusal for the new module, not a witnessed behavioral red.
An initial whole baseline bus attempt left a runtime after its expected
failure; its exact runner was stopped, no process remained, and the
baseline was narrowed to the three bounded broker/socket cases.
Candidate fixture setup errors and a mock-child timeout were corrected
before the final whole-file passes. No suite or Docker/image/proof ran.
Filbert's 09:09 publication note: no dependence on mv -n's skipped-rename
exit status across coreutils versions. Save staging dev/ino before mv;
only its disappearance plus the same dev/ino at the mirror proves a win.
A remaining original staging inode means a loss, requiring a valid winner.
Only that owned original staging inode is removed. Numeric no-clobber
statuses 0 and 1, including a diagnostic, are tested with a competing
winner. Spawn errors, signals/timeouts and inconsistent outcomes refuse.
Coreutils must provide atomic --no-clobber --no-target-directory semantics;
it need not use any particular skipped-rename status convention.
+214
View File
@@ -0,0 +1,214 @@
# C2-04 library business entry implementation plan
> Execution: Herb implements in this session using the executing-plans skill.
> Filbert approves this manifest before any product implementation.
**Goal:** Load a library mapping without giving an agent a checkout, protect
all project mappings from organization writes, and expose human-only bus
reads of the bare mirror's served commit and sync status.
**Architecture:** C2-01 checks mappings and reads commits. Business consumers
skip library entries. A host-only mirror module owns local mirror reads and
initialization. runtime.mjs injects that module into the broker, preserving
the broker's sandbox import closure. Sync and HTTP routes belong to later rows.
**Tech stack:** Existing Node ESM, node:test and bounded local Git helpers.
No dependencies. node_modules remains the plain canonical symlink.
**Spec:** Pinned C2-04 brief; Gosalyn's approved design 1.1, 1.3, 2.1,
M4/M9 and 7.2; BUILD.md's sage-1848/1852/1860 scope and shared-state contract.
## Constraints and review focus
- Base a2c72ac95; rebase only onto Sage's named 212 landing on a new branch.
- Existing code-project entries retain root/repo/remote behavior. Library
entries have no root or repo override and use C2-01 checkMapping.
- Preserve the broker's pinned imports. No library import in broker.mjs and
no new runner mount. Do not edit serve.mjs or add an HTTP route.
- No fetch, remote config, credential read, timer, sync writer, bus write or
live operation. All tests use made-up content and local repositories.
- Singles may run under COMMON; suites and any long run need Sage's grant.
Stop and report two failures in one gate. Do not push without authorization.
Review focus, with owning tasks below: library-first iteration must not
break a later code project (T1); serialized documents cannot bypass M9
(T2); racing initialization must not replace another mirror (T3); stale or
unsafe state must not leak content (T3); async socket replies must preserve
the result and human-only admission (T4).
## Paths and evidence from the base
| Product path | Allowed change |
| --- | --- |
| packages/business/src/business.mjs | Dispatch kind:library to checkMapping before code-root validation (:228-244) |
| packages/business/src/organization.mjs | Compare canonical projects in write before publication (:177 onward) |
| packages/business/src/cli.mjs | Skip library roots in validate, resolve --project and credential exclusion roots (:60-61, :71-79, :118) |
| packages/business/src/resolve.mjs | Skip library entries while matching a project file (:44) |
| packages/business/src/instance-settings.mjs | Skip library before projectFilePath (:142), nothing else |
| packages/seat/src/checkout.mjs | Skip library entries in ensureCheckouts (:315-327) |
| packages/cli/src/launcher.mjs | Pass only code entries to ensureCheckouts (:354) |
| packages/cli/src/config.mjs | Skip library entries in trackerFor's root loop (:61-63), per sage-1860 |
| packages/project/src/paths.mjs | Skip library entries in overlap loop (:41-44); default record use refuses library, mirrorPath internally opts into allowLibrary |
| packages/library/src/mirror.mjs | New bare init, checked state and served-commit file read module |
| packages/library/src/index.mjs | Export mirror module interface and SYNC_STATE_FILE |
| packages/bus/src/broker.mjs | New injected library adapter, human-only admission and two async verbs |
| packages/bus/src/runtime.mjs | Library adapter wiring, mirror initialization and protectedRoots skip only |
| packages/bus/src/server.mjs | Await the two library verbs only |
| packages/webui/src/cli.mjs | Two library read callbacks beside project callbacks on main and tester listener; no routes |
packages/business/src/project.mjs need not change: callers skip a library
before projectFilePath or loadProject. Do not weaken their code-root checks.
scripts/stack-copy.sh already copies packages/library/src and needs no edit.
Test paths to modify: business/tests/business.test.mjs, cli.test.mjs,
resolve.test.mjs, instance-settings.test.mjs; seat/tests/checkout.test.mjs;
cli/tests/launcher.test.mjs, trackers-boot.test.mjs; project/tests/project.test.mjs;
bus/tests/project-runtime.test.mjs. All paths are under packages/.
T5 dependency-audit addition: cli/tests/stack-release.test.mjs and its
stack-release-fixture.mjs helper copy packages/library/src alongside
business/src, whose validator now imports the mapping checker. No assertions
or timeouts change. Whole-file dependent singles require a later Sage slot.
New tests: packages/business/tests/library-projects-protected.test.mjs;
packages/library/tests/mirror.test.mjs; packages/bus/tests/library.test.mjs;
packages/webui/tests/library-cli.test.mjs. Reuse existing made-up fixtures.
If a new fixture is needed, define it within the corresponding new test file.
No unrelated test assertion or timeout changes.
Record paths: agents/herb/work/queue-215/{MANIFEST.md,BUILD.md}, later source
manifest and test receipts there; one append-only BUILD-LOG.md phase for this
row when implementation begins. Sage owns docs/SESSIONS.md.
## T1. Mapping and consumer skips
- [ ] Add failing business tests: accept a frozen library mapping without a
root, retain mixed/only-library documents, and propagate C2-01's mapping
and playground refusal codes. A role publish naming a library project
refuses at load, even with an otherwise valid fork identity and remote.
Existing code validation remains unchanged.
- [ ] Add a separate regression for each consumer: CLI validate and its
credential exclusion roots; CLI resolve --project books has no project
layer; resolveInstance finds a later real project; instance settings still
writes an unrelated setting; ensureCheckouts returns no library checkout;
launcher creates no books workspace clone; projectRepo skips a library
while still rejecting overlap with a real code root. Put books first.
- [ ] Add the real bus boot regression in cli/tests/trackers-boot.test.mjs:
tracker.baseUrl is set and books is a library entry; boot succeeds, books
produces no tracker, and a later real tracker project still works.
- [ ] projectRepo(context, {allowLibrary = false} = {}) refuses a library
entry by default with project-invalid. Only mirrorPath opts in. Test that
ordinary human and tester record initialization leave no directory at the
future mirror path; no public verb can supply allowLibrary.
- [ ] Add runtime's protectedRoots consumer regression in T4's real runtime
fixture. A valid credential outside code/data roots still loads with a
library entry, and existing credential protections still refuse unsafe files.
- [ ] Run each changed test singly, confirm the new cases fail for the
intended root assumption, then implement only the table's guards and
checkMapping dispatch. Re-run the same files singly.
CLI validate reports a library entry as library, without an absent-project
warning. CLI resolve --project books resolves without loading a project file,
as the design's skip requires. It does not fabricate project vars or a root.
## T2. M9 at the organization writer
- [ ] In the new business test file, call organizationWriter.write directly
with a changed playground prefix and, separately, a changed hybrid entry.
Both refuse organization-projects-protected. Snapshot business bytes and
changes.jsonl before each call; assert both remain identical afterward.
- [ ] Cover addition, deletion and changes to code projects too, since M9
protects all projects. Reordered object keys are allowed; array changes
are changes. An ordinary non-project write still succeeds and logs.
- [ ] Add a serialization regression so a proposed object's serialized
projects cannot differ from those checked before publication.
- [ ] Run the file singly to see the missing refusal, compare canonical
projects against old disk JSON and the document that actually lands,
before any publication/receipt, then re-run it singly.
## T3. Bare mirror, served file and read-only state
- [ ] Add failing tests for mirrorPath and ensureMirror: exact projectRepo
default, no checkout/remote, retained no-links/overlap protections,
idempotence, simultaneous creators and preservation of a pre-existing
non-mirror directory or unsafe Git metadata.
- [ ] Add failing state tests for BUILD.md's full version-1 contract: missing
state/ref, valid state, malformed JSON, unknown keys/types, branch mismatch,
invalid times/IDs/problem shape, symlink/hardlink/non-private file and
missing or mismatched served ref. Assert reads change no bytes.
- [ ] Add shared-lock tests: withLibraryLock(mirror, operation) waits at most
5000 ms, refuses library-sync-busy on timeout and releases after a throw.
A reader arriving between the publisher's served move and state write
waits, then reads the consistent pair. Cover first sync and fast-forward
in independent processes, same-process independent acquisitions and
release after owner death. A callback-scoped reader expires on release.
- [ ] Implement the lock by async spawn of flock on the parent's inherited
fd 3, as pinned in BUILD.md. Await acquisition-child exit before calling
operation. Never spawnSync during acquisition, which could freeze the bus
for 5 s. A parallel ordinary bus read remains responsive while a library
read waits. Only the publication/read lock is shared; fetch is outside it.
- [ ] Add served-read tests against local fixtures containing spaces,
capitals, a .canvas and unmapped top-level note. Seed a served ref and state
without fetching. A newer unrelated HEAD does not change the returned
file. R1/R2/R7 exclusions, traversal, a missing path, a caller commit and a
never-synced mirror refuse. Status contains no fixture body marker.
- [ ] Implement mirror.mjs and index exports using the shared contract.
Initialization stages into an owned private sibling directory and renames
atomically; a losing creator validates the winner and removes only its own
staging directory. Git uses existing bounded safe helpers, no remote.
- [ ] Run mirror.test.mjs singly, red then green. Re-run C2-01 reader.test.mjs
singly if a reader assumption changes. Do not change C2-01 reader rules.
mirror.mjs validates a bare mirror's metadata before Git reads it, including
no remote/include/indirection. It does not apply inspectGit's working-tree
.git assumption to a bare repository. Failures reuse library-git or the
approved library-state-invalid code. State reads are bounded and private.
## T4. Human-only bus reads and deferred Console callbacks
- [ ] Add broker tests for library.file and library.status using injected
adapters. Each agent session refuses human-required before adapter access,
including bad arguments and an absent adapter. A reader or tester does not
gain a human capability. Human calls validate keys, then use checked
library context. Secret checking and bounded refusal evidence remain.
- [ ] Add real startBroker/socket tests for both verbs. Seed only local
fixture mirror data. Assert result shape/content and an actual status
field, not merely successful transport or {}. Cover never-synced status
and runtime's protectedRoots skip. Keep the pinned import-closure test.
- [ ] Inject a separate library adapter via runtime, initialize only library
mirrors, add the two async request paths, and include them in server's
awaited list. Do not import library or fs into broker.
- [ ] Unsafe or pre-existing non-mirror initialization refuses bus boot,
closes opened resources and never publishes a ready socket. Add that
runtime regression in library.test.mjs.
- [ ] Wire main and tester Console callbacks beside project-file callbacks.
No HTTP route is shipped. The new CLI test captures the startServer
options with a test-only module loader and calls the supplied callbacks
against a fixture bus, checking the verb and transport identity. No
product test seam, UI page, endpoint or serve.mjs change is needed.
- [ ] Run library.test.mjs, project-runtime.test.mjs and library-cli.test.mjs
individually, red then green. Human admission may use the existing
fixture authentication seam; do not forge or bypass a live identity.
## T5. Review packet and Sage's gate
- [ ] Search packages/*/tests and scripts/test-*.sh for each changed source
path, imports and 64-hex pins. Record the resulting test-file list and run
each changed/touched test singly. Stop and report a scope error.
- [ ] Self-review the diff against this manifest and all acceptance cases.
Record results, commands, skips, source hashes and remaining limits in
BUILD.md. Request Sage's slot for the required business, seat, cli, project,
bus and webui suites and every scripts/test-*.sh. Do not run them ungranted.
- [ ] After applicable tests are green, make the source commit, build the
candidate manifest from the authorized pushed tip, and move the row with
scripts/mosaic. Sage posts token-requiring requests; Filbert reviews the
candidate. Push only when authorized. No merge or live change by Herb.
## Self-review
Every brief acceptance item is in T1-T4. Added host support paths match
sage-1848/1860. BUILD.md records the complete 14-hit root audit. State shape
and authority match sage-1852; the shared read/publication lock follows
sage-1860 and Megavolt's agreement. Fetch, timers, outer sync serialization
and HTTP stay with their later rows.
This is a plan only; no product edits or test execution precede approval.
+26 -1
View File
@@ -88,6 +88,7 @@ export const TASK_PAGE_MAX = 200;
// Row 148 (#1642): the human-only verbs behind the Console's document editor. The save is the one
// request allowed past the ordinary 64 KiB envelope (server.mjs and client.mjs), up to its content cap.
export const PROJECT_FILE_VERBS = Object.freeze(['project.files', 'project.file', 'project.file.save']);
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status']);
export const PROJECT_SAVE_VERB = 'project.file.save';
export const PROJECT_SAVE_REQUEST_MAX = 1024 * 1024 + 64 * 1024;
export const WEB_VERBS =Object.freeze(['web.search', 'web.fetch']);
@@ -267,6 +268,7 @@ export class Broker {
#organization;
#credentialVerbs;
#project;
#library;
#usage;
#limits;
#harnessOn;
@@ -276,7 +278,7 @@ export class Broker {
#stops = new Map();
#clock = 0;
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
this.#businesses = structuredClone(businesses);
this.#store = store;
@@ -287,6 +289,7 @@ export class Broker {
this.#credentialVerbs = credentialVerbs;
this.#hostGrant = hostGrant;
this.#project = project;
this.#library = library;
this.#usage = usage;
this.#limits = limits;
// Row 65: harnessOn(business, name) is true when the organization file's plugins.harness switches
@@ -1743,6 +1746,8 @@ export class Broker {
return this.#requestProject(s, { verb: request.verb, args: structuredClone(args) });
if (PROJECT_FILE_VERBS.includes(request.verb))
return this.#requestProjectFile(s, { verb: request.verb, args: structuredClone(args) });
if (LIBRARY_VERBS.includes(request.verb))
return this.#requestLibrary(s, { verb: request.verb, args: structuredClone(args) });
if (CREDENTIAL_VERBS.includes(request.verb)) return this.#requestCredential(s, request.verb, structuredClone(args));
if (["limits", "limits.set", "organization.set", "organization.revision", "plugins", "plugins.set", "instances", "instances.set", "routes", "routes.set"].includes(request.verb)) {
// Row 123 (#1617, REQ-COS-1, REQ-LIMIT-1): organization.set is limits.set for a role whose
@@ -1992,6 +1997,26 @@ export class Broker {
throw this.#refused(s, ProjectFailure && e instanceof ProjectFailure ? new BusError(e.code) : e, request);
}
}
// C2-04 M4: only a verified human reads library text or status. Host Git
// functions stay injected, preserving this module's sandbox import closure.
async #requestLibrary(s, request) {
const { verb, args } = request;
try {
this.#human(s);
const fields = verb === 'library.file' ? ['project', 'path'] : ['project'];
keys(args, fields, fields);
if (!this.#library?.readFile || !this.#library?.status) fail('project-not-configured');
const context = { business: this.#business(s.business), project: args.project };
const result = verb === 'library.file'
? await this.#library.readFile({ ...context, path: args.path })
: await this.#library.status(context);
this.#secretCheck(result);
return result;
} catch (error) {
const Failure = this.#library?.LibraryError;
throw this.#refused(s, Failure && error instanceof Failure ? new BusError(error.code) : error, request);
}
}
// The eight task verbs reach the trusted S3 handler here. It runs outside any transaction, gets the
// cap to authorize and record through authorize() and recordTask(); refusals are recorded as in request().
async requestTask(cap, request, handler) {
+12 -1
View File
@@ -3,6 +3,7 @@ import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { Store } from './store.mjs';
import * as projectModule from '../../project/src/index.mjs';
import * as libraryModule from '../../library/src/index.mjs';
import { Broker, BusError } from './broker.mjs';
import { Credentials } from './credentials.mjs';
import { serve } from './server.mjs';
@@ -92,7 +93,7 @@ export async function startBroker({
const protectedRoots = [
fileURLToPath(new URL('../../../', import.meta.url)),
...repoRoots,
...Object.values(businesses).flatMap((b) => Object.values(b.projects ?? {}).map((p) => p.root)),
...Object.values(businesses).flatMap((b) => Object.values(b.projects ?? {}).filter(p => p.kind !== 'library').map((p) => p.root)),
];
// Row 199 (#1697, design D1): the per-agent store is <configDir>/secrets/agents.
const storeRoot = organization ? join(organization.dir, 'secrets', 'agents') : null;
@@ -123,12 +124,22 @@ export async function startBroker({
const withTokens = (id, result) => ({ ...result, tokens: limits.report(id) });
// Row 210 (#1707, design D7 step 6) and row 234 (#1740, r6 E2): the run dir and the host enrollment half.
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
// C2-04: mirrors are host state, never session checkouts. An unsafe
// existing mirror refuses startup before a socket can become ready.
for (const business of Object.values(businesses))
for (const [project, entry] of Object.entries(business.projects ?? {}))
if (entry.kind === 'library') await libraryModule.ensureMirror({ business, project, dataRoot });
const broker = new Broker({
hostGrant: runRoot ? hostGrant({ configDir: organization.dir, runRoot }) : null,
usage,
limits,
store,
businesses,
library: {
readFile: context => libraryModule.readLibraryFile({ ...context, dataRoot }),
status: context => libraryModule.libraryStatus({ ...context, dataRoot }),
LibraryError: libraryModule.LibraryError,
},
project: {
readSpec: context => projectModule.readSpec({ ...context, dataRoot }),
writeRecord: context => projectModule.writeRecord({ ...context, dataRoot }),
+2 -2
View File
@@ -1,6 +1,6 @@
import { createServer } from 'node:net';
import { lstatSync, chmodSync, unlinkSync } from 'node:fs';
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, LIBRARY_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
const LIMIT = 65536;
// Row 148: a document save may carry up to 1 MiB of text. Only a request that names that verb near
// its start (the proof object comes first) may pass LIMIT; the parsed verb is checked again below.
@@ -122,7 +122,7 @@ export async function serve({ broker, path, authenticateHuman = null, authentica
}
// Row138: only project Git effects need an await; preserve every other sync path below.
if (r.verb === 'usage.begin' || r.verb === 'usage.end' ||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) ||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) || LIBRARY_VERBS.includes(r.verb) ||
// Row 234 (#1740): a grant raise awaits its keyscan, and hosts.check its ssh.
HOSTS_VERBS.includes(r.verb) ||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit') ||
+123
View File
@@ -0,0 +1,123 @@
// C2-04 M4 and real async transport, with made-up local library data.
import test from 'node:test';
import assert from 'node:assert/strict';
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { Store } from '../src/store.mjs';
import { Broker } from '../src/broker.mjs';
import { serve } from '../src/server.mjs';
import { Client } from '../src/client.mjs';
import { startBroker } from '../src/runtime.mjs';
import { mappingEntry, repo, git, BASE, MARKER } from '../../library/tests/helpers.mjs';
import * as library from '../../library/src/index.mjs';
function world(t, adapter = {}) {
const root = mkdtempSync(join(tmpdir(), 'bus-library-')), store = new Store(root);
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
const business = { id: 'acme', human: 'jason', vars: { 'bus.tester': true },
projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
const broker = new Broker({ store, businesses: { acme: business }, library: adapter });
const human = broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
const agent = broker.bindLaunch({ business: 'acme', role: 'coder', run: 'library-agent', harness: 'pi' });
const reader = broker.bindReader({ business: 'acme' }), tester = broker.bindTester({ business: 'acme' });
const call = (cap, verb, args = {}) => Promise.resolve().then(() => broker.request(cap, { verb, args }));
return { root, store, broker, business, human, agent, reader, tester, call };
}
test('C2-04 M4: both verbs reject every agent before arguments or adapter access', async t => {
let touched = 0;
const f = world(t, { readFile: () => { touched++; }, status: () => { touched++; } });
for (const verb of ['library.file', 'library.status']) {
await assert.rejects(f.call(f.agent, verb, { unknown: true }), { code: 'human-required' });
await assert.rejects(f.call(f.tester, verb, { unknown: true }), { code: 'human-required' });
await assert.rejects(f.call(f.reader, verb, { unknown: true }), { code: 'read-only' });
}
assert.equal(touched, 0);
const refusals = f.store.all("SELECT body FROM events WHERE kind='action.refused'").map(row => JSON.parse(row.body));
assert.equal(refusals.length, 6);
assert.equal(JSON.stringify(refusals).includes('unknown'), false);
const absent = world(t, null);
for (const verb of ['library.file', 'library.status'])
await assert.rejects(absent.call(absent.agent, verb, {}), { code: 'human-required' });
});
test('C2-04 bus: human reads validate keys and return adapter results without path leakage', async t => {
const seen = [];
const f = world(t, { readFile: async args => { seen.push(args); return { project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40), content: '# Made up\n' }; },
status: async args => { seen.push(args); return { project: 'books', served: null, problems: [] }; }, LibraryError: library.LibraryError });
assert.equal((await f.call(f.human, 'library.file', { project: 'books', path: 'Book One/A.md' })).content, '# Made up\n');
assert.equal((await f.call(f.human, 'library.status', { project: 'books' })).served, null);
assert.equal(seen[0].business.id, 'acme'); assert.equal(seen[1].project, 'books');
for (const [verb, args] of [['library.file', { project: 'books' }], ['library.file', { project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40) }], ['library.status', { project: 'books', extra: 1 }]])
await assert.rejects(f.call(f.human, verb, args), { code: 'invalid-request' });
});
test('C2-04 socket: each async library verb resolves its result rather than serializing {}', async t => {
const f = world(t, { readFile: async () => ({ project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40), content: '# Fixture\n' }),
status: async () => ({ project: 'books', served: null, problems: [] }), LibraryError: library.LibraryError });
const path = join(f.root, 'bus.sock');
const server = await serve({ broker: f.broker, path, authenticateHuman: () => f.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true }) });
t.after(() => server.close());
const human = new Client({ path, human: { fixture: true } });
assert.deepEqual(await human.call('library.status', { project: 'books' }), { project: 'books', served: null, problems: [] });
assert.equal((await human.call('library.file', { project: 'books', path: 'Book One/A.md' })).content, '# Fixture\n');
});
test('C2-04 runtime: protectedRoots skips library but retains real code-root credential protection', async t => {
const f = repo(t), dataRoot = join(f.root, 'data'), token = join(f.root, 'fixture-token');
mkdirSync(dataRoot, { mode: 0o700 }); writeFileSync(token, 'made-up-fixture-token-123456', { mode: 0o600 });
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry(), stack: { root: f.repo } },
arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] }, credentials: { gitea: { file: token, rotateBy: '2099-01-01' } } } } };
const runtime = await startBroker({ dataRoot, businesses: { acme: business } });
await runtime.close();
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
assert.equal(git(mirror, 'rev-parse', '--is-bare-repository'), 'true');
const bad = join(f.repo, 'credential'); writeFileSync(bad, 'made-up-fixture-token-123456', { mode: 0o600 });
business.roles.coder.credentials.gitea.file = bad;
await assert.rejects(startBroker({ dataRoot, businesses: { acme: business } }));
});
test('C2-04 runtime: unsafe existing mirror refuses boot and never publishes a socket', async t => {
const f = repo(t), dataRoot = join(f.root, 'data');
mkdirSync(dataRoot, { mode: 0o700 });
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
const mirror = join(dataRoot, 'projects/acme/books'); mkdirSync(mirror, { recursive: true });
writeFileSync(join(mirror, 'keep'), 'made-up note');
await assert.rejects(startBroker({ dataRoot, businesses: { acme: business } }), { code: 'library-git' });
assert.equal(existsSync(join(dataRoot, 'bus.sock')), false);
assert.equal(readFileSync(join(mirror, 'keep'), 'utf8'), 'made-up note');
});
test('C2-04 runtime: real library adapters reach the socket and lock waits leave other bus reads responsive', { timeout: 15000 }, async t => {
const f = repo(t), dataRoot = join(f.root, 'data');
mkdirSync(dataRoot, { mode: 0o700 });
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
const runtime = await startBroker({ dataRoot, businesses: { acme: business } });
t.after(() => runtime.close());
const cap = runtime.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
const path = join(f.root, 'human-fixture.sock');
const server = await serve({ broker: runtime.broker, path, authenticateHuman: () => runtime.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true }) }); t.after(() => server.close());
const human = new Client({ path, human: { fixture: true } });
assert.equal((await human.call('library.status', { project: 'books' })).served, null);
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify({ version: 1, branch: 'main', served: f.commit,
lastFetch: null, lastError: null, diverged: null, problems: [] }), { mode: 0o600 });
const file = await human.call('library.file', { project: 'books', path: 'Book One/Characters/Tomas.md' });
assert.equal(file.content, BASE['Book One/Characters/Tomas.md']);
assert.equal(file.commit, f.commit);
let release, done = false;
const hold = library.withLibraryLock(mirror, () => new Promise(r => { release = r; }));
const deadline = Date.now() + 5000;
while (!release && Date.now() < deadline) await new Promise(r => setTimeout(r, 5));
assert.equal(typeof release, 'function');
t.after(async () => { release(); await hold; });
const pending = human.call('library.status', { project: 'books' }).then(result => { done = true; return result; });
const ordinary = await Promise.race([human.call('agents'), new Promise((_, reject) => {
const timer = setTimeout(() => reject(Error('ordinary bus read blocked by lock acquisition')), 1000); timer.unref();
})]);
assert.ok(ordinary && typeof ordinary === 'object'); assert.equal(done, false);
release(); await hold;
assert.equal((await pending).served, f.commit);
});
+6
View File
@@ -10,6 +10,7 @@ import { checkVars } from "./vars.mjs";
import { parseCredentialRef } from "./credentials.mjs";
import { loadRole } from "./role.mjs";
import { checkPlugins } from "../../plugins/src/kinds.mjs";
import { checkMapping } from "../../library/src/mapping.mjs";
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE } from "./vocabulary.mjs";
import {
requireObject, rejectUnknownKeys, requireId, requirePositiveInt, requireString, requireDistinctList, readJsonFile, deepFreeze,
@@ -229,6 +230,10 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
for (const [id, entry] of Object.entries(document.projects)) {
requireId(id, `${file} project id`);
requireObject(entry, `${file} projects.${id}`);
if (entry.kind === 'library') {
projects[id] = checkMapping(entry, `${file} projects.${id}`, { fail: (code, message) => refuse(`${code}: ${message}`) });
continue;
}
rejectUnknownKeys(entry, ["root", "repo", "remote"], `${file} projects.${id}`);
if (typeof entry.root !== "string" || !isAbsolute(entry.root) || normalize(entry.root) !== entry.root || entry.root.includes("\0")) {
refuse(`${file} projects.${id}.root must be a normalized absolute path`);
@@ -245,6 +250,7 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
for (const [instance, role] of Object.entries(roles)) {
for (const project of role.publish ?? []) {
if (!Object.hasOwn(projects, project)) refuse(`${file} roles.${instance}.publish names ${project}, which the business file doesn't declare`);
if (projects[project].kind === 'library') refuse(`${file} roles.${instance}.publish names library project ${project}; library repositories cannot be published by agents`);
if (!projects[project].remote) refuse(`${file} roles.${instance}.publish names ${project}, which has no "remote"`);
const problem = forkProblem(projects[project].remote, role.identity);
if (problem) refuse(`${file} roles.${instance}.publish names ${project}: ${problem}`);
+3 -2
View File
@@ -58,7 +58,7 @@ function checkRefs(refs, forbiddenRoots) {
}
function forbiddenRoots(config, business) {
return [REPO, config.dataRoot, ...Object.values(business.projects).map((p) => p.root)];
return [REPO, config.dataRoot, ...Object.values(business.projects).filter(p => p.kind !== 'library').map((p) => p.root)];
}
function validate(args) {
@@ -69,6 +69,7 @@ function validate(args) {
const projects = {};
const warnings = [];
for (const [id, entry] of Object.entries(business.projects)) {
if (entry.kind === 'library') { projects[id] = 'library'; continue; }
const file = projectFilePath(entry.root);
if (!existsSync(file)) {
projects[id] = "absent";
@@ -115,7 +116,7 @@ function resolveCommand(args) {
if (project !== null) {
const entry = Object.hasOwn(business.projects, project) ? business.projects[project] : null;
if (!entry) fail(2, `business ${business.id} declares no project ${JSON.stringify(project)}`);
loaded = loadProject(entry.root);
if (entry.kind !== 'library') loaded = loadProject(entry.root);
}
const resolved = resolveInstance({ system: systemVars(config), business, project: loaded, instance });
const checked = checkRefs(Object.values(resolved.credentials), forbiddenRoots(config, business));
@@ -139,6 +139,7 @@ export function instanceSettings({ dir, rolesDir, system, levels = () => null })
}
checkRoutes({ proposed, old: before, levels });
for (const [project, entry] of Object.entries(proposed.projects)) {
if (entry.kind === 'library') continue;
if (!existsSync(projectFilePath(entry.root))) continue;
const loaded = loadProject(entry.root);
if (loaded.id !== project) throw new OrganizationError("organization-project-mismatch");
+7 -1
View File
@@ -185,12 +185,18 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) {
const oldBytes = regular(file);
const before = sha256(oldBytes);
if (revision !== before) fail("revision-conflict", "organization revision changed; read again");
validateBusinessDocument(document, file, { rolesDir });
const old = JSON.parse(oldBytes);
// C2-04 M9: every organization setter preserves the whole project
// mapping, including library playgrounds and hybrid universes.
if (canonicalJson(document?.projects) !== canonicalJson(old.projects))
fail('organization-projects-protected');
validateBusinessDocument(document, file, { rolesDir });
const bytes = JSON.stringify(document, null, 2) + "\n";
// Check and log exactly what lands on disk, for every setter: a value JSON
// can't carry (a Date, a Map) serializes as something else (row 131, sage-0314).
const written = JSON.parse(bytes);
if (canonicalJson(written.projects) !== canonicalJson(old.projects))
fail('organization-projects-protected');
validateBusinessDocument(written, file, { rolesDir });
const after = sha256(bytes);
const transaction = randomUUID();
+1 -1
View File
@@ -41,7 +41,7 @@ export function resolveInstance({ system, business, project = null, instance })
];
let projectId = null;
if (project) {
const declared = Object.entries(business.projects).find(([, p]) => projectFilePath(p.root) === project.file);
const declared = Object.entries(business.projects).find(([, p]) => p.kind !== 'library' && projectFilePath(p.root) === project.file);
if (!declared || declared[0] !== project.id) {
refuse(`project ${project.id} (${project.file}) isn't declared under that id in business ${business.id}`);
}
+30
View File
@@ -8,6 +8,36 @@ import { BusinessError, businessFilePath, configDir, loadBusiness, validateBusin
import { THREAD_IDLE_SECONDS } from "../src/business.mjs";
import { launchDoc, businessDoc, REPO, REPO_ROLES, rolesCopy, tmp, writeJson } from "./helpers.mjs";
test('C2-04: library mappings load without roots, remain frozen, and retain mapping refusals', async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const { rmSync } = await import('node:fs');
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
const doc = businessDoc(root), file = join(root, 'businesses/acme.json');
doc.projects = { books: mappingEntry(), ...doc.projects };
const loaded = validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES });
const { checkMapping } = await import('../../library/src/mapping.mjs');
assert.deepEqual(loaded.projects.books, checkMapping(mappingEntry()));
assert.equal(loaded.projects.books.root, undefined);
assert.ok(Object.isFrozen(loaded.projects.books.universes.harbor.books));
const only = structuredClone(doc); delete only.projects.stack;
assert.deepEqual(Object.keys(validateBusinessDocument(only, file, { rolesDir: REPO_ROLES }).projects), ['books']);
doc.projects.books.universes.harbor.playground = ['Elsewhere/Playground/'];
refuses(() => validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES }), /library-playground-outside-book/);
doc.projects.books = { ...mappingEntry(), root: '/forbidden' };
refuses(() => validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES }), /library-mapping-invalid/);
});
test('C2-04: an otherwise valid publish grant cannot name a library repo', async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const { rmSync } = await import('node:fs');
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
const doc = businessDoc(root);
doc.projects.books = { ...mappingEntry(), remote: 'https://git.example/writer/books' };
doc.roles.coder.identity = { name: 'Coder', email: '[email protected]', gitea: { url: 'https://git.example', user: 'acme-coder' } };
doc.roles.coder.publish = ['books'];
refuses(() => validateBusinessDocument(doc, join(root, 'businesses/acme.json'), { rolesDir: REPO_ROLES }), /publish.*library/);
});
function refuses(fn, pattern, exitCode = 2) {
assert.throws(fn, (error) => {
assert.ok(error instanceof BusinessError, `expected BusinessError, got ${error}`);
+18
View File
@@ -8,6 +8,24 @@ import { businessDoc, REPO, rolesCopy, systemConfig, tmp, tokenFile, writeJson }
const CLI = join(REPO, "packages", "business", "src", "cli.mjs");
test('C2-04: business validate and resolve skip a library before code projects and credential checks', async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const { rmSync } = await import('node:fs');
const s = setup(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; });
t.after(() => rmSync(s.root, { recursive: true, force: true }));
writeJson(projectFilePath(s.doc.projects.stack.root), { projectVersion: 1, id: 'stack', vars: { 'tracker.project': 3 } });
const checked = run(s, 'validate', 'acme');
assert.equal(checked.code, 0, checked.err);
assert.deepEqual(JSON.parse(checked.out).projects, { books: 'library', stack: 'valid' });
assert.doesNotMatch(checked.err, /project books: no project file/);
const library = run(s, 'resolve', 'acme', 'coder', '--project', 'books');
assert.equal(library.code, 0, library.err);
assert.equal(JSON.parse(library.out).project, null);
const code = run(s, 'resolve', 'acme', 'coder', '--project', 'stack');
assert.equal(code.code, 0, code.err);
assert.equal(JSON.parse(code.out).vars['tracker.project'], 3);
});
// A complete scratch setup: system config, business file, roles copy.
// HOME points into the scratch tree too, so nothing reads the real one.
function setup(mutate) {
@@ -19,6 +19,16 @@ after(() => roots.forEach(root => rmSync(root, { recursive: true, force: true })
const harnessOn = (doc, names = ["pi", "claude-code"]) => { doc.plugins = { ...(doc.plugins ?? {}), harness: names }; };
const LEVELS = { pi: ["off", "minimal", "low", "medium", "high", "xhigh", "max"], "claude-code": ["low", "medium", "high", "xhigh", "max"] };
test('C2-04: instance settings skip a library while checking a later code project', async () => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const f = fixture(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; });
writeJson(projectFilePath(f.doc.projects.stack.root), { projectVersion: 1, id: 'stack', vars: { 'tracker.project': 3 } });
const before = f.settings.read('acme');
f.settings.set('acme', { revision: before.revision, instance: 'gate-215', op: 'add', definition: 'merge-gate' }, 'jason');
assert.deepEqual(JSON.parse(readFileSync(f.file, 'utf8')).projects.books, mappingEntry());
assert.ok(f.settings.read('acme').instances['gate-215']);
});
function fixture(mutate, { levels } = {}) {
const root = tmp();
roots.push(root);
@@ -0,0 +1,68 @@
// C2-04 M9: use the writer directly, rather than relying on broker key filtering.
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync, rmSync } from 'node:fs';
import { join } from 'node:path';
import { organizationWriter } from '../src/organization.mjs';
import { businessDoc, rolesCopy, tmp, writeJson } from './helpers.mjs';
import { mappingEntry } from '../../library/tests/helpers.mjs';
function fixture(t) {
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
const doc = businessDoc(root);
const books = mappingEntry();
books.universes.hybrid = { kind: 'hybrid', name: 'An invented crossover',
sources: ['harbor', 'second'], books: ['Hybrid Book'], playground: [], constraints: [], notes: 'Fixture only' };
doc.projects = { books, ...doc.projects };
const dir = join(root, 'config'), file = writeJson(join(dir, 'businesses/acme.json'), doc);
const writer = organizationWriter({ dir, rolesDir: rolesCopy(root) });
// Give both snapshots an existing ordinary committed receipt to preserve.
doc.vars['tracker.pollSeconds'] = 61;
writer.write('acme', doc, { revision: writer.read('acme').revision, actor: 'jason' });
const log = writer.logFile('acme');
return { root, file, doc, writer, log };
}
function protectedWrite(f, change) {
const before = readFileSync(f.file), logBefore = readFileSync(f.log);
const next = structuredClone(f.writer.read('acme').document); change(next);
assert.throws(() => f.writer.write('acme', next, { revision: f.writer.read('acme').revision, actor: 'jason' }),
{ code: 'organization-projects-protected' });
assert.deepEqual(readFileSync(f.file), before);
assert.deepEqual(readFileSync(f.log), logBefore);
}
test('C2-04 M9: changed playground and hybrid entries leave business and changes.jsonl byte-identical', t => {
const f = fixture(t);
protectedWrite(f, next => { next.projects.books.universes.harbor.playground = ['Book One/New Playground/']; });
protectedWrite(f, next => { next.projects.books.universes.hybrid.notes = 'Changed hybrid rule'; });
});
test('C2-04 M9: code projects, additions, deletions and array changes are protected too', t => {
const f = fixture(t);
protectedWrite(f, next => { next.projects.stack.root = join(f.root, 'different-code'); });
protectedWrite(f, next => { next.projects.extra = { root: join(f.root, 'extra') }; });
protectedWrite(f, next => { delete next.projects.stack; });
protectedWrite(f, next => { next.projects.books.universes.hybrid.sources.reverse(); });
});
test('C2-04 M9: canonical key order is allowed and a non-project setter still records its change', t => {
const f = fixture(t), old = f.writer.read('acme');
const next = structuredClone(old.document);
next.projects = Object.fromEntries(Object.entries(next.projects).reverse());
next.vars['tracker.pollSeconds'] = 62;
f.writer.write('acme', next, { revision: old.revision, actor: 'jason' });
assert.equal(f.writer.read('acme').document.vars['tracker.pollSeconds'], 62);
const receipt = JSON.parse(readFileSync(f.log, 'utf8').trim().split('\n').at(-1));
assert.equal(receipt.state, 'committed');
assert.deepEqual(receipt.changes.map(c => c.path), ['vars.tracker.pollSeconds']);
});
test('C2-04 M9: a serialization hook cannot change projects after the canonical comparison', t => {
const f = fixture(t);
protectedWrite(f, next => {
const changed = structuredClone(next.projects);
changed.books.universes.hybrid.notes = 'Serialized change';
Object.defineProperty(next.projects, 'toJSON', { value: () => changed });
});
});
+10
View File
@@ -28,6 +28,16 @@ function setup(mutate, projectDoc) {
const resolve = (s, instance) => resolveInstance({ system: s.system, business: s.business, project: s.project, instance });
test('C2-04: a library first in projects does not hide a later project layer', async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const { rmSync } = await import('node:fs');
const s = setup(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; }, { vars: { 'tracker.project': 7 } });
t.after(() => rmSync(s.root, { recursive: true, force: true }));
const result = resolve(s, 'coder');
assert.equal(result.project, 'stack');
assert.equal(result.vars['tracker.project'], 7);
});
test("systemVars flattens the validated config", () => {
const vars = systemVars({ environment: "development", dataRoot: "/d", execution: { backend: "docker", provider: "zai", model: "m", adapter: "pi" } });
assert.deepEqual({ ...vars }, {
+1
View File
@@ -59,6 +59,7 @@ function trackerFor(system, b, warn) {
if (base["tracker.baseUrl"] === undefined) return null;
const named = [];
for (const [id, entry] of Object.entries(b.projects)) {
if (entry.kind === 'library') continue;
if (!existsSync(projectFilePath(entry.root))) continue;
const project = loadProject(entry.root);
if (project.id !== id) throw new CliError(`project file ${project.file} has id ${project.id}, but business ${b.id} declares it as ${id}`, 3);
+1 -1
View File
@@ -351,7 +351,7 @@ export function createLauncher({
mkdirSync(workspace, { recursive: true, mode: 0o700 });
let checkouts;
try {
checkouts = await ensureCheckouts({ workspace, projects: b.projects, env });
checkouts = await ensureCheckouts({ workspace, projects: Object.fromEntries(Object.entries(b.projects).filter(([, entry]) => entry.kind !== 'library')), env });
} catch (e) {
if (e instanceof CheckoutError) throw new Refusal("checkout-failed", `${e.project}: ${e.message}`);
throw e;
+10
View File
@@ -29,6 +29,16 @@ const FAKE = fileURLToPath(new URL("../../harness/tests/fixtures/fake-adapter.mj
const HOST = fileURLToPath(new URL("./fixtures/launch-host.mjs", import.meta.url));
const skip = sandboxProblem() ?? false;
test('C2-04: launcher workspace clones code but never the library', { skip, timeout: 30000 }, async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const f = await setup(t, doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; return doc; });
await f.launcher.launchPm();
const workspace = workspaceDir(f.f.dataRoot, 'acme', 'chief');
assert.equal(existsSync(join(workspace, 'stack', '.git')), true);
assert.equal(existsSync(join(workspace, 'books')), false);
assert.equal(await f.close(), 0);
});
// chief opus, coder and reviewer sonnet, one of each family; cto is listed but
// runs the system model, which is in no family. The chief of staff is the launcher.
function edit(doc) {
+1 -1
View File
@@ -35,7 +35,7 @@ export function setup(t, realPin=false){
test('shared copy: npm ci, pushed reachability, readonly/existing copy, webui delegation',t=>{
const s=setup(t),r=s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]);assert.equal(r.status,0,r.stderr);
const copy=join(s.copies,s.id);assert.equal(statSync(copy).mode&0o777,0o555);
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/library/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
const npm=join(s.bin,'npm');writeFileSync(npm,'#!/bin/sh\nexit 99\n');chmodSync(npm,0o755);
const existing=s.run('webui-service.sh',['copy',s.id,'--copies',s.copies]);assert.equal(existing.status,0);assert.equal(existing.stdout,`exists: ${copy}\n`);unlinkSync(npm);
git(s.repo,'update-ref','refs/remotes/origin/refactor',git(s.repo,'hash-object','-w','package.json'));
+1 -1
View File
@@ -306,7 +306,7 @@ for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeTe
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
}
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
for(const p of ['packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
execFileSync('chmod',['-R','a-w',d]);
}
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
+1 -1
View File
@@ -1769,7 +1769,7 @@ export class Store {
const RCOPIES=join(base,'rcopies');mkdirSync(RCOPIES);
for(const [id,v,froms] of [[B,'3g',['3f']],[C,'3h',['3f','3g']],[A,'3g',['3f']],[C1,'3g',['3f']]]){
const d=join(RCOPIES,id);
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/bus/src/business.mjs','packages/plugins/src/kinds.mjs','packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/bus/src/business.mjs','packages/plugins/src/kinds.mjs','packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
if(id===C1)writeFileSync(join(d,'scripts','mosaic'),`#!/usr/bin/env bash\nexec node "${join(RCOPIES,B,'packages','cli','src','cli.mjs')}" "$@"\n`,{mode:0o755});
for(const [f,text] of [['packages/cli/src/cli.mjs',id===A?'// fixture host from before row 163\nprocess.exit(1);\n':script(rHost)],['packages/webui/src/cli.mjs',script(rConsole)],['packages/bus/src/store.mjs',rStore(v,froms)],['packages/bus/src/migrate-schema.mjs',script(rMigrate)],['packages/bus/src/broker.mjs','export class BusError extends Error { constructor(code) { super(code); this.code = code; } }\n']]){mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text)}
execFileSync('chmod',['-R','a-w',d]);
+14
View File
@@ -12,7 +12,20 @@ import { FakeVikunja } from "../../tasks/src/fake.mjs";
import { SCOPES } from "../../tasks/tests/world.mjs";
import { fixture, tmp } from "./helpers.mjs";
test('C2-04: a library-only business with tracker.baseUrl boots without a project tracker', async t => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const f = fixture(tmp(t), 'acme', doc => { doc.projects = { books: mappingEntry() }; return doc; });
const warnings = [];
const boot = bootConfig({ system: loadSystem({ env: f.env }), businessId: 'acme', env: f.env, warn: text => warnings.push(text) });
assert.equal(boot.trackers, undefined);
assert.ok(warnings.some(text => text.includes('no project file sets tracker.project')));
const host = await startHost({ boot, business: 'acme', log: () => {} });
t.after(() => host.close(0));
assert.equal(await host.close(0), 0);
});
test("bootConfig trackers reach the S3 adapter in the real broker child, which goes ready against a fake Vikunja", async (t) => {
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
const root = tmp(t);
const fake = new FakeVikunja();
const srv = await fake.listen();
@@ -26,6 +39,7 @@ test("bootConfig trackers reach the S3 adapter in the real broker child, which g
fake.install(project);
const f = fixture(root, "acme", (doc) => {
doc.projects = { books: mappingEntry(), ...doc.projects };
doc.vars["tracker.baseUrl"] = srv.url;
doc.tracker.sync.botId = bots.sync;
// The chief of staff (row 119) has no tracker bot.
+128
View File
@@ -0,0 +1,128 @@
# packages/library
Reads a writing repo by a mapping of folders to books, types, canon and playground. This is Case 2
row C2-01 (`docs/plans/2026-10-20_case2-rows.md`), built from design sections 1.1 to 1.3 and 7.4 in
`agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`.
It is pure code with no state. Business file loading (C2-04), sync (C2-03a), the index (C2-05a) and the
guard (C2-06) build on it.
## Mapping
`checkMapping(entry, where, { fail })` checks a `kind: "library"` project entry and returns a
deep-frozen copy. `where` names the entry in messages. `fail(code, message)` defaults to throwing a
`LibraryError`. C2-04 passes its own so a business file refusal carries the file name.
```json
{
"kind": "library",
"remote": "https://git.example.invalid/her/books.git",
"branch": "main",
"vaultPrefix": "books",
"types": { "Characters": "character", "Places": "place", "Research": "research" },
"manuscript": ["Chapters"],
"universes": {
"harbor": { "name": "Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"] }
}
}
```
- `remote` is https with no credentials, query or fragment. `branch` is a plain ref name.
- `vaultPrefix` is optional, one folder name.
- `types` maps a folder name to `character`, `place`, `research`, `tracker` or `summary`.
`manuscript` lists folder names and shares none with `types`.
- A universe has `name`, `books` and `playground`. A hybrid adds `kind: "hybrid"`, two or more
non-hybrid `sources`, `constraints` (each one `{ "exclude": ... }` or
`{ "pin": { "entity", "field", "value" } }`) and `notes`.
Refusals:
| Code | When |
|---|---|
| `library-playground-outside-book` | a playground prefix is not strictly inside a book of its own universe |
| `library-mapping-invalid` | any other shape fault: unknown key, bad remote or branch, a book folder in two universes, a book folder that holds another, a playground prefix that holds another, an unknown hybrid source |
## Scope
`classify(mapping, path)` decides scope from the path alone. Only paths ending in lowercase `.md` are
mapped. A path under a playground prefix is `playground`. Any other path under a book folder is
`canon`. Everything else is `unmapped`. The type is the first folder after the book, or after the
playground prefix: a `types` name, `manuscript`, or `null`.
## Reader
`readLibrary({ repo, commit, mapping })` reads one commit, never a working tree. `repo` may be a
clone or a bare mirror. `commit` must be a full 40-hex id that names a commit, else `library-commit`.
Git runs through `git()` and `gitBytes()` from `packages/project/src/git.mjs`, so the same safe config,
environment and ceiling directory apply. A failed git call refuses `library-git`.
It returns `{ commit, files, problems }`, deep-frozen and sorted. Each file has its path, scope,
universe, book, type, numbered lines, `bodyLine`, parsed `frontmatter` or `null`, `frontmatterRaw`
and `links`. Front matter `appears_in` and `relationships` wikilinks count as links.
Problems carry `{ code, path, line, message }`:
| Rule | Meaning | File indexed |
|---|---|---|
| R1 | not a regular `100644` blob of at most 1 MiB, or not UTF-8, or holds a NUL | no |
| R2 | path is unmapped | no |
| R3 | front matter not fully parsed, `line` is where parsing stopped | yes, front matter kept raw |
| R4 | link resolves to no file or to more than one | yes, link dropped |
| R5 | canon file links into the playground | yes |
| R6 | canon file links into another universe, a hybrid's sources excepted | yes |
| R7 | two paths differ only in case | no, neither |
The front matter parser takes scalars, quoted strings, flow lists, block lists and wikilinks. Nested
maps, anchors, block scalars and duplicate keys stop it with R3.
## Links
`resolver(paths, { vaultPrefix })` tries four steps in order, each with the target as written and
then with `.md` added:
1. `root`: the path from the repo root.
2. `vault-prefix`: the same path with `vaultPrefix/` stripped.
3. `relative`: relative to the linking file, never above the repo root.
4. `shortest`: a target with no `/` that matches the base name of exactly one indexed file.
It resolves only against indexed files. A link to a `.canvas` file or an attachment is R4. A link into
another repo resolves to nothing. `extractLinks` skips fenced and inline code.
## renderItems
`renderItems(body, { scopeOf, changed })` turns a reply body into the plain text both channels show.
It is pure. Item kinds are `fact`, `finding`, `correction` (with `from` and `to`), `question` and
`idea`. Each cite is a path with commit, line range and quote, or a url with a quote.
```
1. Correction: Chapter 1 says grey.
From: "green eyes"
To: "grey eyes"
[canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes" (changed since)
```
A body of `{ "items": [], "notice": ... }` gives `No cited answer`, `No answer` or `The turn failed`.
Any other shape gives the fixed line `This reply could not be shown.` and none of the body's text.
## Survey
```sh
mosaic library survey <repo> [--mapping <file>]
```
It reads the repo's HEAD commit and prints folder names, file counts by extension, front matter key
sets and link-form counts. It never prints file text, front matter values, file names or link targets.
A key that is not a short identifier prints as `(other key)`, since it could be her words. With
`--mapping` it adds scope counts, resolver step counts and problem counts by rule.
Exit codes follow `packages/project`: 0 done, 2 the repo can't be read (`library-git`,
`library-commit`), 3 a bad mapping, 4 usage. `scripts/mosaic` sends `library survey` here. Other
library verbs will be bus verbs.
## Tests
```sh
node --test packages/library/tests/
```
Fixtures are made-up repos built per test in a temp dir. A marker string stands in for her text, and
the survey tests check that it never appears in output.
+11
View File
@@ -0,0 +1,11 @@
{
"name": "@mosaic/library",
"version": "0.1.0",
"private": true,
"type": "module",
"license": "UNLICENSED",
"exports": "./src/index.mjs",
"scripts": {
"test": "node --test tests/*.test.mjs"
}
}
+9
View File
@@ -0,0 +1,9 @@
// packages/library: her writing repo read by a mapping (Case 2, design 1 and 7.4). See README.md.
export { LibraryError, RULES, problem, sortProblems } from './problems.mjs';
export { checkMapping, TYPES } from './mapping.mjs';
export { classify } from './scope.mjs';
export { extractLinks, parseWikilink, resolver, formOf } from './links.mjs';
export { readLibrary, listTree, readTexts, parseFile, parseFrontMatter, headCommit, MAX_BLOB, LINK_FIELDS } from './reader.mjs';
export { renderItems, KINDS, NOTICES, UNREADABLE } from './render.mjs';
export { survey, formatSurvey, surveyMain } from './survey.mjs';
export { SYNC_STATE_FILE, mirrorPath, ensureMirror, withLibraryLock, servedCommit, readSyncState, libraryStatus, readLibraryFile } from './mirror.mjs';
+79
View File
@@ -0,0 +1,79 @@
// Wikilinks and the link resolver of design 1.3. Links are [[target]], [[target|label]] and
// [[target#heading]], with ![[...]] embeds read the same way. Links inside fenced code blocks and
// inline code spans are not links, as in Obsidian.
import { posix } from 'node:path';
const WIKILINK = /(!?)\[\[([^\[\]\n]+?)\]\]/g;
const MARKDOWN_LINK = /(?<!!)\[[^\[\]\n]*\]\(([^()\s]+)\)/g;
const FENCE = /^\s{0,3}(`{3,}|~{3,})/;
// One wikilink's parts, or null for text that isn't one.
export function parseWikilink(text) {
const match = /^(!?)\[\[([^\[\]\n]+?)\]\]$/.exec(text);
return match ? parts(match[2], match[1] === '!') : null;
}
function parts(inner, embed) {
const bar = inner.indexOf('|');
const target = bar < 0 ? inner : inner.slice(0, bar);
const hash = target.indexOf('#');
return {
file: (hash < 0 ? target : target.slice(0, hash)).trim(),
heading: hash < 0 ? null : target.slice(hash + 1).trim(),
label: bar < 0 ? null : inner.slice(bar + 1),
embed,
};
}
// The form a link is counted under in the survey.
export const formOf = link => link.markdown ? 'markdown' : link.embed ? 'embed' : !link.file ? 'self'
: link.label !== null ? 'label' : link.heading !== null ? 'heading' : 'plain';
// Links in body lines. `lines` is [[number, text], ...]. Markdown links are returned only with
// `markdown: true`, for the survey's counts. The resolver follows wikilinks only.
export function extractLinks(lines, { markdown = false } = {}) {
const found = [];
let fence = null;
for (const [line, text] of lines) {
const open = FENCE.exec(text);
if (fence) { if (open && open[1][0] === fence[0] && open[1].length >= fence.length) fence = null; continue; }
if (open) { fence = open[1]; continue; }
const plain = text.replace(/(`+)[^`]*?\1/g, match => ' '.repeat(match.length));
for (const m of plain.matchAll(WIKILINK)) found.push({ ...parts(m[2], m[1] === '!'), line });
if (markdown) {
for (const m of plain.matchAll(MARKDOWN_LINK)) {
if (!/^[a-z][a-z0-9+.-]*:/i.test(m[1]) && !m[1].startsWith('#'))
found.push({ file: m[1], heading: null, label: null, embed: false, markdown: true, line });
}
}
}
return found;
}
// A resolver over the indexed paths. resolve(from, file) tries, in order: a path from the repo
// root, the same path with `vaultPrefix/` stripped, a path relative to the linking file, and
// Obsidian's shortest-path form, a base name that matches exactly one indexed file. Each step tries
// the target as written and then with `.md` added. A target in another of her repos is not in the
// index, so it resolves to nothing.
export function resolver(paths, { vaultPrefix } = {}) {
const known = new Set(paths), byBase = new Map();
for (const path of known) {
const base = posix.basename(path);
for (const key of new Set([base, base.replace(/\.md$/, '')])) byBase.set(key, [...(byBase.get(key) ?? []), path]);
}
const hit = candidate => [candidate, `${candidate}.md`].find(c => known.has(c));
return function resolve(from, file) {
if (typeof file !== 'string' || !file) return { path: null, step: 'missing' };
let path = hit(file);
if (path) return { path, step: 'root' };
if (vaultPrefix && file.startsWith(`${vaultPrefix}/`) && (path = hit(file.slice(vaultPrefix.length + 1))))
return { path, step: 'vault-prefix' };
const relative = posix.normalize(posix.join(posix.dirname(from), file));
if (relative !== '..' && !relative.startsWith('../') && !relative.startsWith('/') && (path = hit(relative)))
return { path, step: 'relative' };
if (!file.includes('/')) {
const matches = byBase.get(file) ?? [];
if (matches.length === 1) return { path: matches[0], step: 'shortest' };
if (matches.length > 1) return { path: null, step: 'ambiguous', count: matches.length };
}
return { path: null, step: 'missing' };
};
}
+132
View File
@@ -0,0 +1,132 @@
// The library project entry of a business file (design 1.1): her folders mapped onto universes,
// books, type folders, manuscript folders and playground prefixes. A pure validator. Business file
// loading wires it in C2-04, which passes its own `fail` so the refusal reads as a business file
// problem. Agents can't write this entry (M9, C2-04).
import { raise } from './problems.mjs';
export const TYPES = Object.freeze(['character', 'place', 'research', 'tracker', 'summary']);
const ID = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
const BRANCH = /^[A-Za-z0-9][A-Za-z0-9._/-]{0,127}$/;
const CONSTRAINT_KEYS = ['exclude', 'pin'];
// A folder segment from her vault: spaces and capitals are fine, separators, control characters
// and dot names are not.
function segment(value) {
return typeof value === 'string' && value.length > 0 && value.length <= 255 && value !== '.' && value !== '..'
&& !/[\/\\\u0000-\u001f\u007f]/.test(value) && value.trim() === value;
}
function folder(value) {
return typeof value === 'string' && value.length <= 1024 && value.split('/').every(segment);
}
const holds = (outer, inner) => inner.startsWith(outer.endsWith('/') ? outer : `${outer}/`);
export function checkMapping(entry, where = 'library', { fail = raise } = {}) {
const bad = message => fail('library-mapping-invalid', `${where} ${message}`);
const object = (value, name) => {
if (value === null || typeof value !== 'object' || Array.isArray(value)) bad(`${name} must be an object`);
};
const only = (value, keys, name) => {
for (const key of Object.keys(value)) if (!keys.includes(key)) bad(`${name} has unknown key ${JSON.stringify(key)}`);
};
const list = (value, name, check, what) => {
if (!Array.isArray(value)) bad(`${name} must be a list`);
for (const item of value) if (!check(item)) bad(`${name} holds ${JSON.stringify(item)}, which is not ${what}`);
if (new Set(value).size !== value.length) bad(`${name} names an entry twice`);
return [...value];
};
object(entry, '');
only(entry, ['kind', 'remote', 'branch', 'vaultPrefix', 'universes', 'types', 'manuscript'], 'entry');
if (entry.kind !== 'library') bad('kind must be "library"');
let url;
try { url = new URL(entry.remote); } catch { bad('remote must be an https:// URL'); }
if (url.protocol !== 'https:' || url.username || url.password || url.search || url.hash || entry.remote.includes('\0'))
bad('remote must be an https:// URL with no credentials, query or fragment');
if (typeof entry.branch !== 'string' || !BRANCH.test(entry.branch) || entry.branch.includes('..')
|| entry.branch.includes('//') || entry.branch.endsWith('/') || entry.branch.endsWith('.lock'))
bad('branch must be a plain branch name');
if (Object.hasOwn(entry, 'vaultPrefix') && !segment(entry.vaultPrefix)) bad('vaultPrefix must be one folder name');
object(entry.types, 'types');
for (const [name, type] of Object.entries(entry.types)) {
if (!segment(name)) bad(`types has ${JSON.stringify(name)}, which is not one folder name`);
if (!TYPES.includes(type)) bad(`types.${name} must be one of ${TYPES.join(', ')}`);
}
const manuscript = list(entry.manuscript, 'manuscript', segment, 'one folder name');
for (const name of manuscript) if (Object.hasOwn(entry.types, name)) bad(`manuscript folder ${JSON.stringify(name)} is also a type folder`);
object(entry.universes, 'universes');
if (!Object.keys(entry.universes).length) bad('universes must name at least one universe');
const universes = {}, bookOwner = new Map(), prefixes = [];
for (const [id, universe] of Object.entries(entry.universes)) {
const name = `universes.${id}`;
if (!ID.test(id)) bad(`universe id ${JSON.stringify(id)} is not a plain id`);
object(universe, name);
const hybrid = universe.kind === 'hybrid';
only(universe, hybrid ? ['kind', 'name', 'sources', 'books', 'playground', 'constraints', 'notes'] : ['name', 'books', 'playground'], name);
if (typeof universe.name !== 'string' || !universe.name.trim() || universe.name.length > 128 || /[\u0000-\u001f]/.test(universe.name))
bad(`${name}.name must be a short display name`);
const books = list(universe.books, `${name}.books`, folder, 'a folder path');
if (!hybrid && !books.length) bad(`${name}.books must name at least one book folder`);
for (const book of books) {
if (bookOwner.has(book)) bad(`book folder ${JSON.stringify(book)} is in two universes`);
bookOwner.set(book, id);
}
const playground = list(universe.playground ?? [], `${name}.playground`,
value => typeof value === 'string' && value.endsWith('/') && folder(value.slice(0, -1)), 'a folder path ending in /');
for (const prefix of playground) {
// Every playground file must have a universe and a book (1.1, B1).
if (!books.some(book => holds(book, prefix) && prefix !== `${book}/`))
fail('library-playground-outside-book', `${where} ${name}.playground ${JSON.stringify(prefix)} is not inside one of its books`);
prefixes.push(prefix);
}
const out = { name: universe.name, books, playground };
if (hybrid) {
out.kind = 'hybrid';
out.sources = list(universe.sources, `${name}.sources`, value => typeof value === 'string' && ID.test(value), 'a universe id');
if (out.sources.length < 2) bad(`${name}.sources must name two or more universes`);
if (!Array.isArray(universe.constraints ?? [])) bad(`${name}.constraints must be a list`);
out.constraints = (universe.constraints ?? []).map((constraint, i) => {
const at = `${name}.constraints[${i}]`;
object(constraint, at);
only(constraint, CONSTRAINT_KEYS, at);
if (Object.keys(constraint).length !== 1) bad(`${at} must hold exactly one of ${CONSTRAINT_KEYS.join(', ')}`);
if (Object.hasOwn(constraint, 'exclude')) {
if (!folder(constraint.exclude)) bad(`${at}.exclude must be a source path`);
return { exclude: constraint.exclude };
}
object(constraint.pin, `${at}.pin`);
only(constraint.pin, ['entity', 'field', 'value'], `${at}.pin`);
if (!folder(constraint.pin.entity) || typeof constraint.pin.field !== 'string' || !constraint.pin.field
|| !['string', 'number', 'boolean'].includes(typeof constraint.pin.value))
bad(`${at}.pin needs an entity path, a field name and a scalar value`);
return { pin: { ...constraint.pin } };
});
if (Object.hasOwn(universe, 'notes') && (typeof universe.notes !== 'string' || universe.notes.length > 4000))
bad(`${name}.notes must be text of at most 4000 characters`);
if (Object.hasOwn(universe, 'notes')) out.notes = universe.notes;
}
universes[id] = out;
}
for (const [id, universe] of Object.entries(universes)) {
for (const source of universe.sources ?? []) {
if (!Object.hasOwn(universes, source) || universes[source].kind === 'hybrid')
bad(`universes.${id}.sources names ${JSON.stringify(source)}, which is not a non-hybrid universe`);
}
}
const books = [...bookOwner.keys()];
for (const a of books) for (const b of books) if (a !== b && holds(a, b)) bad(`book folder ${JSON.stringify(a)} holds book folder ${JSON.stringify(b)}`);
for (const a of prefixes) for (const b of prefixes) if (a !== b && holds(a, b)) bad(`playground ${JSON.stringify(a)} holds playground ${JSON.stringify(b)}`);
if (new Set(prefixes).size !== prefixes.length) bad('a playground prefix is listed twice');
return deepFreeze({
kind: 'library', remote: entry.remote, branch: entry.branch,
...(Object.hasOwn(entry, 'vaultPrefix') ? { vaultPrefix: entry.vaultPrefix } : {}),
universes, types: { ...entry.types }, manuscript,
});
}
function deepFreeze(value) {
if (value && typeof value === 'object') { for (const item of Object.values(value)) deepFreeze(item); Object.freeze(value); }
return value;
}
+257
View File
@@ -0,0 +1,257 @@
// C2-04: host-only bare library storage. No remote, fetch, credential or
// working tree. The broker receives these functions through the runtime.
import { constants, closeSync, existsSync, fstatSync, lstatSync, mkdirSync, mkdtempSync,
openSync, readSync, readdirSync, realpathSync, rmSync } from 'node:fs';
import { dirname, isAbsolute, join, normalize } from 'node:path';
import { execFile, spawn } from 'node:child_process';
import { projectRepo, noLinks } from '../../project/src/paths.mjs';
import { git } from '../../project/src/git.mjs';
import { checkMapping } from './mapping.mjs';
import { classify } from './scope.mjs';
import { LibraryError, RULES } from './problems.mjs';
import { listTree, readLibrary, readTexts } from './reader.mjs';
export const SYNC_STATE_FILE = 'mosaic-library-sync.json';
const LOCK_FILE = 'mosaic-library-sync.lock';
const STATE_MAX = 8 * 1024 * 1024;
const SHA = /^[a-f0-9]{40}$/;
const fail = (code, message = code) => { throw new LibraryError(code, message); };
const object = value => value !== null && typeof value === 'object' && !Array.isArray(value);
const keys = (value, expected) => object(value) && Object.keys(value).length === expected.length
&& Object.keys(value).every(key => expected.includes(key));
const optional = (value, check) => value === null || check(value);
const utc = value => {
if (typeof value !== 'string') return false;
try { return new Date(value).toISOString() === value; } catch { return false; }
};
const empty = branch => ({ version: 1, branch, served: null, lastFetch: null, lastError: null, diverged: null, problems: [] });
function contextEntry(context) {
const { business, project } = context;
if (!Object.hasOwn(business?.projects ?? {}, project)) fail('project-not-declared');
const entry = business.projects[project];
if (entry?.kind !== 'library') fail('library-mapping-invalid', 'expected a library project');
return checkMapping(entry);
}
export function mirrorPath(context) {
contextEntry(context);
try { return projectRepo(context, { allowLibrary: true }); }
catch (error) { throw new LibraryError(error.code ?? 'library-git', 'unsafe mirror path'); }
}
function ownedRoot(mirror) {
try {
if (typeof mirror !== 'string' || !isAbsolute(mirror) || normalize(mirror) !== mirror) throw Error();
noLinks(mirror);
const st = lstatSync(mirror);
if (!st.isDirectory() || st.uid !== process.getuid() || (st.mode & 0o077) || realpathSync(mirror) !== mirror) throw Error();
} catch { fail('library-git', 'expected a private owned bare mirror'); }
}
function privateFile(st) {
return st.isFile() && st.uid === process.getuid() && st.nlink === 1 && (st.mode & 0o077) === 0;
}
async function inspectMirrorUnchecked(mirror) {
ownedRoot(mirror);
const normal = ['HEAD', 'config', 'description', 'hooks', 'info', 'objects', 'refs', 'logs', 'packed-refs', 'shallow', 'FETCH_HEAD', 'ORIG_HEAD'];
const allowed = new Set([...normal, ...normal.filter(name => !['hooks', 'info', 'objects', 'refs', 'logs', 'description'].includes(name)).map(name => name + '.lock'), SYNC_STATE_FILE, LOCK_FILE]);
const top = readdirSync(mirror);
if (top.some(name => !allowed.has(name)) || ['HEAD', 'config', 'objects', 'refs'].some(name => !top.includes(name)))
fail('library-git', 'unexpected bare mirror layout');
for (const path of ['commondir', 'objects/info/alternates', 'objects/info/http-alternates'])
if (existsSync(join(mirror, path))) fail('library-git', 'Git metadata indirection refused');
const walk = (folder, topLevel = false) => {
for (const name of readdirSync(folder)) {
const file = join(folder, name); let st;
try { st = lstatSync(file); }
catch (error) { if (error.code === 'ENOENT') continue; throw error; }
const state = topLevel && [SYNC_STATE_FILE, LOCK_FILE].includes(name);
if (state) { if (!privateFile(st)) fail('library-state-invalid', 'unsafe state or lock metadata'); }
else if (st.isDirectory() && !st.isSymbolicLink()) walk(file);
else if (!st.isFile() || st.nlink !== 1 || st.uid !== process.getuid()) fail('library-git', 'unsafe Git metadata');
}
};
walk(mirror, true);
if (!top.includes(LOCK_FILE)) fail('library-state-invalid', 'mirror lock file missing');
if (lstatSync(join(mirror, 'config')).size > 65536) fail('library-git', 'Git config too large');
let config;
try { config = await git(mirror, ['config', '--no-includes', '--file', join(mirror, 'config'), '--list', '--null']); }
catch { fail('library-git', 'cannot inspect mirror config'); }
const values = { 'core.repositoryformatversion': ['0'], 'core.filemode': ['true', 'false'], 'core.bare': ['true'],
'core.logallrefupdates': ['true', 'false'], 'core.ignorecase': ['true', 'false'] };
let bare = false;
for (const item of config.split('\0').filter(Boolean)) {
const at = item.indexOf('\n'), key = item.slice(0, at), value = item.slice(at + 1);
if (key === 'core.bare' && value === 'true') bare = true;
if (at < 0 || !values[key]?.includes(value)) fail('library-git', 'unsupported bare mirror config');
}
if (!bare) fail('library-git', 'expected a bare repository');
}
async function inspectMirror(mirror) {
try { await inspectMirrorUnchecked(mirror); }
catch (error) {
if (error instanceof LibraryError) throw error;
fail('library-git', 'unreadable bare mirror metadata');
}
}
export async function ensureMirror(context) {
const mirror = mirrorPath(context);
if (existsSync(mirror)) { await inspectMirror(mirror); return mirror; }
const parent = dirname(mirror); mkdirSync(parent, { recursive: true, mode: 0o700 }); noLinks(parent);
const staged = mkdtempSync(join(parent, '.library-init-'));
const original = lstatSync(staged);
const isOwnStage = st => st.isDirectory() && st.uid === process.getuid() && st.dev === original.dev && st.ino === original.ino;
try {
await git(staged, ['init', '--bare', '--initial-branch=served', '--template=']);
const fd = openSync(join(staged, LOCK_FILE), constants.O_CREAT | constants.O_EXCL | constants.O_WRONLY | constants.O_NOFOLLOW, 0o600);
closeSync(fd);
await inspectMirror(staged);
// Coreutils versions disagree on the status of a skipped no-clobber
// rename. The original inode, rather than that status, decides who won.
const moveError = await new Promise(resolve => execFile('/usr/bin/mv', ['--no-clobber', '--no-target-directory', staged, mirror],
{ timeout: 5000, maxBuffer: 65536, env: { PATH: '/usr/bin:/bin', LC_ALL: 'C' } }, error => resolve(error)));
let remaining;
try { remaining = lstatSync(staged); } catch (error) { if (error.code !== 'ENOENT') throw error; }
if (remaining) {
if (!isOwnStage(remaining)) fail('library-git', 'staging directory was replaced');
// A skipped rename is valid only when another intact mirror won.
// Signals, timeouts and spawn failures are always genuine errors.
if (moveError && (typeof moveError.code !== 'number' || moveError.killed || moveError.signal)) throw moveError;
} else {
if (!isOwnStage(lstatSync(mirror))) fail('library-git', 'mirror publication identity mismatch');
if (moveError) throw moveError;
}
await inspectMirror(mirror);
return mirror;
} catch (error) {
if (error instanceof LibraryError) throw error;
fail('library-git', 'bare mirror initialization refused');
} finally {
let remaining;
try { remaining = lstatSync(staged); } catch (error) { if (error.code !== 'ENOENT') throw error; }
if (remaining) {
if (!isOwnStage(remaining)) fail('library-git', 'refusing to remove replaced staging directory');
rmSync(staged, { recursive: true });
}
}
}
// Each acquisition opens a distinct description. flock(1) inherits it as
// fd 3 and exits after acquisition; this parent's fd retains the lock.
export async function withLibraryLock(mirror, operation) {
ownedRoot(mirror);
let fd;
try {
try {
fd = openSync(join(mirror, LOCK_FILE), constants.O_RDWR | constants.O_NOFOLLOW | constants.O_NONBLOCK);
if (!privateFile(fstatSync(fd))) fail('library-state-invalid', 'unsafe library lock');
} catch { fail('library-state-invalid', 'library lock missing or unsafe'); }
await new Promise((resolve, reject) => {
const child = spawn('/usr/bin/flock', ['--exclusive', '--wait', '5', '--conflict-exit-code', '75', '3'],
{ env: { PATH: '/usr/bin:/bin', LC_ALL: 'C' }, stdio: ['ignore', 'ignore', 'ignore', fd] });
child.once('error', () => reject(new LibraryError('library-state-invalid', 'host requires /usr/bin/flock')));
child.once('close', code => code === 0 ? resolve() : reject(new LibraryError(code === 75 ? 'library-sync-busy' : 'library-state-invalid', 'library lock acquisition refused')));
});
try {
const heldStat = fstatSync(fd), pathStat = lstatSync(join(mirror, LOCK_FILE));
if (!privateFile(pathStat) || pathStat.ino !== heldStat.ino || pathStat.dev !== heldStat.dev) throw Error();
} catch { fail('library-state-invalid', 'library lock was replaced'); }
let held = true;
const pending = new Set();
const readSyncState = options => {
if (!held) return Promise.reject(new LibraryError('library-state-invalid', 'library lock scope expired'));
const read = readStateHeld(mirror, options).finally(() => pending.delete(read));
pending.add(read); read.catch(() => {});
return read;
};
try { return await operation({ readSyncState }); }
finally {
held = false;
// An accidentally unawaited scoped read must finish before this fd
// releases the lock. No new read can start after the callback ends.
await Promise.allSettled([...pending]);
}
} finally { if (fd !== undefined) closeSync(fd); }
}
export async function servedCommit(mirror) {
await inspectMirror(mirror);
try {
const refs = await git(mirror, ['for-each-ref', '--format=%(refname) %(objectname)', 'refs/heads/served']);
const line = refs.split('\n').find(line => line.startsWith('refs/heads/served '));
if (!line) {
if (existsSync(join(mirror, 'refs/heads/served'))) fail('library-state-invalid', 'invalid served ref');
return null;
}
const commit = line.slice('refs/heads/served '.length);
if (!SHA.test(commit) || await git(mirror, ['cat-file', '-t', commit]) !== 'commit') fail('library-state-invalid', 'served ref must name a commit');
return commit;
} catch { fail('library-state-invalid', 'served ref is unreadable'); }
}
function validProblem(value) {
return object(value) && Object.keys(value).every(key => ['code', 'path', 'message', 'line'].includes(key))
&& Object.hasOwn(RULES, value.code) && value.message === RULES[value.code]
&& typeof value.path === 'string' && value.path.length > 0 && value.path.length <= 4096 && !value.path.includes('\0')
&& (!Object.hasOwn(value, 'line') || Number.isSafeInteger(value.line) && value.line > 0);
}
async function readStateHeld(mirror, { branch } = {}) {
if (typeof branch !== 'string' || !branch) fail('library-state-invalid', 'configured branch required');
const served = await servedCommit(mirror);
let fd, value;
try {
try { fd = openSync(join(mirror, SYNC_STATE_FILE), constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
catch (error) {
if (error.code === 'ENOENT' && served === null) return empty(branch);
fail('library-state-invalid', 'sync state missing or unsafe');
}
const st = fstatSync(fd);
if (!privateFile(st) || st.size > STATE_MAX) fail('library-state-invalid', 'sync state must be bounded and private');
const bytes = Buffer.alloc(Math.min(st.size + 1, STATE_MAX + 1));
let used = 0;
while (used < bytes.length) { const read = readSync(fd, bytes, used, bytes.length - used, null); if (!read) break; used += read; }
if (used > STATE_MAX || used > st.size) fail('library-state-invalid', 'state changed during bounded read');
value = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes.subarray(0, used)));
} catch { fail('library-state-invalid', 'invalid sync state'); }
finally { if (fd !== undefined) closeSync(fd); }
if (!keys(value, ['version', 'branch', 'served', 'lastFetch', 'lastError', 'diverged', 'problems'])
|| value.version !== 1 || value.branch !== branch || value.served !== served
|| !optional(value.served, v => typeof v === 'string' && SHA.test(v)) || !optional(value.lastFetch, utc)
|| !optional(value.lastError, v => keys(v, ['code', 'at']) && typeof v.code === 'string' && /^[a-z-]{1,64}$/.test(v.code) && utc(v.at))
|| !optional(value.diverged, v => keys(v, ['commit', 'at']) && typeof v.commit === 'string' && SHA.test(v.commit) && utc(v.at))
|| !Array.isArray(value.problems) || !value.problems.every(validProblem))
fail('library-state-invalid', 'sync state does not match its contract or served ref');
return value;
}
export async function readSyncState(mirror, options) {
return withLibraryLock(mirror, lock => lock.readSyncState(options));
}
export async function libraryStatus(context) {
const mapping = contextEntry(context), mirror = mirrorPath(context);
return { project: context.project, ...await readSyncState(mirror, { branch: mapping.branch }) };
}
export async function readLibraryFile(context) {
if (!object(context) || Object.keys(context).some(key => !['business', 'project', 'dataRoot', 'path'].includes(key)))
fail('project-read-path', 'unknown library read arguments');
const mapping = contextEntry(context), path = context.path;
if (typeof path !== 'string' || !path || isAbsolute(path) || path.includes('\0') || path.split('/').some(p => !p || p === '.' || p === '..')
|| classify(mapping, path).scope === 'unmapped') fail('project-read-path', 'expected a mapped library note');
const mirror = mirrorPath(context);
// Snapshot under the publication lock, then read only immutable objects.
const state = await readSyncState(mirror, { branch: mapping.branch });
if (!state.served) fail('library-commit', 'library has never been synced');
const read = await readLibrary({ repo: mirror, commit: state.served, mapping });
if (!read.files.some(file => file.path === path)) fail('project-read-missing', 'file is absent or excluded at served commit');
const entry = (await listTree(mirror, state.served)).find(entry => entry.path === path);
const texts = await readTexts(mirror, [entry]);
if (!texts.has(path)) fail('project-read-missing', 'file is not readable');
return { project: context.project, path, commit: state.served, content: texts.get(path) };
}
+27
View File
@@ -0,0 +1,27 @@
// Errors and the R-rule problems of design 1.3. A problem never holds back a commit: it names a
// path and, where there is one, a line, and the index leaves the file or link out.
export class LibraryError extends Error {
constructor(code, message) { super(`${code}: ${message}`); this.code = code; }
}
export const raise = (code, message) => { throw new LibraryError(code, message); };
export const RULES = Object.freeze({
R1: 'not a regular blob up to 1 MiB of UTF-8 text',
R2: 'path is unmapped',
R3: 'front matter not fully parsed',
R4: 'link resolves to no file or to more than one',
R5: 'canon file refers into the playground',
R6: 'canon file refers into another universe',
R7: 'paths differ only in case',
});
export function problem(code, path, line, message = RULES[code]) {
if (!Object.hasOwn(RULES, code)) raise('library-problem-code', `unknown problem code ${code}`);
return Object.freeze({ code, path, ...(Number.isSafeInteger(line) ? { line } : {}), message });
}
// Stable order for status output and tests: path, then line, then code.
export function sortProblems(list) {
return [...list].sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1
: (a.line ?? 0) - (b.line ?? 0) || (a.code < b.code ? -1 : a.code > b.code ? 1 : 0));
}
+219
View File
@@ -0,0 +1,219 @@
// The commit reader of design 1.3. It reads a commit of her repo, never a working tree: it lists the
// tree with `git ls-tree -r -z` and reads blobs with `cat-file`, through the same bounded git()
// and gitBytes() as packages/project, so hooks, protocols and the environment are fixed the same
// way. Nothing in the repo is checked out or run (D7). Each R-rule is a problem, never a refusal
// of the commit; the affected file or link is left out of the index.
import { git, gitBytes } from '../../project/src/git.mjs';
import { classify } from './scope.mjs';
import { extractLinks, parseWikilink, resolver } from './links.mjs';
import { LibraryError, problem, raise, sortProblems } from './problems.mjs';
export const MAX_BLOB = 1024 * 1024;
const MAX_LISTING = 32 * 1024 * 1024;
const SHA = /^[0-9a-f]{40}$/;
const ENTRY = /^([0-7]{6}) (blob|tree|commit) ([0-9a-f]{40}) +(-|\d+)\t(.+)$/s;
export const LINK_FIELDS = Object.freeze(['appears_in', 'relationships']);
const READERS = 8;
async function local(promise, message) {
try { return await promise; } catch (e) { raise('library-git', `${message} (${e.code ?? 'error'})`); }
}
export async function headCommit(repo) {
return local(git(repo, ['rev-parse', '--verify', '--end-of-options', 'HEAD^{commit}']), 'repository has no readable HEAD commit');
}
// Every entry of the commit's tree: { mode, type, oid, size, path }. Sizes come from `-l`, so no
// blob is opened to list it.
export async function listTree(repo, commit) {
if (typeof commit !== 'string' || !SHA.test(commit)) raise('library-commit', 'expected a full commit id');
const kind = await local(git(repo, ['cat-file', '-t', commit]), 'commit is not in the repository');
if (kind !== 'commit') raise('library-commit', 'not a commit');
const bytes = await local(gitBytes(repo, ['ls-tree', '-r', '-z', '-l', '--full-tree', commit], MAX_LISTING), 'tree listing refused');
let text;
try { text = new TextDecoder('utf-8', { fatal: true }).decode(bytes); }
catch { raise('library-git', 'tree listing is not UTF-8'); }
const entries = text.split('\0');
if (entries.pop() !== '') raise('library-git', 'invalid tree listing');
return entries.map(entry => {
const m = ENTRY.exec(entry);
if (!m) raise('library-git', 'invalid tree listing');
return { mode: m[1], type: m[2], oid: m[3], size: m[4] === '-' ? null : Number(m[4]), path: m[5] };
});
}
// R1: only a 100644 blob up to 1 MiB of UTF-8 text without NUL is read.
export const readable = entry => entry.mode === '100644' && entry.type === 'blob'
&& Number.isSafeInteger(entry.size) && entry.size <= MAX_BLOB;
// Text of each readable entry, by path. An entry whose bytes aren't UTF-8 text is missing from the
// map, and the caller reports it under R1.
export async function readTexts(repo, entries) {
const texts = new Map(), queue = [...entries];
const decoder = new TextDecoder('utf-8', { fatal: true, ignoreBOM: false });
const worker = async () => {
for (let entry = queue.shift(); entry; entry = queue.shift()) {
const bytes = await local(gitBytes(repo, ['cat-file', 'blob', entry.oid], MAX_BLOB), 'blob read refused');
let value;
try { value = decoder.decode(bytes); } catch { continue; }
if (!value.includes('\0')) texts.set(entry.path, value);
}
};
await Promise.all(Array.from({ length: READERS }, worker));
return texts;
}
const QUOTED = /^"(?:[^"\\]|\\.)*"$|^'(?:[^']|'')*'$/;
const PLAIN_START = /^[^\s\[\]{}&*!|>%@`"'#,]/;
function scalar(raw) {
const value = raw.trim();
if (parseWikilink(value)) return { ok: true, value };
if (QUOTED.test(value)) {
if (value[0] === "'") return { ok: true, value: value.slice(1, -1).replace(/''/g, "'") };
try { return { ok: true, value: JSON.parse(value) }; } catch { return { ok: false }; }
}
if (value === '' || value === '~' || value === 'null') return { ok: true, value: null };
if (!PLAIN_START.test(value) || /: |\s#/.test(value) || value.endsWith(':')) return { ok: false };
if (value === 'true' || value === 'false') return { ok: true, value: value === 'true' };
if (/^-?(0|[1-9]\d*)(\.\d+)?$/.test(value)) return { ok: true, value: Number(value) };
return { ok: true, value };
}
function flow(raw) {
const inner = raw.trim().slice(1, -1), items = [];
let current = '', quote = null;
for (const ch of inner) {
if (quote) { current += ch; if (ch === quote) quote = null; continue; }
if (ch === '"' || ch === "'") { quote = ch; current += ch; continue; }
if (ch === '[' || ch === ']' || ch === '{' || ch === '}') return { ok: false };
if (ch === ',') { items.push(current); current = ''; continue; }
current += ch;
}
if (quote) return { ok: false };
if (current.trim() || items.length) items.push(current);
const values = [];
for (const item of items) { const s = scalar(item); if (!s.ok || !item.trim()) return { ok: false }; values.push(s.value); }
return { ok: true, value: values };
}
function value(raw) {
const text = raw.trim();
if (text.startsWith('[') && !parseWikilink(text)) return text.endsWith(']') ? flow(text) : { ok: false };
return scalar(text);
}
// Front matter of design 1.3 (R3): scalars, quoted strings, flow lists and block lists of scalars or
// wikilinks. Anything else, a nested map, a block scalar or an anchor for example, leaves the whole
// front matter as raw text with one R3 problem at the line where parsing stopped.
// Returns { frontmatter, raw, bodyLine, stop } with 1-based line numbers.
export function parseFrontMatter(lines) {
if (lines[0]?.replace(/^/, '').trimEnd() !== '---') return { frontmatter: null, raw: null, bodyLine: 1, stop: null };
const close = lines.findIndex((text, i) => i > 0 && /^(---|\.\.\.)\s*$/.test(text));
if (close < 0) return { frontmatter: null, raw: null, bodyLine: 1, stop: 1 };
const raw = lines.slice(1, close).join('\n'), data = {};
let list = null;
for (let i = 1; i < close; i++) {
const text = lines[i], line = i + 1;
if (!text.trim() || /^\s*#/.test(text)) continue;
const item = /^(\s*)- (.*)$|^(\s*)-$/.exec(text);
if (item && list) {
const parsed = scalar(item[2] ?? '');
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
data[list].push(parsed.value);
continue;
}
const pair = /^([^\s:#\-][^:]*?|"[^"]+"|'[^']+')\s*:(?:\s+(.*))?$/.exec(text);
if (!pair) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
const key = pair[1].replace(/^(["'])(.*)\1$/, '$2');
if (Object.hasOwn(data, key) || key === '__proto__') return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
if (pair[2] === undefined || !pair[2].trim()) {
const next = lines[i + 1];
if (i + 1 < close && /^\s*-( |$)/.test(next)) { data[key] = []; list = key; }
else { data[key] = null; list = null; }
continue;
}
list = null;
const parsed = value(pair[2]);
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
data[key] = parsed.value;
}
return { frontmatter: data, raw: null, bodyLine: close + 2, stop: null };
}
export const splitLines = text => text.replace(/^/, '').split('\n').map(l => l.endsWith('\r') ? l.slice(0, -1) : l);
// One parsed file: its lines (all of them, numbered from 1), front matter and the links it holds.
// Front matter links come from LINK_FIELDS values that are wikilinks; other values are names.
export function parseFile(text, { markdown = false } = {}) {
const lines = splitLines(text);
const fm = parseFrontMatter(lines);
const numbered = lines.map((t, i) => [i + 1, t]);
const links = extractLinks(numbered.slice(fm.bodyLine - 1), { markdown });
if (fm.frontmatter) {
for (const field of LINK_FIELDS) {
const raw = fm.frontmatter[field];
const values = Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw];
for (const v of values) {
const link = typeof v === 'string' ? parseWikilink(v.trim()) : null;
if (link) links.push({ ...link, field, line: frontMatterLine(lines, fm.bodyLine, field) });
}
}
}
return { lines: numbered, bodyLine: fm.bodyLine, frontmatter: fm.frontmatter, frontmatterRaw: fm.raw, stop: fm.stop, links };
}
function frontMatterLine(lines, bodyLine, field) {
const i = lines.slice(0, bodyLine - 1).findIndex(t => new RegExp(`^["']?${field}["']?\\s*:`).test(t));
return i < 0 ? undefined : i + 1;
}
// Read a commit under a checked mapping (mapping.mjs). Returns the indexed files and every
// R1-R7 problem: { commit, files: [...], problems: [...] }, frozen, sorted by path.
export async function readLibrary({ repo, commit, mapping }) {
if (!mapping || mapping.kind !== 'library') throw new LibraryError('library-mapping-invalid', 'expected a checked library mapping');
const entries = await listTree(repo, commit);
const problems = [], twins = new Set(), byLower = new Map();
for (const e of entries) byLower.set(e.path.toLowerCase(), [...(byLower.get(e.path.toLowerCase()) ?? []), e.path]);
for (const group of byLower.values()) if (group.length > 1) for (const path of group) twins.add(path);
const candidates = [];
for (const entry of entries) {
const where = classify(mapping, entry.path);
if (where.scope === 'unmapped') { problems.push(problem('R2', entry.path)); continue; }
if (twins.has(entry.path)) { problems.push(problem('R7', entry.path)); continue; }
if (!readable(entry)) { problems.push(problem('R1', entry.path)); continue; }
candidates.push({ entry, where });
}
const texts = await readTexts(repo, candidates.map(c => c.entry));
const files = [];
for (const { entry, where } of candidates) {
if (!texts.has(entry.path)) { problems.push(problem('R1', entry.path)); continue; }
const parsed = parseFile(texts.get(entry.path));
if (parsed.stop !== null) problems.push(problem('R3', entry.path, parsed.stop));
files.push({ path: entry.path, ...where, name: entry.path.split('/').pop().replace(/\.md$/, ''), ...parsed });
}
const resolve = resolver(files.map(f => f.path), { vaultPrefix: mapping.vaultPrefix });
const byPath = new Map(files.map(f => [f.path, f]));
for (const file of files) {
const kept = [];
for (const link of file.links) {
if (!link.file) continue;
const target = resolve(file.path, link.file);
if (!target.path) { problems.push(problem('R4', file.path, link.line)); continue; }
const to = byPath.get(target.path);
if (file.scope === 'canon') {
if (to.scope === 'playground') problems.push(problem('R5', file.path, link.line));
const sources = mapping.universes[file.universe].sources ?? [];
if (to.universe !== file.universe && !sources.includes(to.universe)) problems.push(problem('R6', file.path, link.line));
}
kept.push({ ...link, path: target.path, step: target.step });
}
file.links = kept;
delete file.stop;
}
files.sort((a, b) => a.path < b.path ? -1 : 1);
return deepFreeze({ commit, files, problems: sortProblems(problems) });
}
function deepFreeze(value) {
if (value && typeof value === 'object' && !Object.isFrozen(value)) { Object.freeze(value); for (const v of Object.values(value)) deepFreeze(v); }
return value;
}
+59
View File
@@ -0,0 +1,59 @@
// renderItems(body): how a stored cited reply looks (design 7.4), shared by the Console chat (C2-10b)
// and the Discord relay (C2-18). Pure: no git, no clock. The caller passes what it knows about cites.
// The guard (C2-06) checks limits before a reply is stored. This only refuses to show a body that
// doesn't have the reply shape, and then it shows a fixed line, never the body's text.
export const KINDS = Object.freeze({ fact: 'Fact', finding: 'Finding', correction: 'Correction', question: 'Question', idea: 'Idea' });
export const NOTICES = Object.freeze({ 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' });
export const UNREADABLE = 'This reply could not be shown.';
const SHA = /^[0-9a-f]{40}$/;
const plain = value => value !== null && typeof value === 'object' && !Array.isArray(value);
const line = n => Number.isSafeInteger(n) && n > 0;
function cite(value, { scopeOf, changed }) {
if (!plain(value) || typeof value.quote !== 'string') return null;
if (Object.hasOwn(value, 'url')) {
if (typeof value.url !== 'string' || Object.hasOwn(value, 'path')) return null;
return { badge: 'web', url: value.url, ref: value.url, quote: value.quote, changed: false };
}
const { path, commit, lines } = value;
if (typeof path !== 'string' || !SHA.test(commit ?? '') || !Array.isArray(lines) || lines.length !== 2
|| !line(lines[0]) || !line(lines[1]) || lines[1] < lines[0]) return null;
const badge = scopeOf?.(path);
const ref = `${path}:${lines[0] === lines[1] ? lines[0] : `${lines[0]}-${lines[1]}`} @${commit.slice(0, 7)}`;
return { badge: badge === 'canon' || badge === 'playground' ? badge : null, path, commit, lines: [...lines], ref,
quote: value.quote, changed: changed?.({ path, commit }) === true };
}
const quoted = text => JSON.stringify(text);
// body: the stored message body (a JSON string or the parsed object).
// options.scopeOf(path) -> 'canon' | 'playground' | undefined, for the scope badge.
// options.changed({path, commit}) -> true when the file's last commit is newer than the cited one.
// Returns { kind: 'items', items, text } | { kind: 'notice', notice, text } | { kind: 'unreadable', text }.
export function renderItems(body, options = {}) {
const unreadable = { kind: 'unreadable', text: UNREADABLE };
let value = body;
if (typeof body === 'string') { try { value = JSON.parse(body); } catch { return unreadable; } }
if (!plain(value) || !Array.isArray(value.items)) return unreadable;
const keys = Object.keys(value);
if (value.items.length === 0) {
if (keys.length !== 2 || !Object.hasOwn(NOTICES, value.notice)) return unreadable;
return { kind: 'notice', notice: value.notice, text: NOTICES[value.notice] };
}
if (keys.length !== 1) return unreadable;
const items = [];
for (const item of value.items) {
if (!plain(item) || !Object.hasOwn(KINDS, item.kind) || typeof item.text !== 'string' || !Array.isArray(item.cites)) return unreadable;
const correction = item.kind === 'correction';
if (correction && (typeof item.from !== 'string' || typeof item.to !== 'string')) return unreadable;
const cites = item.cites.map(c => cite(c, options));
if (cites.some(c => c === null)) return unreadable;
items.push({ kind: item.kind, label: KINDS[item.kind], text: item.text,
...(correction ? { from: item.from, to: item.to } : {}), cites });
}
const text = items.map((item, i) => [
`${i + 1}. ${item.label}: ${item.text}`,
...(item.kind === 'correction' ? [` From: ${quoted(item.from)}`, ` To: ${quoted(item.to)}`] : []),
...item.cites.map(c => ` ${c.badge ? `[${c.badge}] ` : ''}${c.ref} ${quoted(c.quote)}${c.changed ? ' (changed since)' : ''}`),
].join('\n')).join('\n');
return { kind: 'items', items, text };
}
+21
View File
@@ -0,0 +1,21 @@
// The scope rule of design 1.2. Scope comes from the path alone (D2): no file content can change it.
// Paths are repo-relative and compared exactly, case included (R7 handles case-only twins).
const inside = (folder, path) => path.startsWith(folder.endsWith('/') ? folder : `${folder}/`);
// { scope: 'playground' | 'canon', universe, book, type, folder } or { scope: 'unmapped' }.
// `type` is the mapped type of the first folder under the book (canon) or under the playground
// prefix (playground), 'manuscript' for a manuscript folder, else null. `folder` is that folder name.
export function classify(mapping, path) {
if (typeof path !== 'string' || !path.endsWith('.md')) return { scope: 'unmapped' };
for (const [universe, entry] of Object.entries(mapping.universes)) {
const book = entry.books.find(b => inside(b, path));
if (!book) continue;
const prefix = entry.playground.find(p => inside(p, path));
const rest = path.slice((prefix ?? `${book}/`).length).split('/');
const folder = rest.length > 1 ? rest[0] : null;
const type = folder === null ? null : Object.hasOwn(mapping.types, folder) ? mapping.types[folder]
: mapping.manuscript.includes(folder) ? 'manuscript' : null;
return { scope: prefix ? 'playground' : 'canon', universe, book, type, folder };
}
return { scope: 'unmapped' };
}
+126
View File
@@ -0,0 +1,126 @@
// `mosaic library survey <repo> [--mapping <file>]` (design 1.1, 10 step 7). It prints what a mapping
// needs: folder names, file counts by extension, front matter key sets and link-form counts. It
// never prints file text, front matter values, file names or link targets. With a mapping it adds
// scope counts, resolver step counts and R-problem counts by code. It reads the repo's HEAD commit
// through the commit reader, never the working tree.
import { readFileSync, realpathSync, statSync } from 'node:fs';
import { resolve as resolvePath } from 'node:path';
import { pathToFileURL } from 'node:url';
import { checkMapping } from './mapping.mjs';
import { classify } from './scope.mjs';
import { formOf } from './links.mjs';
import { headCommit, listTree, parseFile, readable, readLibrary, readTexts } from './reader.mjs';
import { LibraryError, RULES } from './problems.mjs';
export const USAGE = 'usage: mosaic library survey <repo> [--mapping <file>]';
// Keys are printed only in the short identifier forms property names take. A key with spaces or of
// any other shape could be a phrase of hers, so it is counted as one "(other key)".
const KEY = /^[A-Za-z_][A-Za-z0-9_-]{0,31}$/;
const OTHER_KEY = '(other key)';
const FORMS = ['plain', 'label', 'heading', 'embed', 'self', 'markdown'];
const STEPS = ['root', 'vault-prefix', 'relative', 'shortest'];
const extension = path => /\.md$/.test(path) ? 'md' : /\.canvas$/.test(path) ? 'canvas' : /\.base$/.test(path) ? 'base' : 'other';
// The survey as data. `mapping` is a checked mapping or undefined.
export async function survey({ repo, commit, mapping }) {
const head = commit ?? await headCommit(repo);
const entries = await listTree(repo, head);
const folders = new Map();
for (const entry of entries) {
const slash = entry.path.lastIndexOf('/');
const name = slash < 0 ? '' : entry.path.slice(0, slash);
const counts = folders.get(name) ?? { md: 0, canvas: 0, base: 0, other: 0 };
counts[entry.type === 'blob' ? extension(entry.path) : 'other']++;
folders.set(name, counts);
}
const markdown = entries.filter(e => e.path.endsWith('.md'));
const texts = await readTexts(repo, markdown.filter(readable));
const keySets = new Map(), forms = Object.fromEntries(FORMS.map(f => [f, 0]));
const fields = { appears_in: { wikilink: 0, name: 0 }, relationships: { wikilink: 0, name: 0 } };
let noFrontMatter = 0, unparsed = 0;
for (const text of texts.values()) {
const file = parseFile(text, { markdown: true });
if (file.stop !== null) unparsed++;
else if (!file.frontmatter) noFrontMatter++;
else {
const set = [...new Set(Object.keys(file.frontmatter).map(k => KEY.test(k) ? k : OTHER_KEY))].sort().join(', ');
keySets.set(set, (keySets.get(set) ?? 0) + 1);
for (const field of Object.keys(fields)) {
const raw = file.frontmatter[field];
for (const v of Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw])
fields[field][typeof v === 'string' && /^\s*!?\[\[[^\]]+\]\]\s*$/.test(v) ? 'wikilink' : 'name']++;
}
}
for (const link of file.links) if (!link.field) forms[formOf(link)]++;
}
const result = {
commit: head, entries: entries.length,
folders: [...folders].sort(([a], [b]) => a < b ? -1 : 1).map(([name, counts]) => ({ name, ...counts })),
markdown: { files: markdown.length, read: texts.size, noFrontMatter, unparsed },
keySets: [...keySets].sort(([a, x], [b, y]) => y - x || (a < b ? -1 : 1)).map(([keys, count]) => ({ keys, count })),
forms, fields,
};
if (mapping) {
const scopes = { canon: 0, playground: 0, unmapped: 0 };
for (const entry of entries) scopes[classify(mapping, entry.path).scope]++;
const library = await readLibrary({ repo, commit: head, mapping });
const steps = Object.fromEntries(STEPS.map(s => [s, 0]));
for (const file of library.files) for (const link of file.links) steps[link.step]++;
const problems = Object.fromEntries(Object.keys(RULES).map(code => [code, 0]));
for (const p of library.problems) problems[p.code]++;
steps.unresolved = problems.R4;
Object.assign(result, { scopes, indexed: library.files.length, steps, problems });
}
return result;
}
// Folder names are quoted as JSON strings, so no name can carry a control sequence to the terminal.
export function formatSurvey(s) {
const out = [`commit ${s.commit}`, `entries ${s.entries}`, 'folders (direct children: md canvas base other)'];
for (const f of s.folders) out.push(` ${f.name === '' ? '(root)' : JSON.stringify(f.name)} ${f.md} ${f.canvas} ${f.base} ${f.other}`);
out.push(`markdown ${s.markdown.files} files, ${s.markdown.read} read, ${s.markdown.noFrontMatter} without front matter, ${s.markdown.unparsed} with front matter not fully parsed`);
out.push('front matter key sets');
for (const k of s.keySets) out.push(` ${k.count} [${k.keys}]`);
out.push(`body links ${FORMS.map(f => `${f} ${s.forms[f]}`).join(', ')}`);
for (const [field, c] of Object.entries(s.fields)) out.push(`field ${field} wikilink ${c.wikilink}, name ${c.name}`);
if (s.scopes) {
out.push(`scopes canon ${s.scopes.canon}, playground ${s.scopes.playground}, unmapped ${s.scopes.unmapped}; indexed ${s.indexed}`);
out.push(`resolved ${STEPS.map(st => `${st} ${s.steps[st]}`).join(', ')}, unresolved ${s.steps.unresolved}`);
out.push(`problems ${Object.entries(s.problems).map(([code, n]) => `${code} ${n}`).join(', ')}`);
}
return `${out.join('\n')}\n`;
}
// Exit codes as packages/project's CLI: 0 done, 2 the repo can't be read, 3 a bad mapping, 4 usage.
export async function surveyMain(argv, io = { stdout: process.stdout, stderr: process.stderr }) {
const fail = (code, message) => { io.stderr.write(`library survey: ${message}\n`); return code; };
const [verb, ...rest] = argv;
if (verb !== 'survey') return fail(4, USAGE);
let repo, mappingFile;
for (let i = 0; i < rest.length; i++) {
if (rest[i] === '--mapping' && i + 1 < rest.length && mappingFile === undefined) mappingFile = rest[++i];
else if (!rest[i].startsWith('-') && repo === undefined) repo = rest[i];
else return fail(4, USAGE);
}
if (repo === undefined) return fail(4, USAGE);
let mapping;
if (mappingFile !== undefined) {
let entry;
try { entry = JSON.parse(readFileSync(mappingFile, 'utf8')); }
catch { return fail(3, `library-mapping-invalid: ${mappingFile} is not a readable JSON file`); }
try { mapping = checkMapping(entry, mappingFile); }
catch (e) { if (e instanceof LibraryError) return fail(3, e.message); throw e; }
}
const path = resolvePath(repo);
try { if (!statSync(path).isDirectory()) return fail(2, `library-git: ${repo} is not a directory`); }
catch { return fail(2, `library-git: ${repo} is not a directory`); }
try { io.stdout.write(formatSurvey(await survey({ repo: path, mapping }))); }
catch (e) { if (e instanceof LibraryError) return fail(2, e.message); throw e; }
return 0;
}
const entry = () => { try { return import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href; } catch { return false; } };
if (process.argv[1] && entry()) {
const code = await surveyMain(process.argv.slice(2));
if (code !== 0) process.exitCode = code;
}
+12
View File
@@ -0,0 +1,12 @@
{
"kind": "library",
"remote": "https://git.example.invalid/writer/books.git",
"branch": "main",
"vaultPrefix": "books",
"universes": {
"harbor": {"name": "The Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"]},
"second": {"name": "Second World", "books": ["Second World"]}
},
"types": {"Characters": "character", "Places": "place", "Research": "research", "Tracker": "tracker", "Summaries": "summary"},
"manuscript": ["Chapters"]
}
+54
View File
@@ -0,0 +1,54 @@
// A repo the test builds in her layout's shape: spaces and capitals in folder names, a .canvas file
// and an unmapped top-level note. Bodies are made up; MARKER stands in for her text.
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync, symlinkSync, chmodSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { checkMapping } from '../src/mapping.mjs';
export const MARKER = 'MARKER-7f3a-HER-TEXT';
export const FIXTURES = fileURLToPath(new URL('./fixtures/', import.meta.url));
export const mappingEntry = () => JSON.parse(readFileSync(join(FIXTURES, 'mapping.json'), 'utf8'));
export const mapping = (change = e => e) => checkMapping(change(mappingEntry()));
export const git = (repo, ...args) => execFileSync('git', ['-c', 'user.name=fixture', '-c', '[email protected]',
'-c', 'commit.gpgsign=false', '-c', 'core.hooksPath=/dev/null', '-C', repo, ...args], { encoding: 'utf8' }).trim();
export const BASE = {
'Book One/Characters/Mara Vell.md': [
'---', 'type: character', 'name: Mara Vell', 'appears_in: "[[Book One/Chapters/Chapter 01]]"',
'relationships:', ' - "[[Tomas]]"', ' - The Ferryman', '---', `Grey eyes. ${MARKER}`, ''].join('\n'),
'Book One/Characters/Tomas.md': ['---', 'type: character', 'name: Tomas', 'appears_in: ["[[Chapter 01]]"]', '---', MARKER, ''].join('\n'),
'Book One/Chapters/Chapter 01.md': [
'# One', `${MARKER} [[Mara Vell]] met [[Book One/Places/Harbor Town|the harbor]].`,
'See [[../Places/Harbor Town#Docks]] and [[books/Book One/Characters/Tomas]].',
'```', '[[Not A Link]]', '```', 'Inline `[[Not Either]]` and [a note](Notes.md).', ''].join('\n'),
'Book One/Places/Harbor Town.md': ['---', 'type: place', '---', `Salt. ${MARKER}`, ''].join('\n'),
'Book One/Playground/What If.md': ['---', 'name: What If', '---', `${MARKER} [[Mara Vell]]`, ''].join('\n'),
'Book One/Board.canvas': '{"nodes":[]}\n',
'Book One/Playground/Sketch.canvas': '{"nodes":[]}\n',
'Inbox Note.md': `${MARKER}\n`,
'Second World/Chapters/01.md': `# Elsewhere ${MARKER}\n`,
};
// repo({files, extra}) writes files (path -> text, or { link } / { mode, text }), commits, and
// returns { repo, commit }.
export function repo(t, files = BASE) {
const root = mkdtempSync(join(tmpdir(), 'library-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const dir = join(root, 'books');
mkdirSync(dir);
git(dir, 'init', '-q', '-b', 'main');
for (const [path, value] of Object.entries(files)) {
const file = join(dir, path);
mkdirSync(dirname(file), { recursive: true });
if (typeof value === 'object' && value.link) symlinkSync(value.link, file);
else if (typeof value === 'object' && value.bytes) writeFileSync(file, value.bytes);
else writeFileSync(file, typeof value === 'object' ? value.text : value);
if (typeof value === 'object' && value.mode) chmodSync(file, value.mode);
}
git(dir, 'add', '-A');
git(dir, 'commit', '-qm', 'fixture');
return { root, repo: dir, commit: git(dir, 'rev-parse', 'HEAD') };
}
+50
View File
@@ -0,0 +1,50 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { extractLinks, parseWikilink, resolver } from '../src/index.mjs';
const PATHS = ['Book One/Characters/Mara Vell.md', 'Book One/Characters/Tomas.md', 'Book One/Places/Harbor Town.md',
'Book One/Chapters/Chapter 01.md', 'Book One/Places/Notes.md', 'Book One/Characters/Notes.md', 'Mara Vell.md'];
const resolve = resolver(PATHS, { vaultPrefix: 'books' });
const FROM = 'Book One/Chapters/Chapter 01.md';
test('resolver step 1: a path from the repo root, with or without .md', () => {
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
// Root wins over shortest: Mara Vell.md at the root, not the character page.
assert.deepEqual(resolve(FROM, 'Mara Vell'), { path: 'Mara Vell.md', step: 'root' });
});
test('resolver step 2: the same path with vaultPrefix/ stripped', () => {
assert.deepEqual(resolve(FROM, 'books/Book One/Characters/Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'vault-prefix' });
assert.deepEqual(resolver(PATHS)(FROM, 'books/Book One/Characters/Tomas'), { path: null, step: 'missing' });
});
test('resolver step 3: a path relative to the linking file, never above the repo root', () => {
assert.deepEqual(resolve(FROM, '../Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'relative' });
assert.deepEqual(resolve('Book One/Places/Harbor Town.md', './Notes'), { path: 'Book One/Places/Notes.md', step: 'relative' });
assert.deepEqual(resolve(FROM, '../../../Book One/Places/Harbor Town'), { path: null, step: 'missing' });
});
test('resolver step 4: a base name that matches exactly one indexed file', () => {
assert.deepEqual(resolve(FROM, 'Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'shortest' });
assert.deepEqual(resolve(FROM, 'Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'shortest' });
assert.deepEqual(resolve(FROM, 'Notes'), { path: null, step: 'ambiguous', count: 2 });
assert.deepEqual(resolve(FROM, 'Nobody'), { path: null, step: 'missing' });
// A partial path is not a base name.
assert.deepEqual(resolve(FROM, 'Places/Harbor Town'), { path: null, step: 'missing' });
});
test('a link into another of her repos resolves to nothing', () => {
assert.deepEqual(resolve(FROM, 'bean-brain/Journal/Today'), { path: null, step: 'missing' });
assert.deepEqual(resolve(FROM, '../bean-brain/Journal/Today'), { path: null, step: 'missing' });
});
test('wikilink forms, embeds, and links skipped inside code', () => {
assert.deepEqual(parseWikilink('[[A/B#Head|Label]]'), { file: 'A/B', heading: 'Head', label: 'Label', embed: false });
assert.deepEqual(parseWikilink('![[pic.png]]'), { file: 'pic.png', heading: null, label: null, embed: true });
assert.equal(parseWikilink('[[a]] and more'), null);
const lines = [[1, '[[One]] [[Two|2]] [[Three#H]] ![[Four]] [[#Self]]'], [2, '```js'], [3, '[[Fenced]]'], [4, '```'],
[5, 'Inline `[[Code]]` and [md](Five.md) and [web](https://example.invalid/x)']];
assert.deepEqual(extractLinks(lines).map(l => [l.file, l.line]), [['One', 1], ['Two', 1], ['Three', 1], ['Four', 1], ['', 1]]);
assert.deepEqual(extractLinks(lines, { markdown: true }).filter(l => l.markdown).map(l => l.file), ['Five.md']);
});
+82
View File
@@ -0,0 +1,82 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { checkMapping, classify, LibraryError } from '../src/index.mjs';
import { mapping, mappingEntry } from './helpers.mjs';
const refuses = (change, code, pattern) => assert.throws(() => checkMapping(change(mappingEntry())),
e => e instanceof LibraryError && e.code === code && (!pattern || pattern.test(e.message)));
test('the fixture mapping checks and comes back frozen', () => {
const m = mapping();
assert.equal(m.universes.harbor.playground[0], 'Book One/Playground/');
assert.ok(Object.isFrozen(m.universes.harbor.books));
assert.throws(() => { m.types.Extra = 'place'; });
});
test('a playground prefix outside its universe books refuses library-playground-outside-book', () => {
refuses(e => { e.universes.harbor.playground = ['Second World/Playground/']; return e; }, 'library-playground-outside-book');
refuses(e => { e.universes.harbor.playground = ['Elsewhere/']; return e; }, 'library-playground-outside-book');
// The whole book as its playground would leave no canon: not inside the book.
refuses(e => { e.universes.harbor.playground = ['Book One/']; return e; }, 'library-playground-outside-book');
});
test('no playground prefix holds another, and no book folder holds another', () => {
refuses(e => { e.universes.harbor.playground = ['Book One/Playground/', 'Book One/Playground/Deep/']; return e; },
'library-mapping-invalid', /holds playground/);
refuses(e => { e.universes.second.books = ['Second World', 'Book One/Part Two']; return e; }, 'library-mapping-invalid', /holds book folder/);
refuses(e => { e.universes.second.books = ['Book One']; return e; }, 'library-mapping-invalid', /in two universes/);
});
test('shape refusals: kind, remote, branch, unknown keys, types, manuscript, folder names', () => {
refuses(e => ({ ...e, kind: 'code' }), 'library-mapping-invalid', /kind/);
refuses(e => ({ ...e, remote: 'http://git.example.invalid/x.git' }), 'library-mapping-invalid', /https/);
refuses(e => ({ ...e, remote: 'https://user:[email protected]/x.git' }), 'library-mapping-invalid', /credentials/);
refuses(e => ({ ...e, remote: '[email protected]:x.git' }), 'library-mapping-invalid', /https/);
refuses(e => ({ ...e, branch: 'main..x' }), 'library-mapping-invalid', /branch/);
refuses(e => ({ ...e, root: '/tmp' }), 'library-mapping-invalid', /unknown key "root"/);
refuses(e => { e.universes.harbor.extra = 1; return e; }, 'library-mapping-invalid', /unknown key/);
refuses(e => { e.types.Characters = 'villain'; return e; }, 'library-mapping-invalid', /types.Characters/);
refuses(e => { e.types['A/B'] = 'place'; return e; }, 'library-mapping-invalid', /one folder name/);
refuses(e => ({ ...e, manuscript: ['Characters'] }), 'library-mapping-invalid', /also a type folder/);
refuses(e => { e.universes.harbor.books = ['../Book One']; return e; }, 'library-mapping-invalid', /folder path/);
refuses(e => ({ ...e, vaultPrefix: 'a/b' }), 'library-mapping-invalid', /vaultPrefix/);
refuses(e => ({ ...e, universes: {} }), 'library-mapping-invalid', /at least one universe/);
refuses(e => { e.universes['bad id'] = e.universes.second; delete e.universes.second; return e; }, 'library-mapping-invalid', /plain id/);
});
test('a hybrid universe is a shape the query code can take (1.4), with sources that must exist', () => {
const hybrid = e => { e.universes.cross = { kind: 'hybrid', name: 'Crossover', sources: ['harbor', 'second'], books: ['Crossover'],
constraints: [{ exclude: 'second/Characters' }, { pin: { entity: 'Book One/Characters/Mara Vell.md', field: 'eyes', value: 'grey' } }],
notes: 'Set after book one.' }; return e; };
const m = checkMapping(hybrid(mappingEntry()));
assert.deepEqual(m.universes.cross.sources, ['harbor', 'second']);
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor', 'nowhere']; return e; }, 'library-mapping-invalid', /non-hybrid universe/);
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor']; return e; }, 'library-mapping-invalid', /two or more/);
refuses(e => { hybrid(e); e.universes.cross.constraints = [{ exclude: 'x', pin: {} }]; return e; }, 'library-mapping-invalid', /exactly one/);
});
test('a caller-supplied fail receives the code, as business file loading will pass it (C2-04)', () => {
const seen = [];
const fail = (code, message) => { seen.push(code); throw new Error(message); };
const entry = mappingEntry();
entry.universes.harbor.playground = ['Elsewhere/'];
assert.throws(() => checkMapping(entry, 'businesses/acme.json projects.books', { fail }), /businesses\/acme.json projects.books/);
assert.deepEqual(seen, ['library-playground-outside-book']);
});
test('scope comes from the path alone (design 1.2)', () => {
const m = mapping();
assert.deepEqual(classify(m, 'Book One/Characters/Mara Vell.md'),
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
assert.deepEqual(classify(m, 'Book One/Chapters/Chapter 01.md'),
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'manuscript', folder: 'Chapters' });
assert.deepEqual(classify(m, 'Book One/Playground/What If.md'),
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: null, folder: null });
assert.deepEqual(classify(m, 'Book One/Playground/Characters/New.md'),
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
assert.deepEqual(classify(m, 'Book One/Notes.md'), { scope: 'canon', universe: 'harbor', book: 'Book One', type: null, folder: null });
assert.equal(classify(m, 'Second World/Chapters/01.md').universe, 'second');
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas', 'Book One/Board.base',
'Book One/cover.png', 'Book One/Upper.MD', 'Book Onex/a.md', 'book one/a.md', 'Elsewhere/Book One/a.md'])
assert.deepEqual(classify(m, path), { scope: 'unmapped' }, path);
});
+247
View File
@@ -0,0 +1,247 @@
// C2-04: local made-up repositories only. No remote credential or live config.
import test from 'node:test';
import assert from 'node:assert/strict';
import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, readFileSync, readdirSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { spawn } from 'node:child_process';
import { once } from 'node:events';
import { mappingEntry, repo, git, BASE, MARKER } from './helpers.mjs';
import * as api from '../src/mirror.mjs';
const NOW = '2026-10-21T00:00:00.000Z';
const MOD = new URL('../src/mirror.mjs', import.meta.url).href;
const state = (served = null) => ({ version: 1, branch: 'main', served, lastFetch: null, lastError: null, diverged: null, problems: [] });
const stateFile = mirror => join(mirror, api.SYNC_STATE_FILE);
async function fixture(t, files = BASE) {
const f = repo(t, files), dataRoot = join(f.root, 'data');
const context = { business: { id: 'acme', projects: { books: mappingEntry(), code: { root: f.repo } } }, project: 'books', dataRoot };
const mirror = await api.ensureMirror(context);
const writeState = value => writeFileSync(stateFile(mirror), JSON.stringify(value), { mode: 0o600 });
const seed = (commit = f.commit) => {
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, `${commit}:refs/heads/served`);
writeState(state(commit));
};
return { ...f, dataRoot, context, mirror, writeState, seed };
}
test('C2-04 mirror: exact default path, bare without checkout or remote, and idempotent initialization', async t => {
const f = await fixture(t);
assert.equal(api.SYNC_STATE_FILE, 'mosaic-library-sync.json');
assert.equal(api.mirrorPath(f.context), join(f.dataRoot, 'projects/acme/books'));
assert.equal(f.mirror, api.mirrorPath(f.context));
assert.equal(git(f.mirror, 'rev-parse', '--is-bare-repository'), 'true');
assert.equal(git(f.mirror, 'remote'), '');
assert.equal(existsSync(join(f.mirror, '.git')), false);
const lock = join(f.mirror, 'mosaic-library-sync.lock'), st = lstatSync(lock);
assert.equal(st.mode & 0o777, 0o600);
assert.equal(st.nlink, 1);
assert.equal(await api.ensureMirror(f.context), f.mirror);
assert.equal(lstatSync(lock).ino, st.ino, 'the lock inode is stable');
assert.equal(await api.servedCommit(f.mirror), null);
assert.deepEqual(await api.libraryStatus(f.context), { project: 'books', ...state() });
});
test('C2-04 mirror: simultaneous creators publish one intact mirror and leave no staging paths', async t => {
const f = repo(t), dataRoot = join(f.root, 'data');
const c = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
const results = await Promise.all(Array.from({ length: 4 }, () => api.ensureMirror(c)));
assert.deepEqual([...new Set(results)], [join(dataRoot, 'projects/acme/books')]);
assert.deepEqual(readdirSync(join(dataRoot, 'projects/acme')), ['books']);
assert.equal(git(results[0], 'rev-parse', '--is-bare-repository'), 'true');
});
test('C2-04 mirror: skipped publication accepts both coreutils statuses without removing the winner', { timeout: 15000 }, async t => {
for (const status of [0, 1]) {
const f = repo(t), dataRoot = join(f.root, 'data');
const context = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
const creator = child(t, `import cp from 'node:child_process';import {cpSync,chmodSync,lstatSync,readdirSync} from 'node:fs';import {dirname} from 'node:path';import {syncBuiltinESMExports} from 'node:module';
const original=cp.execFile;let winner;
cp.execFile=(command,args,options,callback)=>{if(command!=='/usr/bin/mv')return original(command,args,options,callback);
cpSync(args[2],args[3],{recursive:true});chmodSync(args[3],0o700);winner=lstatSync(args[3]).ino;
const error=${status} ? Object.assign(Error('fixture no-clobber diagnostic'),{code:${status}}) : null;
queueMicrotask(()=>callback(error));};syncBuiltinESMExports();
const {ensureMirror}=await import(${JSON.stringify(MOD)});const mirror=await ensureMirror(${JSON.stringify(context)});
process.send({same:lstatSync(mirror).ino===winner,entries:readdirSync(dirname(mirror))});process.disconnect();`);
const [result] = await Promise.race([once(creator, 'message'), creator.finished.then(result => { throw Error(JSON.stringify(result)); })]);
assert.deepEqual(result, { same: true, entries: ['books'] });
assert.deepEqual(await creator.finished, { code: 0, stderr: '' });
}
});
test('C2-04 mirror: preserve occupied directories and reject symlink ancestors and code overlaps', async t => {
const f = repo(t), dataRoot = join(f.root, 'data');
const c = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
const path = join(dataRoot, 'projects/acme/books'); mkdirSync(path, { recursive: true });
writeFileSync(join(path, 'keep'), 'made-up human file');
await assert.rejects(api.ensureMirror(c));
assert.equal(readFileSync(join(path, 'keep'), 'utf8'), 'made-up human file');
rmSync(dataRoot, { recursive: true }); symlinkSync(f.repo, dataRoot);
assert.throws(() => api.mirrorPath(c), { code: 'project-path' });
unlinkSync(dataRoot);
c.business.projects.code = { root: dataRoot };
assert.throws(() => api.mirrorPath(c), { code: 'project-path' });
});
test('C2-04 mirror: existing metadata accepts only bare Git plus state and the precreated lock', async t => {
const f = await fixture(t);
writeFileSync(join(f.mirror, 'unknown-note'), 'do not silently allow this');
await assert.rejects(api.ensureMirror(f.context), { code: 'library-git' });
assert.equal(readFileSync(join(f.mirror, 'unknown-note'), 'utf8'), 'do not silently allow this');
unlinkSync(join(f.mirror, 'unknown-note'));
git(f.mirror, 'config', 'remote.origin.url', 'https://example.invalid/fixture.git');
await assert.rejects(api.ensureMirror(f.context), { code: 'library-git' });
git(f.mirror, 'config', '--remove-section', 'remote.origin');
unlinkSync(join(f.mirror, 'mosaic-library-sync.lock'));
await assert.rejects(api.ensureMirror(f.context), { code: 'library-state-invalid' });
assert.equal(existsSync(join(f.mirror, 'mosaic-library-sync.lock')), false);
});
test('C2-04 state: missing state is empty only while the served ref is absent', async t => {
const f = await fixture(t);
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
f.seed(); unlinkSync(stateFile(f.mirror));
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
});
test('C2-04 state: valid problems and sync metadata return no content and change no state bytes', async t => {
const f = await fixture(t); f.seed();
const value = { ...state(f.commit), lastFetch: NOW, lastError: { code: 'library-git', at: NOW },
diverged: { commit: 'a'.repeat(40), at: NOW },
problems: [{ code: 'R4', path: 'Book One/Chapters/A.md', line: 2, message: 'link resolves to no file or to more than one' }] };
f.writeState(value); const before = readFileSync(stateFile(f.mirror));
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), value);
assert.equal(JSON.stringify(await api.libraryStatus(f.context)).includes(MARKER), false);
assert.deepEqual(readFileSync(stateFile(f.mirror)), before);
});
test('C2-04 state: strict keys, types, times, branch, commit and fixed R messages fail closed', async t => {
const f = await fixture(t); f.seed();
const mutations = [
s => { s.extra = true; }, s => { s.version = 2; }, s => { s.branch = 'other'; },
s => { s.served = 'a'.repeat(40); }, s => { s.served = null; }, s => { s.served = 'ABC'; },
s => { s.lastFetch = 'yesterday'; }, s => { s.lastFetch = 1; },
s => { s.lastError = { code: 'bad code', at: NOW }; },
s => { s.lastError = { code: 'library-git', at: NOW, stderr: MARKER }; },
s => { s.diverged = { commit: 'not-a-sha', at: NOW }; },
s => { s.problems = [{ code: 'R1', path: 'Book One/A.md', message: MARKER }]; },
s => { s.problems = [{ code: 'R4', path: 'Book One/A.md', line: 0, message: 'link resolves to no file or to more than one' }]; },
s => { s.problems = null; },
];
for (const mutate of mutations) {
const s = state(f.commit); mutate(s); f.writeState(s);
const before = readFileSync(stateFile(f.mirror));
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
assert.deepEqual(readFileSync(stateFile(f.mirror)), before);
}
writeFileSync(stateFile(f.mirror), '{');
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
});
test('C2-04 state: symlink, hardlink, non-private state and missing lock refuse without replacement', async t => {
const f = await fixture(t); f.seed();
const outside = join(f.root, 'outside.json'); writeFileSync(outside, JSON.stringify(state(f.commit)), { mode: 0o600 });
unlinkSync(stateFile(f.mirror)); symlinkSync(outside, stateFile(f.mirror));
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
unlinkSync(stateFile(f.mirror)); linkSync(outside, stateFile(f.mirror));
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
unlinkSync(stateFile(f.mirror)); f.writeState(state(f.commit)); chmodSync(stateFile(f.mirror), 0o644);
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
chmodSync(stateFile(f.mirror), 0o600); unlinkSync(join(f.mirror, 'mosaic-library-sync.lock'));
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
assert.equal(existsSync(join(f.mirror, 'mosaic-library-sync.lock')), false);
});
test('C2-04 file: served commit, not HEAD or a caller commit, owns the exact returned text', async t => {
const f = await fixture(t); f.seed();
const path = 'Book One/Characters/Tomas.md';
writeFileSync(join(f.repo, path), 'newer text\n'); git(f.repo, 'commit', '-qam', 'newer');
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/unserved');
git(f.mirror, 'symbolic-ref', 'HEAD', 'refs/heads/unserved');
assert.deepEqual(await api.readLibraryFile({ ...f.context, path }), { project: 'books', path, commit: f.commit, content: BASE[path] });
await assert.rejects(api.readLibraryFile({ ...f.context, path, commit: git(f.repo, 'rev-parse', 'HEAD') }), { code: 'project-read-path' });
for (const path of ['../escape.md', '/absolute.md', 'Book One/Board.canvas', 'Inbox Note.md'])
await assert.rejects(api.readLibraryFile({ ...f.context, path }), { code: 'project-read-path' });
await assert.rejects(api.readLibraryFile({ ...f.context, path: 'Book One/Missing.md' }), { code: 'project-read-missing' });
});
test('C2-04 file: R1, R2 and R7 exclusions and never-synced mirrors cannot return file bodies', async t => {
const f = await fixture(t, { ...BASE, 'Book One/Bad.md': { link: 'Characters/Tomas.md' },
'Book One/Executable.md': { mode: 0o755, text: MARKER },
'Book One/Twin.md': MARKER, 'Book One/twin.md': MARKER });
await assert.rejects(api.readLibraryFile({ ...f.context, path: 'Book One/Characters/Tomas.md' }), { code: 'library-commit' });
f.seed();
for (const path of ['Book One/Bad.md', 'Book One/Executable.md', 'Book One/Twin.md', 'Book One/twin.md'])
await assert.rejects(api.readLibraryFile({ ...f.context, path }), { code: 'project-read-missing' });
});
function child(t, source) {
const env = { ...process.env }; delete env.NODE_TEST_CONTEXT;
const p = spawn(process.execPath, ['--input-type=module', '-e', source], { env, stdio: ['ignore', 'pipe', 'pipe', 'ipc'] });
let stderr = ''; p.stderr.on('data', b => { stderr += b; }); p.stdout.resume();
p.finished = new Promise(resolve => { p.once('error', e => resolve({ error: e })); p.once('close', code => resolve({ code, stderr })); });
t.after(async () => { if (p.exitCode === null) p.kill('SIGKILL'); await p.finished; });
return p;
}
test('C2-04 lock: a first-sync and fast-forward reader wait for a consistent pair across processes', { timeout: 15000 }, async t => {
const f = await fixture(t);
// Transfer objects without serving them. No fetch or remote is added by the API.
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/candidate');
for (const first of [true, false]) {
if (!first) { f.seed(); writeFileSync(join(f.repo, 'Book One/New.md'), '# New\n'); git(f.repo, 'add', '.'); git(f.repo, 'commit', '-qm', 'new');
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/candidate'); }
const commit = git(f.repo, 'rev-parse', 'HEAD');
const writer = child(t, `import {withLibraryLock} from ${JSON.stringify(MOD)};import {writeFileSync} from 'node:fs';import {git} from ${JSON.stringify(new URL('../../project/src/git.mjs', import.meta.url).href)};
await withLibraryLock(${JSON.stringify(f.mirror)},async()=>{await git(${JSON.stringify(f.mirror)},['update-ref','refs/heads/served',${JSON.stringify(commit)}]);process.send('moved');await new Promise(r=>process.once('message',r));writeFileSync(${JSON.stringify(stateFile(f.mirror))},${JSON.stringify(JSON.stringify(state(commit)))},{mode:0o600});});process.disconnect();`);
const [message] = await once(writer, 'message'); assert.equal(message, 'moved');
let complete = false;
const read = api.readSyncState(f.mirror, { branch: 'main' }).then(value => { complete = true; return value; });
await new Promise(r => setTimeout(r, 50)); assert.equal(complete, false, 'the inconsistent pair is not observed');
writer.send('publish');
assert.deepEqual(await read, state(commit));
assert.deepEqual(await writer.finished, { code: 0, stderr: '' });
}
});
test('C2-04 lock: same-process waits, timeout code, callback expiry and throw release', { timeout: 15000 }, async t => {
const f = await fixture(t); let release, scoped;
const held = api.withLibraryLock(f.mirror, async lock => { scoped = lock.readSyncState; await new Promise(r => { release = r; }); });
const deadline = Date.now() + 5000;
while (!release && Date.now() < deadline) await new Promise(r => setTimeout(r, 5));
assert.equal(typeof release, 'function', 'first lock acquired before deadline');
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-sync-busy' });
release(); await held;
await assert.rejects(scoped({ branch: 'main' }), { code: 'library-state-invalid' });
await assert.rejects(api.withLibraryLock(f.mirror, async () => { throw Error('fixture throw'); }), /fixture throw/);
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
});
test('C2-04 lock: owner death releases the kernel lock without replacing its file', { timeout: 15000 }, async t => {
const f = await fixture(t), lock = join(f.mirror, 'mosaic-library-sync.lock'), ino = lstatSync(lock).ino;
const owner = child(t, `import {withLibraryLock} from ${JSON.stringify(MOD)};await withLibraryLock(${JSON.stringify(f.mirror)},async()=>{process.send('held');await new Promise(r=>process.once('message',r));});`);
const [message] = await once(owner, 'message'); assert.equal(message, 'held');
owner.kill('SIGKILL'); await owner.finished;
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
assert.equal(lstatSync(lock).ino, ino);
});
test('C2-04 lock: a missing flock binary becomes a handled refusal', { timeout: 10000 }, async t => {
const f = await fixture(t);
const missing = child(t, `import cp from 'node:child_process';import {syncBuiltinESMExports} from 'node:module';import {EventEmitter} from 'node:events';
const original=cp.spawn;cp.spawn=(command,...args)=>{if(command!=='/usr/bin/flock')return original(command,...args);const p=new EventEmitter();process.nextTick(()=>p.emit('error',Object.assign(Error('fixture missing flock'),{code:'ENOENT'})));return p;};syncBuiltinESMExports();
const {readSyncState}=await import(${JSON.stringify(MOD)});try{await readSyncState(${JSON.stringify(f.mirror)},{branch:'main'});process.send('unexpected success');}catch(e){process.send(e.code);}process.disconnect();`);
const [code] = await once(missing, 'message');
assert.equal(code, 'library-state-invalid');
assert.deepEqual(await missing.finished, { code: 0, stderr: '' });
});
test('C2-04 lock: an unawaited scoped read drains before the lock is released', { timeout: 10000 }, async t => {
const f = await fixture(t); let scoped, pending, complete = false;
await api.withLibraryLock(f.mirror, lock => {
scoped = lock.readSyncState;
pending = scoped({ branch: 'main' }).then(value => { complete = true; return value; });
});
assert.equal(complete, true);
assert.deepEqual(await pending, state());
await assert.rejects(scoped({ branch: 'main' }), { code: 'library-state-invalid' });
});
+151
View File
@@ -0,0 +1,151 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { readLibrary, parseFrontMatter, LibraryError, MAX_BLOB } from '../src/index.mjs';
import { BASE, git, mapping, repo } from './helpers.mjs';
const codes = (result, path) => result.problems.filter(p => p.path === path).map(p => p.code);
const file = (result, path) => result.files.find(f => f.path === path);
test('the fixture reads clean: every resolver step, appears_in and relationships wikilinks indexed', async t => {
const r = repo(t);
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
assert.equal(result.commit, r.commit);
assert.deepEqual(result.files.map(f => [f.path, f.scope]), [
['Book One/Chapters/Chapter 01.md', 'canon'], ['Book One/Characters/Mara Vell.md', 'canon'],
['Book One/Characters/Tomas.md', 'canon'], ['Book One/Places/Harbor Town.md', 'canon'],
['Book One/Playground/What If.md', 'playground'], ['Second World/Chapters/01.md', 'canon']]);
const chapter = file(result, 'Book One/Chapters/Chapter 01.md');
assert.deepEqual(chapter.links.map(l => [l.path, l.step, l.line]), [
['Book One/Characters/Mara Vell.md', 'shortest', 2], ['Book One/Places/Harbor Town.md', 'root', 2],
['Book One/Places/Harbor Town.md', 'relative', 3], ['Book One/Characters/Tomas.md', 'vault-prefix', 3]]);
const mara = file(result, 'Book One/Characters/Mara Vell.md');
assert.deepEqual(mara.frontmatter, { type: 'character', name: 'Mara Vell', appears_in: '[[Book One/Chapters/Chapter 01]]',
relationships: ['[[Tomas]]', 'The Ferryman'] });
assert.deepEqual(mara.links.map(l => [l.field, l.path, l.line]), [
['appears_in', 'Book One/Chapters/Chapter 01.md', 4], ['relationships', 'Book One/Characters/Tomas.md', 5]]);
assert.deepEqual(file(result, 'Book One/Characters/Tomas.md').links.map(l => [l.field, l.path]),
[['appears_in', 'Book One/Chapters/Chapter 01.md']]);
assert.equal(mara.bodyLine, 9);
assert.equal(mara.lines[8][0], 9);
assert.equal(mara.type, 'character');
// Only the unmapped paths are problems: the top-level note and both .canvas files.
assert.deepEqual(result.problems.map(p => [p.code, p.path]), [
['R2', 'Book One/Board.canvas'], ['R2', 'Book One/Playground/Sketch.canvas'], ['R2', 'Inbox Note.md']]);
assert.ok(Object.isFrozen(result.files[0].links));
});
test('R1: executables, symlinks, gitlinks, blobs over 1 MiB and non-UTF-8 text are not indexed', async t => {
const r = repo(t, { ...BASE,
'Book One/Places/Run.md': { text: 'x\n', mode: 0o755 },
'Book One/Places/Link.md': { link: 'Harbor Town.md' },
'Book One/Places/Big.md': 'x'.repeat(MAX_BLOB + 1),
'Book One/Places/Edge.md': 'y'.repeat(MAX_BLOB),
'Book One/Places/Latin.md': { bytes: Buffer.from([0x63, 0x61, 0x66, 0xe9, 0x0a]) },
'Book One/Places/Nul.md': 'a\0b\n' });
git(r.repo, 'update-index', '--add', '--cacheinfo', `160000,${r.commit},Book One/Places/Sub.md`);
git(r.repo, 'commit', '-qm', 'gitlink');
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
for (const name of ['Run', 'Link', 'Big', 'Latin', 'Nul', 'Sub']) {
assert.deepEqual(codes(result, `Book One/Places/${name}.md`), ['R1'], name);
assert.equal(file(result, `Book One/Places/${name}.md`), undefined, name);
}
assert.ok(file(result, 'Book One/Places/Edge.md'), 'exactly 1 MiB is read');
});
test('R2: unmapped paths are reported and not indexed', async t => {
const r = repo(t);
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas']) {
assert.deepEqual(codes(result, path), ['R2']);
assert.equal(file(result, path), undefined);
}
});
test('R3: front matter not fully parsed stays raw, with the line where parsing stopped', async t => {
const r = repo(t, { ...BASE,
'Book One/Places/Nested.md': ['---', 'type: place', 'coords:', ' lat: 1', '---', 'Body [[Tomas]]', ''].join('\n'),
'Book One/Places/Block.md': ['---', 'summary: |', ' text', '---', ''].join('\n'),
'Book One/Places/Open.md': ['---', 'type: place', 'never closed', ''].join('\n') });
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
const nested = file(result, 'Book One/Places/Nested.md');
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Places/Nested.md'), [{ code: 'R3', path: 'Book One/Places/Nested.md', line: 4, message: 'front matter not fully parsed' }]);
assert.equal(nested.frontmatter, null);
assert.equal(nested.frontmatterRaw, 'type: place\ncoords:\n lat: 1');
assert.deepEqual(nested.links.map(l => l.path), ['Book One/Characters/Tomas.md'], 'still indexed by body');
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Block.md').line, 2);
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Open.md').line, 1);
assert.equal(file(result, 'Book One/Places/Open.md').bodyLine, 1);
});
test('front matter forms the reader accepts and refuses', () => {
const parse = text => parseFrontMatter(text.split('\n'));
assert.deepEqual(parse('---\na: plain words\nb: "quoted: yes"\nc: \'it\'\'s\'\nd: [x, "y z", 3]\ne:\n- one\n- "[[Two]]"\nf: [[Wiki]]\ng: true\nh:\n---').frontmatter,
{ a: 'plain words', b: 'quoted: yes', c: "it's", d: ['x', 'y z', 3], e: ['one', '[[Two]]'], f: '[[Wiki]]', g: true, h: null });
for (const bad of ['a: {x: 1}', 'a: &anchor x', 'a: [x, [y]]', 'a: x: y', 'a: 1\na: 2', ' a: 1', 'a: "open', '- loose'])
assert.notEqual(parse(`---\n${bad}\n---`).stop, null, bad);
assert.deepEqual(parse('no front matter'), { frontmatter: null, raw: null, bodyLine: 1, stop: null });
});
test('R4: a link to no file, to more than one, or into another repo is reported and not followed', async t => {
const r = repo(t, { ...BASE,
'Book One/Research/Notes.md': 'a\n', 'Book One/Places/Notes.md': 'b\n',
'Book One/Chapters/Links.md': ['[[Nobody]]', '[[Notes]]', '[[bean-brain/Journal/Today]]', '[[Tomas]]', ''].join('\n') });
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Chapters/Links.md').map(p => [p.code, p.line]), [['R4', 1], ['R4', 2], ['R4', 3]]);
assert.deepEqual(file(result, 'Book One/Chapters/Links.md').links.map(l => l.path), ['Book One/Characters/Tomas.md']);
});
test('R5: a canon file that refers into the playground is reported, and the link still goes by path', async t => {
const r = repo(t, { ...BASE, 'Book One/Places/Dock.md': 'See [[What If]].\n' });
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
assert.deepEqual(codes(result, 'Book One/Places/Dock.md'), ['R5']);
assert.deepEqual(file(result, 'Book One/Places/Dock.md').links.map(l => l.path), ['Book One/Playground/What If.md']);
// The playground file links to canon: no problem, R5 is only about canon.
assert.deepEqual(codes(result, 'Book One/Playground/What If.md'), []);
});
test('R6: a canon file that refers into another universe is reported; a hybrid may refer into its sources', async t => {
const r = repo(t, { ...BASE, 'Book One/Places/Far.md': 'Like [[Second World/Chapters/01]].\n',
'Crossover/Chapters/A.md': 'From [[Mara Vell]] and [[Second World/Chapters/01]].\n' });
const hybrid = mapping(e => { e.universes.cross = { kind: 'hybrid', name: 'Cross', sources: ['harbor', 'second'], books: ['Crossover'] }; return e; });
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: hybrid });
assert.deepEqual(codes(result, 'Book One/Places/Far.md'), ['R6']);
assert.deepEqual(file(result, 'Book One/Places/Far.md').links.map(l => l.path), ['Second World/Chapters/01.md']);
assert.deepEqual(codes(result, 'Crossover/Chapters/A.md'), []);
});
test('R7: two paths that differ only in case are both left out', async t => {
const r = repo(t, BASE);
writeFileSync(join(r.root, 'twin'), 'twin\n');
const blob = git(r.repo, 'hash-object', '-w', join(r.root, 'twin'));
git(r.repo, 'update-index', '--add', '--cacheinfo', `100644,${blob},Book One/Characters/tomas.md`);
git(r.repo, 'commit', '-qm', 'case twin');
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
for (const path of ['Book One/Characters/Tomas.md', 'Book One/Characters/tomas.md']) {
assert.deepEqual(codes(result, path), ['R7'], path);
assert.equal(file(result, path), undefined);
}
});
test('the reader reads the commit, never the working tree, and refuses a bad commit id', async t => {
const r = repo(t);
writeFileSync(join(r.repo, 'Book One/Places/Harbor Town.md'), 'unsaved [[Nobody]]\n');
writeFileSync(join(r.repo, 'Book One/Places/Untracked.md'), 'x\n');
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
assert.equal(file(result, 'Book One/Places/Harbor Town.md').lines[0][1], '---');
assert.equal(file(result, 'Book One/Places/Untracked.md'), undefined);
for (const commit of ['HEAD', r.commit.slice(0, 12), '0'.repeat(40), git(r.repo, 'rev-parse', 'HEAD^{tree}')])
await assert.rejects(readLibrary({ repo: r.repo, commit, mapping: mapping() }), e => e instanceof LibraryError, commit);
await assert.rejects(readLibrary({ repo: r.repo, commit: r.commit, mapping: {} }), /library-mapping-invalid/);
});
test('a bare mirror reads the same as the clone', async t => {
const r = repo(t);
const bare = join(r.root, 'mirror.git');
git(r.root, 'clone', '-q', '--bare', r.repo, bare);
const a = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
const b = await readLibrary({ repo: bare, commit: r.commit, mapping: mapping() });
assert.deepEqual(b, a);
});
+80
View File
@@ -0,0 +1,80 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { renderItems, UNREADABLE } from '../src/index.mjs';
const A = 'a'.repeat(40), B = 'b'.repeat(40);
const cite = (path, lines, quote, commit = A) => ({ path, commit, lines, quote });
const options = {
scopeOf: path => path.includes('/Playground/') ? 'playground' : path.endsWith('.md') ? 'canon' : undefined,
changed: ({ path, commit }) => path === 'Book One/Characters/Mara Vell.md' && commit === A,
};
const render = items => renderItems(JSON.stringify({ items }), options);
// One snapshot per item kind: the plain text both channels show.
const SNAPSHOTS = {
fact: [{ kind: 'fact', text: 'Tomas is the ferryman.', cites: [cite('Book One/Characters/Tomas.md', [3, 3], 'the ferryman', B)] }],
finding: [{ kind: 'finding', text: 'Eye colour differs.', cites: [cite('Book One/Chapters/Chapter 01.md', [41, 41], 'her grey eyes'),
cite('Book One/Characters/Mara Vell.md', [9, 10], 'green eyes')] }],
correction: [{ kind: 'correction', text: 'Chapter 1 says grey.', from: 'green eyes', to: 'grey eyes',
cites: [cite('Book One/Characters/Mara Vell.md', [9, 9], 'green eyes', B)] }],
question: [{ kind: 'question', text: 'Is the harbor north of the town?', cites: [cite('Book One/Places/Harbor Town.md', [2, 4], 'the harbor', B)] }],
idea: [{ kind: 'idea', text: 'A storm could close the harbor.', cites: [cite('Book One/Playground/What If.md', [5, 5], 'a storm', B),
{ url: 'https://example.invalid/tides', quote: 'spring tides' }] }],
};
const EXPECTED = {
fact: '1. Fact: Tomas is the ferryman.\n [canon] Book One/Characters/Tomas.md:3 @bbbbbbb "the ferryman"',
finding: '1. Finding: Eye colour differs.\n [canon] Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
+ ' [canon] Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes" (changed since)',
correction: '1. Correction: Chapter 1 says grey.\n From: "green eyes"\n To: "grey eyes"\n'
+ ' [canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes"',
question: '1. Question: Is the harbor north of the town?\n [canon] Book One/Places/Harbor Town.md:2-4 @bbbbbbb "the harbor"',
idea: '1. Idea: A storm could close the harbor.\n [playground] Book One/Playground/What If.md:5 @bbbbbbb "a storm"\n'
+ ' [web] https://example.invalid/tides "spring tides"',
};
for (const kind of Object.keys(SNAPSHOTS)) {
test(`renderItems snapshot: ${kind}`, () => {
const out = render(SNAPSHOTS[kind]);
assert.equal(out.kind, 'items');
assert.equal(out.text, EXPECTED[kind]);
assert.equal(out.items[0].kind, kind);
});
}
test('renderItems numbers several items and takes a parsed body as well as a string', () => {
const items = [...SNAPSHOTS.fact, ...SNAPSHOTS.question];
const out = renderItems({ items }, options);
assert.equal(out.text, `${EXPECTED.fact}\n${EXPECTED.question.replace(/^1\./, '2.')}`);
assert.deepEqual(out.items[1].cites[0], { badge: 'canon', path: 'Book One/Places/Harbor Town.md', commit: B, lines: [2, 4],
ref: 'Book One/Places/Harbor Town.md:2-4 @bbbbbbb', quote: 'the harbor', changed: false });
});
test('without options a path cite has no badge and is never marked changed', () => {
assert.equal(renderItems({ items: SNAPSHOTS.finding }).text,
'1. Finding: Eye colour differs.\n Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
+ ' Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes"');
});
const NOTICE_SNAPSHOTS = { 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' };
for (const [notice, text] of Object.entries(NOTICE_SNAPSHOTS)) {
test(`renderItems snapshot: notice ${notice}`, () => {
assert.deepEqual(renderItems(JSON.stringify({ items: [], notice }), options), { kind: 'notice', notice, text });
});
}
test('a body without the reply shape shows a fixed line and none of its text', () => {
const secret = 'free prose the guard would refuse';
const bodies = [secret, JSON.stringify(secret), '{"items": [], "notice": "other"}', JSON.stringify({ items: [], notice: 'no-answer', extra: secret }),
JSON.stringify({ items: [] }), JSON.stringify({ items: [{ kind: 'story', text: secret, cites: [] }] }),
JSON.stringify({ items: [{ kind: 'fact', text: secret }] }), JSON.stringify({ items: SNAPSHOTS.fact, prose: secret }),
JSON.stringify({ items: [{ kind: 'correction', text: secret, cites: [] }] }),
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [cite('a.md', [2, 1], 'q')] }] }),
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ path: 'a.md', commit: 'abc', lines: [1, 1], quote: 'q' }] }] }),
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ url: 'https://x.invalid', path: 'a.md', quote: 'q' }] }] }),
null, 42, '[]'];
for (const body of bodies) {
const out = renderItems(body, options);
assert.deepEqual(out, { kind: 'unreadable', text: UNREADABLE }, String(body));
assert.ok(!JSON.stringify(out).includes(secret));
}
});
+82
View File
@@ -0,0 +1,82 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { spawnSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { surveyMain } from '../src/index.mjs';
import { BASE, FIXTURES, MARKER, repo } from './helpers.mjs';
const ROOT = fileURLToPath(new URL('../../..', import.meta.url));
async function run(args) {
let stdout = '', stderr = '';
const code = await surveyMain(args, { stdout: { write: s => { stdout += s; } }, stderr: { write: s => { stderr += s; } } });
return { code, stdout, stderr };
}
// Strings from the fixture that are text, values, file names or link targets: none may be printed.
const NEVER = [MARKER, 'Grey eyes', 'The Ferryman', 'Mara Vell', 'Tomas', 'Harbor Town', 'What If', 'Inbox Note', 'Chapter 01',
'Sketch', 'Board', 'Not A Link', 'the harbor', 'Docks', 'character'];
test('the survey prints folder names, counts, key sets and link-form counts, and never text', async t => {
const r = repo(t, { ...BASE, 'Book One/Places/Odd.md': ['---', `"${MARKER} key": 1`, '---', ''].join('\n') });
const out = await run(['survey', r.repo]);
assert.equal(out.code, 0, out.stderr);
assert.equal(out.stdout, [
`commit ${r.commit}`,
'entries 10',
'folders (direct children: md canvas base other)',
' (root) 1 0 0 0',
' "Book One" 0 1 0 0',
' "Book One/Chapters" 1 0 0 0',
' "Book One/Characters" 2 0 0 0',
' "Book One/Places" 2 0 0 0',
' "Book One/Playground" 1 1 0 0',
' "Second World/Chapters" 1 0 0 0',
'markdown 8 files, 8 read, 3 without front matter, 0 with front matter not fully parsed',
'front matter key sets',
' 1 [(other key)]',
' 1 [appears_in, name, relationships, type]',
' 1 [appears_in, name, type]',
' 1 [name]',
' 1 [type]',
'body links plain 3, label 1, heading 1, embed 0, self 0, markdown 1',
'field appears_in wikilink 2, name 0',
'field relationships wikilink 1, name 1',
''].join('\n'));
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
});
test('with --mapping the survey adds scope, resolver step and problem counts', async t => {
const r = repo(t);
const out = await run(['survey', r.repo, '--mapping', join(FIXTURES, 'mapping.json')]);
assert.equal(out.code, 0, out.stderr);
const tail = out.stdout.trimEnd().split('\n').slice(-3);
assert.deepEqual(tail, [
'scopes canon 5, playground 1, unmapped 3; indexed 6',
'resolved root 2, vault-prefix 1, relative 2, shortest 3, unresolved 0',
'problems R1 0, R2 3, R3 0, R4 0, R5 0, R6 0, R7 0']);
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
});
test('survey exit codes: usage 4, bad mapping 3, unreadable repo 2', async t => {
const r = repo(t);
for (const args of [[], ['scan', r.repo], ['survey'], ['survey', r.repo, '--mapping'], ['survey', r.repo, '--other'], ['survey', r.repo, r.repo]])
assert.equal((await run(args)).code, 4, args.join(' '));
const bad = join(r.root, 'bad.json');
writeFileSync(bad, JSON.stringify({ kind: 'library' }));
const refused = await run(['survey', r.repo, '--mapping', bad]);
assert.equal(refused.code, 3);
assert.match(refused.stderr, /library-mapping-invalid/);
writeFileSync(bad, 'not json');
assert.equal((await run(['survey', r.repo, '--mapping', bad])).code, 3);
assert.equal((await run(['survey', join(r.root, 'missing')])).code, 2);
assert.equal((await run(['survey', r.root])).code, 2, 'a directory that is not a repo');
});
test('scripts/mosaic library survey reaches the survey', t => {
const r = repo(t);
const out = spawnSync(join(ROOT, 'scripts/mosaic'), ['library', 'survey', r.repo], { encoding: 'utf8', timeout: 30000 });
assert.equal(out.status, 0, out.stderr);
assert.match(out.stdout, new RegExp(`^commit ${r.commit}\n`));
assert.ok(!out.stdout.includes(MARKER));
});
+4 -1
View File
@@ -31,14 +31,17 @@ const contains = (base, child) => {
const rel = relative(base, child);
return rel === '' || (rel !== '..' && !rel.startsWith('..' + sep) && !isAbsolute(rel));
};
export function projectRepo({ business, project, dataRoot }) {
export function projectRepo({ business, project, dataRoot }, { allowLibrary = false } = {}) {
checkedId(business?.id); checkedId(project);
if (!Object.hasOwn(business?.projects ?? {}, project))
throw new ProjectError('project-not-declared', `business does not declare ${project}`);
absolute(dataRoot);
const entry = business.projects[project];
if (entry.kind === 'library' && !allowLibrary)
throw new ProjectError('project-invalid', 'library projects are not record repositories');
const repo = absolute(entry.repo ?? join(dataRoot, 'projects', business.id, project));
for (const project of Object.values(business.projects)) {
if (project.kind === 'library') continue;
const code = absolute(project.root);
if (contains(code, repo) || contains(repo, code))
throw new ProjectError('project-path', 'record repository and code checkout overlap');
+13
View File
@@ -24,6 +24,19 @@ async function ready(t, custom) {
return { c, repo: initialized.repo, commit: initialized.commit };
}
test('C2-04: record APIs refuse a library before creating a repository', async t => {
const c = context(t);
c.business.projects = { books: { kind: 'library' }, ...c.business.projects };
assert.equal(api.projectRepo(c), join(c.dataRoot, 'projects/acme/stack'));
const library = { ...c, project: 'books' };
assert.throws(() => api.projectRepo(library), { code: 'project-invalid' });
for (const author of ['jason', 'tester'])
await assert.rejects(api.initProject(library, { author }), { code: 'project-invalid' });
assert.equal(readdirSync(c.root).includes('data'), false);
c.business.projects.stack.root = c.dataRoot;
assert.throws(() => api.projectRepo(c), /overlap/);
});
test('init creates a checked versioned layout, valid templates and human-authored local history', async t => {
const { c, repo, commit } = await ready(t);
assert.equal(repo, join(c.dataRoot, 'projects/acme/stack'));
+3 -1
View File
@@ -311,7 +311,9 @@ async function update(root, dest, g) {
export async function ensureCheckouts({ workspace, projects, env = process.env, limits = GIT_LIMITS }) {
const g = (args, extra) => git(args, env, extra, limits);
const out = [];
for (const [project, { root }] of Object.entries(projects)) {
for (const [project, entry] of Object.entries(projects)) {
if (entry.kind === 'library') continue;
const { root } = entry;
const dest = join(workspace, project);
let exists = true;
try {
+9
View File
@@ -12,6 +12,15 @@ import { join } from "node:path";
import { deflateSync } from "node:zlib";
import { CONFIG_MAX_BYTES, CheckoutError, ensureCheckouts, gitEnv } from "../src/checkout.mjs";
test('C2-04: ensureCheckouts skips a library before a real clone and leaves no books path', async t => {
const w = world(t);
const result = await ensureCheckouts({ workspace: w.workspace, projects: { books: { kind: 'library' }, stack: { root: w.root } } });
assert.deepEqual(result.map(r => r.project), ['stack']);
assert.equal(result[0].outcome, 'cloned');
assert.equal(existsSync(join(w.workspace, 'books')), false);
assert.deepEqual(await ensureCheckouts({ workspace: w.workspace, projects: { books: { kind: 'library' } } }), []);
});
function tmp(t) {
const dir = mkdtempSync(join(tmpdir(), "mosaic-checkout-"));
t.after(() => {
+4 -2
View File
@@ -64,7 +64,7 @@ try {
// the system config, Business can't resolve authority and Settings has
// no notifier. A refusal printed here names no path (redactor).
// --send without a bus refuses to start rather than serve a composer that can't send.
let bus = null, send = null, settingsWrite = null, projectFiles = null, projectFile = null, projectSave = null, chat = null, nameWrite = null, assistantStop = null, instancesWrite = null, routesWrite = null, busLine, system = null;
let bus = null, send = null, settingsWrite = null, projectFiles = null, projectFile = null, libraryFile = null, libraryStatus = null, projectSave = null, chat = null, nameWrite = null, assistantStop = null, instancesWrite = null, routesWrite = null, busLine, system = null;
try {
system = loadSystem();
const { dataRoot } = system;
@@ -77,6 +77,7 @@ try {
// Row 148: the spec and docs editor reads like the other reads and saves only under --send.
projectFiles = args => call('project.files', args);
projectFile = args => call('project.file', args);
libraryFile = args => call('library.file', args); libraryStatus = args => call('library.status', args);
if (sendOn) {send = args => call('message.send', args);settingsWrite=args=>call('limits.set',args);projectSave=args=>call('project.file.save',args);nameWrite=args=>call('name.set',args);assistantStop=args=>call('assistant.stop',args);instancesWrite=args=>call('instances.set',args);routesWrite=args=>call('routes.set',args);}
busLine = `Bus: ${options.business ?? 'the running bus host\'s business'}`;
} catch (err) {
@@ -97,7 +98,7 @@ try {
}
let server;
try {
server = await startServer({ board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite });
server = await startServer({ board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, libraryFile, libraryStatus, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite });
} catch (err) { sso?.close(); throw err; }
let testerServer = null;
if (testerPort !== null) {
@@ -107,6 +108,7 @@ try {
testerServer = await startServer({ host: '127.0.0.1', port: testerPort, board: options.board,
tester: true, testerNoSignIn: ssoOn, testerProofHeader: proof?.header ?? null, testerAuthHeader: options.preview?.header ?? null, bus: testerBus, reads,
projectFiles: args => call('project.files', args), projectFile: args => call('project.file', args),
libraryFile: args => call('library.file', args), libraryStatus: args => call('library.status', args),
instancesWrite: args => call('instances.set', args), routesWrite: args => call('routes.set', args), nameWrite: args => call('name.set', args),
// Row 177 (#1675): the tester's own chat, through the tester CLI; 7332's stays the human's.
chat: createChat({ call }),
+34
View File
@@ -0,0 +1,34 @@
// C2-04 only wires callbacks. HTTP routes and pages are later rows.
import test from 'node:test';
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { tmp, fixture } from '../../cli/tests/helpers.mjs';
import { HUMAN_CLI, TESTER_CLI } from '../../cli/src/transport.mjs';
test('C2-04 CLI: both listener callback pairs dispatch library verbs with their own transport', t => {
const root = tmp(t, 'library-cli-'), f = fixture(root), log = join(root, 'calls.jsonl');
writeFileSync(log, '');
const hook = join(root, 'hooks.mjs');
// The loader supplies a fixture bus and captures callbacks instead of
// opening an HTTP listener. Production imports and routes are unchanged.
const serveSource = "export const DEFAULT_BOARD='http://127.0.0.1:9';export const previewOptions=()=>null;export const proofOptions=()=>null;export async function startServer(o){if(typeof o.libraryFile!=='function'||typeof o.libraryStatus!=='function')throw Error('missing library callbacks');await o.libraryFile({project:'books',path:'Book One/A.md'});await o.libraryStatus({project:'books'});return {address:()=>({port:0}),close:()=>{}};}";
const busSource = `import {appendFileSync} from 'node:fs';export class BusReadError extends Error{};export const busCaller=o=>async(verb,args)=>{appendFileSync(${JSON.stringify(log)},JSON.stringify({verb,args,cli:o.cli??${JSON.stringify(HUMAN_CLI)}})+'\\n');return {};};`;
writeFileSync(hook, `import {registerHooks} from 'node:module';
registerHooks({load(url,context,next){
if(url.endsWith('/webui/src/serve.mjs'))return {format:'module',shortCircuit:true,source:${JSON.stringify(serveSource)}};
if(url.endsWith('/webui/src/bus.mjs'))return {format:'module',shortCircuit:true,source:${JSON.stringify(busSource)}};
return next(url,context);
}});`);
const result = spawnSync(process.execPath, [new URL('../src/cli.mjs', import.meta.url).pathname,
'serve', '--business', 'acme', '--port', '0', '--tester-port', '7444'],
{ env: { ...f.env, NODE_OPTIONS: `--import=${hook}` }, encoding: 'utf8', timeout: 10000 });
assert.equal(result.status, 0, result.stdout + result.stderr);
const calls = readFileSync(log, 'utf8').trim().split('\n').filter(Boolean).map(JSON.parse);
assert.deepEqual(calls.map(c => [c.verb, c.cli]), [
['library.file', HUMAN_CLI], ['library.status', HUMAN_CLI],
['library.file', TESTER_CLI], ['library.status', TESTER_CLI],
]);
assert.equal(calls[0].args.path, 'Book One/A.md');
});
+6
View File
@@ -34,6 +34,12 @@ if [ "${1:-}" = business ]; then
shift
exec node "$REPO/packages/business/src/cli.mjs" "$@"
fi
# `mosaic library survey`: names and counts from a writing repo, never its text
# (packages/library/README.md). Other library verbs are bus verbs.
if [ "${1:-}" = library ] && [ "${2:-}" = survey ]; then
shift
exec node "$REPO/packages/library/src/survey.mjs" "$@"
fi
case "${1:-}" in
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|hosts|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
esac
+1 -1
View File
@@ -33,7 +33,7 @@ COPY_PATHS=(
packages/conversation/src packages/discord/src packages/discord/bin packages/discord/extension packages/harness/src
packages/seat/src packages/tasks/src packages/tasks/fields.json packages/tasks/requirements.json
packages/webui/src packages/webui/README.md packages/cli/README.md
packages/queue/src
packages/queue/src packages/library/src
# Whole directory: every in-repo plug-in module ships with the copy (row 131).
packages/plugins
)