Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b6eec16839 | ||
|
|
2522025afc | ||
|
|
6a508804f0 | ||
|
|
a2c72ac952 |
No files matched your search
@@ -5176,3 +5176,4 @@ Before: managed project/thread verbs had no Console conversation area. After: fi
|
||||
2026-10-20 05:10:34 UTC | Row 178 #1676 round 2 verified | Before: C1 render refusals lacked killing tests and row 181 base was pending. Rebased onto 931651f2d, kept all COUNTED/count code and tests, appended T178 tail; added independent owner/collision/absolute/charset/header and empty-loopback proof tests. Four files singly 156/156, all 24 mutants killed, metadata and sandbox fixture probes pass; scratch sweep empty. Packet agents/rocko/work/queue-178/r2/, candidate 9300a670, inventory d0dcde7b (59/59), scan 58 files no findings; in-review revs 2166-2167, expected no-token request failure. No live actions or commits.
|
||||
2026-10-21T01:28:05Z | Darkwing | row 232 #1734 installer I1 build, round 1 in review | Before: stack-release.sh managed only user units (systemctl --user), and stack-copy.sh had no bundle copy. After: --units external (adopt, activate, rollback --to, status, confirm, revert, gate --since, AWAITING-UNITS, stale-start, external-pm, --plan --expect-from) and stack-copy check --units external, copy --bundle FILE SHA, wrapper ID --create; user mode byte-identical to the base by test. First build 0af0c73c6 on 00d4cc1f6; rebased onto row 200 (cb5fdb064) as a0ffabc3f on darkwing/row232-r1; --sso refused under external (D9), row 200 config_checks run under external (D10). Gate on Sage's slot (sage-1618): cli 561/0, every scripts/test-*.sh green; 17/17 mutants killed. Results commit 84f1a91e6 (pushed, seat branch only). Manifest 7a3b4ea1 (9 paths), queue revs 2595-2596, request comment 29825, reviewer Filbert. Packet agents/darkwing/work/queue-232/BUILD.md.
|
||||
2026-10-21T04:50:44Z | Darkwing | row 233 #1735 installer I3 build, round 1 in review | Before: root had no upgrade driver for root-owned units and no host run sheet; stack-release.sh had no rehearse --backup last and no external plan refusals. After: scripts/stack-upgrade.sh upgrade and rollback (r6 A1-A5, A2f, R1-R4) with 20 tests, docs/guides/host-run-sheet.md, TOOLS.md lines, and on Sage's rulings stack-release.sh --backup last, BACKUP_STORE and the --plan refusals open-activation and migrate-required (3 tests). Base e65a3ab4d (row 232's landing). Gate on Sage's slot (sage-1704) on d99166ade: cli 616/0, every scripts/test-*.sh green, 1201/0 summed by awk; m1-m8 8/8 killed. Correction: the process sweep after the gate found an orphaned sleep 60 from the lock test's flock holder (N1); 08db6ac3f fixes it test only, Sage ruled the gate stands (sage-1707), rerun c0 20/0 and m1 killed with 0 left (sage-1708). Tip 1a1dc80ca (pushed, seat branch only). Manifest 464ce9eb (7 paths), queue revs 2644 and 2647, request comment 29901, reviewer Filbert. Packet agents/darkwing/work/queue-233/BUILD.md.
|
||||
2026-10-21T09:00Z | Herb | row 215 C2-04 implementation started | Before: business files reject library entries and root consumers assume code projects; the organization writer can change all project mappings. Plan: approved manifest 2522025af on C2-01 a2c72ac95, Filbert reply to herb-0056, scope sage-1848/1852/1860. Focused slot sage-1869 observed expected red mapping and M9 cases, then 14/0 green for new T1/T2 cases, no skips; clear herb-0062 with empty test/fixture process scan. After that clear, T3/T4 tests and host-only mirror/bus implementation continue under sage-1874, unrun while Sage holds the slot. Full singles, suites and implementation review remain pending. No source commit, push or live change.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Row 212 build notes, C2-01 packages/library (#1713, Vikunja #416)
|
||||
|
||||
## Rounds
|
||||
|
||||
- Round 1: bdf18272a on gosalyn/c2-01, base 32b54ac3b. Webby approved it in Gitea comment 29704.
|
||||
- Round 2: d3c7bbf839d9c3bf50f154b18e7e4472fbd005ed on gosalyn/c2-01-r2, rebased onto 9b5362018 after row 232 drifted three paths (sage-1714). Webby approved it at merge scope, rev 2666 against comment 29921 (webby-0052).
|
||||
- Round 3: this commit on gosalyn/c2-01-r3, one commit on the row 234 landing c771a36f4 (sage-1736, sage-1779, mos-1160). Row 234 changed scripts/mosaic, so round 3 review covers that one file.
|
||||
|
||||
## Round 2 merge
|
||||
|
||||
range-diff 32b54ac3b..bdf18272a against 9b5362018..d3c7bbf83 shows one change, the context line where upstream added `credentials` to the scripts/mosaic case line. Patch-id moved from 8f80dfa3 to 175f45c6 for that reason only. All three drifted hunks keep both sides:
|
||||
|
||||
- scripts/stack-copy.sh COPY_PATHS reads `packages/queue/src packages/library/src`. Row 232's bundle, --create, --units and instance code is unchanged.
|
||||
- scripts/mosaic runs the `library survey` exec block before the case line, and the case line keeps `credentials`.
|
||||
- packages/cli/tests/stack-copy.test.mjs adds `packages/library/src/index.mjs` to the copy file list.
|
||||
|
||||
## Round 2 singles, evidence for the 19 paths other than scripts/mosaic
|
||||
|
||||
Run on d3c7bbf83 in Sage's slot (sage-1723) from 2026-10-21T05:00:09Z to 05:11:29Z. The HEAD and a clean tree were checked first. The counts are an awk sum over the 23 logs: 481 passed, 0 failed, 0 skipped, 0 cancelled, and every file exited rc=0.
|
||||
|
||||
| Package | File | Pass |
|
||||
|---|---|---|
|
||||
| library | links | 6 |
|
||||
| library | mapping | 7 |
|
||||
| library | reader | 11 |
|
||||
| library | render | 11 |
|
||||
| library | survey | 4 |
|
||||
| cli | app | 30 |
|
||||
| cli | case2-unit | 30 |
|
||||
| cli | credentials | 5 |
|
||||
| cli | host | 42 |
|
||||
| cli | instances | 5 |
|
||||
| cli | project | 4 |
|
||||
| cli | stack-copy | 16 |
|
||||
| cli | stack-instances | 23 |
|
||||
| cli | stack-release-external | 25 |
|
||||
| cli | stack-release | 192 |
|
||||
| cli | tester-reads | 6 |
|
||||
| cli | tester-threads | 5 |
|
||||
| cli | usage | 2 |
|
||||
| queue | dispatch | 4 |
|
||||
| runs | task-cli | 9 |
|
||||
| webui | pinned-copy | 10 |
|
||||
| webui | service | 31 |
|
||||
| webui | views | 3 |
|
||||
|
||||
The file list comes from the library tests plus every test that names scripts/mosaic or stack-copy.sh, by basename and by join() path. No 64-hex pin in the tests names a merged path.
|
||||
|
||||
## Round 3 scripts/mosaic
|
||||
|
||||
The only path both row 234 and this row change is scripts/mosaic. Row 234 added `hosts` to the cli case line and changed nothing else in the file. Round 3 takes the landed blob cce1fdee4 and adds this row's six-line `library survey` block before the case line by hand, so `hosts` stays. The mode stays 100755. The other 19 paths are byte-identical to d3c7bbf83. Against d3c7bbf83 the only difference is the `hosts` context line.
|
||||
|
||||
Round 3 runs only the tests that read scripts/mosaic, in Sage's slot. Their counts go to Sage in T3 and onto #1713, since this file can't hold them and stay one commit. Those tests are library survey, cli app, case2-unit, credentials, hosts, host, instances, project, stack-copy, stack-release, stack-release-external, stack-upgrade, tester-reads, tester-threads and usage, queue dispatch, runs task-cli, webui pinned-copy, service and views, plus scripts/test-queue.sh. hosts.test.mjs is new with row 234.
|
||||
@@ -0,0 +1,271 @@
|
||||
# Row 215 build record
|
||||
|
||||
Owner: Herb. Reviewer: Filbert. Claim: herb-215-start, rev 2753.
|
||||
Brief: docs/plans/2026-10-20_case2-rows.md, C2-04, pin
|
||||
16847e94ae1768881711318fe81cbb5e6c66b971. Design: Gosalyn r11,
|
||||
sections 1.1, 1.3, 2.1, M4, M9 and 7.2.
|
||||
|
||||
Base: a2c72ac952df94ecab7e391d5042f49b7f88b816, row 212's reviewed
|
||||
candidate, under Q177. Branch: herb/row215. Rebuild on the combined 212+197
|
||||
landing when Sage names it. Do not replay or revert landed changes.
|
||||
|
||||
## Scope rulings (sage-1848)
|
||||
|
||||
1. Add packages/bus/src/runtime.mjs for library adapter injection and the
|
||||
library skip in protectedRoots. The skip has its own consumer test. Change
|
||||
nothing else there.
|
||||
2. Add packages/library/src/mirror.mjs, named mirror rather than store. It
|
||||
creates a bare mirror without a remote, reads at the served commit and
|
||||
reads sync state. Export one state filename constant, and export this
|
||||
module through packages/library/src/index.mjs. Row 222 imports the constant.
|
||||
3. Add packages/bus/src/server.mjs only to await library.file and
|
||||
library.status, with socket tests proving their replies are results, not {}.
|
||||
4. HTTP exposure is deferred. Row 225 owns the library.status route;
|
||||
C2-11b owns library pages. Leave serve.mjs alone. webui/src/cli.mjs gets
|
||||
only callback wiring next to the project-file callbacks on both listeners.
|
||||
5. Reuse existing refusal codes where possible.
|
||||
6. Build on a2c72ac95 until Sage supplies the 212 landing SHA.
|
||||
|
||||
Additional ruling, sage-1852: library-state-invalid is approved for malformed,
|
||||
unsafe or mismatched state, including a missing file while the served ref
|
||||
exists. The version-1 shared contract below is approved. Row 222 also owns
|
||||
runtime.mjs for its sync and timer wiring, recorded in that row's manifest.
|
||||
|
||||
Additional rulings, sage-1860, after Filbert's CHANGES on 6a508804f:
|
||||
|
||||
- B1 adds packages/cli/src/config.mjs and cli/tests/trackers-boot.test.mjs.
|
||||
trackerFor skips library entries. A real bus boot with tracker.baseUrl and
|
||||
a library entry succeeds and gets no tracker from that entry.
|
||||
- B2 refuses a role publish grant naming a library project in the loader.
|
||||
- B3 readers use the sync lock. mirror.mjs exports withLibraryLock, with a
|
||||
5000 ms wait bound and library-sync-busy on timeout. Sync locks only the
|
||||
served-ref move and state publication, never the fetch.
|
||||
- N3, git-socket doPush/recordPush accepting a library ID, is a later row.
|
||||
Sage records it in DEFERRED.md when 215 lands. No fix here.
|
||||
|
||||
## Shared contract with row 222
|
||||
|
||||
Agreed in herb-0048, megavolt-0081/0082 and sage-1852. This section is the
|
||||
contract Filbert reviews and Megavolt cites for Dewey. It is not a sync writer.
|
||||
|
||||
Module: packages/library/src/mirror.mjs. Export
|
||||
`SYNC_STATE_FILE = 'mosaic-library-sync.json'`. The full path is
|
||||
`join(mirror, SYNC_STATE_FILE)`, inside the bare repository. Only sync.mjs
|
||||
writes it, using a sibling temp file, fsync, rename, and directory fsync.
|
||||
|
||||
Version 1 has exactly these keys:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"branch": "main",
|
||||
"served": null,
|
||||
"lastFetch": null,
|
||||
"lastError": null,
|
||||
"diverged": null,
|
||||
"problems": []
|
||||
}
|
||||
```
|
||||
|
||||
- branch is the configured mapping branch and must match the reader's branch.
|
||||
- served is null or a full lowercase 40-hex commit ID. refs/heads/served is
|
||||
authority. The state value must equal that ref, which must name a commit.
|
||||
- lastFetch is null or a canonical ISO UTC timestamp.
|
||||
- lastError is null or exactly {code, at}. code matches [a-z-]{1,64}; at is
|
||||
canonical ISO UTC. It never holds stderr, prose or a credential value.
|
||||
- diverged is null or exactly {commit, at}, with the same commit/time rules.
|
||||
- problems uses C2-01's exact {code, path, message, line?}. code is R1-R7,
|
||||
message is that code's fixed RULES text, line is a positive safe integer
|
||||
when present. No file body or arbitrary message is accepted.
|
||||
- An absent file and absent served ref return the empty object above with
|
||||
the configured branch. An absent file with an existing served ref refuses.
|
||||
- Unknown keys, bad types, unsafe filesystem metadata, invalid JSON and a
|
||||
mismatch refuse library-state-invalid. Reads use a bounded regular owned
|
||||
private file without following links. The read never repairs state.
|
||||
- refs/remotes/library/<branch> is not duplicated in this file. Row 222 may
|
||||
read it when it needs the fetched tip.
|
||||
|
||||
Exports consumed by row 222:
|
||||
|
||||
| Export | Contract |
|
||||
| --- | --- |
|
||||
| mirrorPath({business, project, dataRoot}) | Require a library entry; return projectRepo's checked path |
|
||||
| ensureMirror(context) | Promise of the mirror path; race-safe idempotent bare init; no remote or fetch |
|
||||
| servedCommit(mirror) | Promise of full commit ID or null; refs/heads/served only |
|
||||
| readSyncState(mirror, {branch}) | Promise of checked version-1 state; compare with served ref |
|
||||
| withLibraryLock(mirror, operation) | Await the same exclusive lock as readers, then await operation({readSyncState}); always release |
|
||||
|
||||
The lock is a stable private regular file, mosaic-library-sync.lock inside
|
||||
the mirror. It is never unlinked. A kernel flock is held by the caller's
|
||||
file descriptor. Acquisition waits at most 5000 ms and refuses
|
||||
library-sync-busy on timeout. Unsafe lock metadata refuses library-state-invalid.
|
||||
|
||||
There is no fs.flock API. The exact protocol is: open the stable file with
|
||||
O_NOFOLLOW, check regular/private/owned/single-link metadata, pass its fd as
|
||||
child FD 3 to `flock --exclusive --wait 5 --conflict-exit-code 75 3` using
|
||||
asynchronous spawn, await
|
||||
that child's exit 0, then run the callback while the parent still holds its
|
||||
original descriptor. Both descriptors share the open file description, so
|
||||
the lock survives acquisition-child exit. Finally close the parent's fd.
|
||||
Process death also closes it. No holder child stays alive in the callback,
|
||||
no stale file is deleted, and unrelated operation children inherit only 0-2.
|
||||
Acquisition never uses spawnSync: the bus must serve other calls while a
|
||||
library reader waits (megavolt-0087).
|
||||
|
||||
A scratch-only probe on node v26.8.1 at 2026-10-21T08:34:00Z confirmed this:
|
||||
acquisition child exited 0; a second flock -n returned 1 while the parent fd
|
||||
stayed open; after closeSync it returned 0. The probe removed its own root.
|
||||
This was a design probe, not a package test, suite, Docker command or proof.
|
||||
|
||||
readSyncState takes that lock around reading the served ref and state file.
|
||||
The callback-scoped readSyncState({branch}) supplied by withLibraryLock reads
|
||||
under the already-held lock and does not reacquire it. The helper awaits
|
||||
asynchronous callbacks and releases in finally. A callback-scoped reader
|
||||
used after its callback ends refuses; it cannot escape its lock lifetime.
|
||||
|
||||
Row 222 fetches without this lock, then calls withLibraryLock across the
|
||||
served-ref move and atomic state write. Its tests hold publication between
|
||||
those steps and prove a concurrent reader waits, then sees the new consistent
|
||||
pair, including first sync. Row 215 tests that publication shape and timeout.
|
||||
A file read releases the lock after snapshotting a consistent served SHA,
|
||||
then reads that immutable commit. Row 222's separate outer sync lock spans
|
||||
fetch to prevent two sync operations interleaving (megavolt-0086).
|
||||
|
||||
Host adapter exports for this row:
|
||||
|
||||
- libraryStatus(context): return {project, ...state}, with no repository path
|
||||
or content. Do not fetch or rebuild on this read.
|
||||
- readLibraryFile({...context, path}): read a mapped, readable file at the
|
||||
served commit, returning {project, path, commit, content}. No caller-picked
|
||||
commit, working-tree fallback, fetch or write. C2-01's R1/R2/R7 exclusions
|
||||
apply. A never-synced mirror refuses library-commit for a file read.
|
||||
|
||||
Row 222 owns syncCommand, sync, writeSyncState, its outer sync lock, the timer
|
||||
and state publication. Row 215 owns the shared publication/read lock per
|
||||
sage-1860. Sync rebuilds problems with existing readLibrary({repo, commit,
|
||||
mapping}). It does not need a second rebuild function from this row.
|
||||
|
||||
## Root consumer audit and reviewer notes
|
||||
|
||||
On a2c72ac95, ran exactly the sage-1860 audit:
|
||||
`git grep -n 'entry\.root\|\.root)' -- 'packages/*/src/*.mjs'`.
|
||||
It has 14 hits. runtime.mjs:95, business/cli.mjs:61,72,78,118,
|
||||
instance-settings.mjs:142-143, resolve.mjs:44, cli/config.mjs:62-63 and
|
||||
project/paths.mjs:42 all get the library guard in this row. business.mjs:233
|
||||
and :242 are code-project validation after the library dispatch, not an
|
||||
unguarded consumer. queue.mjs:1144 compares queue genesis roots and never
|
||||
iterates business.projects; it is safe and unchanged. A second search for
|
||||
projectFilePath/business.projects/b.projects found no additional root reader.
|
||||
ensureCheckouts destructures root rather than spelling .root; its guard and
|
||||
the launcher's filter remain in T1. Bare projectFilePath/loadProject remain
|
||||
strict and unchanged.
|
||||
|
||||
Filbert N1 is handled at the existing owned paths.mjs boundary: projectRepo
|
||||
defaults to record-project use and refuses a library entry with project-invalid.
|
||||
Only mirrorPath's internal call opts into allowLibrary. Thus ordinary human
|
||||
and tester project init cannot occupy an uninitialized library mirror path.
|
||||
No record API or bus argument exposes that internal option.
|
||||
|
||||
Filbert N2: ensureMirror failure refuses bus boot. No socket becomes ready.
|
||||
The runtime closes resources it opened, following its existing failure path.
|
||||
A T4 test uses an unsafe/pre-existing mirror and proves the refusal, rather
|
||||
than serving a half-configured library adapter.
|
||||
|
||||
## Verification record
|
||||
|
||||
Planning stage. No product code, package test, suite, image build, live operation
|
||||
or push for row 215 has happened. Product implementation awaits Filbert's
|
||||
manifest approval. Sage owns suite and full-gate grants. Row 197's proof
|
||||
commit is separate and is not part of this row's changes.
|
||||
|
||||
## Implementation ledger
|
||||
|
||||
Plan: agents/herb/work/queue-215/MANIFEST.md at 2522025af, approved by
|
||||
Filbert in the reply to herb-0056. Execution is inline by Herb under the
|
||||
executing-plans and test-driven-development skills. No agent delegation.
|
||||
|
||||
Pre-flight: T1's validated mapping feeds T2's protected document and T3's
|
||||
mirror context; T3's lock/state helpers feed T4's injected host adapter.
|
||||
The shared contract is pinned above, with no interface conflict found.
|
||||
|
||||
Sage-1867 overrides COMMON's singles allowance: every test and executable
|
||||
probe needs an explicit slot grant. No further probe/test has run since
|
||||
that reminder. Read-only inspection and test authoring continue while the
|
||||
slot is held. The first red execution is pending; product code is unchanged.
|
||||
|
||||
Filbert's approved-manifest build notes: ensureMirror creates the lock in
|
||||
its staged directory before publication; a missing lock in an existing
|
||||
mirror refuses library-state-invalid. The top-level metadata allowlist
|
||||
permits only normal bare Git entries plus the state and stable lock files.
|
||||
Tests cover an unknown entry. The host needs /usr/bin/flock; ENOENT yields
|
||||
a handled library-state-invalid refusal, never an unhandled child error.
|
||||
Lock acquisition uses /usr/bin/flock directly and asynchronous spawn.
|
||||
|
||||
T1: consumer and loader regression tests authored, not run. T2: direct M9
|
||||
write and serialization regression tests authored, not run. No completion
|
||||
or passing claim is made for either task.
|
||||
|
||||
Update, sage-1869 focused slot, 2026-10-21T08:47:48Z to 08:51:07Z:
|
||||
the busy check was CLEAR before the first test. T1's mapping regression
|
||||
failed 0/1 on unsupported kind as expected, then passed 2/0 with the publish
|
||||
refusal included. Other focused greens: business CLI 1/0, resolver 1/0,
|
||||
instance settings 1/0, seat checkout 1/0, project paths 1/0, sandboxed launcher
|
||||
1/0, and trackers-boot whole 2/0. T2's direct writer case failed 0/1 on the
|
||||
missing refusal, then its whole new file passed 4/0. Green total 14/0 with
|
||||
no skips or cancellations. Logs and counts: granted-t1/. The final /proc
|
||||
scan found no test process or fixture cwd/argv reference; its private root
|
||||
was removed. Clear sent as herb-0062. Existing files' other cases have not
|
||||
been run yet, so T1/T2's full-file acceptance is still pending.
|
||||
|
||||
Sage-1874 directs continued authoring and implementation while the landing
|
||||
gate/proof holds the slot. T3/T4 tests were written first. Their initial
|
||||
red runs cannot execute during that hold; implementation proceeds under
|
||||
that owner direction, with no claim of a witnessed red or green for T3/T4.
|
||||
Their baseline red and candidate checks need a later explicit grant.
|
||||
|
||||
Implementation ruling: initialization uses GNU /usr/bin/mv --no-clobber
|
||||
--no-target-directory for its bounded staging rename, rather than Node's
|
||||
renameSync, which could replace an empty directory made by a competing
|
||||
actor. This preserves pre-existing directories and needs no new package on
|
||||
the Linux/coreutils bus host. The host also needs /usr/bin/flock. Neither
|
||||
command has been executed for T3/T4 yet. Publication errors fail closed.
|
||||
Sync-state reads are capped at 8 MiB, including a checked extra byte; an
|
||||
oversized or concurrently changed file refuses library-state-invalid.
|
||||
|
||||
T5 static dependency audit found two partial pinned-copy fixtures that copy
|
||||
business/src but not its newly imported library mapping validator:
|
||||
cli/tests/stack-release.test.mjs:1772 and stack-release-fixture.mjs:309 on
|
||||
the base. Each now includes packages/library/src in the existing copy list.
|
||||
This changes fixture closure only, no assertion or timeout. The complete
|
||||
stack-release file and each test file importing that shared fixture require
|
||||
a later granted single-file run. They are outside the four-file T3/T4 slot.
|
||||
|
||||
Sage-1879 places the four-file T3/T4 focused run after their 212+197 proof,
|
||||
before Gizmoduck, and requires their explicit clear before starting. No run
|
||||
has occurred after herb-0062. Row 197's N7 base-image-tag finding is left to
|
||||
Sage's DEFERRED entry; no row 197 source change is made here.
|
||||
|
||||
|
||||
Focused T3/T4 grant sage-1882, 2026-10-21 09:13-09:18Z: busy check
|
||||
CLEAR. Final whole-file results: mirror 16/0, bus library 6/0, WebUI
|
||||
library-cli 1/0, project-runtime 3/0, all with zero skips/cancellations.
|
||||
Syntax checks passed for the four files and five changed host modules.
|
||||
Logs, all intermediate failures, counts and empty /proc cleanup scan are
|
||||
in granted-t34/. Baseline a2c72ac95: three focused bus cases 0/3
|
||||
(unknown-verb), CLI 0/1 (missing callbacks). Mirror baseline 0/1 is an
|
||||
import refusal for the new module, not a witnessed behavioral red.
|
||||
An initial whole baseline bus attempt left a runtime after its expected
|
||||
failure; its exact runner was stopped, no process remained, and the
|
||||
baseline was narrowed to the three bounded broker/socket cases.
|
||||
Candidate fixture setup errors and a mock-child timeout were corrected
|
||||
before the final whole-file passes. No suite or Docker/image/proof ran.
|
||||
|
||||
Filbert's 09:09 publication note: no dependence on mv -n's skipped-rename
|
||||
exit status across coreutils versions. Save staging dev/ino before mv;
|
||||
only its disappearance plus the same dev/ino at the mirror proves a win.
|
||||
A remaining original staging inode means a loss, requiring a valid winner.
|
||||
Only that owned original staging inode is removed. Numeric no-clobber
|
||||
statuses 0 and 1, including a diagnostic, are tested with a competing
|
||||
winner. Spawn errors, signals/timeouts and inconsistent outcomes refuse.
|
||||
Coreutils must provide atomic --no-clobber --no-target-directory semantics;
|
||||
it need not use any particular skipped-rename status convention.
|
||||
@@ -0,0 +1,214 @@
|
||||
# C2-04 library business entry implementation plan
|
||||
|
||||
> Execution: Herb implements in this session using the executing-plans skill.
|
||||
> Filbert approves this manifest before any product implementation.
|
||||
|
||||
**Goal:** Load a library mapping without giving an agent a checkout, protect
|
||||
all project mappings from organization writes, and expose human-only bus
|
||||
reads of the bare mirror's served commit and sync status.
|
||||
|
||||
**Architecture:** C2-01 checks mappings and reads commits. Business consumers
|
||||
skip library entries. A host-only mirror module owns local mirror reads and
|
||||
initialization. runtime.mjs injects that module into the broker, preserving
|
||||
the broker's sandbox import closure. Sync and HTTP routes belong to later rows.
|
||||
|
||||
**Tech stack:** Existing Node ESM, node:test and bounded local Git helpers.
|
||||
No dependencies. node_modules remains the plain canonical symlink.
|
||||
|
||||
**Spec:** Pinned C2-04 brief; Gosalyn's approved design 1.1, 1.3, 2.1,
|
||||
M4/M9 and 7.2; BUILD.md's sage-1848/1852/1860 scope and shared-state contract.
|
||||
|
||||
## Constraints and review focus
|
||||
|
||||
- Base a2c72ac95; rebase only onto Sage's named 212 landing on a new branch.
|
||||
- Existing code-project entries retain root/repo/remote behavior. Library
|
||||
entries have no root or repo override and use C2-01 checkMapping.
|
||||
- Preserve the broker's pinned imports. No library import in broker.mjs and
|
||||
no new runner mount. Do not edit serve.mjs or add an HTTP route.
|
||||
- No fetch, remote config, credential read, timer, sync writer, bus write or
|
||||
live operation. All tests use made-up content and local repositories.
|
||||
- Singles may run under COMMON; suites and any long run need Sage's grant.
|
||||
Stop and report two failures in one gate. Do not push without authorization.
|
||||
|
||||
Review focus, with owning tasks below: library-first iteration must not
|
||||
break a later code project (T1); serialized documents cannot bypass M9
|
||||
(T2); racing initialization must not replace another mirror (T3); stale or
|
||||
unsafe state must not leak content (T3); async socket replies must preserve
|
||||
the result and human-only admission (T4).
|
||||
|
||||
## Paths and evidence from the base
|
||||
|
||||
| Product path | Allowed change |
|
||||
| --- | --- |
|
||||
| packages/business/src/business.mjs | Dispatch kind:library to checkMapping before code-root validation (:228-244) |
|
||||
| packages/business/src/organization.mjs | Compare canonical projects in write before publication (:177 onward) |
|
||||
| packages/business/src/cli.mjs | Skip library roots in validate, resolve --project and credential exclusion roots (:60-61, :71-79, :118) |
|
||||
| packages/business/src/resolve.mjs | Skip library entries while matching a project file (:44) |
|
||||
| packages/business/src/instance-settings.mjs | Skip library before projectFilePath (:142), nothing else |
|
||||
| packages/seat/src/checkout.mjs | Skip library entries in ensureCheckouts (:315-327) |
|
||||
| packages/cli/src/launcher.mjs | Pass only code entries to ensureCheckouts (:354) |
|
||||
| packages/cli/src/config.mjs | Skip library entries in trackerFor's root loop (:61-63), per sage-1860 |
|
||||
| packages/project/src/paths.mjs | Skip library entries in overlap loop (:41-44); default record use refuses library, mirrorPath internally opts into allowLibrary |
|
||||
| packages/library/src/mirror.mjs | New bare init, checked state and served-commit file read module |
|
||||
| packages/library/src/index.mjs | Export mirror module interface and SYNC_STATE_FILE |
|
||||
| packages/bus/src/broker.mjs | New injected library adapter, human-only admission and two async verbs |
|
||||
| packages/bus/src/runtime.mjs | Library adapter wiring, mirror initialization and protectedRoots skip only |
|
||||
| packages/bus/src/server.mjs | Await the two library verbs only |
|
||||
| packages/webui/src/cli.mjs | Two library read callbacks beside project callbacks on main and tester listener; no routes |
|
||||
|
||||
packages/business/src/project.mjs need not change: callers skip a library
|
||||
before projectFilePath or loadProject. Do not weaken their code-root checks.
|
||||
scripts/stack-copy.sh already copies packages/library/src and needs no edit.
|
||||
|
||||
Test paths to modify: business/tests/business.test.mjs, cli.test.mjs,
|
||||
resolve.test.mjs, instance-settings.test.mjs; seat/tests/checkout.test.mjs;
|
||||
cli/tests/launcher.test.mjs, trackers-boot.test.mjs; project/tests/project.test.mjs;
|
||||
bus/tests/project-runtime.test.mjs. All paths are under packages/.
|
||||
|
||||
T5 dependency-audit addition: cli/tests/stack-release.test.mjs and its
|
||||
stack-release-fixture.mjs helper copy packages/library/src alongside
|
||||
business/src, whose validator now imports the mapping checker. No assertions
|
||||
or timeouts change. Whole-file dependent singles require a later Sage slot.
|
||||
|
||||
New tests: packages/business/tests/library-projects-protected.test.mjs;
|
||||
packages/library/tests/mirror.test.mjs; packages/bus/tests/library.test.mjs;
|
||||
packages/webui/tests/library-cli.test.mjs. Reuse existing made-up fixtures.
|
||||
If a new fixture is needed, define it within the corresponding new test file.
|
||||
No unrelated test assertion or timeout changes.
|
||||
|
||||
Record paths: agents/herb/work/queue-215/{MANIFEST.md,BUILD.md}, later source
|
||||
manifest and test receipts there; one append-only BUILD-LOG.md phase for this
|
||||
row when implementation begins. Sage owns docs/SESSIONS.md.
|
||||
|
||||
## T1. Mapping and consumer skips
|
||||
|
||||
- [ ] Add failing business tests: accept a frozen library mapping without a
|
||||
root, retain mixed/only-library documents, and propagate C2-01's mapping
|
||||
and playground refusal codes. A role publish naming a library project
|
||||
refuses at load, even with an otherwise valid fork identity and remote.
|
||||
Existing code validation remains unchanged.
|
||||
- [ ] Add a separate regression for each consumer: CLI validate and its
|
||||
credential exclusion roots; CLI resolve --project books has no project
|
||||
layer; resolveInstance finds a later real project; instance settings still
|
||||
writes an unrelated setting; ensureCheckouts returns no library checkout;
|
||||
launcher creates no books workspace clone; projectRepo skips a library
|
||||
while still rejecting overlap with a real code root. Put books first.
|
||||
- [ ] Add the real bus boot regression in cli/tests/trackers-boot.test.mjs:
|
||||
tracker.baseUrl is set and books is a library entry; boot succeeds, books
|
||||
produces no tracker, and a later real tracker project still works.
|
||||
- [ ] projectRepo(context, {allowLibrary = false} = {}) refuses a library
|
||||
entry by default with project-invalid. Only mirrorPath opts in. Test that
|
||||
ordinary human and tester record initialization leave no directory at the
|
||||
future mirror path; no public verb can supply allowLibrary.
|
||||
- [ ] Add runtime's protectedRoots consumer regression in T4's real runtime
|
||||
fixture. A valid credential outside code/data roots still loads with a
|
||||
library entry, and existing credential protections still refuse unsafe files.
|
||||
- [ ] Run each changed test singly, confirm the new cases fail for the
|
||||
intended root assumption, then implement only the table's guards and
|
||||
checkMapping dispatch. Re-run the same files singly.
|
||||
|
||||
CLI validate reports a library entry as library, without an absent-project
|
||||
warning. CLI resolve --project books resolves without loading a project file,
|
||||
as the design's skip requires. It does not fabricate project vars or a root.
|
||||
|
||||
## T2. M9 at the organization writer
|
||||
|
||||
- [ ] In the new business test file, call organizationWriter.write directly
|
||||
with a changed playground prefix and, separately, a changed hybrid entry.
|
||||
Both refuse organization-projects-protected. Snapshot business bytes and
|
||||
changes.jsonl before each call; assert both remain identical afterward.
|
||||
- [ ] Cover addition, deletion and changes to code projects too, since M9
|
||||
protects all projects. Reordered object keys are allowed; array changes
|
||||
are changes. An ordinary non-project write still succeeds and logs.
|
||||
- [ ] Add a serialization regression so a proposed object's serialized
|
||||
projects cannot differ from those checked before publication.
|
||||
- [ ] Run the file singly to see the missing refusal, compare canonical
|
||||
projects against old disk JSON and the document that actually lands,
|
||||
before any publication/receipt, then re-run it singly.
|
||||
|
||||
## T3. Bare mirror, served file and read-only state
|
||||
|
||||
- [ ] Add failing tests for mirrorPath and ensureMirror: exact projectRepo
|
||||
default, no checkout/remote, retained no-links/overlap protections,
|
||||
idempotence, simultaneous creators and preservation of a pre-existing
|
||||
non-mirror directory or unsafe Git metadata.
|
||||
- [ ] Add failing state tests for BUILD.md's full version-1 contract: missing
|
||||
state/ref, valid state, malformed JSON, unknown keys/types, branch mismatch,
|
||||
invalid times/IDs/problem shape, symlink/hardlink/non-private file and
|
||||
missing or mismatched served ref. Assert reads change no bytes.
|
||||
- [ ] Add shared-lock tests: withLibraryLock(mirror, operation) waits at most
|
||||
5000 ms, refuses library-sync-busy on timeout and releases after a throw.
|
||||
A reader arriving between the publisher's served move and state write
|
||||
waits, then reads the consistent pair. Cover first sync and fast-forward
|
||||
in independent processes, same-process independent acquisitions and
|
||||
release after owner death. A callback-scoped reader expires on release.
|
||||
- [ ] Implement the lock by async spawn of flock on the parent's inherited
|
||||
fd 3, as pinned in BUILD.md. Await acquisition-child exit before calling
|
||||
operation. Never spawnSync during acquisition, which could freeze the bus
|
||||
for 5 s. A parallel ordinary bus read remains responsive while a library
|
||||
read waits. Only the publication/read lock is shared; fetch is outside it.
|
||||
- [ ] Add served-read tests against local fixtures containing spaces,
|
||||
capitals, a .canvas and unmapped top-level note. Seed a served ref and state
|
||||
without fetching. A newer unrelated HEAD does not change the returned
|
||||
file. R1/R2/R7 exclusions, traversal, a missing path, a caller commit and a
|
||||
never-synced mirror refuse. Status contains no fixture body marker.
|
||||
- [ ] Implement mirror.mjs and index exports using the shared contract.
|
||||
Initialization stages into an owned private sibling directory and renames
|
||||
atomically; a losing creator validates the winner and removes only its own
|
||||
staging directory. Git uses existing bounded safe helpers, no remote.
|
||||
- [ ] Run mirror.test.mjs singly, red then green. Re-run C2-01 reader.test.mjs
|
||||
singly if a reader assumption changes. Do not change C2-01 reader rules.
|
||||
|
||||
mirror.mjs validates a bare mirror's metadata before Git reads it, including
|
||||
no remote/include/indirection. It does not apply inspectGit's working-tree
|
||||
.git assumption to a bare repository. Failures reuse library-git or the
|
||||
approved library-state-invalid code. State reads are bounded and private.
|
||||
|
||||
## T4. Human-only bus reads and deferred Console callbacks
|
||||
|
||||
- [ ] Add broker tests for library.file and library.status using injected
|
||||
adapters. Each agent session refuses human-required before adapter access,
|
||||
including bad arguments and an absent adapter. A reader or tester does not
|
||||
gain a human capability. Human calls validate keys, then use checked
|
||||
library context. Secret checking and bounded refusal evidence remain.
|
||||
- [ ] Add real startBroker/socket tests for both verbs. Seed only local
|
||||
fixture mirror data. Assert result shape/content and an actual status
|
||||
field, not merely successful transport or {}. Cover never-synced status
|
||||
and runtime's protectedRoots skip. Keep the pinned import-closure test.
|
||||
- [ ] Inject a separate library adapter via runtime, initialize only library
|
||||
mirrors, add the two async request paths, and include them in server's
|
||||
awaited list. Do not import library or fs into broker.
|
||||
- [ ] Unsafe or pre-existing non-mirror initialization refuses bus boot,
|
||||
closes opened resources and never publishes a ready socket. Add that
|
||||
runtime regression in library.test.mjs.
|
||||
- [ ] Wire main and tester Console callbacks beside project-file callbacks.
|
||||
No HTTP route is shipped. The new CLI test captures the startServer
|
||||
options with a test-only module loader and calls the supplied callbacks
|
||||
against a fixture bus, checking the verb and transport identity. No
|
||||
product test seam, UI page, endpoint or serve.mjs change is needed.
|
||||
- [ ] Run library.test.mjs, project-runtime.test.mjs and library-cli.test.mjs
|
||||
individually, red then green. Human admission may use the existing
|
||||
fixture authentication seam; do not forge or bypass a live identity.
|
||||
|
||||
## T5. Review packet and Sage's gate
|
||||
|
||||
- [ ] Search packages/*/tests and scripts/test-*.sh for each changed source
|
||||
path, imports and 64-hex pins. Record the resulting test-file list and run
|
||||
each changed/touched test singly. Stop and report a scope error.
|
||||
- [ ] Self-review the diff against this manifest and all acceptance cases.
|
||||
Record results, commands, skips, source hashes and remaining limits in
|
||||
BUILD.md. Request Sage's slot for the required business, seat, cli, project,
|
||||
bus and webui suites and every scripts/test-*.sh. Do not run them ungranted.
|
||||
- [ ] After applicable tests are green, make the source commit, build the
|
||||
candidate manifest from the authorized pushed tip, and move the row with
|
||||
scripts/mosaic. Sage posts token-requiring requests; Filbert reviews the
|
||||
candidate. Push only when authorized. No merge or live change by Herb.
|
||||
|
||||
## Self-review
|
||||
|
||||
Every brief acceptance item is in T1-T4. Added host support paths match
|
||||
sage-1848/1860. BUILD.md records the complete 14-hit root audit. State shape
|
||||
and authority match sage-1852; the shared read/publication lock follows
|
||||
sage-1860 and Megavolt's agreement. Fetch, timers, outer sync serialization
|
||||
and HTTP stay with their later rows.
|
||||
This is a plan only; no product edits or test execution precede approval.
|
||||
@@ -88,6 +88,7 @@ export const TASK_PAGE_MAX = 200;
|
||||
// Row 148 (#1642): the human-only verbs behind the Console's document editor. The save is the one
|
||||
// request allowed past the ordinary 64 KiB envelope (server.mjs and client.mjs), up to its content cap.
|
||||
export const PROJECT_FILE_VERBS = Object.freeze(['project.files', 'project.file', 'project.file.save']);
|
||||
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status']);
|
||||
export const PROJECT_SAVE_VERB = 'project.file.save';
|
||||
export const PROJECT_SAVE_REQUEST_MAX = 1024 * 1024 + 64 * 1024;
|
||||
export const WEB_VERBS =Object.freeze(['web.search', 'web.fetch']);
|
||||
@@ -267,6 +268,7 @@ export class Broker {
|
||||
#organization;
|
||||
#credentialVerbs;
|
||||
#project;
|
||||
#library;
|
||||
#usage;
|
||||
#limits;
|
||||
#harnessOn;
|
||||
@@ -276,7 +278,7 @@ export class Broker {
|
||||
#stops = new Map();
|
||||
#clock = 0;
|
||||
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
|
||||
constructor({ store, businesses, secretCheck = () => {}, credentialStatus = () => [], organization = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, hostGrant = null }) {
|
||||
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
|
||||
this.#businesses = structuredClone(businesses);
|
||||
this.#store = store;
|
||||
@@ -287,6 +289,7 @@ export class Broker {
|
||||
this.#credentialVerbs = credentialVerbs;
|
||||
this.#hostGrant = hostGrant;
|
||||
this.#project = project;
|
||||
this.#library = library;
|
||||
this.#usage = usage;
|
||||
this.#limits = limits;
|
||||
// Row 65: harnessOn(business, name) is true when the organization file's plugins.harness switches
|
||||
@@ -1743,6 +1746,8 @@ export class Broker {
|
||||
return this.#requestProject(s, { verb: request.verb, args: structuredClone(args) });
|
||||
if (PROJECT_FILE_VERBS.includes(request.verb))
|
||||
return this.#requestProjectFile(s, { verb: request.verb, args: structuredClone(args) });
|
||||
if (LIBRARY_VERBS.includes(request.verb))
|
||||
return this.#requestLibrary(s, { verb: request.verb, args: structuredClone(args) });
|
||||
if (CREDENTIAL_VERBS.includes(request.verb)) return this.#requestCredential(s, request.verb, structuredClone(args));
|
||||
if (["limits", "limits.set", "organization.set", "organization.revision", "plugins", "plugins.set", "instances", "instances.set", "routes", "routes.set"].includes(request.verb)) {
|
||||
// Row 123 (#1617, REQ-COS-1, REQ-LIMIT-1): organization.set is limits.set for a role whose
|
||||
@@ -1992,6 +1997,26 @@ export class Broker {
|
||||
throw this.#refused(s, ProjectFailure && e instanceof ProjectFailure ? new BusError(e.code) : e, request);
|
||||
}
|
||||
}
|
||||
// C2-04 M4: only a verified human reads library text or status. Host Git
|
||||
// functions stay injected, preserving this module's sandbox import closure.
|
||||
async #requestLibrary(s, request) {
|
||||
const { verb, args } = request;
|
||||
try {
|
||||
this.#human(s);
|
||||
const fields = verb === 'library.file' ? ['project', 'path'] : ['project'];
|
||||
keys(args, fields, fields);
|
||||
if (!this.#library?.readFile || !this.#library?.status) fail('project-not-configured');
|
||||
const context = { business: this.#business(s.business), project: args.project };
|
||||
const result = verb === 'library.file'
|
||||
? await this.#library.readFile({ ...context, path: args.path })
|
||||
: await this.#library.status(context);
|
||||
this.#secretCheck(result);
|
||||
return result;
|
||||
} catch (error) {
|
||||
const Failure = this.#library?.LibraryError;
|
||||
throw this.#refused(s, Failure && error instanceof Failure ? new BusError(error.code) : error, request);
|
||||
}
|
||||
}
|
||||
// The eight task verbs reach the trusted S3 handler here. It runs outside any transaction, gets the
|
||||
// cap to authorize and record through authorize() and recordTask(); refusals are recorded as in request().
|
||||
async requestTask(cap, request, handler) {
|
||||
|
||||
@@ -3,6 +3,7 @@ import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { Store } from './store.mjs';
|
||||
import * as projectModule from '../../project/src/index.mjs';
|
||||
import * as libraryModule from '../../library/src/index.mjs';
|
||||
import { Broker, BusError } from './broker.mjs';
|
||||
import { Credentials } from './credentials.mjs';
|
||||
import { serve } from './server.mjs';
|
||||
@@ -92,7 +93,7 @@ export async function startBroker({
|
||||
const protectedRoots = [
|
||||
fileURLToPath(new URL('../../../', import.meta.url)),
|
||||
...repoRoots,
|
||||
...Object.values(businesses).flatMap((b) => Object.values(b.projects ?? {}).map((p) => p.root)),
|
||||
...Object.values(businesses).flatMap((b) => Object.values(b.projects ?? {}).filter(p => p.kind !== 'library').map((p) => p.root)),
|
||||
];
|
||||
// Row 199 (#1697, design D1): the per-agent store is <configDir>/secrets/agents.
|
||||
const storeRoot = organization ? join(organization.dir, 'secrets', 'agents') : null;
|
||||
@@ -123,12 +124,22 @@ export async function startBroker({
|
||||
const withTokens = (id, result) => ({ ...result, tokens: limits.report(id) });
|
||||
// Row 210 (#1707, design D7 step 6) and row 234 (#1740, r6 E2): the run dir and the host enrollment half.
|
||||
const runRoot = organization ? join(organization.dir, 'secrets', 'run') : null;
|
||||
// C2-04: mirrors are host state, never session checkouts. An unsafe
|
||||
// existing mirror refuses startup before a socket can become ready.
|
||||
for (const business of Object.values(businesses))
|
||||
for (const [project, entry] of Object.entries(business.projects ?? {}))
|
||||
if (entry.kind === 'library') await libraryModule.ensureMirror({ business, project, dataRoot });
|
||||
const broker = new Broker({
|
||||
hostGrant: runRoot ? hostGrant({ configDir: organization.dir, runRoot }) : null,
|
||||
usage,
|
||||
limits,
|
||||
store,
|
||||
businesses,
|
||||
library: {
|
||||
readFile: context => libraryModule.readLibraryFile({ ...context, dataRoot }),
|
||||
status: context => libraryModule.libraryStatus({ ...context, dataRoot }),
|
||||
LibraryError: libraryModule.LibraryError,
|
||||
},
|
||||
project: {
|
||||
readSpec: context => projectModule.readSpec({ ...context, dataRoot }),
|
||||
writeRecord: context => projectModule.writeRecord({ ...context, dataRoot }),
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { createServer } from 'node:net';
|
||||
import { lstatSync, chmodSync, unlinkSync } from 'node:fs';
|
||||
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
|
||||
import { BusError, TASK_VERBS, WEB_VERBS, HOST_VERBS, HOSTS_VERBS, PROJECT_FILE_VERBS, LIBRARY_VERBS, PROJECT_SAVE_VERB, PROJECT_SAVE_REQUEST_MAX } from './broker.mjs';
|
||||
const LIMIT = 65536;
|
||||
// Row 148: a document save may carry up to 1 MiB of text. Only a request that names that verb near
|
||||
// its start (the proof object comes first) may pass LIMIT; the parsed verb is checked again below.
|
||||
@@ -122,7 +122,7 @@ export async function serve({ broker, path, authenticateHuman = null, authentica
|
||||
}
|
||||
// Row138: only project Git effects need an await; preserve every other sync path below.
|
||||
if (r.verb === 'usage.begin' || r.verb === 'usage.end' ||
|
||||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) ||
|
||||
r.verb === 'spec.read' || r.verb === 'record.write' || PROJECT_FILE_VERBS.includes(r.verb) || LIBRARY_VERBS.includes(r.verb) ||
|
||||
// Row 234 (#1740): a grant raise awaits its keyscan, and hosts.check its ssh.
|
||||
HOSTS_VERBS.includes(r.verb) ||
|
||||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit') ||
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
// C2-04 M4 and real async transport, with made-up local library data.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { Store } from '../src/store.mjs';
|
||||
import { Broker } from '../src/broker.mjs';
|
||||
import { serve } from '../src/server.mjs';
|
||||
import { Client } from '../src/client.mjs';
|
||||
import { startBroker } from '../src/runtime.mjs';
|
||||
import { mappingEntry, repo, git, BASE, MARKER } from '../../library/tests/helpers.mjs';
|
||||
import * as library from '../../library/src/index.mjs';
|
||||
|
||||
function world(t, adapter = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'bus-library-')), store = new Store(root);
|
||||
t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); });
|
||||
const business = { id: 'acme', human: 'jason', vars: { 'bus.tester': true },
|
||||
projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
|
||||
const broker = new Broker({ store, businesses: { acme: business }, library: adapter });
|
||||
const human = broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
const agent = broker.bindLaunch({ business: 'acme', role: 'coder', run: 'library-agent', harness: 'pi' });
|
||||
const reader = broker.bindReader({ business: 'acme' }), tester = broker.bindTester({ business: 'acme' });
|
||||
const call = (cap, verb, args = {}) => Promise.resolve().then(() => broker.request(cap, { verb, args }));
|
||||
return { root, store, broker, business, human, agent, reader, tester, call };
|
||||
}
|
||||
|
||||
test('C2-04 M4: both verbs reject every agent before arguments or adapter access', async t => {
|
||||
let touched = 0;
|
||||
const f = world(t, { readFile: () => { touched++; }, status: () => { touched++; } });
|
||||
for (const verb of ['library.file', 'library.status']) {
|
||||
await assert.rejects(f.call(f.agent, verb, { unknown: true }), { code: 'human-required' });
|
||||
await assert.rejects(f.call(f.tester, verb, { unknown: true }), { code: 'human-required' });
|
||||
await assert.rejects(f.call(f.reader, verb, { unknown: true }), { code: 'read-only' });
|
||||
}
|
||||
assert.equal(touched, 0);
|
||||
const refusals = f.store.all("SELECT body FROM events WHERE kind='action.refused'").map(row => JSON.parse(row.body));
|
||||
assert.equal(refusals.length, 6);
|
||||
assert.equal(JSON.stringify(refusals).includes('unknown'), false);
|
||||
const absent = world(t, null);
|
||||
for (const verb of ['library.file', 'library.status'])
|
||||
await assert.rejects(absent.call(absent.agent, verb, {}), { code: 'human-required' });
|
||||
});
|
||||
|
||||
test('C2-04 bus: human reads validate keys and return adapter results without path leakage', async t => {
|
||||
const seen = [];
|
||||
const f = world(t, { readFile: async args => { seen.push(args); return { project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40), content: '# Made up\n' }; },
|
||||
status: async args => { seen.push(args); return { project: 'books', served: null, problems: [] }; }, LibraryError: library.LibraryError });
|
||||
assert.equal((await f.call(f.human, 'library.file', { project: 'books', path: 'Book One/A.md' })).content, '# Made up\n');
|
||||
assert.equal((await f.call(f.human, 'library.status', { project: 'books' })).served, null);
|
||||
assert.equal(seen[0].business.id, 'acme'); assert.equal(seen[1].project, 'books');
|
||||
for (const [verb, args] of [['library.file', { project: 'books' }], ['library.file', { project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40) }], ['library.status', { project: 'books', extra: 1 }]])
|
||||
await assert.rejects(f.call(f.human, verb, args), { code: 'invalid-request' });
|
||||
});
|
||||
|
||||
test('C2-04 socket: each async library verb resolves its result rather than serializing {}', async t => {
|
||||
const f = world(t, { readFile: async () => ({ project: 'books', path: 'Book One/A.md', commit: 'a'.repeat(40), content: '# Fixture\n' }),
|
||||
status: async () => ({ project: 'books', served: null, problems: [] }), LibraryError: library.LibraryError });
|
||||
const path = join(f.root, 'bus.sock');
|
||||
const server = await serve({ broker: f.broker, path, authenticateHuman: () => f.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true }) });
|
||||
t.after(() => server.close());
|
||||
const human = new Client({ path, human: { fixture: true } });
|
||||
assert.deepEqual(await human.call('library.status', { project: 'books' }), { project: 'books', served: null, problems: [] });
|
||||
assert.equal((await human.call('library.file', { project: 'books', path: 'Book One/A.md' })).content, '# Fixture\n');
|
||||
});
|
||||
|
||||
test('C2-04 runtime: protectedRoots skips library but retains real code-root credential protection', async t => {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data'), token = join(f.root, 'fixture-token');
|
||||
mkdirSync(dataRoot, { mode: 0o700 }); writeFileSync(token, 'made-up-fixture-token-123456', { mode: 0o600 });
|
||||
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry(), stack: { root: f.repo } },
|
||||
arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] }, credentials: { gitea: { file: token, rotateBy: '2099-01-01' } } } } };
|
||||
const runtime = await startBroker({ dataRoot, businesses: { acme: business } });
|
||||
await runtime.close();
|
||||
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
|
||||
assert.equal(git(mirror, 'rev-parse', '--is-bare-repository'), 'true');
|
||||
const bad = join(f.repo, 'credential'); writeFileSync(bad, 'made-up-fixture-token-123456', { mode: 0o600 });
|
||||
business.roles.coder.credentials.gitea.file = bad;
|
||||
await assert.rejects(startBroker({ dataRoot, businesses: { acme: business } }));
|
||||
});
|
||||
|
||||
test('C2-04 runtime: unsafe existing mirror refuses boot and never publishes a socket', async t => {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data');
|
||||
mkdirSync(dataRoot, { mode: 0o700 });
|
||||
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
|
||||
const mirror = join(dataRoot, 'projects/acme/books'); mkdirSync(mirror, { recursive: true });
|
||||
writeFileSync(join(mirror, 'keep'), 'made-up note');
|
||||
await assert.rejects(startBroker({ dataRoot, businesses: { acme: business } }), { code: 'library-git' });
|
||||
assert.equal(existsSync(join(dataRoot, 'bus.sock')), false);
|
||||
assert.equal(readFileSync(join(mirror, 'keep'), 'utf8'), 'made-up note');
|
||||
});
|
||||
|
||||
test('C2-04 runtime: real library adapters reach the socket and lock waits leave other bus reads responsive', { timeout: 15000 }, async t => {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data');
|
||||
mkdirSync(dataRoot, { mode: 0o700 });
|
||||
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' }, roles: { coder: { authority: { withinRole: [], crossRole: [] } } } };
|
||||
const runtime = await startBroker({ dataRoot, businesses: { acme: business } });
|
||||
t.after(() => runtime.close());
|
||||
const cap = runtime.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
const path = join(f.root, 'human-fixture.sock');
|
||||
const server = await serve({ broker: runtime.broker, path, authenticateHuman: () => runtime.broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true }) }); t.after(() => server.close());
|
||||
const human = new Client({ path, human: { fixture: true } });
|
||||
assert.equal((await human.call('library.status', { project: 'books' })).served, null);
|
||||
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
|
||||
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify({ version: 1, branch: 'main', served: f.commit,
|
||||
lastFetch: null, lastError: null, diverged: null, problems: [] }), { mode: 0o600 });
|
||||
const file = await human.call('library.file', { project: 'books', path: 'Book One/Characters/Tomas.md' });
|
||||
assert.equal(file.content, BASE['Book One/Characters/Tomas.md']);
|
||||
assert.equal(file.commit, f.commit);
|
||||
let release, done = false;
|
||||
const hold = library.withLibraryLock(mirror, () => new Promise(r => { release = r; }));
|
||||
const deadline = Date.now() + 5000;
|
||||
while (!release && Date.now() < deadline) await new Promise(r => setTimeout(r, 5));
|
||||
assert.equal(typeof release, 'function');
|
||||
t.after(async () => { release(); await hold; });
|
||||
const pending = human.call('library.status', { project: 'books' }).then(result => { done = true; return result; });
|
||||
const ordinary = await Promise.race([human.call('agents'), new Promise((_, reject) => {
|
||||
const timer = setTimeout(() => reject(Error('ordinary bus read blocked by lock acquisition')), 1000); timer.unref();
|
||||
})]);
|
||||
assert.ok(ordinary && typeof ordinary === 'object'); assert.equal(done, false);
|
||||
release(); await hold;
|
||||
assert.equal((await pending).served, f.commit);
|
||||
});
|
||||
@@ -10,6 +10,7 @@ import { checkVars } from "./vars.mjs";
|
||||
import { parseCredentialRef } from "./credentials.mjs";
|
||||
import { loadRole } from "./role.mjs";
|
||||
import { checkPlugins } from "../../plugins/src/kinds.mjs";
|
||||
import { checkMapping } from "../../library/src/mapping.mjs";
|
||||
import { forkProblem, giteaName, parseGiteaUrl, HOST_SERVICE } from "./vocabulary.mjs";
|
||||
import {
|
||||
requireObject, rejectUnknownKeys, requireId, requirePositiveInt, requireString, requireDistinctList, readJsonFile, deepFreeze,
|
||||
@@ -229,6 +230,10 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
|
||||
for (const [id, entry] of Object.entries(document.projects)) {
|
||||
requireId(id, `${file} project id`);
|
||||
requireObject(entry, `${file} projects.${id}`);
|
||||
if (entry.kind === 'library') {
|
||||
projects[id] = checkMapping(entry, `${file} projects.${id}`, { fail: (code, message) => refuse(`${code}: ${message}`) });
|
||||
continue;
|
||||
}
|
||||
rejectUnknownKeys(entry, ["root", "repo", "remote"], `${file} projects.${id}`);
|
||||
if (typeof entry.root !== "string" || !isAbsolute(entry.root) || normalize(entry.root) !== entry.root || entry.root.includes("\0")) {
|
||||
refuse(`${file} projects.${id}.root must be a normalized absolute path`);
|
||||
@@ -245,6 +250,7 @@ export function validateBusinessDocument(document, file, { rolesDir }) {
|
||||
for (const [instance, role] of Object.entries(roles)) {
|
||||
for (const project of role.publish ?? []) {
|
||||
if (!Object.hasOwn(projects, project)) refuse(`${file} roles.${instance}.publish names ${project}, which the business file doesn't declare`);
|
||||
if (projects[project].kind === 'library') refuse(`${file} roles.${instance}.publish names library project ${project}; library repositories cannot be published by agents`);
|
||||
if (!projects[project].remote) refuse(`${file} roles.${instance}.publish names ${project}, which has no "remote"`);
|
||||
const problem = forkProblem(projects[project].remote, role.identity);
|
||||
if (problem) refuse(`${file} roles.${instance}.publish names ${project}: ${problem}`);
|
||||
|
||||
@@ -58,7 +58,7 @@ function checkRefs(refs, forbiddenRoots) {
|
||||
}
|
||||
|
||||
function forbiddenRoots(config, business) {
|
||||
return [REPO, config.dataRoot, ...Object.values(business.projects).map((p) => p.root)];
|
||||
return [REPO, config.dataRoot, ...Object.values(business.projects).filter(p => p.kind !== 'library').map((p) => p.root)];
|
||||
}
|
||||
|
||||
function validate(args) {
|
||||
@@ -69,6 +69,7 @@ function validate(args) {
|
||||
const projects = {};
|
||||
const warnings = [];
|
||||
for (const [id, entry] of Object.entries(business.projects)) {
|
||||
if (entry.kind === 'library') { projects[id] = 'library'; continue; }
|
||||
const file = projectFilePath(entry.root);
|
||||
if (!existsSync(file)) {
|
||||
projects[id] = "absent";
|
||||
@@ -115,7 +116,7 @@ function resolveCommand(args) {
|
||||
if (project !== null) {
|
||||
const entry = Object.hasOwn(business.projects, project) ? business.projects[project] : null;
|
||||
if (!entry) fail(2, `business ${business.id} declares no project ${JSON.stringify(project)}`);
|
||||
loaded = loadProject(entry.root);
|
||||
if (entry.kind !== 'library') loaded = loadProject(entry.root);
|
||||
}
|
||||
const resolved = resolveInstance({ system: systemVars(config), business, project: loaded, instance });
|
||||
const checked = checkRefs(Object.values(resolved.credentials), forbiddenRoots(config, business));
|
||||
|
||||
@@ -139,6 +139,7 @@ export function instanceSettings({ dir, rolesDir, system, levels = () => null })
|
||||
}
|
||||
checkRoutes({ proposed, old: before, levels });
|
||||
for (const [project, entry] of Object.entries(proposed.projects)) {
|
||||
if (entry.kind === 'library') continue;
|
||||
if (!existsSync(projectFilePath(entry.root))) continue;
|
||||
const loaded = loadProject(entry.root);
|
||||
if (loaded.id !== project) throw new OrganizationError("organization-project-mismatch");
|
||||
|
||||
@@ -185,12 +185,18 @@ export function organizationWriter({ dir, rolesDir, barrier = () => {} }) {
|
||||
const oldBytes = regular(file);
|
||||
const before = sha256(oldBytes);
|
||||
if (revision !== before) fail("revision-conflict", "organization revision changed; read again");
|
||||
validateBusinessDocument(document, file, { rolesDir });
|
||||
const old = JSON.parse(oldBytes);
|
||||
// C2-04 M9: every organization setter preserves the whole project
|
||||
// mapping, including library playgrounds and hybrid universes.
|
||||
if (canonicalJson(document?.projects) !== canonicalJson(old.projects))
|
||||
fail('organization-projects-protected');
|
||||
validateBusinessDocument(document, file, { rolesDir });
|
||||
const bytes = JSON.stringify(document, null, 2) + "\n";
|
||||
// Check and log exactly what lands on disk, for every setter: a value JSON
|
||||
// can't carry (a Date, a Map) serializes as something else (row 131, sage-0314).
|
||||
const written = JSON.parse(bytes);
|
||||
if (canonicalJson(written.projects) !== canonicalJson(old.projects))
|
||||
fail('organization-projects-protected');
|
||||
validateBusinessDocument(written, file, { rolesDir });
|
||||
const after = sha256(bytes);
|
||||
const transaction = randomUUID();
|
||||
|
||||
@@ -41,7 +41,7 @@ export function resolveInstance({ system, business, project = null, instance })
|
||||
];
|
||||
let projectId = null;
|
||||
if (project) {
|
||||
const declared = Object.entries(business.projects).find(([, p]) => projectFilePath(p.root) === project.file);
|
||||
const declared = Object.entries(business.projects).find(([, p]) => p.kind !== 'library' && projectFilePath(p.root) === project.file);
|
||||
if (!declared || declared[0] !== project.id) {
|
||||
refuse(`project ${project.id} (${project.file}) isn't declared under that id in business ${business.id}`);
|
||||
}
|
||||
|
||||
@@ -8,6 +8,36 @@ import { BusinessError, businessFilePath, configDir, loadBusiness, validateBusin
|
||||
import { THREAD_IDLE_SECONDS } from "../src/business.mjs";
|
||||
import { launchDoc, businessDoc, REPO, REPO_ROLES, rolesCopy, tmp, writeJson } from "./helpers.mjs";
|
||||
|
||||
test('C2-04: library mappings load without roots, remain frozen, and retain mapping refusals', async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const { rmSync } = await import('node:fs');
|
||||
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const doc = businessDoc(root), file = join(root, 'businesses/acme.json');
|
||||
doc.projects = { books: mappingEntry(), ...doc.projects };
|
||||
const loaded = validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES });
|
||||
const { checkMapping } = await import('../../library/src/mapping.mjs');
|
||||
assert.deepEqual(loaded.projects.books, checkMapping(mappingEntry()));
|
||||
assert.equal(loaded.projects.books.root, undefined);
|
||||
assert.ok(Object.isFrozen(loaded.projects.books.universes.harbor.books));
|
||||
const only = structuredClone(doc); delete only.projects.stack;
|
||||
assert.deepEqual(Object.keys(validateBusinessDocument(only, file, { rolesDir: REPO_ROLES }).projects), ['books']);
|
||||
doc.projects.books.universes.harbor.playground = ['Elsewhere/Playground/'];
|
||||
refuses(() => validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES }), /library-playground-outside-book/);
|
||||
doc.projects.books = { ...mappingEntry(), root: '/forbidden' };
|
||||
refuses(() => validateBusinessDocument(doc, file, { rolesDir: REPO_ROLES }), /library-mapping-invalid/);
|
||||
});
|
||||
|
||||
test('C2-04: an otherwise valid publish grant cannot name a library repo', async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const { rmSync } = await import('node:fs');
|
||||
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const doc = businessDoc(root);
|
||||
doc.projects.books = { ...mappingEntry(), remote: 'https://git.example/writer/books' };
|
||||
doc.roles.coder.identity = { name: 'Coder', email: '[email protected]', gitea: { url: 'https://git.example', user: 'acme-coder' } };
|
||||
doc.roles.coder.publish = ['books'];
|
||||
refuses(() => validateBusinessDocument(doc, join(root, 'businesses/acme.json'), { rolesDir: REPO_ROLES }), /publish.*library/);
|
||||
});
|
||||
|
||||
function refuses(fn, pattern, exitCode = 2) {
|
||||
assert.throws(fn, (error) => {
|
||||
assert.ok(error instanceof BusinessError, `expected BusinessError, got ${error}`);
|
||||
|
||||
@@ -8,6 +8,24 @@ import { businessDoc, REPO, rolesCopy, systemConfig, tmp, tokenFile, writeJson }
|
||||
|
||||
const CLI = join(REPO, "packages", "business", "src", "cli.mjs");
|
||||
|
||||
test('C2-04: business validate and resolve skip a library before code projects and credential checks', async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const { rmSync } = await import('node:fs');
|
||||
const s = setup(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; });
|
||||
t.after(() => rmSync(s.root, { recursive: true, force: true }));
|
||||
writeJson(projectFilePath(s.doc.projects.stack.root), { projectVersion: 1, id: 'stack', vars: { 'tracker.project': 3 } });
|
||||
const checked = run(s, 'validate', 'acme');
|
||||
assert.equal(checked.code, 0, checked.err);
|
||||
assert.deepEqual(JSON.parse(checked.out).projects, { books: 'library', stack: 'valid' });
|
||||
assert.doesNotMatch(checked.err, /project books: no project file/);
|
||||
const library = run(s, 'resolve', 'acme', 'coder', '--project', 'books');
|
||||
assert.equal(library.code, 0, library.err);
|
||||
assert.equal(JSON.parse(library.out).project, null);
|
||||
const code = run(s, 'resolve', 'acme', 'coder', '--project', 'stack');
|
||||
assert.equal(code.code, 0, code.err);
|
||||
assert.equal(JSON.parse(code.out).vars['tracker.project'], 3);
|
||||
});
|
||||
|
||||
// A complete scratch setup: system config, business file, roles copy.
|
||||
// HOME points into the scratch tree too, so nothing reads the real one.
|
||||
function setup(mutate) {
|
||||
|
||||
@@ -19,6 +19,16 @@ after(() => roots.forEach(root => rmSync(root, { recursive: true, force: true })
|
||||
const harnessOn = (doc, names = ["pi", "claude-code"]) => { doc.plugins = { ...(doc.plugins ?? {}), harness: names }; };
|
||||
const LEVELS = { pi: ["off", "minimal", "low", "medium", "high", "xhigh", "max"], "claude-code": ["low", "medium", "high", "xhigh", "max"] };
|
||||
|
||||
test('C2-04: instance settings skip a library while checking a later code project', async () => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const f = fixture(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; });
|
||||
writeJson(projectFilePath(f.doc.projects.stack.root), { projectVersion: 1, id: 'stack', vars: { 'tracker.project': 3 } });
|
||||
const before = f.settings.read('acme');
|
||||
f.settings.set('acme', { revision: before.revision, instance: 'gate-215', op: 'add', definition: 'merge-gate' }, 'jason');
|
||||
assert.deepEqual(JSON.parse(readFileSync(f.file, 'utf8')).projects.books, mappingEntry());
|
||||
assert.ok(f.settings.read('acme').instances['gate-215']);
|
||||
});
|
||||
|
||||
function fixture(mutate, { levels } = {}) {
|
||||
const root = tmp();
|
||||
roots.push(root);
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
// C2-04 M9: use the writer directly, rather than relying on broker key filtering.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync, rmSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { organizationWriter } from '../src/organization.mjs';
|
||||
import { businessDoc, rolesCopy, tmp, writeJson } from './helpers.mjs';
|
||||
import { mappingEntry } from '../../library/tests/helpers.mjs';
|
||||
|
||||
function fixture(t) {
|
||||
const root = tmp(); t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const doc = businessDoc(root);
|
||||
const books = mappingEntry();
|
||||
books.universes.hybrid = { kind: 'hybrid', name: 'An invented crossover',
|
||||
sources: ['harbor', 'second'], books: ['Hybrid Book'], playground: [], constraints: [], notes: 'Fixture only' };
|
||||
doc.projects = { books, ...doc.projects };
|
||||
const dir = join(root, 'config'), file = writeJson(join(dir, 'businesses/acme.json'), doc);
|
||||
const writer = organizationWriter({ dir, rolesDir: rolesCopy(root) });
|
||||
// Give both snapshots an existing ordinary committed receipt to preserve.
|
||||
doc.vars['tracker.pollSeconds'] = 61;
|
||||
writer.write('acme', doc, { revision: writer.read('acme').revision, actor: 'jason' });
|
||||
const log = writer.logFile('acme');
|
||||
return { root, file, doc, writer, log };
|
||||
}
|
||||
|
||||
function protectedWrite(f, change) {
|
||||
const before = readFileSync(f.file), logBefore = readFileSync(f.log);
|
||||
const next = structuredClone(f.writer.read('acme').document); change(next);
|
||||
assert.throws(() => f.writer.write('acme', next, { revision: f.writer.read('acme').revision, actor: 'jason' }),
|
||||
{ code: 'organization-projects-protected' });
|
||||
assert.deepEqual(readFileSync(f.file), before);
|
||||
assert.deepEqual(readFileSync(f.log), logBefore);
|
||||
}
|
||||
|
||||
test('C2-04 M9: changed playground and hybrid entries leave business and changes.jsonl byte-identical', t => {
|
||||
const f = fixture(t);
|
||||
protectedWrite(f, next => { next.projects.books.universes.harbor.playground = ['Book One/New Playground/']; });
|
||||
protectedWrite(f, next => { next.projects.books.universes.hybrid.notes = 'Changed hybrid rule'; });
|
||||
});
|
||||
|
||||
test('C2-04 M9: code projects, additions, deletions and array changes are protected too', t => {
|
||||
const f = fixture(t);
|
||||
protectedWrite(f, next => { next.projects.stack.root = join(f.root, 'different-code'); });
|
||||
protectedWrite(f, next => { next.projects.extra = { root: join(f.root, 'extra') }; });
|
||||
protectedWrite(f, next => { delete next.projects.stack; });
|
||||
protectedWrite(f, next => { next.projects.books.universes.hybrid.sources.reverse(); });
|
||||
});
|
||||
|
||||
test('C2-04 M9: canonical key order is allowed and a non-project setter still records its change', t => {
|
||||
const f = fixture(t), old = f.writer.read('acme');
|
||||
const next = structuredClone(old.document);
|
||||
next.projects = Object.fromEntries(Object.entries(next.projects).reverse());
|
||||
next.vars['tracker.pollSeconds'] = 62;
|
||||
f.writer.write('acme', next, { revision: old.revision, actor: 'jason' });
|
||||
assert.equal(f.writer.read('acme').document.vars['tracker.pollSeconds'], 62);
|
||||
const receipt = JSON.parse(readFileSync(f.log, 'utf8').trim().split('\n').at(-1));
|
||||
assert.equal(receipt.state, 'committed');
|
||||
assert.deepEqual(receipt.changes.map(c => c.path), ['vars.tracker.pollSeconds']);
|
||||
});
|
||||
|
||||
test('C2-04 M9: a serialization hook cannot change projects after the canonical comparison', t => {
|
||||
const f = fixture(t);
|
||||
protectedWrite(f, next => {
|
||||
const changed = structuredClone(next.projects);
|
||||
changed.books.universes.hybrid.notes = 'Serialized change';
|
||||
Object.defineProperty(next.projects, 'toJSON', { value: () => changed });
|
||||
});
|
||||
});
|
||||
@@ -28,6 +28,16 @@ function setup(mutate, projectDoc) {
|
||||
|
||||
const resolve = (s, instance) => resolveInstance({ system: s.system, business: s.business, project: s.project, instance });
|
||||
|
||||
test('C2-04: a library first in projects does not hide a later project layer', async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const { rmSync } = await import('node:fs');
|
||||
const s = setup(doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; }, { vars: { 'tracker.project': 7 } });
|
||||
t.after(() => rmSync(s.root, { recursive: true, force: true }));
|
||||
const result = resolve(s, 'coder');
|
||||
assert.equal(result.project, 'stack');
|
||||
assert.equal(result.vars['tracker.project'], 7);
|
||||
});
|
||||
|
||||
test("systemVars flattens the validated config", () => {
|
||||
const vars = systemVars({ environment: "development", dataRoot: "/d", execution: { backend: "docker", provider: "zai", model: "m", adapter: "pi" } });
|
||||
assert.deepEqual({ ...vars }, {
|
||||
|
||||
@@ -59,6 +59,7 @@ function trackerFor(system, b, warn) {
|
||||
if (base["tracker.baseUrl"] === undefined) return null;
|
||||
const named = [];
|
||||
for (const [id, entry] of Object.entries(b.projects)) {
|
||||
if (entry.kind === 'library') continue;
|
||||
if (!existsSync(projectFilePath(entry.root))) continue;
|
||||
const project = loadProject(entry.root);
|
||||
if (project.id !== id) throw new CliError(`project file ${project.file} has id ${project.id}, but business ${b.id} declares it as ${id}`, 3);
|
||||
|
||||
@@ -351,7 +351,7 @@ export function createLauncher({
|
||||
mkdirSync(workspace, { recursive: true, mode: 0o700 });
|
||||
let checkouts;
|
||||
try {
|
||||
checkouts = await ensureCheckouts({ workspace, projects: b.projects, env });
|
||||
checkouts = await ensureCheckouts({ workspace, projects: Object.fromEntries(Object.entries(b.projects).filter(([, entry]) => entry.kind !== 'library')), env });
|
||||
} catch (e) {
|
||||
if (e instanceof CheckoutError) throw new Refusal("checkout-failed", `${e.project}: ${e.message}`);
|
||||
throw e;
|
||||
|
||||
@@ -29,6 +29,16 @@ const FAKE = fileURLToPath(new URL("../../harness/tests/fixtures/fake-adapter.mj
|
||||
const HOST = fileURLToPath(new URL("./fixtures/launch-host.mjs", import.meta.url));
|
||||
const skip = sandboxProblem() ?? false;
|
||||
|
||||
test('C2-04: launcher workspace clones code but never the library', { skip, timeout: 30000 }, async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const f = await setup(t, doc => { doc.projects = { books: mappingEntry(), ...doc.projects }; return doc; });
|
||||
await f.launcher.launchPm();
|
||||
const workspace = workspaceDir(f.f.dataRoot, 'acme', 'chief');
|
||||
assert.equal(existsSync(join(workspace, 'stack', '.git')), true);
|
||||
assert.equal(existsSync(join(workspace, 'books')), false);
|
||||
assert.equal(await f.close(), 0);
|
||||
});
|
||||
|
||||
// chief opus, coder and reviewer sonnet, one of each family; cto is listed but
|
||||
// runs the system model, which is in no family. The chief of staff is the launcher.
|
||||
function edit(doc) {
|
||||
|
||||
@@ -35,7 +35,7 @@ export function setup(t, realPin=false){
|
||||
test('shared copy: npm ci, pushed reachability, readonly/existing copy, webui delegation',t=>{
|
||||
const s=setup(t),r=s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]);assert.equal(r.status,0,r.stderr);
|
||||
const copy=join(s.copies,s.id);assert.equal(statSync(copy).mode&0o777,0o555);
|
||||
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
|
||||
for(const f of ['scripts/mosaic','adapters/pi/adapter.sh','packages/queue/src/cli.mjs','package.json','package-lock.json','packages/tasks/requirements.json','packages/project/src/index.mjs','packages/library/src/index.mjs','packages/project/schemas/project.schema.json','packages/project/templates/north-star.md','packages/plugins/src/kinds.mjs','packages/plugins/src/loader.mjs'])assert.ok(existsSync(join(copy,f)),f);
|
||||
const npm=join(s.bin,'npm');writeFileSync(npm,'#!/bin/sh\nexit 99\n');chmodSync(npm,0o755);
|
||||
const existing=s.run('webui-service.sh',['copy',s.id,'--copies',s.copies]);assert.equal(existing.status,0);assert.equal(existing.stdout,`exists: ${copy}\n`);unlinkSync(npm);
|
||||
git(s.repo,'update-ref','refs/remotes/origin/refactor',git(s.repo,'hash-object','-w','package.json'));
|
||||
|
||||
@@ -306,7 +306,7 @@ for(const [id,store] of [[F,'// a copy from before SCHEMA_VERSION\n'],[A,storeTe
|
||||
for(const [f,text,mode] of [['scripts/mosaic',script(mosaicStub),0o755],['packages/webui/src/cli.mjs',[B,C1,C].includes(id)?`// fixture Console, never run\n// serve accepts '--tester-port'\n// serve accepts '--proof-header' (row 178)\n${[C1,C].includes(id)?"// serve accepts '--sso' (row 192)\n":''}`:'// fixture Console before tester listener, never run\n',0o644],['packages/bus/src/store.mjs',store,0o644],['packages/bus/src/migrate-schema.mjs',`(${migrateStub})();\n`,0o644],['packages/plugins/src/kinds.mjs',[C1,C].includes(id)?KINDS_NEW:KINDS_OLD,0o644],...ssoFiles(id).map(([f,text])=>[f,text,0o644])]){
|
||||
mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text,{mode});
|
||||
}
|
||||
for(const p of ['packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
for(const p of ['packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
}
|
||||
for(const [name,fn] of [['systemctl',systemctl],['journalctl',journalctl],['systemd-analyze',systemdAnalyze]])writeFileSync(join(SBIN,name),script(fn),{mode:0o755});
|
||||
|
||||
@@ -1769,7 +1769,7 @@ export class Store {
|
||||
const RCOPIES=join(base,'rcopies');mkdirSync(RCOPIES);
|
||||
for(const [id,v,froms] of [[B,'3g',['3f']],[C,'3h',['3f','3g']],[A,'3g',['3f']],[C1,'3g',['3f']]]){
|
||||
const d=join(RCOPIES,id);
|
||||
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/bus/src/business.mjs','packages/plugins/src/kinds.mjs','packages/business/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/bus/src/business.mjs','packages/plugins/src/kinds.mjs','packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
|
||||
if(id===C1)writeFileSync(join(d,'scripts','mosaic'),`#!/usr/bin/env bash\nexec node "${join(RCOPIES,B,'packages','cli','src','cli.mjs')}" "$@"\n`,{mode:0o755});
|
||||
for(const [f,text] of [['packages/cli/src/cli.mjs',id===A?'// fixture host from before row 163\nprocess.exit(1);\n':script(rHost)],['packages/webui/src/cli.mjs',script(rConsole)],['packages/bus/src/store.mjs',rStore(v,froms)],['packages/bus/src/migrate-schema.mjs',script(rMigrate)],['packages/bus/src/broker.mjs','export class BusError extends Error { constructor(code) { super(code); this.code = code; } }\n']]){mkdirSync(dirname(join(d,f)),{recursive:true});writeFileSync(join(d,f),text)}
|
||||
execFileSync('chmod',['-R','a-w',d]);
|
||||
|
||||
@@ -12,7 +12,20 @@ import { FakeVikunja } from "../../tasks/src/fake.mjs";
|
||||
import { SCOPES } from "../../tasks/tests/world.mjs";
|
||||
import { fixture, tmp } from "./helpers.mjs";
|
||||
|
||||
test('C2-04: a library-only business with tracker.baseUrl boots without a project tracker', async t => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const f = fixture(tmp(t), 'acme', doc => { doc.projects = { books: mappingEntry() }; return doc; });
|
||||
const warnings = [];
|
||||
const boot = bootConfig({ system: loadSystem({ env: f.env }), businessId: 'acme', env: f.env, warn: text => warnings.push(text) });
|
||||
assert.equal(boot.trackers, undefined);
|
||||
assert.ok(warnings.some(text => text.includes('no project file sets tracker.project')));
|
||||
const host = await startHost({ boot, business: 'acme', log: () => {} });
|
||||
t.after(() => host.close(0));
|
||||
assert.equal(await host.close(0), 0);
|
||||
});
|
||||
|
||||
test("bootConfig trackers reach the S3 adapter in the real broker child, which goes ready against a fake Vikunja", async (t) => {
|
||||
const { mappingEntry } = await import('../../library/tests/helpers.mjs');
|
||||
const root = tmp(t);
|
||||
const fake = new FakeVikunja();
|
||||
const srv = await fake.listen();
|
||||
@@ -26,6 +39,7 @@ test("bootConfig trackers reach the S3 adapter in the real broker child, which g
|
||||
fake.install(project);
|
||||
|
||||
const f = fixture(root, "acme", (doc) => {
|
||||
doc.projects = { books: mappingEntry(), ...doc.projects };
|
||||
doc.vars["tracker.baseUrl"] = srv.url;
|
||||
doc.tracker.sync.botId = bots.sync;
|
||||
// The chief of staff (row 119) has no tracker bot.
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
# packages/library
|
||||
|
||||
Reads a writing repo by a mapping of folders to books, types, canon and playground. This is Case 2
|
||||
row C2-01 (`docs/plans/2026-10-20_case2-rows.md`), built from design sections 1.1 to 1.3 and 7.4 in
|
||||
`agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`.
|
||||
It is pure code with no state. Business file loading (C2-04), sync (C2-03a), the index (C2-05a) and the
|
||||
guard (C2-06) build on it.
|
||||
|
||||
## Mapping
|
||||
|
||||
`checkMapping(entry, where, { fail })` checks a `kind: "library"` project entry and returns a
|
||||
deep-frozen copy. `where` names the entry in messages. `fail(code, message)` defaults to throwing a
|
||||
`LibraryError`. C2-04 passes its own so a business file refusal carries the file name.
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "library",
|
||||
"remote": "https://git.example.invalid/her/books.git",
|
||||
"branch": "main",
|
||||
"vaultPrefix": "books",
|
||||
"types": { "Characters": "character", "Places": "place", "Research": "research" },
|
||||
"manuscript": ["Chapters"],
|
||||
"universes": {
|
||||
"harbor": { "name": "Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"] }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `remote` is https with no credentials, query or fragment. `branch` is a plain ref name.
|
||||
- `vaultPrefix` is optional, one folder name.
|
||||
- `types` maps a folder name to `character`, `place`, `research`, `tracker` or `summary`.
|
||||
`manuscript` lists folder names and shares none with `types`.
|
||||
- A universe has `name`, `books` and `playground`. A hybrid adds `kind: "hybrid"`, two or more
|
||||
non-hybrid `sources`, `constraints` (each one `{ "exclude": ... }` or
|
||||
`{ "pin": { "entity", "field", "value" } }`) and `notes`.
|
||||
|
||||
Refusals:
|
||||
|
||||
| Code | When |
|
||||
|---|---|
|
||||
| `library-playground-outside-book` | a playground prefix is not strictly inside a book of its own universe |
|
||||
| `library-mapping-invalid` | any other shape fault: unknown key, bad remote or branch, a book folder in two universes, a book folder that holds another, a playground prefix that holds another, an unknown hybrid source |
|
||||
|
||||
## Scope
|
||||
|
||||
`classify(mapping, path)` decides scope from the path alone. Only paths ending in lowercase `.md` are
|
||||
mapped. A path under a playground prefix is `playground`. Any other path under a book folder is
|
||||
`canon`. Everything else is `unmapped`. The type is the first folder after the book, or after the
|
||||
playground prefix: a `types` name, `manuscript`, or `null`.
|
||||
|
||||
## Reader
|
||||
|
||||
`readLibrary({ repo, commit, mapping })` reads one commit, never a working tree. `repo` may be a
|
||||
clone or a bare mirror. `commit` must be a full 40-hex id that names a commit, else `library-commit`.
|
||||
Git runs through `git()` and `gitBytes()` from `packages/project/src/git.mjs`, so the same safe config,
|
||||
environment and ceiling directory apply. A failed git call refuses `library-git`.
|
||||
|
||||
It returns `{ commit, files, problems }`, deep-frozen and sorted. Each file has its path, scope,
|
||||
universe, book, type, numbered lines, `bodyLine`, parsed `frontmatter` or `null`, `frontmatterRaw`
|
||||
and `links`. Front matter `appears_in` and `relationships` wikilinks count as links.
|
||||
|
||||
Problems carry `{ code, path, line, message }`:
|
||||
|
||||
| Rule | Meaning | File indexed |
|
||||
|---|---|---|
|
||||
| R1 | not a regular `100644` blob of at most 1 MiB, or not UTF-8, or holds a NUL | no |
|
||||
| R2 | path is unmapped | no |
|
||||
| R3 | front matter not fully parsed, `line` is where parsing stopped | yes, front matter kept raw |
|
||||
| R4 | link resolves to no file or to more than one | yes, link dropped |
|
||||
| R5 | canon file links into the playground | yes |
|
||||
| R6 | canon file links into another universe, a hybrid's sources excepted | yes |
|
||||
| R7 | two paths differ only in case | no, neither |
|
||||
|
||||
The front matter parser takes scalars, quoted strings, flow lists, block lists and wikilinks. Nested
|
||||
maps, anchors, block scalars and duplicate keys stop it with R3.
|
||||
|
||||
## Links
|
||||
|
||||
`resolver(paths, { vaultPrefix })` tries four steps in order, each with the target as written and
|
||||
then with `.md` added:
|
||||
|
||||
1. `root`: the path from the repo root.
|
||||
2. `vault-prefix`: the same path with `vaultPrefix/` stripped.
|
||||
3. `relative`: relative to the linking file, never above the repo root.
|
||||
4. `shortest`: a target with no `/` that matches the base name of exactly one indexed file.
|
||||
|
||||
It resolves only against indexed files. A link to a `.canvas` file or an attachment is R4. A link into
|
||||
another repo resolves to nothing. `extractLinks` skips fenced and inline code.
|
||||
|
||||
## renderItems
|
||||
|
||||
`renderItems(body, { scopeOf, changed })` turns a reply body into the plain text both channels show.
|
||||
It is pure. Item kinds are `fact`, `finding`, `correction` (with `from` and `to`), `question` and
|
||||
`idea`. Each cite is a path with commit, line range and quote, or a url with a quote.
|
||||
|
||||
```
|
||||
1. Correction: Chapter 1 says grey.
|
||||
From: "green eyes"
|
||||
To: "grey eyes"
|
||||
[canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes" (changed since)
|
||||
```
|
||||
|
||||
A body of `{ "items": [], "notice": ... }` gives `No cited answer`, `No answer` or `The turn failed`.
|
||||
Any other shape gives the fixed line `This reply could not be shown.` and none of the body's text.
|
||||
|
||||
## Survey
|
||||
|
||||
```sh
|
||||
mosaic library survey <repo> [--mapping <file>]
|
||||
```
|
||||
|
||||
It reads the repo's HEAD commit and prints folder names, file counts by extension, front matter key
|
||||
sets and link-form counts. It never prints file text, front matter values, file names or link targets.
|
||||
A key that is not a short identifier prints as `(other key)`, since it could be her words. With
|
||||
`--mapping` it adds scope counts, resolver step counts and problem counts by rule.
|
||||
|
||||
Exit codes follow `packages/project`: 0 done, 2 the repo can't be read (`library-git`,
|
||||
`library-commit`), 3 a bad mapping, 4 usage. `scripts/mosaic` sends `library survey` here. Other
|
||||
library verbs will be bus verbs.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
node --test packages/library/tests/
|
||||
```
|
||||
|
||||
Fixtures are made-up repos built per test in a temp dir. A marker string stands in for her text, and
|
||||
the survey tests check that it never appears in output.
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"name": "@mosaic/library",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"license": "UNLICENSED",
|
||||
"exports": "./src/index.mjs",
|
||||
"scripts": {
|
||||
"test": "node --test tests/*.test.mjs"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
// packages/library: her writing repo read by a mapping (Case 2, design 1 and 7.4). See README.md.
|
||||
export { LibraryError, RULES, problem, sortProblems } from './problems.mjs';
|
||||
export { checkMapping, TYPES } from './mapping.mjs';
|
||||
export { classify } from './scope.mjs';
|
||||
export { extractLinks, parseWikilink, resolver, formOf } from './links.mjs';
|
||||
export { readLibrary, listTree, readTexts, parseFile, parseFrontMatter, headCommit, MAX_BLOB, LINK_FIELDS } from './reader.mjs';
|
||||
export { renderItems, KINDS, NOTICES, UNREADABLE } from './render.mjs';
|
||||
export { survey, formatSurvey, surveyMain } from './survey.mjs';
|
||||
export { SYNC_STATE_FILE, mirrorPath, ensureMirror, withLibraryLock, servedCommit, readSyncState, libraryStatus, readLibraryFile } from './mirror.mjs';
|
||||
@@ -0,0 +1,79 @@
|
||||
// Wikilinks and the link resolver of design 1.3. Links are [[target]], [[target|label]] and
|
||||
// [[target#heading]], with ![[...]] embeds read the same way. Links inside fenced code blocks and
|
||||
// inline code spans are not links, as in Obsidian.
|
||||
import { posix } from 'node:path';
|
||||
|
||||
const WIKILINK = /(!?)\[\[([^\[\]\n]+?)\]\]/g;
|
||||
const MARKDOWN_LINK = /(?<!!)\[[^\[\]\n]*\]\(([^()\s]+)\)/g;
|
||||
const FENCE = /^\s{0,3}(`{3,}|~{3,})/;
|
||||
|
||||
// One wikilink's parts, or null for text that isn't one.
|
||||
export function parseWikilink(text) {
|
||||
const match = /^(!?)\[\[([^\[\]\n]+?)\]\]$/.exec(text);
|
||||
return match ? parts(match[2], match[1] === '!') : null;
|
||||
}
|
||||
function parts(inner, embed) {
|
||||
const bar = inner.indexOf('|');
|
||||
const target = bar < 0 ? inner : inner.slice(0, bar);
|
||||
const hash = target.indexOf('#');
|
||||
return {
|
||||
file: (hash < 0 ? target : target.slice(0, hash)).trim(),
|
||||
heading: hash < 0 ? null : target.slice(hash + 1).trim(),
|
||||
label: bar < 0 ? null : inner.slice(bar + 1),
|
||||
embed,
|
||||
};
|
||||
}
|
||||
// The form a link is counted under in the survey.
|
||||
export const formOf = link => link.markdown ? 'markdown' : link.embed ? 'embed' : !link.file ? 'self'
|
||||
: link.label !== null ? 'label' : link.heading !== null ? 'heading' : 'plain';
|
||||
|
||||
// Links in body lines. `lines` is [[number, text], ...]. Markdown links are returned only with
|
||||
// `markdown: true`, for the survey's counts. The resolver follows wikilinks only.
|
||||
export function extractLinks(lines, { markdown = false } = {}) {
|
||||
const found = [];
|
||||
let fence = null;
|
||||
for (const [line, text] of lines) {
|
||||
const open = FENCE.exec(text);
|
||||
if (fence) { if (open && open[1][0] === fence[0] && open[1].length >= fence.length) fence = null; continue; }
|
||||
if (open) { fence = open[1]; continue; }
|
||||
const plain = text.replace(/(`+)[^`]*?\1/g, match => ' '.repeat(match.length));
|
||||
for (const m of plain.matchAll(WIKILINK)) found.push({ ...parts(m[2], m[1] === '!'), line });
|
||||
if (markdown) {
|
||||
for (const m of plain.matchAll(MARKDOWN_LINK)) {
|
||||
if (!/^[a-z][a-z0-9+.-]*:/i.test(m[1]) && !m[1].startsWith('#'))
|
||||
found.push({ file: m[1], heading: null, label: null, embed: false, markdown: true, line });
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
// A resolver over the indexed paths. resolve(from, file) tries, in order: a path from the repo
|
||||
// root, the same path with `vaultPrefix/` stripped, a path relative to the linking file, and
|
||||
// Obsidian's shortest-path form, a base name that matches exactly one indexed file. Each step tries
|
||||
// the target as written and then with `.md` added. A target in another of her repos is not in the
|
||||
// index, so it resolves to nothing.
|
||||
export function resolver(paths, { vaultPrefix } = {}) {
|
||||
const known = new Set(paths), byBase = new Map();
|
||||
for (const path of known) {
|
||||
const base = posix.basename(path);
|
||||
for (const key of new Set([base, base.replace(/\.md$/, '')])) byBase.set(key, [...(byBase.get(key) ?? []), path]);
|
||||
}
|
||||
const hit = candidate => [candidate, `${candidate}.md`].find(c => known.has(c));
|
||||
return function resolve(from, file) {
|
||||
if (typeof file !== 'string' || !file) return { path: null, step: 'missing' };
|
||||
let path = hit(file);
|
||||
if (path) return { path, step: 'root' };
|
||||
if (vaultPrefix && file.startsWith(`${vaultPrefix}/`) && (path = hit(file.slice(vaultPrefix.length + 1))))
|
||||
return { path, step: 'vault-prefix' };
|
||||
const relative = posix.normalize(posix.join(posix.dirname(from), file));
|
||||
if (relative !== '..' && !relative.startsWith('../') && !relative.startsWith('/') && (path = hit(relative)))
|
||||
return { path, step: 'relative' };
|
||||
if (!file.includes('/')) {
|
||||
const matches = byBase.get(file) ?? [];
|
||||
if (matches.length === 1) return { path: matches[0], step: 'shortest' };
|
||||
if (matches.length > 1) return { path: null, step: 'ambiguous', count: matches.length };
|
||||
}
|
||||
return { path: null, step: 'missing' };
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
// The library project entry of a business file (design 1.1): her folders mapped onto universes,
|
||||
// books, type folders, manuscript folders and playground prefixes. A pure validator. Business file
|
||||
// loading wires it in C2-04, which passes its own `fail` so the refusal reads as a business file
|
||||
// problem. Agents can't write this entry (M9, C2-04).
|
||||
import { raise } from './problems.mjs';
|
||||
|
||||
export const TYPES = Object.freeze(['character', 'place', 'research', 'tracker', 'summary']);
|
||||
const ID = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
|
||||
const BRANCH = /^[A-Za-z0-9][A-Za-z0-9._/-]{0,127}$/;
|
||||
const CONSTRAINT_KEYS = ['exclude', 'pin'];
|
||||
|
||||
// A folder segment from her vault: spaces and capitals are fine, separators, control characters
|
||||
// and dot names are not.
|
||||
function segment(value) {
|
||||
return typeof value === 'string' && value.length > 0 && value.length <= 255 && value !== '.' && value !== '..'
|
||||
&& !/[\/\\\u0000-\u001f\u007f]/.test(value) && value.trim() === value;
|
||||
}
|
||||
function folder(value) {
|
||||
return typeof value === 'string' && value.length <= 1024 && value.split('/').every(segment);
|
||||
}
|
||||
const holds = (outer, inner) => inner.startsWith(outer.endsWith('/') ? outer : `${outer}/`);
|
||||
|
||||
export function checkMapping(entry, where = 'library', { fail = raise } = {}) {
|
||||
const bad = message => fail('library-mapping-invalid', `${where} ${message}`);
|
||||
const object = (value, name) => {
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) bad(`${name} must be an object`);
|
||||
};
|
||||
const only = (value, keys, name) => {
|
||||
for (const key of Object.keys(value)) if (!keys.includes(key)) bad(`${name} has unknown key ${JSON.stringify(key)}`);
|
||||
};
|
||||
const list = (value, name, check, what) => {
|
||||
if (!Array.isArray(value)) bad(`${name} must be a list`);
|
||||
for (const item of value) if (!check(item)) bad(`${name} holds ${JSON.stringify(item)}, which is not ${what}`);
|
||||
if (new Set(value).size !== value.length) bad(`${name} names an entry twice`);
|
||||
return [...value];
|
||||
};
|
||||
|
||||
object(entry, '');
|
||||
only(entry, ['kind', 'remote', 'branch', 'vaultPrefix', 'universes', 'types', 'manuscript'], 'entry');
|
||||
if (entry.kind !== 'library') bad('kind must be "library"');
|
||||
let url;
|
||||
try { url = new URL(entry.remote); } catch { bad('remote must be an https:// URL'); }
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.search || url.hash || entry.remote.includes('\0'))
|
||||
bad('remote must be an https:// URL with no credentials, query or fragment');
|
||||
if (typeof entry.branch !== 'string' || !BRANCH.test(entry.branch) || entry.branch.includes('..')
|
||||
|| entry.branch.includes('//') || entry.branch.endsWith('/') || entry.branch.endsWith('.lock'))
|
||||
bad('branch must be a plain branch name');
|
||||
if (Object.hasOwn(entry, 'vaultPrefix') && !segment(entry.vaultPrefix)) bad('vaultPrefix must be one folder name');
|
||||
|
||||
object(entry.types, 'types');
|
||||
for (const [name, type] of Object.entries(entry.types)) {
|
||||
if (!segment(name)) bad(`types has ${JSON.stringify(name)}, which is not one folder name`);
|
||||
if (!TYPES.includes(type)) bad(`types.${name} must be one of ${TYPES.join(', ')}`);
|
||||
}
|
||||
const manuscript = list(entry.manuscript, 'manuscript', segment, 'one folder name');
|
||||
for (const name of manuscript) if (Object.hasOwn(entry.types, name)) bad(`manuscript folder ${JSON.stringify(name)} is also a type folder`);
|
||||
|
||||
object(entry.universes, 'universes');
|
||||
if (!Object.keys(entry.universes).length) bad('universes must name at least one universe');
|
||||
const universes = {}, bookOwner = new Map(), prefixes = [];
|
||||
for (const [id, universe] of Object.entries(entry.universes)) {
|
||||
const name = `universes.${id}`;
|
||||
if (!ID.test(id)) bad(`universe id ${JSON.stringify(id)} is not a plain id`);
|
||||
object(universe, name);
|
||||
const hybrid = universe.kind === 'hybrid';
|
||||
only(universe, hybrid ? ['kind', 'name', 'sources', 'books', 'playground', 'constraints', 'notes'] : ['name', 'books', 'playground'], name);
|
||||
if (typeof universe.name !== 'string' || !universe.name.trim() || universe.name.length > 128 || /[\u0000-\u001f]/.test(universe.name))
|
||||
bad(`${name}.name must be a short display name`);
|
||||
const books = list(universe.books, `${name}.books`, folder, 'a folder path');
|
||||
if (!hybrid && !books.length) bad(`${name}.books must name at least one book folder`);
|
||||
for (const book of books) {
|
||||
if (bookOwner.has(book)) bad(`book folder ${JSON.stringify(book)} is in two universes`);
|
||||
bookOwner.set(book, id);
|
||||
}
|
||||
const playground = list(universe.playground ?? [], `${name}.playground`,
|
||||
value => typeof value === 'string' && value.endsWith('/') && folder(value.slice(0, -1)), 'a folder path ending in /');
|
||||
for (const prefix of playground) {
|
||||
// Every playground file must have a universe and a book (1.1, B1).
|
||||
if (!books.some(book => holds(book, prefix) && prefix !== `${book}/`))
|
||||
fail('library-playground-outside-book', `${where} ${name}.playground ${JSON.stringify(prefix)} is not inside one of its books`);
|
||||
prefixes.push(prefix);
|
||||
}
|
||||
const out = { name: universe.name, books, playground };
|
||||
if (hybrid) {
|
||||
out.kind = 'hybrid';
|
||||
out.sources = list(universe.sources, `${name}.sources`, value => typeof value === 'string' && ID.test(value), 'a universe id');
|
||||
if (out.sources.length < 2) bad(`${name}.sources must name two or more universes`);
|
||||
if (!Array.isArray(universe.constraints ?? [])) bad(`${name}.constraints must be a list`);
|
||||
out.constraints = (universe.constraints ?? []).map((constraint, i) => {
|
||||
const at = `${name}.constraints[${i}]`;
|
||||
object(constraint, at);
|
||||
only(constraint, CONSTRAINT_KEYS, at);
|
||||
if (Object.keys(constraint).length !== 1) bad(`${at} must hold exactly one of ${CONSTRAINT_KEYS.join(', ')}`);
|
||||
if (Object.hasOwn(constraint, 'exclude')) {
|
||||
if (!folder(constraint.exclude)) bad(`${at}.exclude must be a source path`);
|
||||
return { exclude: constraint.exclude };
|
||||
}
|
||||
object(constraint.pin, `${at}.pin`);
|
||||
only(constraint.pin, ['entity', 'field', 'value'], `${at}.pin`);
|
||||
if (!folder(constraint.pin.entity) || typeof constraint.pin.field !== 'string' || !constraint.pin.field
|
||||
|| !['string', 'number', 'boolean'].includes(typeof constraint.pin.value))
|
||||
bad(`${at}.pin needs an entity path, a field name and a scalar value`);
|
||||
return { pin: { ...constraint.pin } };
|
||||
});
|
||||
if (Object.hasOwn(universe, 'notes') && (typeof universe.notes !== 'string' || universe.notes.length > 4000))
|
||||
bad(`${name}.notes must be text of at most 4000 characters`);
|
||||
if (Object.hasOwn(universe, 'notes')) out.notes = universe.notes;
|
||||
}
|
||||
universes[id] = out;
|
||||
}
|
||||
for (const [id, universe] of Object.entries(universes)) {
|
||||
for (const source of universe.sources ?? []) {
|
||||
if (!Object.hasOwn(universes, source) || universes[source].kind === 'hybrid')
|
||||
bad(`universes.${id}.sources names ${JSON.stringify(source)}, which is not a non-hybrid universe`);
|
||||
}
|
||||
}
|
||||
const books = [...bookOwner.keys()];
|
||||
for (const a of books) for (const b of books) if (a !== b && holds(a, b)) bad(`book folder ${JSON.stringify(a)} holds book folder ${JSON.stringify(b)}`);
|
||||
for (const a of prefixes) for (const b of prefixes) if (a !== b && holds(a, b)) bad(`playground ${JSON.stringify(a)} holds playground ${JSON.stringify(b)}`);
|
||||
if (new Set(prefixes).size !== prefixes.length) bad('a playground prefix is listed twice');
|
||||
|
||||
return deepFreeze({
|
||||
kind: 'library', remote: entry.remote, branch: entry.branch,
|
||||
...(Object.hasOwn(entry, 'vaultPrefix') ? { vaultPrefix: entry.vaultPrefix } : {}),
|
||||
universes, types: { ...entry.types }, manuscript,
|
||||
});
|
||||
}
|
||||
|
||||
function deepFreeze(value) {
|
||||
if (value && typeof value === 'object') { for (const item of Object.values(value)) deepFreeze(item); Object.freeze(value); }
|
||||
return value;
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
// C2-04: host-only bare library storage. No remote, fetch, credential or
|
||||
// working tree. The broker receives these functions through the runtime.
|
||||
import { constants, closeSync, existsSync, fstatSync, lstatSync, mkdirSync, mkdtempSync,
|
||||
openSync, readSync, readdirSync, realpathSync, rmSync } from 'node:fs';
|
||||
import { dirname, isAbsolute, join, normalize } from 'node:path';
|
||||
import { execFile, spawn } from 'node:child_process';
|
||||
import { projectRepo, noLinks } from '../../project/src/paths.mjs';
|
||||
import { git } from '../../project/src/git.mjs';
|
||||
import { checkMapping } from './mapping.mjs';
|
||||
import { classify } from './scope.mjs';
|
||||
import { LibraryError, RULES } from './problems.mjs';
|
||||
import { listTree, readLibrary, readTexts } from './reader.mjs';
|
||||
|
||||
export const SYNC_STATE_FILE = 'mosaic-library-sync.json';
|
||||
const LOCK_FILE = 'mosaic-library-sync.lock';
|
||||
const STATE_MAX = 8 * 1024 * 1024;
|
||||
const SHA = /^[a-f0-9]{40}$/;
|
||||
const fail = (code, message = code) => { throw new LibraryError(code, message); };
|
||||
const object = value => value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
const keys = (value, expected) => object(value) && Object.keys(value).length === expected.length
|
||||
&& Object.keys(value).every(key => expected.includes(key));
|
||||
const optional = (value, check) => value === null || check(value);
|
||||
const utc = value => {
|
||||
if (typeof value !== 'string') return false;
|
||||
try { return new Date(value).toISOString() === value; } catch { return false; }
|
||||
};
|
||||
const empty = branch => ({ version: 1, branch, served: null, lastFetch: null, lastError: null, diverged: null, problems: [] });
|
||||
|
||||
function contextEntry(context) {
|
||||
const { business, project } = context;
|
||||
if (!Object.hasOwn(business?.projects ?? {}, project)) fail('project-not-declared');
|
||||
const entry = business.projects[project];
|
||||
if (entry?.kind !== 'library') fail('library-mapping-invalid', 'expected a library project');
|
||||
return checkMapping(entry);
|
||||
}
|
||||
|
||||
export function mirrorPath(context) {
|
||||
contextEntry(context);
|
||||
try { return projectRepo(context, { allowLibrary: true }); }
|
||||
catch (error) { throw new LibraryError(error.code ?? 'library-git', 'unsafe mirror path'); }
|
||||
}
|
||||
|
||||
function ownedRoot(mirror) {
|
||||
try {
|
||||
if (typeof mirror !== 'string' || !isAbsolute(mirror) || normalize(mirror) !== mirror) throw Error();
|
||||
noLinks(mirror);
|
||||
const st = lstatSync(mirror);
|
||||
if (!st.isDirectory() || st.uid !== process.getuid() || (st.mode & 0o077) || realpathSync(mirror) !== mirror) throw Error();
|
||||
} catch { fail('library-git', 'expected a private owned bare mirror'); }
|
||||
}
|
||||
|
||||
function privateFile(st) {
|
||||
return st.isFile() && st.uid === process.getuid() && st.nlink === 1 && (st.mode & 0o077) === 0;
|
||||
}
|
||||
|
||||
async function inspectMirrorUnchecked(mirror) {
|
||||
ownedRoot(mirror);
|
||||
const normal = ['HEAD', 'config', 'description', 'hooks', 'info', 'objects', 'refs', 'logs', 'packed-refs', 'shallow', 'FETCH_HEAD', 'ORIG_HEAD'];
|
||||
const allowed = new Set([...normal, ...normal.filter(name => !['hooks', 'info', 'objects', 'refs', 'logs', 'description'].includes(name)).map(name => name + '.lock'), SYNC_STATE_FILE, LOCK_FILE]);
|
||||
const top = readdirSync(mirror);
|
||||
if (top.some(name => !allowed.has(name)) || ['HEAD', 'config', 'objects', 'refs'].some(name => !top.includes(name)))
|
||||
fail('library-git', 'unexpected bare mirror layout');
|
||||
for (const path of ['commondir', 'objects/info/alternates', 'objects/info/http-alternates'])
|
||||
if (existsSync(join(mirror, path))) fail('library-git', 'Git metadata indirection refused');
|
||||
const walk = (folder, topLevel = false) => {
|
||||
for (const name of readdirSync(folder)) {
|
||||
const file = join(folder, name); let st;
|
||||
try { st = lstatSync(file); }
|
||||
catch (error) { if (error.code === 'ENOENT') continue; throw error; }
|
||||
const state = topLevel && [SYNC_STATE_FILE, LOCK_FILE].includes(name);
|
||||
if (state) { if (!privateFile(st)) fail('library-state-invalid', 'unsafe state or lock metadata'); }
|
||||
else if (st.isDirectory() && !st.isSymbolicLink()) walk(file);
|
||||
else if (!st.isFile() || st.nlink !== 1 || st.uid !== process.getuid()) fail('library-git', 'unsafe Git metadata');
|
||||
}
|
||||
};
|
||||
walk(mirror, true);
|
||||
if (!top.includes(LOCK_FILE)) fail('library-state-invalid', 'mirror lock file missing');
|
||||
if (lstatSync(join(mirror, 'config')).size > 65536) fail('library-git', 'Git config too large');
|
||||
let config;
|
||||
try { config = await git(mirror, ['config', '--no-includes', '--file', join(mirror, 'config'), '--list', '--null']); }
|
||||
catch { fail('library-git', 'cannot inspect mirror config'); }
|
||||
const values = { 'core.repositoryformatversion': ['0'], 'core.filemode': ['true', 'false'], 'core.bare': ['true'],
|
||||
'core.logallrefupdates': ['true', 'false'], 'core.ignorecase': ['true', 'false'] };
|
||||
let bare = false;
|
||||
for (const item of config.split('\0').filter(Boolean)) {
|
||||
const at = item.indexOf('\n'), key = item.slice(0, at), value = item.slice(at + 1);
|
||||
if (key === 'core.bare' && value === 'true') bare = true;
|
||||
if (at < 0 || !values[key]?.includes(value)) fail('library-git', 'unsupported bare mirror config');
|
||||
}
|
||||
if (!bare) fail('library-git', 'expected a bare repository');
|
||||
}
|
||||
|
||||
async function inspectMirror(mirror) {
|
||||
try { await inspectMirrorUnchecked(mirror); }
|
||||
catch (error) {
|
||||
if (error instanceof LibraryError) throw error;
|
||||
fail('library-git', 'unreadable bare mirror metadata');
|
||||
}
|
||||
}
|
||||
|
||||
export async function ensureMirror(context) {
|
||||
const mirror = mirrorPath(context);
|
||||
if (existsSync(mirror)) { await inspectMirror(mirror); return mirror; }
|
||||
const parent = dirname(mirror); mkdirSync(parent, { recursive: true, mode: 0o700 }); noLinks(parent);
|
||||
const staged = mkdtempSync(join(parent, '.library-init-'));
|
||||
const original = lstatSync(staged);
|
||||
const isOwnStage = st => st.isDirectory() && st.uid === process.getuid() && st.dev === original.dev && st.ino === original.ino;
|
||||
try {
|
||||
await git(staged, ['init', '--bare', '--initial-branch=served', '--template=']);
|
||||
const fd = openSync(join(staged, LOCK_FILE), constants.O_CREAT | constants.O_EXCL | constants.O_WRONLY | constants.O_NOFOLLOW, 0o600);
|
||||
closeSync(fd);
|
||||
await inspectMirror(staged);
|
||||
// Coreutils versions disagree on the status of a skipped no-clobber
|
||||
// rename. The original inode, rather than that status, decides who won.
|
||||
const moveError = await new Promise(resolve => execFile('/usr/bin/mv', ['--no-clobber', '--no-target-directory', staged, mirror],
|
||||
{ timeout: 5000, maxBuffer: 65536, env: { PATH: '/usr/bin:/bin', LC_ALL: 'C' } }, error => resolve(error)));
|
||||
let remaining;
|
||||
try { remaining = lstatSync(staged); } catch (error) { if (error.code !== 'ENOENT') throw error; }
|
||||
if (remaining) {
|
||||
if (!isOwnStage(remaining)) fail('library-git', 'staging directory was replaced');
|
||||
// A skipped rename is valid only when another intact mirror won.
|
||||
// Signals, timeouts and spawn failures are always genuine errors.
|
||||
if (moveError && (typeof moveError.code !== 'number' || moveError.killed || moveError.signal)) throw moveError;
|
||||
} else {
|
||||
if (!isOwnStage(lstatSync(mirror))) fail('library-git', 'mirror publication identity mismatch');
|
||||
if (moveError) throw moveError;
|
||||
}
|
||||
await inspectMirror(mirror);
|
||||
return mirror;
|
||||
} catch (error) {
|
||||
if (error instanceof LibraryError) throw error;
|
||||
fail('library-git', 'bare mirror initialization refused');
|
||||
} finally {
|
||||
let remaining;
|
||||
try { remaining = lstatSync(staged); } catch (error) { if (error.code !== 'ENOENT') throw error; }
|
||||
if (remaining) {
|
||||
if (!isOwnStage(remaining)) fail('library-git', 'refusing to remove replaced staging directory');
|
||||
rmSync(staged, { recursive: true });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Each acquisition opens a distinct description. flock(1) inherits it as
|
||||
// fd 3 and exits after acquisition; this parent's fd retains the lock.
|
||||
export async function withLibraryLock(mirror, operation) {
|
||||
ownedRoot(mirror);
|
||||
let fd;
|
||||
try {
|
||||
try {
|
||||
fd = openSync(join(mirror, LOCK_FILE), constants.O_RDWR | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
if (!privateFile(fstatSync(fd))) fail('library-state-invalid', 'unsafe library lock');
|
||||
} catch { fail('library-state-invalid', 'library lock missing or unsafe'); }
|
||||
await new Promise((resolve, reject) => {
|
||||
const child = spawn('/usr/bin/flock', ['--exclusive', '--wait', '5', '--conflict-exit-code', '75', '3'],
|
||||
{ env: { PATH: '/usr/bin:/bin', LC_ALL: 'C' }, stdio: ['ignore', 'ignore', 'ignore', fd] });
|
||||
child.once('error', () => reject(new LibraryError('library-state-invalid', 'host requires /usr/bin/flock')));
|
||||
child.once('close', code => code === 0 ? resolve() : reject(new LibraryError(code === 75 ? 'library-sync-busy' : 'library-state-invalid', 'library lock acquisition refused')));
|
||||
});
|
||||
try {
|
||||
const heldStat = fstatSync(fd), pathStat = lstatSync(join(mirror, LOCK_FILE));
|
||||
if (!privateFile(pathStat) || pathStat.ino !== heldStat.ino || pathStat.dev !== heldStat.dev) throw Error();
|
||||
} catch { fail('library-state-invalid', 'library lock was replaced'); }
|
||||
let held = true;
|
||||
const pending = new Set();
|
||||
const readSyncState = options => {
|
||||
if (!held) return Promise.reject(new LibraryError('library-state-invalid', 'library lock scope expired'));
|
||||
const read = readStateHeld(mirror, options).finally(() => pending.delete(read));
|
||||
pending.add(read); read.catch(() => {});
|
||||
return read;
|
||||
};
|
||||
try { return await operation({ readSyncState }); }
|
||||
finally {
|
||||
held = false;
|
||||
// An accidentally unawaited scoped read must finish before this fd
|
||||
// releases the lock. No new read can start after the callback ends.
|
||||
await Promise.allSettled([...pending]);
|
||||
}
|
||||
} finally { if (fd !== undefined) closeSync(fd); }
|
||||
}
|
||||
|
||||
export async function servedCommit(mirror) {
|
||||
await inspectMirror(mirror);
|
||||
try {
|
||||
const refs = await git(mirror, ['for-each-ref', '--format=%(refname) %(objectname)', 'refs/heads/served']);
|
||||
const line = refs.split('\n').find(line => line.startsWith('refs/heads/served '));
|
||||
if (!line) {
|
||||
if (existsSync(join(mirror, 'refs/heads/served'))) fail('library-state-invalid', 'invalid served ref');
|
||||
return null;
|
||||
}
|
||||
const commit = line.slice('refs/heads/served '.length);
|
||||
if (!SHA.test(commit) || await git(mirror, ['cat-file', '-t', commit]) !== 'commit') fail('library-state-invalid', 'served ref must name a commit');
|
||||
return commit;
|
||||
} catch { fail('library-state-invalid', 'served ref is unreadable'); }
|
||||
}
|
||||
|
||||
function validProblem(value) {
|
||||
return object(value) && Object.keys(value).every(key => ['code', 'path', 'message', 'line'].includes(key))
|
||||
&& Object.hasOwn(RULES, value.code) && value.message === RULES[value.code]
|
||||
&& typeof value.path === 'string' && value.path.length > 0 && value.path.length <= 4096 && !value.path.includes('\0')
|
||||
&& (!Object.hasOwn(value, 'line') || Number.isSafeInteger(value.line) && value.line > 0);
|
||||
}
|
||||
|
||||
async function readStateHeld(mirror, { branch } = {}) {
|
||||
if (typeof branch !== 'string' || !branch) fail('library-state-invalid', 'configured branch required');
|
||||
const served = await servedCommit(mirror);
|
||||
let fd, value;
|
||||
try {
|
||||
try { fd = openSync(join(mirror, SYNC_STATE_FILE), constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
|
||||
catch (error) {
|
||||
if (error.code === 'ENOENT' && served === null) return empty(branch);
|
||||
fail('library-state-invalid', 'sync state missing or unsafe');
|
||||
}
|
||||
const st = fstatSync(fd);
|
||||
if (!privateFile(st) || st.size > STATE_MAX) fail('library-state-invalid', 'sync state must be bounded and private');
|
||||
const bytes = Buffer.alloc(Math.min(st.size + 1, STATE_MAX + 1));
|
||||
let used = 0;
|
||||
while (used < bytes.length) { const read = readSync(fd, bytes, used, bytes.length - used, null); if (!read) break; used += read; }
|
||||
if (used > STATE_MAX || used > st.size) fail('library-state-invalid', 'state changed during bounded read');
|
||||
value = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes.subarray(0, used)));
|
||||
} catch { fail('library-state-invalid', 'invalid sync state'); }
|
||||
finally { if (fd !== undefined) closeSync(fd); }
|
||||
if (!keys(value, ['version', 'branch', 'served', 'lastFetch', 'lastError', 'diverged', 'problems'])
|
||||
|| value.version !== 1 || value.branch !== branch || value.served !== served
|
||||
|| !optional(value.served, v => typeof v === 'string' && SHA.test(v)) || !optional(value.lastFetch, utc)
|
||||
|| !optional(value.lastError, v => keys(v, ['code', 'at']) && typeof v.code === 'string' && /^[a-z-]{1,64}$/.test(v.code) && utc(v.at))
|
||||
|| !optional(value.diverged, v => keys(v, ['commit', 'at']) && typeof v.commit === 'string' && SHA.test(v.commit) && utc(v.at))
|
||||
|| !Array.isArray(value.problems) || !value.problems.every(validProblem))
|
||||
fail('library-state-invalid', 'sync state does not match its contract or served ref');
|
||||
return value;
|
||||
}
|
||||
|
||||
export async function readSyncState(mirror, options) {
|
||||
return withLibraryLock(mirror, lock => lock.readSyncState(options));
|
||||
}
|
||||
|
||||
export async function libraryStatus(context) {
|
||||
const mapping = contextEntry(context), mirror = mirrorPath(context);
|
||||
return { project: context.project, ...await readSyncState(mirror, { branch: mapping.branch }) };
|
||||
}
|
||||
|
||||
export async function readLibraryFile(context) {
|
||||
if (!object(context) || Object.keys(context).some(key => !['business', 'project', 'dataRoot', 'path'].includes(key)))
|
||||
fail('project-read-path', 'unknown library read arguments');
|
||||
const mapping = contextEntry(context), path = context.path;
|
||||
if (typeof path !== 'string' || !path || isAbsolute(path) || path.includes('\0') || path.split('/').some(p => !p || p === '.' || p === '..')
|
||||
|| classify(mapping, path).scope === 'unmapped') fail('project-read-path', 'expected a mapped library note');
|
||||
const mirror = mirrorPath(context);
|
||||
// Snapshot under the publication lock, then read only immutable objects.
|
||||
const state = await readSyncState(mirror, { branch: mapping.branch });
|
||||
if (!state.served) fail('library-commit', 'library has never been synced');
|
||||
const read = await readLibrary({ repo: mirror, commit: state.served, mapping });
|
||||
if (!read.files.some(file => file.path === path)) fail('project-read-missing', 'file is absent or excluded at served commit');
|
||||
const entry = (await listTree(mirror, state.served)).find(entry => entry.path === path);
|
||||
const texts = await readTexts(mirror, [entry]);
|
||||
if (!texts.has(path)) fail('project-read-missing', 'file is not readable');
|
||||
return { project: context.project, path, commit: state.served, content: texts.get(path) };
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
// Errors and the R-rule problems of design 1.3. A problem never holds back a commit: it names a
|
||||
// path and, where there is one, a line, and the index leaves the file or link out.
|
||||
export class LibraryError extends Error {
|
||||
constructor(code, message) { super(`${code}: ${message}`); this.code = code; }
|
||||
}
|
||||
export const raise = (code, message) => { throw new LibraryError(code, message); };
|
||||
|
||||
export const RULES = Object.freeze({
|
||||
R1: 'not a regular blob up to 1 MiB of UTF-8 text',
|
||||
R2: 'path is unmapped',
|
||||
R3: 'front matter not fully parsed',
|
||||
R4: 'link resolves to no file or to more than one',
|
||||
R5: 'canon file refers into the playground',
|
||||
R6: 'canon file refers into another universe',
|
||||
R7: 'paths differ only in case',
|
||||
});
|
||||
|
||||
export function problem(code, path, line, message = RULES[code]) {
|
||||
if (!Object.hasOwn(RULES, code)) raise('library-problem-code', `unknown problem code ${code}`);
|
||||
return Object.freeze({ code, path, ...(Number.isSafeInteger(line) ? { line } : {}), message });
|
||||
}
|
||||
|
||||
// Stable order for status output and tests: path, then line, then code.
|
||||
export function sortProblems(list) {
|
||||
return [...list].sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1
|
||||
: (a.line ?? 0) - (b.line ?? 0) || (a.code < b.code ? -1 : a.code > b.code ? 1 : 0));
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
// The commit reader of design 1.3. It reads a commit of her repo, never a working tree: it lists the
|
||||
// tree with `git ls-tree -r -z` and reads blobs with `cat-file`, through the same bounded git()
|
||||
// and gitBytes() as packages/project, so hooks, protocols and the environment are fixed the same
|
||||
// way. Nothing in the repo is checked out or run (D7). Each R-rule is a problem, never a refusal
|
||||
// of the commit; the affected file or link is left out of the index.
|
||||
import { git, gitBytes } from '../../project/src/git.mjs';
|
||||
import { classify } from './scope.mjs';
|
||||
import { extractLinks, parseWikilink, resolver } from './links.mjs';
|
||||
import { LibraryError, problem, raise, sortProblems } from './problems.mjs';
|
||||
|
||||
export const MAX_BLOB = 1024 * 1024;
|
||||
const MAX_LISTING = 32 * 1024 * 1024;
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
const ENTRY = /^([0-7]{6}) (blob|tree|commit) ([0-9a-f]{40}) +(-|\d+)\t(.+)$/s;
|
||||
export const LINK_FIELDS = Object.freeze(['appears_in', 'relationships']);
|
||||
const READERS = 8;
|
||||
|
||||
async function local(promise, message) {
|
||||
try { return await promise; } catch (e) { raise('library-git', `${message} (${e.code ?? 'error'})`); }
|
||||
}
|
||||
|
||||
export async function headCommit(repo) {
|
||||
return local(git(repo, ['rev-parse', '--verify', '--end-of-options', 'HEAD^{commit}']), 'repository has no readable HEAD commit');
|
||||
}
|
||||
|
||||
// Every entry of the commit's tree: { mode, type, oid, size, path }. Sizes come from `-l`, so no
|
||||
// blob is opened to list it.
|
||||
export async function listTree(repo, commit) {
|
||||
if (typeof commit !== 'string' || !SHA.test(commit)) raise('library-commit', 'expected a full commit id');
|
||||
const kind = await local(git(repo, ['cat-file', '-t', commit]), 'commit is not in the repository');
|
||||
if (kind !== 'commit') raise('library-commit', 'not a commit');
|
||||
const bytes = await local(gitBytes(repo, ['ls-tree', '-r', '-z', '-l', '--full-tree', commit], MAX_LISTING), 'tree listing refused');
|
||||
let text;
|
||||
try { text = new TextDecoder('utf-8', { fatal: true }).decode(bytes); }
|
||||
catch { raise('library-git', 'tree listing is not UTF-8'); }
|
||||
const entries = text.split('\0');
|
||||
if (entries.pop() !== '') raise('library-git', 'invalid tree listing');
|
||||
return entries.map(entry => {
|
||||
const m = ENTRY.exec(entry);
|
||||
if (!m) raise('library-git', 'invalid tree listing');
|
||||
return { mode: m[1], type: m[2], oid: m[3], size: m[4] === '-' ? null : Number(m[4]), path: m[5] };
|
||||
});
|
||||
}
|
||||
|
||||
// R1: only a 100644 blob up to 1 MiB of UTF-8 text without NUL is read.
|
||||
export const readable = entry => entry.mode === '100644' && entry.type === 'blob'
|
||||
&& Number.isSafeInteger(entry.size) && entry.size <= MAX_BLOB;
|
||||
|
||||
// Text of each readable entry, by path. An entry whose bytes aren't UTF-8 text is missing from the
|
||||
// map, and the caller reports it under R1.
|
||||
export async function readTexts(repo, entries) {
|
||||
const texts = new Map(), queue = [...entries];
|
||||
const decoder = new TextDecoder('utf-8', { fatal: true, ignoreBOM: false });
|
||||
const worker = async () => {
|
||||
for (let entry = queue.shift(); entry; entry = queue.shift()) {
|
||||
const bytes = await local(gitBytes(repo, ['cat-file', 'blob', entry.oid], MAX_BLOB), 'blob read refused');
|
||||
let value;
|
||||
try { value = decoder.decode(bytes); } catch { continue; }
|
||||
if (!value.includes('\0')) texts.set(entry.path, value);
|
||||
}
|
||||
};
|
||||
await Promise.all(Array.from({ length: READERS }, worker));
|
||||
return texts;
|
||||
}
|
||||
|
||||
const QUOTED = /^"(?:[^"\\]|\\.)*"$|^'(?:[^']|'')*'$/;
|
||||
const PLAIN_START = /^[^\s\[\]{}&*!|>%@`"'#,]/;
|
||||
function scalar(raw) {
|
||||
const value = raw.trim();
|
||||
if (parseWikilink(value)) return { ok: true, value };
|
||||
if (QUOTED.test(value)) {
|
||||
if (value[0] === "'") return { ok: true, value: value.slice(1, -1).replace(/''/g, "'") };
|
||||
try { return { ok: true, value: JSON.parse(value) }; } catch { return { ok: false }; }
|
||||
}
|
||||
if (value === '' || value === '~' || value === 'null') return { ok: true, value: null };
|
||||
if (!PLAIN_START.test(value) || /: |\s#/.test(value) || value.endsWith(':')) return { ok: false };
|
||||
if (value === 'true' || value === 'false') return { ok: true, value: value === 'true' };
|
||||
if (/^-?(0|[1-9]\d*)(\.\d+)?$/.test(value)) return { ok: true, value: Number(value) };
|
||||
return { ok: true, value };
|
||||
}
|
||||
function flow(raw) {
|
||||
const inner = raw.trim().slice(1, -1), items = [];
|
||||
let current = '', quote = null;
|
||||
for (const ch of inner) {
|
||||
if (quote) { current += ch; if (ch === quote) quote = null; continue; }
|
||||
if (ch === '"' || ch === "'") { quote = ch; current += ch; continue; }
|
||||
if (ch === '[' || ch === ']' || ch === '{' || ch === '}') return { ok: false };
|
||||
if (ch === ',') { items.push(current); current = ''; continue; }
|
||||
current += ch;
|
||||
}
|
||||
if (quote) return { ok: false };
|
||||
if (current.trim() || items.length) items.push(current);
|
||||
const values = [];
|
||||
for (const item of items) { const s = scalar(item); if (!s.ok || !item.trim()) return { ok: false }; values.push(s.value); }
|
||||
return { ok: true, value: values };
|
||||
}
|
||||
function value(raw) {
|
||||
const text = raw.trim();
|
||||
if (text.startsWith('[') && !parseWikilink(text)) return text.endsWith(']') ? flow(text) : { ok: false };
|
||||
return scalar(text);
|
||||
}
|
||||
|
||||
// Front matter of design 1.3 (R3): scalars, quoted strings, flow lists and block lists of scalars or
|
||||
// wikilinks. Anything else, a nested map, a block scalar or an anchor for example, leaves the whole
|
||||
// front matter as raw text with one R3 problem at the line where parsing stopped.
|
||||
// Returns { frontmatter, raw, bodyLine, stop } with 1-based line numbers.
|
||||
export function parseFrontMatter(lines) {
|
||||
if (lines[0]?.replace(/^/, '').trimEnd() !== '---') return { frontmatter: null, raw: null, bodyLine: 1, stop: null };
|
||||
const close = lines.findIndex((text, i) => i > 0 && /^(---|\.\.\.)\s*$/.test(text));
|
||||
if (close < 0) return { frontmatter: null, raw: null, bodyLine: 1, stop: 1 };
|
||||
const raw = lines.slice(1, close).join('\n'), data = {};
|
||||
let list = null;
|
||||
for (let i = 1; i < close; i++) {
|
||||
const text = lines[i], line = i + 1;
|
||||
if (!text.trim() || /^\s*#/.test(text)) continue;
|
||||
const item = /^(\s*)- (.*)$|^(\s*)-$/.exec(text);
|
||||
if (item && list) {
|
||||
const parsed = scalar(item[2] ?? '');
|
||||
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
data[list].push(parsed.value);
|
||||
continue;
|
||||
}
|
||||
const pair = /^([^\s:#\-][^:]*?|"[^"]+"|'[^']+')\s*:(?:\s+(.*))?$/.exec(text);
|
||||
if (!pair) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
const key = pair[1].replace(/^(["'])(.*)\1$/, '$2');
|
||||
if (Object.hasOwn(data, key) || key === '__proto__') return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
if (pair[2] === undefined || !pair[2].trim()) {
|
||||
const next = lines[i + 1];
|
||||
if (i + 1 < close && /^\s*-( |$)/.test(next)) { data[key] = []; list = key; }
|
||||
else { data[key] = null; list = null; }
|
||||
continue;
|
||||
}
|
||||
list = null;
|
||||
const parsed = value(pair[2]);
|
||||
if (!parsed.ok) return { frontmatter: null, raw, bodyLine: close + 2, stop: line };
|
||||
data[key] = parsed.value;
|
||||
}
|
||||
return { frontmatter: data, raw: null, bodyLine: close + 2, stop: null };
|
||||
}
|
||||
|
||||
export const splitLines = text => text.replace(/^/, '').split('\n').map(l => l.endsWith('\r') ? l.slice(0, -1) : l);
|
||||
|
||||
// One parsed file: its lines (all of them, numbered from 1), front matter and the links it holds.
|
||||
// Front matter links come from LINK_FIELDS values that are wikilinks; other values are names.
|
||||
export function parseFile(text, { markdown = false } = {}) {
|
||||
const lines = splitLines(text);
|
||||
const fm = parseFrontMatter(lines);
|
||||
const numbered = lines.map((t, i) => [i + 1, t]);
|
||||
const links = extractLinks(numbered.slice(fm.bodyLine - 1), { markdown });
|
||||
if (fm.frontmatter) {
|
||||
for (const field of LINK_FIELDS) {
|
||||
const raw = fm.frontmatter[field];
|
||||
const values = Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw];
|
||||
for (const v of values) {
|
||||
const link = typeof v === 'string' ? parseWikilink(v.trim()) : null;
|
||||
if (link) links.push({ ...link, field, line: frontMatterLine(lines, fm.bodyLine, field) });
|
||||
}
|
||||
}
|
||||
}
|
||||
return { lines: numbered, bodyLine: fm.bodyLine, frontmatter: fm.frontmatter, frontmatterRaw: fm.raw, stop: fm.stop, links };
|
||||
}
|
||||
function frontMatterLine(lines, bodyLine, field) {
|
||||
const i = lines.slice(0, bodyLine - 1).findIndex(t => new RegExp(`^["']?${field}["']?\\s*:`).test(t));
|
||||
return i < 0 ? undefined : i + 1;
|
||||
}
|
||||
|
||||
// Read a commit under a checked mapping (mapping.mjs). Returns the indexed files and every
|
||||
// R1-R7 problem: { commit, files: [...], problems: [...] }, frozen, sorted by path.
|
||||
export async function readLibrary({ repo, commit, mapping }) {
|
||||
if (!mapping || mapping.kind !== 'library') throw new LibraryError('library-mapping-invalid', 'expected a checked library mapping');
|
||||
const entries = await listTree(repo, commit);
|
||||
const problems = [], twins = new Set(), byLower = new Map();
|
||||
for (const e of entries) byLower.set(e.path.toLowerCase(), [...(byLower.get(e.path.toLowerCase()) ?? []), e.path]);
|
||||
for (const group of byLower.values()) if (group.length > 1) for (const path of group) twins.add(path);
|
||||
|
||||
const candidates = [];
|
||||
for (const entry of entries) {
|
||||
const where = classify(mapping, entry.path);
|
||||
if (where.scope === 'unmapped') { problems.push(problem('R2', entry.path)); continue; }
|
||||
if (twins.has(entry.path)) { problems.push(problem('R7', entry.path)); continue; }
|
||||
if (!readable(entry)) { problems.push(problem('R1', entry.path)); continue; }
|
||||
candidates.push({ entry, where });
|
||||
}
|
||||
const texts = await readTexts(repo, candidates.map(c => c.entry));
|
||||
const files = [];
|
||||
for (const { entry, where } of candidates) {
|
||||
if (!texts.has(entry.path)) { problems.push(problem('R1', entry.path)); continue; }
|
||||
const parsed = parseFile(texts.get(entry.path));
|
||||
if (parsed.stop !== null) problems.push(problem('R3', entry.path, parsed.stop));
|
||||
files.push({ path: entry.path, ...where, name: entry.path.split('/').pop().replace(/\.md$/, ''), ...parsed });
|
||||
}
|
||||
|
||||
const resolve = resolver(files.map(f => f.path), { vaultPrefix: mapping.vaultPrefix });
|
||||
const byPath = new Map(files.map(f => [f.path, f]));
|
||||
for (const file of files) {
|
||||
const kept = [];
|
||||
for (const link of file.links) {
|
||||
if (!link.file) continue;
|
||||
const target = resolve(file.path, link.file);
|
||||
if (!target.path) { problems.push(problem('R4', file.path, link.line)); continue; }
|
||||
const to = byPath.get(target.path);
|
||||
if (file.scope === 'canon') {
|
||||
if (to.scope === 'playground') problems.push(problem('R5', file.path, link.line));
|
||||
const sources = mapping.universes[file.universe].sources ?? [];
|
||||
if (to.universe !== file.universe && !sources.includes(to.universe)) problems.push(problem('R6', file.path, link.line));
|
||||
}
|
||||
kept.push({ ...link, path: target.path, step: target.step });
|
||||
}
|
||||
file.links = kept;
|
||||
delete file.stop;
|
||||
}
|
||||
files.sort((a, b) => a.path < b.path ? -1 : 1);
|
||||
return deepFreeze({ commit, files, problems: sortProblems(problems) });
|
||||
}
|
||||
|
||||
function deepFreeze(value) {
|
||||
if (value && typeof value === 'object' && !Object.isFrozen(value)) { Object.freeze(value); for (const v of Object.values(value)) deepFreeze(v); }
|
||||
return value;
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
// renderItems(body): how a stored cited reply looks (design 7.4), shared by the Console chat (C2-10b)
|
||||
// and the Discord relay (C2-18). Pure: no git, no clock. The caller passes what it knows about cites.
|
||||
// The guard (C2-06) checks limits before a reply is stored. This only refuses to show a body that
|
||||
// doesn't have the reply shape, and then it shows a fixed line, never the body's text.
|
||||
export const KINDS = Object.freeze({ fact: 'Fact', finding: 'Finding', correction: 'Correction', question: 'Question', idea: 'Idea' });
|
||||
export const NOTICES = Object.freeze({ 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' });
|
||||
export const UNREADABLE = 'This reply could not be shown.';
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
|
||||
const plain = value => value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
const line = n => Number.isSafeInteger(n) && n > 0;
|
||||
function cite(value, { scopeOf, changed }) {
|
||||
if (!plain(value) || typeof value.quote !== 'string') return null;
|
||||
if (Object.hasOwn(value, 'url')) {
|
||||
if (typeof value.url !== 'string' || Object.hasOwn(value, 'path')) return null;
|
||||
return { badge: 'web', url: value.url, ref: value.url, quote: value.quote, changed: false };
|
||||
}
|
||||
const { path, commit, lines } = value;
|
||||
if (typeof path !== 'string' || !SHA.test(commit ?? '') || !Array.isArray(lines) || lines.length !== 2
|
||||
|| !line(lines[0]) || !line(lines[1]) || lines[1] < lines[0]) return null;
|
||||
const badge = scopeOf?.(path);
|
||||
const ref = `${path}:${lines[0] === lines[1] ? lines[0] : `${lines[0]}-${lines[1]}`} @${commit.slice(0, 7)}`;
|
||||
return { badge: badge === 'canon' || badge === 'playground' ? badge : null, path, commit, lines: [...lines], ref,
|
||||
quote: value.quote, changed: changed?.({ path, commit }) === true };
|
||||
}
|
||||
const quoted = text => JSON.stringify(text);
|
||||
|
||||
// body: the stored message body (a JSON string or the parsed object).
|
||||
// options.scopeOf(path) -> 'canon' | 'playground' | undefined, for the scope badge.
|
||||
// options.changed({path, commit}) -> true when the file's last commit is newer than the cited one.
|
||||
// Returns { kind: 'items', items, text } | { kind: 'notice', notice, text } | { kind: 'unreadable', text }.
|
||||
export function renderItems(body, options = {}) {
|
||||
const unreadable = { kind: 'unreadable', text: UNREADABLE };
|
||||
let value = body;
|
||||
if (typeof body === 'string') { try { value = JSON.parse(body); } catch { return unreadable; } }
|
||||
if (!plain(value) || !Array.isArray(value.items)) return unreadable;
|
||||
const keys = Object.keys(value);
|
||||
if (value.items.length === 0) {
|
||||
if (keys.length !== 2 || !Object.hasOwn(NOTICES, value.notice)) return unreadable;
|
||||
return { kind: 'notice', notice: value.notice, text: NOTICES[value.notice] };
|
||||
}
|
||||
if (keys.length !== 1) return unreadable;
|
||||
const items = [];
|
||||
for (const item of value.items) {
|
||||
if (!plain(item) || !Object.hasOwn(KINDS, item.kind) || typeof item.text !== 'string' || !Array.isArray(item.cites)) return unreadable;
|
||||
const correction = item.kind === 'correction';
|
||||
if (correction && (typeof item.from !== 'string' || typeof item.to !== 'string')) return unreadable;
|
||||
const cites = item.cites.map(c => cite(c, options));
|
||||
if (cites.some(c => c === null)) return unreadable;
|
||||
items.push({ kind: item.kind, label: KINDS[item.kind], text: item.text,
|
||||
...(correction ? { from: item.from, to: item.to } : {}), cites });
|
||||
}
|
||||
const text = items.map((item, i) => [
|
||||
`${i + 1}. ${item.label}: ${item.text}`,
|
||||
...(item.kind === 'correction' ? [` From: ${quoted(item.from)}`, ` To: ${quoted(item.to)}`] : []),
|
||||
...item.cites.map(c => ` ${c.badge ? `[${c.badge}] ` : ''}${c.ref} ${quoted(c.quote)}${c.changed ? ' (changed since)' : ''}`),
|
||||
].join('\n')).join('\n');
|
||||
return { kind: 'items', items, text };
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
// The scope rule of design 1.2. Scope comes from the path alone (D2): no file content can change it.
|
||||
// Paths are repo-relative and compared exactly, case included (R7 handles case-only twins).
|
||||
const inside = (folder, path) => path.startsWith(folder.endsWith('/') ? folder : `${folder}/`);
|
||||
|
||||
// { scope: 'playground' | 'canon', universe, book, type, folder } or { scope: 'unmapped' }.
|
||||
// `type` is the mapped type of the first folder under the book (canon) or under the playground
|
||||
// prefix (playground), 'manuscript' for a manuscript folder, else null. `folder` is that folder name.
|
||||
export function classify(mapping, path) {
|
||||
if (typeof path !== 'string' || !path.endsWith('.md')) return { scope: 'unmapped' };
|
||||
for (const [universe, entry] of Object.entries(mapping.universes)) {
|
||||
const book = entry.books.find(b => inside(b, path));
|
||||
if (!book) continue;
|
||||
const prefix = entry.playground.find(p => inside(p, path));
|
||||
const rest = path.slice((prefix ?? `${book}/`).length).split('/');
|
||||
const folder = rest.length > 1 ? rest[0] : null;
|
||||
const type = folder === null ? null : Object.hasOwn(mapping.types, folder) ? mapping.types[folder]
|
||||
: mapping.manuscript.includes(folder) ? 'manuscript' : null;
|
||||
return { scope: prefix ? 'playground' : 'canon', universe, book, type, folder };
|
||||
}
|
||||
return { scope: 'unmapped' };
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
// `mosaic library survey <repo> [--mapping <file>]` (design 1.1, 10 step 7). It prints what a mapping
|
||||
// needs: folder names, file counts by extension, front matter key sets and link-form counts. It
|
||||
// never prints file text, front matter values, file names or link targets. With a mapping it adds
|
||||
// scope counts, resolver step counts and R-problem counts by code. It reads the repo's HEAD commit
|
||||
// through the commit reader, never the working tree.
|
||||
import { readFileSync, realpathSync, statSync } from 'node:fs';
|
||||
import { resolve as resolvePath } from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import { checkMapping } from './mapping.mjs';
|
||||
import { classify } from './scope.mjs';
|
||||
import { formOf } from './links.mjs';
|
||||
import { headCommit, listTree, parseFile, readable, readLibrary, readTexts } from './reader.mjs';
|
||||
import { LibraryError, RULES } from './problems.mjs';
|
||||
|
||||
export const USAGE = 'usage: mosaic library survey <repo> [--mapping <file>]';
|
||||
// Keys are printed only in the short identifier forms property names take. A key with spaces or of
|
||||
// any other shape could be a phrase of hers, so it is counted as one "(other key)".
|
||||
const KEY = /^[A-Za-z_][A-Za-z0-9_-]{0,31}$/;
|
||||
const OTHER_KEY = '(other key)';
|
||||
const FORMS = ['plain', 'label', 'heading', 'embed', 'self', 'markdown'];
|
||||
const STEPS = ['root', 'vault-prefix', 'relative', 'shortest'];
|
||||
const extension = path => /\.md$/.test(path) ? 'md' : /\.canvas$/.test(path) ? 'canvas' : /\.base$/.test(path) ? 'base' : 'other';
|
||||
|
||||
// The survey as data. `mapping` is a checked mapping or undefined.
|
||||
export async function survey({ repo, commit, mapping }) {
|
||||
const head = commit ?? await headCommit(repo);
|
||||
const entries = await listTree(repo, head);
|
||||
const folders = new Map();
|
||||
for (const entry of entries) {
|
||||
const slash = entry.path.lastIndexOf('/');
|
||||
const name = slash < 0 ? '' : entry.path.slice(0, slash);
|
||||
const counts = folders.get(name) ?? { md: 0, canvas: 0, base: 0, other: 0 };
|
||||
counts[entry.type === 'blob' ? extension(entry.path) : 'other']++;
|
||||
folders.set(name, counts);
|
||||
}
|
||||
const markdown = entries.filter(e => e.path.endsWith('.md'));
|
||||
const texts = await readTexts(repo, markdown.filter(readable));
|
||||
const keySets = new Map(), forms = Object.fromEntries(FORMS.map(f => [f, 0]));
|
||||
const fields = { appears_in: { wikilink: 0, name: 0 }, relationships: { wikilink: 0, name: 0 } };
|
||||
let noFrontMatter = 0, unparsed = 0;
|
||||
for (const text of texts.values()) {
|
||||
const file = parseFile(text, { markdown: true });
|
||||
if (file.stop !== null) unparsed++;
|
||||
else if (!file.frontmatter) noFrontMatter++;
|
||||
else {
|
||||
const set = [...new Set(Object.keys(file.frontmatter).map(k => KEY.test(k) ? k : OTHER_KEY))].sort().join(', ');
|
||||
keySets.set(set, (keySets.get(set) ?? 0) + 1);
|
||||
for (const field of Object.keys(fields)) {
|
||||
const raw = file.frontmatter[field];
|
||||
for (const v of Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw])
|
||||
fields[field][typeof v === 'string' && /^\s*!?\[\[[^\]]+\]\]\s*$/.test(v) ? 'wikilink' : 'name']++;
|
||||
}
|
||||
}
|
||||
for (const link of file.links) if (!link.field) forms[formOf(link)]++;
|
||||
}
|
||||
const result = {
|
||||
commit: head, entries: entries.length,
|
||||
folders: [...folders].sort(([a], [b]) => a < b ? -1 : 1).map(([name, counts]) => ({ name, ...counts })),
|
||||
markdown: { files: markdown.length, read: texts.size, noFrontMatter, unparsed },
|
||||
keySets: [...keySets].sort(([a, x], [b, y]) => y - x || (a < b ? -1 : 1)).map(([keys, count]) => ({ keys, count })),
|
||||
forms, fields,
|
||||
};
|
||||
if (mapping) {
|
||||
const scopes = { canon: 0, playground: 0, unmapped: 0 };
|
||||
for (const entry of entries) scopes[classify(mapping, entry.path).scope]++;
|
||||
const library = await readLibrary({ repo, commit: head, mapping });
|
||||
const steps = Object.fromEntries(STEPS.map(s => [s, 0]));
|
||||
for (const file of library.files) for (const link of file.links) steps[link.step]++;
|
||||
const problems = Object.fromEntries(Object.keys(RULES).map(code => [code, 0]));
|
||||
for (const p of library.problems) problems[p.code]++;
|
||||
steps.unresolved = problems.R4;
|
||||
Object.assign(result, { scopes, indexed: library.files.length, steps, problems });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Folder names are quoted as JSON strings, so no name can carry a control sequence to the terminal.
|
||||
export function formatSurvey(s) {
|
||||
const out = [`commit ${s.commit}`, `entries ${s.entries}`, 'folders (direct children: md canvas base other)'];
|
||||
for (const f of s.folders) out.push(` ${f.name === '' ? '(root)' : JSON.stringify(f.name)} ${f.md} ${f.canvas} ${f.base} ${f.other}`);
|
||||
out.push(`markdown ${s.markdown.files} files, ${s.markdown.read} read, ${s.markdown.noFrontMatter} without front matter, ${s.markdown.unparsed} with front matter not fully parsed`);
|
||||
out.push('front matter key sets');
|
||||
for (const k of s.keySets) out.push(` ${k.count} [${k.keys}]`);
|
||||
out.push(`body links ${FORMS.map(f => `${f} ${s.forms[f]}`).join(', ')}`);
|
||||
for (const [field, c] of Object.entries(s.fields)) out.push(`field ${field} wikilink ${c.wikilink}, name ${c.name}`);
|
||||
if (s.scopes) {
|
||||
out.push(`scopes canon ${s.scopes.canon}, playground ${s.scopes.playground}, unmapped ${s.scopes.unmapped}; indexed ${s.indexed}`);
|
||||
out.push(`resolved ${STEPS.map(st => `${st} ${s.steps[st]}`).join(', ')}, unresolved ${s.steps.unresolved}`);
|
||||
out.push(`problems ${Object.entries(s.problems).map(([code, n]) => `${code} ${n}`).join(', ')}`);
|
||||
}
|
||||
return `${out.join('\n')}\n`;
|
||||
}
|
||||
|
||||
// Exit codes as packages/project's CLI: 0 done, 2 the repo can't be read, 3 a bad mapping, 4 usage.
|
||||
export async function surveyMain(argv, io = { stdout: process.stdout, stderr: process.stderr }) {
|
||||
const fail = (code, message) => { io.stderr.write(`library survey: ${message}\n`); return code; };
|
||||
const [verb, ...rest] = argv;
|
||||
if (verb !== 'survey') return fail(4, USAGE);
|
||||
let repo, mappingFile;
|
||||
for (let i = 0; i < rest.length; i++) {
|
||||
if (rest[i] === '--mapping' && i + 1 < rest.length && mappingFile === undefined) mappingFile = rest[++i];
|
||||
else if (!rest[i].startsWith('-') && repo === undefined) repo = rest[i];
|
||||
else return fail(4, USAGE);
|
||||
}
|
||||
if (repo === undefined) return fail(4, USAGE);
|
||||
let mapping;
|
||||
if (mappingFile !== undefined) {
|
||||
let entry;
|
||||
try { entry = JSON.parse(readFileSync(mappingFile, 'utf8')); }
|
||||
catch { return fail(3, `library-mapping-invalid: ${mappingFile} is not a readable JSON file`); }
|
||||
try { mapping = checkMapping(entry, mappingFile); }
|
||||
catch (e) { if (e instanceof LibraryError) return fail(3, e.message); throw e; }
|
||||
}
|
||||
const path = resolvePath(repo);
|
||||
try { if (!statSync(path).isDirectory()) return fail(2, `library-git: ${repo} is not a directory`); }
|
||||
catch { return fail(2, `library-git: ${repo} is not a directory`); }
|
||||
try { io.stdout.write(formatSurvey(await survey({ repo: path, mapping }))); }
|
||||
catch (e) { if (e instanceof LibraryError) return fail(2, e.message); throw e; }
|
||||
return 0;
|
||||
}
|
||||
|
||||
const entry = () => { try { return import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href; } catch { return false; } };
|
||||
if (process.argv[1] && entry()) {
|
||||
const code = await surveyMain(process.argv.slice(2));
|
||||
if (code !== 0) process.exitCode = code;
|
||||
}
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"kind": "library",
|
||||
"remote": "https://git.example.invalid/writer/books.git",
|
||||
"branch": "main",
|
||||
"vaultPrefix": "books",
|
||||
"universes": {
|
||||
"harbor": {"name": "The Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"]},
|
||||
"second": {"name": "Second World", "books": ["Second World"]}
|
||||
},
|
||||
"types": {"Characters": "character", "Places": "place", "Research": "research", "Tracker": "tracker", "Summaries": "summary"},
|
||||
"manuscript": ["Chapters"]
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
// A repo the test builds in her layout's shape: spaces and capitals in folder names, a .canvas file
|
||||
// and an unmapped top-level note. Bodies are made up; MARKER stands in for her text.
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync, symlinkSync, chmodSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { checkMapping } from '../src/mapping.mjs';
|
||||
|
||||
export const MARKER = 'MARKER-7f3a-HER-TEXT';
|
||||
export const FIXTURES = fileURLToPath(new URL('./fixtures/', import.meta.url));
|
||||
export const mappingEntry = () => JSON.parse(readFileSync(join(FIXTURES, 'mapping.json'), 'utf8'));
|
||||
export const mapping = (change = e => e) => checkMapping(change(mappingEntry()));
|
||||
|
||||
export const git = (repo, ...args) => execFileSync('git', ['-c', 'user.name=fixture', '-c', '[email protected]',
|
||||
'-c', 'commit.gpgsign=false', '-c', 'core.hooksPath=/dev/null', '-C', repo, ...args], { encoding: 'utf8' }).trim();
|
||||
|
||||
export const BASE = {
|
||||
'Book One/Characters/Mara Vell.md': [
|
||||
'---', 'type: character', 'name: Mara Vell', 'appears_in: "[[Book One/Chapters/Chapter 01]]"',
|
||||
'relationships:', ' - "[[Tomas]]"', ' - The Ferryman', '---', `Grey eyes. ${MARKER}`, ''].join('\n'),
|
||||
'Book One/Characters/Tomas.md': ['---', 'type: character', 'name: Tomas', 'appears_in: ["[[Chapter 01]]"]', '---', MARKER, ''].join('\n'),
|
||||
'Book One/Chapters/Chapter 01.md': [
|
||||
'# One', `${MARKER} [[Mara Vell]] met [[Book One/Places/Harbor Town|the harbor]].`,
|
||||
'See [[../Places/Harbor Town#Docks]] and [[books/Book One/Characters/Tomas]].',
|
||||
'```', '[[Not A Link]]', '```', 'Inline `[[Not Either]]` and [a note](Notes.md).', ''].join('\n'),
|
||||
'Book One/Places/Harbor Town.md': ['---', 'type: place', '---', `Salt. ${MARKER}`, ''].join('\n'),
|
||||
'Book One/Playground/What If.md': ['---', 'name: What If', '---', `${MARKER} [[Mara Vell]]`, ''].join('\n'),
|
||||
'Book One/Board.canvas': '{"nodes":[]}\n',
|
||||
'Book One/Playground/Sketch.canvas': '{"nodes":[]}\n',
|
||||
'Inbox Note.md': `${MARKER}\n`,
|
||||
'Second World/Chapters/01.md': `# Elsewhere ${MARKER}\n`,
|
||||
};
|
||||
|
||||
// repo({files, extra}) writes files (path -> text, or { link } / { mode, text }), commits, and
|
||||
// returns { repo, commit }.
|
||||
export function repo(t, files = BASE) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'library-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const dir = join(root, 'books');
|
||||
mkdirSync(dir);
|
||||
git(dir, 'init', '-q', '-b', 'main');
|
||||
for (const [path, value] of Object.entries(files)) {
|
||||
const file = join(dir, path);
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
if (typeof value === 'object' && value.link) symlinkSync(value.link, file);
|
||||
else if (typeof value === 'object' && value.bytes) writeFileSync(file, value.bytes);
|
||||
else writeFileSync(file, typeof value === 'object' ? value.text : value);
|
||||
if (typeof value === 'object' && value.mode) chmodSync(file, value.mode);
|
||||
}
|
||||
git(dir, 'add', '-A');
|
||||
git(dir, 'commit', '-qm', 'fixture');
|
||||
return { root, repo: dir, commit: git(dir, 'rev-parse', 'HEAD') };
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { extractLinks, parseWikilink, resolver } from '../src/index.mjs';
|
||||
|
||||
const PATHS = ['Book One/Characters/Mara Vell.md', 'Book One/Characters/Tomas.md', 'Book One/Places/Harbor Town.md',
|
||||
'Book One/Chapters/Chapter 01.md', 'Book One/Places/Notes.md', 'Book One/Characters/Notes.md', 'Mara Vell.md'];
|
||||
const resolve = resolver(PATHS, { vaultPrefix: 'books' });
|
||||
const FROM = 'Book One/Chapters/Chapter 01.md';
|
||||
|
||||
test('resolver step 1: a path from the repo root, with or without .md', () => {
|
||||
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
|
||||
assert.deepEqual(resolve(FROM, 'Book One/Places/Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'root' });
|
||||
// Root wins over shortest: Mara Vell.md at the root, not the character page.
|
||||
assert.deepEqual(resolve(FROM, 'Mara Vell'), { path: 'Mara Vell.md', step: 'root' });
|
||||
});
|
||||
|
||||
test('resolver step 2: the same path with vaultPrefix/ stripped', () => {
|
||||
assert.deepEqual(resolve(FROM, 'books/Book One/Characters/Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'vault-prefix' });
|
||||
assert.deepEqual(resolver(PATHS)(FROM, 'books/Book One/Characters/Tomas'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('resolver step 3: a path relative to the linking file, never above the repo root', () => {
|
||||
assert.deepEqual(resolve(FROM, '../Places/Harbor Town'), { path: 'Book One/Places/Harbor Town.md', step: 'relative' });
|
||||
assert.deepEqual(resolve('Book One/Places/Harbor Town.md', './Notes'), { path: 'Book One/Places/Notes.md', step: 'relative' });
|
||||
assert.deepEqual(resolve(FROM, '../../../Book One/Places/Harbor Town'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('resolver step 4: a base name that matches exactly one indexed file', () => {
|
||||
assert.deepEqual(resolve(FROM, 'Tomas'), { path: 'Book One/Characters/Tomas.md', step: 'shortest' });
|
||||
assert.deepEqual(resolve(FROM, 'Harbor Town.md'), { path: 'Book One/Places/Harbor Town.md', step: 'shortest' });
|
||||
assert.deepEqual(resolve(FROM, 'Notes'), { path: null, step: 'ambiguous', count: 2 });
|
||||
assert.deepEqual(resolve(FROM, 'Nobody'), { path: null, step: 'missing' });
|
||||
// A partial path is not a base name.
|
||||
assert.deepEqual(resolve(FROM, 'Places/Harbor Town'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('a link into another of her repos resolves to nothing', () => {
|
||||
assert.deepEqual(resolve(FROM, 'bean-brain/Journal/Today'), { path: null, step: 'missing' });
|
||||
assert.deepEqual(resolve(FROM, '../bean-brain/Journal/Today'), { path: null, step: 'missing' });
|
||||
});
|
||||
|
||||
test('wikilink forms, embeds, and links skipped inside code', () => {
|
||||
assert.deepEqual(parseWikilink('[[A/B#Head|Label]]'), { file: 'A/B', heading: 'Head', label: 'Label', embed: false });
|
||||
assert.deepEqual(parseWikilink('![[pic.png]]'), { file: 'pic.png', heading: null, label: null, embed: true });
|
||||
assert.equal(parseWikilink('[[a]] and more'), null);
|
||||
const lines = [[1, '[[One]] [[Two|2]] [[Three#H]] ![[Four]] [[#Self]]'], [2, '```js'], [3, '[[Fenced]]'], [4, '```'],
|
||||
[5, 'Inline `[[Code]]` and [md](Five.md) and [web](https://example.invalid/x)']];
|
||||
assert.deepEqual(extractLinks(lines).map(l => [l.file, l.line]), [['One', 1], ['Two', 1], ['Three', 1], ['Four', 1], ['', 1]]);
|
||||
assert.deepEqual(extractLinks(lines, { markdown: true }).filter(l => l.markdown).map(l => l.file), ['Five.md']);
|
||||
});
|
||||
@@ -0,0 +1,82 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { checkMapping, classify, LibraryError } from '../src/index.mjs';
|
||||
import { mapping, mappingEntry } from './helpers.mjs';
|
||||
|
||||
const refuses = (change, code, pattern) => assert.throws(() => checkMapping(change(mappingEntry())),
|
||||
e => e instanceof LibraryError && e.code === code && (!pattern || pattern.test(e.message)));
|
||||
|
||||
test('the fixture mapping checks and comes back frozen', () => {
|
||||
const m = mapping();
|
||||
assert.equal(m.universes.harbor.playground[0], 'Book One/Playground/');
|
||||
assert.ok(Object.isFrozen(m.universes.harbor.books));
|
||||
assert.throws(() => { m.types.Extra = 'place'; });
|
||||
});
|
||||
|
||||
test('a playground prefix outside its universe books refuses library-playground-outside-book', () => {
|
||||
refuses(e => { e.universes.harbor.playground = ['Second World/Playground/']; return e; }, 'library-playground-outside-book');
|
||||
refuses(e => { e.universes.harbor.playground = ['Elsewhere/']; return e; }, 'library-playground-outside-book');
|
||||
// The whole book as its playground would leave no canon: not inside the book.
|
||||
refuses(e => { e.universes.harbor.playground = ['Book One/']; return e; }, 'library-playground-outside-book');
|
||||
});
|
||||
|
||||
test('no playground prefix holds another, and no book folder holds another', () => {
|
||||
refuses(e => { e.universes.harbor.playground = ['Book One/Playground/', 'Book One/Playground/Deep/']; return e; },
|
||||
'library-mapping-invalid', /holds playground/);
|
||||
refuses(e => { e.universes.second.books = ['Second World', 'Book One/Part Two']; return e; }, 'library-mapping-invalid', /holds book folder/);
|
||||
refuses(e => { e.universes.second.books = ['Book One']; return e; }, 'library-mapping-invalid', /in two universes/);
|
||||
});
|
||||
|
||||
test('shape refusals: kind, remote, branch, unknown keys, types, manuscript, folder names', () => {
|
||||
refuses(e => ({ ...e, kind: 'code' }), 'library-mapping-invalid', /kind/);
|
||||
refuses(e => ({ ...e, remote: 'http://git.example.invalid/x.git' }), 'library-mapping-invalid', /https/);
|
||||
refuses(e => ({ ...e, remote: 'https://user:[email protected]/x.git' }), 'library-mapping-invalid', /credentials/);
|
||||
refuses(e => ({ ...e, remote: '[email protected]:x.git' }), 'library-mapping-invalid', /https/);
|
||||
refuses(e => ({ ...e, branch: 'main..x' }), 'library-mapping-invalid', /branch/);
|
||||
refuses(e => ({ ...e, root: '/tmp' }), 'library-mapping-invalid', /unknown key "root"/);
|
||||
refuses(e => { e.universes.harbor.extra = 1; return e; }, 'library-mapping-invalid', /unknown key/);
|
||||
refuses(e => { e.types.Characters = 'villain'; return e; }, 'library-mapping-invalid', /types.Characters/);
|
||||
refuses(e => { e.types['A/B'] = 'place'; return e; }, 'library-mapping-invalid', /one folder name/);
|
||||
refuses(e => ({ ...e, manuscript: ['Characters'] }), 'library-mapping-invalid', /also a type folder/);
|
||||
refuses(e => { e.universes.harbor.books = ['../Book One']; return e; }, 'library-mapping-invalid', /folder path/);
|
||||
refuses(e => ({ ...e, vaultPrefix: 'a/b' }), 'library-mapping-invalid', /vaultPrefix/);
|
||||
refuses(e => ({ ...e, universes: {} }), 'library-mapping-invalid', /at least one universe/);
|
||||
refuses(e => { e.universes['bad id'] = e.universes.second; delete e.universes.second; return e; }, 'library-mapping-invalid', /plain id/);
|
||||
});
|
||||
|
||||
test('a hybrid universe is a shape the query code can take (1.4), with sources that must exist', () => {
|
||||
const hybrid = e => { e.universes.cross = { kind: 'hybrid', name: 'Crossover', sources: ['harbor', 'second'], books: ['Crossover'],
|
||||
constraints: [{ exclude: 'second/Characters' }, { pin: { entity: 'Book One/Characters/Mara Vell.md', field: 'eyes', value: 'grey' } }],
|
||||
notes: 'Set after book one.' }; return e; };
|
||||
const m = checkMapping(hybrid(mappingEntry()));
|
||||
assert.deepEqual(m.universes.cross.sources, ['harbor', 'second']);
|
||||
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor', 'nowhere']; return e; }, 'library-mapping-invalid', /non-hybrid universe/);
|
||||
refuses(e => { hybrid(e); e.universes.cross.sources = ['harbor']; return e; }, 'library-mapping-invalid', /two or more/);
|
||||
refuses(e => { hybrid(e); e.universes.cross.constraints = [{ exclude: 'x', pin: {} }]; return e; }, 'library-mapping-invalid', /exactly one/);
|
||||
});
|
||||
|
||||
test('a caller-supplied fail receives the code, as business file loading will pass it (C2-04)', () => {
|
||||
const seen = [];
|
||||
const fail = (code, message) => { seen.push(code); throw new Error(message); };
|
||||
const entry = mappingEntry();
|
||||
entry.universes.harbor.playground = ['Elsewhere/'];
|
||||
assert.throws(() => checkMapping(entry, 'businesses/acme.json projects.books', { fail }), /businesses\/acme.json projects.books/);
|
||||
assert.deepEqual(seen, ['library-playground-outside-book']);
|
||||
});
|
||||
|
||||
test('scope comes from the path alone (design 1.2)', () => {
|
||||
const m = mapping();
|
||||
assert.deepEqual(classify(m, 'Book One/Characters/Mara Vell.md'),
|
||||
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Chapters/Chapter 01.md'),
|
||||
{ scope: 'canon', universe: 'harbor', book: 'Book One', type: 'manuscript', folder: 'Chapters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Playground/What If.md'),
|
||||
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: null, folder: null });
|
||||
assert.deepEqual(classify(m, 'Book One/Playground/Characters/New.md'),
|
||||
{ scope: 'playground', universe: 'harbor', book: 'Book One', type: 'character', folder: 'Characters' });
|
||||
assert.deepEqual(classify(m, 'Book One/Notes.md'), { scope: 'canon', universe: 'harbor', book: 'Book One', type: null, folder: null });
|
||||
assert.equal(classify(m, 'Second World/Chapters/01.md').universe, 'second');
|
||||
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas', 'Book One/Board.base',
|
||||
'Book One/cover.png', 'Book One/Upper.MD', 'Book Onex/a.md', 'book one/a.md', 'Elsewhere/Book One/a.md'])
|
||||
assert.deepEqual(classify(m, path), { scope: 'unmapped' }, path);
|
||||
});
|
||||
@@ -0,0 +1,247 @@
|
||||
// C2-04: local made-up repositories only. No remote credential or live config.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, readFileSync, readdirSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { once } from 'node:events';
|
||||
import { mappingEntry, repo, git, BASE, MARKER } from './helpers.mjs';
|
||||
import * as api from '../src/mirror.mjs';
|
||||
|
||||
const NOW = '2026-10-21T00:00:00.000Z';
|
||||
const MOD = new URL('../src/mirror.mjs', import.meta.url).href;
|
||||
const state = (served = null) => ({ version: 1, branch: 'main', served, lastFetch: null, lastError: null, diverged: null, problems: [] });
|
||||
const stateFile = mirror => join(mirror, api.SYNC_STATE_FILE);
|
||||
async function fixture(t, files = BASE) {
|
||||
const f = repo(t, files), dataRoot = join(f.root, 'data');
|
||||
const context = { business: { id: 'acme', projects: { books: mappingEntry(), code: { root: f.repo } } }, project: 'books', dataRoot };
|
||||
const mirror = await api.ensureMirror(context);
|
||||
const writeState = value => writeFileSync(stateFile(mirror), JSON.stringify(value), { mode: 0o600 });
|
||||
const seed = (commit = f.commit) => {
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, `${commit}:refs/heads/served`);
|
||||
writeState(state(commit));
|
||||
};
|
||||
return { ...f, dataRoot, context, mirror, writeState, seed };
|
||||
}
|
||||
|
||||
test('C2-04 mirror: exact default path, bare without checkout or remote, and idempotent initialization', async t => {
|
||||
const f = await fixture(t);
|
||||
assert.equal(api.SYNC_STATE_FILE, 'mosaic-library-sync.json');
|
||||
assert.equal(api.mirrorPath(f.context), join(f.dataRoot, 'projects/acme/books'));
|
||||
assert.equal(f.mirror, api.mirrorPath(f.context));
|
||||
assert.equal(git(f.mirror, 'rev-parse', '--is-bare-repository'), 'true');
|
||||
assert.equal(git(f.mirror, 'remote'), '');
|
||||
assert.equal(existsSync(join(f.mirror, '.git')), false);
|
||||
const lock = join(f.mirror, 'mosaic-library-sync.lock'), st = lstatSync(lock);
|
||||
assert.equal(st.mode & 0o777, 0o600);
|
||||
assert.equal(st.nlink, 1);
|
||||
assert.equal(await api.ensureMirror(f.context), f.mirror);
|
||||
assert.equal(lstatSync(lock).ino, st.ino, 'the lock inode is stable');
|
||||
assert.equal(await api.servedCommit(f.mirror), null);
|
||||
assert.deepEqual(await api.libraryStatus(f.context), { project: 'books', ...state() });
|
||||
});
|
||||
|
||||
test('C2-04 mirror: simultaneous creators publish one intact mirror and leave no staging paths', async t => {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data');
|
||||
const c = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
|
||||
const results = await Promise.all(Array.from({ length: 4 }, () => api.ensureMirror(c)));
|
||||
assert.deepEqual([...new Set(results)], [join(dataRoot, 'projects/acme/books')]);
|
||||
assert.deepEqual(readdirSync(join(dataRoot, 'projects/acme')), ['books']);
|
||||
assert.equal(git(results[0], 'rev-parse', '--is-bare-repository'), 'true');
|
||||
});
|
||||
|
||||
test('C2-04 mirror: skipped publication accepts both coreutils statuses without removing the winner', { timeout: 15000 }, async t => {
|
||||
for (const status of [0, 1]) {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data');
|
||||
const context = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
|
||||
const creator = child(t, `import cp from 'node:child_process';import {cpSync,chmodSync,lstatSync,readdirSync} from 'node:fs';import {dirname} from 'node:path';import {syncBuiltinESMExports} from 'node:module';
|
||||
const original=cp.execFile;let winner;
|
||||
cp.execFile=(command,args,options,callback)=>{if(command!=='/usr/bin/mv')return original(command,args,options,callback);
|
||||
cpSync(args[2],args[3],{recursive:true});chmodSync(args[3],0o700);winner=lstatSync(args[3]).ino;
|
||||
const error=${status} ? Object.assign(Error('fixture no-clobber diagnostic'),{code:${status}}) : null;
|
||||
queueMicrotask(()=>callback(error));};syncBuiltinESMExports();
|
||||
const {ensureMirror}=await import(${JSON.stringify(MOD)});const mirror=await ensureMirror(${JSON.stringify(context)});
|
||||
process.send({same:lstatSync(mirror).ino===winner,entries:readdirSync(dirname(mirror))});process.disconnect();`);
|
||||
const [result] = await Promise.race([once(creator, 'message'), creator.finished.then(result => { throw Error(JSON.stringify(result)); })]);
|
||||
assert.deepEqual(result, { same: true, entries: ['books'] });
|
||||
assert.deepEqual(await creator.finished, { code: 0, stderr: '' });
|
||||
}
|
||||
});
|
||||
|
||||
test('C2-04 mirror: preserve occupied directories and reject symlink ancestors and code overlaps', async t => {
|
||||
const f = repo(t), dataRoot = join(f.root, 'data');
|
||||
const c = { business: { id: 'acme', projects: { books: mappingEntry() } }, project: 'books', dataRoot };
|
||||
const path = join(dataRoot, 'projects/acme/books'); mkdirSync(path, { recursive: true });
|
||||
writeFileSync(join(path, 'keep'), 'made-up human file');
|
||||
await assert.rejects(api.ensureMirror(c));
|
||||
assert.equal(readFileSync(join(path, 'keep'), 'utf8'), 'made-up human file');
|
||||
rmSync(dataRoot, { recursive: true }); symlinkSync(f.repo, dataRoot);
|
||||
assert.throws(() => api.mirrorPath(c), { code: 'project-path' });
|
||||
unlinkSync(dataRoot);
|
||||
c.business.projects.code = { root: dataRoot };
|
||||
assert.throws(() => api.mirrorPath(c), { code: 'project-path' });
|
||||
});
|
||||
|
||||
test('C2-04 mirror: existing metadata accepts only bare Git plus state and the precreated lock', async t => {
|
||||
const f = await fixture(t);
|
||||
writeFileSync(join(f.mirror, 'unknown-note'), 'do not silently allow this');
|
||||
await assert.rejects(api.ensureMirror(f.context), { code: 'library-git' });
|
||||
assert.equal(readFileSync(join(f.mirror, 'unknown-note'), 'utf8'), 'do not silently allow this');
|
||||
unlinkSync(join(f.mirror, 'unknown-note'));
|
||||
git(f.mirror, 'config', 'remote.origin.url', 'https://example.invalid/fixture.git');
|
||||
await assert.rejects(api.ensureMirror(f.context), { code: 'library-git' });
|
||||
git(f.mirror, 'config', '--remove-section', 'remote.origin');
|
||||
unlinkSync(join(f.mirror, 'mosaic-library-sync.lock'));
|
||||
await assert.rejects(api.ensureMirror(f.context), { code: 'library-state-invalid' });
|
||||
assert.equal(existsSync(join(f.mirror, 'mosaic-library-sync.lock')), false);
|
||||
});
|
||||
|
||||
test('C2-04 state: missing state is empty only while the served ref is absent', async t => {
|
||||
const f = await fixture(t);
|
||||
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
|
||||
f.seed(); unlinkSync(stateFile(f.mirror));
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
});
|
||||
|
||||
test('C2-04 state: valid problems and sync metadata return no content and change no state bytes', async t => {
|
||||
const f = await fixture(t); f.seed();
|
||||
const value = { ...state(f.commit), lastFetch: NOW, lastError: { code: 'library-git', at: NOW },
|
||||
diverged: { commit: 'a'.repeat(40), at: NOW },
|
||||
problems: [{ code: 'R4', path: 'Book One/Chapters/A.md', line: 2, message: 'link resolves to no file or to more than one' }] };
|
||||
f.writeState(value); const before = readFileSync(stateFile(f.mirror));
|
||||
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), value);
|
||||
assert.equal(JSON.stringify(await api.libraryStatus(f.context)).includes(MARKER), false);
|
||||
assert.deepEqual(readFileSync(stateFile(f.mirror)), before);
|
||||
});
|
||||
|
||||
test('C2-04 state: strict keys, types, times, branch, commit and fixed R messages fail closed', async t => {
|
||||
const f = await fixture(t); f.seed();
|
||||
const mutations = [
|
||||
s => { s.extra = true; }, s => { s.version = 2; }, s => { s.branch = 'other'; },
|
||||
s => { s.served = 'a'.repeat(40); }, s => { s.served = null; }, s => { s.served = 'ABC'; },
|
||||
s => { s.lastFetch = 'yesterday'; }, s => { s.lastFetch = 1; },
|
||||
s => { s.lastError = { code: 'bad code', at: NOW }; },
|
||||
s => { s.lastError = { code: 'library-git', at: NOW, stderr: MARKER }; },
|
||||
s => { s.diverged = { commit: 'not-a-sha', at: NOW }; },
|
||||
s => { s.problems = [{ code: 'R1', path: 'Book One/A.md', message: MARKER }]; },
|
||||
s => { s.problems = [{ code: 'R4', path: 'Book One/A.md', line: 0, message: 'link resolves to no file or to more than one' }]; },
|
||||
s => { s.problems = null; },
|
||||
];
|
||||
for (const mutate of mutations) {
|
||||
const s = state(f.commit); mutate(s); f.writeState(s);
|
||||
const before = readFileSync(stateFile(f.mirror));
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
assert.deepEqual(readFileSync(stateFile(f.mirror)), before);
|
||||
}
|
||||
writeFileSync(stateFile(f.mirror), '{');
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
});
|
||||
|
||||
test('C2-04 state: symlink, hardlink, non-private state and missing lock refuse without replacement', async t => {
|
||||
const f = await fixture(t); f.seed();
|
||||
const outside = join(f.root, 'outside.json'); writeFileSync(outside, JSON.stringify(state(f.commit)), { mode: 0o600 });
|
||||
unlinkSync(stateFile(f.mirror)); symlinkSync(outside, stateFile(f.mirror));
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
unlinkSync(stateFile(f.mirror)); linkSync(outside, stateFile(f.mirror));
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
unlinkSync(stateFile(f.mirror)); f.writeState(state(f.commit)); chmodSync(stateFile(f.mirror), 0o644);
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
chmodSync(stateFile(f.mirror), 0o600); unlinkSync(join(f.mirror, 'mosaic-library-sync.lock'));
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-state-invalid' });
|
||||
assert.equal(existsSync(join(f.mirror, 'mosaic-library-sync.lock')), false);
|
||||
});
|
||||
|
||||
test('C2-04 file: served commit, not HEAD or a caller commit, owns the exact returned text', async t => {
|
||||
const f = await fixture(t); f.seed();
|
||||
const path = 'Book One/Characters/Tomas.md';
|
||||
writeFileSync(join(f.repo, path), 'newer text\n'); git(f.repo, 'commit', '-qam', 'newer');
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/unserved');
|
||||
git(f.mirror, 'symbolic-ref', 'HEAD', 'refs/heads/unserved');
|
||||
assert.deepEqual(await api.readLibraryFile({ ...f.context, path }), { project: 'books', path, commit: f.commit, content: BASE[path] });
|
||||
await assert.rejects(api.readLibraryFile({ ...f.context, path, commit: git(f.repo, 'rev-parse', 'HEAD') }), { code: 'project-read-path' });
|
||||
for (const path of ['../escape.md', '/absolute.md', 'Book One/Board.canvas', 'Inbox Note.md'])
|
||||
await assert.rejects(api.readLibraryFile({ ...f.context, path }), { code: 'project-read-path' });
|
||||
await assert.rejects(api.readLibraryFile({ ...f.context, path: 'Book One/Missing.md' }), { code: 'project-read-missing' });
|
||||
});
|
||||
|
||||
test('C2-04 file: R1, R2 and R7 exclusions and never-synced mirrors cannot return file bodies', async t => {
|
||||
const f = await fixture(t, { ...BASE, 'Book One/Bad.md': { link: 'Characters/Tomas.md' },
|
||||
'Book One/Executable.md': { mode: 0o755, text: MARKER },
|
||||
'Book One/Twin.md': MARKER, 'Book One/twin.md': MARKER });
|
||||
await assert.rejects(api.readLibraryFile({ ...f.context, path: 'Book One/Characters/Tomas.md' }), { code: 'library-commit' });
|
||||
f.seed();
|
||||
for (const path of ['Book One/Bad.md', 'Book One/Executable.md', 'Book One/Twin.md', 'Book One/twin.md'])
|
||||
await assert.rejects(api.readLibraryFile({ ...f.context, path }), { code: 'project-read-missing' });
|
||||
});
|
||||
|
||||
function child(t, source) {
|
||||
const env = { ...process.env }; delete env.NODE_TEST_CONTEXT;
|
||||
const p = spawn(process.execPath, ['--input-type=module', '-e', source], { env, stdio: ['ignore', 'pipe', 'pipe', 'ipc'] });
|
||||
let stderr = ''; p.stderr.on('data', b => { stderr += b; }); p.stdout.resume();
|
||||
p.finished = new Promise(resolve => { p.once('error', e => resolve({ error: e })); p.once('close', code => resolve({ code, stderr })); });
|
||||
t.after(async () => { if (p.exitCode === null) p.kill('SIGKILL'); await p.finished; });
|
||||
return p;
|
||||
}
|
||||
|
||||
test('C2-04 lock: a first-sync and fast-forward reader wait for a consistent pair across processes', { timeout: 15000 }, async t => {
|
||||
const f = await fixture(t);
|
||||
// Transfer objects without serving them. No fetch or remote is added by the API.
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/candidate');
|
||||
for (const first of [true, false]) {
|
||||
if (!first) { f.seed(); writeFileSync(join(f.repo, 'Book One/New.md'), '# New\n'); git(f.repo, 'add', '.'); git(f.repo, 'commit', '-qm', 'new');
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/candidate'); }
|
||||
const commit = git(f.repo, 'rev-parse', 'HEAD');
|
||||
const writer = child(t, `import {withLibraryLock} from ${JSON.stringify(MOD)};import {writeFileSync} from 'node:fs';import {git} from ${JSON.stringify(new URL('../../project/src/git.mjs', import.meta.url).href)};
|
||||
await withLibraryLock(${JSON.stringify(f.mirror)},async()=>{await git(${JSON.stringify(f.mirror)},['update-ref','refs/heads/served',${JSON.stringify(commit)}]);process.send('moved');await new Promise(r=>process.once('message',r));writeFileSync(${JSON.stringify(stateFile(f.mirror))},${JSON.stringify(JSON.stringify(state(commit)))},{mode:0o600});});process.disconnect();`);
|
||||
const [message] = await once(writer, 'message'); assert.equal(message, 'moved');
|
||||
let complete = false;
|
||||
const read = api.readSyncState(f.mirror, { branch: 'main' }).then(value => { complete = true; return value; });
|
||||
await new Promise(r => setTimeout(r, 50)); assert.equal(complete, false, 'the inconsistent pair is not observed');
|
||||
writer.send('publish');
|
||||
assert.deepEqual(await read, state(commit));
|
||||
assert.deepEqual(await writer.finished, { code: 0, stderr: '' });
|
||||
}
|
||||
});
|
||||
|
||||
test('C2-04 lock: same-process waits, timeout code, callback expiry and throw release', { timeout: 15000 }, async t => {
|
||||
const f = await fixture(t); let release, scoped;
|
||||
const held = api.withLibraryLock(f.mirror, async lock => { scoped = lock.readSyncState; await new Promise(r => { release = r; }); });
|
||||
const deadline = Date.now() + 5000;
|
||||
while (!release && Date.now() < deadline) await new Promise(r => setTimeout(r, 5));
|
||||
assert.equal(typeof release, 'function', 'first lock acquired before deadline');
|
||||
await assert.rejects(api.readSyncState(f.mirror, { branch: 'main' }), { code: 'library-sync-busy' });
|
||||
release(); await held;
|
||||
await assert.rejects(scoped({ branch: 'main' }), { code: 'library-state-invalid' });
|
||||
await assert.rejects(api.withLibraryLock(f.mirror, async () => { throw Error('fixture throw'); }), /fixture throw/);
|
||||
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
|
||||
});
|
||||
|
||||
test('C2-04 lock: owner death releases the kernel lock without replacing its file', { timeout: 15000 }, async t => {
|
||||
const f = await fixture(t), lock = join(f.mirror, 'mosaic-library-sync.lock'), ino = lstatSync(lock).ino;
|
||||
const owner = child(t, `import {withLibraryLock} from ${JSON.stringify(MOD)};await withLibraryLock(${JSON.stringify(f.mirror)},async()=>{process.send('held');await new Promise(r=>process.once('message',r));});`);
|
||||
const [message] = await once(owner, 'message'); assert.equal(message, 'held');
|
||||
owner.kill('SIGKILL'); await owner.finished;
|
||||
assert.deepEqual(await api.readSyncState(f.mirror, { branch: 'main' }), state());
|
||||
assert.equal(lstatSync(lock).ino, ino);
|
||||
});
|
||||
|
||||
test('C2-04 lock: a missing flock binary becomes a handled refusal', { timeout: 10000 }, async t => {
|
||||
const f = await fixture(t);
|
||||
const missing = child(t, `import cp from 'node:child_process';import {syncBuiltinESMExports} from 'node:module';import {EventEmitter} from 'node:events';
|
||||
const original=cp.spawn;cp.spawn=(command,...args)=>{if(command!=='/usr/bin/flock')return original(command,...args);const p=new EventEmitter();process.nextTick(()=>p.emit('error',Object.assign(Error('fixture missing flock'),{code:'ENOENT'})));return p;};syncBuiltinESMExports();
|
||||
const {readSyncState}=await import(${JSON.stringify(MOD)});try{await readSyncState(${JSON.stringify(f.mirror)},{branch:'main'});process.send('unexpected success');}catch(e){process.send(e.code);}process.disconnect();`);
|
||||
const [code] = await once(missing, 'message');
|
||||
assert.equal(code, 'library-state-invalid');
|
||||
assert.deepEqual(await missing.finished, { code: 0, stderr: '' });
|
||||
});
|
||||
|
||||
test('C2-04 lock: an unawaited scoped read drains before the lock is released', { timeout: 10000 }, async t => {
|
||||
const f = await fixture(t); let scoped, pending, complete = false;
|
||||
await api.withLibraryLock(f.mirror, lock => {
|
||||
scoped = lock.readSyncState;
|
||||
pending = scoped({ branch: 'main' }).then(value => { complete = true; return value; });
|
||||
});
|
||||
assert.equal(complete, true);
|
||||
assert.deepEqual(await pending, state());
|
||||
await assert.rejects(scoped({ branch: 'main' }), { code: 'library-state-invalid' });
|
||||
});
|
||||
@@ -0,0 +1,151 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { readLibrary, parseFrontMatter, LibraryError, MAX_BLOB } from '../src/index.mjs';
|
||||
import { BASE, git, mapping, repo } from './helpers.mjs';
|
||||
|
||||
const codes = (result, path) => result.problems.filter(p => p.path === path).map(p => p.code);
|
||||
const file = (result, path) => result.files.find(f => f.path === path);
|
||||
|
||||
test('the fixture reads clean: every resolver step, appears_in and relationships wikilinks indexed', async t => {
|
||||
const r = repo(t);
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.equal(result.commit, r.commit);
|
||||
assert.deepEqual(result.files.map(f => [f.path, f.scope]), [
|
||||
['Book One/Chapters/Chapter 01.md', 'canon'], ['Book One/Characters/Mara Vell.md', 'canon'],
|
||||
['Book One/Characters/Tomas.md', 'canon'], ['Book One/Places/Harbor Town.md', 'canon'],
|
||||
['Book One/Playground/What If.md', 'playground'], ['Second World/Chapters/01.md', 'canon']]);
|
||||
const chapter = file(result, 'Book One/Chapters/Chapter 01.md');
|
||||
assert.deepEqual(chapter.links.map(l => [l.path, l.step, l.line]), [
|
||||
['Book One/Characters/Mara Vell.md', 'shortest', 2], ['Book One/Places/Harbor Town.md', 'root', 2],
|
||||
['Book One/Places/Harbor Town.md', 'relative', 3], ['Book One/Characters/Tomas.md', 'vault-prefix', 3]]);
|
||||
const mara = file(result, 'Book One/Characters/Mara Vell.md');
|
||||
assert.deepEqual(mara.frontmatter, { type: 'character', name: 'Mara Vell', appears_in: '[[Book One/Chapters/Chapter 01]]',
|
||||
relationships: ['[[Tomas]]', 'The Ferryman'] });
|
||||
assert.deepEqual(mara.links.map(l => [l.field, l.path, l.line]), [
|
||||
['appears_in', 'Book One/Chapters/Chapter 01.md', 4], ['relationships', 'Book One/Characters/Tomas.md', 5]]);
|
||||
assert.deepEqual(file(result, 'Book One/Characters/Tomas.md').links.map(l => [l.field, l.path]),
|
||||
[['appears_in', 'Book One/Chapters/Chapter 01.md']]);
|
||||
assert.equal(mara.bodyLine, 9);
|
||||
assert.equal(mara.lines[8][0], 9);
|
||||
assert.equal(mara.type, 'character');
|
||||
// Only the unmapped paths are problems: the top-level note and both .canvas files.
|
||||
assert.deepEqual(result.problems.map(p => [p.code, p.path]), [
|
||||
['R2', 'Book One/Board.canvas'], ['R2', 'Book One/Playground/Sketch.canvas'], ['R2', 'Inbox Note.md']]);
|
||||
assert.ok(Object.isFrozen(result.files[0].links));
|
||||
});
|
||||
|
||||
test('R1: executables, symlinks, gitlinks, blobs over 1 MiB and non-UTF-8 text are not indexed', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Places/Run.md': { text: 'x\n', mode: 0o755 },
|
||||
'Book One/Places/Link.md': { link: 'Harbor Town.md' },
|
||||
'Book One/Places/Big.md': 'x'.repeat(MAX_BLOB + 1),
|
||||
'Book One/Places/Edge.md': 'y'.repeat(MAX_BLOB),
|
||||
'Book One/Places/Latin.md': { bytes: Buffer.from([0x63, 0x61, 0x66, 0xe9, 0x0a]) },
|
||||
'Book One/Places/Nul.md': 'a\0b\n' });
|
||||
git(r.repo, 'update-index', '--add', '--cacheinfo', `160000,${r.commit},Book One/Places/Sub.md`);
|
||||
git(r.repo, 'commit', '-qm', 'gitlink');
|
||||
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
|
||||
for (const name of ['Run', 'Link', 'Big', 'Latin', 'Nul', 'Sub']) {
|
||||
assert.deepEqual(codes(result, `Book One/Places/${name}.md`), ['R1'], name);
|
||||
assert.equal(file(result, `Book One/Places/${name}.md`), undefined, name);
|
||||
}
|
||||
assert.ok(file(result, 'Book One/Places/Edge.md'), 'exactly 1 MiB is read');
|
||||
});
|
||||
|
||||
test('R2: unmapped paths are reported and not indexed', async t => {
|
||||
const r = repo(t);
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
for (const path of ['Inbox Note.md', 'Book One/Board.canvas', 'Book One/Playground/Sketch.canvas']) {
|
||||
assert.deepEqual(codes(result, path), ['R2']);
|
||||
assert.equal(file(result, path), undefined);
|
||||
}
|
||||
});
|
||||
|
||||
test('R3: front matter not fully parsed stays raw, with the line where parsing stopped', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Places/Nested.md': ['---', 'type: place', 'coords:', ' lat: 1', '---', 'Body [[Tomas]]', ''].join('\n'),
|
||||
'Book One/Places/Block.md': ['---', 'summary: |', ' text', '---', ''].join('\n'),
|
||||
'Book One/Places/Open.md': ['---', 'type: place', 'never closed', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
const nested = file(result, 'Book One/Places/Nested.md');
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Places/Nested.md'), [{ code: 'R3', path: 'Book One/Places/Nested.md', line: 4, message: 'front matter not fully parsed' }]);
|
||||
assert.equal(nested.frontmatter, null);
|
||||
assert.equal(nested.frontmatterRaw, 'type: place\ncoords:\n lat: 1');
|
||||
assert.deepEqual(nested.links.map(l => l.path), ['Book One/Characters/Tomas.md'], 'still indexed by body');
|
||||
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Block.md').line, 2);
|
||||
assert.equal(result.problems.find(p => p.path === 'Book One/Places/Open.md').line, 1);
|
||||
assert.equal(file(result, 'Book One/Places/Open.md').bodyLine, 1);
|
||||
});
|
||||
|
||||
test('front matter forms the reader accepts and refuses', () => {
|
||||
const parse = text => parseFrontMatter(text.split('\n'));
|
||||
assert.deepEqual(parse('---\na: plain words\nb: "quoted: yes"\nc: \'it\'\'s\'\nd: [x, "y z", 3]\ne:\n- one\n- "[[Two]]"\nf: [[Wiki]]\ng: true\nh:\n---').frontmatter,
|
||||
{ a: 'plain words', b: 'quoted: yes', c: "it's", d: ['x', 'y z', 3], e: ['one', '[[Two]]'], f: '[[Wiki]]', g: true, h: null });
|
||||
for (const bad of ['a: {x: 1}', 'a: &anchor x', 'a: [x, [y]]', 'a: x: y', 'a: 1\na: 2', ' a: 1', 'a: "open', '- loose'])
|
||||
assert.notEqual(parse(`---\n${bad}\n---`).stop, null, bad);
|
||||
assert.deepEqual(parse('no front matter'), { frontmatter: null, raw: null, bodyLine: 1, stop: null });
|
||||
});
|
||||
|
||||
test('R4: a link to no file, to more than one, or into another repo is reported and not followed', async t => {
|
||||
const r = repo(t, { ...BASE,
|
||||
'Book One/Research/Notes.md': 'a\n', 'Book One/Places/Notes.md': 'b\n',
|
||||
'Book One/Chapters/Links.md': ['[[Nobody]]', '[[Notes]]', '[[bean-brain/Journal/Today]]', '[[Tomas]]', ''].join('\n') });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(result.problems.filter(p => p.path === 'Book One/Chapters/Links.md').map(p => [p.code, p.line]), [['R4', 1], ['R4', 2], ['R4', 3]]);
|
||||
assert.deepEqual(file(result, 'Book One/Chapters/Links.md').links.map(l => l.path), ['Book One/Characters/Tomas.md']);
|
||||
});
|
||||
|
||||
test('R5: a canon file that refers into the playground is reported, and the link still goes by path', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Dock.md': 'See [[What If]].\n' });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(codes(result, 'Book One/Places/Dock.md'), ['R5']);
|
||||
assert.deepEqual(file(result, 'Book One/Places/Dock.md').links.map(l => l.path), ['Book One/Playground/What If.md']);
|
||||
// The playground file links to canon: no problem, R5 is only about canon.
|
||||
assert.deepEqual(codes(result, 'Book One/Playground/What If.md'), []);
|
||||
});
|
||||
|
||||
test('R6: a canon file that refers into another universe is reported; a hybrid may refer into its sources', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Far.md': 'Like [[Second World/Chapters/01]].\n',
|
||||
'Crossover/Chapters/A.md': 'From [[Mara Vell]] and [[Second World/Chapters/01]].\n' });
|
||||
const hybrid = mapping(e => { e.universes.cross = { kind: 'hybrid', name: 'Cross', sources: ['harbor', 'second'], books: ['Crossover'] }; return e; });
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: hybrid });
|
||||
assert.deepEqual(codes(result, 'Book One/Places/Far.md'), ['R6']);
|
||||
assert.deepEqual(file(result, 'Book One/Places/Far.md').links.map(l => l.path), ['Second World/Chapters/01.md']);
|
||||
assert.deepEqual(codes(result, 'Crossover/Chapters/A.md'), []);
|
||||
});
|
||||
|
||||
test('R7: two paths that differ only in case are both left out', async t => {
|
||||
const r = repo(t, BASE);
|
||||
writeFileSync(join(r.root, 'twin'), 'twin\n');
|
||||
const blob = git(r.repo, 'hash-object', '-w', join(r.root, 'twin'));
|
||||
git(r.repo, 'update-index', '--add', '--cacheinfo', `100644,${blob},Book One/Characters/tomas.md`);
|
||||
git(r.repo, 'commit', '-qm', 'case twin');
|
||||
const result = await readLibrary({ repo: r.repo, commit: git(r.repo, 'rev-parse', 'HEAD'), mapping: mapping() });
|
||||
for (const path of ['Book One/Characters/Tomas.md', 'Book One/Characters/tomas.md']) {
|
||||
assert.deepEqual(codes(result, path), ['R7'], path);
|
||||
assert.equal(file(result, path), undefined);
|
||||
}
|
||||
});
|
||||
|
||||
test('the reader reads the commit, never the working tree, and refuses a bad commit id', async t => {
|
||||
const r = repo(t);
|
||||
writeFileSync(join(r.repo, 'Book One/Places/Harbor Town.md'), 'unsaved [[Nobody]]\n');
|
||||
writeFileSync(join(r.repo, 'Book One/Places/Untracked.md'), 'x\n');
|
||||
const result = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
assert.equal(file(result, 'Book One/Places/Harbor Town.md').lines[0][1], '---');
|
||||
assert.equal(file(result, 'Book One/Places/Untracked.md'), undefined);
|
||||
for (const commit of ['HEAD', r.commit.slice(0, 12), '0'.repeat(40), git(r.repo, 'rev-parse', 'HEAD^{tree}')])
|
||||
await assert.rejects(readLibrary({ repo: r.repo, commit, mapping: mapping() }), e => e instanceof LibraryError, commit);
|
||||
await assert.rejects(readLibrary({ repo: r.repo, commit: r.commit, mapping: {} }), /library-mapping-invalid/);
|
||||
});
|
||||
|
||||
test('a bare mirror reads the same as the clone', async t => {
|
||||
const r = repo(t);
|
||||
const bare = join(r.root, 'mirror.git');
|
||||
git(r.root, 'clone', '-q', '--bare', r.repo, bare);
|
||||
const a = await readLibrary({ repo: r.repo, commit: r.commit, mapping: mapping() });
|
||||
const b = await readLibrary({ repo: bare, commit: r.commit, mapping: mapping() });
|
||||
assert.deepEqual(b, a);
|
||||
});
|
||||
@@ -0,0 +1,80 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { renderItems, UNREADABLE } from '../src/index.mjs';
|
||||
|
||||
const A = 'a'.repeat(40), B = 'b'.repeat(40);
|
||||
const cite = (path, lines, quote, commit = A) => ({ path, commit, lines, quote });
|
||||
const options = {
|
||||
scopeOf: path => path.includes('/Playground/') ? 'playground' : path.endsWith('.md') ? 'canon' : undefined,
|
||||
changed: ({ path, commit }) => path === 'Book One/Characters/Mara Vell.md' && commit === A,
|
||||
};
|
||||
const render = items => renderItems(JSON.stringify({ items }), options);
|
||||
|
||||
// One snapshot per item kind: the plain text both channels show.
|
||||
const SNAPSHOTS = {
|
||||
fact: [{ kind: 'fact', text: 'Tomas is the ferryman.', cites: [cite('Book One/Characters/Tomas.md', [3, 3], 'the ferryman', B)] }],
|
||||
finding: [{ kind: 'finding', text: 'Eye colour differs.', cites: [cite('Book One/Chapters/Chapter 01.md', [41, 41], 'her grey eyes'),
|
||||
cite('Book One/Characters/Mara Vell.md', [9, 10], 'green eyes')] }],
|
||||
correction: [{ kind: 'correction', text: 'Chapter 1 says grey.', from: 'green eyes', to: 'grey eyes',
|
||||
cites: [cite('Book One/Characters/Mara Vell.md', [9, 9], 'green eyes', B)] }],
|
||||
question: [{ kind: 'question', text: 'Is the harbor north of the town?', cites: [cite('Book One/Places/Harbor Town.md', [2, 4], 'the harbor', B)] }],
|
||||
idea: [{ kind: 'idea', text: 'A storm could close the harbor.', cites: [cite('Book One/Playground/What If.md', [5, 5], 'a storm', B),
|
||||
{ url: 'https://example.invalid/tides', quote: 'spring tides' }] }],
|
||||
};
|
||||
const EXPECTED = {
|
||||
fact: '1. Fact: Tomas is the ferryman.\n [canon] Book One/Characters/Tomas.md:3 @bbbbbbb "the ferryman"',
|
||||
finding: '1. Finding: Eye colour differs.\n [canon] Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
|
||||
+ ' [canon] Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes" (changed since)',
|
||||
correction: '1. Correction: Chapter 1 says grey.\n From: "green eyes"\n To: "grey eyes"\n'
|
||||
+ ' [canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes"',
|
||||
question: '1. Question: Is the harbor north of the town?\n [canon] Book One/Places/Harbor Town.md:2-4 @bbbbbbb "the harbor"',
|
||||
idea: '1. Idea: A storm could close the harbor.\n [playground] Book One/Playground/What If.md:5 @bbbbbbb "a storm"\n'
|
||||
+ ' [web] https://example.invalid/tides "spring tides"',
|
||||
};
|
||||
|
||||
for (const kind of Object.keys(SNAPSHOTS)) {
|
||||
test(`renderItems snapshot: ${kind}`, () => {
|
||||
const out = render(SNAPSHOTS[kind]);
|
||||
assert.equal(out.kind, 'items');
|
||||
assert.equal(out.text, EXPECTED[kind]);
|
||||
assert.equal(out.items[0].kind, kind);
|
||||
});
|
||||
}
|
||||
|
||||
test('renderItems numbers several items and takes a parsed body as well as a string', () => {
|
||||
const items = [...SNAPSHOTS.fact, ...SNAPSHOTS.question];
|
||||
const out = renderItems({ items }, options);
|
||||
assert.equal(out.text, `${EXPECTED.fact}\n${EXPECTED.question.replace(/^1\./, '2.')}`);
|
||||
assert.deepEqual(out.items[1].cites[0], { badge: 'canon', path: 'Book One/Places/Harbor Town.md', commit: B, lines: [2, 4],
|
||||
ref: 'Book One/Places/Harbor Town.md:2-4 @bbbbbbb', quote: 'the harbor', changed: false });
|
||||
});
|
||||
|
||||
test('without options a path cite has no badge and is never marked changed', () => {
|
||||
assert.equal(renderItems({ items: SNAPSHOTS.finding }).text,
|
||||
'1. Finding: Eye colour differs.\n Book One/Chapters/Chapter 01.md:41 @aaaaaaa "her grey eyes"\n'
|
||||
+ ' Book One/Characters/Mara Vell.md:9-10 @aaaaaaa "green eyes"');
|
||||
});
|
||||
|
||||
const NOTICE_SNAPSHOTS = { 'no-cited-answer': 'No cited answer', 'no-answer': 'No answer', 'turn-failed': 'The turn failed' };
|
||||
for (const [notice, text] of Object.entries(NOTICE_SNAPSHOTS)) {
|
||||
test(`renderItems snapshot: notice ${notice}`, () => {
|
||||
assert.deepEqual(renderItems(JSON.stringify({ items: [], notice }), options), { kind: 'notice', notice, text });
|
||||
});
|
||||
}
|
||||
|
||||
test('a body without the reply shape shows a fixed line and none of its text', () => {
|
||||
const secret = 'free prose the guard would refuse';
|
||||
const bodies = [secret, JSON.stringify(secret), '{"items": [], "notice": "other"}', JSON.stringify({ items: [], notice: 'no-answer', extra: secret }),
|
||||
JSON.stringify({ items: [] }), JSON.stringify({ items: [{ kind: 'story', text: secret, cites: [] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret }] }), JSON.stringify({ items: SNAPSHOTS.fact, prose: secret }),
|
||||
JSON.stringify({ items: [{ kind: 'correction', text: secret, cites: [] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [cite('a.md', [2, 1], 'q')] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ path: 'a.md', commit: 'abc', lines: [1, 1], quote: 'q' }] }] }),
|
||||
JSON.stringify({ items: [{ kind: 'fact', text: secret, cites: [{ url: 'https://x.invalid', path: 'a.md', quote: 'q' }] }] }),
|
||||
null, 42, '[]'];
|
||||
for (const body of bodies) {
|
||||
const out = renderItems(body, options);
|
||||
assert.deepEqual(out, { kind: 'unreadable', text: UNREADABLE }, String(body));
|
||||
assert.ok(!JSON.stringify(out).includes(secret));
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,82 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { surveyMain } from '../src/index.mjs';
|
||||
import { BASE, FIXTURES, MARKER, repo } from './helpers.mjs';
|
||||
|
||||
const ROOT = fileURLToPath(new URL('../../..', import.meta.url));
|
||||
async function run(args) {
|
||||
let stdout = '', stderr = '';
|
||||
const code = await surveyMain(args, { stdout: { write: s => { stdout += s; } }, stderr: { write: s => { stderr += s; } } });
|
||||
return { code, stdout, stderr };
|
||||
}
|
||||
// Strings from the fixture that are text, values, file names or link targets: none may be printed.
|
||||
const NEVER = [MARKER, 'Grey eyes', 'The Ferryman', 'Mara Vell', 'Tomas', 'Harbor Town', 'What If', 'Inbox Note', 'Chapter 01',
|
||||
'Sketch', 'Board', 'Not A Link', 'the harbor', 'Docks', 'character'];
|
||||
|
||||
test('the survey prints folder names, counts, key sets and link-form counts, and never text', async t => {
|
||||
const r = repo(t, { ...BASE, 'Book One/Places/Odd.md': ['---', `"${MARKER} key": 1`, '---', ''].join('\n') });
|
||||
const out = await run(['survey', r.repo]);
|
||||
assert.equal(out.code, 0, out.stderr);
|
||||
assert.equal(out.stdout, [
|
||||
`commit ${r.commit}`,
|
||||
'entries 10',
|
||||
'folders (direct children: md canvas base other)',
|
||||
' (root) 1 0 0 0',
|
||||
' "Book One" 0 1 0 0',
|
||||
' "Book One/Chapters" 1 0 0 0',
|
||||
' "Book One/Characters" 2 0 0 0',
|
||||
' "Book One/Places" 2 0 0 0',
|
||||
' "Book One/Playground" 1 1 0 0',
|
||||
' "Second World/Chapters" 1 0 0 0',
|
||||
'markdown 8 files, 8 read, 3 without front matter, 0 with front matter not fully parsed',
|
||||
'front matter key sets',
|
||||
' 1 [(other key)]',
|
||||
' 1 [appears_in, name, relationships, type]',
|
||||
' 1 [appears_in, name, type]',
|
||||
' 1 [name]',
|
||||
' 1 [type]',
|
||||
'body links plain 3, label 1, heading 1, embed 0, self 0, markdown 1',
|
||||
'field appears_in wikilink 2, name 0',
|
||||
'field relationships wikilink 1, name 1',
|
||||
''].join('\n'));
|
||||
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
|
||||
});
|
||||
|
||||
test('with --mapping the survey adds scope, resolver step and problem counts', async t => {
|
||||
const r = repo(t);
|
||||
const out = await run(['survey', r.repo, '--mapping', join(FIXTURES, 'mapping.json')]);
|
||||
assert.equal(out.code, 0, out.stderr);
|
||||
const tail = out.stdout.trimEnd().split('\n').slice(-3);
|
||||
assert.deepEqual(tail, [
|
||||
'scopes canon 5, playground 1, unmapped 3; indexed 6',
|
||||
'resolved root 2, vault-prefix 1, relative 2, shortest 3, unresolved 0',
|
||||
'problems R1 0, R2 3, R3 0, R4 0, R5 0, R6 0, R7 0']);
|
||||
for (const s of NEVER) assert.ok(!out.stdout.includes(s), s);
|
||||
});
|
||||
|
||||
test('survey exit codes: usage 4, bad mapping 3, unreadable repo 2', async t => {
|
||||
const r = repo(t);
|
||||
for (const args of [[], ['scan', r.repo], ['survey'], ['survey', r.repo, '--mapping'], ['survey', r.repo, '--other'], ['survey', r.repo, r.repo]])
|
||||
assert.equal((await run(args)).code, 4, args.join(' '));
|
||||
const bad = join(r.root, 'bad.json');
|
||||
writeFileSync(bad, JSON.stringify({ kind: 'library' }));
|
||||
const refused = await run(['survey', r.repo, '--mapping', bad]);
|
||||
assert.equal(refused.code, 3);
|
||||
assert.match(refused.stderr, /library-mapping-invalid/);
|
||||
writeFileSync(bad, 'not json');
|
||||
assert.equal((await run(['survey', r.repo, '--mapping', bad])).code, 3);
|
||||
assert.equal((await run(['survey', join(r.root, 'missing')])).code, 2);
|
||||
assert.equal((await run(['survey', r.root])).code, 2, 'a directory that is not a repo');
|
||||
});
|
||||
|
||||
test('scripts/mosaic library survey reaches the survey', t => {
|
||||
const r = repo(t);
|
||||
const out = spawnSync(join(ROOT, 'scripts/mosaic'), ['library', 'survey', r.repo], { encoding: 'utf8', timeout: 30000 });
|
||||
assert.equal(out.status, 0, out.stderr);
|
||||
assert.match(out.stdout, new RegExp(`^commit ${r.commit}\n`));
|
||||
assert.ok(!out.stdout.includes(MARKER));
|
||||
});
|
||||
@@ -31,14 +31,17 @@ const contains = (base, child) => {
|
||||
const rel = relative(base, child);
|
||||
return rel === '' || (rel !== '..' && !rel.startsWith('..' + sep) && !isAbsolute(rel));
|
||||
};
|
||||
export function projectRepo({ business, project, dataRoot }) {
|
||||
export function projectRepo({ business, project, dataRoot }, { allowLibrary = false } = {}) {
|
||||
checkedId(business?.id); checkedId(project);
|
||||
if (!Object.hasOwn(business?.projects ?? {}, project))
|
||||
throw new ProjectError('project-not-declared', `business does not declare ${project}`);
|
||||
absolute(dataRoot);
|
||||
const entry = business.projects[project];
|
||||
if (entry.kind === 'library' && !allowLibrary)
|
||||
throw new ProjectError('project-invalid', 'library projects are not record repositories');
|
||||
const repo = absolute(entry.repo ?? join(dataRoot, 'projects', business.id, project));
|
||||
for (const project of Object.values(business.projects)) {
|
||||
if (project.kind === 'library') continue;
|
||||
const code = absolute(project.root);
|
||||
if (contains(code, repo) || contains(repo, code))
|
||||
throw new ProjectError('project-path', 'record repository and code checkout overlap');
|
||||
|
||||
@@ -24,6 +24,19 @@ async function ready(t, custom) {
|
||||
return { c, repo: initialized.repo, commit: initialized.commit };
|
||||
}
|
||||
|
||||
test('C2-04: record APIs refuse a library before creating a repository', async t => {
|
||||
const c = context(t);
|
||||
c.business.projects = { books: { kind: 'library' }, ...c.business.projects };
|
||||
assert.equal(api.projectRepo(c), join(c.dataRoot, 'projects/acme/stack'));
|
||||
const library = { ...c, project: 'books' };
|
||||
assert.throws(() => api.projectRepo(library), { code: 'project-invalid' });
|
||||
for (const author of ['jason', 'tester'])
|
||||
await assert.rejects(api.initProject(library, { author }), { code: 'project-invalid' });
|
||||
assert.equal(readdirSync(c.root).includes('data'), false);
|
||||
c.business.projects.stack.root = c.dataRoot;
|
||||
assert.throws(() => api.projectRepo(c), /overlap/);
|
||||
});
|
||||
|
||||
test('init creates a checked versioned layout, valid templates and human-authored local history', async t => {
|
||||
const { c, repo, commit } = await ready(t);
|
||||
assert.equal(repo, join(c.dataRoot, 'projects/acme/stack'));
|
||||
|
||||
@@ -311,7 +311,9 @@ async function update(root, dest, g) {
|
||||
export async function ensureCheckouts({ workspace, projects, env = process.env, limits = GIT_LIMITS }) {
|
||||
const g = (args, extra) => git(args, env, extra, limits);
|
||||
const out = [];
|
||||
for (const [project, { root }] of Object.entries(projects)) {
|
||||
for (const [project, entry] of Object.entries(projects)) {
|
||||
if (entry.kind === 'library') continue;
|
||||
const { root } = entry;
|
||||
const dest = join(workspace, project);
|
||||
let exists = true;
|
||||
try {
|
||||
|
||||
@@ -12,6 +12,15 @@ import { join } from "node:path";
|
||||
import { deflateSync } from "node:zlib";
|
||||
import { CONFIG_MAX_BYTES, CheckoutError, ensureCheckouts, gitEnv } from "../src/checkout.mjs";
|
||||
|
||||
test('C2-04: ensureCheckouts skips a library before a real clone and leaves no books path', async t => {
|
||||
const w = world(t);
|
||||
const result = await ensureCheckouts({ workspace: w.workspace, projects: { books: { kind: 'library' }, stack: { root: w.root } } });
|
||||
assert.deepEqual(result.map(r => r.project), ['stack']);
|
||||
assert.equal(result[0].outcome, 'cloned');
|
||||
assert.equal(existsSync(join(w.workspace, 'books')), false);
|
||||
assert.deepEqual(await ensureCheckouts({ workspace: w.workspace, projects: { books: { kind: 'library' } } }), []);
|
||||
});
|
||||
|
||||
function tmp(t) {
|
||||
const dir = mkdtempSync(join(tmpdir(), "mosaic-checkout-"));
|
||||
t.after(() => {
|
||||
|
||||
@@ -64,7 +64,7 @@ try {
|
||||
// the system config, Business can't resolve authority and Settings has
|
||||
// no notifier. A refusal printed here names no path (redactor).
|
||||
// --send without a bus refuses to start rather than serve a composer that can't send.
|
||||
let bus = null, send = null, settingsWrite = null, projectFiles = null, projectFile = null, projectSave = null, chat = null, nameWrite = null, assistantStop = null, instancesWrite = null, routesWrite = null, busLine, system = null;
|
||||
let bus = null, send = null, settingsWrite = null, projectFiles = null, projectFile = null, libraryFile = null, libraryStatus = null, projectSave = null, chat = null, nameWrite = null, assistantStop = null, instancesWrite = null, routesWrite = null, busLine, system = null;
|
||||
try {
|
||||
system = loadSystem();
|
||||
const { dataRoot } = system;
|
||||
@@ -77,6 +77,7 @@ try {
|
||||
// Row 148: the spec and docs editor reads like the other reads and saves only under --send.
|
||||
projectFiles = args => call('project.files', args);
|
||||
projectFile = args => call('project.file', args);
|
||||
libraryFile = args => call('library.file', args); libraryStatus = args => call('library.status', args);
|
||||
if (sendOn) {send = args => call('message.send', args);settingsWrite=args=>call('limits.set',args);projectSave=args=>call('project.file.save',args);nameWrite=args=>call('name.set',args);assistantStop=args=>call('assistant.stop',args);instancesWrite=args=>call('instances.set',args);routesWrite=args=>call('routes.set',args);}
|
||||
busLine = `Bus: ${options.business ?? 'the running bus host\'s business'}`;
|
||||
} catch (err) {
|
||||
@@ -97,7 +98,7 @@ try {
|
||||
}
|
||||
let server;
|
||||
try {
|
||||
server = await startServer({ board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite });
|
||||
server = await startServer({ board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, libraryFile, libraryStatus, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite });
|
||||
} catch (err) { sso?.close(); throw err; }
|
||||
let testerServer = null;
|
||||
if (testerPort !== null) {
|
||||
@@ -107,6 +108,7 @@ try {
|
||||
testerServer = await startServer({ host: '127.0.0.1', port: testerPort, board: options.board,
|
||||
tester: true, testerNoSignIn: ssoOn, testerProofHeader: proof?.header ?? null, testerAuthHeader: options.preview?.header ?? null, bus: testerBus, reads,
|
||||
projectFiles: args => call('project.files', args), projectFile: args => call('project.file', args),
|
||||
libraryFile: args => call('library.file', args), libraryStatus: args => call('library.status', args),
|
||||
instancesWrite: args => call('instances.set', args), routesWrite: args => call('routes.set', args), nameWrite: args => call('name.set', args),
|
||||
// Row 177 (#1675): the tester's own chat, through the tester CLI; 7332's stays the human's.
|
||||
chat: createChat({ call }),
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
// C2-04 only wires callbacks. HTTP routes and pages are later rows.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { readFileSync, writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { tmp, fixture } from '../../cli/tests/helpers.mjs';
|
||||
import { HUMAN_CLI, TESTER_CLI } from '../../cli/src/transport.mjs';
|
||||
|
||||
test('C2-04 CLI: both listener callback pairs dispatch library verbs with their own transport', t => {
|
||||
const root = tmp(t, 'library-cli-'), f = fixture(root), log = join(root, 'calls.jsonl');
|
||||
writeFileSync(log, '');
|
||||
const hook = join(root, 'hooks.mjs');
|
||||
// The loader supplies a fixture bus and captures callbacks instead of
|
||||
// opening an HTTP listener. Production imports and routes are unchanged.
|
||||
const serveSource = "export const DEFAULT_BOARD='http://127.0.0.1:9';export const previewOptions=()=>null;export const proofOptions=()=>null;export async function startServer(o){if(typeof o.libraryFile!=='function'||typeof o.libraryStatus!=='function')throw Error('missing library callbacks');await o.libraryFile({project:'books',path:'Book One/A.md'});await o.libraryStatus({project:'books'});return {address:()=>({port:0}),close:()=>{}};}";
|
||||
const busSource = `import {appendFileSync} from 'node:fs';export class BusReadError extends Error{};export const busCaller=o=>async(verb,args)=>{appendFileSync(${JSON.stringify(log)},JSON.stringify({verb,args,cli:o.cli??${JSON.stringify(HUMAN_CLI)}})+'\\n');return {};};`;
|
||||
writeFileSync(hook, `import {registerHooks} from 'node:module';
|
||||
registerHooks({load(url,context,next){
|
||||
if(url.endsWith('/webui/src/serve.mjs'))return {format:'module',shortCircuit:true,source:${JSON.stringify(serveSource)}};
|
||||
if(url.endsWith('/webui/src/bus.mjs'))return {format:'module',shortCircuit:true,source:${JSON.stringify(busSource)}};
|
||||
return next(url,context);
|
||||
}});`);
|
||||
const result = spawnSync(process.execPath, [new URL('../src/cli.mjs', import.meta.url).pathname,
|
||||
'serve', '--business', 'acme', '--port', '0', '--tester-port', '7444'],
|
||||
{ env: { ...f.env, NODE_OPTIONS: `--import=${hook}` }, encoding: 'utf8', timeout: 10000 });
|
||||
assert.equal(result.status, 0, result.stdout + result.stderr);
|
||||
const calls = readFileSync(log, 'utf8').trim().split('\n').filter(Boolean).map(JSON.parse);
|
||||
assert.deepEqual(calls.map(c => [c.verb, c.cli]), [
|
||||
['library.file', HUMAN_CLI], ['library.status', HUMAN_CLI],
|
||||
['library.file', TESTER_CLI], ['library.status', TESTER_CLI],
|
||||
]);
|
||||
assert.equal(calls[0].args.path, 'Book One/A.md');
|
||||
});
|
||||
@@ -34,6 +34,12 @@ if [ "${1:-}" = business ]; then
|
||||
shift
|
||||
exec node "$REPO/packages/business/src/cli.mjs" "$@"
|
||||
fi
|
||||
# `mosaic library survey`: names and counts from a writing repo, never its text
|
||||
# (packages/library/README.md). Other library verbs are bus verbs.
|
||||
if [ "${1:-}" = library ] && [ "${2:-}" = survey ]; then
|
||||
shift
|
||||
exec node "$REPO/packages/library/src/survey.mjs" "$@"
|
||||
fi
|
||||
case "${1:-}" in
|
||||
usage|project|limits|inbox|decide|tasks|agents|trail|talk|stop|launches|bus|plugins|credentials|hosts|name|names|follow|threads|thread) exec node "$REPO/packages/cli/src/cli.mjs" "$@" ;;
|
||||
esac
|
||||
|
||||
@@ -33,7 +33,7 @@ COPY_PATHS=(
|
||||
packages/conversation/src packages/discord/src packages/discord/bin packages/discord/extension packages/harness/src
|
||||
packages/seat/src packages/tasks/src packages/tasks/fields.json packages/tasks/requirements.json
|
||||
packages/webui/src packages/webui/README.md packages/cli/README.md
|
||||
packages/queue/src
|
||||
packages/queue/src packages/library/src
|
||||
# Whole directory: every in-repo plug-in module ships with the copy (row 131).
|
||||
packages/plugins
|
||||
)
|
||||
|
||||
Reference in new issue
Block a user