Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3094e3bfa4 | ||
|
|
a8f0cf9248 |
No files matched your search
@@ -1,170 +0,0 @@
|
||||
# Row 224 — C2-05b broker lookups and typed tools (#1725)
|
||||
|
||||
Owner: Della; reviewer: Dewey. Assignment sage-2795; scope extension sage-2804.
|
||||
Base: `4f3fbc25a` (Researcher C2-05a). Branch: `della/row224`.
|
||||
Brief: C2-05b in `docs/plans/2026-10-20_case2-rows.md` at `aca16404c`
|
||||
and `51cc0445c`; design section 3, 5.3 M3 and 7.2; C2-05a BUILD rulings.
|
||||
|
||||
## Execution boundary and progress
|
||||
|
||||
Code/docs and local Git only while row 253 holds the gate. No Node (including
|
||||
syntax checks), HTTPS Git, queue claim, test or push until Sage's CLEAR.
|
||||
Tests are written first; their RED/GREEN runs are deferred, not claimed.
|
||||
Queue next is deferred too: its executable uses Node. Sage assigned this row
|
||||
explicitly. After 253 and 216 land, rebase under a new branch name.
|
||||
|
||||
- Pre-flight: query.mjs consumes `{repo,commit,mapping}`; broker cannot import
|
||||
Git/filesystem library code because it is in the sandbox import closure.
|
||||
Runtime injects the host lookup adapter; existing human file/status/sync stay.
|
||||
- Pre-flight: resolved has no project list and a library is never a checkout.
|
||||
Bundle needs an explicit library flag from launcher business configuration.
|
||||
- M3 reading: exact creative base is five lookups plus send_message; retain
|
||||
explicitly granted broker web_search/web_fetch for research/companion under
|
||||
landed C2-08 and sage-2804. No generic spec, task, evidence or decision tools. Residual (mos-1308): those
|
||||
reads lack broker action checks; their typed-tool absence is the only fence
|
||||
for creative roles. M1/M2 deny bash; broker-side refusal is outside this row.
|
||||
Mos confirmed this interpretation in mos-1308; design wording fix belongs to Gosalyn. Other definitions retain their current tools.
|
||||
- Multi-library businesses use an optional project ID; omit it only with one
|
||||
configured library. Universe is required for all five tools, including get.
|
||||
- Missing library uses the brief's `library-not-configured`; argument errors
|
||||
reuse `invalid-request` and C2-05a query codes. No new action or authority.
|
||||
|
||||
## Manifest (draft code, unrun)
|
||||
|
||||
- bus/src/broker.mjs: five lookup verbs, strict args, holder/business rechecks.
|
||||
- bus/src/runtime.mjs: host lookup adapter, stale-before-read sync, served pin.
|
||||
- harness/src/tools.mjs: five schemas; creative exact-set filtering.
|
||||
- harness/src/bundle.mjs: library flag forwarded to typedTools.
|
||||
- cli/src/launcher.mjs: one flag derived from trusted business projects.
|
||||
- bus/tests/library-lookups.test.mjs: real query/broker/socket/runtime cases.
|
||||
- harness/tests/library-tools.test.mjs: schemas, role snapshots, bundle parity.
|
||||
- cli/tests/thread-launch.test.mjs: flag reaches a real launched bundle with no library checkout.
|
||||
- agents/della/work/queue-224/BUILD.md: manifest, rulings and deferred verification ledger.
|
||||
|
||||
## Verification ledger
|
||||
|
||||
No test or syntax-check run yet. Acceptance requires full bus/cli/harness suites
|
||||
and every scripts/test-*.sh on Sage's gate; focused files and affected CLI/
|
||||
seat import-closure tests precede that only in an authorized CLEAR.
|
||||
|
||||
## Coordination
|
||||
|
||||
Huey272 reports no runtime/bundle/launcher overlap. Rocko219 changes launcher
|
||||
THREAD_DEFINITIONS checks only, separate from our buildBundle flag argument.
|
||||
The launcher argument is required: bundle cannot derive library presence from
|
||||
checkout IDs (a library has no agent clone). No role, authority or config schema
|
||||
changes; library project entries are already validated and protected by M9.
|
||||
|
||||
## Mount and pin impact (inspection only)
|
||||
|
||||
No new imports in broker.mjs or tools.mjs; no RUNNER_FILES or sandbox mount
|
||||
addition. runtime.mjs is host-only and already imports library index.mjs.
|
||||
Bundle forwards only a boolean, never a root/mapping/slot/credential. The
|
||||
existing import-closure tests are bus/tests/project-runtime.test.mjs and
|
||||
seat/tests/sandbox.test.mjs. Grep found no literal sha256 pin for runtime.mjs
|
||||
or bundle.mjs; bundle tests derive current source digests through SOURCES.
|
||||
Library sources remain in the existing whole-directory release copy.
|
||||
|
||||
## Runs to schedule only after CLEAR
|
||||
|
||||
Set TMPDIR=/mnt/storage/scratch/tmp. Focused new bus/harness files first,
|
||||
then existing library.test.mjs, library-sync.test.mjs, project-runtime.test.mjs,
|
||||
harness creative/tools/spec-tools/bundle files and affected CLI launch tests.
|
||||
Follow with the required whole bus/cli/harness suites and Sage's full shell gate.
|
||||
No timing claim yet: use the repository/Sage per-file limits and record every
|
||||
command, outcome, elapsed time and timeout before acceptance. No focused skip
|
||||
or pattern run substitutes for whole-file gate evidence.
|
||||
|
||||
## Read-through findings and residuals
|
||||
|
||||
- Library get requires universe although the older design table omitted it;
|
||||
section 3 isolation rule and C2-05a's accepted API are binding.
|
||||
- Query type enum matches mapping.TYPES (five values); manuscript is a distinct
|
||||
lookup and not a find type. Unknown host-path/commit/business/accept fields
|
||||
are refused; explicit project is restricted to this business's library list.
|
||||
- Async reads recheck the live holder and current project revision. A stale
|
||||
read attempts sync, then takes the new served snapshot; a returned fetch
|
||||
failure/divergence keeps serving the old pin. Busy sync uses the current pin.
|
||||
Unreadable state, no served commit or unsafe configuration refuse. Query
|
||||
and sync never hold a SQLite transaction over await.
|
||||
- Processing halt refuses new agent lookup work and rechecks before returning
|
||||
async results. Existing human library file/status/sync behavior is unchanged.
|
||||
- Timer sync and reads still use C2-03a's 60-second contract. No timer tuning or
|
||||
resource-limit schema is introduced by this row. Q222 creative reply settings
|
||||
are in C2-06; no reply guard or writing capability is implemented here.
|
||||
- The generic tool socket's existing 60-second client timeout can precede the
|
||||
server's 120-second Git timeout. This row does not change the generic timeout;
|
||||
a timed-out lookup is read-only and retryable, never a write/outcome claim.
|
||||
- All tests are authored but unrun. No RED, GREEN, syntax, runtime performance,
|
||||
sandbox, broker/harness suite, shell gate, claim or push evidence exists yet.
|
||||
|
||||
## Candidate review checklist (read-through only)
|
||||
|
||||
- [x] Five verbs mapped to real C2-05a methods, all require universe.
|
||||
- [x] Default sole library or explicit owned project; no caller repo/commit.
|
||||
- [x] Human-only file/status/sync gates kept; lookup holder rechecked.
|
||||
- [x] Async socket dispatch includes lookup verbs through LIBRARY_VERBS.
|
||||
- [x] Stale/unknown fetch syncs first; results use current immutable served pin.
|
||||
- [x] Six/eight creative names per mos-1308; other roles retain their tools.
|
||||
- [x] Launcher -> bundle -> typedTools passes presence without a library clone.
|
||||
- [x] No imports added to sandbox source closure; no mount change.
|
||||
- [ ] Execute new tests, RED characterization, full files/suites after CLEAR.
|
||||
- [ ] Rebase after rows 253/216 land, under a new branch name.
|
||||
- [ ] Claim/push only in Sage-authorized gap; Dewey review and Sage gate.
|
||||
|
||||
Candidate preservation commits are explicitly unrun under sage-2795; they are
|
||||
not completion, green-suite or landing claims. No dependencies installed.
|
||||
|
||||
Correction: the first committed-range diff --check found a blank line at EOF
|
||||
in the new library-tools test, missed while it was untracked. Removed in a
|
||||
follow-up local commit; the complete range must be checked again. No tests ran.
|
||||
|
||||
## Shared-log receipts for Sage (sage-2815)
|
||||
|
||||
The shared logs are unchanged from the base in this row. Sage appends these
|
||||
receipts in the landing sweep; they are preserved here, not in the manifest.
|
||||
|
||||
For `BUILD-LOG.md`:
|
||||
|
||||
```text
|
||||
2026-10-22T16:36:30Z | Della row224 C2-05b #1725 preparation | Before: C2-05a queries exist at 4f3fbc25a but no broker lookup verbs or typed tools. Scope: broker/tools plus Sage-authorized runtime/bundle integration, tests and docs. Existing human-only library reads retained; five agent queries require a live holder and trusted configured source. Code/docs only under row253 gate; no Node/test/push/claim. Runtime ownership coordinated with Rocko219/Huey272.
|
||||
```
|
||||
|
||||
For `docs/SESSIONS.md`:
|
||||
|
||||
```text
|
||||
2026-10-22T16:36:30Z | Della (sage-2795 / sage-2804, Q220) | Row 224 C2-05b #1725, della/row224 base 4f3fbc25a | Read both pinned briefs and C2-05a source; preparing broker/typed tools and authorized host runtime/bundle seams. No queue command, Node, HTTPS Git, claim, test or push during Gander gate. Dewey reviewer; Rocko/Huey runtime coordination sent. Tests and integration verification deferred to CLEAR.
|
||||
```
|
||||
|
||||
Local-only draft pins 19c0a7adf/7cf39c390 are superseded by the
|
||||
log-free candidate. Saved as local branch della/row224-draft-with-logs before
|
||||
rebuilding the unpublished candidate commit; no remote history changed.
|
||||
|
||||
## Dewey dewey-0852 and Sage sage-2819 — focused corrections
|
||||
|
||||
B1: inline freshness uses max(lastFetch, lastError.at), not lastFetch alone.
|
||||
An error within 60 seconds counts as a recent attempt, so later lookups serve
|
||||
the old served pin without another fetch. Keep inline sync: the brief requires
|
||||
it and businesses without timed libraries have no sync timer. The extended
|
||||
failed-fetch test makes reads at +1, +59,999 and +60,000 ms (one attempt total),
|
||||
then +60,001 ms (second attempt). All reads must return the old commit.
|
||||
|
||||
Required mutant M-B1: remove lastError.at from recentAttempt. The repeated
|
||||
reads must fail the attempted-count assertion. This mutant has NOT been run.
|
||||
|
||||
Residual (Sage will defer): the first inline fetch in each outage window can
|
||||
still exceed the tool timeout (60 seconds, or the runner broker client’s
|
||||
30 seconds). Bounding that fetch is out of scope for 224. Subsequent reads
|
||||
are paced by persisted lastError.at. No claim of first-read outage latency.
|
||||
|
||||
N1: whole bus, cli and harness suites are required, plus the shell gate.
|
||||
If the new thread-launch integration test skips due to sandboxProblem(),
|
||||
name the skip; a skip is not launcher integration proof.
|
||||
N2: cli/tests/thread-launch.test.mjs now depends on library/tests/helpers.mjs
|
||||
mappingEntry() and library/tests/fixtures/mapping.json. Changes to that fixture
|
||||
must rerun the CLI launch file.
|
||||
N3: harness/tests/library-tools.test.mjs compares the type enum to mapping.mjs
|
||||
TYPES, so adding a mapped type cannot silently leave the typed tool behind.
|
||||
|
||||
These edits are local code/docs only. No Node, RED/GREEN run, mutant, claim,
|
||||
HTTPS Git or push. New local pin goes to Dewey/Mos/Sage for focused review.
|
||||
@@ -0,0 +1,135 @@
|
||||
# Row 258, E01: mail PRD addendum draft and schemas
|
||||
|
||||
Ludwig, 2026-10-22. Issue #1763. Assigned by sage-2802 under Q220. Reviewer: Della.
|
||||
Base: aca16404c. Branch: ludwig/row258.
|
||||
|
||||
Sources: brief `docs/plans/2026-10-21_post-v1-row-briefs.md` (blob d7e5eaeea), "Brief: E01" and
|
||||
"Gate: email rows". Email plan row E01 at origin/della/email-plan 179391055 (approved ludwig-0037).
|
||||
Q2/Q222 terms (sage-2802): `mailboxScope` defaults to `[]`, changes are human-only, and the default-none
|
||||
and refusal tests apply.
|
||||
|
||||
## Deliverable
|
||||
|
||||
| Path | What |
|
||||
| --- | --- |
|
||||
| `docs/prd/mail-addendum-draft.md` | REQ-MAIL-1 to 16, a draft. Each requirement names its rows. Two questions for Mos. |
|
||||
| `packages/mail/schemas/*.schema.json` | source, data, access, service, content-key |
|
||||
| `packages/mail/src/validate.mjs` | Closed-subset validator, binding, location, file and key-lifecycle checks |
|
||||
| `packages/mail/tests/schema.test.mjs` | T1-T14, with T10b |
|
||||
| `packages/mail/{package.json,README.md}` | Package manifest and notes |
|
||||
|
||||
Not changed: `docs/prd/mosaic-stack.md`, its requirements JSON and tests, `packages/plugins/src/kinds.mjs`,
|
||||
`packages/business/src/vocabulary.mjs`, roles, `scripts/stack-copy.sh` COPY_PATHS. No new dependency.
|
||||
|
||||
## Choices and tradeoffs
|
||||
|
||||
1. **Validator.** A copy of the `packages/project/src/validate.mjs` pattern: a closed keyword set
|
||||
that throws on an unknown keyword. Added `integer`, `minimum`, `maximum`, `uniqueItems` and `default`.
|
||||
No JSON Schema library, so there is no dependency review. Tradeoff: two small validators to keep apart.
|
||||
A shared one is a later refactor, not this row.
|
||||
2. **Defaults live in the schema** (`default` keyword), read by `defaults(kind)`. One definition per
|
||||
default and range, as Q222 asks.
|
||||
3. **Ranges (implementer choice, for the reviewer to challenge).** Defaults come from the plan.
|
||||
|
||||
| Setting | Default | Min | Max |
|
||||
| --- | --- | --- | --- |
|
||||
| pollSeconds | 60 | 15 | 3600 |
|
||||
| sliceObservations | 50 | 1 | 500 |
|
||||
| sliceBytes | 10 MiB | 64 KiB | 64 MiB |
|
||||
| pendingDeliveries | 1,000 | 1 | 100,000 |
|
||||
| retainedBytes | 100 MiB | 1 MiB | 10 GiB |
|
||||
| parserInputBytes | 30 MiB | 1 MiB | 64 MiB |
|
||||
| mimeDepth | 10 | 1 | 32 |
|
||||
| mimeParts | 100 | 1 | 1,000 |
|
||||
| normalizedTextBytes | 256 KiB | 4 KiB | 4 MiB |
|
||||
| attachmentPartBytes | 10 MiB | 0 | 50 MiB |
|
||||
| attachmentMessageBytes | 10 MiB | 0 | 100 MiB |
|
||||
| staleScanIntervals | 3 | 2 | 20 |
|
||||
| pageSize | 20 | 1 | 100 |
|
||||
| bodyDraftDays | 30 | 1 | 3,650 |
|
||||
| attachmentDays | 7 | 1 | 3,650 |
|
||||
| metadataDays | 365 | 1 | 3,650 |
|
||||
|
||||
Cross-checks: part bytes ≤ message bytes; metadataDays ≥ both content durations (tombstones outlive
|
||||
content, D4).
|
||||
4. **Fixed false until later rows.** `persistContent` and `automaticPrune` are `const false`: the plan
|
||||
says metadata only until retention authority exists (E13), and settings never grant `data.delete`.
|
||||
`sourceMutations` has `maxItems: 0`: no mutation capability exists (Q3). E13 or a later reviewed row
|
||||
widens these.
|
||||
5. **Human-only.** Settings records require `changedBy.kind: "human"`. The schema can't tell a real human
|
||||
from an agent proxy. The E02 writer proves that, as the plan's E02 row says.
|
||||
6. **Incomplete scopes.** An enabled source needs an adapter. Unattended wake needs at least one exact
|
||||
sender: wake on with an empty allowlist would be on in name only. Grants need at least one right and
|
||||
one mailbox. Service scopes are `["mail.read"]` only.
|
||||
7. **Secret values.** Unknown keys are refused. All string values are also checked against secret
|
||||
shapes (private key header, JWT, common token prefixes, URL userinfo password, `password=` style).
|
||||
Messages name the path, never the value. This is a backstop, not a scanner.
|
||||
8. **Content-key record location.** The plan names the key path. The lifecycle record's own file is left
|
||||
to E03. `checkKeyLocation` and `checkProtectedFile` apply to both.
|
||||
9. **Key lifecycle.** active → retired → expired, or active → expired. `keyTransitionAllowed` is a pure
|
||||
check on one state edge. It doesn't persist state, stop a restored backup rolling a record back, or
|
||||
destroy key copies; E03 owns that proof (della-0023 N1). `keyUse` is none outside
|
||||
[createdAt, expiresAt), whatever the state says.
|
||||
10. **v1 compatibility.** T13 asserts that kinds.mjs names no mail, vocabulary.mjs has no `mail.access`,
|
||||
and the approved PRD has no REQ-MAIL. E05 and the approved addendum update T13 when they land.
|
||||
|
||||
11. **Use helpers fail closed (della-0023 F1).** `effectiveMailboxes` and `keyUse` validate their
|
||||
records first and read the clock only from a valid `Date`. An invalid record or clock, or a time
|
||||
outside [issuedAt, expiresAt) or [createdAt, expiresAt), gives no read and no key use. Tradeoff: a
|
||||
caller with a slightly malformed but otherwise usable grant gets nothing, not a partial read. E02
|
||||
still owns principal, right, manual-mode and revoke enforcement.
|
||||
12. **Placement compares resolved paths too (della-0023 F2).** `checkSlotLocation` and
|
||||
`checkKeyLocation` check the paths as written first, so planned paths that don't exist yet still
|
||||
work. Then they resolve the nearest existing ancestor of the secret path and of each forbidden root,
|
||||
and refuse containment. That catches a configDir linked into dataRoot and a root named through an
|
||||
alias. `checkProtectedFile` now needs the file lexically under configDir, refuses a symbolic link
|
||||
on configDir itself or any directory below it, and takes the forbidden roots for the same resolved
|
||||
comparison. Symbolic links above configDir (a linked `/home`, say) stay allowed: they don't move
|
||||
the secret relative to the roots, and the resolved comparison still covers them. Tradeoff: one
|
||||
`realpath` per root on each check.
|
||||
13. **Draft wording (mos-1313).** W1: rulings are cited as "DECISIONS Q…", plan questions as "email
|
||||
plan Q…", and "Q4" is replaced by what it stands for. W2: retention says expiry stops serving an
|
||||
item and removal waits for `data.delete`. W3: an invalid mail setting is refused when written and
|
||||
holds its source with an alert when found at load; no fallback to the default. W4: the questions
|
||||
section is gone; Mos's two answers are recorded in the status lines.
|
||||
|
||||
## Test plan (not run: Gander holds the slot, sage-2802)
|
||||
|
||||
`cd packages/mail && node --test tests/schema.test.mjs`. It writes only under `os.tmpdir()`, which the gate
|
||||
sets to a `mktemp -d /mnt/storage/scratch/tmp/gXXXXXX` directory. No network and no `~/.mosaic-dev`.
|
||||
|
||||
Static checks done: schemas written by an uncommitted scratch script (the JSON files are the source),
|
||||
JSON parses, every keyword is in the validator's closed set, every required key has a property, T13/T14 assertions checked against
|
||||
the repository text with python3 -B, secret and wildcard patterns checked against every fixture and
|
||||
sentinel. Node was not run.
|
||||
|
||||
Existing suites to run whole, because they scan every package's `src` or copy tracked package files:
|
||||
- `packages/bus/tests/credential-store.test.mjs` (credential use sites)
|
||||
- `packages/bus/tests/host-run.test.mjs` (T24 kind lists and use sites)
|
||||
- `packages/plugins/tests/harness.test.mjs` (harness names outside plugins)
|
||||
- `packages/cli/tests/stack-copy.test.mjs`, `packages/cli/tests/stack-release.test.mjs` (fixture copies)
|
||||
|
||||
Mutants, named by Della (della-0023). Each must die on a semantic assertion in the whole
|
||||
`schema.test.mjs`, run on a CLEAR slot. The wildcard tests stay.
|
||||
|
||||
| Mutant | Change | Must fail |
|
||||
| --- | --- | --- |
|
||||
| M1 | `mailboxScope` default nonempty | T1/T2 |
|
||||
| M2 | allow an agent settings author | T3 |
|
||||
| M3 | remove the source/business binding comparison | T8 |
|
||||
| M4 | disable secret-shape rejection | T6 |
|
||||
| M5 | omit the grant intersection | T2 |
|
||||
| M6 | omit the forbidden-root exclusion | T10 (config under dataRoot) |
|
||||
| M7 | allow mode 0640 | T11 |
|
||||
| M8 | allow retired → active | T12 |
|
||||
| M9 | bypass an enforced numeric lower or upper comparison in `visit` | T9 |
|
||||
| M10 | omit the malformed-state/time guard | T2/T12 lifetime and malformed cases |
|
||||
| M11 | omit the canonical or root-symlink guard | T10b/T11 |
|
||||
|
||||
## Round 2 (della-0023, mos-1313)
|
||||
|
||||
Round 1 was a8f0cf924, CHANGES (della-0023). Round 2 adds F1, F2 and N1 above, and Mos's W1-W4 on the
|
||||
draft. New cases: T2 lifetime edges and malformed grant or clock, T12 lifetime edges and malformed key
|
||||
or clock, T10b (configDir linked into dataRoot with a real 0600 key, a planned slot through the link,
|
||||
a dataRoot alias), T11 (file outside configDir, unnormalized path, linked configDir, root alias).
|
||||
Static checks only; Node not run (Gander holds the slot).
|
||||
@@ -1,184 +0,0 @@
|
||||
# Row 216 build notes (C2-05a, lookup index and queries, #1717)
|
||||
|
||||
Phase A only: code, tests and this file. No test, suite or mutant has been run. `node --check` passed on every changed `.mjs`.
|
||||
|
||||
- Branch: `researcher/row216`
|
||||
- Base: `d372102882d6858fa77af4aee453f866c886ad5c` (origin/refactor)
|
||||
- Design: `agents/gosalyn/work/case2/DESIGN.md` on origin/gosalyn/case2-design, section 3 (table at lines 625-631, bullets 633-641), 1.2-1.4, B12, D2
|
||||
- Brief: `docs/plans/2026-10-20_case2-rows.md`, C2-05a at lines 320-357
|
||||
|
||||
## Files
|
||||
|
||||
| Path | Status |
|
||||
|---|---|
|
||||
| `packages/library/src/index-build.mjs` | new |
|
||||
| `packages/library/src/query.mjs` | new |
|
||||
| `packages/library/src/index.mjs` | edit: 2 export lines added |
|
||||
| `packages/library/tests/query-fixture.mjs` | new, test helper |
|
||||
| `packages/library/tests/query.test.mjs` | new |
|
||||
| `packages/library/tests/index-build.test.mjs` | new |
|
||||
| `agents/researcher/work/queue-216/BUILD.md` | new (this file) |
|
||||
|
||||
No existing test or source file changed beyond `index.mjs`.
|
||||
|
||||
## Design rows and where each is tested
|
||||
|
||||
| Design row | Code | Tests (query.test.mjs unless noted) |
|
||||
|---|---|---|
|
||||
| `library_find` | `find` | find is bound to its universe; every result carries its scope; long list pages; large items page under the cap |
|
||||
| `library_get` | `get` | hybrid gets a source file; get lists visible links both ways; get pages its lines |
|
||||
| `library_search` | `search` | search returns scoped line hits; item that cannot fit alone is refused |
|
||||
| `library_manuscript` | `manuscript` | one chapter at most 400 lines; refused without chapter; hybrid reads a source chapter |
|
||||
| `library_universe` | `universeInfo` | universe info reports a hybrid's sources, constraints, notes; unknown universe refused |
|
||||
| Hybrid semantics (1.4) | `sees` | a hybrid sees its sources canon minus exclusions, no playground |
|
||||
| Unmapped never appears | index build + `sees` | unmapped files never appear (query.test); index holds mapped files only (index-build.test) |
|
||||
| Per-commit cache (3) | `indexFor` | same commit, path, mapping return one index; different path, mapping or commit get their own; eight-entry bound; failed read not kept; unchecked mapping refused (index-build.test) |
|
||||
| Rebuild time (3, C2-05a gate) | `indexFor` | 3,000-file rebuild, time reported with `t.diagnostic`, no threshold asserted |
|
||||
| REPLY_CAP parity | `QUERY_REPLY_CAP` | equals `REPLY_CAP` from harness tools.mjs |
|
||||
|
||||
## Choices where the design table is silent or differs
|
||||
|
||||
These are departures or gaps. Each is built as stated, so Sage can rule before Phase B. A ruling that changes one of them is a small edit.
|
||||
|
||||
1. **Paging arguments (gap a).** The table has no offset. `find` and `search` take an optional `offset` (a position in the filtered list, default 0). A reply carries `next`: the position of the next item, or `null`.
|
||||
2. **`library_get` has no universe (gap b).** Built with a required `universe`. Without it the reply could not be isolated. A missing file and a file the universe cannot see both return `null`, so a caller cannot tell them apart. `get` also takes an optional `from` line.
|
||||
3. **`get` over the cap (gap c).** Lines page by `from` with `next`. `links_out` and `links_in` come back whole on every page. If the frontmatter and links alone exceed the cap, the call is refused with `query-item-too-large`. A hub note with many backlinks may be refused. This needs a ruling: link paging, or accept the refusal.
|
||||
4. **`manuscript` without a chapter (gap d).** Refused with `query-chapter-required`. The design names no list form. A chapter name that matches two visible files is refused with `query-chapter-ambiguous`.
|
||||
5. **`search` excerpt (gap e).** The excerpt is the whole line, not a truncation. A line that alone exceeds the cap is refused. Truncating it would cut an item.
|
||||
6. **`exclude` form (gap f).** `checkMapping` accepts only a folder path for `exclude` (`folder()`), so an exclusion is a path prefix. There is no type or source form to build. Confirm.
|
||||
7. **Pin constraints.** `pin` does not hide any file. It is reported by `universeInfo` only. Confirm.
|
||||
8. **Playground through `get`.** `scope` applies to lists. `get` by an explicit path returns a playground file of the caller's own universe. Confirm.
|
||||
9. **Links.** A link into a file the universe cannot see is dropped from `links_out` and `links_in`, in line with N1. Links from a visible file that point into another universe are kept in the index (the reader keeps them under R6), and the query layer drops them for universes that cannot see them.
|
||||
10. **`find` matching.** Case-insensitive substring over the front matter `name` and the file name. An empty `query` matches every file in scope.
|
||||
11. **Cache key.** `repo`, `commit` and a sha256 of the canonical mapping. Keying by commit alone would return the wrong scopes after a mapping change. The cache holds eight entries, least recently used out. Failed reads are not kept.
|
||||
12. **`universeInfo`.** Reads only the checked mapping and the served commit. It builds no index.
|
||||
13. **Reply commit.** Every reply carries the index commit, which is the served commit passed in.
|
||||
|
||||
## Provisional refusal codes (pending Sage approval)
|
||||
|
||||
The brief says new refusal codes need Sage's approval first. These are provisional and must not be treated as settled:
|
||||
|
||||
- `query-argument-invalid`
|
||||
- `query-universe-unknown`
|
||||
- `query-chapter-required`
|
||||
- `query-chapter-ambiguous`
|
||||
- `query-item-too-large`
|
||||
|
||||
Reused, existing codes: `library-mapping-invalid`, plus the reader's `library-git`, `library-commit` through `indexFor`.
|
||||
|
||||
## Grep results (grep only, not a test run)
|
||||
|
||||
- Test references to the changed paths: `packages/library/tests/*.mjs` import `../src/index.mjs` and `../src/mapping.mjs`. `packages/bus/tests/library.test.mjs` and `library-sync.test.mjs` import the library index as a namespace. Neither enumerates its exports, so the two new export lines break no assertion.
|
||||
- `packages/cli/tests/stack-copy.test.mjs:52` checks that `packages/library/src/index.mjs` exists, and it still does.
|
||||
- 64-hex pins: none in `packages/library/tests`, none on `index.mjs`, `query.mjs` or `index-build.mjs`.
|
||||
- `scripts/test-*.sh`: no reference to the new modules.
|
||||
|
||||
## Flag for Sage
|
||||
|
||||
- `packages/library/tests/helpers.mjs` `repo()` builds its fixtures under `tmpdir()`, which is `/tmp`. The launcher rules forbid that. The new tests do not use it; they use `tests/query-fixture.mjs`, which builds under `/mnt/storage/scratch/tmp`. The existing helper should be fixed in a separate row, not here.
|
||||
|
||||
## Phase B commands (not run)
|
||||
|
||||
Run only after Sage sends "slot clear". Set the scratch TMPDIR first, per the launcher rule.
|
||||
|
||||
```
|
||||
cd /mnt/storage/src/mosaic-stack/.claude/worktrees/agent-aa3cc98787bc5152c/packages/library
|
||||
export TMPDIR=/mnt/storage/scratch/tmp
|
||||
node --test tests/query.test.mjs tests/index-build.test.mjs
|
||||
node --test tests/*.test.mjs
|
||||
```
|
||||
|
||||
Also run `packages/bus/tests/library.test.mjs` and `library-sync.test.mjs`, since they import the index.
|
||||
|
||||
## Status
|
||||
|
||||
Not yet verified. Nothing has been tested and nothing is committed. The working tree holds the Phase A changes. AGENTS.md invariant 8 allows a commit only after the applicable suites are green, so the commit waits for Phase B. The dispatch asked for a local commit in Phase A; Sage to confirm that the commit waits for Phase B.
|
||||
|
||||
## Round 2 (2026-10-22, sage-2708, sage-2716): rulings applied, rebased onto aca16404c
|
||||
|
||||
Added after the text above; nothing above is edited. "Nothing is committed", "Provisional refusal codes",
|
||||
`query-chapter-required` and the tmpdir flag above were true when written and are superseded here.
|
||||
|
||||
- Branch `researcher/row216-r2`, one local commit on origin/refactor `aca16404c`. Row 268 (C2-01c) is in
|
||||
that base. Row 253 has not landed; the work rebases onto 253's tip before any gate. Not pushed, not run.
|
||||
- Rulings: `agents/sage/work/queue-216/RULINGS.md` at da9555ca2, rulings 1-13 and the tmpdir flag.
|
||||
- Drift check, d37210288..aca16404c against my paths: the library files that changed are `mapping.mjs`,
|
||||
`scope.mjs`, `README.md` and the mapping and scope tests. None is a file I add or edit. `query.mjs` reads
|
||||
the checked mapping through them, so the whole library package runs in phase B.
|
||||
|
||||
### What changed from phase A
|
||||
|
||||
| Ruling | Change in `query.mjs` | Test |
|
||||
|---|---|---|
|
||||
| 3 `get` over the cap | `links_out` and `links_in` page together with `links_from`; every reply carries `links_next`, `next` and the front matter. A page with lines left carries at least one line. Links take at most half the cap while lines remain. `query-item-too-large` is left for front matter alone or one line alone (or one link alone) | hub note with 900 backlinks pages and adds up; front matter alone and a line alone refused; bad `links_from` |
|
||||
| 4 `manuscript` no chapter | `chapter` optional: the first visible chapter in path order. Every reply carries `next_chapter`. `query-chapter-required` removed. Ambiguity still refuses | first chapter and `next_chapter`; ambiguous name refused |
|
||||
| 5 `search` long line | hit comes back `{path, line, excerpt: null, excerpt_too_long: true, scope}`, other hits still return | marked hit and unmarked hits in one reply |
|
||||
| 6 `exclude` | path-prefix reading only. The test names `EXCLUSION` and `EXCLUDED_PATHS` in `query-fixture.mjs`, so C2-15 (row 267) swaps two constants | exclusion hides from the hybrid and not from the source |
|
||||
| 8 hybrid `get` | unchanged code; test added | hybrid gets a source's playground file as `null`; the source gets it with `scope: "playground"` |
|
||||
| PRD 1.14 (row 268) | `universeInfo` reports `kind` from the mapping (`draft` included) | four tests below |
|
||||
|
||||
PRD 1.14 tests, one each: a lookup naming a `book` leaves `book: null` files out (find and search);
|
||||
`library_manuscript` never reads a `book: null` file; a draft universe has no canon, so its lookups return
|
||||
files only with `scope: "playground"`; `library_universe` on a draft returns kind, name and prefixes.
|
||||
The levels fixture is `WITH_LEVELS` in `query.test.mjs`: harbor gets `Notes/Harbor/` as a universe-level
|
||||
playground prefix, and `idea` is a draft over `Drafts/New World/`.
|
||||
|
||||
`query-fixture.mjs` builds under `tmpdir()`, as `helpers.mjs` does (ruling "tmpdir flag"). The earlier
|
||||
flag about `helpers.mjs` building under `/tmp` is closed by that ruling: the gate sets `TMPDIR`.
|
||||
|
||||
Refusal codes now: `query-argument-invalid`, `query-universe-unknown`, `query-chapter-ambiguous`,
|
||||
`query-item-too-large`. All four approved in the rulings.
|
||||
|
||||
### Grep results (grep only, not a test run)
|
||||
|
||||
- `git grep` over `packages/*/tests/*` and `scripts/*` for `query.mjs` and `index-build`: no hit outside
|
||||
`packages/library/tests/query*.test.mjs` and `index-build.test.mjs`.
|
||||
- 64-hex pins naming a library path: none. Positive control: the same pathspec finds the sha256 lines in
|
||||
`packages/cli/tests/case1-hostcheck.test.mjs` and the `packages/library/src/index.mjs` existence check at
|
||||
`packages/cli/tests/stack-copy.test.mjs:52`.
|
||||
- Tests that read `packages/library/src` as a directory: `packages/cli/tests/stack-release-fixture.mjs:309`
|
||||
copies `packages/library/src` whole into a release fixture. It carries no file list, so two new files break
|
||||
no assertion. `stack-release.test.mjs` is not in the gate unless Sage wants it.
|
||||
- Tests that import the library index as a namespace: `packages/bus/tests/library.test.mjs` and
|
||||
`library-sync.test.mjs`. Neither enumerates exports, so the two export lines add no break. Both run
|
||||
whole in phase B.
|
||||
|
||||
### Run limits (fixed before any run)
|
||||
|
||||
Every node file runs as its own `setsid -w timeout -k 10 LIMIT node --test FILE`; nothing runs unbounded.
|
||||
|
||||
- Node test files: LIMIT 900 s. The library test files have no recorded time. 900 s is above the longest
|
||||
recorded whole-package run I know of (cli, 591 s, row 281 BUILD.md "Runs" step 3), so it holds the 3x rule
|
||||
for any single file. The first run records each file's time, and a file past 300 s gets a limit re-derived
|
||||
from it before the next run. `query.test.mjs` builds a 900-file repo and a 3,000-file index rebuild is in
|
||||
`index-build.test.mjs`; both are the files most likely to need it.
|
||||
- No shell script is in this row's gate, so no script limit applies.
|
||||
|
||||
Any timeout goes to Sage, then Mos, unplaced (mos-1275).
|
||||
|
||||
### Phase B (not run; Sage's CLEAR first, then a rebase onto 253's tip)
|
||||
|
||||
```
|
||||
export TMPDIR=/mnt/storage/scratch/tmp
|
||||
cd <worktree>/packages/library
|
||||
for f in tests/*.test.mjs; do setsid -w timeout -k 10 900 node --test "$f"; done
|
||||
cd ../bus
|
||||
for f in tests/library.test.mjs tests/library-sync.test.mjs; do setsid -w timeout -k 10 900 node --test "$f"; done
|
||||
cd ../cli
|
||||
setsid -w timeout -k 10 900 node --test tests/stack-copy.test.mjs
|
||||
```
|
||||
|
||||
Mutants I propose (my list, not read off the brief; each is meant to be killed by the named test, and none
|
||||
has been thrown; the kill claims are unverified until CLEAR):
|
||||
|
||||
- M-a: `sees()` returns true for a file of another universe: killed by "find is bound to its universe".
|
||||
- M-b: the playground filter dropped: killed by "playground appears only when asked for".
|
||||
- M-c: the hybrid reads a source's playground: killed by "a hybrid gets a source playground file as null".
|
||||
- M-d: `links_next` always null: killed by the hub paging test.
|
||||
- M-e: the book filter lets `book: null` through: killed by the PRD 1.14 book test.
|
||||
|
||||
### Status
|
||||
|
||||
Built, not run. `node --check` has not been run on this round's files (no node before the CLEAR); the
|
||||
phase A pass above covered the earlier text only. The commit is local and waits for no gate under Sage's
|
||||
sage-2716 instruction, which allows git and docs work only; push and claim wait for Sage's go.
|
||||
@@ -0,0 +1,164 @@
|
||||
# PRD addendum draft: mail intake and access
|
||||
|
||||
- Status: draft, not approved. Ludwig drafted it for email plan row E01
|
||||
(Vikunja 460, issue #1763). Its wording goes to Mos first, then Jason.
|
||||
It does not change `docs/prd/mosaic-stack.md`. Once Jason approves it,
|
||||
it lands there as a new PRD version, and only then does any requirement
|
||||
here bind.
|
||||
- Source: `docs/plans/2026-10-21_email-plan.md`, Della's revision
|
||||
179391055, approved by Ludwig (ludwig-0037). DECISIONS Q217 settles
|
||||
the mailbox (one agent mailbox on woltje.com at InMotion, which Jason
|
||||
creates). DECISIONS Q222 settles personal mail (none for now) and the
|
||||
settings rule: things are tunable, not hardcoded.
|
||||
- Shape (mos-1313): once approved, this lands as one new PRD version with
|
||||
a Mail section, in its own reviewed row. `mail.access` stays a hard
|
||||
wait in that version.
|
||||
- Scope: post-v1. None of it is needed for 2026-10-27.
|
||||
- Q numbers: "DECISIONS Q…" is a ruling. "Email plan Q…" is a question
|
||||
still open in the email plan.
|
||||
|
||||
## How to read this draft
|
||||
|
||||
Every limit, scope and time below is a setting with a default, unless
|
||||
Jason fixed it himself. The default is the starting value, not a rule.
|
||||
The schemas in `packages/mail/schemas/` hold each default and the range a
|
||||
setting may take. Implementers and reviewers set that range. Widening it
|
||||
needs a reviewed schema and test change. Only an authenticated human
|
||||
changes a setting, through the audited CLI and later the Console. An
|
||||
agent can't change one, not even to narrow it. For mail, an invalid
|
||||
setting doesn't fall back to its default. A write with an invalid value
|
||||
is refused. An invalid value found when the service loads holds that
|
||||
source and raises an alert.
|
||||
|
||||
Each requirement lists the build rows that deliver it. The rows are in
|
||||
the email plan's "Ordered build rows".
|
||||
|
||||
## Requirements
|
||||
|
||||
### Mail
|
||||
|
||||
- **REQ-MAIL-1.** The stack may run one mail service per instance. It is
|
||||
off by default and opens no connection while off. A disabled source
|
||||
makes no probe and no read. Each configured source has one leased
|
||||
owner. Accounts stay separate by instance, business and source. One
|
||||
service never means one shared credential or mailbox.
|
||||
Rows: E01, E03, E14.
|
||||
- **REQ-MAIL-2.** Each source has a mailbox scope, `mailboxScope`. It
|
||||
defaults to none, which reads nothing, agent ingress included. A human
|
||||
sets exact mailbox IDs. No wildcard is accepted. The scope is
|
||||
rechecked before each scan and fetch. A mailbox removed from scope is
|
||||
fenced before in-flight content returns. A read is the intersection of
|
||||
the source scope, the provider's rights and the principal's current
|
||||
grant. Widening a scope grants no agent anything. No personal mailbox
|
||||
is enrolled until a later consent and grant (DECISIONS Q222).
|
||||
Rows: E01, E02, E13.
|
||||
- **REQ-MAIL-3.** A new plug-in kind, `mail-source`, carries providers
|
||||
(REQ-PLUG-1). Its modules export `descriptor`, `scan`, `fetch` and
|
||||
`health`. A module only reads: it never sends, deletes, labels, moves
|
||||
or marks mail read. Microsoft 365 (Graph) and IMAP are the first
|
||||
modules. POP3, Gmail API and JMAP wait for a named need. The kind
|
||||
enters `packages/plugins/src/kinds.mjs` with its first production
|
||||
module. That waits for the approved PRD version, and for the source,
|
||||
grant and route approvals in REQ-MAIL-4, 7 and 8.
|
||||
Rows: E05, E06.
|
||||
- **REQ-MAIL-4.** A source connects through its own service principal,
|
||||
`mail-service`. This is a new approved extension, not an exception to
|
||||
REQ-CRED-1. Its record names the business, source, provider, mailbox,
|
||||
scopes (read only), slot path, rotation date, approval and revision.
|
||||
The slot sits under
|
||||
`<configDir>/secrets/mosaic-stack/mail/<business>/<source>/`, never in
|
||||
the repository or dataRoot. Agents never see the credential. A missing,
|
||||
expired or revoked slot fails closed. Authentication may retry once,
|
||||
then stops and alerts.
|
||||
Rows: E01, E02.
|
||||
- **REQ-MAIL-5.** Intake keeps a durable record of what it saw. The
|
||||
provider cursor and the new observations commit together. A message's
|
||||
identity is its provider ID within business, source and account, never
|
||||
Message-ID alone. Delivery uses a stable key and a receipt at the
|
||||
recipient, so a crash causes no second task. First activation starts
|
||||
from now and acts on no old mail. Importing history is a separate
|
||||
approved window. Per-source quotas default to 1,000 pending deliveries
|
||||
and 100 MiB of content. A full quota, a full disk or missing audit
|
||||
storage holds intake and raises an alert.
|
||||
Rows: E03.
|
||||
- **REQ-MAIL-6.** Parsing is bounded. Defaults: 30 MiB input, depth 10,
|
||||
100 parts, 256 KiB of normalized text. Bad or truncated mail is
|
||||
quarantined, never dropped. Active content never runs. Attachments
|
||||
start as metadata. An authorized fetch quarantines the bytes, by
|
||||
default up to 10 MiB per part and per message. Extracting attachment
|
||||
content stays Jason's choice (email plan Q8). If he chooses it, extraction runs
|
||||
isolated with no network and a scanner.
|
||||
Rows: E04, E16.
|
||||
- **REQ-MAIL-7.** A route table maps trusted delivery evidence to a Stack
|
||||
user or role instance. It lives in the organization file and changes
|
||||
only through the human-approved write path. A route grants no read
|
||||
right. No agent reroutes mail or widens its own route. Bcc, ambiguous
|
||||
alias or forwarded mail goes to restricted review. Unknown addresses
|
||||
create no agent.
|
||||
Rows: E07, E08.
|
||||
- **REQ-MAIL-8.** Agent access to mail needs a `mail.access` grant. It is
|
||||
separate from `credential.mint`. The chief raises it, and the human
|
||||
approves the exact principal, source, mailboxes, rights and expiry.
|
||||
Metadata, body, attachment and draft are separate rights. Each read
|
||||
rechecks the current grant. A revoke fences new reads and in-flight
|
||||
reads before content returns. Manual mode asks the human for each use.
|
||||
`mail.access` is a new vocabulary action, so it is a hard wait
|
||||
(REQ-WAIT-3) until a PRD version Jason approves says otherwise
|
||||
(REQ-WAIT-2).
|
||||
Rows: E02, E09.
|
||||
- **REQ-MAIL-9.** Every harness gets the same mail tools through the
|
||||
broker: list, get, thread, attachment, acknowledge, create a local
|
||||
draft and health. A list page defaults to 20 items. Acknowledge
|
||||
changes Stack state only. A draft is a local unsent record. There is
|
||||
no send tool and no reroute tool.
|
||||
Rows: E09.
|
||||
- **REQ-MAIL-10.** Mail content is untrusted. Tool results carry
|
||||
`untrusted: true`. A sender can't approve a decision, change policy,
|
||||
mint a grant or pose as the human. Mail never wakes an agent by
|
||||
default (`unattendedWake` is off, and its sender allowlist is empty).
|
||||
A human may turn it on for a route. Then a wake also needs aligned
|
||||
DMARC, a current grant and admission by the system manager
|
||||
(DECISIONS Q198).
|
||||
The woken work runs in a fresh restricted profile, and anything it
|
||||
starts keeps `source: mail`.
|
||||
Rows: E10.
|
||||
- **REQ-MAIL-11.** A mail-triggered decision goes through the stack's one
|
||||
decision module (DECISIONS Q207). Its input is metadata only, inside a closed
|
||||
schema. The mail service never calls a model itself. Without a model,
|
||||
deterministic intake keeps working and model work is held.
|
||||
Rows: E11.
|
||||
- **REQ-MAIL-12.** Mail health reports real progress: last scan, last
|
||||
cursor commit, lag, backlog and credential state. A live process alone
|
||||
is not healthy. Outages pause only the affected source. Recovery
|
||||
follows the system manager's protected mode and its recovery latch
|
||||
(DECISIONS Q198). A health check sends no mail and calls no model.
|
||||
Rows: E12.
|
||||
- **REQ-MAIL-13.** Mail data lives under `<dataRoot>/mail/<business>/`
|
||||
and never enters git or an image. By default the stack keeps metadata
|
||||
only. Content keys sit under
|
||||
`<configDir>/secrets/mosaic-stack/mail-content/<business>/`, outside
|
||||
dataRoot and its backups, mode 600, with no symbolic link. Retention
|
||||
defaults: bodies and drafts 30 days, attachments 7 days, metadata 365
|
||||
days. When an item's time runs out, the stack stops serving it. A
|
||||
retention setting never authorizes deletion: removing the item waits
|
||||
for `data.delete` or a retention authority Jason approves later. A
|
||||
restored backup must not bring back expired content or redeliver old
|
||||
work.
|
||||
Rows: E01, E03, E13.
|
||||
- **REQ-MAIL-14.** The mail service never sends. No SMTP, Graph send,
|
||||
forward, auto-reply or emailed digest. The email specialist reads and
|
||||
drafts, never sends (R21). A future sender needs its own plan and
|
||||
`message.external` approval.
|
||||
Rows: E05, E06, E09, E14.
|
||||
- **REQ-MAIL-15.** Every mail limit, scope and retention time is a
|
||||
setting. The schema holds its default and range. Only a human changes
|
||||
it. A change is versioned and audited with who made it. An invalid
|
||||
value is refused when written, and holds its source with an alert when
|
||||
found at load. Settings never
|
||||
override grants, hard waits or the system manager's contract.
|
||||
Rows: E01, E04, E13.
|
||||
- **REQ-MAIL-16.** A live pilot is separate. It needs Jason's mailbox,
|
||||
the approved PRD version, the source, grant and route approvals in
|
||||
REQ-MAIL-4, 7 and 8, and a reviewed re-pin. It sends no test mail, and it
|
||||
is judged on evidence for each message, not counts.
|
||||
Rows: E15.
|
||||
@@ -96,16 +96,7 @@ export const TASK_PAGE_MAX = 200;
|
||||
// request allowed past the ordinary 64 KiB envelope (server.mjs and client.mjs), up to its content cap.
|
||||
export const PROJECT_FILE_VERBS = Object.freeze(['project.files', 'project.file', 'project.file.save']);
|
||||
// C2-03a: library.sync is the human's fetch, and with `accept` the one way past library-diverged.
|
||||
// Row 224: pure agent lookups are separate from the three human-only verbs. Listing them
|
||||
// here also makes server.mjs await their host effects instead of serializing a Promise as {}.
|
||||
const LIBRARY_LOOKUPS = Object.freeze({
|
||||
'library.find': { method: 'find', fields: ['universe', 'query', 'type', 'book', 'scope', 'offset'], required: ['universe', 'query'] },
|
||||
'library.get': { method: 'get', fields: ['universe', 'path', 'from', 'links_from'], required: ['universe', 'path'] },
|
||||
'library.search': { method: 'search', fields: ['universe', 'text', 'book', 'scope', 'offset'], required: ['universe', 'text'] },
|
||||
'library.manuscript': { method: 'manuscript', fields: ['universe', 'book', 'scope', 'chapter', 'from', 'to'], required: ['universe', 'book'] },
|
||||
'library.universe': { method: 'universeInfo', fields: ['universe'], required: ['universe'] },
|
||||
});
|
||||
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status', 'library.sync', ...Object.keys(LIBRARY_LOOKUPS)]);
|
||||
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status', 'library.sync']);
|
||||
export const PROJECT_SAVE_VERB = 'project.file.save';
|
||||
export const PROJECT_SAVE_REQUEST_MAX = 1024 * 1024 + 64 * 1024;
|
||||
export const WEB_VERBS =Object.freeze(['web.search', 'web.fetch']);
|
||||
@@ -4610,42 +4601,10 @@ export class Broker {
|
||||
throw this.#refused(s, ProjectFailure && e instanceof ProjectFailure ? new BusError(e.code) : e, request);
|
||||
}
|
||||
}
|
||||
// Row 224 C2-05b: the agent never supplies a host repo, commit or business. Its current
|
||||
// holder and business entry are rechecked before/after awaits; runtime owns Git and queries.
|
||||
async #requestLibraryLookup(s, { verb, args }) {
|
||||
try {
|
||||
this.#agent(s);
|
||||
const business = this.#business(s.business);
|
||||
const projects = Object.entries(business.projects ?? {}).filter(([, entry]) => entry.kind === 'library');
|
||||
if (!projects.length || typeof this.#library?.lookup !== 'function') fail('library-not-configured');
|
||||
const spec = LIBRARY_LOOKUPS[verb];
|
||||
keys(args, ['project', ...spec.fields], spec.required);
|
||||
if (!text(args.universe, 160)) fail('invalid-request');
|
||||
if (args.project === undefined && projects.length !== 1) fail('invalid-request');
|
||||
const project = args.project ?? projects[0][0];
|
||||
if (!text(project, 160) || !projects.some(([name]) => name === project)) fail('library-not-configured');
|
||||
const revision = JSON.stringify(business.projects[project]);
|
||||
const authorize = () => {
|
||||
this.#agent(s);
|
||||
this.#processingGate(s, verb);
|
||||
if (JSON.stringify(this.#business(s.business).projects?.[project]) !== revision) fail('library-not-configured');
|
||||
};
|
||||
const { project: ignored, ...query } = args;
|
||||
authorize();
|
||||
const result = await this.#library.lookup({ business, project }, { method: spec.method, args: query, authorize });
|
||||
authorize();
|
||||
this.#secretCheck(result);
|
||||
return result;
|
||||
} catch (error) {
|
||||
const Failure = this.#library?.LibraryError;
|
||||
throw this.#refused(s, Failure && error instanceof Failure ? new BusError(error.code) : error, { verb, args });
|
||||
}
|
||||
}
|
||||
// C2-04 M4: only a verified human reads library text or status. Host Git
|
||||
// functions stay injected, preserving this module's sandbox import closure.
|
||||
async #requestLibrary(s, request) {
|
||||
const { verb, args } = request;
|
||||
if (Object.hasOwn(LIBRARY_LOOKUPS, verb)) return this.#requestLibraryLookup(s, request);
|
||||
try {
|
||||
this.#human(s);
|
||||
const fields = verb === 'library.file' ? ['project', 'path'] : verb === 'library.sync' ? ['project', 'accept'] : ['project'];
|
||||
|
||||
@@ -84,38 +84,6 @@ export async function harnessLevels({ log = (e) => process.stderr.write(`plugins
|
||||
}
|
||||
return (name) => levels.get(name) ?? null;
|
||||
}
|
||||
// Row 224 C2-05b: a host-only adapter. A failed fetch keeps serving immutable old
|
||||
// objects, as C2-03a requires; no caller may select a repo, commit or sync --accept.
|
||||
export function createLibraryLookup({ dataRoot, sync, now = () => Date.now() }) {
|
||||
const methods = { find: libraryModule.find, get: libraryModule.get, search: libraryModule.search,
|
||||
manuscript: libraryModule.manuscript, universeInfo: libraryModule.universeInfo };
|
||||
return async (context, { method, args, authorize }) => {
|
||||
if (!Object.hasOwn(methods, method)) throw new libraryModule.LibraryError('query-argument-invalid', 'unknown lookup');
|
||||
authorize();
|
||||
const source = { ...context, dataRoot };
|
||||
let state = await libraryModule.libraryStatus(source);
|
||||
authorize();
|
||||
// sage-2819 / dewey-0852: failed attempts also pace inline retries. Otherwise an
|
||||
// outage makes every lookup refetch while lastFetch remains old or null.
|
||||
const recentAttempt = Math.max(state.lastFetch === null ? -Infinity : Date.parse(state.lastFetch),
|
||||
state.lastError === null ? -Infinity : Date.parse(state.lastError.at));
|
||||
const age = now() - recentAttempt;
|
||||
if (age > libraryModule.SYNC_INTERVAL_MS || age < 0) {
|
||||
try { await sync(source); }
|
||||
catch (error) {
|
||||
// A timer may already be fetching this mirror. Its current served state is still safe.
|
||||
if (!(error instanceof libraryModule.LibraryError) || error.code !== 'library-sync-busy') throw error;
|
||||
}
|
||||
authorize();
|
||||
state = await libraryModule.libraryStatus(source);
|
||||
}
|
||||
if (!state.served) throw new libraryModule.LibraryError('library-commit', 'library has never been synced');
|
||||
authorize();
|
||||
return methods[method]({ repo: libraryModule.mirrorPath(source), commit: state.served,
|
||||
mapping: libraryModule.checkMapping(context.business.projects[context.project]) }, args);
|
||||
};
|
||||
}
|
||||
|
||||
export async function startBroker({
|
||||
dataRoot,
|
||||
businesses,
|
||||
@@ -305,8 +273,6 @@ export async function startBroker({
|
||||
store,
|
||||
businesses,
|
||||
library: {
|
||||
lookup: createLibraryLookup({ dataRoot, sync: context =>
|
||||
processing.work(() => libraryModule.sync(context, syncOptions(context.business))) }),
|
||||
readFile: context => libraryModule.readLibraryFile({ ...context, dataRoot }),
|
||||
status: context => libraryModule.libraryStatus({ ...context, dataRoot }),
|
||||
sync: (context, { accept }) => processing.work(() => libraryModule.sync({ ...context, dataRoot }, { ...syncOptions(context.business), accept })),
|
||||
|
||||
@@ -1,235 +0,0 @@
|
||||
// Row 224: exercise broker authorization, real C2-05a queries and host served snapshots.
|
||||
// All bodies and remotes are made up. External sync is replaced only at its trusted host seam.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdirSync, writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { Store } from '../src/store.mjs';
|
||||
import { Broker } from '../src/broker.mjs';
|
||||
import { createLibraryLookup, startBroker } from '../src/runtime.mjs';
|
||||
import { serve } from '../src/server.mjs';
|
||||
import { Client } from '../src/client.mjs';
|
||||
import * as library from '../../library/src/index.mjs';
|
||||
import { scratchRepo, note } from '../../library/tests/query-fixture.mjs';
|
||||
import { mappingEntry, git } from '../../library/tests/helpers.mjs';
|
||||
|
||||
const NOW = Date.parse('2026-10-22T16:00:00.000Z');
|
||||
const VERBS = ['library.find', 'library.get', 'library.search', 'library.manuscript', 'library.universe'];
|
||||
const GET = { universe: 'harbor', path: 'Book One/Characters/Tomas.md' };
|
||||
const ARGS = [
|
||||
{ universe: 'harbor', query: 'Tomas' }, GET,
|
||||
{ universe: 'harbor', text: 'ferry' }, { universe: 'harbor', book: 'Book One' }, { universe: 'harbor' },
|
||||
];
|
||||
const TEXTS = {
|
||||
'Book One/Characters/Tomas.md': note('character', 'Tomas', 'He runs the ferry. [[Elsewhere]]'),
|
||||
'Book One/Chapters/01.md': '# A ferry\nOne line.\n',
|
||||
'Book One/Playground/What If.md': note('character', 'Maybe Tomas', 'Unwritten ferry.'),
|
||||
'Second World/Characters/Elsewhere.md': note('character', 'Elsewhere', 'Other ferry.'),
|
||||
};
|
||||
|
||||
async function world(t, { projects, adapter, claimed = true, now = () => NOW } = {}) {
|
||||
const f = scratchRepo(t, TEXTS), dataRoot = join(f.root, 'data');
|
||||
mkdirSync(dataRoot);
|
||||
const business = { id: 'acme', human: 'jason', vars: { 'bus.tester': true },
|
||||
projects: projects ?? { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' },
|
||||
roles: { coder: { definition: 'coder', authority: { withinRole: [], crossRole: [] } } } };
|
||||
const context = { business, project: 'books', dataRoot };
|
||||
let mirror = null, state = null, fetches = 0;
|
||||
if (business.projects.books?.kind === 'library') {
|
||||
mirror = await library.ensureMirror(context);
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
|
||||
state = { version: 1, branch: 'main', served: f.commit, lastFetch: new Date(NOW).toISOString(),
|
||||
lastError: null, diverged: null, problems: [] };
|
||||
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify(state), { mode: 0o600 });
|
||||
}
|
||||
const setState = patch => {
|
||||
state = { ...state, ...patch };
|
||||
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify(state), { mode: 0o600 });
|
||||
};
|
||||
const lookup = createLibraryLookup({ dataRoot, now, sync: async () => {
|
||||
fetches++;
|
||||
setState({ lastFetch: new Date(now()).toISOString() });
|
||||
return { state, code: null };
|
||||
} });
|
||||
const store = new Store(dataRoot);
|
||||
t.after(() => store.close());
|
||||
const broker = new Broker({ store, businesses: { acme: business },
|
||||
library: adapter ?? { lookup, LibraryError: library.LibraryError } });
|
||||
const agent = broker.bindLaunch({ business: 'acme', role: 'coder', run: 'r224', harness: 'pi' });
|
||||
if (claimed) broker.request(agent, { verb: 'role.claim' });
|
||||
const human = broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
|
||||
const reader = broker.bindReader({ business: 'acme' }), tester = broker.bindTester({ business: 'acme' });
|
||||
const call = (verb, args = {}, cap = agent) => Promise.resolve().then(() => broker.request(cap, { verb, args }));
|
||||
return { ...f, dataRoot, business, context, mirror, store, broker, agent, human, reader, tester,
|
||||
call, lookup, setState, fetches: () => fetches };
|
||||
}
|
||||
|
||||
test('all five agent verbs read the configured served commit through real queries', async t => {
|
||||
const f = await world(t);
|
||||
const [find, get, search, manuscript, universe] = await Promise.all(VERBS.map((verb, i) => f.call(verb, ARGS[i])));
|
||||
for (const result of [find, get, search, manuscript, universe]) assert.equal(result.commit, f.commit);
|
||||
assert.deepEqual(find.results.map(x => x.path), ['Book One/Characters/Tomas.md']);
|
||||
assert.match(JSON.stringify(get.lines), /He runs the ferry/);
|
||||
assert.deepEqual(get.links_out, []); // The only target is in another universe.
|
||||
assert.ok(search.results.length > 0);
|
||||
assert.ok(search.results.every(x => !x.path.startsWith('Second World/') && x.scope === 'canon'));
|
||||
assert.equal(manuscript.path, 'Book One/Chapters/01.md');
|
||||
assert.deepEqual(universe.books, ['Book One']);
|
||||
assert.equal(f.fetches(), 0);
|
||||
const playground = await f.call('library.find', { universe: 'harbor', query: '', scope: 'playground' });
|
||||
assert.deepEqual(playground.results.map(x => x.path), ['Book One/Playground/What If.md']);
|
||||
assert.equal(await f.call('library.get', { universe: 'harbor', path: 'Second World/Characters/Elsewhere.md' }), null);
|
||||
});
|
||||
|
||||
test('each verb refuses missing universe and unknown keys before host work', async t => {
|
||||
let calls = 0;
|
||||
const f = await world(t, { adapter: { lookup: () => { calls++; } } });
|
||||
for (const [i, verb] of VERBS.entries()) {
|
||||
const { universe, ...missing } = ARGS[i];
|
||||
await assert.rejects(f.call(verb, missing), { code: 'invalid-request' });
|
||||
for (const extra of [{ repo: '/elsewhere' }, { commit: 'b'.repeat(40) }, { business: 'other' }, { accept: 'b'.repeat(40) }])
|
||||
await assert.rejects(f.call(verb, { ...ARGS[i], ...extra }), { code: 'invalid-request' });
|
||||
await assert.rejects(f.call(verb, { ...ARGS[i], universe: '' }), { code: 'invalid-request' });
|
||||
}
|
||||
assert.equal(calls, 0);
|
||||
});
|
||||
|
||||
test('no library, missing host adapter and undeclared project refuse library-not-configured', async t => {
|
||||
const noLibrary = await world(t, { projects: {} });
|
||||
const noAdapter = await world(t, { adapter: {} });
|
||||
const f = await world(t);
|
||||
for (const [i, verb] of VERBS.entries()) {
|
||||
await assert.rejects(noLibrary.call(verb, ARGS[i]), { code: 'library-not-configured' });
|
||||
await assert.rejects(noAdapter.call(verb, ARGS[i]), { code: 'library-not-configured' });
|
||||
await assert.rejects(f.call(verb, { ...ARGS[i], project: 'other' }), { code: 'library-not-configured' });
|
||||
}
|
||||
f.business.projects.second = mappingEntry();
|
||||
await assert.rejects(f.call('library.get', GET), { code: 'invalid-request' });
|
||||
assert.equal((await f.call('library.get', { ...GET, project: 'books' })).commit, f.commit);
|
||||
});
|
||||
|
||||
test('reads need the current holder on each call; human/tester/reader are not agents', async t => {
|
||||
const f = await world(t, { claimed: false });
|
||||
for (const [i, verb] of VERBS.entries()) {
|
||||
await assert.rejects(f.call(verb, ARGS[i]), { code: 'not-holder' });
|
||||
await assert.rejects(f.call(verb, ARGS[i], f.human), { code: 'agent-required' });
|
||||
await assert.rejects(f.call(verb, ARGS[i], f.tester), { code: 'tester-refused' });
|
||||
await assert.rejects(f.call(verb, ARGS[i], f.reader), { code: 'read-only' });
|
||||
}
|
||||
f.broker.request(f.agent, { verb: 'role.claim' });
|
||||
assert.equal((await f.call('library.get', GET)).commit, f.commit);
|
||||
f.broker.request(f.agent, { verb: 'role.release' });
|
||||
await assert.rejects(f.call('library.get', GET), { code: 'not-holder' });
|
||||
});
|
||||
|
||||
test('releasing a claim during an async lookup fences the response', { timeout: 30_000 }, async t => {
|
||||
let release, entered;
|
||||
const started = new Promise(resolve => { entered = resolve; });
|
||||
const f = await world(t, { adapter: { lookup: async () => {
|
||||
entered(); await new Promise(resolve => { release = resolve; }); return { lines: [[1, 'not returned']] };
|
||||
} } });
|
||||
const pending = f.call('library.get', GET);
|
||||
await started;
|
||||
f.broker.request(f.agent, { verb: 'role.release' });
|
||||
release();
|
||||
await assert.rejects(pending, { code: 'not-holder' });
|
||||
});
|
||||
|
||||
test('a business library removed during an async read cannot leak its result', { timeout: 30_000 }, async t => {
|
||||
let release, entered;
|
||||
const started = new Promise(resolve => { entered = resolve; });
|
||||
const f = await world(t, { adapter: { lookup: async () => {
|
||||
entered(); await new Promise(resolve => { release = resolve; }); return { lines: [[1, 'not returned']] };
|
||||
} } });
|
||||
const pending = f.call('library.get', GET);
|
||||
await started;
|
||||
delete f.business.projects.books;
|
||||
release();
|
||||
await assert.rejects(pending, { code: 'library-not-configured' });
|
||||
});
|
||||
|
||||
test('freshness boundary, unknown last fetch and failed fetch still use immutable served state', async t => {
|
||||
const f = await world(t);
|
||||
f.setState({ lastFetch: new Date(NOW - 60_000).toISOString() });
|
||||
await f.call('library.get', GET);
|
||||
assert.equal(f.fetches(), 0);
|
||||
f.setState({ lastFetch: new Date(NOW - 60_001).toISOString() });
|
||||
await f.call('library.get', GET);
|
||||
assert.equal(f.fetches(), 1);
|
||||
f.setState({ lastFetch: null });
|
||||
await f.call('library.universe', { universe: 'harbor' });
|
||||
assert.equal(f.fetches(), 2);
|
||||
let attempted = 0, clock = NOW;
|
||||
const failing = createLibraryLookup({ dataRoot: f.dataRoot, now: () => clock, sync: async () => {
|
||||
attempted++;
|
||||
f.setState({ lastError: { code: 'library-fetch-failed', at: new Date(clock).toISOString() } });
|
||||
return { code: 'library-fetch-failed' };
|
||||
} });
|
||||
f.setState({ lastFetch: null });
|
||||
const result = await failing(f.context, { method: 'get', args: GET, authorize() {} });
|
||||
assert.equal(attempted, 1);
|
||||
assert.equal(result.commit, f.commit);
|
||||
for (const elapsed of [1, 59_999, 60_000]) {
|
||||
clock = NOW + elapsed;
|
||||
const retry = await failing(f.context, { method: 'get', args: GET, authorize() {} });
|
||||
assert.equal(retry.commit, f.commit, 'serve the old pin during the failure cooldown');
|
||||
assert.equal(attempted, 1, 'recent lastError prevents another sync even though lastFetch is null');
|
||||
}
|
||||
clock = NOW + 60_001;
|
||||
assert.equal((await failing(f.context, { method: 'get', args: GET, authorize() {} })).commit, f.commit);
|
||||
assert.equal(attempted, 2, 'retry only after the failed-attempt interval expires');
|
||||
});
|
||||
|
||||
test('an unreadable sync state fails closed; a busy sync keeps serving the existing pin', async t => {
|
||||
const f = await world(t);
|
||||
const lookup = createLibraryLookup({ dataRoot: f.dataRoot, now: () => NOW, sync: async () => {
|
||||
throw new library.LibraryError('library-sync-busy', 'another fetch');
|
||||
} });
|
||||
f.setState({ lastFetch: null });
|
||||
assert.equal((await lookup(f.context, { method: 'get', args: GET, authorize() {} })).commit, f.commit);
|
||||
writeFileSync(join(f.mirror, library.SYNC_STATE_FILE), '{bad json', { mode: 0o600 });
|
||||
await assert.rejects(f.call('library.get', GET), { code: 'library-state-invalid' });
|
||||
});
|
||||
|
||||
test('a stale lookup sees the commit published by its sync, not the prior served commit', async t => {
|
||||
const f = await world(t);
|
||||
writeFileSync(join(f.repo, GET.path), note('character', 'Tomas', 'Changed ferry.'));
|
||||
git(f.repo, 'add', '-A'); git(f.repo, 'commit', '-qm', 'second fixture');
|
||||
const next = git(f.repo, 'rev-parse', 'HEAD');
|
||||
const lookup = createLibraryLookup({ dataRoot: f.dataRoot, now: () => NOW, sync: async () => {
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/served');
|
||||
f.setState({ served: next, lastFetch: new Date(NOW).toISOString() });
|
||||
return { code: null };
|
||||
} });
|
||||
f.setState({ lastFetch: null });
|
||||
const result = await lookup(f.context, { method: 'get', args: GET, authorize() {} });
|
||||
assert.equal(result.commit, next);
|
||||
assert.match(JSON.stringify(result.lines), /Changed ferry/);
|
||||
});
|
||||
|
||||
test('real socket awaits lookup results and rechecks rejected query arguments', { timeout: 30_000 }, async t => {
|
||||
const f = await world(t);
|
||||
const path = join(f.root, 'lookup.sock');
|
||||
const server = await serve({ broker: f.broker, path }); t.after(() => server.close());
|
||||
const client = new Client({ path, cap: f.agent });
|
||||
const result = await client.call('library.get', GET);
|
||||
assert.equal(result.commit, f.commit);
|
||||
await assert.rejects(client.call('library.find', { universe: 'harbor', query: '', scope: 'anything' }), /query-argument-invalid/);
|
||||
});
|
||||
|
||||
test('runtime actually injects the lookup adapter, without exposing human library.file', { timeout: 30_000 }, async t => {
|
||||
const f = scratchRepo(t, TEXTS), dataRoot = join(f.root, 'runtime'); mkdirSync(dataRoot);
|
||||
const business = { id: 'acme', human: 'jason', projects: { books: mappingEntry() },
|
||||
arbiters: { technical: 'coder', delivery: 'coder' },
|
||||
roles: { coder: { definition: 'coder', authority: { withinRole: [], crossRole: [] } } } };
|
||||
const runtime = await startBroker({ dataRoot, businesses: { acme: business } }); t.after(() => runtime.close());
|
||||
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
|
||||
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
|
||||
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify({ version: 1, branch: 'main', served: f.commit,
|
||||
lastFetch: new Date().toISOString(), lastError: null, diverged: null, problems: [] }), { mode: 0o600 });
|
||||
const cap = runtime.broker.bindLaunch({ business: 'acme', role: 'coder', run: 'r224-runtime', harness: 'pi' });
|
||||
runtime.broker.request(cap, { verb: 'role.claim' });
|
||||
assert.equal((await runtime.broker.request(cap, { verb: 'library.get', args: GET })).commit, f.commit);
|
||||
await assert.rejects(runtime.broker.request(cap, { verb: 'library.file', args: { project: 'books', path: GET.path } }),
|
||||
{ code: 'human-required' });
|
||||
});
|
||||
@@ -439,7 +439,6 @@ export function createLauncher({
|
||||
contract: readFileSync(resolved.contract, "utf8"),
|
||||
arbiter: Object.values(b.arbiters).includes(instance),
|
||||
tracker,
|
||||
library: Object.values(b.projects ?? {}).some(project => project.kind === "library"),
|
||||
credentials,
|
||||
session: { run, launchedBy, workspace, toolSocket, turnMarker, checkouts },
|
||||
harness,
|
||||
|
||||
@@ -27,7 +27,6 @@ import { bootConfig, loadSystem } from "../src/config.mjs";
|
||||
import { startTimeOf } from "../src/host.mjs";
|
||||
import { createLauncher } from "../src/launcher.mjs";
|
||||
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
|
||||
import { mappingEntry } from "../../library/tests/helpers.mjs";
|
||||
// Row 264a (#1769): a thread made by hand gets the default session the 3k-3l migration gives a
|
||||
// legacy thread, with the thread's id.
|
||||
const DEFAULT_SESSION = [
|
||||
@@ -443,24 +442,3 @@ test("REQ-LIMIT-4 a thread run at its token limit stops at the end of its turn:
|
||||
assert.deepEqual(threadRuns(f, A), []);
|
||||
assert.equal(host.runtime.threadsState("acme").find((x) => x.thread === A).holder, null);
|
||||
});
|
||||
|
||||
// Row224: the launcher must derive library presence from business projects, not
|
||||
// from session checkouts (a library never has an agent clone).
|
||||
test("C2-05b a real launched bundle gets lookups for a library project without a library checkout", { skip, timeout: 60000 }, async t => {
|
||||
const prepared = prepare(t, { idle: 10, edit: doc => {
|
||||
doc.projects.books = mappingEntry();
|
||||
return doc;
|
||||
} });
|
||||
const { f } = prepared;
|
||||
const host = await start(t, prepared);
|
||||
const project = host.call("project.create", { name: "library work", repo: "stack" }).id;
|
||||
const thread = host.call("thread.create", { project, name: "lookup" }).id;
|
||||
await ask(host.call, thread, "inspect bundle");
|
||||
const run = launchLog(f).find(e => e.event === "launch" && e.thread === thread).run;
|
||||
const tools = JSON.parse(readFileSync(join(f.dataRoot, "launches", "acme", run, "bundle", "tools.json"), "utf8"));
|
||||
assert.deepEqual(tools.filter(tool => tool.name.startsWith("library_")).map(tool => tool.name),
|
||||
["library_find", "library_get", "library_search", "library_manuscript", "library_universe"]);
|
||||
const [session] = threadRuns(f, thread);
|
||||
assert.ok(session);
|
||||
assert.ok(!session.checkouts?.some(checkout => checkout.project === "books"));
|
||||
});
|
||||
@@ -292,7 +292,6 @@ function prompt({ contract, resolved, session, tools, untyped, builtins, persona
|
||||
// resolved resolveInstance(...) output
|
||||
// contract the role contract's text
|
||||
// arbiter true when the business names this instance as an arbiter
|
||||
// library true when the trusted business configuration has a library project
|
||||
// tracker true when the broker has task verbs for the business
|
||||
// credentials the broker's credentialStatus(...) for the instance (metadata only)
|
||||
// session { run, launchedBy, workspace, toolSocket, turnMarker, checkouts }
|
||||
@@ -310,11 +309,11 @@ function prompt({ contract, resolved, session, tools, untyped, builtins, persona
|
||||
// (row 213); an instance with persona.sha256 and no personaDir refuses
|
||||
// Returns { manifest, files, policy, tools, env } with absolute paths; env is the module's
|
||||
// adapter variables, which the host writes into session.json.
|
||||
export function buildBundle({ dir, resolved, contract, arbiter = false, tracker = false, library = false, credentials = [], session, harness, route = "primary", routeReason = null, harnessVersion = null, models = null, land = false, skillsDir = null, personaDir = null }) {
|
||||
export function buildBundle({ dir, resolved, contract, arbiter = false, tracker = false, credentials = [], session, harness, route = "primary", routeReason = null, harnessVersion = null, models = null, land = false, skillsDir = null, personaDir = null }) {
|
||||
if (route !== "primary" && route !== "fallback") throw new BundleError(`route ${JSON.stringify(route)} isn't primary or fallback`);
|
||||
const model = sessionModel(resolved, models, route);
|
||||
const web = webPlan(resolved, model.harness, harness.builtinWeb());
|
||||
const tools = typedTools({ resolved, arbiter, tracker, library, projects: (session.checkouts ?? []).map((c) => c.project), webTyped: !web.builtin, land });
|
||||
const tools = typedTools({ resolved, arbiter, tracker, projects: (session.checkouts ?? []).map((c) => c.project), webTyped: !web.builtin, land });
|
||||
const untyped = untypedActions(resolved, tools);
|
||||
const persona = readPersona(resolved, personaDir);
|
||||
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
|
||||
@@ -78,10 +78,9 @@ const TASK_TOOLS = {
|
||||
|
||||
const MESSAGE_CLASSES = ["REQUEST", "ASSIGNMENT", "REVIEW-REQUEST", "REVIEW-RESULT", "RESULT", "INFO", "DECISION", "REACTION", "BLOCK"];
|
||||
|
||||
// typedTools({ resolved, arbiter, tracker, projects, land, library }) -> [{ name, verb, action, description, parameters }]
|
||||
// typedTools({ resolved, arbiter, tracker, projects, land }) -> [{ name, verb, action, description, parameters }]
|
||||
// resolved resolveInstance(...) output for the session's instance
|
||||
// arbiter true when the business names this instance as an arbiter
|
||||
// library true when the trusted business configuration has a library project
|
||||
// tracker true when the broker has task verbs for the business
|
||||
// projects the project ids the session has a clone of
|
||||
// land true when the role and thread hold a merge decision a landing could use (landHeld)
|
||||
@@ -89,7 +88,7 @@ const MESSAGE_CLASSES = ["REQUEST", "ASSIGNMENT", "REVIEW-REQUEST", "REVIEW-RESU
|
||||
// `verb` is the broker verb; the `launch` tool has verb "launch" and goes
|
||||
// to the host instead, and "working.push", "working.fetch" and
|
||||
// "working.land" go to the host's git socket.
|
||||
export function typedTools({ resolved, arbiter = false, tracker = false, projects = [], webTyped = true, land = false, library = false }) {
|
||||
export function typedTools({ resolved, arbiter = false, tracker = false, projects = [], webTyped = true, land = false }) {
|
||||
const held = new Set([...resolved.limits.authority.withinRole, ...resolved.limits.authority.crossRole]);
|
||||
const tools = [];
|
||||
const add = (name, verb, action, description, parameters) => tools.push({ name, verb, action, description, parameters });
|
||||
@@ -223,32 +222,6 @@ export function typedTools({ resolved, arbiter = false, tracker = false, project
|
||||
url: str("The https url to fetch", { maxLength: 2048 }),
|
||||
}, ["url"]));
|
||||
}
|
||||
// Row224: no library clone or token reaches a session. The broker chooses the served
|
||||
// commit and requires a universe on every call; cursors page without cutting replies.
|
||||
if (library) {
|
||||
const universe = str("Universe ID in the configured library", { minLength: 1 });
|
||||
const project = str("Library project ID; omit only when the business has one library");
|
||||
const scope = str("canon by default; playground is opt-in", { enum: ["canon", "playground"] });
|
||||
const book = str("Book folder from library_universe");
|
||||
const offset = { type: "integer", minimum: 0, description: "next from the previous page" };
|
||||
const line = { type: "integer", minimum: 1, description: "One-based line number" };
|
||||
const base = { universe, project };
|
||||
add("library_find", "library.find", null, "Find names in one universe, at the served commit. At most 50 results; continue with next as offset.", obj({
|
||||
...base, query: str("Case-insensitive name or filename substring; empty matches all"),
|
||||
type: str("Mapped type", { enum: ["character", "place", "research", "tracker", "summary"] }),
|
||||
book, scope, offset,
|
||||
}, ["universe", "query"]));
|
||||
add("library_get", "library.get", null, "Read a mapped note and visible links in one universe at the served commit. next pages lines; links_next pages links. An explicit own-universe playground path is readable.", obj({
|
||||
...base, path: str("Mapped note path relative to the library"), from: line, links_from: offset,
|
||||
}, ["universe", "path"]));
|
||||
add("library_search", "library.search", null, "Search text in one universe at the served commit. At most 50 hits; use next as offset. Playground is opt-in.", obj({
|
||||
...base, text: str("Nonempty case-insensitive substring", { minLength: 1 }), book, scope, offset,
|
||||
}, ["universe", "text"]));
|
||||
add("library_manuscript", "library.manuscript", null, "Read one chapter of a book at the served commit, at most 400 lines. Omit chapter for the first; continue with next or next_chapter.", obj({
|
||||
...base, book, scope, chapter: str("Chapter filename without extension"), from: line, to: line,
|
||||
}, ["universe", "book"]));
|
||||
add("library_universe", "library.universe", null, "Read a universe's mapping at the served commit: books, types, playground prefixes, and hybrid sources/constraints/notes or draft metadata.", obj(base, ["universe"]));
|
||||
}
|
||||
add("read_spec", "spec.read", null, "Read a project's committed north-star spec and its file history. commit is an optional full reachable commit ID. The file commit, rather than HEAD, is what a decision cites. confirmed says the file commit was authored by the business's human.", obj({
|
||||
project: str("Project ID from the business file"),
|
||||
commit: str("Optional full historical commit ID", { pattern: "^[0-9a-f]{40}$" }),
|
||||
@@ -304,11 +277,6 @@ export function typedTools({ resolved, arbiter = false, tracker = false, project
|
||||
if (held.has("task.create"))
|
||||
add("list_requirements", "requirements", null, "List the approved PRD's requirement ids, each with a one-line title. task_create takes only these. When none fits a request, raise a prd.approve decision instead of creating the task.", obj({}));
|
||||
}
|
||||
// mos-1308 / M3: no generic spec, evidence, tracker, decision or write tools. The
|
||||
// C2-08 web grant still supplies broker web reads to companion and research only.
|
||||
if (CREATIVE_DEFINITIONS.includes(resolved.definition))
|
||||
return tools.filter(tool => tool.name === "send_message" || tool.name.startsWith("library_") ||
|
||||
tool.name === "web_search" || tool.name === "web_fetch");
|
||||
return tools;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
// Row224 M3: exact creative tool sets preserve the existing C2-08 web grants.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdirSync, readFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { typedTools } from '../src/tools.mjs';
|
||||
import { buildBundle } from '../src/bundle.mjs';
|
||||
import { TYPES } from '../../library/src/mapping.mjs';
|
||||
import { HARNESS, REPO_SKILLS, resolvedFor, scratch } from './helpers.mjs';
|
||||
|
||||
const LOOKUPS = ['library_find', 'library_get', 'library_search', 'library_manuscript', 'library_universe'];
|
||||
const CREATIVE = ['companion', 'research', 'ideas', 'proofreader', 'fact-checker', 'continuity'];
|
||||
const role = (t, definition, harness = 'pi') => resolvedFor(t, 'bean', d => {
|
||||
d.roles.bean = { definition, vars: { harness, model: 'claude-sonnet-5-5', thinking: 'high' } };
|
||||
});
|
||||
const expected = definition => ['send_message', ...(['companion', 'research'].includes(definition)
|
||||
? ['web_search', 'web_fetch'] : []), ...LOOKUPS];
|
||||
|
||||
test('M3: per-role snapshots contain six names, or eight with the C2-08 web grant', t => {
|
||||
for (const definition of CREATIVE) {
|
||||
const tools = typedTools({ resolved: role(t, definition), library: true, tracker: true, arbiter: true });
|
||||
assert.deepEqual(tools.map(t => t.name), expected(definition), definition);
|
||||
assert.ok(tools.every(t => t.parameters.additionalProperties === false));
|
||||
}
|
||||
});
|
||||
|
||||
test('every noncreative role gets lookups when a library is configured, not otherwise', t => {
|
||||
for (const instance of ['coder', 'reviewer', 'pm', 'chief']) {
|
||||
const resolved = resolvedFor(t, instance);
|
||||
const ordinary = typedTools({ resolved }).map(t => t.name);
|
||||
assert.ok(!ordinary.some(name => name.startsWith('library_')));
|
||||
const tools = typedTools({ resolved, library: true });
|
||||
assert.deepEqual(tools.filter(t => t.name.startsWith('library_')).map(t => t.name), LOOKUPS);
|
||||
assert.deepEqual(tools.filter(t => !t.name.startsWith('library_')).map(t => t.name), ordinary);
|
||||
}
|
||||
});
|
||||
|
||||
test('lookup schemas require a universe even for get, include paging, and cannot select repo or commit', t => {
|
||||
const tools = typedTools({ resolved: role(t, 'ideas'), library: true });
|
||||
const required = [ ['universe', 'query'], ['universe', 'path'], ['universe', 'text'], ['universe', 'book'], ['universe'] ];
|
||||
for (const [i, tool] of tools.filter(t => t.name.startsWith('library_')).entries()) {
|
||||
assert.deepEqual(tool.parameters.required, required[i]);
|
||||
assert.equal(tool.verb, LOOKUPS[i].replace('_', '.'));
|
||||
assert.equal(tool.action, null);
|
||||
assert.equal(tool.parameters.additionalProperties, false);
|
||||
for (const key of ['repo', 'commit', 'business', 'accept']) assert.equal(Object.hasOwn(tool.parameters.properties, key), false);
|
||||
}
|
||||
const byName = Object.fromEntries(tools.map(t => [t.name, t]));
|
||||
assert.deepEqual(byName.library_find.parameters.properties.scope.enum, ['canon', 'playground']);
|
||||
assert.deepEqual(byName.library_find.parameters.properties.type.enum, [...TYPES]);
|
||||
assert.equal(byName.library_find.parameters.properties.offset.minimum, 0);
|
||||
assert.equal(byName.library_get.parameters.properties.from.minimum, 1);
|
||||
assert.equal(byName.library_get.parameters.properties.links_from.minimum, 0);
|
||||
assert.equal(byName.library_manuscript.parameters.properties.chapter.type, 'string');
|
||||
});
|
||||
|
||||
test('bundle flag reaches both harness tool manifests and creative prompt without generic tools', t => {
|
||||
for (const definition of CREATIVE) for (const harness of ['pi', 'claude-code']) {
|
||||
const dir = scratch(t), workspace = join(dir, 'ws'); mkdirSync(workspace);
|
||||
const b = buildBundle({ dir: join(dir, 'bundle'), resolved: role(t, definition, harness), library: true,
|
||||
contract: '# Creative\n', tracker: true, arbiter: true, skillsDir: REPO_SKILLS,
|
||||
session: { run: 'r224', launchedBy: 'chief', workspace, toolSocket: join(dir, 't.sock'), turnMarker: join(dir, 'turn.done') },
|
||||
harness: HARNESS[harness] });
|
||||
assert.deepEqual(b.tools.map(t => t.name), expected(definition), `${definition} ${harness}`);
|
||||
const serialized = JSON.parse(readFileSync(b.files.tools, 'utf8'));
|
||||
assert.deepEqual(serialized.map(t => t.name), expected(definition));
|
||||
const prompt = readFileSync(b.files.prompt, 'utf8');
|
||||
assert.match(prompt, /library_find/);
|
||||
assert.doesNotMatch(prompt, /Bus tools: [^\n]*(read_spec|list_agents|inbox|trail|write_record|raise_decision|list_tasks)/);
|
||||
}
|
||||
});
|
||||
@@ -1,75 +0,0 @@
|
||||
// The lookup index of design 3 (C2-05a): types, names, links and backlinks for one commit of her repo.
|
||||
// It is built from the reader's output, so it holds only mapped, readable files: R1, R2 and R7 leave a
|
||||
// file out, and an unresolved link is not in it. It is cached in memory by commit id. The key also
|
||||
// carries the repo path and a digest of the mapping, because one commit reads differently under another
|
||||
// mapping. The index is not universe-aware; query.mjs decides what a universe may see.
|
||||
import { createHash } from 'node:crypto';
|
||||
import { readLibrary } from './reader.mjs';
|
||||
import { raise } from './problems.mjs';
|
||||
|
||||
const CACHE_LIMIT = 8;
|
||||
const cache = new Map();
|
||||
|
||||
const push = (map, key, value) => {
|
||||
if (!map.has(key)) map.set(key, []);
|
||||
map.get(key).push(value);
|
||||
};
|
||||
// A file's name for lookups: its front matter `name` when it is text, else its file name.
|
||||
const displayName = file => {
|
||||
const name = file.frontmatter?.name;
|
||||
return typeof name === 'string' && name.trim() ? name.trim() : file.name;
|
||||
};
|
||||
// A link as the index keeps it: the file at the other end, its line, and the field when front matter holds it.
|
||||
const linkAt = (path, link) => ({ path, line: link.line, ...(link.field === undefined ? {} : { field: link.field }) });
|
||||
|
||||
// Builds the index from readLibrary's output. Pure: no git, no clock.
|
||||
export function buildIndex(library, mapping) {
|
||||
const files = new Map(), types = new Map(), names = new Map(), linksOut = new Map(), linksIn = new Map();
|
||||
for (const file of library.files) {
|
||||
const name = displayName(file);
|
||||
files.set(file.path, {
|
||||
path: file.path, scope: file.scope, universe: file.universe, book: file.book, type: file.type,
|
||||
folder: file.folder, stem: file.name, name, frontmatter: file.frontmatter, lines: file.lines,
|
||||
});
|
||||
if (file.type !== null) push(types, file.type, file.path);
|
||||
push(names, name.toLowerCase(), file.path);
|
||||
linksOut.set(file.path, file.links.map(link => linkAt(link.path, link)));
|
||||
for (const link of file.links) push(linksIn, link.path, linkAt(file.path, link));
|
||||
}
|
||||
return Object.freeze({
|
||||
commit: library.commit, mapping, files, paths: [...files.keys()], types, names, linksOut, linksIn,
|
||||
problems: library.problems,
|
||||
});
|
||||
}
|
||||
|
||||
const digest = mapping => createHash('sha256').update(JSON.stringify(mapping)).digest('hex');
|
||||
|
||||
// The index for one commit under one mapping, read once and kept in memory. A repeat call returns the same
|
||||
// object. Past CACHE_LIMIT entries the oldest goes. A failed read is not kept.
|
||||
export async function indexFor({ repo, commit, mapping }) {
|
||||
if (!mapping || mapping.kind !== 'library') raise('library-mapping-invalid', 'expected a checked library mapping');
|
||||
const key = JSON.stringify([repo, commit, digest(mapping)]);
|
||||
if (cache.has(key)) {
|
||||
const hit = cache.get(key);
|
||||
cache.delete(key);
|
||||
cache.set(key, hit);
|
||||
return hit;
|
||||
}
|
||||
const pending = readLibrary({ repo, commit, mapping }).then(library => buildIndex(library, mapping));
|
||||
cache.set(key, pending);
|
||||
while (cache.size > CACHE_LIMIT) cache.delete(cache.keys().next().value);
|
||||
try {
|
||||
return await pending;
|
||||
} catch (error) {
|
||||
if (cache.get(key) === pending) cache.delete(key);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function clearIndexCache() {
|
||||
cache.clear();
|
||||
}
|
||||
|
||||
export function indexCacheSize() {
|
||||
return cache.size;
|
||||
}
|
||||
@@ -8,5 +8,3 @@ export { renderItems, KINDS, NOTICES, UNREADABLE } from './render.mjs';
|
||||
export { survey, formatSurvey, surveyMain } from './survey.mjs';
|
||||
export { SYNC_STATE_FILE, mirrorPath, ensureMirror, withLibraryLock, servedCommit, readSyncState, libraryStatus, readLibraryFile } from './mirror.mjs';
|
||||
export { syncCommand, sync, ASKPASS, SLOT, SYNC_INTERVAL_MS } from './sync.mjs';
|
||||
export { buildIndex, indexFor, clearIndexCache, indexCacheSize } from './index-build.mjs';
|
||||
export { sees, find, get, search, manuscript, universeInfo, QUERY_REPLY_CAP, LOOKUP_LIMIT, MANUSCRIPT_LINES } from './query.mjs';
|
||||
@@ -1,240 +0,0 @@
|
||||
// The lookup queries of design 3 (C2-05a): find, get, search, manuscript and universeInfo. Each takes a
|
||||
// source {repo, commit, mapping} and an argument object, and returns a reply that fits QUERY_REPLY_CAP.
|
||||
// Universe isolation is here, in sees(): a file reaches a reply only when sees() allows it. Scope comes
|
||||
// from the path (D2). Unmapped files are not in the index, so they never appear. Lists page by position
|
||||
// and never cut an item. A search line too long to fit alone comes back as a marked hit with no excerpt;
|
||||
// a find item, a file's front matter or one of its lines that cannot fit alone is refused.
|
||||
import { raise } from './problems.mjs';
|
||||
import { TYPES } from './mapping.mjs';
|
||||
import { indexFor } from './index-build.mjs';
|
||||
|
||||
// The same cap as packages/harness/src/tools.mjs REPLY_CAP. The query tests assert the two stay equal.
|
||||
export const QUERY_REPLY_CAP = 40_000;
|
||||
export const LOOKUP_LIMIT = 50;
|
||||
export const MANUSCRIPT_LINES = 400;
|
||||
const SCOPES = Object.freeze(['canon', 'playground']);
|
||||
|
||||
const bytes = value => JSON.stringify(value).length;
|
||||
const invalid = message => raise('query-argument-invalid', message);
|
||||
const inside = (folder, path) => path.startsWith(folder.endsWith('/') ? folder : `${folder}/`);
|
||||
|
||||
function textArg(value, name) {
|
||||
if (typeof value !== 'string') invalid(`${name} must be text`);
|
||||
return value;
|
||||
}
|
||||
function scopeArg(value) {
|
||||
if (value === undefined) return 'canon';
|
||||
if (!SCOPES.includes(value)) invalid('scope must be "canon" or "playground"');
|
||||
return value;
|
||||
}
|
||||
function positionArg(value, name, fallback) {
|
||||
if (value === undefined) return fallback;
|
||||
if (!Number.isSafeInteger(value) || value < 1) invalid(`${name} must be a line number of 1 or more`);
|
||||
return value;
|
||||
}
|
||||
function offsetArg(value) {
|
||||
if (value === undefined) return 0;
|
||||
if (!Number.isSafeInteger(value) || value < 0) invalid('offset must be a whole number of 0 or more');
|
||||
return value;
|
||||
}
|
||||
function universeArg(mapping, value) {
|
||||
if (typeof value !== 'string' || !value) invalid('universe is required');
|
||||
if (!Object.hasOwn(mapping.universes, value)) raise('query-universe-unknown', `this library has no universe ${JSON.stringify(value)}`);
|
||||
return value;
|
||||
}
|
||||
|
||||
// Whether universe U may see a file. Its own files, in either scope. A hybrid also sees the canon of
|
||||
// its declared sources, minus the folders its exclude constraints name. A hybrid never sees a source's
|
||||
// playground. Pin constraints do not hide anything; they are reported by universeInfo only.
|
||||
export function sees(mapping, universe, file) {
|
||||
if (file.universe === universe) return true;
|
||||
const entry = mapping.universes[universe];
|
||||
if (entry.kind !== 'hybrid' || !entry.sources.includes(file.universe) || file.scope !== 'canon') return false;
|
||||
return !entry.constraints.some(c => c.exclude !== undefined && inside(c.exclude, file.path));
|
||||
}
|
||||
|
||||
// The files one universe may see in one scope, with optional book and type filters, in path order.
|
||||
async function visible(source, universe, { scope, book, type }) {
|
||||
const index = await indexFor(source);
|
||||
const files = index.paths
|
||||
.map(path => index.files.get(path))
|
||||
.filter(file => file.scope === scope && sees(index.mapping, universe, file)
|
||||
&& (book === undefined || file.book === book) && (type === undefined || file.type === type));
|
||||
return { index, files };
|
||||
}
|
||||
|
||||
// A reply whose size is over the cap is refused, never cut.
|
||||
function checked(reply) {
|
||||
if (bytes(reply) > QUERY_REPLY_CAP) raise('query-item-too-large', `the reply is over the ${QUERY_REPLY_CAP}-character cap`);
|
||||
return reply;
|
||||
}
|
||||
|
||||
// Picks items from `offset`, at most `limit`, while the reply still fits. build(picked, next) returns
|
||||
// the reply; `next` is the position of the first item not picked, or null when none is left. An item
|
||||
// that cannot fit even alone is refused: skipping it would hide it, and cutting it would break it.
|
||||
function fitPage(items, { offset = 0, limit = LOOKUP_LIMIT, build }) {
|
||||
const picked = [];
|
||||
let at = offset;
|
||||
while (at < items.length && picked.length < limit) {
|
||||
// The size check uses items.length for `next`, the widest number a cursor can take, so a page never runs over.
|
||||
if (bytes(build([...picked, items[at]], items.length)) > QUERY_REPLY_CAP) {
|
||||
if (!picked.length) raise('query-item-too-large', `the item at position ${at} alone is over the ${QUERY_REPLY_CAP}-character cap`);
|
||||
break;
|
||||
}
|
||||
picked.push(items[at]);
|
||||
at += 1;
|
||||
}
|
||||
return checked(build(picked, at < items.length ? at : null));
|
||||
}
|
||||
|
||||
// find: the files whose name or file name holds `query`, case-insensitive. Up to 50 per page.
|
||||
export async function find(source, { universe: name, query, type, book, scope, offset } = {}) {
|
||||
const universe = universeArg(source.mapping, name);
|
||||
const needle = textArg(query, 'query').toLowerCase();
|
||||
if (type !== undefined && !TYPES.includes(type)) invalid(`type must be one of ${TYPES.join(', ')}`);
|
||||
const { index, files } = await visible(source, universe, {
|
||||
scope: scopeArg(scope), book: book === undefined ? undefined : textArg(book, 'book'), type,
|
||||
});
|
||||
const hits = files.filter(file => file.name.toLowerCase().includes(needle) || file.stem.toLowerCase().includes(needle));
|
||||
return fitPage(hits, {
|
||||
offset: offsetArg(offset),
|
||||
build: (picked, next) => ({
|
||||
commit: index.commit,
|
||||
results: picked.map(file => ({
|
||||
path: file.path, type: file.type, name: file.name, appears_in: file.frontmatter?.appears_in ?? null,
|
||||
scope: file.scope, universe: file.universe, book: file.book,
|
||||
})),
|
||||
next,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
// get: one visible file, its front matter, its lines from `from`, and its visible links both ways.
|
||||
// Returns null when the file is missing or not visible, so a caller cannot tell the two apart. Lines page
|
||||
// by `from` (reply `next`), links page together by `links_from` (reply `links_next`); `links_out` come
|
||||
// before `links_in`. Every page carries the front matter. Links take at most half the cap, so a hub note
|
||||
// with many backlinks stays readable and still leaves room for lines. A page with lines left carries at
|
||||
// least one line, so links shrink when a line needs the room; read on with `from: next` and
|
||||
// `links_from: links_next`. A `from` past the last line returns links alone. Only front matter, or a
|
||||
// line that cannot fit with no links beside it, is refused.
|
||||
export async function get(source, { universe: name, path, from, links_from: linksFrom } = {}) {
|
||||
const universe = universeArg(source.mapping, name);
|
||||
const startLine = positionArg(from, 'from', 1);
|
||||
const linkStart = offsetArg(linksFrom);
|
||||
if (typeof path !== 'string') invalid('path must be text');
|
||||
const index = await indexFor(source);
|
||||
const file = index.files.get(path);
|
||||
if (!file || !sees(index.mapping, universe, file)) return null;
|
||||
const reachable = links => links.filter(link => {
|
||||
const other = index.files.get(link.path);
|
||||
return other !== undefined && sees(index.mapping, universe, other);
|
||||
});
|
||||
const outs = reachable(index.linksOut.get(file.path) ?? []);
|
||||
const ins = reachable(index.linksIn.get(file.path) ?? []);
|
||||
const links = [...outs.map(link => ({ out: true, link })), ...ins.map(link => ({ out: false, link }))];
|
||||
const lines = file.lines.slice(startLine - 1);
|
||||
const linkRest = Math.max(0, links.length - linkStart);
|
||||
const page = (nLines, nLinks) => {
|
||||
const taken = links.slice(linkStart, linkStart + nLinks);
|
||||
return {
|
||||
path: file.path, commit: index.commit, scope: file.scope, frontmatter: file.frontmatter,
|
||||
lines: lines.slice(0, nLines),
|
||||
links_out: taken.filter(item => item.out).map(item => item.link),
|
||||
links_in: taken.filter(item => !item.out).map(item => item.link),
|
||||
next: nLines < lines.length ? startLine + nLines : null,
|
||||
links_next: nLinks < linkRest ? linkStart + nLinks : null,
|
||||
};
|
||||
};
|
||||
// How many leading items fit in `room`, by each item's own size.
|
||||
const count = (items, room) => {
|
||||
let used = 0, n = 0;
|
||||
while (n < items.length && used + bytes(items[n]) + 1 <= room) used += bytes(items[n++]) + 1;
|
||||
return n;
|
||||
};
|
||||
const base = bytes(page(0, 0));
|
||||
if (base > QUERY_REPLY_CAP) raise('query-item-too-large', `the front matter of ${file.path} alone is over the ${QUERY_REPLY_CAP}-character cap`);
|
||||
if (lines.length && bytes(page(1, 0)) > QUERY_REPLY_CAP) {
|
||||
raise('query-item-too-large', `line ${lines[0][0]} of ${file.path} alone is over the ${QUERY_REPLY_CAP}-character cap`);
|
||||
}
|
||||
const rest = links.slice(linkStart);
|
||||
let nLinks = count(rest, (lines.length ? Math.floor(QUERY_REPLY_CAP / 2) : QUERY_REPLY_CAP) - base);
|
||||
if (!nLinks && rest.length) nLinks = count(rest, QUERY_REPLY_CAP - base);
|
||||
if (!nLinks && rest.length) raise('query-item-too-large', `a link of ${file.path} alone is over the ${QUERY_REPLY_CAP}-character cap`);
|
||||
while (lines.length && nLinks > 0 && bytes(page(1, nLinks)) > QUERY_REPLY_CAP) nLinks = Math.floor(nLinks / 2);
|
||||
let nLines = Math.max(lines.length ? 1 : 0, count(lines, QUERY_REPLY_CAP - bytes(page(0, nLinks))));
|
||||
while (nLines > 1 && bytes(page(nLines, nLinks)) > QUERY_REPLY_CAP) nLines -= 1;
|
||||
return checked(page(nLines, nLinks));
|
||||
}
|
||||
|
||||
// search: lines whose text holds `text`, case-insensitive. The excerpt is the whole line. Up to 50 per page.
|
||||
export async function search(source, { universe: name, text, book, scope, offset } = {}) {
|
||||
const universe = universeArg(source.mapping, name);
|
||||
const needle = textArg(text, 'text').toLowerCase();
|
||||
if (!needle) invalid('text must not be empty');
|
||||
const { index, files } = await visible(source, universe, {
|
||||
scope: scopeArg(scope), book: book === undefined ? undefined : textArg(book, 'book'),
|
||||
});
|
||||
const hits = [];
|
||||
for (const file of files) {
|
||||
for (const [line, body] of file.lines) {
|
||||
if (!body.toLowerCase().includes(needle)) continue;
|
||||
const hit = { path: file.path, line, excerpt: body, scope: file.scope };
|
||||
// The excerpt is the whole line. A line too long to fit alone is marked, not cut and not refused.
|
||||
hits.push(bytes({ commit: index.commit, results: [hit], next: Number.MAX_SAFE_INTEGER }) > QUERY_REPLY_CAP
|
||||
? { path: file.path, line, excerpt: null, excerpt_too_long: true, scope: file.scope } : hit);
|
||||
}
|
||||
}
|
||||
return fitPage(hits, {
|
||||
offset: offsetArg(offset),
|
||||
build: (picked, next) => ({ commit: index.commit, results: picked, next }),
|
||||
});
|
||||
}
|
||||
|
||||
// manuscript: one chapter file of a book, by its file name, up to 400 lines per call from `from` to `to`.
|
||||
// Without a chapter it returns the book's first chapter file in path order. Every reply carries
|
||||
// `next_chapter`, the next chapter file's name in path order or null. A name that matches two visible
|
||||
// files is refused. Files outside every book (book null) are never chapters of a book.
|
||||
export async function manuscript(source, { universe: name, book, scope, chapter, from, to } = {}) {
|
||||
const universe = universeArg(source.mapping, name);
|
||||
const bookName = textArg(book, 'book');
|
||||
const chapterName = chapter === undefined ? undefined : textArg(chapter, 'chapter');
|
||||
const start = positionArg(from, 'from', 1);
|
||||
const end = positionArg(to, 'to', Infinity);
|
||||
if (end < start) invalid('to must not be before from');
|
||||
const { index, files } = await visible(source, universe, { scope: scopeArg(scope), book: bookName, type: 'manuscript' });
|
||||
let at = 0;
|
||||
if (chapterName !== undefined) {
|
||||
const matches = files.filter(file => file.stem === chapterName);
|
||||
if (matches.length > 1) raise('query-chapter-ambiguous', `${matches.length} visible files are named ${JSON.stringify(chapterName)}`);
|
||||
if (!matches.length) return null;
|
||||
at = files.indexOf(matches[0]);
|
||||
}
|
||||
const file = files[at];
|
||||
if (!file) return null;
|
||||
const nextChapter = files[at + 1]?.stem ?? null;
|
||||
return fitPage(file.lines.slice(start - 1, end), {
|
||||
limit: MANUSCRIPT_LINES,
|
||||
build: (picked, next) => ({
|
||||
path: file.path, commit: index.commit, scope: file.scope, lines: picked,
|
||||
next: next === null ? null : start + next, next_chapter: nextChapter,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
// universeInfo: a universe's kind, name, books, playground prefixes, type folders and manuscript folders.
|
||||
// A hybrid adds its sources, constraints and notes. A draft has no books and no canon: its kind, name
|
||||
// and prefixes are the whole answer beside the mapping's types. It reads only the checked mapping and the
|
||||
// served commit.
|
||||
export async function universeInfo(source, { universe: name } = {}) {
|
||||
const universe = universeArg(source.mapping, name);
|
||||
const entry = source.mapping.universes[universe];
|
||||
const hybrid = entry.kind === 'hybrid';
|
||||
return checked({
|
||||
commit: source.commit, universe, name: entry.name, kind: entry.kind ?? 'universe',
|
||||
books: entry.books, playground: entry.playground, types: source.mapping.types, manuscript: source.mapping.manuscript,
|
||||
...(hybrid ? {
|
||||
sources: entry.sources, constraints: entry.constraints,
|
||||
...(entry.notes === undefined ? {} : { notes: entry.notes }),
|
||||
} : {}),
|
||||
});
|
||||
}
|
||||
@@ -1,91 +0,0 @@
|
||||
// Index tests (C2-05a): the cache key and its bound, failed reads, mapped-only contents, and the rebuild
|
||||
// time on a 3,000-file fixture. Fixtures are made up and built under the scratch area.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { symlinkSync, writeFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { performance } from 'node:perf_hooks';
|
||||
import { clearIndexCache, indexCacheSize, indexFor } from '../src/index-build.mjs';
|
||||
import { git, mapping, MARKER } from './helpers.mjs';
|
||||
import { note, scratchRepo } from './query-fixture.mjs';
|
||||
|
||||
const FILES = {
|
||||
'Book One/Characters/Mara Vell.md': note('character', 'Mara Vell', MARKER),
|
||||
'Book One/Chapters/Chapter 01.md': '# One\n[[Mara Vell]]\n',
|
||||
'Inbox Note.md': `${MARKER}\n`,
|
||||
};
|
||||
|
||||
// A mapping whose harbor universe has the given display name. Each name gives a new cache key.
|
||||
const named = name => mapping(e => ({ ...e, universes: { ...e.universes, harbor: { ...e.universes.harbor, name } } }));
|
||||
|
||||
test('the same commit, repo path and mapping return the same index, and only mapped files are in it', async t => {
|
||||
clearIndexCache();
|
||||
const { repo, commit } = scratchRepo(t, FILES);
|
||||
const a = await indexFor({ repo, commit, mapping: mapping() });
|
||||
const b = await indexFor({ repo, commit, mapping: mapping() });
|
||||
assert.equal(a, b);
|
||||
assert.equal(indexCacheSize(), 1);
|
||||
assert.equal(a.commit, commit);
|
||||
assert.ok(a.files.has('Book One/Characters/Mara Vell.md'));
|
||||
assert.ok(!a.files.has('Inbox Note.md'));
|
||||
assert.deepEqual(a.names.get('mara vell'), ['Book One/Characters/Mara Vell.md']);
|
||||
assert.deepEqual(a.linksIn.get('Book One/Characters/Mara Vell.md').map(l => l.path), ['Book One/Chapters/Chapter 01.md']);
|
||||
});
|
||||
|
||||
test('a different repo path, mapping or commit gets its own index', async t => {
|
||||
clearIndexCache();
|
||||
const { root, repo, commit } = scratchRepo(t, FILES);
|
||||
const base = await indexFor({ repo, commit, mapping: mapping() });
|
||||
const alias = join(root, 'alias');
|
||||
symlinkSync(repo, alias);
|
||||
assert.notEqual(await indexFor({ repo: alias, commit, mapping: mapping() }), base);
|
||||
assert.notEqual(await indexFor({ repo, commit, mapping: named('Renamed Harbor') }), base);
|
||||
writeFileSync(join(repo, 'Book One/Places/New.md'), note('place', 'New'));
|
||||
git(repo, 'add', '-A');
|
||||
git(repo, 'commit', '-qm', 'second');
|
||||
const next = git(repo, 'rev-parse', 'HEAD');
|
||||
const later = await indexFor({ repo, commit: next, mapping: mapping() });
|
||||
assert.notEqual(later, base);
|
||||
assert.equal(later.commit, next);
|
||||
assert.ok(later.files.has('Book One/Places/New.md'));
|
||||
});
|
||||
|
||||
test('the cache holds at most eight indexes and evicts the least recently used', async t => {
|
||||
clearIndexCache();
|
||||
const { repo, commit } = scratchRepo(t, FILES);
|
||||
const maps = Array.from({ length: 9 }, (_, i) => named(`Harbor ${i}`));
|
||||
const first = await indexFor({ repo, commit, mapping: maps[0] });
|
||||
for (const m of maps.slice(1)) await indexFor({ repo, commit, mapping: m });
|
||||
assert.equal(indexCacheSize(), 8);
|
||||
assert.notEqual(await indexFor({ repo, commit, mapping: maps[0] }), first);
|
||||
assert.equal(indexCacheSize(), 8);
|
||||
});
|
||||
|
||||
test('a failed read is not kept in the cache', async t => {
|
||||
clearIndexCache();
|
||||
const { repo } = scratchRepo(t, FILES);
|
||||
await assert.rejects(indexFor({ repo, commit: '0'.repeat(40), mapping: mapping() }));
|
||||
assert.equal(indexCacheSize(), 0);
|
||||
});
|
||||
|
||||
test('a missing or unchecked mapping is refused before any read', async t => {
|
||||
clearIndexCache();
|
||||
const { repo, commit } = scratchRepo(t, FILES);
|
||||
await assert.rejects(indexFor({ repo, commit, mapping: undefined }), { code: 'library-mapping-invalid' });
|
||||
await assert.rejects(indexFor({ repo, commit, mapping: { kind: 'hybrid' } }), { code: 'library-mapping-invalid' });
|
||||
});
|
||||
|
||||
test('a 3,000-file rebuild completes and its time is reported', { timeout: 300_000 }, async t => {
|
||||
clearIndexCache();
|
||||
const folders = ['Characters', 'Places', 'Research'];
|
||||
const files = {};
|
||||
for (let i = 0; i < 3000; i++) {
|
||||
files[`Book One/${folders[i % 3]}/Note ${i}.md`] = note('character', `Note ${i}`, `Body ${i} [[Note ${(i + 1) % 3000}]]`);
|
||||
}
|
||||
const { repo, commit } = scratchRepo(t, files);
|
||||
const started = performance.now();
|
||||
const index = await indexFor({ repo, commit, mapping: mapping() });
|
||||
const ms = performance.now() - started;
|
||||
t.diagnostic(`3,000-file index rebuild: ${ms.toFixed(0)} ms`);
|
||||
assert.equal(index.paths.length, 3000);
|
||||
});
|
||||
@@ -1,31 +0,0 @@
|
||||
// Scratch repos for the query and index tests (C2-05a). They are built under tmpdir(), as helpers.mjs
|
||||
// repo() does; the gate sets TMPDIR under /mnt/storage/scratch/tmp, never /tmp (RULINGS "tmpdir flag").
|
||||
// Bodies are made up.
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { git } from './helpers.mjs';
|
||||
|
||||
// A repo with one commit holding `files` (path -> text). Removed when the test ends.
|
||||
export function scratchRepo(t, files) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'r216-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const repo = join(root, 'books');
|
||||
mkdirSync(repo);
|
||||
git(repo, 'init', '-q', '-b', 'main');
|
||||
for (const [path, text] of Object.entries(files)) {
|
||||
mkdirSync(dirname(join(repo, path)), { recursive: true });
|
||||
writeFileSync(join(repo, path), text);
|
||||
}
|
||||
git(repo, 'add', '-A');
|
||||
git(repo, 'commit', '-qm', 'fixture');
|
||||
return { root, repo, commit: git(repo, 'rev-parse', 'HEAD') };
|
||||
}
|
||||
|
||||
// A note: front matter with a type and a name, optional extra front matter lines, then a body.
|
||||
export const note = (type, name, body = '', extra = []) => ['---', `type: ${type}`, `name: ${name}`, ...extra, '---', body, ''].join('\n');
|
||||
|
||||
// The one exclusion the hybrid tests use, and the paths it hides. C2-15 (row 267) replaces the reading of
|
||||
// `exclude` (a path prefix today); swapping these two constants is the whole change to the tests.
|
||||
export const EXCLUSION = { exclude: 'Book One/Research' };
|
||||
export const EXCLUDED_PATHS = ['Book One/Research/Ferry Logs.md'];
|
||||
@@ -1,369 +0,0 @@
|
||||
// Query tests for design 3 (C2-05a): universe isolation, scope labels, unmapped files, hybrids, paging
|
||||
// under the reply cap, refusals and manuscript paging. Fixtures are made up and built under the scratch area.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { REPLY_CAP } from '../../harness/src/tools.mjs';
|
||||
import { find, get, search, manuscript, universeInfo, QUERY_REPLY_CAP, LOOKUP_LIMIT, MANUSCRIPT_LINES } from '../src/query.mjs';
|
||||
import { mapping, MARKER } from './helpers.mjs';
|
||||
import { scratchRepo, note, EXCLUSION, EXCLUDED_PATHS } from './query-fixture.mjs';
|
||||
|
||||
// A hybrid over harbor and second. Its one constraint is EXCLUSION (query-fixture.mjs) and it declares no
|
||||
// books or playground.
|
||||
const WITH_BLEND = e => ({
|
||||
...e,
|
||||
universes: {
|
||||
...e.universes,
|
||||
blend: {
|
||||
kind: 'hybrid', name: 'Blend', sources: ['harbor', 'second'], books: [], playground: [],
|
||||
constraints: [EXCLUSION], notes: 'made-up hybrid',
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
// PRD 1.14 levels (C2-01c): harbor gets a universe-level playground prefix outside every book, and a draft
|
||||
// universe has only playground prefixes. Files there have book null.
|
||||
const WITH_LEVELS = e => ({
|
||||
...WITH_BLEND(e),
|
||||
universes: {
|
||||
...WITH_BLEND(e).universes,
|
||||
harbor: { ...e.universes.harbor, playground: ['Book One/Playground/', 'Notes/Harbor/'] },
|
||||
idea: { kind: 'draft', name: 'New World', playground: ['Drafts/New World/'] },
|
||||
},
|
||||
});
|
||||
const LEVEL_FILES = {
|
||||
'Notes/Harbor/Scraps.md': note('summary', 'Scraps', `${MARKER} loose`),
|
||||
'Notes/Harbor/Chapters/Idea Chapter.md': `# Idea\n${MARKER} unbooked chapter\n`,
|
||||
'Drafts/New World/Characters/Rin.md': note('character', 'Rin', `${MARKER} draft`),
|
||||
'Drafts/New World/Summary.md': note('summary', 'Summary', `${MARKER} draft`),
|
||||
};
|
||||
|
||||
const FILES = {
|
||||
'Book One/Characters/Mara Vell.md': note('character', 'Mara Vell', `Grey eyes. ${MARKER}`,
|
||||
['appears_in: "[[Book One/Chapters/Chapter 01]]"', 'relationships:', ' - "[[Tomas]]"']),
|
||||
'Book One/Characters/Tomas.md': note('character', 'Tomas', MARKER),
|
||||
'Book One/Chapters/Chapter 01.md': `# One\n${MARKER} [[Mara Vell]] met [[Book One/Places/Harbor Town|the harbor]].\n`,
|
||||
'Book One/Chapters/Chapter 02.md': 'Quiet. [[Ferry Logs]] [[Ilse]]\n',
|
||||
'Book One/Places/Harbor Town.md': note('place', 'Harbor Town', `Salt. ${MARKER}`),
|
||||
'Book One/Research/Ferry Logs.md': note('research', 'Ferry Logs', `Timetable. ${MARKER}`),
|
||||
'Book One/Playground/What If.md': note('summary', 'What If', `${MARKER} [[Mara Vell]]`),
|
||||
'Second World/Characters/Ilse.md': note('character', 'Ilse', `${MARKER} Ilse.`),
|
||||
'Second World/Chapters/01.md': '# Elsewhere\n[[Ilse]] waits.\n',
|
||||
'Inbox Note.md': `${MARKER}\n`,
|
||||
'Book One/Board.canvas': '{"nodes":[]}\n',
|
||||
};
|
||||
|
||||
const src = (ref, map = mapping()) => ({ repo: ref.repo, commit: ref.commit, mapping: map });
|
||||
const paths = reply => reply.results.map(r => r.path);
|
||||
|
||||
test('the query reply cap equals the harness reply cap', () => {
|
||||
assert.equal(QUERY_REPLY_CAP, REPLY_CAP);
|
||||
});
|
||||
|
||||
test('find is bound to its universe and refuses a call without one', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
const names = paths(await find(s, { universe: 'harbor', query: '' }));
|
||||
assert.ok(names.length > 0);
|
||||
assert.ok(names.every(p => p.startsWith('Book One/')));
|
||||
assert.ok(!names.includes('Second World/Characters/Ilse.md'));
|
||||
await assert.rejects(find(s, { query: '' }), { code: 'query-argument-invalid' });
|
||||
await assert.rejects(find(s, { universe: 'nowhere', query: '' }), { code: 'query-universe-unknown' });
|
||||
});
|
||||
|
||||
test('every result carries its scope, and playground appears only when asked for', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
const canon = (await find(s, { universe: 'harbor', query: '' })).results;
|
||||
assert.ok(canon.every(r => r.scope === 'canon'));
|
||||
assert.ok(!canon.some(r => r.path.includes('/Playground/')));
|
||||
const play = (await find(s, { universe: 'harbor', query: '', scope: 'playground' })).results;
|
||||
assert.deepEqual(play.map(r => r.path), ['Book One/Playground/What If.md']);
|
||||
assert.ok(play.every(r => r.scope === 'playground'));
|
||||
});
|
||||
|
||||
test('unmapped files never appear in any reply', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
assert.deepEqual((await find(s, { universe: 'harbor', query: 'Inbox' })).results, []);
|
||||
assert.deepEqual((await find(s, { universe: 'harbor', query: 'Board' })).results, []);
|
||||
const hits = (await search(s, { universe: 'harbor', text: MARKER })).results;
|
||||
assert.ok(hits.length > 0);
|
||||
assert.ok(hits.every(r => r.path !== 'Inbox Note.md' && r.path !== 'Book One/Board.canvas'));
|
||||
});
|
||||
|
||||
test('a hybrid sees its sources canon, minus exclusions, and no playground', async t => {
|
||||
const s = src(scratchRepo(t, FILES), mapping(WITH_BLEND));
|
||||
const reply = await find(s, { universe: 'blend', query: '' });
|
||||
assert.deepEqual(paths(reply).sort(), [
|
||||
'Book One/Characters/Mara Vell.md', 'Book One/Characters/Tomas.md', 'Book One/Chapters/Chapter 01.md',
|
||||
'Book One/Chapters/Chapter 02.md', 'Book One/Places/Harbor Town.md',
|
||||
'Second World/Characters/Ilse.md', 'Second World/Chapters/01.md',
|
||||
]);
|
||||
assert.ok(reply.results.every(r => r.scope === 'canon' && ['harbor', 'second'].includes(r.universe)));
|
||||
assert.deepEqual((await find(s, { universe: 'blend', query: '', scope: 'playground' })).results, []);
|
||||
assert.deepEqual((await find(s, { universe: 'blend', query: 'Ferry' })).results, []);
|
||||
});
|
||||
|
||||
test('a hybrid gets a source file it can see, and not an excluded or foreign one', async t => {
|
||||
const ref = scratchRepo(t, FILES);
|
||||
const hybrid = src(ref, mapping(WITH_BLEND));
|
||||
const ilse = await get(hybrid, { universe: 'blend', path: 'Second World/Characters/Ilse.md' });
|
||||
assert.equal(ilse.scope, 'canon');
|
||||
assert.ok(ilse.links_in.some(l => l.path === 'Second World/Chapters/01.md'));
|
||||
assert.equal(await get(hybrid, { universe: 'blend', path: 'Book One/Research/Ferry Logs.md' }), null);
|
||||
assert.equal(await get(src(ref), { universe: 'harbor', path: 'Second World/Characters/Ilse.md' }), null);
|
||||
});
|
||||
|
||||
test('get lists visible links both ways and drops a target another universe cannot see', async t => {
|
||||
const ref = scratchRepo(t, FILES);
|
||||
const harbor = src(ref);
|
||||
const ch1 = await get(harbor, { universe: 'harbor', path: 'Book One/Chapters/Chapter 01.md' });
|
||||
assert.ok(ch1.links_out.some(l => l.path === 'Book One/Characters/Mara Vell.md' && l.line === 2));
|
||||
assert.ok(ch1.links_out.some(l => l.path === 'Book One/Places/Harbor Town.md'));
|
||||
const ch2 = await get(harbor, { universe: 'harbor', path: 'Book One/Chapters/Chapter 02.md' });
|
||||
assert.ok(ch2.links_out.some(l => l.path === 'Book One/Research/Ferry Logs.md'));
|
||||
assert.ok(!ch2.links_out.some(l => l.path === 'Second World/Characters/Ilse.md'));
|
||||
const mara = await get(harbor, { universe: 'harbor', path: 'Book One/Characters/Mara Vell.md' });
|
||||
assert.ok(mara.links_in.some(l => l.path === 'Book One/Chapters/Chapter 01.md'));
|
||||
assert.ok(mara.links_in.some(l => l.path === 'Book One/Playground/What If.md'));
|
||||
assert.ok(mara.links_out.some(l => l.path === 'Book One/Characters/Tomas.md' && l.field === 'relationships'));
|
||||
assert.ok(mara.links_out.some(l => l.path === 'Book One/Chapters/Chapter 01.md' && l.field === 'appears_in'));
|
||||
const blendCh2 = await get(src(ref, mapping(WITH_BLEND)), { universe: 'blend', path: 'Book One/Chapters/Chapter 02.md' });
|
||||
assert.ok(blendCh2.links_out.some(l => l.path === 'Second World/Characters/Ilse.md'));
|
||||
});
|
||||
|
||||
test('a long list pages by position, never cuts an item, and returns 50 per page', async t => {
|
||||
const files = { ...FILES };
|
||||
for (let i = 1; i <= 120; i++) files[`Book One/Places/Note ${i}.md`] = note('place', `Note ${i}`);
|
||||
const s = src(scratchRepo(t, files));
|
||||
const seen = [];
|
||||
let offset = 0;
|
||||
for (let pages = 0; pages < 10; pages++) {
|
||||
const reply = await find(s, { universe: 'harbor', query: 'Note', offset });
|
||||
assert.ok(JSON.stringify(reply).length <= QUERY_REPLY_CAP);
|
||||
assert.ok(reply.results.length <= LOOKUP_LIMIT);
|
||||
if (pages === 0) assert.equal(reply.results.length, LOOKUP_LIMIT);
|
||||
seen.push(...paths(reply));
|
||||
if (reply.next === null) break;
|
||||
offset = reply.next;
|
||||
}
|
||||
assert.equal(seen.length, 120);
|
||||
assert.equal(new Set(seen).size, 120);
|
||||
});
|
||||
|
||||
test('a page stays under the reply cap when items are large', async t => {
|
||||
const big = 'x'.repeat(2000);
|
||||
const files = { ...FILES };
|
||||
for (let i = 1; i <= 40; i++) files[`Book One/Places/Big ${i}.md`] = note('place', `Big ${i}`, '', [`appears_in: "${big}"`]);
|
||||
const s = src(scratchRepo(t, files));
|
||||
const first = await find(s, { universe: 'harbor', query: 'Big' });
|
||||
assert.ok(JSON.stringify(first).length <= QUERY_REPLY_CAP);
|
||||
assert.ok(first.results.length > 0 && first.results.length < 40);
|
||||
assert.notEqual(first.next, null);
|
||||
const seen = [];
|
||||
let offset = 0;
|
||||
for (let pages = 0; pages < 10; pages++) {
|
||||
const reply = await find(s, { universe: 'harbor', query: 'Big', offset });
|
||||
assert.ok(JSON.stringify(reply).length <= QUERY_REPLY_CAP);
|
||||
seen.push(...paths(reply));
|
||||
if (reply.next === null) break;
|
||||
offset = reply.next;
|
||||
}
|
||||
assert.equal(new Set(seen).size, 40);
|
||||
});
|
||||
|
||||
test('a search line too long to fit alone comes back marked, with no excerpt, and the other hits still return', async t => {
|
||||
const huge = 'y'.repeat(QUERY_REPLY_CAP + 1000);
|
||||
const s = src(scratchRepo(t, { ...FILES, 'Book One/Places/Wall.md': note('place', 'Wall', `${huge} ${MARKER}`) }));
|
||||
const reply = await search(s, { universe: 'harbor', text: 'yyyyyyyy' });
|
||||
assert.deepEqual(reply.results, [{ path: 'Book One/Places/Wall.md', line: 5, excerpt: null, excerpt_too_long: true, scope: 'canon' }]);
|
||||
const mixed = await search(s, { universe: 'harbor', text: MARKER.toLowerCase() });
|
||||
assert.ok(mixed.results.some(r => r.excerpt_too_long === true));
|
||||
assert.ok(mixed.results.some(r => typeof r.excerpt === 'string' && !('excerpt_too_long' in r)));
|
||||
assert.ok(JSON.stringify(mixed).length <= QUERY_REPLY_CAP);
|
||||
});
|
||||
|
||||
test('a find item that cannot fit alone is refused, not cut', async t => {
|
||||
const huge = 'y'.repeat(QUERY_REPLY_CAP + 1000);
|
||||
const s = src(scratchRepo(t, { ...FILES, 'Book One/Places/Wall.md': note('place', 'Wall', '', [`appears_in: "${huge}"`]) }));
|
||||
await assert.rejects(find(s, { universe: 'harbor', query: 'Wall' }), { code: 'query-item-too-large' });
|
||||
});
|
||||
|
||||
test('get pages its lines, and the next line picks up where the page stopped', async t => {
|
||||
const body = `${Array.from({ length: 500 }, (_, i) => `Line ${i + 1} ${'z'.repeat(120)}`).join('\n')}\n`;
|
||||
const s = src(scratchRepo(t, { ...FILES, 'Book One/Chapters/Long.md': body }));
|
||||
const first = await get(s, { universe: 'harbor', path: 'Book One/Chapters/Long.md' });
|
||||
assert.equal(first.lines[0][0], 1);
|
||||
assert.ok(first.lines.length < 501);
|
||||
assert.notEqual(first.next, null);
|
||||
let from = 1, count = 0;
|
||||
for (let pages = 0; pages < 10; pages++) {
|
||||
const page = await get(s, { universe: 'harbor', path: 'Book One/Chapters/Long.md', from });
|
||||
count += page.lines.length;
|
||||
if (page.next === null) break;
|
||||
from = page.next;
|
||||
}
|
||||
assert.equal(count, 501);
|
||||
});
|
||||
|
||||
test('get refuses a file whose front matter alone is over the cap, and a line alone is over the cap', async t => {
|
||||
const huge = 'q'.repeat(QUERY_REPLY_CAP + 1000);
|
||||
const s = src(scratchRepo(t, {
|
||||
...FILES,
|
||||
'Book One/Places/Heavy.md': note('place', 'Heavy', '', [`appears_in: "${huge}"`]),
|
||||
'Book One/Chapters/Wide.md': `${huge}\n`,
|
||||
}));
|
||||
await assert.rejects(get(s, { universe: 'harbor', path: 'Book One/Places/Heavy.md' }), { code: 'query-item-too-large' });
|
||||
await assert.rejects(get(s, { universe: 'harbor', path: 'Book One/Chapters/Wide.md' }), { code: 'query-item-too-large' });
|
||||
});
|
||||
|
||||
test('get pages the links of a hub note, every page carries the front matter, and the pages add up', async t => {
|
||||
const files = { ...FILES };
|
||||
for (let i = 1; i <= 900; i++) files[`Book One/Chapters/Mention ${i}.md`] = `[[Tomas]] ${'w'.repeat(60)}\n`;
|
||||
const s = src(scratchRepo(t, files));
|
||||
const seen = [];
|
||||
const PAST_END = 1000;
|
||||
let linksFrom = 0, pages = 0, lines = 0, from = 1;
|
||||
for (; pages < 40; pages++) {
|
||||
const page = await get(s, { universe: 'harbor', path: 'Book One/Characters/Tomas.md', from, links_from: linksFrom });
|
||||
assert.ok(JSON.stringify(page).length <= QUERY_REPLY_CAP);
|
||||
assert.equal(page.frontmatter.name, 'Tomas');
|
||||
seen.push(...page.links_in.map(l => l.path));
|
||||
if (from !== PAST_END) lines += page.lines.length;
|
||||
// Once the lines are read, ask past the last one: the page then carries links alone.
|
||||
from = page.next ?? PAST_END;
|
||||
if (page.links_next === null && page.next === null) break;
|
||||
if (page.links_next !== null) linksFrom = page.links_next;
|
||||
}
|
||||
assert.ok(pages > 1, 'the hub needed more than one page');
|
||||
assert.ok(pages < 40);
|
||||
assert.equal(seen.filter(p => p.startsWith('Book One/Chapters/Mention ')).length, 900);
|
||||
assert.equal(new Set(seen).size, seen.length, 'no link comes back twice');
|
||||
assert.equal(lines, 6, 'the five lines of the note and the empty one after its last newline');
|
||||
const tail = await get(s, { universe: 'harbor', path: 'Book One/Characters/Tomas.md', from: PAST_END });
|
||||
assert.deepEqual(tail.lines, []);
|
||||
assert.equal(tail.next, null);
|
||||
});
|
||||
|
||||
test('get rejects a bad links_from', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
await assert.rejects(get(s, { universe: 'harbor', path: 'Book One/Characters/Tomas.md', links_from: -1 }), { code: 'query-argument-invalid' });
|
||||
const past = await get(s, { universe: 'harbor', path: 'Book One/Characters/Tomas.md', links_from: 99 });
|
||||
assert.deepEqual([past.links_out, past.links_in, past.links_next], [[], [], null]);
|
||||
});
|
||||
|
||||
test('search returns scoped line hits, case-insensitively, and refuses empty text', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
const reply = await search(s, { universe: 'harbor', text: 'marker' });
|
||||
assert.ok(reply.results.length > 0);
|
||||
assert.ok(reply.results.every(r => r.scope === 'canon' && r.path.startsWith('Book One/') && Number.isInteger(r.line)));
|
||||
assert.ok(reply.results.some(r => r.path === 'Book One/Chapters/Chapter 01.md' && r.line === 2));
|
||||
await assert.rejects(search(s, { universe: 'harbor', text: '' }), { code: 'query-argument-invalid' });
|
||||
});
|
||||
|
||||
test('a manuscript call returns one chapter, at most 400 lines, and names its next line', async t => {
|
||||
const long = `${Array.from({ length: 900 }, (_, i) => `Scene ${i + 1}`).join('\n')}\n`;
|
||||
const s = src(scratchRepo(t, { ...FILES, 'Book One/Chapters/Long Chapter.md': long }));
|
||||
const first = await manuscript(s, { universe: 'harbor', book: 'Book One', chapter: 'Long Chapter' });
|
||||
assert.equal(first.lines.length, MANUSCRIPT_LINES);
|
||||
assert.equal(first.next, MANUSCRIPT_LINES + 1);
|
||||
const second = await manuscript(s, { universe: 'harbor', book: 'Book One', chapter: 'Long Chapter', from: first.next });
|
||||
assert.equal(second.lines[0][0], MANUSCRIPT_LINES + 1);
|
||||
});
|
||||
|
||||
test('a manuscript call without a chapter returns the first chapter in path order and names the next', async t => {
|
||||
const s = src(scratchRepo(t, FILES));
|
||||
const first = await manuscript(s, { universe: 'harbor', book: 'Book One' });
|
||||
assert.equal(first.path, 'Book One/Chapters/Chapter 01.md');
|
||||
assert.equal(first.next_chapter, 'Chapter 02');
|
||||
const last = await manuscript(s, { universe: 'harbor', book: 'Book One', chapter: 'Chapter 02' });
|
||||
assert.equal(last.path, 'Book One/Chapters/Chapter 02.md');
|
||||
assert.equal(last.next_chapter, null);
|
||||
assert.equal(await manuscript(s, { universe: 'harbor', book: 'Book One', chapter: 'Nope' }), null);
|
||||
});
|
||||
|
||||
test('a chapter name that matches two visible files is refused', async t => {
|
||||
const s = src(scratchRepo(t, { ...FILES, 'Book One/Chapters/Part/Chapter 01.md': 'Again.\n' }));
|
||||
await assert.rejects(manuscript(s, { universe: 'harbor', book: 'Book One', chapter: 'Chapter 01' }), { code: 'query-chapter-ambiguous' });
|
||||
});
|
||||
|
||||
test('a hybrid reads a source chapter through its declared sources only', async t => {
|
||||
const ref = scratchRepo(t, FILES);
|
||||
const hybrid = await manuscript(src(ref, mapping(WITH_BLEND)), { universe: 'blend', book: 'Second World', chapter: '01' });
|
||||
assert.equal(hybrid.path, 'Second World/Chapters/01.md');
|
||||
assert.equal(hybrid.scope, 'canon');
|
||||
assert.equal(await manuscript(src(ref), { universe: 'harbor', book: 'Second World', chapter: '01' }), null);
|
||||
});
|
||||
|
||||
test('universe info reports a hybrid sources, constraints and notes, and refuses an unknown universe', async t => {
|
||||
const s = src(scratchRepo(t, FILES), mapping(WITH_BLEND));
|
||||
const blend = await universeInfo(s, { universe: 'blend' });
|
||||
assert.equal(blend.kind, 'hybrid');
|
||||
assert.deepEqual(blend.sources, ['harbor', 'second']);
|
||||
assert.deepEqual(blend.constraints, [EXCLUSION]);
|
||||
assert.equal(blend.notes, 'made-up hybrid');
|
||||
const harbor = await universeInfo(s, { universe: 'harbor' });
|
||||
assert.equal(harbor.kind, 'universe');
|
||||
assert.equal(harbor.sources, undefined);
|
||||
await assert.rejects(universeInfo(s, { universe: 'ghost' }), { code: 'query-universe-unknown' });
|
||||
});
|
||||
|
||||
test('an exclusion hides what it names from a hybrid and from nothing else', async t => {
|
||||
const ref = scratchRepo(t, FILES);
|
||||
const hybrid = src(ref, mapping(WITH_BLEND));
|
||||
const seen = paths(await find(hybrid, { universe: 'blend', query: '' }));
|
||||
for (const path of EXCLUDED_PATHS) assert.ok(!seen.includes(path), `${path} stays hidden`);
|
||||
const own = paths(await find(src(ref), { universe: 'harbor', query: '' }));
|
||||
for (const path of EXCLUDED_PATHS) assert.ok(own.includes(path), `${path} stays visible to its own universe`);
|
||||
});
|
||||
|
||||
test('PRD 1.14: a lookup that names a book leaves book-null files out', async t => {
|
||||
const s = src(scratchRepo(t, { ...FILES, ...LEVEL_FILES }), mapping(WITH_LEVELS));
|
||||
const all = paths(await find(s, { universe: 'harbor', query: '', scope: 'playground' }));
|
||||
assert.deepEqual(all.sort(), ['Book One/Playground/What If.md', 'Notes/Harbor/Chapters/Idea Chapter.md', 'Notes/Harbor/Scraps.md']);
|
||||
const booked = await find(s, { universe: 'harbor', query: '', scope: 'playground', book: 'Book One' });
|
||||
assert.deepEqual(paths(booked), ['Book One/Playground/What If.md']);
|
||||
assert.ok(booked.results.every(r => r.book === 'Book One'));
|
||||
const hits = (await search(s, { universe: 'harbor', text: 'unbooked', scope: 'playground' })).results;
|
||||
assert.deepEqual(hits.map(r => r.path), ['Notes/Harbor/Chapters/Idea Chapter.md']);
|
||||
assert.deepEqual((await search(s, { universe: 'harbor', text: 'unbooked', scope: 'playground', book: 'Book One' })).results, []);
|
||||
});
|
||||
|
||||
test('PRD 1.14: library_manuscript never reads a book-null file', async t => {
|
||||
const s = src(scratchRepo(t, { ...FILES, ...LEVEL_FILES }), mapping(WITH_LEVELS));
|
||||
// The unbooked file sits under a manuscript folder, so it has type manuscript and book null.
|
||||
const listed = await find(s, { universe: 'harbor', query: 'Idea', scope: 'playground' });
|
||||
assert.deepEqual(listed.results.map(r => [r.path, r.type, r.book]), [['Notes/Harbor/Chapters/Idea Chapter.md', 'manuscript', null]]);
|
||||
assert.equal(await manuscript(s, { universe: 'harbor', book: 'Book One', scope: 'playground', chapter: 'Idea Chapter' }), null);
|
||||
assert.equal(await manuscript(s, { universe: 'harbor', book: 'Book One', scope: 'playground' }), null);
|
||||
assert.equal(await manuscript(s, { universe: 'harbor', book: 'Notes', scope: 'playground', chapter: 'Idea Chapter' }), null);
|
||||
});
|
||||
|
||||
test('PRD 1.14: a draft universe has no canon, so its lookups return files only with scope playground', async t => {
|
||||
const s = src(scratchRepo(t, { ...FILES, ...LEVEL_FILES }), mapping(WITH_LEVELS));
|
||||
assert.deepEqual((await find(s, { universe: 'idea', query: '' })).results, []);
|
||||
assert.deepEqual((await search(s, { universe: 'idea', text: MARKER })).results, []);
|
||||
const found = await find(s, { universe: 'idea', query: '', scope: 'playground' });
|
||||
assert.deepEqual(paths(found).sort(), ['Drafts/New World/Characters/Rin.md', 'Drafts/New World/Summary.md']);
|
||||
assert.ok(found.results.every(r => r.scope === 'playground' && r.universe === 'idea' && r.book === null));
|
||||
const hits = (await search(s, { universe: 'idea', text: 'draft', scope: 'playground' })).results;
|
||||
assert.equal(hits.length, 2);
|
||||
const rin = await get(s, { universe: 'idea', path: 'Drafts/New World/Characters/Rin.md' });
|
||||
assert.equal(rin.scope, 'playground');
|
||||
assert.equal(await get(s, { universe: 'harbor', path: 'Drafts/New World/Characters/Rin.md' }), null);
|
||||
});
|
||||
|
||||
test('PRD 1.14: library_universe on a draft returns its kind, name and prefixes', async t => {
|
||||
const s = src(scratchRepo(t, { ...FILES, ...LEVEL_FILES }), mapping(WITH_LEVELS));
|
||||
const draft = await universeInfo(s, { universe: 'idea' });
|
||||
assert.equal(draft.kind, 'draft');
|
||||
assert.equal(draft.name, 'New World');
|
||||
assert.deepEqual(draft.playground, ['Drafts/New World/']);
|
||||
assert.deepEqual(draft.books, []);
|
||||
assert.equal(draft.sources, undefined);
|
||||
});
|
||||
|
||||
test('a hybrid gets a source playground file as null, while the source gets it with scope playground', async t => {
|
||||
const ref = scratchRepo(t, FILES);
|
||||
const path = 'Book One/Playground/What If.md';
|
||||
assert.equal(await get(src(ref, mapping(WITH_BLEND)), { universe: 'blend', path }), null);
|
||||
const own = await get(src(ref), { universe: 'harbor', path });
|
||||
assert.equal(own.scope, 'playground');
|
||||
});
|
||||
@@ -0,0 +1,59 @@
|
||||
# Mail records — schemas only (email plan row E01)
|
||||
|
||||
Post-v1. This package holds the record schemas for mail intake and access,
|
||||
and a validator. Nothing here opens a connection, reads a secret or
|
||||
registers a plug-in kind. Core `mail-source` registration waits for the
|
||||
approved PRD version and the source, grant and route approvals. The addendum draft is
|
||||
`docs/prd/mail-addendum-draft.md`; it binds nothing until Jason approves it.
|
||||
|
||||
| Schema | Record |
|
||||
| --- | --- |
|
||||
| `schemas/source.schema.json` | Per-source settings: enabled, adapter, `mailboxScope`, wake, limits |
|
||||
| `schemas/data.schema.json` | Per-business content and retention settings |
|
||||
| `schemas/access.schema.json` | A `mail.access` grant to one agent or user |
|
||||
| `schemas/service.schema.json` | A source's service principal and its slot path |
|
||||
| `schemas/content-key.schema.json` | A content key's location and lifecycle |
|
||||
|
||||
## Settings (DECISIONS Q222)
|
||||
|
||||
Every limit, scope and retention time is a setting. Its default and its
|
||||
range sit in the schema, so they have one definition. `defaults(kind)`
|
||||
reads them. Implementers and reviewers set the range; widening it is a
|
||||
reviewed schema and test change. A settings record carries
|
||||
`changedBy.kind: "human"`, and the validator refuses any other author.
|
||||
|
||||
Defaults are off and empty: `sourceEnabled: false`, `mailboxScope: []`,
|
||||
`sourceMutations: []`, `accessMode: "manual"`, `unattendedWake: false`,
|
||||
`wakeSenderAllowlist: []`, `persistContent: false` and
|
||||
`automaticPrune: false`. `persistContent` and `automaticPrune` are fixed
|
||||
false until retention authority exists (E13).
|
||||
|
||||
## Checks
|
||||
|
||||
- `validateRecord(kind, value)` returns a list of problems. It refuses
|
||||
unknown keys, wildcards, secret-shaped values, values outside the
|
||||
schema range and incomplete scopes. A refusal never echoes the value.
|
||||
- `checkBinding({source, service, access})` refuses records that name a
|
||||
different business, source or provider, and a grant past the source's
|
||||
mailbox scope.
|
||||
- `effectiveMailboxes(...)` intersects the source scope, the provider's
|
||||
mailboxes and the grant, from `issuedAt` up to `expiresAt`. The
|
||||
defaults read nothing. An invalid record or clock reads nothing.
|
||||
Principal, right, manual-mode and revoke checks belong to E02.
|
||||
- `checkSlotLocation` and `checkKeyLocation` keep provider slots and
|
||||
content keys under `<configDir>/secrets/mosaic-stack/`, outside
|
||||
dataRoot, the repository and any backup root. They compare the paths
|
||||
as written and again after resolving symbolic links, so a linked
|
||||
configDir or an aliased root can't hide a secret inside a forbidden root.
|
||||
- `checkProtectedFile` wants a regular file under configDir, owned, mode
|
||||
600, with no symbolic link from configDir itself down to the file. Given
|
||||
the forbidden roots, it compares resolved paths against them too.
|
||||
- `keyUse` gives active keys encrypt and decrypt, retired keys decrypt
|
||||
only, and expired keys nothing. An invalid record or clock, or a time
|
||||
outside `createdAt` to `expiresAt`, gives nothing.
|
||||
- `keyTransitionAllowed` checks one state edge: active to retired or
|
||||
expired, retired to expired. It is a pure predicate. It doesn't persist
|
||||
state, stop a restored backup from rolling a record back, or destroy
|
||||
key copies. E03 owns that proof.
|
||||
|
||||
Tests: `node --test tests/*.test.mjs` from this directory.
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"name": "@mosaic/mail",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"license": "UNLICENSED",
|
||||
"exports": "./src/validate.mjs",
|
||||
"scripts": {
|
||||
"test": "node --test tests/*.test.mjs"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "mosaic:mail:1:access",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"recordVersion",
|
||||
"kind",
|
||||
"business",
|
||||
"source",
|
||||
"principal",
|
||||
"rights",
|
||||
"mailboxes",
|
||||
"mode",
|
||||
"approvalId",
|
||||
"issuedAt",
|
||||
"expiresAt",
|
||||
"revision"
|
||||
],
|
||||
"properties": {
|
||||
"recordVersion": {
|
||||
"const": 1
|
||||
},
|
||||
"kind": {
|
||||
"const": "mail-access"
|
||||
},
|
||||
"business": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"source": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"principal": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"kind",
|
||||
"id"
|
||||
],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"enum": [
|
||||
"agent",
|
||||
"user"
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"rights": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 4,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"metadata",
|
||||
"body",
|
||||
"attachment",
|
||||
"draft"
|
||||
]
|
||||
}
|
||||
},
|
||||
"mailboxes": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 100,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9._@+-]{0,253}$"
|
||||
}
|
||||
},
|
||||
"mode": {
|
||||
"enum": [
|
||||
"manual",
|
||||
"grant"
|
||||
]
|
||||
},
|
||||
"approvalId": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$"
|
||||
},
|
||||
"issuedAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"expiresAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 2147483647
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "mosaic:mail:1:content-key",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"recordVersion",
|
||||
"kind",
|
||||
"business",
|
||||
"keyId",
|
||||
"path",
|
||||
"state",
|
||||
"createdAt",
|
||||
"expiresAt",
|
||||
"revision"
|
||||
],
|
||||
"properties": {
|
||||
"recordVersion": {
|
||||
"const": 1
|
||||
},
|
||||
"kind": {
|
||||
"const": "mail-content-key"
|
||||
},
|
||||
"business": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"keyId": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"pattern": "^/[^\\u0000]{0,4095}$"
|
||||
},
|
||||
"state": {
|
||||
"enum": [
|
||||
"active",
|
||||
"retired",
|
||||
"expired"
|
||||
]
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"expiresAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 2147483647
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "mosaic:mail:1:data",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"recordVersion",
|
||||
"kind",
|
||||
"business",
|
||||
"persistContent",
|
||||
"bodyDraftDays",
|
||||
"attachmentDays",
|
||||
"metadataDays",
|
||||
"automaticPrune",
|
||||
"revision",
|
||||
"changedBy",
|
||||
"changedAt"
|
||||
],
|
||||
"properties": {
|
||||
"recordVersion": {
|
||||
"const": 1
|
||||
},
|
||||
"kind": {
|
||||
"const": "mail-data-settings"
|
||||
},
|
||||
"business": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"persistContent": {
|
||||
"const": false,
|
||||
"default": false
|
||||
},
|
||||
"bodyDraftDays": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 3650,
|
||||
"default": 30
|
||||
},
|
||||
"attachmentDays": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 3650,
|
||||
"default": 7
|
||||
},
|
||||
"metadataDays": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 3650,
|
||||
"default": 365
|
||||
},
|
||||
"automaticPrune": {
|
||||
"const": false,
|
||||
"default": false
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 2147483647
|
||||
},
|
||||
"changedBy": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"kind",
|
||||
"id"
|
||||
],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"const": "human"
|
||||
},
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"changedAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "mosaic:mail:1:service",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"recordVersion",
|
||||
"kind",
|
||||
"business",
|
||||
"source",
|
||||
"provider",
|
||||
"mailbox",
|
||||
"scopes",
|
||||
"slotPath",
|
||||
"rotateBy",
|
||||
"approvalId",
|
||||
"revision"
|
||||
],
|
||||
"properties": {
|
||||
"recordVersion": {
|
||||
"const": 1
|
||||
},
|
||||
"kind": {
|
||||
"const": "mail-service"
|
||||
},
|
||||
"business": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"source": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"provider": {
|
||||
"enum": [
|
||||
"m365",
|
||||
"imap"
|
||||
]
|
||||
},
|
||||
"mailbox": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9._@+-]{0,253}$"
|
||||
},
|
||||
"scopes": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"mail.read"
|
||||
]
|
||||
}
|
||||
},
|
||||
"slotPath": {
|
||||
"type": "string",
|
||||
"pattern": "^/[^\\u0000]{0,4095}$"
|
||||
},
|
||||
"rotateBy": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"approvalId": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$"
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 2147483647
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "mosaic:mail:1:source",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"recordVersion",
|
||||
"kind",
|
||||
"business",
|
||||
"source",
|
||||
"provider",
|
||||
"sourceEnabled",
|
||||
"sourceAdapter",
|
||||
"mailboxScope",
|
||||
"sourceMutations",
|
||||
"accessMode",
|
||||
"unattendedWake",
|
||||
"wakeSenderAllowlist",
|
||||
"limits",
|
||||
"revision",
|
||||
"changedBy",
|
||||
"changedAt"
|
||||
],
|
||||
"properties": {
|
||||
"recordVersion": {
|
||||
"const": 1
|
||||
},
|
||||
"kind": {
|
||||
"const": "mail-source-settings"
|
||||
},
|
||||
"business": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"source": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
},
|
||||
"provider": {
|
||||
"enum": [
|
||||
"m365",
|
||||
"imap"
|
||||
]
|
||||
},
|
||||
"sourceEnabled": {
|
||||
"type": "boolean",
|
||||
"default": false
|
||||
},
|
||||
"sourceAdapter": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"pattern": "^[a-z][a-z0-9-]{0,63}$",
|
||||
"default": null
|
||||
},
|
||||
"mailboxScope": {
|
||||
"type": "array",
|
||||
"maxItems": 100,
|
||||
"uniqueItems": true,
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9._@+-]{0,253}$"
|
||||
}
|
||||
},
|
||||
"sourceMutations": {
|
||||
"type": "array",
|
||||
"maxItems": 0,
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"accessMode": {
|
||||
"enum": [
|
||||
"manual",
|
||||
"grant"
|
||||
],
|
||||
"default": "manual"
|
||||
},
|
||||
"unattendedWake": {
|
||||
"type": "boolean",
|
||||
"default": false
|
||||
},
|
||||
"wakeSenderAllowlist": {
|
||||
"type": "array",
|
||||
"maxItems": 100,
|
||||
"uniqueItems": true,
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._+-]{0,63}@[a-z0-9][a-z0-9.-]{0,252}$"
|
||||
}
|
||||
},
|
||||
"limits": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"pollSeconds",
|
||||
"sliceObservations",
|
||||
"sliceBytes",
|
||||
"pendingDeliveries",
|
||||
"retainedBytes",
|
||||
"parserInputBytes",
|
||||
"mimeDepth",
|
||||
"mimeParts",
|
||||
"normalizedTextBytes",
|
||||
"attachmentPartBytes",
|
||||
"attachmentMessageBytes",
|
||||
"staleScanIntervals",
|
||||
"pageSize"
|
||||
],
|
||||
"properties": {
|
||||
"pollSeconds": {
|
||||
"type": "integer",
|
||||
"minimum": 15,
|
||||
"maximum": 3600,
|
||||
"default": 60
|
||||
},
|
||||
"sliceObservations": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 500,
|
||||
"default": 50
|
||||
},
|
||||
"sliceBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 65536,
|
||||
"maximum": 67108864,
|
||||
"default": 10485760
|
||||
},
|
||||
"pendingDeliveries": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100000,
|
||||
"default": 1000
|
||||
},
|
||||
"retainedBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 1048576,
|
||||
"maximum": 10737418240,
|
||||
"default": 104857600
|
||||
},
|
||||
"parserInputBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 1048576,
|
||||
"maximum": 67108864,
|
||||
"default": 31457280
|
||||
},
|
||||
"mimeDepth": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 32,
|
||||
"default": 10
|
||||
},
|
||||
"mimeParts": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 1000,
|
||||
"default": 100
|
||||
},
|
||||
"normalizedTextBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 4096,
|
||||
"maximum": 4194304,
|
||||
"default": 262144
|
||||
},
|
||||
"attachmentPartBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 52428800,
|
||||
"default": 10485760
|
||||
},
|
||||
"attachmentMessageBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 104857600,
|
||||
"default": 10485760
|
||||
},
|
||||
"staleScanIntervals": {
|
||||
"type": "integer",
|
||||
"minimum": 2,
|
||||
"maximum": 20,
|
||||
"default": 3
|
||||
},
|
||||
"pageSize": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100,
|
||||
"default": 20
|
||||
}
|
||||
}
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 2147483647
|
||||
},
|
||||
"changedBy": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"kind",
|
||||
"id"
|
||||
],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"const": "human"
|
||||
},
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"changedAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,237 @@
|
||||
import { lstatSync, readFileSync, realpathSync } from 'node:fs';
|
||||
import { basename, dirname, isAbsolute, join, normalize, relative, sep } from 'node:path';
|
||||
|
||||
// Mail record schemas (email plan row E01). Nothing here reads a secret, opens a connection or
|
||||
// registers a plug-in kind: core mail-source registration waits for the approved PRD version and the
|
||||
// source, grant and route approvals.
|
||||
export const KINDS = Object.freeze(['source', 'data', 'access', 'service', 'content-key']);
|
||||
const KEYWORDS = new Set(['$schema', '$id', 'type', 'const', 'enum', 'properties', 'required',
|
||||
'additionalProperties', 'items', 'minItems', 'maxItems', 'uniqueItems', 'minLength', 'maxLength',
|
||||
'minimum', 'maximum', 'pattern', 'format', 'default']);
|
||||
const schemas = Object.fromEntries(KINDS.map(kind => [kind,
|
||||
JSON.parse(readFileSync(new URL(`../schemas/${kind}.schema.json`, import.meta.url), 'utf8')),
|
||||
]));
|
||||
// Package-owned schemas use this deliberately small closed subset. Never silently ignore a new constraint.
|
||||
function known(schema) {
|
||||
for (const key of Object.keys(schema)) if (!KEYWORDS.has(key)) throw Error(`unsupported schema keyword: ${key}`);
|
||||
for (const child of Object.values(schema.properties ?? {})) known(child);
|
||||
if (schema.items) known(schema.items);
|
||||
}
|
||||
for (const schema of Object.values(schemas)) known(schema);
|
||||
|
||||
const isObject = value => value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
function visit(schema, value, path, problems) {
|
||||
const bad = text => problems.push(`${path}: ${text}`);
|
||||
if (Object.hasOwn(schema, 'const') && value !== schema.const) bad(`must equal ${JSON.stringify(schema.const)}`);
|
||||
if (schema.enum && !schema.enum.includes(value)) bad(`must be one of ${schema.enum.join(', ')}`);
|
||||
const types = schema.type ? [schema.type].flat() : [];
|
||||
const matches = type => type === 'null' ? value === null : type === 'object' ? isObject(value)
|
||||
: type === 'array' ? Array.isArray(value) : type === 'integer' ? Number.isSafeInteger(value)
|
||||
: typeof value === type;
|
||||
if (types.length && !types.some(matches)) { bad(`must be ${types.join(' or ')}`); return; }
|
||||
if (typeof value === 'number') {
|
||||
if (schema.minimum !== undefined && value < schema.minimum) bad(`below the schema bound ${schema.minimum}`);
|
||||
if (schema.maximum !== undefined && value > schema.maximum) bad(`above the schema bound ${schema.maximum}`);
|
||||
}
|
||||
if (typeof value === 'string') {
|
||||
if (schema.minLength !== undefined && value.length < schema.minLength) bad('too short');
|
||||
if (schema.maxLength !== undefined && value.length > schema.maxLength) bad('too long');
|
||||
if (schema.pattern && !new RegExp(schema.pattern).test(value)) bad('invalid format');
|
||||
if (schema.format === 'date-time' && (!Number.isFinite(Date.parse(value)) ||
|
||||
new Date(value).toISOString() !== value || value.length !== 24)) bad('invalid canonical UTC date-time');
|
||||
if (value.includes('\0')) bad('NUL is forbidden');
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
if (schema.minItems !== undefined && value.length < schema.minItems) bad('too few items');
|
||||
if (schema.maxItems !== undefined && value.length > schema.maxItems) bad('too many items');
|
||||
if (schema.uniqueItems && new Set(value.map(item => JSON.stringify(item))).size !== value.length) bad('duplicate items');
|
||||
if (schema.items) value.forEach((item, index) => visit(schema.items, item, `${path}[${index}]`, problems));
|
||||
}
|
||||
if (isObject(value)) {
|
||||
for (const key of schema.required ?? []) if (!Object.hasOwn(value, key)) bad(`missing ${key}`);
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (Object.hasOwn(schema.properties ?? {}, key)) visit(schema.properties[key], child, `${path}.${key}`, problems);
|
||||
else if (schema.additionalProperties === false) bad(`unknown key ${key}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Wildcards are refused by every pattern; this names the reason plainly. Scopes are exact IDs only.
|
||||
const WILDCARD = /[*?%]/;
|
||||
// Shapes of secret values. A record names a protected slot path, never a credential (invariant 3).
|
||||
const SECRET_SHAPES = [
|
||||
/-----BEGIN [A-Z ]*PRIVATE KEY-----/,
|
||||
/\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\./,
|
||||
/\b(?:gh[pousr]_|glpat-|xox[abprs]-|sk-)[A-Za-z0-9_-]{8,}/,
|
||||
/:\/\/[^/\s:@]+:[^/\s@]+@/,
|
||||
/\b(?:password|passwd|secret|token|apikey|api_key|client_secret)\s*[=:]/i,
|
||||
];
|
||||
export const looksSecret = text => SECRET_SHAPES.some(shape => shape.test(text));
|
||||
function strings(value, path, out) {
|
||||
if (typeof value === 'string') out.push([path, value]);
|
||||
else if (Array.isArray(value)) value.forEach((item, index) => strings(item, `${path}[${index}]`, out));
|
||||
else if (isObject(value)) for (const [key, child] of Object.entries(value)) strings(child, `${path}.${key}`, out);
|
||||
return out;
|
||||
}
|
||||
|
||||
export function validateRecord(kind, value) {
|
||||
if (!Object.hasOwn(schemas, kind)) return [`unknown record kind: ${kind}`];
|
||||
const problems = [];
|
||||
visit(schemas[kind], value, kind, problems);
|
||||
for (const [path, text] of strings(value, kind, [])) {
|
||||
if (looksSecret(text)) problems.push(`${path}: looks like a secret value; name a protected slot instead`);
|
||||
else if (WILDCARD.test(text)) problems.push(`${path}: wildcard not allowed; scopes name exact IDs`);
|
||||
}
|
||||
if (problems.length) return problems;
|
||||
if (kind === 'source') {
|
||||
if (value.sourceEnabled && value.sourceAdapter === null)
|
||||
problems.push('source.sourceAdapter: an enabled source needs an enrolled adapter');
|
||||
if (value.unattendedWake && value.wakeSenderAllowlist.length === 0)
|
||||
problems.push('source.wakeSenderAllowlist: unattended wake needs at least one exact sender');
|
||||
if (value.limits.attachmentPartBytes > value.limits.attachmentMessageBytes)
|
||||
problems.push('source.limits: attachmentPartBytes exceeds attachmentMessageBytes');
|
||||
}
|
||||
if (kind === 'data' && value.metadataDays < Math.max(value.bodyDraftDays, value.attachmentDays))
|
||||
problems.push('data.metadataDays: metadata and tombstones must outlive content');
|
||||
if (kind === 'access' && Date.parse(value.expiresAt) <= Date.parse(value.issuedAt))
|
||||
problems.push('access.expiresAt: must be after issuedAt');
|
||||
if (kind === 'content-key' && Date.parse(value.expiresAt) <= Date.parse(value.createdAt))
|
||||
problems.push('content-key.expiresAt: must be after createdAt');
|
||||
return problems;
|
||||
}
|
||||
|
||||
// Defaults come from the schema, so a default and its bounds have one definition (Q222).
|
||||
function defaultsOf(schema) {
|
||||
if (Object.hasOwn(schema, 'default')) return structuredClone(schema.default);
|
||||
if (schema.type !== 'object') return undefined;
|
||||
const out = {};
|
||||
for (const [key, child] of Object.entries(schema.properties ?? {})) {
|
||||
const value = defaultsOf(child);
|
||||
if (value !== undefined) out[key] = value;
|
||||
}
|
||||
return Object.keys(out).length ? out : undefined;
|
||||
}
|
||||
export function defaults(kind) {
|
||||
if (kind !== 'source' && kind !== 'data') throw Error(`no settings defaults for ${kind}`);
|
||||
return defaultsOf(schemas[kind]);
|
||||
}
|
||||
export const schemaBounds = (kind, key) => {
|
||||
const node = key.split('.').reduce((schema, part) => schema?.properties?.[part], schemas[kind]);
|
||||
if (!node || node.minimum === undefined) throw Error(`no bounds for ${kind}.${key}`);
|
||||
return { minimum: node.minimum, maximum: node.maximum, default: node.default };
|
||||
};
|
||||
|
||||
// Records that name one another must agree, and a grant never reaches past the source's mailbox scope.
|
||||
// (The access schema already limits a grant's principal to an agent or a user, never the service.)
|
||||
export function checkBinding({ source, service, access }) {
|
||||
const problems = [];
|
||||
const same = (label, a, b) => { for (const key of ['business', 'source']) if (a[key] !== b[key]) problems.push(`${label}: ${key} mismatch`); };
|
||||
if (service) {
|
||||
same('service', service, source);
|
||||
if (service.provider !== source.provider) problems.push('service: provider mismatch');
|
||||
}
|
||||
if (access) {
|
||||
same('access', access, source);
|
||||
const scope = new Set(source.mailboxScope);
|
||||
for (const mailbox of access.mailboxes) if (!scope.has(mailbox)) problems.push(`access: mailbox ${mailbox} outside the source scope`);
|
||||
}
|
||||
return problems;
|
||||
}
|
||||
|
||||
const instant = now => (now instanceof Date && Number.isFinite(now.getTime()) ? now.getTime() : NaN);
|
||||
|
||||
// What a principal may read: the source scope, the provider's mailboxes and the grant, intersected.
|
||||
// A disabled source, the default empty scope, a missing, invalid, not-yet-issued or expired grant, or an
|
||||
// invalid clock reads nothing. Authenticated principal, right, manual-mode and revoke checks belong to E02.
|
||||
export function effectiveMailboxes({ source, providerMailboxes = [], access = null, now = new Date() }) {
|
||||
if (!access || validateRecord('source', source).length || validateRecord('access', access).length) return [];
|
||||
if (!source.sourceEnabled || !Array.isArray(providerMailboxes)) return [];
|
||||
const at = instant(now);
|
||||
if (!(at >= Date.parse(access.issuedAt) && at < Date.parse(access.expiresAt))) return [];
|
||||
if (access.business !== source.business || access.source !== source.source) return [];
|
||||
const provider = new Set(providerMailboxes);
|
||||
const granted = new Set(access.mailboxes);
|
||||
return source.mailboxScope.filter(mailbox => provider.has(mailbox) && granted.has(mailbox));
|
||||
}
|
||||
|
||||
function inside(root, path) {
|
||||
const rel = relative(root, path);
|
||||
return rel === '' || (!rel.startsWith(`..${sep}`) && rel !== '..' && !isAbsolute(rel));
|
||||
}
|
||||
// The path with every existing part resolved, so a symbolic link or alias can't hide where it lands.
|
||||
// Parts that don't exist yet (a planned slot) are kept as written under their nearest existing ancestor.
|
||||
function canonical(path) {
|
||||
const rest = [];
|
||||
for (let at = path; ; at = dirname(at)) {
|
||||
try { return join(realpathSync(at), ...rest.reverse()); } catch (error) {
|
||||
if (error.code !== 'ENOENT' || at === dirname(at)) return null;
|
||||
rest.push(basename(at));
|
||||
}
|
||||
}
|
||||
}
|
||||
// Lexical checks first, for planned paths. Then the same forbidden roots compared after resolution, so
|
||||
// a configDir that links into dataRoot, the repository or a backup root is refused too.
|
||||
function forbidden(label, path, { dataRoot, repoRoot, backupRoots = [] }) {
|
||||
const problems = [];
|
||||
const roots = [['dataRoot', dataRoot], ['the repository', repoRoot], ...backupRoots.map(root => ['a backup root', root])]
|
||||
.filter(([, root]) => root);
|
||||
for (const [name, root] of roots) if (inside(root, path)) problems.push(`${label}: inside ${name}; mail secrets live outside it`);
|
||||
if (problems.length) return problems;
|
||||
const real = canonical(path);
|
||||
if (real === null) return [`${label}: path can't be resolved`];
|
||||
for (const [name, root] of roots) {
|
||||
const realRoot = canonical(root);
|
||||
if (realRoot === null || inside(realRoot, real)) problems.push(`${label}: resolves inside ${name}; mail secrets live outside it`);
|
||||
}
|
||||
return problems;
|
||||
}
|
||||
function placed(label, path, expectedDir, roots) {
|
||||
if (!isAbsolute(path) || normalize(path) !== path) return [`${label}: must be a normalized absolute path`];
|
||||
const problems = !inside(expectedDir, path) || path === expectedDir ? [`${label}: must sit under ${expectedDir}`] : [];
|
||||
return [...problems, ...forbidden(label, path, roots)];
|
||||
}
|
||||
// A source service's provider credential slot (plan D9).
|
||||
export const slotDir = ({ configDir, business, source }) => join(configDir, 'secrets', 'mosaic-stack', 'mail', business, source);
|
||||
export const checkSlotLocation = (service, roots) =>
|
||||
placed('service.slotPath', service.slotPath, slotDir({ ...roots, ...service }), roots);
|
||||
// A content key, outside dataRoot and its backup set, apart from provider slots (plan D7).
|
||||
export const keyPath = ({ configDir, business, keyId }) => join(configDir, 'secrets', 'mosaic-stack', 'mail-content', business, keyId);
|
||||
export function checkKeyLocation(record, roots) {
|
||||
const problems = placed('content-key.path', record.path, join(roots.configDir, 'secrets', 'mosaic-stack', 'mail-content', record.business), roots);
|
||||
if (!problems.length && record.path !== keyPath({ ...roots, ...record })) problems.push(`content-key.path: must be ${keyPath({ ...roots, ...record })}`);
|
||||
return problems;
|
||||
}
|
||||
|
||||
// A protected file: inside configDir, owned, mode 600, regular, with no symbolic link from configDir
|
||||
// itself down to the file, and not resolving into a forbidden root.
|
||||
export function checkProtectedFile(path, { configDir, uid = process.getuid(), ...roots }) {
|
||||
if (!isAbsolute(path) || normalize(path) !== path || !isAbsolute(configDir) || normalize(configDir) !== configDir)
|
||||
return [`${path}: must be a normalized absolute path`];
|
||||
if (!inside(configDir, path) || path === configDir) return [`${path}: must sit under ${configDir}`];
|
||||
let stat;
|
||||
try { stat = lstatSync(path); } catch { return [`${path}: not found`]; }
|
||||
if (stat.isSymbolicLink() || !stat.isFile()) return [`${path}: must be a regular file, not a symbolic link`];
|
||||
for (let at = dirname(path); ; at = dirname(at)) {
|
||||
let dir;
|
||||
try { dir = lstatSync(at); } catch { return [`${path}: path can't be resolved`]; }
|
||||
if (dir.isSymbolicLink()) return [`${path}: a symbolic link sits on the path at ${at}`];
|
||||
if (at === configDir) break;
|
||||
}
|
||||
const problems = [];
|
||||
if (stat.uid !== uid) problems.push(`${path}: owned by uid ${stat.uid}, not ${uid}`);
|
||||
if ((stat.mode & 0o777) !== 0o600) problems.push(`${path}: mode ${(stat.mode & 0o777).toString(8)}; use 600`);
|
||||
return [...problems, ...forbidden(path, path, roots)];
|
||||
}
|
||||
|
||||
// Key lifecycle states: active keys encrypt and decrypt, retired keys only decrypt, expired keys do
|
||||
// neither. keyTransitionAllowed is a pure check on one state edge. It doesn't persist state, stop a
|
||||
// restored backup from rolling a record back, or destroy key copies; E03 owns that proof.
|
||||
const KEY_STEPS = { active: ['retired', 'expired'], retired: ['expired'], expired: [] };
|
||||
export const keyTransitionAllowed = (from, to) => Object.hasOwn(KEY_STEPS, from) && KEY_STEPS[from].includes(to);
|
||||
// The use a record allows now. An invalid record or clock, or a time outside [createdAt, expiresAt), is none.
|
||||
export function keyUse(record, now = new Date()) {
|
||||
if (validateRecord('content-key', record).length) return 'none';
|
||||
const at = instant(now);
|
||||
if (!(at >= Date.parse(record.createdAt) && at < Date.parse(record.expiresAt))) return 'none';
|
||||
return record.state === 'active' ? 'encrypt-decrypt' : record.state === 'retired' ? 'decrypt' : 'none';
|
||||
}
|
||||
@@ -0,0 +1,337 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
checkBinding, checkKeyLocation, checkProtectedFile, checkSlotLocation, defaults, effectiveMailboxes, keyPath,
|
||||
keyTransitionAllowed, keyUse, schemaBounds, slotDir, validateRecord,
|
||||
} from '../src/validate.mjs';
|
||||
|
||||
// Synthetic records only. No credential, mailbox or provider is real, and nothing here opens a connection.
|
||||
const HUMAN = { kind: 'human', id: 'jason' };
|
||||
const AT = '2026-10-22T16:00:00.000Z';
|
||||
const source = (over = {}) => ({
|
||||
recordVersion: 1, kind: 'mail-source-settings', business: 'acme', source: 'ingress', provider: 'imap',
|
||||
...defaults('source'), revision: 1, changedBy: HUMAN, changedAt: AT, ...over,
|
||||
});
|
||||
const data = (over = {}) => ({
|
||||
recordVersion: 1, kind: 'mail-data-settings', business: 'acme', ...defaults('data'),
|
||||
revision: 1, changedBy: HUMAN, changedAt: AT, ...over,
|
||||
});
|
||||
const access = (over = {}) => ({
|
||||
recordVersion: 1, kind: 'mail-access', business: 'acme', source: 'ingress',
|
||||
principal: { kind: 'agent', id: 'mail-specialist' }, rights: ['metadata', 'body'],
|
||||
mailboxes: ['[email protected]'], mode: 'manual', approvalId: 'decision-1',
|
||||
issuedAt: AT, expiresAt: '2026-10-29T16:00:00.000Z', revision: 1, ...over,
|
||||
});
|
||||
const CONFIG = '/home/op/.config/mosaic-acme';
|
||||
const ROOTS = { configDir: CONFIG, dataRoot: '/home/op/.mosaic-acme', repoRoot: '/src/stack', backupRoots: ['/backup/mosaic'] };
|
||||
const service = (over = {}) => ({
|
||||
recordVersion: 1, kind: 'mail-service', business: 'acme', source: 'ingress', provider: 'imap',
|
||||
mailbox: '[email protected]', scopes: ['mail.read'],
|
||||
slotPath: `${CONFIG}/secrets/mosaic-stack/mail/acme/ingress/imap`, rotateBy: '2027-01-01T00:00:00.000Z',
|
||||
approvalId: 'decision-2', revision: 1, ...over,
|
||||
});
|
||||
const contentKey = (over = {}) => ({
|
||||
recordVersion: 1, kind: 'mail-content-key', business: 'acme', keyId: 'k1',
|
||||
path: `${CONFIG}/secrets/mosaic-stack/mail-content/acme/k1`, state: 'active',
|
||||
createdAt: AT, expiresAt: '2027-10-22T16:00:00.000Z', revision: 1, ...over,
|
||||
});
|
||||
const refused = (kind, value, text) => {
|
||||
const problems = validateRecord(kind, value);
|
||||
assert.ok(problems.some(problem => problem.includes(text)), `${kind} should be refused with "${text}": ${JSON.stringify(problems)}`);
|
||||
};
|
||||
|
||||
test('T1 the defaults are off, empty and valid, and every record kind validates', () => {
|
||||
const settings = defaults('source');
|
||||
assert.equal(settings.sourceEnabled, false);
|
||||
assert.deepEqual(settings.mailboxScope, []);
|
||||
assert.deepEqual(settings.sourceMutations, []);
|
||||
assert.equal(settings.sourceAdapter, null);
|
||||
assert.equal(settings.accessMode, 'manual');
|
||||
assert.equal(settings.unattendedWake, false);
|
||||
assert.deepEqual(settings.wakeSenderAllowlist, []);
|
||||
assert.deepEqual(settings.limits, {
|
||||
pollSeconds: 60, sliceObservations: 50, sliceBytes: 10485760, pendingDeliveries: 1000, retainedBytes: 104857600,
|
||||
parserInputBytes: 31457280, mimeDepth: 10, mimeParts: 100, normalizedTextBytes: 262144,
|
||||
attachmentPartBytes: 10485760, attachmentMessageBytes: 10485760, staleScanIntervals: 3, pageSize: 20,
|
||||
});
|
||||
assert.deepEqual(defaults('data'), { persistContent: false, bodyDraftDays: 30, attachmentDays: 7, metadataDays: 365, automaticPrune: false });
|
||||
for (const [kind, value] of [['source', source()], ['data', data()], ['access', access()], ['service', service()], ['content-key', contentKey()]])
|
||||
assert.deepEqual(validateRecord(kind, value), [], kind);
|
||||
assert.throws(() => defaults('access'), /no settings defaults/);
|
||||
assert.deepEqual(validateRecord('mailbox', {}), ['unknown record kind: mailbox']);
|
||||
});
|
||||
|
||||
test('T2 the default scope reads nothing, and a read is the intersection of scope, provider and grant', () => {
|
||||
const grant = access({ mailboxes: ['[email protected]'] });
|
||||
const providerMailboxes = ['[email protected]', '[email protected]'];
|
||||
assert.deepEqual(effectiveMailboxes({ source: source(), providerMailboxes, access: grant }), []);
|
||||
const enabled = source({ sourceEnabled: true, sourceAdapter: 'imap' });
|
||||
assert.deepEqual(effectiveMailboxes({ source: enabled, providerMailboxes, access: grant }), [], 'enabled with the default empty scope');
|
||||
const scoped = source({ sourceEnabled: true, sourceAdapter: 'imap', mailboxScope: ['[email protected]', '[email protected]'] });
|
||||
assert.deepEqual(effectiveMailboxes({ source: scoped, providerMailboxes, access: grant, now: new Date(AT) }), ['[email protected]']);
|
||||
assert.deepEqual(effectiveMailboxes({ source: scoped, providerMailboxes, access: null }), [], 'no grant');
|
||||
assert.deepEqual(effectiveMailboxes({ source: scoped, providerMailboxes: [], access: grant, now: new Date(AT) }), [], 'provider denies');
|
||||
assert.deepEqual(effectiveMailboxes({ source: scoped, providerMailboxes, access: grant, now: new Date('2026-10-29T16:00:00.000Z') }), [], 'expired grant');
|
||||
assert.deepEqual(effectiveMailboxes({ source: scoped, providerMailboxes, access: access({ source: 'other' }), now: new Date(AT) }), [], 'grant for another source');
|
||||
assert.deepEqual(effectiveMailboxes({ source: { ...scoped, sourceEnabled: false }, providerMailboxes, access: grant, now: new Date(AT) }), [], 'disabled');
|
||||
// Lifetime edges: the grant reads from issuedAt up to, not including, expiresAt.
|
||||
const read = (over = {}) => effectiveMailboxes({ source: scoped, providerMailboxes, access: grant, now: new Date(AT), ...over });
|
||||
assert.deepEqual(read({ now: new Date('2026-10-22T15:59:59.999Z') }), [], 'before issuedAt');
|
||||
assert.deepEqual(read({ now: new Date('2026-10-29T15:59:59.999Z') }), ['[email protected]'], 'the last instant before expiresAt');
|
||||
// Malformed records or clock fail closed.
|
||||
assert.deepEqual(read({ now: new Date(Number.NaN) }), [], 'invalid clock');
|
||||
assert.deepEqual(read({ now: AT }), [], 'a clock that is not a Date');
|
||||
assert.deepEqual(read({ access: access({ expiresAt: 'not-a-date' }) }), [], 'invalid expiresAt');
|
||||
assert.deepEqual(read({ access: access({ issuedAt: 'not-a-date' }) }), [], 'invalid issuedAt');
|
||||
assert.deepEqual(read({ access: access({ issuedAt: '2026-10-23T16:00:00.000Z' }) }), [], 'issued in the future');
|
||||
const { mailboxes, ...noMailboxes } = grant;
|
||||
assert.deepEqual(read({ access: noMailboxes }), [], 'grant without mailboxes');
|
||||
assert.deepEqual(read({ source: { ...scoped, mailboxScope: ['*'] } }), [], 'invalid source');
|
||||
assert.deepEqual(read({ providerMailboxes: '[email protected]' }), [], 'provider list that is not an array');
|
||||
});
|
||||
|
||||
test('T3 only a human changes settings', () => {
|
||||
for (const kind of ['source', 'data']) {
|
||||
const make = kind === 'source' ? source : data;
|
||||
refused(kind, make({ changedBy: { kind: 'agent', id: 'mail-specialist' } }), 'changedBy.kind: must equal "human"');
|
||||
refused(kind, make({ changedBy: { kind: 'system', id: 'mail' } }), 'changedBy.kind: must equal "human"');
|
||||
const { changedBy, ...unsigned } = make();
|
||||
refused(kind, unsigned, 'missing changedBy');
|
||||
}
|
||||
});
|
||||
|
||||
test('T4 unknown fields are refused at every level', () => {
|
||||
refused('source', source({ secretRef: 'x' }), 'unknown key secretRef');
|
||||
refused('source', source({ limits: { ...defaults('source').limits, unlimited: 1 } }), 'unknown key unlimited');
|
||||
refused('data', data({ purgeNow: true }), 'unknown key purgeNow');
|
||||
refused('access', access({ principal: { kind: 'agent', id: 'a', asHuman: true } }), 'unknown key asHuman');
|
||||
refused('service', service({ password: 'x' }), 'unknown key password');
|
||||
refused('content-key', contentKey({ material: 'x' }), 'unknown key material');
|
||||
});
|
||||
|
||||
test('T5 wildcards are refused in scopes, grants, allowlists and principals', () => {
|
||||
refused('source', source({ mailboxScope: ['*'] }), 'wildcard not allowed');
|
||||
refused('source', source({ mailboxScope: ['%'] }), 'wildcard not allowed');
|
||||
refused('source', source({ wakeSenderAllowlist: ['*@example.test'] }), 'wildcard not allowed');
|
||||
refused('access', access({ mailboxes: ['[email protected]'] }), 'wildcard not allowed');
|
||||
refused('access', access({ principal: { kind: 'agent', id: '*' } }), 'wildcard not allowed');
|
||||
refused('service', service({ mailbox: '*' }), 'wildcard not allowed');
|
||||
});
|
||||
|
||||
test('T6 secret values are refused wherever they appear', () => {
|
||||
const sentinels = [
|
||||
'-----BEGIN OPENSSH PRIVATE KEY-----', 'eyJhbGciOiJub25lIn0.eyJzdWIiOiJzZW50aW5lbCJ9.x',
|
||||
'ghp_SENTINELSENTINEL', 'imaps://user:[email protected]', 'password=SENTINEL', 'client_secret: SENTINEL',
|
||||
];
|
||||
for (const sentinel of sentinels) {
|
||||
refused('service', service({ slotPath: `${CONFIG}/secrets/mosaic-stack/mail/acme/ingress/${sentinel}` }), 'looks like a secret value');
|
||||
refused('service', service({ approvalId: sentinel }), 'looks like a secret value');
|
||||
refused('access', access({ approvalId: sentinel }), 'looks like a secret value');
|
||||
}
|
||||
const problems = validateRecord('service', service({ approvalId: 'password=SENTINEL' }));
|
||||
assert.ok(!problems.join('\n').includes('SENTINEL'), 'a refusal never echoes the value');
|
||||
});
|
||||
|
||||
test('T7 incomplete scopes are refused', () => {
|
||||
refused('access', access({ rights: [] }), 'access.rights: too few items');
|
||||
refused('access', access({ mailboxes: [] }), 'access.mailboxes: too few items');
|
||||
refused('access', access({ rights: ['metadata', 'send'] }), 'must be one of');
|
||||
refused('access', access({ rights: ['body', 'body'] }), 'duplicate items');
|
||||
refused('access', access({ expiresAt: AT }), 'must be after issuedAt');
|
||||
refused('service', service({ scopes: [] }), 'service.scopes: too few items');
|
||||
refused('service', service({ scopes: ['mail.send'] }), 'must be one of');
|
||||
refused('source', source({ sourceEnabled: true }), 'an enabled source needs an enrolled adapter');
|
||||
refused('source', source({ unattendedWake: true }), 'unattended wake needs at least one exact sender');
|
||||
refused('source', source({ sourceMutations: ['mark-read'] }), 'too many items');
|
||||
refused('source', source({ mailboxScope: ['[email protected]', '[email protected]'] }), 'duplicate items');
|
||||
assert.deepEqual(validateRecord('source', source({ unattendedWake: true, wakeSenderAllowlist: ['[email protected]'] })), []);
|
||||
});
|
||||
|
||||
test('T8 mismatched principal and source are refused', () => {
|
||||
const scoped = source({ mailboxScope: ['[email protected]'] });
|
||||
assert.deepEqual(checkBinding({ source: scoped, service: service(), access: access() }), []);
|
||||
assert.deepEqual(checkBinding({ source: scoped, service: service({ source: 'other' }) }), ['service: source mismatch']);
|
||||
assert.deepEqual(checkBinding({ source: scoped, service: service({ business: 'beta' }) }), ['service: business mismatch']);
|
||||
assert.deepEqual(checkBinding({ source: scoped, service: service({ provider: 'm365' }) }), ['service: provider mismatch']);
|
||||
assert.deepEqual(checkBinding({ source: scoped, access: access({ business: 'beta' }) }), ['access: business mismatch']);
|
||||
assert.deepEqual(checkBinding({ source: scoped, access: access({ mailboxes: ['[email protected]'] }) }),
|
||||
['access: mailbox [email protected] outside the source scope']);
|
||||
assert.deepEqual(checkBinding({ source: source(), access: access() }), ['access: mailbox [email protected] outside the source scope'],
|
||||
'the default empty scope admits no grant');
|
||||
refused('access', access({ principal: { kind: 'service', id: 'mail' } }), 'must be one of agent, user');
|
||||
assert.deepEqual(checkSlotLocation(service({ slotPath: `${CONFIG}/secrets/mosaic-stack/mail/acme/other/imap` }), ROOTS),
|
||||
[`service.slotPath: must sit under ${CONFIG}/secrets/mosaic-stack/mail/acme/ingress`]);
|
||||
});
|
||||
|
||||
test('T9 settings take changed values inside the schema bounds and refuse values outside them', () => {
|
||||
for (const key of ['pollSeconds', 'parserInputBytes', 'mimeDepth', 'mimeParts', 'normalizedTextBytes', 'pageSize']) {
|
||||
const { minimum, maximum } = schemaBounds('source', `limits.${key}`);
|
||||
const at = value => source({ limits: { ...defaults('source').limits, [key]: value } });
|
||||
assert.deepEqual(validateRecord('source', at(minimum)), [], `${key} at its minimum`);
|
||||
assert.deepEqual(validateRecord('source', at(maximum)), [], `${key} at its maximum`);
|
||||
refused('source', at(minimum - 1), 'below the schema bound');
|
||||
refused('source', at(maximum + 1), 'above the schema bound');
|
||||
refused('source', at(minimum + 0.5), 'must be integer');
|
||||
}
|
||||
assert.equal(schemaBounds('source', 'limits.pageSize').maximum, 100);
|
||||
refused('source', source({ limits: { ...defaults('source').limits, attachmentPartBytes: 20971520 } }), 'attachmentPartBytes exceeds attachmentMessageBytes');
|
||||
assert.deepEqual(validateRecord('data', data({ bodyDraftDays: 90, metadataDays: 400 })), []);
|
||||
refused('data', data({ metadataDays: 10 }), 'metadata and tombstones must outlive content');
|
||||
refused('data', data({ persistContent: true }), 'persistContent: must equal false');
|
||||
refused('data', data({ automaticPrune: true }), 'automaticPrune: must equal false');
|
||||
});
|
||||
|
||||
test('T10 slot and key locations stay outside dataRoot, the repository and backups', () => {
|
||||
assert.equal(slotDir({ configDir: CONFIG, business: 'acme', source: 'ingress' }), `${CONFIG}/secrets/mosaic-stack/mail/acme/ingress`);
|
||||
assert.deepEqual(checkSlotLocation(service(), ROOTS), []);
|
||||
assert.deepEqual(checkKeyLocation(contentKey(), ROOTS), []);
|
||||
const nested = { ...ROOTS, dataRoot: CONFIG };
|
||||
assert.ok(checkSlotLocation(service(), nested).some(problem => problem.includes('inside dataRoot')));
|
||||
assert.ok(checkKeyLocation(contentKey(), nested).some(problem => problem.includes('inside dataRoot')));
|
||||
assert.ok(checkKeyLocation(contentKey(), { ...ROOTS, backupRoots: ['/home/op/.config'] }).some(problem => problem.includes('inside a backup root')));
|
||||
assert.ok(checkKeyLocation(contentKey(), { ...ROOTS, repoRoot: CONFIG }).some(problem => problem.includes('inside the repository')));
|
||||
assert.deepEqual(checkKeyLocation(contentKey({ path: '/home/op/.mosaic-acme/mail/acme/k1' }), ROOTS),
|
||||
[`content-key.path: must sit under ${CONFIG}/secrets/mosaic-stack/mail-content/acme`, 'content-key.path: inside dataRoot; mail secrets live outside it']);
|
||||
assert.deepEqual(checkKeyLocation(contentKey({ path: `${CONFIG}/secrets/mosaic-stack/mail-content/acme/k2` }), ROOTS),
|
||||
[`content-key.path: must be ${keyPath({ configDir: CONFIG, business: 'acme', keyId: 'k1' })}`]);
|
||||
assert.ok(checkKeyLocation(contentKey({ path: `${CONFIG}/secrets/mosaic-stack/mail/acme/k1` }), ROOTS)[0].includes('must sit under'), 'apart from provider slots');
|
||||
assert.deepEqual(checkKeyLocation(contentKey({ path: `${CONFIG}/secrets/mosaic-stack/mail-content/acme/../acme/k1` }), ROOTS),
|
||||
['content-key.path: must be a normalized absolute path']);
|
||||
});
|
||||
|
||||
test('T10b a symbolic link or alias can\'t move a slot or key into a forbidden root', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'mail-place-'));
|
||||
try {
|
||||
const dataRoot = join(root, 'data');
|
||||
const roots = { dataRoot, repoRoot: join(root, 'repo'), backupRoots: [join(root, 'backup')] };
|
||||
// configDir is a symbolic link into dataRoot, with a real 0600 key behind it.
|
||||
mkdirSync(join(dataRoot, 'cfg'), { recursive: true, mode: 0o700 });
|
||||
const configDir = join(root, 'config');
|
||||
symlinkSync(join(dataRoot, 'cfg'), configDir);
|
||||
const dir = join(configDir, 'secrets', 'mosaic-stack', 'mail-content', 'acme');
|
||||
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
const file = join(dir, 'k1');
|
||||
writeFileSync(file, 'synthetic', { mode: 0o600 });
|
||||
const key = contentKey({ path: file });
|
||||
assert.deepEqual(checkKeyLocation(key, { ...roots, configDir }), ['content-key.path: resolves inside dataRoot; mail secrets live outside it']);
|
||||
// A planned slot that doesn't exist yet resolves through its nearest existing ancestor.
|
||||
const slot = service({ slotPath: join(configDir, 'secrets', 'mosaic-stack', 'mail', 'acme', 'ingress', 'imap') });
|
||||
assert.deepEqual(checkSlotLocation(slot, { ...roots, configDir }), ['service.slotPath: resolves inside dataRoot; mail secrets live outside it']);
|
||||
// A forbidden root named through an alias still covers the real configDir.
|
||||
const realConfig = join(dataRoot, 'cfg');
|
||||
const alias = join(root, 'alias');
|
||||
symlinkSync(dataRoot, alias);
|
||||
const realKey = contentKey({ path: join(realConfig, 'secrets', 'mosaic-stack', 'mail-content', 'acme', 'k1') });
|
||||
assert.deepEqual(checkKeyLocation(realKey, { ...roots, dataRoot: alias, configDir: realConfig }),
|
||||
['content-key.path: resolves inside dataRoot; mail secrets live outside it']);
|
||||
// The same layout outside every forbidden root passes.
|
||||
const safeConfig = join(root, 'safe');
|
||||
mkdirSync(safeConfig, { mode: 0o700 });
|
||||
const safeKey = contentKey({ path: join(safeConfig, 'secrets', 'mosaic-stack', 'mail-content', 'acme', 'k1') });
|
||||
assert.deepEqual(checkKeyLocation(safeKey, { ...roots, configDir: safeConfig }), []);
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('T11 a protected file must be a regular owned 600 file with no symbolic link on its path', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'mail-schema-'));
|
||||
try {
|
||||
const configDir = join(root, 'config');
|
||||
const dir = join(configDir, 'secrets', 'mosaic-stack', 'mail-content', 'acme');
|
||||
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
const file = join(dir, 'k1');
|
||||
writeFileSync(file, 'synthetic', { mode: 0o600 });
|
||||
assert.deepEqual(checkProtectedFile(file, { configDir }), []);
|
||||
chmodSync(file, 0o640);
|
||||
assert.ok(checkProtectedFile(file, { configDir }).some(problem => problem.includes('mode 640')));
|
||||
chmodSync(file, 0o400);
|
||||
assert.ok(checkProtectedFile(file, { configDir }).some(problem => problem.includes('mode 400')));
|
||||
chmodSync(file, 0o600);
|
||||
assert.ok(checkProtectedFile(file, { configDir, uid: process.getuid() + 1 }).some(problem => problem.includes('owned by uid')));
|
||||
const link = join(dir, 'k2');
|
||||
symlinkSync(file, link);
|
||||
assert.deepEqual(checkProtectedFile(link, { configDir }), [`${link}: must be a regular file, not a symbolic link`]);
|
||||
const realDir = join(root, 'elsewhere');
|
||||
mkdirSync(realDir, { mode: 0o700 });
|
||||
writeFileSync(join(realDir, 'k3'), 'synthetic', { mode: 0o600 });
|
||||
symlinkSync(realDir, join(configDir, 'secrets', 'mosaic-stack', 'mail-content', 'beta'));
|
||||
const viaLink = join(configDir, 'secrets', 'mosaic-stack', 'mail-content', 'beta', 'k3');
|
||||
assert.deepEqual(checkProtectedFile(viaLink, { configDir }),
|
||||
[`${viaLink}: a symbolic link sits on the path at ${join(configDir, 'secrets', 'mosaic-stack', 'mail-content', 'beta')}`]);
|
||||
assert.deepEqual(checkProtectedFile(join(dir, 'missing'), { configDir }), [`${join(dir, 'missing')}: not found`]);
|
||||
mkdirSync(join(dir, 'k4'));
|
||||
assert.deepEqual(checkProtectedFile(join(dir, 'k4'), { configDir }), [`${join(dir, 'k4')}: must be a regular file, not a symbolic link`]);
|
||||
// The file must sit under configDir, lexically.
|
||||
const outside = join(realDir, 'k3');
|
||||
assert.deepEqual(checkProtectedFile(outside, { configDir }), [`${outside}: must sit under ${configDir}`]);
|
||||
const dotted = `${configDir}/../elsewhere/k3`;
|
||||
assert.deepEqual(checkProtectedFile(dotted, { configDir }), [`${dotted}: must be a normalized absolute path`]);
|
||||
// A symbolic-link configDir is refused, even with a real 0600 file behind it.
|
||||
const linkedConfig = join(root, 'linked-config');
|
||||
symlinkSync(configDir, linkedConfig);
|
||||
const viaConfig = join(linkedConfig, 'secrets', 'mosaic-stack', 'mail-content', 'acme', 'k1');
|
||||
assert.deepEqual(checkProtectedFile(viaConfig, { configDir: linkedConfig }), [`${viaConfig}: a symbolic link sits on the path at ${linkedConfig}`]);
|
||||
// A forbidden root named through an alias still covers the file.
|
||||
const alias = join(root, 'alias');
|
||||
symlinkSync(root, alias);
|
||||
assert.deepEqual(checkProtectedFile(file, { configDir, dataRoot: alias }), [`${file}: resolves inside dataRoot; mail secrets live outside it`]);
|
||||
assert.deepEqual(checkProtectedFile(file, { configDir, dataRoot: join(root, 'data') }), []);
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('T12 the key lifecycle moves forward only, and an expired key gives no use', () => {
|
||||
assert.ok(keyTransitionAllowed('active', 'retired'));
|
||||
assert.ok(keyTransitionAllowed('active', 'expired'));
|
||||
assert.ok(keyTransitionAllowed('retired', 'expired'));
|
||||
for (const [from, to] of [['retired', 'active'], ['expired', 'active'], ['expired', 'retired'], ['active', 'active'], ['lost', 'active'], ['constructor', 'active']])
|
||||
assert.equal(keyTransitionAllowed(from, to), false, `${from} -> ${to}`);
|
||||
const now = new Date('2027-01-01T00:00:00.000Z');
|
||||
assert.equal(keyUse(contentKey(), now), 'encrypt-decrypt');
|
||||
assert.equal(keyUse(contentKey({ state: 'retired' }), now), 'decrypt');
|
||||
assert.equal(keyUse(contentKey({ state: 'expired' }), now), 'none');
|
||||
assert.equal(keyUse(contentKey(), new Date('2027-10-22T16:00:00.000Z')), 'none', 'past expiresAt, whatever the state says');
|
||||
assert.equal(keyUse(contentKey({ state: 'retired' }), new Date('2027-10-22T15:59:59.999Z')), 'decrypt', 'the last instant before expiresAt');
|
||||
assert.equal(keyUse(contentKey(), new Date(AT)), 'encrypt-decrypt', 'from createdAt');
|
||||
assert.equal(keyUse(contentKey(), new Date('2026-10-22T15:59:59.999Z')), 'none', 'before createdAt');
|
||||
// Malformed records or clock fail closed.
|
||||
assert.equal(keyUse(contentKey({ state: 'escrowed' }), now), 'none', 'unknown state');
|
||||
assert.equal(keyUse(contentKey({ state: 'constructor' }), now), 'none', 'prototype key as state');
|
||||
assert.equal(keyUse(contentKey({ createdAt: 'not-a-date' }), now), 'none', 'invalid createdAt');
|
||||
assert.equal(keyUse(contentKey({ expiresAt: 'not-a-date' }), now), 'none', 'invalid expiresAt');
|
||||
assert.equal(keyUse(contentKey(), new Date(Number.NaN)), 'none', 'invalid clock');
|
||||
assert.equal(keyUse(contentKey(), '2027-01-01T00:00:00.000Z'), 'none', 'a clock that is not a Date');
|
||||
refused('content-key', contentKey({ state: 'escrowed' }), 'must be one of');
|
||||
refused('content-key', contentKey({ expiresAt: AT }), 'must be after createdAt');
|
||||
});
|
||||
|
||||
test('T13 the disabled schema leaves v1 unchanged: no mail kind, no mail action', () => {
|
||||
const kinds = readFileSync(new URL('../../plugins/src/kinds.mjs', import.meta.url), 'utf8');
|
||||
const vocabulary = readFileSync(new URL('../../business/src/vocabulary.mjs', import.meta.url), 'utf8');
|
||||
assert.ok(!/mail/i.test(kinds), 'core mail-source registration waits for the approved PRD version and the source, grant and route approvals');
|
||||
assert.ok(!/mail\.access/.test(vocabulary), 'mail.access waits for a Jason-approved PRD version');
|
||||
const prd = readFileSync(new URL('../../../docs/prd/mosaic-stack.md', import.meta.url), 'utf8');
|
||||
assert.ok(!/REQ-MAIL-/.test(prd), 'the addendum is a draft until Jason approves it');
|
||||
});
|
||||
|
||||
test('T14 every addendum requirement maps to build rows, and every row E01-E16 is covered', () => {
|
||||
const draft = readFileSync(new URL('../../../docs/prd/mail-addendum-draft.md', import.meta.url), 'utf8');
|
||||
assert.match(draft, /^- Status: draft, not approved\./m);
|
||||
const blocks = draft.split(/^- \*\*(?=REQ-MAIL-)/m).slice(1);
|
||||
const ids = blocks.map(block => block.match(/^(REQ-MAIL-\d+)\.\*\*/)[1]);
|
||||
assert.deepEqual(ids, Array.from({ length: ids.length }, (_, index) => `REQ-MAIL-${index + 1}`));
|
||||
assert.equal((draft.match(/\*\*REQ-MAIL-/g) ?? []).length, ids.length);
|
||||
const covered = new Set();
|
||||
for (const [index, block] of blocks.entries()) {
|
||||
const rows = block.match(/^ {2}Rows: (E\d\d(?:, E\d\d)*)\.$/m);
|
||||
assert.ok(rows, `${ids[index]} names its rows`);
|
||||
for (const row of rows[1].split(', ')) covered.add(row);
|
||||
}
|
||||
const all = Array.from({ length: 16 }, (_, index) => `E${String(index + 1).padStart(2, '0')}`);
|
||||
assert.deepEqual([...covered].sort(), all);
|
||||
});
|
||||
Reference in new issue
Block a user