Files
stack/packages/library/README.md
T

227 lines
13 KiB
Markdown

# packages/library
Reads a writing repo by a mapping of folders to books, types, canon and playground. This is Case 2
row C2-01 (`docs/plans/2026-10-20_case2-rows.md`), built from design sections 1.1 to 1.3 and 7.4 in
`agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`.
It is pure code with no state. Business file loading (C2-04), sync (C2-03a), the index (C2-05a) and the
guard (C2-06) build on it.
## Mapping
`checkMapping(entry, where, { fail })` checks a `kind: "library"` project entry and returns a
deep-frozen copy. `where` names the entry in messages. `fail(code, message)` defaults to throwing a
`LibraryError`. C2-04 passes its own so a business file refusal carries the file name.
```json
{
"kind": "library",
"remote": "https://git.example.invalid/her/books.git",
"branch": "main",
"user": "books-bot",
"vaultPrefix": "books",
"types": { "Characters": "character", "Places": "place", "Research": "research" },
"manuscript": ["Chapters"],
"universes": {
"harbor": { "name": "Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"] }
}
}
```
- `remote` is https with no credentials, query or fragment. `branch` is a plain ref name.
- `user` is the bot account name from design step 3, required for sync. It is optional in the
mapping and checked with row 199's account name pattern, `[A-Za-z0-9_.-]`, 1 to 100 characters.
- `vaultPrefix` is optional, one folder name.
- `editor` is optional, exactly `{ "name", "email" }`: the ident her Console saves commit under
(C2-01d). Each value is text that is non-empty after trim, with no control character
(`U+0000`-`U+001F`, `U+007F`-`U+009F`), `<` or `>`. It comes back only when given.
- `types` maps a folder name to `character`, `place`, `research`, `tracker` or `summary`.
`manuscript` lists folder names and shares none with `types`.
- A universe has `name`, `books` and `playground`. A hybrid adds `kind: "hybrid"`, two or more
plain-universe `sources`, `constraints` (each one `{ "exclude": ... }` or
`{ "pin": { "entity", "field", "value" } }`) and `notes`, and may have `books: []`. A draft
universe has only `kind: "draft"`, `name` and one or more `playground` prefixes, and comes back as
`{ kind: "draft", name, books: [], playground, supporting }`. `kind` is absent, `"hybrid"` or `"draft"`.
- A playground prefix ends in `/` and has one of three levels (C2-01c). Book level lies inside one of
its own entry's book folders. Universe level, in a plain universe or a hybrid, lies outside every
book folder. A draft's prefixes all lie outside every book folder. No prefix holds another or is
listed twice, across every entry.
- Every universe kind may also have `supporting`, a list of prefixes where agents may write (C2-01d,
M12). It comes back on every universe, `[]` when absent. A supporting prefix follows the playground's
rules: it ends in `/`, has the same three levels, holds no other supporting prefix and is listed
once, across every entry.
- Between the two lists: a supporting prefix may equal a playground prefix of its own entry, and a
playground prefix may hold one of its own entry's supporting prefixes. A supporting prefix may not
hold a playground prefix, which scope would then never reach, and may not equal, hold or lie inside
another entry's playground prefix.
Refusals:
| Code | When |
|---|---|
| `library-playground-other-universe` | a playground prefix lies inside, or holds, a book folder of another entry |
| `library-supporting-other-universe` | a supporting prefix lies inside, or holds, a book folder of another entry |
| `library-mapping-invalid` | any other shape fault: unknown key, bad remote or branch, an unknown `kind`, a book folder in two universes, a book folder that holds another, a playground or supporting prefix that holds another of its list or is listed twice, a prefix equal to a book folder or holding its own entry's book, a supporting prefix that holds a playground prefix or overlaps another entry's, a bad `editor`, a draft with `books` or with no prefix, a hybrid source that is unknown, a hybrid or a draft |
## Scope
`classify(mapping, path)` decides scope from the path alone. Only paths ending in lowercase `.md` are
mapped. It checks prefixes first, supporting before playground. A path under any entry's supporting
prefix is `supporting`, and otherwise one under a playground prefix is `playground`. Either way that
entry is `universe` and the book folder holding the prefix is `book`, or `null` at universe level and
in a draft. Any other path under a book folder is `canon`. Everything else is `unmapped`. The type is
the first folder after the prefix, or after the book: a `types` name, `manuscript`, or `null`.
## Agent write paths
`supportingPath(mapping, universe, path, tree)` returns `path` when an agent may write it in that
universe, else it refuses `library-write-story-path` (C2-01d, M12). `tree` is the array `listTree`
returns for the commit the write lands on; anything else is a TypeError. It reads no git. The checks
run in this order, so a path with two faults always names the same one:
1. a string of at most 1,024 UTF-8 bytes;
2. no control character (the `editor` class), and the message leaves the path out;
3. no backslash;
4. not absolute;
5. no empty, `.` or `..` segment, which also refuses `//` and a trailing `/`;
6. `classify` gives `supporting` in that same universe, which refuses unmapped, `.canvas`, `.base`
and attachment paths, canon, a playground-only folder and another universe's prefix;
7. no folder on the way, and not the path itself, is a symlink (`120000`) or gitlink (`160000`) in
`tree`;
8. no folder on the way, and not the path itself, matches a tree path or tree folder after
`normalize('NFC').toLowerCase()` unless it is that exact name. APFS folds case and Unicode form,
so her checkout would put the file somewhere else. A tree that already holds case twins refuses a
write to either (R7).
A delete's further rule, that the path exists at the tip as a `100644` blob, belongs to the C2-02
broker.
## Reader
`readLibrary({ repo, commit, mapping })` reads one commit, never a working tree. `repo` may be a
clone or a bare mirror. `commit` must be a full 40-hex id that names a commit, else `library-commit`.
Git runs through `git()` and `gitBytes()` from `packages/project/src/git.mjs`, so the same safe config,
environment and ceiling directory apply. A failed git call refuses `library-git`.
It returns `{ commit, files, problems }`, deep-frozen and sorted. Each file has its path, scope,
universe, book, type, numbered lines, `bodyLine`, parsed `frontmatter` or `null`, `frontmatterRaw`
and `links`. Front matter `appears_in` and `relationships` wikilinks count as links. A link in a block
list carries its list item's line. One in a scalar or flow value carries its key's line.
Problems carry `{ code, path, line, message }`:
| Rule | Meaning | File indexed |
|---|---|---|
| R1 | not a regular `100644` blob of at most 1 MiB, or not UTF-8, or holds a NUL | no |
| R2 | path is unmapped | no |
| R3 | front matter not fully parsed, `line` is where parsing stopped | yes, front matter kept raw |
| R4 | link resolves to no file or to more than one | yes, link dropped |
| R5 | canon file links into the playground | yes |
| R6 | canon file links into another universe, a hybrid's sources excepted | yes |
| R7 | two paths differ only in case | no, neither |
The front matter parser takes scalars, quoted strings, flow lists, block lists and wikilinks. Nested
maps, anchors, block scalars and duplicate keys stop it with R3.
## Links
`resolver(paths, { vaultPrefix })` tries four steps in order, each with the target as written and
then with `.md` added:
1. `root`: the path from the repo root.
2. `vault-prefix`: the same path with `vaultPrefix/` stripped.
3. `relative`: relative to the linking file, never above the repo root.
4. `shortest`: a target with no `/` that matches the base name of exactly one indexed file.
It resolves only against indexed files. A link to a `.canvas` file or an attachment is R4. A link into
another repo resolves to nothing. `extractLinks` skips fenced and inline code.
A `![[...]]` embed resolves like any link. A body embed that resolves to nothing and whose target ends
in an extension other than `.md`, in any case, embeds an attachment, and it is never R4. An extension is
a dot and up to 8 letters and digits, at least one a letter. Any other unresolved embed is R4, and so is
an unresolved front matter value written as an embed (C2-01b N3, sage-1900).
## renderItems
`renderItems(body, { scopeOf, changed })` turns a reply body into the plain text both channels show.
It is pure. Item kinds are `fact`, `finding`, `correction` (with `from` and `to`), `question` and
`idea`. Each cite is a path with commit, line range and quote, or a url with a quote.
```
1. Correction: Chapter 1 says grey.
From: "green eyes"
To: "grey eyes"
[canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes" (changed since)
```
A body of `{ "items": [], "notice": ... }` gives `No cited answer`, `No answer` or `The turn failed`.
Any other shape gives the fixed line `This reply could not be shown.` and none of the body's text.
## Survey
```sh
mosaic library survey <repo> [--mapping <file>]
```
It reads the repo's HEAD commit and prints folder names, file counts by extension, front matter key
sets, link-form counts and the count of body embeds whose target has an extension other than `.md`,
which still count as embeds. That count goes by the name alone, so an embed of `Dr.Quinn.md` written
`![[Dr.Quinn]]` counts there too. Folder names print as JSON strings, with DEL, C1 and bidi controls,
U+061C among them, also as `\uXXXX`. It never prints file text, front matter values, file names or
link targets.
A key that is not a short identifier prints as `(other key)`, since it could be her words. With
`--mapping` it adds scope counts, resolver step counts and problem counts by rule.
Exit codes follow `packages/project`: 0 done, 2 the repo can't be read (`library-git`,
`library-commit`), 3 a bad mapping, 4 usage. `scripts/mosaic` sends `library survey` here. Other
library verbs will be bus verbs.
## Sync
`sync(context, { slot, accept })` (C2-03a, design 7.2) fetches the entry's `branch` from its `remote`
into `refs/remotes/library/<branch>` of the C2-04 mirror, then publishes under the library lock:
| Fetched tip | `served` | State |
|---|---|---|
| a fast-forward, or the first sync | moves to the tip | `problems` rebuilt from the tip |
| the served commit | stays | `lastFetch` set |
| not a fast-forward | stays | `diverged` holds the tip, `library-diverged`, until `accept` |
| fetch failed, slot missing | stays | `lastError` holds `library-fetch-failed` or `library-credential-missing` |
`lastFetch` is the last fetch that succeeded. Once diverged, `served` stays put until a human runs
`library.sync` with `accept` set to the fetched tip, even if a later tip is a fast-forward again. A
tip back at the served commit clears it. A
stale `accept` refuses `library-accept-stale`. `accept` also starts the state over, so it is how a
changed `branch` gets a fresh record. An entry with no `user` refuses `library-remote-invalid` and
runs no git.
`syncCommand('fetch', url, { mirror, branch, user, slot })` gives the one argv and env. The argv is
`SAFE` from `packages/project/src/git.mjs`, then `-c protocol.https.allow=always -c
http.followRedirects=false -c credential.helper= -c core.createObject=rename`, then `-C <mirror>` and
the fetch. `core.createObject=rename` is there because git's default files an object by link then
unlink, and a reader's mirror check refuses a file with two links. The env is
`gitEnv()` with `LC_ALL=C`, `GIT_ASKPASS` naming `bin/askpass`, and the host, user and slot path. The
helper answers only `Username for 'https://<host>': ` and `Password for 'https://<user>@<host>': `,
the second from the slot's first line, and exits 1 on any other prompt. Sync checks that the slot is
a private regular file and never opens it. The token is in no argv, env, URL, config or log.
Sync never runs gc or prune. A file read takes `served` under the lock and reads its objects after it
lets go, so an object must outlive the next move of `served`. The bus host runs `sync` for every entry
with a `user`, once at start and then every 60 seconds; the slot is
`<configDir>/secrets/<business>/library-gitea`. `library.sync` is the human's bus verb for the same.
The human transport (`packages/bus/src/human-cli.mjs`) waits 130 seconds for `library.sync`, past the
server's 120-second hold, and 5 seconds for every other verb. A client that gives up before the answer
gets `outcome-unknown`, but the sync keeps running: a second call meanwhile gets `library-sync-busy`,
and `library.status` shows the result once it ends. A sync that outlasts the 120-second hold gets
`request-timeout` from the server and still finishes the same way. The Console has no library route yet: `cli.mjs`
passes `librarySync` to `startServer`, and `serve.mjs` doesn't read it.
## Tests
```sh
node --test packages/library/tests/
```
Fixtures are made-up repos built per test in a temp dir. A marker string stands in for her text, and
the survey tests check that it never appears in output.