Files
stack/packages/library/README.md
T
jason.woltje eb9672c910 row 254: README request-timeout sentence and a DEFERRED entry after the landing (#1759)
Doc-only commit after the row 254 landing. The library README now says
that a sync outlasting the server's 120-second hold gets request-timeout
and still finishes (Dewey N1, #1759 comment 30173; server.mjs sets the
120 s socket timeout and the broker promise runs on). DEFERRED records
Dewey N2: library-mapping-invalid, library-remote-invalid,
library-state-invalid and library-git still map to 502 until the Console
library routes row decides them. No test reads either file's content.
2026-10-21 18:55:26 -05:00

183 lines
9.6 KiB
Markdown

# packages/library
Reads a writing repo by a mapping of folders to books, types, canon and playground. This is Case 2
row C2-01 (`docs/plans/2026-10-20_case2-rows.md`), built from design sections 1.1 to 1.3 and 7.4 in
`agents/gosalyn/work/case2/DESIGN.md` on `gosalyn/case2-design`.
It is pure code with no state. Business file loading (C2-04), sync (C2-03a), the index (C2-05a) and the
guard (C2-06) build on it.
## Mapping
`checkMapping(entry, where, { fail })` checks a `kind: "library"` project entry and returns a
deep-frozen copy. `where` names the entry in messages. `fail(code, message)` defaults to throwing a
`LibraryError`. C2-04 passes its own so a business file refusal carries the file name.
```json
{
"kind": "library",
"remote": "https://git.example.invalid/her/books.git",
"branch": "main",
"user": "books-bot",
"vaultPrefix": "books",
"types": { "Characters": "character", "Places": "place", "Research": "research" },
"manuscript": ["Chapters"],
"universes": {
"harbor": { "name": "Harbor", "books": ["Book One"], "playground": ["Book One/Playground/"] }
}
}
```
- `remote` is https with no credentials, query or fragment. `branch` is a plain ref name.
- `user` is the bot account name from design step 3, required for sync. It is optional in the
mapping and checked with row 199's account name pattern, `[A-Za-z0-9_.-]`, 1 to 100 characters.
- `vaultPrefix` is optional, one folder name.
- `types` maps a folder name to `character`, `place`, `research`, `tracker` or `summary`.
`manuscript` lists folder names and shares none with `types`.
- A universe has `name`, `books` and `playground`. A hybrid adds `kind: "hybrid"`, two or more
non-hybrid `sources`, `constraints` (each one `{ "exclude": ... }` or
`{ "pin": { "entity", "field", "value" } }`) and `notes`.
Refusals:
| Code | When |
|---|---|
| `library-playground-outside-book` | a playground prefix is not strictly inside a book of its own universe |
| `library-mapping-invalid` | any other shape fault: unknown key, bad remote or branch, a book folder in two universes, a book folder that holds another, a playground prefix that holds another, an unknown hybrid source |
## Scope
`classify(mapping, path)` decides scope from the path alone. Only paths ending in lowercase `.md` are
mapped. A path under a playground prefix is `playground`. Any other path under a book folder is
`canon`. Everything else is `unmapped`. The type is the first folder after the book, or after the
playground prefix: a `types` name, `manuscript`, or `null`.
## Reader
`readLibrary({ repo, commit, mapping })` reads one commit, never a working tree. `repo` may be a
clone or a bare mirror. `commit` must be a full 40-hex id that names a commit, else `library-commit`.
Git runs through `git()` and `gitBytes()` from `packages/project/src/git.mjs`, so the same safe config,
environment and ceiling directory apply. A failed git call refuses `library-git`.
It returns `{ commit, files, problems }`, deep-frozen and sorted. Each file has its path, scope,
universe, book, type, numbered lines, `bodyLine`, parsed `frontmatter` or `null`, `frontmatterRaw`
and `links`. Front matter `appears_in` and `relationships` wikilinks count as links. A link in a block
list carries its list item's line. One in a scalar or flow value carries its key's line.
Problems carry `{ code, path, line, message }`:
| Rule | Meaning | File indexed |
|---|---|---|
| R1 | not a regular `100644` blob of at most 1 MiB, or not UTF-8, or holds a NUL | no |
| R2 | path is unmapped | no |
| R3 | front matter not fully parsed, `line` is where parsing stopped | yes, front matter kept raw |
| R4 | link resolves to no file or to more than one | yes, link dropped |
| R5 | canon file links into the playground | yes |
| R6 | canon file links into another universe, a hybrid's sources excepted | yes |
| R7 | two paths differ only in case | no, neither |
The front matter parser takes scalars, quoted strings, flow lists, block lists and wikilinks. Nested
maps, anchors, block scalars and duplicate keys stop it with R3.
## Links
`resolver(paths, { vaultPrefix })` tries four steps in order, each with the target as written and
then with `.md` added:
1. `root`: the path from the repo root.
2. `vault-prefix`: the same path with `vaultPrefix/` stripped.
3. `relative`: relative to the linking file, never above the repo root.
4. `shortest`: a target with no `/` that matches the base name of exactly one indexed file.
It resolves only against indexed files. A link to a `.canvas` file or an attachment is R4. A link into
another repo resolves to nothing. `extractLinks` skips fenced and inline code.
A `![[...]]` embed resolves like any link. A body embed that resolves to nothing and whose target ends
in an extension other than `.md`, in any case, embeds an attachment, and it is never R4. An extension is
a dot and up to 8 letters and digits, at least one a letter. Any other unresolved embed is R4, and so is
an unresolved front matter value written as an embed (C2-01b N3, sage-1900).
## renderItems
`renderItems(body, { scopeOf, changed })` turns a reply body into the plain text both channels show.
It is pure. Item kinds are `fact`, `finding`, `correction` (with `from` and `to`), `question` and
`idea`. Each cite is a path with commit, line range and quote, or a url with a quote.
```
1. Correction: Chapter 1 says grey.
From: "green eyes"
To: "grey eyes"
[canon] Book One/Characters/Mara Vell.md:9 @bbbbbbb "green eyes" (changed since)
```
A body of `{ "items": [], "notice": ... }` gives `No cited answer`, `No answer` or `The turn failed`.
Any other shape gives the fixed line `This reply could not be shown.` and none of the body's text.
## Survey
```sh
mosaic library survey <repo> [--mapping <file>]
```
It reads the repo's HEAD commit and prints folder names, file counts by extension, front matter key
sets, link-form counts and the count of body embeds whose target has an extension other than `.md`,
which still count as embeds. That count goes by the name alone, so an embed of `Dr.Quinn.md` written
`![[Dr.Quinn]]` counts there too. Folder names print as JSON strings, with DEL, C1 and bidi controls,
U+061C among them, also as `\uXXXX`. It never prints file text, front matter values, file names or
link targets.
A key that is not a short identifier prints as `(other key)`, since it could be her words. With
`--mapping` it adds scope counts, resolver step counts and problem counts by rule.
Exit codes follow `packages/project`: 0 done, 2 the repo can't be read (`library-git`,
`library-commit`), 3 a bad mapping, 4 usage. `scripts/mosaic` sends `library survey` here. Other
library verbs will be bus verbs.
## Sync
`sync(context, { slot, accept })` (C2-03a, design 7.2) fetches the entry's `branch` from its `remote`
into `refs/remotes/library/<branch>` of the C2-04 mirror, then publishes under the library lock:
| Fetched tip | `served` | State |
|---|---|---|
| a fast-forward, or the first sync | moves to the tip | `problems` rebuilt from the tip |
| the served commit | stays | `lastFetch` set |
| not a fast-forward | stays | `diverged` holds the tip, `library-diverged`, until `accept` |
| fetch failed, slot missing | stays | `lastError` holds `library-fetch-failed` or `library-credential-missing` |
`lastFetch` is the last fetch that succeeded. Once diverged, `served` stays put until a human runs
`library.sync` with `accept` set to the fetched tip, even if a later tip is a fast-forward again. A
tip back at the served commit clears it. A
stale `accept` refuses `library-accept-stale`. `accept` also starts the state over, so it is how a
changed `branch` gets a fresh record. An entry with no `user` refuses `library-remote-invalid` and
runs no git.
`syncCommand('fetch', url, { mirror, branch, user, slot })` gives the one argv and env. The argv is
`SAFE` from `packages/project/src/git.mjs`, then `-c protocol.https.allow=always -c
http.followRedirects=false -c credential.helper= -c core.createObject=rename`, then `-C <mirror>` and
the fetch. `core.createObject=rename` is there because git's default files an object by link then
unlink, and a reader's mirror check refuses a file with two links. The env is
`gitEnv()` with `LC_ALL=C`, `GIT_ASKPASS` naming `bin/askpass`, and the host, user and slot path. The
helper answers only `Username for 'https://<host>': ` and `Password for 'https://<user>@<host>': `,
the second from the slot's first line, and exits 1 on any other prompt. Sync checks that the slot is
a private regular file and never opens it. The token is in no argv, env, URL, config or log.
Sync never runs gc or prune. A file read takes `served` under the lock and reads its objects after it
lets go, so an object must outlive the next move of `served`. The bus host runs `sync` for every entry
with a `user`, once at start and then every 60 seconds; the slot is
`<configDir>/secrets/<business>/library-gitea`. `library.sync` is the human's bus verb for the same.
The human transport (`packages/bus/src/human-cli.mjs`) waits 130 seconds for `library.sync`, past the
server's 120-second hold, and 5 seconds for every other verb. A client that gives up before the answer
gets `outcome-unknown`, but the sync keeps running: a second call meanwhile gets `library-sync-busy`,
and `library.status` shows the result once it ends. A sync that outlasts the 120-second hold gets
`request-timeout` from the server and still finishes the same way. The Console has no library route yet: `cli.mjs`
passes `librarySync` to `startServer`, and `serve.mjs` doesn't read it.
## Tests
```sh
node --test packages/library/tests/
```
Fixtures are made-up repos built per test in a temp dir. A marker string stands in for her text, and
the survey tests check that it never appears in output.