Files

284 lines
22 KiB
Markdown

# Row 154 (#1648) phase 1: candidate report
Researcher, 2026-10-18 (figures re-checked against the result files at 15:20Z). Brief: `docs/plans/2026-10-16_q168-wysiwyg-editor.md`
(blob 9d01d989c187). Search table and the rejected candidates: `LIBRARY.md`. This report is for Sage to take to Mos for the
decision 123 go. **No editor code is in the repository and nothing was installed there.** Phase 2 has not started.
## Recommendation
Candidate A: **ProseMirror + prosemirror-markdown + markdown-it**, built here from tagged source into one unminified
bundle with `esbuild`, with a **block-splice save** so an unedited file saves the same bytes. All MIT except one
BSD-2-Clause package (`entities`). No separate sanitizer: markdown parsing runs with `html:false`, and the editor schema
is the whitelist. The schema needs two overrides (links, images) that I measured as necessary, below.
This is a recommendation, not a go. Five points need a ruling from Mos (end of this report).
## 1. Project, origin, pins
Twenty packages are bundled. Every pin is a tag in the project's own repository; each tagged tree was compared with the
npm package of the same version and every file present in both is byte-identical (`results/pins.txt`;
`punycode.js` differs only in `package.json`'s `name`). The ProseMirror tags exist on the project's own forge, not on the
GitHub mirrors (which lack the current tags), so the provenance file should record that URL.
| Package | Version | License | Commit | Tree | Origin |
|---|---|---|---|---|---|
| prosemirror-model | 1.25.12 | MIT | 4211aeeaa3d738454a5d1428a95c1a210e86d787 | 30600a5e07c4ec5c63f1fb1c8ab392e2bc019b9f | code.haverbeke.berlin/prosemirror/prosemirror-model |
| prosemirror-state | 1.4.4 | MIT | d6fdcd19c4f7f68206b0a8d49649860365672585 | b781a85e73fb7c3f37bc669e7dc27e7d16f592d6 | …/prosemirror-state |
| prosemirror-view | 1.42.6 | MIT | f5a5b93a78f6f723c1fbf1a2eec1556a022cd0ee | d73328db75884503bf74157722236efe4a6ef867 | …/prosemirror-view |
| prosemirror-transform | 1.12.2 | MIT | a93e228a10b76698e0878df2e1f881389e3a94ec | a0aaeb587f5fa15efa9fadaab03e7fcb26bbe64e | …/prosemirror-transform |
| prosemirror-inputrules | 1.5.1 | MIT | 089aa3852a55d551211af90aae2560c554ccf476 | 056aa2801460da409048c2eef074f6e513bb1233 | …/prosemirror-inputrules |
| prosemirror-markdown | 1.13.8 | MIT | e3a5926aa3236af2a463d824f4f22455460b008f | 1467a53e37b554d0f0329a5017781db89ecc3da1 | …/prosemirror-markdown |
| prosemirror-commands | 1.7.2 | MIT | 59fee8d89be2d033b28c2bac68e6736e2989bf9d | b5e1865a2d532f1424f409dc5be1d56ac077b855 | …/prosemirror-commands |
| prosemirror-history | 1.5.1 | MIT | 44956b6a1929d7bc99aca9a8ec01c790260fa45f | 8c1e6d7ec040a87886e7f39fbf3029b0c2961ca9 | …/prosemirror-history |
| prosemirror-keymap | 1.2.3 | MIT | 081f24f65b6a394174a4bdf07b9a55a00361f1b7 | 9031f125a6b53618b55b12b9272573a5aba1677c | …/prosemirror-keymap |
| prosemirror-schema-basic | 1.2.5 | MIT | 531f1cad1b8059616dbe68407ab4d446b6f882f4 | c7b80fc1ee25b75fa506bd400d2d479a16b33a9c | …/prosemirror-schema-basic |
| prosemirror-schema-list | 1.5.1 | MIT | e65a144f2285178a20cbdea5b0ee16925be7d6c5 | 12f3dd9c2735f86f54af2442e2600a14dc14f61f | …/prosemirror-schema-list |
| markdown-it | 14.3.3 | MIT | d372f0d8a68bef0b0acaa68868ce8ae8ba85b384 | add6fa27b88402728a549324407204b5ca0b9799 | github.com/markdown-it/markdown-it |
| linkify-it | 5.0.2 | MIT | 50a0c914f834b201cab25ff4faefd1f832b37332 | c8b101da7bbcff3a3de72621fc7e9abf31903eab | github.com/markdown-it/linkify-it |
| uc.micro | 2.1.0 | MIT | d91cec10974f125be7accd4dd870900fd18fdec7 | 085c2a8b6c255efbd5ccd1fb13c4fc012a3521da | github.com/markdown-it/uc.micro |
| mdurl | 2.1.0 | MIT | dd913f6290a4d8772240a401aab993eb65c5786d | 36d7a3131965a4e77531f549b89ecb73eede5080 | github.com/markdown-it/mdurl |
| punycode.js | 2.3.1 (tag v2.3.1) | MIT | 9e1b2cda98d215d3a73fcbfe93c62e021f4ba768 | f95d544642e1599cd27de41416654b7b7a950031 | github.com/mathiasbynens/punycode.js |
| entities | 4.5.0 (tag v4.5.0) | BSD-2-Clause | 61afd4701eaa736978b13c7351cd3de9a96b04bc | 23f047805a33dc7ce35dbdb137c947e5526d8d5c | github.com/fb55/entities |
| orderedmap | 2.1.1 | MIT | 6a30ebbbecd30fed8bda884c7acf4ca9c6a02405 | 49c032b36195f4f3eb5bdf3aa9a0a7b70b0d523c | github.com/marijnh/orderedmap |
| rope-sequence | 1.3.4 | MIT | 11db1ba5aa2abf31ff8040bdf01fabdb9db34c7a | 4006f2f2db7ababcbe84527ccc71afc6b69c19b3 | github.com/marijnh/rope-sequence |
| w3c-keyname | 2.2.8 | MIT | 5ea93b04bb62230c7de4165e3a9b5c840f05cd44 | d6851bd96544a49c9ffd99a0b0eae920656b098d | github.com/marijnh/w3c-keyname |
`prosemirror-markdown` 1.13.8 depends on `markdown-it ^14`, so 14.3.3 (what the npm lock resolves under it) is the
parser, not markdown-it 15. `argparse` (Python-2.0 license) is in the npm lock only for markdown-it's CLI and is not
bundled. `@types/*` packages are in the lock for typings and are not bundled.
## 2. Licenses
`licenses/` holds 10 license files (8 distinct by checksum: `orderedmap` = `w3c-keyname`, and `punycode.js` = `uc.micro`).
I copied one file for the ProseMirror group (`prosemirror-model.license`) and did **not** compare the other ten ProseMirror
packages' license files with it; the provenance step in phase 2 should copy each package's own file. Holders, from the copied files:
- Marijn Haverbeke and others: `prosemirror-model` (copied), assumed the same for the other 10 ProseMirror packages (not checked), `orderedmap`, `rope-sequence`, `w3c-keyname` (MIT).
- Vitaly Puzrin (and Alex Kocharin on `markdown-it`, `mdurl`): `markdown-it`, `linkify-it`, `mdurl` (MIT).
- Mathias Bynens: `uc.micro`, `punycode.js` (MIT; the two files are byte-identical).
- Felix Böhm: `entities` (BSD-2-Clause: keep the notice in source and in binary form, with the disclaimer).
All are permissive: the obligation is to keep the notice and the disclaimer with the code, including the built bundle
(`--legal-comments=inline` keeps the license comments the packages carry; phase 2 should also ship a `NOTICE`/`LICENSE`
file next to the imported source, as the brief asks). **The repository has no root `LICENSE`, and `package.json`
says `"license": "UNLICENSED"`.** MIT and BSD-2 allow that: they impose notice-retention, not a license on our code. I am not a
lawyer and this is not a legal opinion; Mos should know the repository's own license position is unstated.
Among the other candidates in LIBRARY.md, only E (BlockNote core, MPL-2.0) and the transitive dual-licensed `dompurify` in B
and F are outside MIT/BSD/ISC.
## 3. Sanitizer, and the schema overrides this design needs
There is **no DOMPurify** in this closure; nothing in the bundle sanitizes HTML strings because nothing parses HTML into the page.
- **Load path:** Markdown is parsed by markdown-it with `html:false` (raw HTML in a file becomes text), then mapped into
the schema by `prosemirror-markdown`. The editor DOM is rendered by `prosemirror-view` through the schema's `toDOM`.
- **Paste path** (the only HTML sink): `prosemirror-view` parses clipboard HTML in a detached document created with
`document.implementation.createHTMLDocument`, which does not run scripts or load images, and maps it through the
schema's `parseDOM` rules, so only schema nodes and marks survive. This is the same mechanism as the sanitizer: the
schema is the allow-list.
- **Measured with the pinned bundle** (jsdom; `results/hostile-html.txt`; 12 hostile inputs through `DOMParser` and
`DOMSerializer` with the stock schema):
- Dropped: `onclick`, `onmouseover`, `style` attributes, `<script>`, `<svg>`, `<iframe>`, `<style>`, `<form>`,
`<object>`, `<math>`.
- **Kept by the stock schema: `<img src="x">` (the `onerror` is dropped but the image stays), `<a href="javascript:alert(3)">`
and `<a href="data:text/html,…">`.**
The stock `link` and `image` specs therefore do not meet the brief's rendering rules. Phase 2 must replace them: the `link`
mark's `parseDOM` and `toDOM` pass the href through the same `window.MosaicMarkdown.linkTarget` row 148 uses (http:,
https: or relative, with `rel="noopener noreferrer"`, nothing else), and the `image` node is replaced by text (the alt
text with the source shown), never an `<img>`. The paste path then cannot reintroduce either. This must be tested with
the brief's `<script>`, `onerror` and `javascript:` browser test, on the paste path as well as the load path.
- **Advisories.** OSV batch query on the 29-package lockfile (`results/osv-a-pm.json`), 2026-10-18: **0
advisories**. Positive control the same day (output not saved in the packet; `repro/osv.sh` has no control): dompurify 2.3.3
returned 20 and markdown-it 12.2.0 returned 3. The build
tool's own lock (27 packages, esbuild 0.28.2) also returned 0 (`results/osv-i-build.json`). An advisory-free answer
is a snapshot of that day; phase 2 should re-run it in the landing record.
- **eval and network.** No `eval(`, `new Function`, `document.write`, `fetch(`, `XMLHttpRequest` or `import(` in the
built bundle (`grep -cE` returns 0 on the 17,963-line file, which contains all 132 inputs). A grep is a screen, not proof that
no code path builds a function from a string by other means. The bundle makes no network call of its own.
## 4. Source size, and what is removed or rewritten
The built bundle reads **132 input files (131 third-party plus the 11-line prototype entry), 25,899 lines**, from the esbuild
metafile of the pinned build, counting each file's lines on disk. (An earlier count of mine, 127 files and 25,706 lines, used a
different method and is superseded.) The prototype entry has no schema or editor code yet, so phase 2 adds to this.
| Package | Files | Lines |
|---|---|---|
| prosemirror-view | 12 | 6,459 |
| prosemirror-model | 12 | 3,574 |
| prosemirror-transform | 10 | 2,113 |
| prosemirror-state | 5 | 1,092 |
| prosemirror-markdown | 4 | 917 |
| prosemirror-commands | 1 | 786 |
| prosemirror-history | 1 | 466 |
| prosemirror-schema-list, -basic, -keymap | 3 | 538 |
| prosemirror-inputrules | 4 | 247 |
| orderedmap, rope-sequence, w3c-keyname | 3 | 444 |
| **ProseMirror side** | 55 | 16,636 |
| markdown-it | 53 | 6,291 |
| entities | 8 | 1,124 |
| linkify-it | 2 | 839 |
| mdurl | 5 | 541 |
| punycode.js | 1 | 443 |
| uc.micro | 7 | 14 |
| **markdown-it side** | 76 | 9,252 |
| prototype entry (ours) | 1 | 11 |
A reviewer reads about 25,900 lines of third-party source (about 1,200 of them are `entities` tables) plus the editor code and CSS we
write in phase 2. For scale, the existing `markdown-preview.js` is 338 lines. The table counts the files the build reads, not
every file in a package's directory.
**What the imported directory would hold:** the files above (the `src/` of the ProseMirror packages, the `index.mjs` and `lib/`
of the markdown-it family, `entities/src`), each package's license file, and a provenance file with the package, the
commit and the tree hash from section 1. **What it would not hold:** `dist/`, `.d.ts`/`.d.cts`, tests, benchmarks, READMEs,
`rollup.config.*` and anything else the build does not read. The brief's tree-hash-vs-provenance test then hashes each package
directory against the tree hash recorded in the provenance file (the tagged tree minus those excluded paths, so the test
needs a rule for which paths count; I would pin the rule in the test and not in prose).
**Rewritten:** nothing in the imported third-party source. All our changes go in our own files: the entry module, the schema, the
splice save, and the editor CSS.
**Not human-readable, three generated files (`results/` lists the line counts):**
- `entities/src/generated/decode-data-html.ts`: 8 lines, 47,698 bytes, encoded strings. The tag has `maps/` and
`scripts/write-decode-map.ts`, which look like its generator (I did not run it, so I have not shown they reproduce the file).
- `uc.micro/categories/P/regex.mjs` (2,042 bytes, one line) and `.../S/regex.mjs` (2,551 bytes): generated Unicode regexes. I did
not find a generator script in the tagged `uc.micro` tree.
The same generated files exist in every markdown-it build. Options: (a) accept them as data, with the provenance file stating they are
generated (and `entities`' generator and input beside its table); (b) regenerate in the build. I did not run any generator, so (b) is
not shown to work; (a) costs nothing and is what I would do unless a reviewer objects.
## 5. Build, and how a reviewer reproduces it byte for byte
**Tool.** `esbuild` 0.28.2. In my scratch build it is the only dependency of a `package.json` with an exact version
(`"esbuild":"0.28.2"`); **nothing in the repository pins it yet**, so phase 2 would add that pin. Lockfile integrity hash `sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==`
(platform binary `@esbuild/linux-x64` 0.28.2, `sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==`;
file sha256 `e1698a3d5c6c0798fee4fd3b5cc816651f460c63d390a7a26ea4beb0b1884100`, 11.4 MB, statically linked, stripped).
**This is a prebuilt Go binary.** It is a build tool pulled by the lockfile, not vendored or committed, and the bundle it produces is
what Q167 D3 requires to be rebuilt from source. Whether that satisfies D3 is Mos's call (question 2).
**Recipe** (`repro/build-pinned.sh`, run from a directory with the pinned trees checked out; no network):
```
esbuild entry.mjs --bundle --format=iife --target=es2022 --platform=browser \
--legal-comments=inline --log-level=warning \
--alias:prosemirror-{model,state,view,transform,inputrules,markdown}=<pkg>/src/index.ts \
--alias:prosemirror-{commands,history,keymap,schema-basic,schema-list}=<pkg>/src/<name>.ts \
--alias:orderedmap=<pkg>/index.js --alias:rope-sequence=<pkg>/index.js --alias:w3c-keyname=<pkg>/index.js \
--alias:markdown-it=<pkg>/index.mjs --alias:mdurl=<pkg>/index.mjs --alias:uc.micro=<pkg>/index.mjs \
--alias:linkify-it=<pkg>/index.mjs --alias:punycode.js=<pkg>/punycode.js --alias:entities=<pkg>/src/index.ts \
--outfile=editor.js
```
No minify flag: the output is readable. For the ProseMirror packages the alias target is the file each package's
`prepare` script feeds `pm-buildhelper` (checked for `prosemirror-view`: `src/index.ts`; the others follow the script in
`repro/build-pinned.sh`). `punycode.js` is the tag's CommonJS file; esbuild wraps it. No `--external`, no plugins, no `define`.
The entry in `repro/entry.mjs` is a measurement prototype that exports the pieces; the phase 2 entry will differ, so the hash
above is the hash of the prototype, and phase 2 records its own.
**Result.** The pinned trees were exported with `git archive <tag>` (no npm packages in the build input). Two builds in a
sandbox with no network gave identical output: sha256
`9587a1706eb5fa8d4d2c79976be6b1d84a0513fe814eeee381b6ecf384d15077`, 17,963 lines, 722,595 bytes (`results/build-hashes.txt`).
A reviewer repeats the same recipe from the same tags and compares the hash. The earlier prototype that aliased
two packages to npm `dist` also built identically twice but is not the plan; this one uses tag sources only.
One esbuild warning appears and is harmless: `prosemirror-transform`'s `import "./mark"` is ignored because the package says
`sideEffects:false`; `addMark` and `removeMark` are still in the bundle (imported by `transform.ts` directly).
**Caveats on the reproduction claim.** My builds ran with the host's Node and the native binary in a `bwrap --unshare-net`
sandbox. I proved identical output on this machine twice; I have not rebuilt on another machine, another architecture, or another
esbuild platform package, and the macOS and Windows binaries are different files. Phase 2's test should record the hash and fail
on drift. An alternative with no prebuilt tool (a small concatenating build with Node's type stripping) is possible but would be new
code to review, because bare imports cannot use an import map under this CSP; I did not build it.
## 6. Markdown normalisation (the measured part)
Corpus: the 414 `.md` files in this checkout (`results/corpus.txt`). **There is no `spec/north-star.md`**; the nearest file is the 3-line
`packages/project/templates/north-star.md`, which is in the corpus. Test: parse, serialise, compare bytes, no edit.
| Stack | Byte-identical, no edit | Notes |
|---|---|---|
| ProseMirror + prosemirror-markdown, stock | 1 of 414 | `repro/rt.mjs` |
| Tiptap 3.32.0 + @tiptap/markdown | 1 of 414 | **drops tables entirely** |
| Milkdown 7.22.3, commonmark + gfm | 59 of 414 | 142 of 414 with `-` bullets set |
| **Block splice** (below), A | **414 of 414** (12,879 blocks, 98 with front matter) | exact by construction |
What the stock serialiser changes (each seen in the corpus; small cases in `repro/micro.mjs`): bullets `-` to `*`;
hard-wrapped paragraphs joined onto one line; tables flattened into one paragraph; setext headings to ATX; `~~~` fences and indented
code to backtick fences; reference links inlined; entities decoded; `_em_` to `*em*`; a hard break becomes backslash plus newline;
`~~strike~~` and task markers escaped; YAML front matter turns into a rule plus a heading; 3+ blank lines collapse to one; CRLF
collapses.
**Block splice (the proposal for phase 2).** The document is split into top-level blocks using markdown-it's token `map` (the source
line range of each block). The editor holds those blocks; on save each block that is unchanged since load writes back its **original
source text**, and only an edited block is re-serialised. Gaps between blocks (blank lines) are kept as they were. Tables, raw HTML
blocks, rules and front matter (762 HTML blocks, 346 tables and 205 rules in this corpus; 98 files start with front matter) are **opaque source blocks**: shown, and edited as
source text, never reserialised. This satisfies the brief's "same bytes if no edits".
Two honest limits:
1. The 414-of-414 figure measures the splice arithmetic (gaps plus block slices rebuild every file), not the editor. A file the
editor loads and saves untouched is exact because untouched blocks emit their stored text; phase 2's round-trip test
(`spec/north-star.md` as the brief says, which does not exist here, so a repo file chosen by Sage, plus a document with every
preview element) is what proves it end to end.
2. **An edited block is rewritten in the editor's canonical Markdown**, not its original style. Measured on the 11,566 blocks that are not
opaque (`results/per-block-roundtrip.txt`), serialising each block alone and comparing with its own source text:
| Block | Count | Stock serialiser exact | With soft breaks kept and `-` bullets |
|---|---|---|---|
| paragraph | 5,639 | 32% | 97% |
| heading | 3,655 | 99% | 99% |
| bullet list | 1,326 | 0% | 85% |
| fence | 468 | 100% | 100% |
| indented code block | 242 | 0% | 0% (becomes a fence) |
| ordered list | 228 | 25% | 84% |
| blockquote | 8 | 13% | 63% |
| all | 11,566 | 52% | 94% |
So with two serialiser settings, about 94% of blocks would come back unchanged even after an edit, and the other 6% are rewritten
(for example indented code to a fenced block, a list's continuation indent, wrapped lines). A list or quote is one top-level
block, so an edit anywhere in a list reserialises that whole list. The brief says reviewers accept "the exact normalisation"; this
is that statement, and the numbers are from one repository's docs.
## 7. What this means for the Console
- **Routing.** `serve.mjs` serves `packages/webui/src/public/` from an explicit route table (lines 24 and 25: file name to content
type). A single bundle is one route (`/editor.js`) and one CSS file is another (`/editor.css`, carrying
`prosemirror.css`'s 54 lines and ours); unbundled serving would need about 131 routes or a new directory route. **`serve.mjs` is not in the
brief's owned-file list, so the two route entries need Sage's say-so.**
- **CSP.** `script-src 'self'` is enough for a same-origin `<script src="/editor.js">`; no CSP change is planned, and if phase 2
finds one is needed it stops there, as the brief says. I have not run the bundle in a browser; jsdom was the environment, so
the brief's browser test is the first real check.
- **Existing code.** `markdown-preview.js` stays; the editor calls `window.MosaicMarkdown.linkTarget` rather than copy the link rule.
## 8. Open items and risks
- Five questions for Mos, below.
- The build is proven reproducible on one machine, twice; not across machines (section 5).
- Three generated data files are not human-readable (section 4).
- Candidates D, E, G, H were not run through the round trip; B and C were. If Mos wants the choice to be between measured round
trips, D and G would be next.
- The advisory check is a snapshot (2026-10-18).
- Phase 2's sizing: the editor view, schema overrides, splice save, tests and CSS are new code. I have not estimated their size.
## Questions for Mos (decision 123)
1. **Go on candidate A** (ProseMirror + prosemirror-markdown + markdown-it; 19 MIT packages plus `entities` BSD-2-Clause), importing its
source into one new directory in the repository?
2. **Build tool.** Accept `esbuild` 0.28.2 (prebuilt Go binary, exact-pinned with lockfile integrity, not committed) as the pinned build
tool, as the brief's "builds from pinned tools" intends? Or require a no-prebuilt-tool build (more code to review, unbuilt)?
3. **Generated tables.** Accept the three generated data files as imported data with generator and input beside them (section 4 option (a))?
4. **Normalisation.** Accept that an unedited file saves byte-identical and an edited block is rewritten in canonical Markdown, with the
table, raw HTML, rule and front-matter blocks edited only as source text?
5. **Licensing.** The repository has no root `LICENSE` and says `UNLICENSED`; the imported code is MIT and BSD-2-Clause and needs only its
notices kept. Does Mos want anything stated about that before import?
For Sage: `serve.mjs` needs two route entries (`/editor.js`, `/editor.css`) and is outside the brief's owned files.
## Packet contents
`LIBRARY.md` (search), `REPORT.md` (this file), `licenses/` (10 files), `results/` (pins, OSV for 8 lockfiles, build hashes,
hostile-HTML run, splice and per-block round-trip output, the corpus list), `repro/` (build script and entry, the measurement
scripts; they carry the scratch paths I ran them from, which a reviewer must edit). Scratch lives outside the repository at
`/mnt/storage/scratch/researcher/q154/` and is not part of the evidence.