stack/agents/researcher/work/queue-154/REPORT.md

Row 154 (#1648) phase 1: candidate report

Researcher, 2026-10-18 (figures re-checked against the result files at 15:20Z). Brief: docs/plans/2026-10-16_q168-wysiwyg-editor.md (blob 9d01d989c1). Search table and the rejected candidates: LIBRARY.md. This report is for Sage to take to Mos for the decision 123 go. No editor code is in the repository and nothing was installed there. Phase 2 has not started.

Recommendation

Candidate A: ProseMirror + prosemirror-markdown + markdown-it, built here from tagged source into one unminified bundle with esbuild, with a block-splice save so an unedited file saves the same bytes. All MIT except one BSD-2-Clause package (entities). No separate sanitizer: markdown parsing runs with html:false, and the editor schema is the whitelist. The schema needs two overrides (links, images) that I measured as necessary, below.

This is a recommendation, not a go. Five points need a ruling from Mos (end of this report).

1. Project, origin, pins

Twenty packages are bundled. Every pin is a tag in the project's own repository; each tagged tree was compared with the npm package of the same version and every file present in both is byte-identical (results/pins.txt; punycode.js differs only in package.json's name). The ProseMirror tags exist on the project's own forge, not on the GitHub mirrors (which lack the current tags), so the provenance file should record that URL.

Package	Version	License	Commit	Tree	Origin
prosemirror-model	1.25.12	MIT	4211aeeaa3d738454a5d1428a95c1a210e86d787	30600a5e07c4ec5c63f1fb1c8ab392e2bc019b9f	code.haverbeke.berlin/prosemirror/prosemirror-model
prosemirror-state	1.4.4	MIT	d6fdcd19c4f7f68206b0a8d49649860365672585	b781a85e73fb7c3f37bc669e7dc27e7d16f592d6	…/prosemirror-state
prosemirror-view	1.42.6	MIT	f5a5b93a78f6f723c1fbf1a2eec1556a022cd0ee	d73328db75884503bf74157722236efe4a6ef867	…/prosemirror-view
prosemirror-transform	1.12.2	MIT	a93e228a10b76698e0878df2e1f881389e3a94ec	a0aaeb587f5fa15efa9fadaab03e7fcb26bbe64e	…/prosemirror-transform
prosemirror-inputrules	1.5.1	MIT	089aa3852a55d551211af90aae2560c554ccf476	056aa2801460da409048c2eef074f6e513bb1233	…/prosemirror-inputrules
prosemirror-markdown	1.13.8	MIT	e3a5926aa3236af2a463d824f4f22455460b008f	1467a53e37b554d0f0329a5017781db89ecc3da1	…/prosemirror-markdown
prosemirror-commands	1.7.2	MIT	59fee8d89be2d033b28c2bac68e6736e2989bf9d	b5e1865a2d532f1424f409dc5be1d56ac077b855	…/prosemirror-commands
prosemirror-history	1.5.1	MIT	44956b6a1929d7bc99aca9a8ec01c790260fa45f	8c1e6d7ec040a87886e7f39fbf3029b0c2961ca9	…/prosemirror-history
prosemirror-keymap	1.2.3	MIT	081f24f65b6a394174a4bdf07b9a55a00361f1b7	9031f125a6b53618b55b12b9272573a5aba1677c	…/prosemirror-keymap
prosemirror-schema-basic	1.2.5	MIT	531f1cad1b8059616dbe68407ab4d446b6f882f4	c7b80fc1ee25b75fa506bd400d2d479a16b33a9c	…/prosemirror-schema-basic
prosemirror-schema-list	1.5.1	MIT	e65a144f2285178a20cbdea5b0ee16925be7d6c5	12f3dd9c2735f86f54af2442e2600a14dc14f61f	…/prosemirror-schema-list
markdown-it	14.3.3	MIT	d372f0d8a68bef0b0acaa68868ce8ae8ba85b384	add6fa27b88402728a549324407204b5ca0b9799	github.com/markdown-it/markdown-it
linkify-it	5.0.2	MIT	50a0c914f834b201cab25ff4faefd1f832b37332	c8b101da7bbcff3a3de72621fc7e9abf31903eab	github.com/markdown-it/linkify-it
uc.micro	2.1.0	MIT	d91cec10974f125be7accd4dd870900fd18fdec7	085c2a8b6c255efbd5ccd1fb13c4fc012a3521da	github.com/markdown-it/uc.micro
mdurl	2.1.0	MIT	dd913f6290a4d8772240a401aab993eb65c5786d	36d7a3131965a4e77531f549b89ecb73eede5080	github.com/markdown-it/mdurl
punycode.js	2.3.1 (tag v2.3.1)	MIT	9e1b2cda98d215d3a73fcbfe93c62e021f4ba768	f95d544642e1599cd27de41416654b7b7a950031	github.com/mathiasbynens/punycode.js
entities	4.5.0 (tag v4.5.0)	BSD-2-Clause	61afd4701eaa736978b13c7351cd3de9a96b04bc	23f047805a33dc7ce35dbdb137c947e5526d8d5c	github.com/fb55/entities
orderedmap	2.1.1	MIT	6a30ebbbecd30fed8bda884c7acf4ca9c6a02405	49c032b36195f4f3eb5bdf3aa9a0a7b70b0d523c	github.com/marijnh/orderedmap
rope-sequence	1.3.4	MIT	11db1ba5aa2abf31ff8040bdf01fabdb9db34c7a	4006f2f2db7ababcbe84527ccc71afc6b69c19b3	github.com/marijnh/rope-sequence
w3c-keyname	2.2.8	MIT	5ea93b04bb62230c7de4165e3a9b5c840f05cd44	d6851bd96544a49c9ffd99a0b0eae920656b098d	github.com/marijnh/w3c-keyname

prosemirror-markdown 1.13.8 depends on markdown-it ^14, so 14.3.3 (what the npm lock resolves under it) is the parser, not markdown-it 15. argparse (Python-2.0 license) is in the npm lock only for markdown-it's CLI and is not bundled. @types/* packages are in the lock for typings and are not bundled.

2. Licenses

licenses/ holds 10 license files (8 distinct by checksum: orderedmap = w3c-keyname, and punycode.js = uc.micro). I copied one file for the ProseMirror group (prosemirror-model.license) and did not compare the other ten ProseMirror packages' license files with it; the provenance step in phase 2 should copy each package's own file. Holders, from the copied files:

• Marijn Haverbeke and others: prosemirror-model (copied), assumed the same for the other 10 ProseMirror packages (not checked), orderedmap, rope-sequence, w3c-keyname (MIT).
• Vitaly Puzrin (and Alex Kocharin on markdown-it, mdurl): markdown-it, linkify-it, mdurl (MIT).
• Mathias Bynens: uc.micro, punycode.js (MIT; the two files are byte-identical).
• Felix Böhm: entities (BSD-2-Clause: keep the notice in source and in binary form, with the disclaimer).

All are permissive: the obligation is to keep the notice and the disclaimer with the code, including the built bundle (--legal-comments=inline keeps the license comments the packages carry; phase 2 should also ship a NOTICE/LICENSE file next to the imported source, as the brief asks). The repository has no root LICENSE, and package.json says "license": "UNLICENSED". MIT and BSD-2 allow that: they impose notice-retention, not a license on our code. I am not a lawyer and this is not a legal opinion; Mos should know the repository's own license position is unstated. Among the other candidates in LIBRARY.md, only E (BlockNote core, MPL-2.0) and the transitive dual-licensed dompurify in B and F are outside MIT/BSD/ISC.

3. Sanitizer, and the schema overrides this design needs

There is no DOMPurify in this closure; nothing in the bundle sanitizes HTML strings because nothing parses HTML into the page.

• Load path: Markdown is parsed by markdown-it with html:false (raw HTML in a file becomes text), then mapped into the schema by prosemirror-markdown. The editor DOM is rendered by prosemirror-view through the schema's toDOM.
• Paste path (the only HTML sink): prosemirror-view parses clipboard HTML in a detached document created with document.implementation.createHTMLDocument, which does not run scripts or load images, and maps it through the schema's parseDOM rules, so only schema nodes and marks survive. This is the same mechanism as the sanitizer: the schema is the allow-list.
• Measured with the pinned bundle (jsdom; results/hostile-html.txt; 12 hostile inputs through DOMParser and DOMSerializer with the stock schema):
  • Dropped: onclick, onmouseover, style attributes, <script>, <svg>, <iframe>, <style>, <form>, <object>, <math>.
  • Kept by the stock schema: <img src="x"> (the onerror is dropped but the image stays), <a href="javascript:alert(3)"> and <a href="data:text/html,…">. The stock link and image specs therefore do not meet the brief's rendering rules. Phase 2 must replace them: the link mark's parseDOM and toDOM pass the href through the same window.MosaicMarkdown.linkTarget row 148 uses (http:, https: or relative, with rel="noopener noreferrer", nothing else), and the image node is replaced by text (the alt text with the source shown), never an <img>. The paste path then cannot reintroduce either. This must be tested with the brief's <script>, onerror and javascript: browser test, on the paste path as well as the load path.
• Advisories. OSV batch query on the 29-package lockfile (results/osv-a-pm.json), 2026-10-18: 0 advisories. Positive control the same day (output not saved in the packet; repro/osv.sh has no control): dompurify 2.3.3 returned 20 and markdown-it 12.2.0 returned 3. The build tool's own lock (27 packages, esbuild 0.28.2) also returned 0 (results/osv-i-build.json). An advisory-free answer is a snapshot of that day; phase 2 should re-run it in the landing record.
• eval and network. No eval(, new Function, document.write, fetch(, XMLHttpRequest or import( in the built bundle (grep -cE returns 0 on the 17,963-line file, which contains all 132 inputs). A grep is a screen, not proof that no code path builds a function from a string by other means. The bundle makes no network call of its own.

4. Source size, and what is removed or rewritten

The built bundle reads 132 input files (131 third-party plus the 11-line prototype entry), 25,899 lines, from the esbuild metafile of the pinned build, counting each file's lines on disk. (An earlier count of mine, 127 files and 25,706 lines, used a different method and is superseded.) The prototype entry has no schema or editor code yet, so phase 2 adds to this.

Package	Files	Lines
prosemirror-view	12	6,459
prosemirror-model	12	3,574
prosemirror-transform	10	2,113
prosemirror-state	5	1,092
prosemirror-markdown	4	917
prosemirror-commands	1	786
prosemirror-history	1	466
prosemirror-schema-list, -basic, -keymap	3	538
prosemirror-inputrules	4	247
orderedmap, rope-sequence, w3c-keyname	3	444
ProseMirror side	55	16,636
markdown-it	53	6,291
entities	8	1,124
linkify-it	2	839
mdurl	5	541
punycode.js	1	443
uc.micro	7	14
markdown-it side	76	9,252
prototype entry (ours)	1	11

A reviewer reads about 25,900 lines of third-party source (about 1,200 of them are entities tables) plus the editor code and CSS we write in phase 2. For scale, the existing markdown-preview.js is 338 lines. The table counts the files the build reads, not every file in a package's directory.

What the imported directory would hold: the files above (the src/ of the ProseMirror packages, the index.mjs and lib/ of the markdown-it family, entities/src), each package's license file, and a provenance file with the package, the commit and the tree hash from section 1. What it would not hold: dist/, .d.ts/.d.cts, tests, benchmarks, READMEs, rollup.config.* and anything else the build does not read. The brief's tree-hash-vs-provenance test then hashes each package directory against the tree hash recorded in the provenance file (the tagged tree minus those excluded paths, so the test needs a rule for which paths count; I would pin the rule in the test and not in prose).

Rewritten: nothing in the imported third-party source. All our changes go in our own files: the entry module, the schema, the splice save, and the editor CSS.

Not human-readable, three generated files (results/ lists the line counts):

• entities/src/generated/decode-data-html.ts: 8 lines, 47,698 bytes, encoded strings. The tag has maps/ and scripts/write-decode-map.ts, which look like its generator (I did not run it, so I have not shown they reproduce the file).
• uc.micro/categories/P/regex.mjs (2,042 bytes, one line) and .../S/regex.mjs (2,551 bytes): generated Unicode regexes. I did not find a generator script in the tagged uc.micro tree.

The same generated files exist in every markdown-it build. Options: (a) accept them as data, with the provenance file stating they are generated (and entities' generator and input beside its table); (b) regenerate in the build. I did not run any generator, so (b) is not shown to work; (a) costs nothing and is what I would do unless a reviewer objects.

5. Build, and how a reviewer reproduces it byte for byte

Tool. esbuild 0.28.2. In my scratch build it is the only dependency of a package.json with an exact version ("esbuild":"0.28.2"); nothing in the repository pins it yet, so phase 2 would add that pin. Lockfile integrity hash sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA== (platform binary @esbuild/linux-x64 0.28.2, sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==; file sha256 e1698a3d5c6c0798fee4fd3b5cc816651f460c63d390a7a26ea4beb0b1884100, 11.4 MB, statically linked, stripped). This is a prebuilt Go binary. It is a build tool pulled by the lockfile, not vendored or committed, and the bundle it produces is what Q167 D3 requires to be rebuilt from source. Whether that satisfies D3 is Mos's call (question 2).

Recipe (repro/build-pinned.sh, run from a directory with the pinned trees checked out; no network):

esbuild entry.mjs --bundle --format=iife --target=es2022 --platform=browser \
  --legal-comments=inline --log-level=warning \
  --alias:prosemirror-{model,state,view,transform,inputrules,markdown}=<pkg>/src/index.ts \
  --alias:prosemirror-{commands,history,keymap,schema-basic,schema-list}=<pkg>/src/<name>.ts \
  --alias:orderedmap=<pkg>/index.js --alias:rope-sequence=<pkg>/index.js --alias:w3c-keyname=<pkg>/index.js \
  --alias:markdown-it=<pkg>/index.mjs --alias:mdurl=<pkg>/index.mjs --alias:uc.micro=<pkg>/index.mjs \
  --alias:linkify-it=<pkg>/index.mjs --alias:punycode.js=<pkg>/punycode.js --alias:entities=<pkg>/src/index.ts \
  --outfile=editor.js

No minify flag: the output is readable. For the ProseMirror packages the alias target is the file each package's prepare script feeds pm-buildhelper (checked for prosemirror-view: src/index.ts; the others follow the script in repro/build-pinned.sh). punycode.js is the tag's CommonJS file; esbuild wraps it. No --external, no plugins, no define. The entry in repro/entry.mjs is a measurement prototype that exports the pieces; the phase 2 entry will differ, so the hash above is the hash of the prototype, and phase 2 records its own.

Result. The pinned trees were exported with git archive <tag> (no npm packages in the build input). Two builds in a sandbox with no network gave identical output: sha256 9587a1706eb5fa8d4d2c79976be6b1d84a0513fe814eeee381b6ecf384d15077, 17,963 lines, 722,595 bytes (results/build-hashes.txt). A reviewer repeats the same recipe from the same tags and compares the hash. The earlier prototype that aliased two packages to npm dist also built identically twice but is not the plan; this one uses tag sources only. One esbuild warning appears and is harmless: prosemirror-transform's import "./mark" is ignored because the package says sideEffects:false; addMark and removeMark are still in the bundle (imported by transform.ts directly).

Caveats on the reproduction claim. My builds ran with the host's Node and the native binary in a bwrap --unshare-net sandbox. I proved identical output on this machine twice; I have not rebuilt on another machine, another architecture, or another esbuild platform package, and the macOS and Windows binaries are different files. Phase 2's test should record the hash and fail on drift. An alternative with no prebuilt tool (a small concatenating build with Node's type stripping) is possible but would be new code to review, because bare imports cannot use an import map under this CSP; I did not build it.

6. Markdown normalisation (the measured part)

Corpus: the 414 .md files in this checkout (results/corpus.txt). There is no spec/north-star.md; the nearest file is the 3-line packages/project/templates/north-star.md, which is in the corpus. Test: parse, serialise, compare bytes, no edit.

Stack	Byte-identical, no edit	Notes
ProseMirror + prosemirror-markdown, stock	1 of 414	repro/rt.mjs
Tiptap 3.32.0 + @tiptap/markdown	1 of 414	drops tables entirely
Milkdown 7.22.3, commonmark + gfm	59 of 414	142 of 414 with - bullets set
Block splice (below), A	414 of 414 (12,879 blocks, 98 with front matter)	exact by construction

What the stock serialiser changes (each seen in the corpus; small cases in repro/micro.mjs): bullets - to *; hard-wrapped paragraphs joined onto one line; tables flattened into one paragraph; setext headings to ATX; ~~~ fences and indented code to backtick fences; reference links inlined; entities decoded; _em_ to *em*; a hard break becomes backslash plus newline; ~~strike~~ and task markers escaped; YAML front matter turns into a rule plus a heading; 3+ blank lines collapse to one; CRLF collapses.

Block splice (the proposal for phase 2). The document is split into top-level blocks using markdown-it's token map (the source line range of each block). The editor holds those blocks; on save each block that is unchanged since load writes back its original source text, and only an edited block is re-serialised. Gaps between blocks (blank lines) are kept as they were. Tables, raw HTML blocks, rules and front matter (762 HTML blocks, 346 tables and 205 rules in this corpus; 98 files start with front matter) are opaque source blocks: shown, and edited as source text, never reserialised. This satisfies the brief's "same bytes if no edits".

Two honest limits:

1. The 414-of-414 figure measures the splice arithmetic (gaps plus block slices rebuild every file), not the editor. A file the editor loads and saves untouched is exact because untouched blocks emit their stored text; phase 2's round-trip test (spec/north-star.md as the brief says, which does not exist here, so a repo file chosen by Sage, plus a document with every preview element) is what proves it end to end.

2. An edited block is rewritten in the editor's canonical Markdown, not its original style. Measured on the 11,566 blocks that are not opaque (results/per-block-roundtrip.txt), serialising each block alone and comparing with its own source text:

   Block	Count	Stock serialiser exact	With soft breaks kept and - bullets
   paragraph	5,639	32%	97%
   heading	3,655	99%	99%
   bullet list	1,326	0%	85%
   fence	468	100%	100%
   indented code block	242	0%	0% (becomes a fence)
   ordered list	228	25%	84%
   blockquote	8	13%	63%
   all	11,566	52%	94%

   So with two serialiser settings, about 94% of blocks would come back unchanged even after an edit, and the other 6% are rewritten (for example indented code to a fenced block, a list's continuation indent, wrapped lines). A list or quote is one top-level block, so an edit anywhere in a list reserialises that whole list. The brief says reviewers accept "the exact normalisation"; this is that statement, and the numbers are from one repository's docs.

7. What this means for the Console

• Routing. serve.mjs serves packages/webui/src/public/ from an explicit route table (lines 24 and 25: file name to content type). A single bundle is one route (/editor.js) and one CSS file is another (/editor.css, carrying prosemirror.css's 54 lines and ours); unbundled serving would need about 131 routes or a new directory route. serve.mjs is not in the brief's owned-file list, so the two route entries need Sage's say-so.
• CSP. script-src 'self' is enough for a same-origin <script src="/editor.js">; no CSP change is planned, and if phase 2 finds one is needed it stops there, as the brief says. I have not run the bundle in a browser; jsdom was the environment, so the brief's browser test is the first real check.
• Existing code. markdown-preview.js stays; the editor calls window.MosaicMarkdown.linkTarget rather than copy the link rule.

8. Open items and risks

• Five questions for Mos, below.
• The build is proven reproducible on one machine, twice; not across machines (section 5).
• Three generated data files are not human-readable (section 4).
• Candidates D, E, G, H were not run through the round trip; B and C were. If Mos wants the choice to be between measured round trips, D and G would be next.
• The advisory check is a snapshot (2026-10-18).
• Phase 2's sizing: the editor view, schema overrides, splice save, tests and CSS are new code. I have not estimated their size.

Questions for Mos (decision 123)

1. Go on candidate A (ProseMirror + prosemirror-markdown + markdown-it; 19 MIT packages plus entities BSD-2-Clause), importing its source into one new directory in the repository?
2. Build tool. Accept esbuild 0.28.2 (prebuilt Go binary, exact-pinned with lockfile integrity, not committed) as the pinned build tool, as the brief's "builds from pinned tools" intends? Or require a no-prebuilt-tool build (more code to review, unbuilt)?
3. Generated tables. Accept the three generated data files as imported data with generator and input beside them (section 4 option (a))?
4. Normalisation. Accept that an unedited file saves byte-identical and an edited block is rewritten in canonical Markdown, with the table, raw HTML, rule and front-matter blocks edited only as source text?
5. Licensing. The repository has no root LICENSE and says UNLICENSED; the imported code is MIT and BSD-2-Clause and needs only its notices kept. Does Mos want anything stated about that before import?

For Sage: serve.mjs needs two route entries (/editor.js, /editor.css) and is outside the brief's owned files.

Packet contents

LIBRARY.md (search), REPORT.md (this file), licenses/ (10 files), results/ (pins, OSV for 8 lockfiles, build hashes, hostile-HTML run, splice and per-block round-trip output, the corpus list), repro/ (build script and entry, the measurement scripts; they carry the scratch paths I ran them from, which a reviewer must edit). Scratch lives outside the repository at /mnt/storage/scratch/researcher/q154/ and is not part of the evidence.