docs: pin CHAT-00 protocol research and synthetic checks (#1507)

docs/plans/chat-00/README.md

@@ -0,0 +1,246 @@
+# CHAT-00 protocol and boundary investigation
+
+Issue #1507. Charter comment 26094. Darkwing authors; Filbert independently
+reviews the exact candidate. Rocko supplies a separate architecture review under
+comment 26098. Confirmed plan `../2026-09-13_webui-session-chat.md`, SHA-256
+`481428295199c55e0dc2f7f752b64e1dac165f02e44ceb1975004bf327513809`, published
+at `4c436f4ce493028c5f602035a29ce27d8a266d92` after investigation began.
+Owner confirmation 26093 and plan-review closure 26095 are separate from approval
+of this research. Status: research candidate, independent review pending.
+
+## Finding
+
+A mediated conversation controller is a reasonable package boundary, but raw Pi
+RPC or Claude stream-json is not that controller. Both expose engine operations
+without Mosaic's actor, scope, generation or durable request admission rules.
+Neither native response success nor process idle establishes stopped tool cohorts.
+No running seat can be adopted by starting another engine against its UUID.
+
+Protocol documentation supports a bounded adapter proposal, not a live readiness
+claim. Claude has a concrete compatibility concern: the brief observed 2.1.269,
+whereas the installed binary now reports 2.1.270. Its own help says print mode
+skips workspace trust and silently ignores invalid settings. Native approval
+callbacks do not repair that difference. Keep fail-closed preflight and effective
+configuration verification as implementation prerequisites.
+
+## Evidence and pins
+
+`sources.json` records byte hashes, versions, locations and inspected sections.
+Pi declared and installed package both read 0.85.1. README, RPC and session-format
+docs were read completely; RPC declarations and framing helper were also read.
+RPC implementation excerpts confirm prompt preflight, queue and branch commands.
+This is not a complete extension/SDK/launcher audit. The implementer must still
+read the relevant SDK/extensions/TUI documents and examples in full before choosing
+and implementing that integration. No Pi engine, loader or model was started.
+
+Claude 2.1.270 was queried only with `--version` and `--help`, using an empty
+environment and temporary HOME/config directory, without prompt/session flags.
+No actual auth file was read. The installed binary digest and help digest are
+recorded. This does not prove which binary any existing seat is running.
+
+Official Python SDK source is pinned at commit
+`37a52c9fb3f0271de017911914b0d42efea6267e`, project version 0.2.152. Its wire
+request/response implementation and relevant types were inspected, not installed
+or executed. The transport declares minimum Claude 2.0.0; that minimum is not a
+compatibility certificate for 2.1.270. Official headless and TypeScript-reference
+pages are retrieval-hash pins, not release-frozen documentation. Sections name
+minimum CLI capabilities; actual capability negotiation remains untested.
+An initial platform documentation URL returned HTML, not a usable protocol source;
+it is excluded from supporting evidence.
+
+No private history, fleet configuration, account data or live registrations were
+inventoried. Named classes below are requirements, not access grants or a count
+of verified seats. No source package, launcher or historical contract was edited.
+
+## Capability matrix
+
+Source IDs refer to `sources.json`. DOCUMENTED means a cited protocol shape or
+behavior; OBSERVED is restricted to version/help commands and pure offline checks.
+UNVERIFIED is not an optional capability.
+
+| Requirement | Pi 0.85.1 evidence | Claude evidence | Remaining proof |
+|---|---|---|---|
+| Prompt and acknowledgement | P-RPC prompt, P-TYPES, P-MODE. `id` correlates acceptance; success may mean queued or extension-handled | C-TYPES/C-QUERY control envelopes; C-REF SDKUserMessage UUID and replay; C-HELP replay-user-messages | Durable Mosaic admission and duplicate/conflicting identity rejection. Neither `id` nor UUID alone supplies exactly-once effects |
+| Incremental user/assistant/tool conversation | P-RPC message_start/update/end; contentIndex deltas; authoritative message_end; toolCallId | C-REF stream_event and complete assistant/user messages; parent_tool_use_id separates nested work | Bound per-execution stream ordering, temporary-to-persisted reconciliation and actual reported-return regression for both adapters |
+| Settled versus finished | P-RPC agent_end can precede retry/compaction/queued continuation; agent_settled means no automatic continuation | C-HEADLESS result and background-task behavior; C-REF result correlation | Settled is still not cohort termination; do not use either to release a writer claim |
+| Complete branch history | P-SESSION v3 tree, compaction/retainedTail, custom visibility; P-RPC get_entries with since and leafId, get_tree | C-REF user UUID, session_id, compaction events; C-HELP resume identity | Claude persisted branch format and exact leaf selection are not verified. Pi get_entries is append order across branches, not a safe selected-branch transcript or bounded page |
+| Read-only catalogue | Pi header/cwd/tree documented; SessionManager.open can auto-migrate old files | C-HEADLESS resume searches by ID across projects, can accept a transcript path | Separate no-write parser and explicit adoption/access mapping required. Never use a migrating loader or global recent-session search |
+| Native approval and denial | P-RPC extension_ui_request/response uses exact id; select/confirm/input/editor, timeouts and cancellation; not a universal builtin approval system | C-QUERY can_use_tool supplies tool/input/tool_use_id and allow/deny response; C-REF says callback only runs when native flow reaches a prompt | Inventory approved Pi extensions and Claude request choices. Native denies remain authoritative; no general chat yes, permission-mode switch or updatedPermissions forwarding |
+| Approval reconnect | Pi pending UI map is in one process; no documented durable pending-dialog replay command | C-REF initialize wrapper pending_permission_requests, required from CLI 2.1.268; repeats must be idempotent | Pin/verify actual reconnect behavior, persist broker pending intent, fence stale controller replies; missing list is not an empty list |
+| Ordinary Interrupt Q20 | P-RPC explicitly clear_queue BEFORE abort; abort alone continues queued work. Returned queues contain text only | C-REF direct interrupt request can use cancel_queued:true, gated by interrupt_cancel_queued_v1; SDK interrupt() does not send it. Python pinned interrupt also sends only subtype | Broker owns durable drafts and attachment identity; serialize dispatch fencing before cancellation, classify races. Cancel receipt covers UUID-bearing main-thread pending messages only, not every possible input |
+| Queue edit/cancel and takeover | Native steering/followUp queues expand commands and do not offer durable per-request edit/cancel | Interrupt receipts expose still_queued/cancelled but do not establish Mosaic edit/cancel semantics | Keep queued work in broker until dispatch; retain native queues as reconciliation evidence. Never relabel already-dispatched work unsent |
+| Images/files | P-RPC prompt images use base64 data/mimeType; no general private-file upload API | C-REF SDKUserMessage MessageParam; C-HELP --file is startup download, not browser staging; stream-json input advertised | Verify exact image/file limits and supported content schemas for installed Claude; privately stage other files through an approved bridge. Q17 caps cannot be assumed harness limits |
+| Force stop and Resume | abort waits idle; no attributable-cohort proof in RPC response | C-HEADLESS describes SIGTERM and resume of unfinished turns; no Mosaic stop proof | Supervisor cohort identity, external-effect uncertainty, durable stop/recovery transaction and per-seat approved sacrificial test. Resume can have effects; not a read operation |
+| Controller/terminal | Raw stdin is the only RPC input channel; TUI does not expose an attach-RPC command | Native attach/background/remote-control help is not a reviewed Mosaic controller interface | One broker-owned engine and mediated terminal; no second UUID writer. All old input channels must be fenced |
+| Ambient config/tool isolation | P-README disables discovery/builtin tools with flags; extensions run with full process access | C-HELP tools empty, safe-mode/restricted/bare documented; print skips trust/invalid settings; C-REF allowedTools auto-approves, not allowlists | Approved tool bridge plus execution separation; verified configuration discovery/refusal behavior. Do not substitute auth-changing bare mode or permission callbacks for this gate |
+| Fleet | No remote channel in Pi RPC itself | No reviewed private fleet mapping from local Claude help | Identity, transport, transfer and remote-cohort contracts; held behind Q22 prerequisites |
+
+## Protocol details that change the design
+
+1. Frame Pi JSON on LF only, retain split UTF-8 bytes and optional CRLF. A string
+   containing U+2028/U+2029 is one payload, not several records. Place explicit
+   size bounds above the engine. Do not expose unbounded get_tree/get_entries
+   or raw paths to a browser. A missing cursor is a reconciliation error.
+2. Pi `prompt` may invoke an extension command immediately, including while busy.
+   Skills/templates expand before queueing. A broker cannot classify all strings
+   as harmless chat merely because the RPC command is named prompt. Reviewed
+   explicit resources and command policy must cover this route too.
+3. Pi message_update contains deltas, not the former cumulative partial message.
+   Tool execution updates instead contain accumulated partialResult. Applying the
+   same append operation to both duplicates output. Reconcile with authoritative
+   message_end and later selected-branch persisted entries.
+4. Claude wire control is `{type:control_request, request_id, request:{subtype,...}}`;
+   success response correlates inside `response.request_id`. Permission allow uses
+   `updatedInput`; deny uses behavior/message and optional interrupt. Refuse
+   permission mutations beyond the exact reviewed decision, even if SDK types
+   expose updatedPermissions. Raw engine account/config fields never go to UI.
+5. Official Claude documentation exposes an important Q20 distinction. A direct
+   control request may set `cancel_queued:true` only after verifying advertised
+   `interrupt_cancel_queued_v1` and receipt support. SDK `interrupt()` alone does
+   not do this. Receipts may contain unknown external UUIDs and omit unnumbered
+   or subagent input. An empty still_queued list is not universal quiescence.
+   Missing capabilities block dependent behavior; do not rely on version comparison.
+6. Broker-owned draft records must retain text and attachment IDs. Pi clear_queue
+   returns text, not all upload metadata; Claude cancel receipts return IDs. Neither
+   is sufficient to recreate a durable draft without the original admission record.
+7. Native engine state and Mosaic policy are distinct. An engine can acknowledge a
+   prompt before the new answer appears. The required regression must show user,
+   tools and NEW assistant output in the same selected conversation, not just a
+   successful POST or a reused last-message preview.
+
+## Retained foundation requirements and proposed extensions
+
+Foundation RUNTIME sections 3-5 remain the accepted design baseline, not installed
+runtime enforcement. No change to their files is part of CHAT-00.
+
+| Requirement | Disposition | CHAT-01/03 prerequisite |
+|---|---|---|
+| Scope/access/legacy adoption, R22/R30 | RETAIN | Approved mappings and grants for each legacy conversation; UUID/cwd/basename is not a project assignment |
+| One writer and current controller generation, R24 | RETAIN | Durable claim, process incarnation, explicit transfer and controller generation check on admission AND dispatch |
+| Snapshot and selected branch at Resume, R7/R25 | RETAIN | Exact private snapshot/leaf/config pins; no autoload-migration, fallback Fresh or start-before-binding |
+| Mediated terminal, R34 | RETAIN | Terminal is another authorized client, not an independently writable native engine TUI |
+| No builtin-tool escape, reviewed bridge, no tools in credential-bearing engine | RETAIN, ENFORCEMENT UNPROVED | Separate tool executor with least privilege; every builtin/direct shell/plugin/MCP path must be disabled, mediated or explicitly refused. Native approval callbacks are insufficient |
+| No ambient extensions/context | RETAIN, COMPATIBILITY CONFLICT TO RESOLVE | Explicit manifest and discovery controls; native Claude print trust/settings behavior requires preflight/effective-state proof without dropping denies |
+| Exact decision correlation and no permission widening | RETAIN | Dialog id plus actor, intent digest, scope, tool, execution and controller binding; native deny and expired/canceled requests fence replies |
+| Invocation/effect evidence and no blind replay | RETAIN | Write-ahead intent, causal actor, dedup and uncertainty recovery. Signal/EOF/idle do not close effects |
+| Pi-only bounded text | PROPOSED EXTENSION | Claude mappings, safe images/files, content/byte limits, approvals and transcript branches; explicit schema review in CHAT-01 |
+| Browser/server durable drafts and uploads | PROPOSED EXTENSION | Private storage ownership/capacity, state transitions and crash recovery, not workspace files or immutable run-record rewrites |
+| Private remote control | PROPOSED EXTENSION, Q22 HELD | Authenticated host/seat identity and permissions plus remote stop/transfer proof. No fleet path or service authority yet |
+
+Do not solve a conflict by installing the parked registry or weakening role/policy
+contracts. Escalate an exact bounded contract disposition only when evidence shows
+the retained requirement cannot be met within a proposed implementation.
+
+## Proposed identity and access map
+
+These are design fields, not a new accepted schema. Authoritative binding should
+include opaque conversation id, approved project/workspace/seat identity, host id,
+harness/version, private source-root reference, native session id and selected
+branch, execution incarnation, supervisor cohort reference, controller generation,
+authenticated actor/connection, config/permission revision and current access grant.
+
+Required classes: repository Pi on the workstation, repository Claude on the
+workstation, local fleet Pi, remote fleet Pi, and any additional Claude/host class
+identified by the separately authorized all-seat inventory. Do not assert that an
+uninspected class is deployed or accessible. Existing Rocko launch identity is
+not proof of its current execution, model or all fleet identities.
+
+Local proposal: private Unix socket with verified OS peer identity between mediated
+terminal and broker; loopback WebUI server using authenticated, per-browser-session
+credentials and per-conversation observer/controller grants. Bind browser requests
+to exact allowed Origin and Host, reject cross-origin/CSRF mutations, avoid query
+string secrets, and use secure session lifecycle, rotation, revocation and bounded
+idle expiry. Cookie security must match the chosen local HTTPS/origin arrangement;
+loopback HTTP cannot simply be labeled secure. No credentials are created here.
+
+Remote proposal: private authenticated host channel with verified peer identity,
+explicit per-host/seat grants and revocation, plus a host supervisor that attests
+execution/cohort incarnation. SSH or mutual TLS remains a reviewed mechanism choice,
+not an installed service. Separate transport credentials from provider accounts.
+Transport connectivity alone grants neither transcript visibility nor control.
+
+Store drafts/uploads privately, keyed by actor and approved conversation identity.
+Use opaque attachment references with digest, verified bytes/type and declared
+recipient. Transfer to a remote host needs exact-seat acknowledgement and replay
+protection. Sent content follows conversation visibility. Enforce 10 files,
+20 MiB/file and 100 MiB/message plus any stricter verified engine limit; no silent
+expiry, extraction, execution or overwrite. No public staging URL or arbitrary
+browser-supplied path.
+
+## Every-ingress disposition
+
+| Ingress | Required disposition before cutover |
+|---|---|
+| Old board Reply and agent-send/tmux | Refuse delivery to migrated engine paths or route through authenticated broker admission. Existing preamble is untrusted text. Merely hiding the old button is not fencing |
+| Native TUI keyboard and external editor | Replace engine input with mediated terminal connection; observer cannot submit and takeover invalidates old connection generation |
+| Raw RPC, Claude stream-json and direct shell commands | Private supervisor-owned descriptors; allowlist translated operations, never raw client forwarding |
+| Engine follow-ups/steering, hooks and scheduled/internal input | Inventory/disable or mediate; reconcile inherited queues at handoff. No input can evade actor and scope admission |
+| Approval replies and reinitialization | Exact outstanding request and current generation; late/duplicate-conflicting replies refused, lost replies reconciled, no auto-approval |
+| Alternate launcher, crash restart, same UUID Resume | Exclusive durable writer reservation and prior-cohort proof before start; uncertainty retains claim. Record and test rollback without prompt/approval replay |
+
+## Synthetic checks and their limits
+
+Run `node docs/plans/chat-00/check.mjs`. `fixtures.json` contains synthetic cases
+only. Checks cover pinned-source identities where available, LF/UTF-8 framing,
+documented envelope shapes, Q20/takeover state examples, stale binding rejection,
+unknown delivery and stop-proof distinctions, and upload-cap arithmetic.
+They do not launch engines, read sessions/auth, make network calls or test native
+permissions, live races, private remote transport, actual image support or cutover.
+A passing miniature model is not runtime enforcement or Claude compatibility.
+
+## Blocking gates and next bounded work
+
+- B1: prove the exact Claude CLI/protocol combination, capability negotiation,
+  transcript branch selection and image/file input shapes. Public SDK/source
+  examples narrow uncertainty but do not certify the installed binary.
+- B2: prove tool-execution separation, resource manifest isolation and native
+  trust/settings refusal parity. This is a security gate, not UI polish.
+- B3: complete all-ingress writer/controller fencing and supervisor crash/stop
+  proof. No native attach or second UUID process substitutes for it.
+- B4: source/access mapping and authentication design need independent contract
+  review and any required owner grants. No private inventory has been performed.
+- B5: Q22 retains piece 5, required #1508 and fleet-launch dependencies. CHAT-04R
+  cannot proceed ahead of them; all-seat acceptance remains blocked meanwhile.
+- B6: exact per-seat live envelope and rollback approval remains separate from
+  source publication. First-seat approval never authorizes later seats.
+
+Finish independent CHAT-00 review, then propose only the bounded CHAT-01 contract
+charter with these proof gates intact. Do not silently start runtime adapters or
+other queue rows. Rocko findings and Filbert verdict will be recorded as issue
+comments, with exact research hashes, before claiming this task approved.
+
+
+## Rocko architecture review and reconciled qualifications
+
+Rocko independently reviewed the plan and foundation boundaries and returned
+findings through agent-send. He recommends distinguishing strict foundation
+workers from existing host seats, with a possible host-seat ceiling of WebUI
+powers no greater than native terminal powers. This is an owner/security-review
+proposal, NOT an accepted exception to tool isolation. Until an exact ruling,
+retain the foundation requirements and mark native parity/enforcement unproved.
+No policy change or loss of required host-seat functionality is approved here.
+
+Keep Pi and Claude proof matrices separate and propose distinct adapter fixtures
+and a mediated-terminal/old-ingress work boundary at CHAT-01. No such subtask has
+started. Any required change to excluded tmux/comms paths needs explicit bounded
+scope approval; disabling communication without a replacement does not meet the
+all-seat outcome. Observer-mode pasted messages may be retained as untrusted
+recoverable input, not silently dispatched as an authenticated controller.
+
+Four qualifications were sent back and Rocko explicitly accepted them:
+
+- Claude help's copy-when-already-running wording belongs to `--bg` with
+  `--resume`, not proof of ordinary `--resume` behavior. Exact same-identity
+  resume of a stopped seat remains a measured handoff gate. Native attach is
+  unverified, not proved impossible or accepted as an alternative controller.
+- The official TypeScript reference documents `cancel_queued:true`; the claim
+  that Claude has no queue clear was withdrawn. Exact CLI behavior is still
+  unverified, and the SDK convenience method omits that field.
+- Pi clear_queue text cannot reconstruct attachment identifiers or durable
+  metadata. The broker must retain the original queue records.
+- Disconnect is neither auto-allow nor blanket auto-deny. Keep outstanding
+  approval state under the exact native timeout/cancellation semantics and
+  reconcile it before a current-controller reply.
+
+The phase-2 Pi document pin predates 0.85.1. All citations here use the current
+sources.json hash and current sections, never inherited phase-2 line numbers.