review: Ludwig's phase B review of Della's email plan (ludwig-0002, Vikunja 460)

docs/plans/2026-10-21_email-plan-ludwig.md

@@ -0,0 +1,218 @@
+# Email intake and monitoring add-on: plan (Ludwig)
+
+- Status: draft for phase B cross-review. Post-v1, not on the 10-27 path.
+- Rulings: DECISIONS Q208 (Stack-level mail, every harness), Q207 (one decision module), Q205/Q206 (outage recovery), R21 (the email specialist reads and drafts, never sends). Vikunja 460.
+- Base: `refactor` c3d88840c. Q198 citations are `origin/huey/system-manager-design` 2ee68cd69 (r6 in progress, so lines may move).
+- Prior work: `mosaicstack/mail-processor` 95b0298 and `mosaicstack/email-processor` 7a64921, read-only.
+- Planning only. Nothing here connects to a mailbox, reads live mail, or creates an address or account.
+
+## 1. Summary
+
+The Stack gets one mail service per instance. It is the only process that holds mail credentials and connections. It pulls mail from IMAP and Microsoft 365 (Graph) sources, stores each raw message once, applies deterministic routing rules, and delivers a pointer to the target seat or user over the bus. Agents on every harness read mail through the same broker tools that already reach Claude Code (MCP) and pi. No harness opens its own mail connection. The service never sends mail. Agents can write drafts that stay in the Stack. Sending stays a question for Jason (Q4).
+
+## 2. What exists and what to reuse
+
+### 2.1 The two repos are one codebase
+
+`mail-processor` 95b0298 is `email-processor` 7a64921 plus three commits (rehome, CI move, generic PRD). `git merge-base --is-ancestor 7a64921 95b0298` holds, and `src/` and `tests/` are identical. Read `mail-processor` only. `email-processor` adds nothing.
+
+Both are Python (FastAPI, SQLAlchemy, Vue), single tenant, Microsoft 365 only, and deployed for USC LLC (`deploy/email-processor.prod.stack.yml:28-48`, `config/config.example.yaml:12,17`). The Stack is Node with `node:sqlite` (`packages/bus/src/store.mjs:1`). Nothing ports as running code. The value is the design, the safety gates, and the tests as case lists.
+
+### 2.2 Findings in the prior work
+
+- F1. The connector contract is small and right: `poll`, `mark_processed`, `get_email`, `health_check` (`src/email_processor/connectors/base.py:30-55`). The normalized message carries `internet_message_id` and `thread_id` (`base.py:8-27`).
+- F2. Graph auth is app-only client credentials (`connectors/m365.py:47,58-60`). The retry policy is worth keeping: 401 refreshes the token, 429 honours `Retry-After`, 5xx backs off, 5 tries (`m365.py:81-111`).
+- F3. The Graph connector never fetches a body (`body_text=""`, `m365.py:128`) and treats attachments as a flag only (`m365.py:131`). It pages with `nextLink` (`m365.py:167`) and has no delta query.
+- F4. No IMAP or POP3 driver exists. There is only an enum value (`models.py:22`) and the PRD.
+- F5. The cursor is a timestamp set to `now()` after a run (`poller.py:200`). Mail that Graph indexes late is missed. Dedup is a SELECT before insert on a non-unique index (`poller.py:187-195`, `models.py:268-273`), so two runs can double-process.
+- F6. The rule engine is clean, pure logic: condition groups with AND/OR, priority order, first-match or all-match (`rule_engine.py:124-151`, 151 lines, 36 tests in `tests/test_rule_engine.py`).
+- F7. The LLM classifier pastes 1,500 characters of the body straight into the prompt with no delimiter or injection defense (`classifiers/llm.py:42-45,79-85`). It checks only that the output is JSON with a `category` key, not that the value is allowed (`llm.py:51-63,127-130`).
+- F8. Nothing sends, replies, forwards or moves mail. The only mailbox write is a category `PATCH` that overwrites existing categories (`m365.py:170-174`).
+- F9. Secrets use Fernet with the key set to SHA-256 of an env secret, with no KDF or rotation (`crypto.py:9-12`). A failed decrypt returns an empty string silently (`crypto.py:26-27`).
+- F10. `/api/health` reports the enabled flag and never touches the mailbox (`routes/health.py:16-30`). There are no metrics or alerts.
+- F11. Safety gates worth keeping: polling off by default, dry run on by default and capped at 10 messages, and a cold start begins at `now` rather than fetching history (`settings.py:51-53`, `poller.py:102-105,122-131`). They have 15 tests in `tests/test_predeploy_safety.py`.
+- F12. `docs/PRD.md` is a strong spec for a generic service, and most of it is unbuilt. Keep its Graph cursor rules (delta, immutable ids, RBAC-scoped app access, `:386-409`), IMAP UIDVALIDITY plus UID (`:410-428`), POP3 UIDL with no `DELE` (`:429-447`), the prohibited list (no send, reply, forward or delete, `:75-80`), and the executor-only action contract (`:709`).
+- F13. Secret scan of both repos, tree and full history, pattern-based: no credential value found. Hits were placeholders, `${ENV}` references, setting names and test fixtures (for example `tests/test_glpi_action.py:16-17`). Nothing to rotate. The scan is pattern-based, not a proof.
+
+### 2.3 Reuse and drop
+
+| Item | Verdict | Reason |
+|---|---|---|
+| Connector contract (F1), message shape | Reuse as design | Maps directly onto a plug-in kind |
+| Graph retry and 429 handling (F2) | Port | Proven behaviour, small |
+| Rule engine (F6) and its 36 tests | Port to JS, tests become cases | Pure, deterministic, tested |
+| Safety gates (F11) and their 15 tests | Port | Matches the Stack's fail-closed rule |
+| PRD cursor, prohibited-action and executor sections (F12) | Reuse as design | Already reviewed thinking |
+| LLM system prompt text | Reuse after hardening | Needs delimiters and enum checks (F7) |
+| Timestamp cursor, SELECT-then-insert dedup (F5) | Drop | Loses and duplicates mail |
+| `state.py`, `service.py`, `classifiers/rules.py` | Drop | v1 JSON files, USC-specific regex (`rules.py:76,186-203`) |
+| `crypto.py`, settings secret store (F9) | Drop | The Stack's credential path replaces it |
+| GLPI, ntfy, digest actions | Drop for now | USC destinations, not Stack needs |
+| Category `PATCH` (F8), health endpoint (F10) | Drop | Destructive write, false health |
+| Vue UI, OIDC auth, Alembic | Drop | The Console, SSO and bus schema replace them |
+
+The USC deployment of the Python service is outside this plan. This plan neither changes nor retires it (Q10).
+
+### 2.4 Stack parts the add-on plugs into
+
+- Plug-in kinds registry: `packages/plugins/src/kinds.mjs:9-55`. Adding a kind is a core change that lands with its first module (PRD `docs/prd/mosaic-stack.md:876-891`, REQ-PLUG-1).
+- Credentials by path: `parseCredentialRef` (`packages/business/src/credentials.mjs:23`), `storePath` (`:57`), the mode check (`:87`). The closed service list is `SERVICES` (`packages/business/src/vocabulary.mjs:109`, `credentialKind` `:119`). The broker reader `Credentials` (`packages/bus/src/credentials.mjs:112`) opens with O_NOFOLLOW and requires 0600, same uid and at most 16 KiB (`:242-253`). PRD REQ-VAR-2 (`:382`).
+- Agent tools for every harness: `typedTools` (`packages/harness/src/tools.mjs:91`) and `callTool` (`:292`). The runner forwards calls over a 0600 socket so the harness never holds a capability (`packages/harness/src/runner.mjs:10-14`). Claude Code reaches them through `packages/harness/src/mcp-server.mjs` with prefix `mcp__mosaic__` (`gate.mjs:18`). Pi reaches them through `pi-extension.mjs`. Codex has no adapter before v1 (PRD `:925`) and inherits the same tools when its adapter lands.
+- Untrusted-content marking: `WEB_UNTRUSTED` (`packages/bus/src/broker.mjs:109`), attached to results at `:2553`.
+- Bus delivery: `messages` and `deliveries` tables (`packages/bus/schema.sql:70,83`), `message.send` (`broker.mjs:2965`) and `message.receive` (`:3057`).
+- Grants and hard waits: REQ-GRANT-1 "no agent creates or writes a grant" (PRD `:432`). Hard waits (`:578-593`) already gate `message.external` (`:592`), `account.create`, `credential.mint`, `network.expose` and `spend`.
+- Decisions and the decision inbox: `decision.raise` (`broker.mjs:2726`), the `urgent_inbox` view (`schema.sql:207-211`), REQ-DEC-4 (PRD `:535`).
+- Connector precedent: the Discord connector runs authorize, journal, engine, deliver (`packages/discord/src/connector.mjs:1-30`), with replay dedup through `appendInbox` (`journal.mjs:60`) and reconnect handling (`gateway.mjs:129-156`).
+- Q198 system manager: per-service-scope protected mode (D13, `:262-289`), per-service probe schedule (D14, `:304-311`), escalation after three failed automatic exits (`:341-358`), `system.health` (`:366`), Console alerts (`:386-392`), the one decision module (`:399-411`), decision context capped at 4 KiB with a metadata allowlist (DECISIONS Q207 reading, design `:420`).
+
+Gaps the add-on must fill: no mail code exists anywhere in the Stack. The PRD has no prompt-injection requirement and no retention requirement. Q198 has no component registry, only service scopes.
+
+## 3. Mail sources and the one service
+
+### 3.1 Sources
+
+| Source | Plan | Cursor and identity | Notes |
+|---|---|---|---|
+| IMAP (any server, Gmail with app password or OAuth) | Build first | UIDVALIDITY plus UID per folder, IDLE with a poll fallback | Read with `BODY.PEEK` so `\Seen` never changes |
+| Microsoft 365 Graph | Build second | Delta query per folder, immutable ids (`Prefer: IdType="ImmutableId"`) | App-only `Mail.Read`, scoped to named mailboxes with Exchange RBAC for Applications |
+| POP3 | Do not build | UIDL | No folders, no flags, no push. Only if Jason names a POP-only mailbox |
+| JMAP (Fastmail, Stalwart) | Later module | `Email/changes` state | Same kind, no core change |
+| Inbound SMTP or LMTP listener, Graph webhooks | Do not build | n/a | Each opens a listener reachable from outside the host (`network.expose`) |
+
+D1. Sources are a new plug-in kind, `mail-source`, with exports `open`, `changes`, `fetch` and `probe`. It lands with the IMAP module (REQ-PLUG-1). Graph, and later JMAP, are modules with no core change.
+
+D2. Pull only. IMAP IDLE gives near-push latency over an outbound connection. Graph uses delta polling at a configured interval with a 60-second floor. Nothing listens for inbound connections.
+
+D3. Read-only by default. The service never flags, moves, marks read or deletes source mail. Its own cursor records progress. That removes F8's destructive write and keeps the user's mail client unchanged (Q3).
+
+Graph uses plain `fetch` plus `oauth4webapi`, which is already a pinned dependency (`package.json`). IMAP and MIME parsing need two new exact-pinned dependencies (recommend `imapflow` and `mailparser`, Q7). Writing an IMAP client in-house costs more than it saves.
+
+### 3.2 One service for every harness
+
+- The mail service is one supervised Stack process per instance, alongside the broker. It is a bus principal with its own actor `mail`. It runs only when the organization file switches on at least one `mail-source` module.
+- It alone reads mail credentials, through the broker `Credentials` reader. Credential kinds become `mail.<sourceId>`, which extends `SERVICES` and `credentialKind`. The stored file lives at `storePath` (`<configDir>/secrets/agents/<business>/<agent>/mail.<sourceId>`), or under an instance slot `secrets/mosaic-stack/mail-<sourceId>` for an instance-owned mailbox, mode 0600. Config names the path, never the value.
+- Agents never receive a credential, a server name, or a socket to the mail server. They get broker tools only (section 6). A harness-native mail connector (for example a hosted Gmail or Outlook connector in Claude or Codex) is not used for Stack work, which matches DECISIONS Q208.
+- Data ownership follows the data map: `<dataRoot>/mail/` holds `mail.db` (SQLite, owned only by the mail service) and `raw/` (write-once `.eml` blobs addressed by sha256). The bus receives pointers and metadata, never bodies.
+
+## 4. Addressing
+
+Mail reaches the Stack at an address the service reads. A route table maps a recipient address to a target. The options:
+
+| Ref | Option | Cost | Leaks | Fit |
+|---|---|---|---|---|
+| O1 | Plus-addressing on one Stack mailbox: `stack+<agent>@<domain>` | One mailbox | Agent names are guessable | Best start. No new mailbox per agent |
+| O2 | Aliases on one shared mailbox: `<agent>@<sub-domain>` | Free aliases on an M365 shared mailbox or a mail server | Cleaner addresses | Good once a domain is chosen |
+| O3 | Dedicated mailbox per agent | One account each (licence cost on M365) | None extra | Only for an agent that needs its own identity |
+| O4 | A user's own mailbox, read-only | None if already owned | Full personal mail in scope | User-specific mail, per mailbox ruling |
+
+D4. Route on the envelope recipient, not on `To:`. Bcc and list mail do not show the routed address in `To:`. The source module returns the delivered-to address from the provider (IMAP `Delivered-To` or `X-Original-To` as set by the receiving server, Graph recipient fields plus the plus-tag). Row M4 must verify, per provider, which header carries it before routing trusts it. Mail with no routable recipient goes to quarantine.
+
+D5. The route table lives in the organization file and changes only through the write path (REQ-ROLE-3), logged with who and old/new values. A route maps an address pattern to `{target: seat|role|user, senders: allowlist, wake: bool}`. An agent cannot add or widen its own route.
+
+D6. User-specific mail (O4) arrives as a digest and a Console list for that user. It never wakes an agent unless a route says so.
+
+Choices for Jason: the domain and provider that hosts agent addresses (Q1), whether to add aliases or mailboxes (Q1), and which of his own mailboxes, if any, the Stack may read (Q8). Each new address, mailbox, DNS or MX record waits for him.
+
+## 5. Processing
+
+Pipeline per message, each step journaled so a crash resumes without loss or repeat:
+
+1. Intake. The source module yields `(sourceId, folder, cursor key)`. The service fetches the raw RFC 5322 bytes, enforcing a size cap (default 25 MiB) and a per-run message cap.
+2. Store. The service writes the raw bytes once to `raw/<sha256>.eml`, then inserts a `mail_messages` row. Unique keys: `(source_id, folder, uidvalidity, uid)` for IMAP, the immutable id for Graph, and `(source_id, sha256)` for both. `INSERT ... ON CONFLICT DO NOTHING` replaces F5's SELECT-then-insert. The cursor advances only after the row commits.
+3. Parse. MIME tree, charset decode, `text/plain` preferred, HTML converted to text with hidden content removed (section 7). The service keeps `Message-ID`, `In-Reply-To`, `References`, `Authentication-Results` (SPF, DKIM, DMARC) and the envelope recipient. A parse failure moves the message to quarantine with the error class. It is never dropped.
+4. Dedup across sources. The same `Message-ID` and body hash arriving via two routes produces one message with two route hits.
+5. Rules. The ported rule engine (F6) runs on headers, auth results and route. Rules can tag, route, quarantine or hold. Rules cannot send, reply, forward or delete (F12 list).
+6. Route. The route table (D5) picks the target. An unmatched message goes to the default route, which is quarantine for a human.
+7. Model step, only when rules leave a message unclassified and the route allows it. The step calls the one decision module (DECISIONS Q207). The request carries metadata only: sender domain, auth results, subject length and a rule summary, never body text, within the 4 KiB allowlist. The answer must be one of the route's configured labels. Anything else, a low confidence score, or no module answer leaves the message in quarantine. The module's own JEV, smaller-model and rules fallback applies unchanged.
+8. Deliver. To a seat: a bus message from actor `mail` with the message id, sender, subject, auth verdict and the `MAIL_UNTRUSTED` notice. The body is never inlined. Delivery wakes the seat only when the route has `wake: true` and the sender passes the allowlist and DMARC. To a user: a digest entry and Console item. A message that needs a human decision raises a decision through `decision.raise`.
+
+Attachments are stored as separate sha256 blobs with type sniffed from content, not from the filename. Archives are not expanded and nothing is executed or rendered. Agents see name, size and sniffed type. Content comes only through `mail.attachment.get` (section 6), with a per-route type allowlist and size cap.
+
+Retention is a new PRD requirement (REQ-MAIL-RET). Raw mail and attachments are kept N days (Q6 recommends 90). Metadata and routing records are kept M days (Q6 recommends 365). Pruning follows the run-record rule: `mail prune` writes a receipt and deletes nothing that a held decision references. Drafts follow raw-mail retention.
+
+## 6. Agent access
+
+New broker verbs, added to `typedTools` so every harness sees the same tools. Claude Code sees them as `mcp__mosaic__mail_*`, pi through its extension, and Codex once its adapter exists:
+
+| Tool | Returns | Authority |
+|---|---|---|
+| `mail.list` | Messages routed to the caller, metadata only | within-role `mail.read` |
+| `mail.read` | One routed message: headers, text body, attachment list, wrapped as untrusted | within-role `mail.read` |
+| `mail.thread` | Routed messages in the same thread | within-role `mail.read` |
+| `mail.attachment.get` | One attachment written read-only into the caller's workspace | within-role `mail.attachment` |
+| `mail.draft` | Creates or updates a Stack-held draft reply. Never transmitted | within-role `mail.draft` |
+
+- D7. Visibility is per route. A seat can read only messages routed to it. There is no mailbox-wide search, no folder listing, and no access by raw provider id. The broker checks the route on every call.
+- D8. No send tool exists. `message.external` already sits on the hard-wait list (PRD `:592`), so a future send path is gated by construction. This plan adds none (R21, Q4).
+- D9. Role authority for the `mail.*` actions lives in `roles/*.json` and changes by reviewed commit. Route membership lives in the organization file. Both are needed for access.
+- Drafts live in `mail.db` and show in the Console for the human who owns the route. Saving drafts into the provider's Drafts folder needs write scope and is not planned (Q4).
+
+## 7. Security
+
+- R1. Prompt injection through mail. Every mail body is attacker-controlled.
+  - Every `mail.read` and `mail.thread` result carries `untrusted: true` and a `MAIL_UNTRUSTED` notice modeled on `WEB_UNTRUSTED` (`broker.mjs:109`).
+  - HTML-to-text drops hidden elements (`display:none`, zero-size or same-colour text, comments), zero-width and bidi control characters, and remote content. Links are listed as text and never fetched.
+  - A turn woken by mail runs with mail-turn authority: the `mail.*` read tools, `mail.draft`, `message.send` to fleet seats, and `decision.raise`. It has no git push, no credential tools and no `message.external`. Any gated action that turn wants becomes a decision tagged `source: mail`, so the human sees the mail origin.
+  - The model step (section 5, step 7) never sees body text, so a body cannot steer routing.
+  - A red-team fixture corpus (instructions in body, in HTML hidden text, in attachment names, in display names, in subject, spoofed internal sender) is part of the acceptance tests for M7 and M8.
+- R2. Spoofed senders. Wake and allowlist checks use `Authentication-Results` from the receiving server only, require DMARC pass or aligned SPF/DKIM, and ignore any `Authentication-Results` header the sender added. A failing message is delivered, if at all, as quarantine for a human.
+- R3. Attachments. They get a type allowlist, a size cap, no expansion and no execution, and land in the workspace read-only. Malware scanning is not planned (Q9 notes the option).
+- R4. Credentials. Paths only, read by the mail service through the broker reader, mode 0600. They never appear in logs, health output, tool results or errors. Errors carry a class (`auth`, `network`, `throttle`, `server`, `parse`), not provider text that might echo a secret. Graph uses `Mail.Read` application permission limited to named mailboxes by Exchange RBAC for Applications. IMAP accounts are dedicated where possible. Creating an app registration or account is `credential.mint` or `account.create`, which waits for Jason.
+- R5. Least privilege per agent. D7 route scoping plus D9 role authority. No agent sees a mailbox, only messages routed to it.
+- R6. Storage at rest. `mail.db` and `raw/` are 0700 and 0600 under the data root, owned by the instance user. Disk encryption is the host's job, as it is for run records.
+
+## 8. Sending
+
+Out of scope. R21 holds: the email specialist reads and drafts and never sends. The add-on stores drafts for a human to send from their own client. If Jason later rules sending in, it goes through `message.external` per message, with the human proof path and the decision inbox, and gets its own plan (Q4).
+
+## 9. Monitoring
+
+Each source is a Q198 service scope `mail:<sourceId>`.
+
+- Health record per source in `mail.db`: last connect, last successful fetch, cursor lag (newest server message minus newest stored), IDLE alive, error class and count, messages in quarantine, delivery backlog. `system.health` exposes these fields with no message content. The Console System view lists them.
+- Outage (network, server, throttle): the mail service reports the scope down. The system manager puts `mail:<sourceId>` into protected mode (D13) and runs its probe schedule of 60 s, rising by 60 s to a 600 s ceiling, with no attempt limit (D14, `:304-311`). The probe is mechanical: TCP and TLS connect, then IMAP `NOOP` after login or a Graph token plus one `GET /users/{id}`. It makes no model call and spends no tokens. Other scopes continue. One successful probe marks the scope online, and the service resumes from its cursor with a bounded catch-up batch. Mail is not lost during an outage because the server keeps it.
+- Throttle: wait out `Retry-After` and never shorten it (DECISIONS Q206 reading).
+- Auth failure is not an outage. A rejected credential is retried at most once, then the scope stops and raises an alert immediately. Repeated logins with a bad credential can lock the account. Alert path: Console attention plus a gated decision in `urgent_inbox`, so REQ-DEC-4 delivery reaches Jason.
+- Escalation: three failed automatic exits in one incident trigger the Q198 escalation (`:341-358`). Cursor lag above a threshold (default 30 minutes) while the scope reports up raises a warning, which catches a silent IDLE drop.
+- Dead letters: a message that fails on its own (parse error, oversize, unroutable) goes to quarantine, follows Q198 D7 dead-letter semantics, and is replayed per message by a human. An outage never sends mail to quarantine.
+
+## 10. Rows
+
+All rows are post-v1 and carry `post-v1, Vikunja 460` in the queue note. None blocks or touches an MVP row. Sizes match current rows: one piece, one gate, one reviewer.
+
+| Row | Piece | After | Acceptance tests |
+|---|---|---|---|
+| M1 | PRD addendum (design only): REQ-MAIL-1..n covering the service, the `mail-source` kind, routes, tools, untrusted marking, retention and the no-send rule | merged plan reviewed | PRD diff reviewed. Every REQ names its test row |
+| M2 | Mail store: `mail.db` schema, write-once `raw/`, unique keys, cursor-after-commit, parse and normalize, quarantine, `mail prune` with receipt | M1 | Fixture `.eml` corpus (multipart, bad charset, nested, 30 MiB, S/MIME, no Message-ID). Double insert yields one row. Crash between blob and row resumes cleanly. Prune receipt written and held items kept |
+| M3 | Rule engine port and route table in the organization file through the write path | M1 | All 36 cases from `tests/test_rule_engine.py` pass as JS cases. Route change logged with old/new. An agent write to routes is refused |
+| M4 | `mail-source` kind plus the IMAP module, read-only, `BODY.PEEK`, UIDVALIDITY plus UID, IDLE with poll fallback, envelope-recipient check | M2 | Against a local test IMAP server in the gate (no live mailbox): `\Seen` unchanged after fetch, UIDVALIDITY change refetches without duplicates, IDLE drop falls back to poll, delivered-to header verified for the test server |
+| M5 | Graph module: client credentials through `oauth4webapi`, delta per folder, immutable ids, ported 401/429/5xx policy | M4 | Against a fake Graph HTTP server: delta resume, `Retry-After` honoured exactly, 401 refresh once, token never in logs (grep of captured output) |
+| M6 | Mail service process: supervision, credential read through broker `Credentials`, polling off and dry run on by default with a 10-message cap, bus delivery with `MAIL_UNTRUSTED`, cold start at `now` | M3, M4 | Ported 15 safety-gate cases. Credential mode 0644 refused. Delivery carries no body. Service off when no module is switched on |
+| M7 | Agent tools `mail.list/read/thread/attachment.get/draft` in `typedTools`, role actions in `roles/`, route-scoped visibility | M6 | Same tool list from the MCP server and the pi extension. Seat A cannot read seat B's message by id. No send tool in any harness tool list. Attachment lands read-only |
+| M8 | Injection hardening: HTML-to-text with hidden-content removal, mail-turn authority, `source: mail` decisions, DMARC-gated wake | M7 | Red-team corpus: no hidden text reaches the result, a mail-woken turn's tool list lacks push, credentials and `message.external`, forged `Authentication-Results` ignored, spoofed internal sender quarantined |
+| M9 | Model-assisted classification through the decision module, metadata only, enum output | M8, Q198 S6 | Request never contains body bytes (assert on the built request). Out-of-enum or low-confidence answer goes to quarantine. Module unreachable falls back to rules |
+| M10 | Monitoring: health record, `system.health` fields, service scopes, mechanical probes, auth-failure stop and alert, cursor-lag warning | M6, Q198 S2 and S5 | Fake-clock probe schedule 60..600 s. Bad credential tried once, then a gated decision. Network outage leaves other scopes running. Recovery resumes from cursor without duplicates |
+| M11 | User routes: digest and Console list for user-routed mail, drafts view | M7, M10 | User-routed mail never wakes a seat without a route flag. Digest holds metadata only. Draft visible to the route owner only |
+| M12 | Live enablement on one mailbox: dry run, then canary, then live | M1-M11 and Jason's Q1, Q7, Q8 answers | Waits for Jason: address, account, credential. Dry run report reviewed before live. One week of health with no lost or duplicated message, checked against the server's message count |
+
+M5 and M9 to M11 can run in parallel once their dependencies land. M12 is the only row that touches a real mailbox.
+
+## 11. Risks
+
+- R7. Dependency surface. Two new npm packages parse hostile input. Pin exact versions, review them, and fuzz the parser path with the M2 corpus.
+- R8. The decision module is not built yet (Q198 S6). M9 waits for it. Rules alone carry routing until then.
+- R9. Plus-addresses are guessable and get scraped. Sender allowlists (D5) and DMARC-gated wake (R2) keep a scraped address from waking an agent.
+- R10. Scope creep toward a mail client. This plan has no compose UI, no send, no folder management and no search beyond routed mail.
+
+## 12. Questions for Jason
+
+- Q1. Which domain and provider host agent addresses? Recommend one Stack mailbox on a domain Jason owns personally, not the USC tenant, with plus-addressing (O1) first and aliases (O2) later. The prior work targets USC (`deploy/email-processor.prod.stack.yml:28-48`), and the Stack is not USC infrastructure.
+- Q2. Which sources are in scope? Recommend IMAP first, then Microsoft 365 Graph. No POP3 unless a named mailbox needs it. No inbound listener or webhook, so no new internet exposure.
+- Q3. Should the Stack ever change source mail (mark read, flag, move)? Recommend no. It stays read-only, with its own cursor.
+- Q4. Sending: does R21 stand for the add-on? Recommend yes. Drafts stay in the Stack, a human sends from their own client, and nothing is saved to the provider's Drafts folder. Any later send path goes through `message.external` per message with its own plan.
+- Q5. Who may wake an agent by mail? Recommend only allowlisted senders whose mail passes DMARC. Everyone else's mail goes to quarantine for a human.
+- Q6. Retention? Recommend 90 days for raw mail, attachments and drafts, and 365 days for metadata and routing records, pruned with receipts.
+- Q7. May the Stack add two exact-pinned npm dependencies for IMAP and MIME parsing (recommend `imapflow` and `mailparser`)? Recommend yes. An in-house IMAP client is a larger risk than a reviewed, pinned one.
+- Q8. Which of Jason's own mailboxes, if any, may the Stack read for user-specific mail? Recommend none at first. Start with agent addresses and add a personal mailbox later by name, read-only.
+- Q9. Attachment scanning: is a type allowlist, size cap and no-execute rule enough, or should a local scanner (ClamAV) be added? Recommend the allowlist rule now and a scanner only if attachment use grows.
+- Q10. The two Python repos: recommend archiving `email-processor` (fully contained in `mail-processor`), keeping `mail-processor` as the design reference, and leaving the USC deployment untouched by this plan. Does the USC service stay in production?

docs/plans/2026-10-21_email-review-by-ludwig.md

@@ -0,0 +1,102 @@
+# Review of Della's email plan (ludwig-0002)
+
+- Reviewed: `origin/della/email-plan` 191dff32f, `docs/plans/2026-10-21_email-plan-della.md` (341 lines), read in full.
+- Mine: `origin/ludwig/email-plan` 4a27b73ac, `docs/plans/2026-10-21_email-plan-ludwig.md`.
+- Pins: Stack c3d88840c, `mail-processor` 95b0298, `email-processor` 7a64921, Q198 2ee68cd69. Della's lines are cited as `della:N`.
+- Phase B only. My plan is unchanged.
+
+## 1. Wrong or unsupported claims
+
+I checked 44 of Della's file:line citations against the pinned commits: every Stack, `mail-processor` and Q198 line she cites, plus the two `email-processor` PRD lines. Each one points at the construct she names. The plan's evidence is sound. Findings:
+
+- F1. Nit. `della:41` cites `poller.py:186` for the duplicate check. Line 186 initializes a counter. The check is at `poller.py:187-195`.
+- F2. Defect. `della:233` proposes the slot path `<dataRoot>/credentials/mail/<business>/<source>/oauth`. That breaks the Stack's convention and is unsafe:
+  - Credentials live under the config directory: `storePath` builds `<configDir>/secrets/agents/<business>/<agent>/<service>` (`packages/business/src/credentials.mjs:57`), and instance slots follow the same pattern (`packages/webui/src/sso.mjs:33-45`, `secrets/mosaic-stack/<slot>`).
+  - `scripts/reset.sh:47` runs `rm -rf` on the data root, so a reset would delete the mail credentials along with the data.
+  - Fix: put the slot under `<configDir>/secrets/`.
+- F3. Overreach in wording. `della:220` says mail access grants "follow the existing chief → `credential.mint` → authenticated human Approve chain".
+  - The PRD's grant methods today are machine key, Gitea and GitHub (`docs/prd/mosaic-stack.md:463-475`). `credential.mint` describes making a key or token. A mail read right mints no credential.
+  - Della does say a new grant method needs a Jason-approved PRD version (`della:222`), so the claim is hedged, not false. The cleaner form is a new action, for example `mail.access`. It is a hard wait under REQ-WAIT-3 (`:607`) until a PRD version rules otherwise, and it is approved through the same inbox flow.
+- F4. Gap against the brief. The brief asked to report any secret held in either repo or its history, by path and commit. Della's plan reports no scan result. My plan has one (F13: none found, pattern-based). Graft mine.
+
+No other claim is wrong. Della keeps the Microsoft protocol references (`della:104-110`) to "protocol references, not evidence of installed support". The caveats themselves are accurate: immutable IDs do not survive archive moves, and Entra grants are independent of Exchange RBAC scoping.
+
+## 2. What Della has that mine lacks (graft into the base)
+
+- G1. Single leased owner per source (`della:76-81`). One lease owns retries, so adapter and manager loops can't multiply them (`della:270`). Mine has one process but no lease.
+- G2. Delivery outbox with a stable key and a recipient-side unique receipt (`della:133-136`). This covers a crash after delivery and before acknowledgement. Mine dedups intake only.
+- G3. Quotas. Della caps each poll at 50 messages and 10 MiB, and holds new intake at 1,000 pending deliveries or 100 MiB stored per source, with persistent accounting, an alert and continued health checks (`della:138-142`). Mine has only a message-size cap and a per-run count cap.
+- G4. History import as a separately approved, previewed window (`della:143-146`). Mine only starts from now.
+- G5. Identity reconciliation for IMAP moves, epoch resets, expired delta cursors, archive moves and reimports. It also treats a reused or missing Message-ID as a correlation signal, never proof of identity, and an edit as a new version that doesn't rerun prior actions (`della:148-156`). Mine merges by Message-ID plus body hash, which is weaker. Take hers.
+- G6. Dedup tombstones kept beyond content expiry, and suppression so restoring a backup doesn't bring back expired content or old work (`della:155,195-196`).
+- G7. Explicit MIME bounds: depth 10, 100 parts, 256 KiB of normalized text (`della:159-161`).
+- G8. Separate rights for metadata, body, attachment, draft and routing. Each read is rechecked, and a revoke cancels in-flight reads (`della:212-216`). It adds isolation across businesses (`:218`) and independent deliveries for multiple recipients without disclosing the other recipients (`:124-125`).
+- G9. Protected credential-use callbacks (`della:92`). The adapter receives a callback, never the token. This matches the broker's existing `Credentials.use(instance, service, operation)` (`packages/bus/src/credentials.mjs:379`) and its leak check `assertClean` (`:364-371`). Mine had the mail service read the file. Della's form is better, because the token never leaves protected code.
+- G10. Bcc or ambiguous delivery goes to restricted operator review (`della:122-123`). Arbitrary `Delivered-To` headers are insufficient unless the enrolled delivery path proves their provenance (`della:121-122`). Mine said to verify the header per provider. Della's rule is stricter, and correct, because a sender can add a fake `Delivered-To` header.
+- G11. Send is blocked in three places: adapter interfaces, credentials and network policy (`della:246-249`).
+- G12. Recovery canary is a synthetic message through the pipeline, never live mail or test mail (`della:273-275`). Alerts go out once per incident, on severity change and on recovery (`della:257`). Health also reports lease owner, outbox age and disk pressure (`della:253-255`).
+- G13. Tools `mail_health` and `mail_ack` (Stack-local state only), with paginated list defaults. The CLI and Console use the same handlers (`della:203-208`).
+- G14. A test that the disabled add-on makes zero connections and that v1 starts without it (`della:289`). POP3 is an optional last row (`della:301`). Rows name their test files (`della:290-301`).
+
+## 3. What mine has that Della's lacks
+
+- L1. Secret scan of both repos and their history, result reported (my F13). See F4.
+- L2. The prior work targets the USC LLC tenant (`deploy/email-processor.prod.stack.yml:28-48`). The agent addresses should not land in an employer's tenant (my Q1). Della's Q1 asks which provider but doesn't raise this.
+- L3. Concrete test ports with counts: the 36 rule-engine cases (`tests/test_rule_engine.py`) and the 15 safety-gate cases (`tests/test_predeploy_safety.py`). Della says "selected synthetic tests" (`della:30`).
+- L4. Sender authentication. Wake and allowlist checks use `Authentication-Results` from the receiving server only, and ignore any sender-supplied copy of that header. Only allowlisted senders whose mail passes DMARC can wake an agent. Della has "sender cannot approve" (`della:238-239`) but no authentication gate on wake-up.
+- L5. Concrete injection controls:
+  - an untrusted notice modeled on `WEB_UNTRUSTED` (`packages/bus/src/broker.mjs:109,2553`)
+  - removal of hidden HTML text and zero-width/bidi characters
+  - a reduced tool set for a turn woken by mail: no push, no credentials, no `message.external`
+  - `source: mail` tagging on decisions raised from mail
+  - a red-team fixture corpus as acceptance tests
+
+  Della states the principle (`della:240-242`) but not these mechanisms.
+- L6. The routing model step sends metadata only, never body text, so a body cannot steer routing. Della allows "minimal, explicitly permitted content" (`della:171`).
+- L7. A rejected login is retried once and then stops, because repeated bad logins can lock the account. Della holds auth denial for correction (`della:263-264`), which is close. Mine adds the explicit single retry and the lockout reason.
+- L8. Two questions Della doesn't ask: the new npm dependencies (`imapflow`, `mailparser`), and whether the Stack may ever write back to source mail. Della forbids write-back in her design (`della:94`) without asking.
+- L9. Stack precedents to reuse: the Discord connector's authorize, journal, engine, deliver pipeline with replay dedup (`packages/discord/src/connector.mjs:1-30`, `journal.mjs:60`), and the closed credential service list that needs a `mail.<sourceId>` kind (`vocabulary.mjs:109,119`). Della cites the vocabulary but not the connector.
+
+## 4. Disagreements and which side I take
+
+- X1. Agents rerouting one message (`mail_route`, `della:204`). **Take mine: no agent reroute.** Rerouting gives another principal read access to that message, which widens access. REQ-GRANT-2 makes Approve the only thing that creates access (`:454-456`). An injected mail asking to be "forwarded to the chief" is the obvious attack. Reroutes should be a Console or CLI human action. An agent that needs another seat's help sends a bus message, and that seat reads only what is routed to it.
+- X2. Who binds an agent to a route. Mine used role authority plus a route table in the organization file. Della uses an approved grant per agent or user (`della:212-225`). **Take Della's for the agent-to-route binding**, with my F3 wording, since the brief requires the Stack's grant and approval model. **Keep mine for the route table itself**, which maps addresses to targets and changes through the organization write path.
+- X3. Pruning. Mine had `mail prune` with a receipt. Della requires `data.delete` approval or an approved retention authority (`della:197-198`). **Take Della's.** `data.delete` is a hard wait (`docs/prd/mosaic-stack.md:588`), and no role file may grant one (REQ-WAIT-2, `:602-606`). Automatic expiry therefore needs a retention policy Jason approves in a PRD version.
+- X4. Encrypting content at rest. Mine relied on host disk encryption. Della uses encrypted blobs and cursors with separate content keys (`della:183-187`). **Take Della's.** The data root is backed up, and deleting a key is the only way an expiry also reaches the backup copies (G6). It costs a key-management row.
+- X5. Retention lengths. Mine: 90 days for content, 365 for metadata. Della: 30 days for bodies and drafts, 7 days for fetched attachments, 365 days for metadata, and metadata only until Jason approves content retention (`della:191-194`). **Take Della's.** Less stored hostile and third-party content, and the stricter default is fail-closed.
+- X6. Plus-addressing or aliases first. Mine started with plus-addressing. Della starts with aliases and allows plus-addressing only after a provider test proves the recipient survives delivery (`della:114-116`). **Take Della's.** Both plans need the recipient proof, and her order doesn't rely on it at the start.
+- X7. Row order:
+  - Della puts grants (E02) before intake (E03). Intake, parse and rules don't need agent grants, so they can run in parallel. Grants must land before any agent tool (her E08, my M7). **Take a merge:** store, parse and rules in parallel with grants. Tools come after both.
+  - Della builds M365 first (E05). I built IMAP first (M4). **Neither is right by default.** The first source is the provider Jason names in Q1.
+- X8. Model input. **Take mine** (L6) for routing classification. **Take Della's** (`della:173-174`) for summaries and drafts: those run as ordinary governed agent work, not as a model client inside the mail service. The two fit together.
+- X9. Size caps. Mine: 25 MiB per message. Della: 10 MiB per poll and per attachment (`della:139,177`). **Take Della's** smaller pilot defaults.
+
+## 5. Merged questions for Jason
+
+Each one has a recommended answer. Source codes: L is my plan, D is Della's.
+
+- Q1. Which provider and domain host agent and user addresses? (L Q1, D Q1)
+  - One read-only ingress mailbox on a domain Jason owns personally, not the USC tenant.
+  - Aliases per agent and user. Plus-addressing only after the recipient-survives-delivery test.
+  - No catch-all and no MX change. A dedicated mailbox only where confidential separation needs one.
+- Q2. Which sources, in what order? (L Q2, D table)
+  - First the provider Q1 names, then the other of IMAP and Microsoft 365.
+  - POP3, Gmail API and JMAP only for a named need.
+- Q3. Webhooks, inbound listeners or paid mailbox infrastructure? (L Q2, D Q5) No. Outbound IMAP and Graph polling only. Each future webhook, DNS or MX change, or spend gets its own authorization.
+- Q4. A Stack-held source service credential, plus separate approved per-agent and per-user mail-access grants through a post-v1 PRD addendum? (D Q2) Yes. Use a new hard-wait action approved in the inbox (F3). No agent holds provider secrets or reads personal mail implicitly.
+- Q5. May rules route mail and wake a granted agent without per-message approval? (D Q3, L Q5) Yes, for reviewed routes, and only for allowlisted senders whose mail passes DMARC. Everything else goes to quarantine for a human. A model-needed decision goes through the Q207 module.
+- Q6. May the Stack ever change source mail (mark read, flag, move, delete)? (L Q3) No. Read-only, with Stack-side cursors and acknowledgements.
+- Q7. Retention, encryption and deletion authority? (L Q6, D Q4)
+  - 30 days for encrypted bodies and drafts, 7 days for fetched attachments, 365 days for minimal metadata and tombstones.
+  - Metadata only until Jason approves this.
+  - Pruning runs under a retention policy Jason approves in the PRD, because `data.delete` stays a hard wait.
+  - Jason names any business retention or legal-hold needs before live intake.
+- Q8. Sending? (L Q4, D Q6) Out of this add-on. R21 stays permanent for the specialist. Drafts are Stack-local and a human sends them. Any future sender is a separate design using per-recipient `message.external` approval.
+- Q9. Two exact-pinned npm dependencies, `imapflow` and `mailparser`, after license review? (L Q7) Yes. An in-house IMAP and MIME parser is the larger risk.
+- Q10. Which of Jason's own mailboxes, if any, may the Stack read? (L Q8) None at first. Add one later by name, read-only, with its own grant.
+- Q11. Attachment scanning? (L Q9, D D6) An allowlisted type, size-capped raw fetch into quarantine needs no scanner. Content extraction (OCR, archives, office parsing) waits for a local scanner row, as Della's D6 requires.
+- Q12. Is the USC deployment of the Python service still in production? (L Q10) Either way this add-on leaves it untouched. Recommend archiving `email-processor` (it is contained in `mail-processor`) and keeping `mail-processor` as the design reference.
+
+## 6. Base recommendation
+
+Della's plan is the stronger base for data integrity and the grant model (G1-G10, X2-X5). Mine is stronger on injection controls, sender authentication and concrete test ports (L3-L6). I recommend Della's plan as the base, with L1-L9, X1 and the F2 fix grafted in.