stack/docs/plans/2026-10-21_email-plan-della.md

Post-v1 email processing and monitoring plan (Della)

Status: proposal for Mos phase A, Vikunja 460, mos-1213. Planning only. Goal: receive mail for named agents and users through Stack-owned connections, then expose authorized reading, routing and drafting equally to every harness. Boundary: every build row below is post-v1. None is required for the 2026-10-27 MVP. This document changes neither PRD 1.10 nor the live Stack. No service, mailbox connection, live-mail read, account, DNS change or spend is authorized. R21 binds the specialist: “The email specialist reads and drafts, never sends.”

Evidence and reuse

Citations are pinned, not claims about deployed behavior. Prefixes below identify the repository and commit for every file:line citation. Both prior clones were inspected read-only. No prior application or test suite was executed.

Prefix	Pinned evidence
S	Stack c3d88840c5b6ffa62d2a6a4aa4385d388234d777, this worktree
MP	mail-processor 95b02989f4d4276dded00c90ffd74ebb7d03ae8b
EP	email-processor 7a6492190cd376058e29a04fd0593e86bae1a8e5
SM	origin/huey/system-manager-design at 2ee68cd69337bb933a8fabeb04bcd71835170171, Q198 r6, superseding r5 fd44f7c60

F1 Existing mail code: recursive comparison found identical src/ trees in MP and EP. Reuse once, not as two running products. Both implement normalized messages and a connector boundary (src/email_processor/connectors/base.py:8,30), M365 pagination/rate-limit handling (connectors/m365.py:81,138, under that same source directory), ordered AND/OR rules (src/email_processor/rule_engine.py:81,124), and bounded preview/cold-start safety (src/email_processor/poller.py:101,122,152). Port these concepts and selected synthetic tests into Stack-native modules after dependency/license review. Neither copy proves IMAP or POP3 support: connector selection handles only M365 (src/email_processor/poller.py:48).

F2 Prior intent: EP docs/PRD.md:25,39 describes a single-organization M365 application, not the Stack add-on. MP docs/PRD.md:26,66,204,235 specifies generic Graph/IMAP/POP3 drivers, separated intake/execution, fail-closed activation and atomic staging/checkpoints. Those are useful design requirements, not implemented features. Adopt the safety properties and conformance cases, not its independent tenant/profile product, bootstrap authority or mandatory standalone web setup.

F3 Drop unsafe coupling: in both repos, src/email_processor/poller.py:186,200,301 checks for duplicates before effects and advances a timestamp to wall-clock now. src/email_processor/models.py:266 has indexes, not a unique mailbox/message key. Replace these with durable provider cursors, unique identities and transactional delivery. Drop connector caches keyed just by mailbox name (poller.py:48), direct GLPI/ntfy effects (action_executor.py:30,69,159) and source category writes (connectors/m365.py:170, poller.py:359). Drop direct model calls and content logs (classifiers/llm.py:79,105,109,123), replacing decisions through Q207. Do not import deployment manifests, credential settings, existing accounts or data.

F4 Stack enforcement: S packages/harness/src/tools.mjs:1 already maps typed tools through the runner socket with broker checks. Its MCP bridge carries no capability (packages/harness/src/mcp-server.mjs:1,34). Reuse that path for all harnesses, rather than giving each one mailbox credentials or a monitoring plugin. S packages/bus/src/broker.mjs:690,701,1601,1666 supplies human-approved grant consumption and action authority. packages/bus/src/credentials.mjs:142,358,379 supplies scoped references and protected use. Mail kinds and grants do not exist: S packages/business/src/vocabulary.mjs:109,119 accepts Gitea, Vikunja and host kinds. Mail requires reviewed extensions, not relabelling mail credentials as an existing kind.

F5 Product constraints: S docs/prd/mosaic-stack.md:432,454,477,496 defines chief-raised grants, exact human approval, one-time consumption and immediate local revocation. Its :507,542,582,607 covers decision evidence, thread binding and hard waits. PRD “inbox” is the decision inbox, not email. Mail needs a separately approved post-v1 PRD addendum. S packages/plugins/src/kinds.mjs:9,32 has no mail-source kind. Follow REQ-PLUG-1/2 (S PRD :876,910) for provider adapters.

F6 Monitoring precedent: S tools/agent-watch/jarvis-email-watch-liveness.sh:8,26,53 checks timer state, failed execution and staleness. Reuse the distinction between alive, failed and unknown, not its legacy unit names or fleet wiring. SM docs/plans/2026-10-21_q198-system-manager.md:262,291,304,326 supplies protected mode, independent probes and controlled resume. It is a design dependency, not shipped proof.

Architecture and sources

D1 One owner per instance. Add an optional supervised packages/mail/ service, with a single leased owner per configured source. It owns connection pools, OAuth refresh, scheduling, cursors and intake. A harness can stop, restart or change without losing monitoring. Multiple accounts are isolated by instance/business/source ID. “One service” does not mean one global credential, mailbox or shared authorization. Default installation is disabled and opens no connection.

Use Node modules aligned with the Stack, adapting the Python design rather than embedding both applications and their separate databases/authentication/UI. A Python sidecar would reuse more syntax but duplicate operational boundaries. Harness-native connectors would repeat connection state and fail Jason's cross-harness requirement.

D2 Source contract. Propose a reviewed mail-source plugin kind, list-selected, with descriptor(), scan({source,cursor,limits,signal}), fetch({source,item,part,limits,signal}) and health({source,signal}) exports. scan returns bounded observations, continuation, terminal cursor and identity epoch. Adapters receive protected credential-use callbacks, never agent-provided secret paths. Core owns grants, leases, persistence, rule evaluation and audit. No adapter sends, deletes, labels, moves or marks messages read. Use the existing channel kind only where its dispatch contract fits presentation, not as a substitute polling interface.

Source	Proposed implementation and boundary
Microsoft 365	Graph read-only access, per-folder delta/continuations and immutable message IDs. Replace timestamp polling. Restrict application rights to named mailboxes and verify effective scope, including any additive Entra permissions.
IMAP	TLS with certificate verification, OAuth where supported, otherwise a separately approved service/app credential. Read-only selection and BODY.PEEK; UIDVALIDITY plus UID, never sequence number. IDLE is a wake-up hint with periodic reconciliation.
Other providers	Prefer a conforming IMAP source initially. Gmail API and JMAP are later adapters only when a named need justifies them. No new provider account is implied.
POP3	Deferred optional row. Require stable UIDL, TLS and read-only retrieval. No DELE. Reject automatic activation without safe identity/reconciliation.

Graph delta is folder scoped and cursors are opaque (Microsoft delta documentation). Immutable IDs survive moves within a mailbox, not archive-mailbox moves or reimports (Microsoft ID documentation). Mailbox-scoped Application RBAC and unscoped Entra grants are independent (Microsoft RBAC documentation). IMAP identity, read-only selection and PEEK semantics follow RFC 9051. These are protocol references checked 2026-10-21, not evidence of installed support.

D3 Addressing. Version a route table mapping trusted delivery evidence to a stable Stack user or role-instance ID, never a display name or a newly invented role. Aliases on an approved ingress mailbox are the recommended inexpensive start. Plus-addresses are acceptable only after fixtures and a later authorized provider test prove the exact recipient survives delivery. They are not a security boundary. Dedicated mailboxes provide stronger provider-side separation for confidential users or agents, at the cost of account management and possible licenses.

Bind routes to source identity plus provider-controlled original-recipient evidence. To/Cc, forwarded headers and arbitrary Delivered-To headers are insufficient unless the enrolled delivery path establishes their provenance. Bcc or ambiguous alias delivery goes to restricted operator review, never guessed routing. Unknown addresses do not create agents. Multiple recipients produce independently authorized deliveries without revealing other recipients' mail. Jason approves domains, names, aliases, mailboxes and each provider-side change before provisioning. No SMTP/MX server, catch-all domain or public webhook is proposed for the initial increment.

Processing, data and retention

D4 Intake pipeline: approved source → bounded discovery → bounded parse → durable observation/dedup → versioned deterministic rules → authorized delivery outbox → Stack notification/work request. Provider cursor advancement and durable observations commit together under the source lease/generation. Outbox delivery has a stable key and a recipient-side unique receipt, so a crash after delivery but before acknowledgment does not create a second task/message. Never promise exactly-once external effects.

Default candidates: poll every 60 seconds, at most 50 observations and 10 MiB fetched per slice, fair scheduling between sources. Persist continuation when the slice fills. Cap each source at 1,000 pending deliveries and 100 MiB of retained content initially. Persist quota accounting across restarts and slices. Reaching a quota holds new intake and raises an alert, while read-only health checks continue. First activation establishes a from-now baseline without actions on old mail. Historical import is a separate approved time/count window, previewed before activation. Cap violations, disk-full and unavailable audit storage hold work without advancing past unstaged mail. These limits are proposed settings to validate, not measured capacities.

Provider identity is scoped to business/source/account: Graph immutable ID, IMAP folder/UIDVALIDITY/UID or POP3 UIDL. RFC Message-ID, thread ID and content similarity are secondary correlation signals, never sole proof of uniqueness. Preserve distinct messages with missing or reused Message-ID. An IMAP move, epoch reset, expired delta cursor, archive move or reimport enters bounded reconciliation. Record aliases only with sufficient identity evidence. Otherwise hold possible duplicate actions for review. An edit is a new observation version, not automatic permission to rerun prior actions. Preserve dedup tombstones beyond content expiry. Replay needs a new authorized intent, parent link and current grants. A rule edit never retroactively executes preview work.

D5 Parsing and rules. Preserve provenance for normalized sender, recipients, timestamp, subject, text, thread references and attachment descriptors. Bound MIME depth to 10, parts to 100 and normalized text to 256 KiB. Decode safely, flag malformed or truncated content and quarantine failures. Prefer text, sanitize HTML for display, disable remote images/URLs and never execute active content. Rules use finite allowlisted fields/operators, stable priority and first-match by default. Reject executable rules and unbounded regex. Record rule/version and routing reason. Unmatched mail is held for its authorized owner, not dropped or copied fleet-wide. Routing a user notification does not launch an agent. Agent wake-up is a separate admission-controlled work request.

Optional classification and any mail-triggered decision go through the Stack's one decision module under DECISIONS Q207. The mail service neither selects nor falls back to a model locally. Core policy still determines permitted actions and human hard waits. Use minimal, explicitly permitted content and an allowlisted result schema. A model result is advisory data, never a grant. Missing model service leaves deterministic intake working and model-dependent work held. Summaries/drafts run as ordinary governed agent work, not an extra mail-service model client.

D6 Attachments. Initially enumerate metadata only. An authorized explicit fetch quarantines bytes under an opaque ID, with a 10 MiB per-part and per-message total cap. Never trust filenames as paths, unpack archives, execute macros or follow embedded links. Content extraction requires isolated no-network processing, resource bounds, scanner success and a separate read right. Failed, encrypted or unsupported files stay quarantined. A clean scan does not make attachment instructions trustworthy.

D7 Storage. Proposed <dataRoot>/mail/<business>/ contains encrypted content blobs, SQLite observation/outbox state, encrypted cursors, local drafts and append-only event evidence. No mail content enters git or images. Service-only directories are 0700, files 0600. Use scoped keyed fingerprints for private dedup fields. Separate content keys from provider slots. Keep tokens, delta URLs and authentication errors out of ordinary logs. Default durable records hold opaque IDs, state, versions and counts, not bodies, subjects, recipient lists or attachment names. Content access is explicit and audited.

Recommended content policy for an enabled pilot: encrypted bodies and local drafts for 30 days, explicitly fetched attachments for 7 days, minimal metadata/tombstones for 365 days. Without Jason's approved content/retention policy, persist metadata only and show that later content fetch requires the source online. Expired content returns an explicit unavailable result. Define backup/key expiry and restore suppression together: restoring an old backup must not resurrect expired content or redeliver old work. Local pruning needs the Stack's data.delete approval path or an explicitly approved future retention authority. Do not silently grant automatic deletion. Write receipts, preserve legal holds and do not rewrite <dataRoot>/runs/ or existing audit evidence.

Access and security

D8 Identical access contract. Proposed tools are mail_list(source?,cursor?,limit), mail_get(id,view), mail_attachment_get(id,part), mail_route(id,target,expect), mail_ack(id,expect), mail_draft_create(id,text) and mail_health(source?). All map to typed broker verbs via the existing tool socket/MCP bridge. Local CLI and authenticated Console use the same handlers. List is paginated (default 20, max 100). ack changes Stack-local state only. Drafts are local unsent records, not Graph drafts. Health returns only authorized source metadata. No arbitrary URL, shell, source secret path, provider query language or send tool is accepted.

Separate rights for metadata, body, attachment, draft and routing per business, source, address binding and named principal. Recheck current role/grant on every read, notification and action, including caches and derived summaries. Revoke immediately fences new uses, invalidates fetch handles and cancels owned in-flight reads before returning content. Agents receive only mail explicitly routed to their grant scope. User mail requires that user's or explicitly delegated access. An email specialist is not implicitly allowed to read every user's mail. Cross-business IDs return no data.

New mail access grants follow the existing chief → credential.mint → authenticated human Approve chain, with exact target, rights and expiry bound once. Add the mail grant method and action routing only through a Jason-approved PRD version and reviewed vocabulary/role changes. Until then new actions remain hard waits. Within-role mail access can become routine only inside those approved grants. Manual-mode actions still need per-use approval. A hand-edited grant or approving email creates no authority.

D9 Credentials and hostile input. Distinguish a source service principal's provider connection grant from an agent's mediated mail-access grant. Recommend dedicated read-only ingress credentials per source/business, held by protected Stack code and never shared with agents. This system-service ownership is a new approved extension, not an exception silently taken to REQ-CRED-1. Configuration names only an approved absolute slot path, for example <dataRoot>/credentials/mail/<business>/<source>/oauth; no value or partial value is read by a planner, returned through tools or embedded in a plan. Missing/expired/revoked slots fail closed. Validate owned non-symlink files, rotation and revocation behavior.

Treat mail, headers, filenames, OCR and quoted replies as untrusted external data. Sender addresses, even authenticated messages, cannot approve decisions, change policy, mint grants or impersonate the CLI/Console human. Delimit model input and strip tool authority, but rely on broker denial, sandboxing and least privilege for enforcement. An email asking an agent to fetch secrets, forward documents or run commands must not gain those rights. External action proposals use existing approval paths and provenance. No automatic link fetches. Provider endpoints are enrolled allowlists with TLS and redirect checks, not values selected from mail content. Logs use reason codes and IDs.

D10 Sending. Out of scope: SMTP submission, Graph send, forwarding, auto-replies, DSNs generated by this service and emailed digests. Block them in adapter interfaces, credentials and network policy. R21 stays intact even if a future separate sender role is approved. Human export of an unsent draft is not authorization for Stack delivery.

Monitoring and system-manager integration

D11 Health: report desired/actual connection state, last successful scan and durable cursor commit, lag, backlog, lease owner, retry-after, credential status, quarantine, outbox age and disk pressure. Never report “healthy” merely because the process lives. Candidate stale threshold: three missed scan intervals. Alert once per incident, on severity change and recovery through existing internal notifications/Console, not mail. Show alert delivery failure locally. Alerts include IDs and reason codes, no content.

Register dependency scopes mail-source/<business>/<source> with Q198's manager. Network/provider outages pause only affected intake and dependent fetch/dispatch. Independent sources and authorized stored-mail reads can continue. Disk/audit failure halts new intake/effects. Auth/config denial stays held for correction, never retries under another credential. Malformed messages quarantine individually, not as outages.

Apply Q205–Q207 through protected mode, not the human kill switch: mechanical source probes run independently at 60, 120, 180 seconds, increasing by 60 to a 600-second ceiling, indefinitely, while respecting longer Retry-After. No JEV/model availability gate and no model call for a mail health check. Mail health proves TLS/auth and scoped source access without fetching message bodies. One lease owns retries, so adapter and manager loops cannot multiply them. Disabled sources make no probes.

On recovery reconcile cursors/uncertain delivery receipts, pass one synthetic pipeline canary, then resume eligible work in manager-controlled batches without approval per session. Never send test mail or use a live message as a synthetic canary. Uncertain external effects stay fenced. Human kill remains human-only to clear. These mail extensions to SM D13–D15 need contract review and tests before operational reliance.

Ordered build rows (all post-v1, not current queue edits)

Each row is a separately reviewable deliverable. Write failing synthetic cases first, implement only that row, then run its named tests and affected existing suites. Commands below name proposed tests. They are not tests run during this planning assignment. Shared touch points include packages/{bus,business,harness,plugins,cli,webui}/. No rollout uses an implementation candidate until normal review and re-pin gates pass.

Row	Deliverable / files	Acceptance evidence
E01 post-v1	PRD addendum and source/data schemas, docs/prd/, packages/mail/schemas/. Resolve Q1–Q6 before activating corresponding behavior.	Schema fixtures reject unknown fields, missing scopes and secret values. Disabled add-on makes zero connections and v1 startup works without it. Obtain Jason's policy approval, not implementation's implied consent.
E02 post-v1	Protected source slots and mail-access grants, packages/bus/, packages/business/. Build D8/D9 on existing grant executor. Depends E01.	node --test packages/bus/tests/mail-grants.test.mjs: deny non-chief raises, nonhuman approvals, altered targets, duplicate consumption, hand grants and revoked/in-flight access. Secret sentinel absent from all outputs. Existing grant/approval suites pass.
E03 post-v1	Durable intake ledger, leases, outbox and disabled supervisor, packages/mail/src/{store,intake,service}.mjs. Depends E02.	node --test packages/mail/tests/intake.test.mjs: crash at each transaction/outbox boundary, concurrent owners, disk-full, cap exhaustion and restart. No skipped cursor or duplicate recipient effect, no source calls while disabled.
E04 post-v1	Bounded MIME parsing and quarantine, packages/mail/src/{parse,attachments}.mjs. Depends E03.	node --test packages/mail/tests/parse.test.mjs: malformed MIME, oversized/deep parts, traversal filenames, remote HTML, archives and encrypted attachments. Limits enforced before expansion, zero execution/network, explicit truncation/quarantine.
E05 post-v1	M365 driver and first mail-source kind, packages/plugins/src/kinds.mjs, packages/plugins/mail-source/m365/. Depends E03/E04.	node --test packages/mail/tests/m365.test.mjs: fake Graph pagination/delta, immutable ID moves, cursor expiry, token expiry, 429, restarts and scoped-denial cases. Read-only request allowlist, no skipped/repeated delivery. Existing plugin tests pass.
E06 post-v1	IMAP driver, packages/plugins/mail-source/imap/. Depends E05 contract.	node --test packages/mail/tests/imap.test.mjs: fake TLS/IMAP server, certificate denial, IDLE disconnect, sequence renumbering, UIDVALIDITY reset, folder moves and missed wake-ups. No Seen/STORE/EXPUNGE, uncertain identities hold effects.
E07 post-v1	Address table, bounded rules and delivery, packages/mail/src/{addresses,rules,route}.mjs. Depends E03–E06.	node --test packages/mail/tests/routing.test.mjs: aliases, plus tags, Bcc, forged recipient headers, retired agents, multiple recipients, missing/reused Message-ID and duplicate provider pages. No cross-principal disclosure, no duplicate work, no authority from sender.
E08 post-v1	Broker verbs, MCP/typed tools and local drafts, packages/bus/, packages/harness/, packages/mail/src/access.mjs. Depends E02/E07.	node --test packages/harness/tests/mail-tools.test.mjs: identical schema/results/refusals through every supported harness adapter, read/attachment/draft rights separated, manual mode and expiry enforced, send impossible. Existing MCP/tools/approval suites pass.
E09 post-v1	Decision-module integration, packages/mail/src/decisions.mjs, central decision interface. Depends E08 and approved Q198/Q207 interface.	node --test packages/mail/tests/decisions.test.mjs: model outage leaves rules active, one central fallback path, schema-invalid suggestions held, no mail-local model client, hostile mail cannot approve/grant/spend/send. Minimal-context privacy and budget checks pass.
E10 post-v1	Health, alerts and protected-mode recovery, packages/mail/src/health.mjs, Q198 manager adapter. Depends E03/E08 and implemented/verified SM D13–D15.	node --test packages/mail/tests/outage.test.mjs: fake clock/provider verifies outage scope, full cooldown sequence, Retry-After, parallel probes, no model probe, no outage DLQ, restart continuity, canary/batched resume, uncertain delivery fencing and human kill persistence.
E11 post-v1	Authorized Console/CLI mail views, retention/export/prune/restore, packages/{mail,cli,webui}/. Depends E08/E10 and Q4 authority.	Focused mail-retention and mail-ui tests prove ACLs on lists/search/export, expired-content denial, exact delete approval, holds, backup-expiry restore suppression and no old-work replay. Decision inbox remains distinct, alerts survive mail outage.
E12 post-v1	Opt-in operator runbook and acceptance packet under docs/. Depends E01–E11.	Offline end-to-end synthetic message → parse → route → authorized tools → local draft, under every supported harness. Demonstrate revoke, outage recovery and disable/rollback preserving evidence. Live pilot is a separate Jason-authorized step after named address/account/scope/retention approvals. No external mail sent.
E13 post-v1 optional	POP3 adapter, packages/plugins/mail-source/pop3/, only for an approved named need. Depends E12.	Fake server tests UIDL absence/reuse, restart and TLS failure, retrieval caps, reconciliation and zero DELE. Unsupported identity refuses activation. Other adapters remain unaffected.

Questions for Jason (recommendations, not assumed approvals)

1. Q1 Source and address pilot: which provider, domain and existing or new mailbox should receive agent/user mail? Recommend one separately approved read-only ingress mailbox, aliases for named agent/user recipients, no catch-all or MX change. Use dedicated mailboxes for confidential separation. Prove preserved delivery identity before enabling alias/plus routing. New addresses/accounts wait for Jason.
2. Q2 Grant ownership: approve a protected source service principal plus separately scoped per-agent/user mail-access grants? Recommend yes, through a post-v1 PRD amendment and existing chief/human approval model. No agent gets provider secrets or implicit access to personal mail. Enable only the explicitly requested sources.
3. Q3 Automation: may rules route mail and wake a granted agent without per-message human approval? Recommend yes for reviewed within-role routes, subject to manager admission and manual mode. Reading/drafting does not authorize requested external effects. Any model-needed decision uses the one Q207 module.
4. Q4 Content and deletion: approve the 30-day body/draft, 7-day attachment and 365-day minimal-evidence policy, including encrypted backups and deletion authority? Recommend these pilot limits, metadata-only until approved, with an exact data.delete approval binding for pruning. Do not weaken the hard-waits list. Jason must identify any business retention/hold requirements before live intake.
5. Q5 Exposure and costs: allow provider webhooks or paid mailbox infrastructure? Recommend no initially. Use outbound Graph polling/IMAP, existing internal notifications and no new exposure. Each future webhook, DNS/MX change or spend gets its own explicit authorization.
6. Q6 Sending: should any future role send email? Recommend keep sending out of this add-on. Retain R21 permanently for the specialist. Any later sender is a separate design with exact-recipient approval, scoped credentials and effect evidence, not a toggle that silently turns drafts into delivery.

Planning receipt and handoff

Verified pinned commits, source-tree equality, cited code boundaries and ten requested coverage areas. No implementation, dependency installation, legacy test execution, mail connection or live-stack change occurred. Canonical queue next della returned nothing after the linked-worktree refusal. No queue/MVP row changed. Vikunja 460 is the supplied tracking task. Mos/Sage owns any required Gitea issue and future queue enrollment because this seat has no Gitea API credential. Phase B review and phase C synthesis remain with Mos. Della has not read Ludwig's plan. This branch push delivers the planning artifact only, not approval to build or deploy it.