feat(comms): P1 presence — minimal Synapse + fleet presence room + mosaic.presence heartbeat + liveness

Implements RFC-001 P1 (the first shippable slice): deterministic Matrix presence/liveness on a single-instance dev Synapse. - infra/matrix/: DEV Synapse (RFC-002 Mode B, federation OFF, self-signed TLS, enable_registration:false, appservice registration wired). Rendered from parameterized templates — zero hardcoded topology. .data is gitignored. - packages/comms/: minimal MACP presence SDK — set presence, run the mosaic.presence heartbeat (seq + interval per RFC-001 §4.5), and a deterministic liveness reader (online/away/offline from heartbeat age, NOT native-presence-timeout). Liveness core written RED-FIRST. 19 vitest tests. - tools/matrix-presence-harness/: minimal provisioner (registers >=3 agent MXIDs, creates the fleet presence room, joins them — reuses the existing @mosaicstack/appservice intent lib) + an E2E validation harness proving A2/A3/A4 against a real Synapse. - eslint.config.mjs: register packages/comms/vitest.config.ts with the type-aware project service (same as other packages' vitest configs). DEV-compose validated only; production deploy is a separate coordinated step (deploy-holds respected). Part of the comms-evolution program (RFC-001 P1) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0158NZqN2n2ymKFeJAZ4GUCb
2026-07-24 20:18:18 -05:00
parent 529c177830
commit 2e7c124e4d
33 changed files with 2140 additions and 0 deletions
--- a/infra/matrix/.gitignore
+++ b/infra/matrix/.gitignore
@@ -0,0 +1,3 @@
+# DEV runtime state: rendered config, sqlite db, signing key, self-signed
+# certs, throwaway secrets. Never committed.
+.data/
--- a/infra/matrix/README.md
+++ b/infra/matrix/README.md
@@ -0,0 +1,53 @@
+# infra/matrix — DEV Synapse for RFC-001 P1 (presence)
+
+> **DEV-ONLY. LOCAL SANDBOX.** This stack is for local presence validation. It
+> must **never** be pointed at, or run alongside, a live/production homeserver.
+> It is single-instance, federation-OFF, self-signed TLS — the smallest slice
+> RFC-002 §9 says P1 needs (Mode B single-domain, no federation, no IP-only, no
+> secret-rotation story).
+
+## What this is
+
+A single Synapse homeserver rendered from committed **templates** (no hardcoded
+topology — RFC-002 G2). Every topology fact is an environment variable with a
+dev default:
+
+| Var                  | Default            | Meaning                                             |
+| -------------------- | ------------------ | --------------------------------------------------- |
+| `MATRIX_SERVER_NAME` | `matrix.localhost` | Synapse `server_name` (the `:suffix` of every MXID) |
+| `MOSAIC_AS_ID`       | `mosaic-as`        | appservice id / registration filename               |
+| `MATRIX_HTTP_PORT`   | `18008`            | host port → Synapse 8008 (plain HTTP)               |
+| `MATRIX_TLS_PORT`    | `18448`            | host port → Synapse 8448 (self-signed TLS)          |
+
+Secrets (`as_token`, `hs_token`, Synapse macaroon/form/registration secrets)
+are **throwaway values generated at bring-up** into `.data/dev-secrets.env`
+(gitignored). In production these are crown-jewel secrets held by the
+SecretBackend (RFC-001 §8 / RFC-002 §4) — never committed.
+
+## Files
+
+- `docker-compose.dev.yml` — Synapse (+ optional Element under `--profile element`).
+- `synapse/homeserver.dev.yaml.tpl` — rendered Synapse config (Mode B, `enable_registration: false`, appservice wired, native TLS).
+- `synapse/log.config` — Synapse logging.
+- `appservice/mosaic-as.dev.yaml.tpl` — minimal AS registration (declares the `@agent-*` user namespace).
+- `dev-up.sh` / `dev-down.sh` — bring up / tear down (`--purge` wipes `.data`).
+- `.data/` — **gitignored** runtime state (rendered config, sqlite db, signing key, self-signed certs, dev secrets).
+
+## Usage
+
+```bash
+./dev-up.sh                       # render config, gen signing key + TLS cert, boot Synapse
+# ... run the validation harness (tools/matrix-presence-harness/run.sh) ...
+./dev-down.sh            # stop, keep .data
+./dev-down.sh --purge   # stop and wipe .data for a pristine next boot
+
+# optional human view (A4) — Element pointed at the dev server:
+docker compose -f docker-compose.dev.yml --profile element up -d element
+#   -> http://127.0.0.1:18080
+```
+
+## Acceptance evidence (A1)
+
+- TLS (self-signed) reachable: `curl -sk https://127.0.0.1:18448/_matrix/client/versions` → `200`.
+- Open registration OFF: `POST /_matrix/client/v3/register` → `M_FORBIDDEN "Registration has been disabled"`.
+- AS-token registration bypasses the flag by design (that is how agents are provisioned).
--- a/infra/matrix/appservice/mosaic-as.dev.yaml.tpl
+++ b/infra/matrix/appservice/mosaic-as.dev.yaml.tpl
@@ -0,0 +1,30 @@
+# ============================================================================
+# mosaic-as.dev.yaml.tpl — Mosaic Appservice registration (DEV)
+# ============================================================================
+#
+#   *** DEV-ONLY. The tokens below are throwaway placeholders rendered at
+#       bring-up by dev-up.sh. NEVER commit real hs_token/as_token — in
+#       production they are crown-jewel secrets held by the SecretBackend
+#       (RFC-001 §8, RFC-002 §4). ***
+#
+# This is the MINIMAL P1 registration: it declares the @agent-* user
+# namespace so the P1 provisioner can register a few virtual agent MXIDs and
+# carry heartbeats. It intentionally does NOT model the full P2 taxonomy /
+# token-minting appservice.
+#
+# Rendered by infra/matrix/dev-up.sh (envsubst -> .data/${MOSAIC_AS_ID}.yaml).
+# ----------------------------------------------------------------------------
+id: "${MOSAIC_AS_ID}"
+url: null                      # P1: provisioner drives the AS API directly; Synapse pushes no txns.
+as_token: "${MOSAIC_AS_TOKEN}"
+hs_token: "${MOSAIC_HS_TOKEN}"
+sender_localpart: "mosaic-as"
+rate_limited: false
+namespaces:
+  users:
+    - exclusive: true
+      regex: "@agent-.*:${MATRIX_SERVER_NAME}"
+  aliases:
+    - exclusive: false
+      regex: "#mosaic-.*:${MATRIX_SERVER_NAME}"
+  rooms: []
--- a/infra/matrix/dev-down.sh
+++ b/infra/matrix/dev-down.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# dev-down.sh — tear down the DEV Synapse (matrix-p1-dev). DEV-ONLY.
+#   ./dev-down.sh            # stop + remove containers, keep ./.data
+#   ./dev-down.sh --purge    # also delete ./.data (fresh next bring-up)
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+docker compose -f "${HERE}/docker-compose.dev.yml" --profile element down -v --remove-orphans || true
+if [[ "${1:-}" == "--purge" ]]; then
+  rm -rf "${HERE}/.data"
+  echo "[dev-down] purged ${HERE}/.data"
+fi
+echo "[dev-down] matrix-p1-dev stopped"
--- a/infra/matrix/dev-up.sh
+++ b/infra/matrix/dev-up.sh
@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+# ============================================================================
+# dev-up.sh — bring up the DEV Synapse for RFC-001 P1 (presence)
+# ============================================================================
+#
+#   *** DEV-ONLY. LOCAL SANDBOX. This script must never be pointed at live
+#       infra. It writes only into infra/matrix/.data (gitignored) and drives
+#       a dedicated compose project (matrix-p1-dev). ***
+#
+# It renders the Synapse config + appservice registration from the committed
+# templates (envsubst), generates a DEV signing key + self-signed TLS cert,
+# and starts Synapse. All topology facts come from the environment with dev
+# defaults (RFC-002 G2: nothing hardcoded).
+# ----------------------------------------------------------------------------
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+DATA="${HERE}/.data"
+COMPOSE=(docker compose -f "${HERE}/docker-compose.dev.yml")
+
+# ---- Topology inputs (dev defaults; override via env) ----------------------
+export MATRIX_SERVER_NAME="${MATRIX_SERVER_NAME:-matrix.localhost}"
+export MOSAIC_AS_ID="${MOSAIC_AS_ID:-mosaic-as}"
+export MATRIX_HTTP_PORT="${MATRIX_HTTP_PORT:-18008}"
+export MATRIX_TLS_PORT="${MATRIX_TLS_PORT:-18448}"
+
+# ---- DEV secrets (throwaway; regenerated if absent) ------------------------
+SECRETS_ENV="${DATA}/dev-secrets.env"
+mkdir -p "${DATA}"
+if [[ ! -f "${SECRETS_ENV}" ]]; then
+  {
+    echo "MOSAIC_AS_TOKEN=devas_$(openssl rand -hex 16)"
+    echo "MOSAIC_HS_TOKEN=devhs_$(openssl rand -hex 16)"
+    echo "SYNAPSE_REG_SHARED_SECRET=devreg_$(openssl rand -hex 16)"
+    echo "SYNAPSE_MACAROON_SECRET=devmac_$(openssl rand -hex 16)"
+    echo "SYNAPSE_FORM_SECRET=devform_$(openssl rand -hex 16)"
+  } > "${SECRETS_ENV}"
+  echo "[dev-up] generated throwaway DEV secrets -> ${SECRETS_ENV}"
+fi
+# shellcheck disable=SC1090
+set -a; source "${SECRETS_ENV}"; set +a
+
+# DEV: let the synapse container user (uid 991) write the sqlite db / media.
+chmod 0777 "${DATA}" || true
+
+# ---- Render config from templates ------------------------------------------
+envsubst < "${HERE}/synapse/homeserver.dev.yaml.tpl" > "${DATA}/homeserver.yaml"
+envsubst < "${HERE}/appservice/mosaic-as.dev.yaml.tpl" > "${DATA}/${MOSAIC_AS_ID}.yaml"
+cp "${HERE}/synapse/log.config" "${DATA}/log.config"
+echo "[dev-up] rendered homeserver.yaml + ${MOSAIC_AS_ID}.yaml (server_name=${MATRIX_SERVER_NAME})"
+
+# ---- DEV signing key -------------------------------------------------------
+# Generate as the synapse container user (uid 991) so the running container
+# can read it; owned-by-991 mode-600 is fine (the container IS 991).
+SIGNING_KEY="${DATA}/${MATRIX_SERVER_NAME}.signing.key"
+if [[ ! -f "${SIGNING_KEY}" ]]; then
+  docker run --rm --user 991:991 -v "${DATA}:/data" --entrypoint generate_signing_key \
+    matrixdotorg/synapse:latest -o "/data/${MATRIX_SERVER_NAME}.signing.key"
+  echo "[dev-up] generated DEV signing key"
+fi
+
+# ---- DEV self-signed TLS cert (A1) -----------------------------------------
+if [[ ! -f "${DATA}/dev-cert.pem" ]]; then
+  openssl req -x509 -newkey rsa:2048 -nodes \
+    -keyout "${DATA}/dev-key.pem" -out "${DATA}/dev-cert.pem" \
+    -days 90 -subj "/CN=${MATRIX_SERVER_NAME}" \
+    -addext "subjectAltName=DNS:${MATRIX_SERVER_NAME},DNS:localhost,IP:127.0.0.1" >/dev/null 2>&1
+  echo "[dev-up] generated DEV self-signed TLS cert for ${MATRIX_SERVER_NAME}"
+fi
+# Files hermes owns must be world-readable so the synapse (uid 991) container
+# can read them (DEV-only; the signing key stays owned by 991 from above).
+chmod 0644 "${DATA}/dev-cert.pem" "${DATA}/dev-key.pem" \
+  "${DATA}/homeserver.yaml" "${DATA}/${MOSAIC_AS_ID}.yaml" "${DATA}/log.config" || true
+
+# ---- Element config (optional human view, A4) ------------------------------
+cat > "${DATA}/element-config.json" <<JSON
+{
+  "default_server_config": {
+    "m.homeserver": {
+      "base_url": "https://localhost:${MATRIX_TLS_PORT}",
+      "server_name": "${MATRIX_SERVER_NAME}"
+    }
+  },
+  "disable_guests": true,
+  "brand": "Mosaic P1 (DEV)"
+}
+JSON
+
+# ---- Boot ------------------------------------------------------------------
+"${COMPOSE[@]}" up -d synapse
+
+echo -n "[dev-up] waiting for Synapse health"
+for _ in $(seq 1 60); do
+  status="$("${COMPOSE[@]}" ps --format '{{.Health}}' synapse 2>/dev/null || true)"
+  if [[ "${status}" == "healthy" ]]; then echo " ... healthy"; break; fi
+  echo -n "."; sleep 2
+done
+
+if ! curl -fsS "http://127.0.0.1:${MATRIX_HTTP_PORT}/health" >/dev/null 2>&1; then
+  echo "[dev-up] ERROR: Synapse did not become healthy" >&2
+  "${COMPOSE[@]}" logs --tail 60 synapse >&2 || true
+  exit 1
+fi
+
+echo "[dev-up] Synapse UP:"
+echo "         plain HTTP : http://127.0.0.1:${MATRIX_HTTP_PORT}"
+echo "         TLS        : https://127.0.0.1:${MATRIX_TLS_PORT} (self-signed, DEV)"
+echo "         server_name: ${MATRIX_SERVER_NAME}   registration: OFF"
--- a/infra/matrix/docker-compose.dev.yml
+++ b/infra/matrix/docker-compose.dev.yml
@@ -0,0 +1,60 @@
+# ============================================================================
+# docker-compose.dev.yml — Synapse DEV homeserver for RFC-001 P1 (presence)
+# ============================================================================
+#
+#   *** DEV-ONLY. LOCAL SANDBOX. Do NOT point this at, or run alongside, any
+#       live/production homeserver. ***
+#
+# Single-instance Synapse (RFC-002 Mode B, federation OFF) + an optional
+# Element web client for the human view (A4). Brought up by ./dev-up.sh, which
+# renders config into ./.data (gitignored) first.
+#
+# Isolated by design: dedicated project name + network + high host ports so it
+# never collides with other stacks on the box (A5).
+#
+#   Host ports:  ${MATRIX_HTTP_PORT:-18008}  -> Synapse 8008 (plain HTTP)
+#                ${MATRIX_TLS_PORT:-18448}   -> Synapse 8448 (self-signed TLS)
+#                ${ELEMENT_PORT:-18080}      -> Element web (profile: element)
+# ----------------------------------------------------------------------------
+name: matrix-p1-dev
+
+services:
+  synapse:
+    image: matrixdotorg/synapse:latest
+    container_name: matrix-p1-synapse
+    restart: 'no'
+    # Run against the rendered config in /data (dev-up.sh puts it there).
+    environment:
+      SYNAPSE_CONFIG_DIR: /data
+      SYNAPSE_CONFIG_PATH: /data/homeserver.yaml
+    volumes:
+      - ./.data:/data
+    ports:
+      - '127.0.0.1:${MATRIX_HTTP_PORT:-18008}:8008'
+      - '127.0.0.1:${MATRIX_TLS_PORT:-18448}:8448'
+    networks:
+      - matrix-p1-net
+    healthcheck:
+      test: ['CMD-SHELL', 'curl -fsS http://localhost:8008/health || exit 1']
+      interval: 3s
+      timeout: 5s
+      retries: 40
+      start_period: 5s
+
+  # Optional human view (A4). `docker compose --profile element up -d element`.
+  element:
+    image: vectorim/element-web:latest
+    container_name: matrix-p1-element
+    restart: 'no'
+    profiles: [element]
+    volumes:
+      - ./.data/element-config.json:/app/config.json:ro
+    ports:
+      - '127.0.0.1:${ELEMENT_PORT:-18080}:80'
+    networks:
+      - matrix-p1-net
+
+networks:
+  matrix-p1-net:
+    name: matrix-p1-net
+    driver: bridge
--- a/infra/matrix/synapse/homeserver.dev.yaml.tpl
+++ b/infra/matrix/synapse/homeserver.dev.yaml.tpl
@@ -0,0 +1,84 @@
+# ============================================================================
+# homeserver.dev.yaml.tpl — Synapse DEV homeserver config (RFC-002 Mode B)
+# ============================================================================
+#
+#   *** DEV-ONLY. NOT FOR PRODUCTION. ***
+#
+# This is the RFC-001 P1 (presence) single-instance homeserver. It is
+# RFC-002 "Mode B — single-domain" (server_name == homeserver host), with
+# federation OFF (standalone), which is all P1 needs (RFC-002 §9 P1 row).
+#
+# NOTHING here is a production value. Every topology fact is a variable
+# rendered by dev-up.sh from the environment; there is no baked-in fleet
+# domain (RFC-002 G2 "zero hardcoded topology"). The secrets below are
+# throwaway DEV placeholders rendered at bring-up — never reuse them.
+#
+# Rendered by:  infra/matrix/dev-up.sh   (envsubst -> .data/homeserver.yaml)
+# Variables:    MATRIX_SERVER_NAME, MOSAIC_AS_ID (+ dev secrets)
+# ----------------------------------------------------------------------------
+
+server_name: "${MATRIX_SERVER_NAME}"
+pid_file: /data/homeserver.pid
+report_stats: false
+suppress_key_server_warning: true
+
+# --- Listeners -------------------------------------------------------------
+# 8008: plain HTTP (client + federation) for in-container/local tooling.
+# 8448: TLS (self-signed in DEV) — satisfies A1 "reachable over TLS".
+listeners:
+  - port: 8008
+    type: http
+    tls: false
+    bind_addresses: ['0.0.0.0']
+    x_forwarded: true
+    resources:
+      - names: [client, federation]
+        compress: false
+
+  - port: 8448
+    type: http
+    tls: true
+    bind_addresses: ['0.0.0.0']
+    resources:
+      - names: [client, federation]
+        compress: false
+
+# --- DEV TLS (self-signed; generated by dev-up.sh) -------------------------
+tls_certificate_path: "/data/dev-cert.pem"
+tls_private_key_path: "/data/dev-key.pem"
+
+# --- Store: sqlite is the simplest dev store (RFC-002 §1 allows it) --------
+database:
+  name: sqlite3
+  args:
+    database: /data/homeserver.db
+
+log_config: "/data/log.config"
+media_store_path: /data/media_store
+signing_key_path: "/data/${MATRIX_SERVER_NAME}.signing.key"
+
+# --- Hardening (RFC-001 §8 / RFC-002 §8.5), rendered so a dev gets it ------
+# Registration is OFF: agents come ONLY via the appservice (A1). AS user
+# registration bypasses this flag, which is exactly the design.
+enable_registration: false
+enable_registration_without_verification: false
+registration_shared_secret: "${SYNAPSE_REG_SHARED_SECRET}"
+macaroon_secret_key: "${SYNAPSE_MACAROON_SECRET}"
+form_secret: "${SYNAPSE_FORM_SECRET}"
+
+# Presence EDUs ON so Element shows the native dot for humans (RFC-001 §4.5);
+# the AUTHORITATIVE liveness is still the mosaic.presence heartbeat.
+presence:
+  enabled: true
+
+# --- Federation: OFF for P1 (standalone). Empty whitelist = federate with
+# nobody (RFC-002 §2.4 / NG5). No public-network federation. ----------------
+federation_domain_whitelist: []
+trusted_key_servers: []
+
+# --- Appservice registration wired in (A1). The file is rendered next to
+# this one by dev-up.sh. -----------------------------------------------------
+app_service_config_files:
+  - "/data/${MOSAIC_AS_ID}.yaml"
+
+# Keep default rate-limiting ON (RFC-001 §8). No overrides here.
--- a/infra/matrix/synapse/log.config
+++ b/infra/matrix/synapse/log.config
@@ -0,0 +1,16 @@
+# Synapse DEV log config. DEV-ONLY.
+version: 1
+formatters:
+  precise:
+    format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
+handlers:
+  console:
+    class: logging.StreamHandler
+    formatter: precise
+loggers:
+  synapse.storage.SQL:
+    level: WARNING
+root:
+  level: INFO
+  handlers: [console]
+disable_existing_loggers: false