gateway cannot boot in ANY shipped configuration except the federation test harness — FederationModule unconditional despite federation being designed profile-gated optional #1138

New Issue

2026-08-09T20:41:12Z

mos-dt-0 commented

2026-08-09 20:41:12 +00:00

Found by shaggy during a boot smoke of a locally-built gateway image (2026-08-09); scope corrected same day after exhaustive verification (shaggy, re-verified fred). Original filing understated this as a deployment gap; it is a repo-wide startup defect.

Defect

apps/gateway/src/app.module.ts imports and registers FederationModule unconditionally (lines 27, 56). It provides CaService, which throws at construction (ca.service.ts:250+) without:

STEP_CA_URL
STEP_CA_PROVISIONER_KEY_JSON
STEP_CA_ROOT_CERT_PATH

Measured: with BETTER_AUTH_SECRET and MOSAIC_STORAGE_TIER set, the container passes tier detection and EmbeddingService init, then dies:

ERROR [ExceptionHandler] CaServiceError: STEP_CA_URL is not set

Scope — every shipped configuration is unbootable

Exhaustive grep for an ASSIGNMENT of STEP_CA_URL (excluding node_modules) returns exactly one wiring site: tools/federation-harness/docker-compose.two-gateways.yml (:123, :204) — a test harness. (Two further hits are comment headers in federation integration tests, not runnable config.)

It is set nowhere in deploy/, docker-compose.yml, docker-compose.federated.yml, or .env.example. Corroborating:

base docker-compose.yml defines only postgres/valkey/otel-collector/jaeger — no gateway service; the documented dev path is pnpm dev on the host with no STEP_CA_* in .env.example to copy from → dies at startup.
docker-compose.federated.yml defines step-ca but NO gateway service, so it cannot serve as a wiring reference either.
deploy/portainer/federated-test.stack.yml sets none of the STEP_CA_* vars (combined with #1136, that template has never produced a working deployment).

The gateway at current head cannot boot in any configuration the repository ships, except the two-gateway federation test harness.

Fix direction — settled by the repo's own stated intent

docker-compose.federated.yml:67, verbatim:

# Profile: `federated` — this service must not start in non-federated dev.

and step-ca carries profiles: [federated]. Federation is DESIGNED to be optional. FederationModule being unconditional is a defect against the repo's own documented intent — the fix is make FederationModule registration conditional (skip with a clear "federation disabled" log line when STEP_CA_* are absent), not "deploy step-ca everywhere". Likely regressed in whichever commit made the module unconditional (not yet traced).

If a deployment later does want federation ON, infra/step-ca/ already ships init.sh, X.509 templates, and dev-password.example — that's a separate enablement decision, not part of this fix.

Related doc drift (fix alongside or split off)

docs/federation/SETUP.md (~line 61) still calls the gateway "future activated" — language predating the module going unconditional — and lists STEP_CA_PROVISIONER_PASSWORD among required vars, which the code does not require (code requires URL / KEY_JSON / ROOT_CERT_PATH).

Refs: docs/scratchpads/mosaic-woltje-stack-proposal.md §10.4 (jarvis-brain) carries the full trace. The mosaic.woltje.com refresh is blocked on this conditional fix (its stack does not want federation).

Found by shaggy during a boot smoke of a locally-built gateway image (2026-08-09); scope corrected same day after exhaustive verification (shaggy, re-verified fred). Original filing understated this as a deployment gap; it is a repo-wide startup defect. ## Defect `apps/gateway/src/app.module.ts` imports and registers `FederationModule` unconditionally (lines 27, 56). It provides `CaService`, which throws at construction (`ca.service.ts:250+`) without: - `STEP_CA_URL` - `STEP_CA_PROVISIONER_KEY_JSON` - `STEP_CA_ROOT_CERT_PATH` Measured: with `BETTER_AUTH_SECRET` and `MOSAIC_STORAGE_TIER` set, the container passes tier detection and EmbeddingService init, then dies: ``` ERROR [ExceptionHandler] CaServiceError: STEP_CA_URL is not set ``` ## Scope — every shipped configuration is unbootable Exhaustive grep for an ASSIGNMENT of `STEP_CA_URL` (excluding node_modules) returns exactly one wiring site: `tools/federation-harness/docker-compose.two-gateways.yml` (:123, :204) — a test harness. (Two further hits are comment headers in federation integration tests, not runnable config.) It is set **nowhere** in `deploy/`, `docker-compose.yml`, `docker-compose.federated.yml`, or `.env.example`. Corroborating: - base `docker-compose.yml` defines only postgres/valkey/otel-collector/jaeger — no gateway service; the documented dev path is `pnpm dev` on the host with no STEP_CA_* in `.env.example` to copy from → dies at startup. - `docker-compose.federated.yml` defines step-ca but NO gateway service, so it cannot serve as a wiring reference either. - `deploy/portainer/federated-test.stack.yml` sets none of the STEP_CA_* vars (combined with #1136, that template has never produced a working deployment). **The gateway at current head cannot boot in any configuration the repository ships, except the two-gateway federation test harness.** ## Fix direction — settled by the repo's own stated intent `docker-compose.federated.yml:67`, verbatim: ``` # Profile: `federated` — this service must not start in non-federated dev. ``` and step-ca carries `profiles: [federated]`. Federation is DESIGNED to be optional. `FederationModule` being unconditional is a defect against the repo's own documented intent — the fix is **make FederationModule registration conditional** (skip with a clear "federation disabled" log line when STEP_CA_* are absent), not "deploy step-ca everywhere". Likely regressed in whichever commit made the module unconditional (not yet traced). If a deployment later does want federation ON, `infra/step-ca/` already ships `init.sh`, X.509 templates, and `dev-password.example` — that's a separate enablement decision, not part of this fix. ## Related doc drift (fix alongside or split off) `docs/federation/SETUP.md` (~line 61) still calls the gateway "future activated" — language predating the module going unconditional — and lists `STEP_CA_PROVISIONER_PASSWORD` among required vars, which the code does not require (code requires URL / KEY_JSON / ROOT_CERT_PATH). Refs: `docs/scratchpads/mosaic-woltje-stack-proposal.md` §10.4 (jarvis-brain) carries the full trace. The mosaic.woltje.com refresh is blocked on this conditional fix (its stack does not want federation).

mos-dt-0 changed title from ~~gateway cannot boot without step-ca — FederationModule imported unconditionally; no deploy template defines a step-ca service~~ to gateway cannot boot in ANY shipped configuration except the federation test harness — FederationModule unconditional despite federation being designed profile-gated optional

2026-08-09 20:48:30 +00:00