From 373e4558a38c7647418f1010504f2105c3d73b61 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Fri, 22 May 2026 11:58:27 -0500 Subject: [PATCH 1/2] chore(framework): canonize Vault-as-SSOT + ESO-default secrets policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Encodes operator-approved (Jason, 2026-05-22) secrets policy as binding framework rules across all Mosaic agent sessions and projects. Changes: - STANDARDS.md: add "Secrets handling (HARD RULE)" subsection under Non-Negotiables — Vault as SSOT, ESO bridge as default, Direct-Vault opt-in only, forbidden ${VAR:-default} for required values, forbidden .env in prod, required startup schema validation - VAULT-SECRETS.md: add four new sections — architecture decision matrix (ESO vs Direct-Vault), full ESO bridge worked example (Vault path + ExternalSecret + Deployment YAML + zod/pydantic/Go validators), Direct-Vault opt-in pattern (AppRole provisioning + ESO bootstrap for chicken-and-egg), and forbidden patterns CI lint targets - BOOTSTRAP.md: add "Secrets Bootstrap" required subsection with checklist for new apps (Vault path, README docs, ExternalSecret, secretKeyRef, schema validator, Direct-Vault justification) All duplicate file paths kept in sync (md5-equal pairs): guides/ <-> packages/mosaic/framework/guides/ packages/mosaic/framework/defaults/STANDARDS.md (single copy in repo) Co-Authored-By: Claude Opus 4.7 --- guides/BOOTSTRAP.md | 20 + guides/VAULT-SECRETS.md | 361 ++++++++++++++++++ .../mosaic/framework/defaults/STANDARDS.md | 10 + packages/mosaic/framework/guides/BOOTSTRAP.md | 20 + .../mosaic/framework/guides/VAULT-SECRETS.md | 361 ++++++++++++++++++ 5 files changed, 772 insertions(+) diff --git a/guides/BOOTSTRAP.md b/guides/BOOTSTRAP.md index 1c74b6d..b750eb4 100755 --- a/guides/BOOTSTRAP.md +++ b/guides/BOOTSTRAP.md @@ -453,6 +453,26 @@ Initialize standard labels and the first pre-MVP milestone: --- +## Secrets Bootstrap (Required for Every New App) + +Every new application MUST complete the following secrets bootstrap before deploying to any non-local environment. This is a hard gate — deployment without completed secrets bootstrap is forbidden. + +### Secrets bootstrap checklist + +- [ ] Vault path created: `vault kv put secret/k3s// ...` with all required secret fields +- [ ] Required secrets listed in project README under a "Secrets architecture" section, including: + - Vault path(s) used + - All required secret keys and their purpose + - Whether the app uses ESO bridge (default) or Direct-Vault (opt-in, with justification) +- [ ] `external-secret.yaml` manifest committed to repo's `deploy/` or `k8s/` directory +- [ ] Deployment YAML references the synced k8s Secret via `secretKeyRef` (not raw env vars or `.env` files) +- [ ] App startup has schema-based validation for all required env vars (zod / pydantic / envconfig equivalent) that exits non-zero on missing required values +- [ ] Direct-Vault opt-in (if applicable): justification documented in README + AppRole provisioned + bootstrap credentials stored in Vault and synced via a separate `ExternalSecret` + +See `~/.config/mosaic/guides/VAULT-SECRETS.md` for full worked examples of the ESO bridge pattern, the Direct-Vault opt-in pattern, and the forbidden antipatterns. + +--- + ## Checklist After bootstrapping, verify: diff --git a/guides/VAULT-SECRETS.md b/guides/VAULT-SECRETS.md index b1dda96..3692b36 100644 --- a/guides/VAULT-SECRETS.md +++ b/guides/VAULT-SECRETS.md @@ -203,3 +203,364 @@ Error: token expired 3. **Audit logging** - All access is logged; act accordingly 4. **No local copies** - Don't store secrets in files or env vars long-term 5. **Rotate on compromise** - Immediately rotate any exposed secrets + +--- + +## Secrets Architecture Decision Matrix + +Use this table to choose between the ESO bridge (default) and Direct-Vault (opt-in) patterns for every new app or integration. + +| Factor | ESO Bridge (default) | Direct-Vault (opt-in) | +| --- | --- | --- | +| **Use-case** | All static secrets (DB creds, API keys, signing keys, OAuth secrets) | Dynamic creds with short TTLs (DB rotation, AWS STS, PKI), per-request audit trails, or lease renewal mid-pod-lifecycle | +| **App code change** | None — reads standard env vars via `secretKeyRef` | Requires Vault client (`hvac`, `node-vault`, `vault/api`) in application code | +| **Secret rotation** | ESO re-syncs on Vault write; pod restart or secret refresh picks up new value | App manages lease renewal or re-auth within the running process | +| **Audit granularity** | Access logged at Vault when ESO syncs; no per-request app audit | Every app request to Vault is a separate audit log entry | +| **Operational burden** | Low — ESO handles polling, sync, and k8s Secret lifecycle | Higher — app must handle auth, lease renewal, error paths, and token rotation | +| **Justification required?** | No — this is the default | Yes — document in project README under "Secrets architecture" | +| **Example use cases** | Web app DB password, OAuth client secret, JWT signing key, API token | HashiCorp DB secrets engine with 15-min TTL leases, AWS STS assume-role, Vault PKI short-lived certs | + +**Decision rule:** If you are unsure, use ESO. Only justify Direct-Vault when the secret cannot be safely stored in a k8s Secret (too short-lived, per-request TTL required, or mid-lifecycle renewal needed). + +--- + +## ESO Bridge Pattern (Default) + +This is the required default for all k8s workloads. Follow this exact pattern unless a documented dynamic-secrets requirement justifies Direct-Vault. + +### 1. Provision Vault path + +```bash +# Write the secrets for the app (run once; use IaC/Terraform for repeatable provisioning) +vault kv put secret/k3s/ \ + db_password="..." \ + api_key="..." \ + jwt_secret="..." +``` + +Use the canonical path structure: `secret/k3s/` for k3s cluster workloads. + +### 2. ExternalSecret manifest + +Commit this to the repo's `deploy/` or `k8s/` directory: + +```yaml +# deploy/external-secret.yaml +apiVersion: external-secrets.io/v1beta1 +kind: ExternalSecret +metadata: + name: -secrets + namespace: +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-backend # ClusterSecretStore name — verify with cluster admin + kind: ClusterSecretStore + target: + name: -secrets # k8s Secret name that will be created + creationPolicy: Owner + data: + - secretKey: DB_PASSWORD # key in the k8s Secret + remoteRef: + key: secret/k3s/ # Vault path + property: db_password # field within the Vault secret + - secretKey: API_KEY + remoteRef: + key: secret/k3s/ + property: api_key + - secretKey: JWT_SECRET + remoteRef: + key: secret/k3s/ + property: jwt_secret +``` + +### 3. Deployment manifest — reference synced k8s Secret + +```yaml +# deploy/deployment.yaml (env section) +env: + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: -secrets # matches ExternalSecret target.name + key: DB_PASSWORD + - name: API_KEY + valueFrom: + secretKeyRef: + name: -secrets + key: API_KEY + - name: JWT_SECRET + valueFrom: + secretKeyRef: + name: -secrets + key: JWT_SECRET + - name: PORT + value: "3000" # safe-default: non-secret, no Vault needed +``` + +### 4. App-side schema validation — TypeScript (zod) + +Validate all required env vars at startup. Exit non-zero on missing values. + +```typescript +// src/env.ts +import { z } from 'zod'; + +const envSchema = z.object({ + DB_PASSWORD: z.string().min(1, 'DB_PASSWORD is required'), + API_KEY: z.string().min(1, 'API_KEY is required'), + JWT_SECRET: z.string().min(32, 'JWT_SECRET must be at least 32 chars'), + PORT: z.coerce.number().default(3000), + NODE_ENV: z.enum(['development', 'production', 'test']).default('production'), +}); + +const result = envSchema.safeParse(process.env); +if (!result.success) { + console.error('Missing or invalid environment variables:'); + console.error(result.error.flatten().fieldErrors); + process.exit(1); +} + +export const env = result.data; +``` + +### 4b. App-side schema validation — Python (pydantic) + +```python +# src/config.py +from pydantic_settings import BaseSettings, SettingsConfigDict + +class Settings(BaseSettings): + db_password: str + api_key: str + jwt_secret: str + port: int = 3000 + node_env: str = "production" + + model_config = SettingsConfigDict(env_file=None) # no .env in prod + +try: + settings = Settings() +except Exception as e: + import sys + print(f"Missing or invalid environment variables: {e}", file=sys.stderr) + sys.exit(1) +``` + +### 4c. App-side schema validation — Go (envconfig) + +```go +// config/config.go +package config + +import ( + "fmt" + "os" + "github.com/kelseyhightower/envconfig" +) + +type Config struct { + DBPassword string `envconfig:"DB_PASSWORD" required:"true"` + APIKey string `envconfig:"API_KEY" required:"true"` + JWTSecret string `envconfig:"JWT_SECRET" required:"true"` + Port int `envconfig:"PORT" default:"3000"` +} + +func Load() (*Config, error) { + var cfg Config + if err := envconfig.Process("", &cfg); err != nil { + return nil, fmt.Errorf("invalid environment: %w", err) + } + return &cfg, nil +} + +// In main(): +// cfg, err := config.Load() +// if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) } +``` + +--- + +## Direct-Vault Opt-In Pattern + +Use this pattern ONLY when a documented dynamic-secrets requirement applies (DB rotation with short TTLs, AWS STS, PKI, per-request audit). Document the justification in the project README under "Secrets architecture" before implementing. + +### When it is justified + +- Vault DB secrets engine with lease TTLs shorter than a typical pod lifecycle (< 1 hour) +- AWS STS assume-role tokens generated per-request +- Vault PKI short-lived certificates (< 24 hours) that must be renewed within a running pod +- Per-request audit trail requirement (each app call must appear separately in Vault audit log) + +### Provision an AppRole for the app + +```bash +# Enable AppRole auth (if not already enabled) +vault auth enable approle + +# Create a Vault policy for the app +vault policy write -policy - </*" { + capabilities = ["read"] +} +path "database/creds/-role" { + capabilities = ["read"] +} +EOF + +# Create the AppRole +vault write auth/approle/role/-role \ + token_policies="-policy" \ + token_ttl=1h \ + token_max_ttl=4h \ + secret_id_ttl=0 + +# Retrieve role-id and secret-id +vault read auth/approle/role/-role/role-id +vault write -f auth/approle/role/-role/secret-id +``` + +### Bootstrap AppRole credentials via ESO (solving the chicken-and-egg problem) + +The AppRole `role-id` and `secret-id` are themselves secrets. Store them in Vault at a bootstrap path, then use ESO to sync them into a k8s Secret. The app reads that k8s Secret at startup to authenticate with Vault directly. + +```bash +# Store the bootstrap credentials in Vault +vault kv put secret/k3s/-bootstrap \ + role_id="" \ + secret_id="" +``` + +```yaml +# deploy/external-secret-bootstrap.yaml +apiVersion: external-secrets.io/v1beta1 +kind: ExternalSecret +metadata: + name: -vault-auth + namespace: +spec: + refreshInterval: 24h + secretStoreRef: + name: vault-backend + kind: ClusterSecretStore + target: + name: -vault-auth + creationPolicy: Owner + data: + - secretKey: VAULT_ROLE_ID + remoteRef: + key: secret/k3s/-bootstrap + property: role_id + - secretKey: VAULT_SECRET_ID + remoteRef: + key: secret/k3s/-bootstrap + property: secret_id +``` + +```yaml +# deploy/deployment.yaml (env section for Direct-Vault app) +env: + - name: VAULT_ADDR + value: "https://vault.example.com" # safe-default: non-secret cluster address + - name: VAULT_ROLE_ID + valueFrom: + secretKeyRef: + name: -vault-auth + key: VAULT_ROLE_ID + - name: VAULT_SECRET_ID + valueFrom: + secretKeyRef: + name: -vault-auth + key: VAULT_SECRET_ID +``` + +### App-side Vault client pattern + +```typescript +// src/vault-client.ts — only exists in Direct-Vault apps +import vault from 'node-vault'; +import { z } from 'zod'; + +const bootstrapSchema = z.object({ + VAULT_ADDR: z.string().url(), + VAULT_ROLE_ID: z.string().min(1), + VAULT_SECRET_ID: z.string().min(1), +}); + +const bootstrap = bootstrapSchema.parse(process.env); + +const client = vault({ endpoint: bootstrap.VAULT_ADDR }); + +export async function getVaultClient() { + const { auth } = await client.approleLogin({ + role_id: bootstrap.VAULT_ROLE_ID, + secret_id: bootstrap.VAULT_SECRET_ID, + }); + client.token = auth.client_token; + return client; +} +``` + +Document in README under "Secrets architecture": the Vault path, why Direct-Vault is required, and the lease/renewal strategy. + +--- + +## Forbidden Patterns (CI Lint Targets) + +The following patterns are forbidden in all Mosaic projects. CI lint SHOULD catch these automatically (implementation tracked separately). Agents MUST NOT introduce these patterns. + +### 1. Untagged fallback defaults for required values + +```yaml +# FORBIDDEN — required secret with silent fallback +environment: + - DB_PASSWORD=${DB_PASSWORD:-changeme} + - API_KEY=${API_KEY:-} + +# REQUIRED — fast-fail on missing required values +environment: + - DB_PASSWORD=${DB_PASSWORD:?DB_PASSWORD is required} + - API_KEY=${API_KEY:?API_KEY is required} + +# ALLOWED — true convenience default, tagged +environment: + - PORT=${PORT:-3000} # safe-default: non-secret, app works at any port +``` + +This applies to: `docker-compose.yml`, k8s manifests, Helm `values.yaml`, any env file committed to git. + +### 2. Vault KV calls in application source code (ESO-default projects) + +```python +# FORBIDDEN in ESO-default apps — direct Vault client in app source +import hvac +client = hvac.Client(url=os.environ['VAULT_ADDR']) +secret = client.secrets.kv.v2.read_secret_version(path='myapp/db') +``` + +ESO-default apps read env vars only. Direct-Vault clients belong only in apps with a documented dynamic-secrets justification in README. + +### 3. Hardcoded secrets or API keys in committed files + +```python +# FORBIDDEN — hardcoded credential +DB_PASSWORD = "supersecret123" +API_KEY = "sk-live-abc123" +``` + +No exceptions. CI lint must flag any string matching common secret patterns (`password`, `secret`, `api_key`, `token` assigned a literal non-env-var value). + +### 4. `.env` files in production deployment paths + +``` +# FORBIDDEN — .env file in a production deploy path +deploy/.env +k8s/.env +docker/.env + +# ALLOWED — local dev only +.env.example # template only, no real values +.env # local dev, must be in .gitignore +``` + +`.env` files are acceptable in local-dev contexts only and MUST be in `.gitignore`. They are forbidden in any path that a CI pipeline or production deployment process reads directly. diff --git a/packages/mosaic/framework/defaults/STANDARDS.md b/packages/mosaic/framework/defaults/STANDARDS.md index b65cfdc..d9ea7f4 100644 --- a/packages/mosaic/framework/defaults/STANDARDS.md +++ b/packages/mosaic/framework/defaults/STANDARDS.md @@ -27,6 +27,16 @@ Master/slave model: - Do not perform destructive git/file actions without explicit instruction. - Browser automation (Playwright, Cypress, Puppeteer) MUST run in headless mode. Never launch a visible browser — it collides with the user's display and active session. +### Secrets handling (HARD RULE) + +- Vault is the canonical source-of-truth for every secret in every environment. No exceptions. +- For k8s workloads, the default read path is **External Secrets Operator → k8s Secret → env var** (`secretKeyRef`). The app reads standard env vars; no Vault client in app code. +- Direct-Vault clients in application code are **opt-in only**, justified per-app by a documented dynamic-secrets requirement (e.g., DB rotation, AWS STS). Default to ESO. Document the justification in the project's README under "Secrets architecture". +- `${VAR:-default}` fallback syntax in any deployment configuration (compose, k8s manifests, Helm values, env files committed to git) is **forbidden** for required values. Use `${VAR:?VAR is required}` to fast-fail. Defaults are allowed only for true conveniences (e.g. `${PORT:-3000}`) and MUST be tagged `# safe-default: ` so a reviewer can confirm the intent. +- `.env` files in production deployment paths are **forbidden**. `.env.example` and `.env` in local-dev paths are fine. +- App startup MUST validate required secrets against a schema (zod / pydantic / equivalent) and exit non-zero on missing required values. Never run with defaulted weak fallbacks. +- New apps: bootstrap checklist (see `~/.config/mosaic/guides/BOOTSTRAP.md`) MUST include Vault path provisioning + `ExternalSecret` manifest + README declaring the Vault path and required keys. + ## Session Lifecycle Contract - Start: `scripts/agent/session-start.sh` diff --git a/packages/mosaic/framework/guides/BOOTSTRAP.md b/packages/mosaic/framework/guides/BOOTSTRAP.md index 1c74b6d..b750eb4 100755 --- a/packages/mosaic/framework/guides/BOOTSTRAP.md +++ b/packages/mosaic/framework/guides/BOOTSTRAP.md @@ -453,6 +453,26 @@ Initialize standard labels and the first pre-MVP milestone: --- +## Secrets Bootstrap (Required for Every New App) + +Every new application MUST complete the following secrets bootstrap before deploying to any non-local environment. This is a hard gate — deployment without completed secrets bootstrap is forbidden. + +### Secrets bootstrap checklist + +- [ ] Vault path created: `vault kv put secret/k3s// ...` with all required secret fields +- [ ] Required secrets listed in project README under a "Secrets architecture" section, including: + - Vault path(s) used + - All required secret keys and their purpose + - Whether the app uses ESO bridge (default) or Direct-Vault (opt-in, with justification) +- [ ] `external-secret.yaml` manifest committed to repo's `deploy/` or `k8s/` directory +- [ ] Deployment YAML references the synced k8s Secret via `secretKeyRef` (not raw env vars or `.env` files) +- [ ] App startup has schema-based validation for all required env vars (zod / pydantic / envconfig equivalent) that exits non-zero on missing required values +- [ ] Direct-Vault opt-in (if applicable): justification documented in README + AppRole provisioned + bootstrap credentials stored in Vault and synced via a separate `ExternalSecret` + +See `~/.config/mosaic/guides/VAULT-SECRETS.md` for full worked examples of the ESO bridge pattern, the Direct-Vault opt-in pattern, and the forbidden antipatterns. + +--- + ## Checklist After bootstrapping, verify: diff --git a/packages/mosaic/framework/guides/VAULT-SECRETS.md b/packages/mosaic/framework/guides/VAULT-SECRETS.md index b1dda96..3692b36 100644 --- a/packages/mosaic/framework/guides/VAULT-SECRETS.md +++ b/packages/mosaic/framework/guides/VAULT-SECRETS.md @@ -203,3 +203,364 @@ Error: token expired 3. **Audit logging** - All access is logged; act accordingly 4. **No local copies** - Don't store secrets in files or env vars long-term 5. **Rotate on compromise** - Immediately rotate any exposed secrets + +--- + +## Secrets Architecture Decision Matrix + +Use this table to choose between the ESO bridge (default) and Direct-Vault (opt-in) patterns for every new app or integration. + +| Factor | ESO Bridge (default) | Direct-Vault (opt-in) | +| --- | --- | --- | +| **Use-case** | All static secrets (DB creds, API keys, signing keys, OAuth secrets) | Dynamic creds with short TTLs (DB rotation, AWS STS, PKI), per-request audit trails, or lease renewal mid-pod-lifecycle | +| **App code change** | None — reads standard env vars via `secretKeyRef` | Requires Vault client (`hvac`, `node-vault`, `vault/api`) in application code | +| **Secret rotation** | ESO re-syncs on Vault write; pod restart or secret refresh picks up new value | App manages lease renewal or re-auth within the running process | +| **Audit granularity** | Access logged at Vault when ESO syncs; no per-request app audit | Every app request to Vault is a separate audit log entry | +| **Operational burden** | Low — ESO handles polling, sync, and k8s Secret lifecycle | Higher — app must handle auth, lease renewal, error paths, and token rotation | +| **Justification required?** | No — this is the default | Yes — document in project README under "Secrets architecture" | +| **Example use cases** | Web app DB password, OAuth client secret, JWT signing key, API token | HashiCorp DB secrets engine with 15-min TTL leases, AWS STS assume-role, Vault PKI short-lived certs | + +**Decision rule:** If you are unsure, use ESO. Only justify Direct-Vault when the secret cannot be safely stored in a k8s Secret (too short-lived, per-request TTL required, or mid-lifecycle renewal needed). + +--- + +## ESO Bridge Pattern (Default) + +This is the required default for all k8s workloads. Follow this exact pattern unless a documented dynamic-secrets requirement justifies Direct-Vault. + +### 1. Provision Vault path + +```bash +# Write the secrets for the app (run once; use IaC/Terraform for repeatable provisioning) +vault kv put secret/k3s/ \ + db_password="..." \ + api_key="..." \ + jwt_secret="..." +``` + +Use the canonical path structure: `secret/k3s/` for k3s cluster workloads. + +### 2. ExternalSecret manifest + +Commit this to the repo's `deploy/` or `k8s/` directory: + +```yaml +# deploy/external-secret.yaml +apiVersion: external-secrets.io/v1beta1 +kind: ExternalSecret +metadata: + name: -secrets + namespace: +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-backend # ClusterSecretStore name — verify with cluster admin + kind: ClusterSecretStore + target: + name: -secrets # k8s Secret name that will be created + creationPolicy: Owner + data: + - secretKey: DB_PASSWORD # key in the k8s Secret + remoteRef: + key: secret/k3s/ # Vault path + property: db_password # field within the Vault secret + - secretKey: API_KEY + remoteRef: + key: secret/k3s/ + property: api_key + - secretKey: JWT_SECRET + remoteRef: + key: secret/k3s/ + property: jwt_secret +``` + +### 3. Deployment manifest — reference synced k8s Secret + +```yaml +# deploy/deployment.yaml (env section) +env: + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: -secrets # matches ExternalSecret target.name + key: DB_PASSWORD + - name: API_KEY + valueFrom: + secretKeyRef: + name: -secrets + key: API_KEY + - name: JWT_SECRET + valueFrom: + secretKeyRef: + name: -secrets + key: JWT_SECRET + - name: PORT + value: "3000" # safe-default: non-secret, no Vault needed +``` + +### 4. App-side schema validation — TypeScript (zod) + +Validate all required env vars at startup. Exit non-zero on missing values. + +```typescript +// src/env.ts +import { z } from 'zod'; + +const envSchema = z.object({ + DB_PASSWORD: z.string().min(1, 'DB_PASSWORD is required'), + API_KEY: z.string().min(1, 'API_KEY is required'), + JWT_SECRET: z.string().min(32, 'JWT_SECRET must be at least 32 chars'), + PORT: z.coerce.number().default(3000), + NODE_ENV: z.enum(['development', 'production', 'test']).default('production'), +}); + +const result = envSchema.safeParse(process.env); +if (!result.success) { + console.error('Missing or invalid environment variables:'); + console.error(result.error.flatten().fieldErrors); + process.exit(1); +} + +export const env = result.data; +``` + +### 4b. App-side schema validation — Python (pydantic) + +```python +# src/config.py +from pydantic_settings import BaseSettings, SettingsConfigDict + +class Settings(BaseSettings): + db_password: str + api_key: str + jwt_secret: str + port: int = 3000 + node_env: str = "production" + + model_config = SettingsConfigDict(env_file=None) # no .env in prod + +try: + settings = Settings() +except Exception as e: + import sys + print(f"Missing or invalid environment variables: {e}", file=sys.stderr) + sys.exit(1) +``` + +### 4c. App-side schema validation — Go (envconfig) + +```go +// config/config.go +package config + +import ( + "fmt" + "os" + "github.com/kelseyhightower/envconfig" +) + +type Config struct { + DBPassword string `envconfig:"DB_PASSWORD" required:"true"` + APIKey string `envconfig:"API_KEY" required:"true"` + JWTSecret string `envconfig:"JWT_SECRET" required:"true"` + Port int `envconfig:"PORT" default:"3000"` +} + +func Load() (*Config, error) { + var cfg Config + if err := envconfig.Process("", &cfg); err != nil { + return nil, fmt.Errorf("invalid environment: %w", err) + } + return &cfg, nil +} + +// In main(): +// cfg, err := config.Load() +// if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) } +``` + +--- + +## Direct-Vault Opt-In Pattern + +Use this pattern ONLY when a documented dynamic-secrets requirement applies (DB rotation with short TTLs, AWS STS, PKI, per-request audit). Document the justification in the project README under "Secrets architecture" before implementing. + +### When it is justified + +- Vault DB secrets engine with lease TTLs shorter than a typical pod lifecycle (< 1 hour) +- AWS STS assume-role tokens generated per-request +- Vault PKI short-lived certificates (< 24 hours) that must be renewed within a running pod +- Per-request audit trail requirement (each app call must appear separately in Vault audit log) + +### Provision an AppRole for the app + +```bash +# Enable AppRole auth (if not already enabled) +vault auth enable approle + +# Create a Vault policy for the app +vault policy write -policy - </*" { + capabilities = ["read"] +} +path "database/creds/-role" { + capabilities = ["read"] +} +EOF + +# Create the AppRole +vault write auth/approle/role/-role \ + token_policies="-policy" \ + token_ttl=1h \ + token_max_ttl=4h \ + secret_id_ttl=0 + +# Retrieve role-id and secret-id +vault read auth/approle/role/-role/role-id +vault write -f auth/approle/role/-role/secret-id +``` + +### Bootstrap AppRole credentials via ESO (solving the chicken-and-egg problem) + +The AppRole `role-id` and `secret-id` are themselves secrets. Store them in Vault at a bootstrap path, then use ESO to sync them into a k8s Secret. The app reads that k8s Secret at startup to authenticate with Vault directly. + +```bash +# Store the bootstrap credentials in Vault +vault kv put secret/k3s/-bootstrap \ + role_id="" \ + secret_id="" +``` + +```yaml +# deploy/external-secret-bootstrap.yaml +apiVersion: external-secrets.io/v1beta1 +kind: ExternalSecret +metadata: + name: -vault-auth + namespace: +spec: + refreshInterval: 24h + secretStoreRef: + name: vault-backend + kind: ClusterSecretStore + target: + name: -vault-auth + creationPolicy: Owner + data: + - secretKey: VAULT_ROLE_ID + remoteRef: + key: secret/k3s/-bootstrap + property: role_id + - secretKey: VAULT_SECRET_ID + remoteRef: + key: secret/k3s/-bootstrap + property: secret_id +``` + +```yaml +# deploy/deployment.yaml (env section for Direct-Vault app) +env: + - name: VAULT_ADDR + value: "https://vault.example.com" # safe-default: non-secret cluster address + - name: VAULT_ROLE_ID + valueFrom: + secretKeyRef: + name: -vault-auth + key: VAULT_ROLE_ID + - name: VAULT_SECRET_ID + valueFrom: + secretKeyRef: + name: -vault-auth + key: VAULT_SECRET_ID +``` + +### App-side Vault client pattern + +```typescript +// src/vault-client.ts — only exists in Direct-Vault apps +import vault from 'node-vault'; +import { z } from 'zod'; + +const bootstrapSchema = z.object({ + VAULT_ADDR: z.string().url(), + VAULT_ROLE_ID: z.string().min(1), + VAULT_SECRET_ID: z.string().min(1), +}); + +const bootstrap = bootstrapSchema.parse(process.env); + +const client = vault({ endpoint: bootstrap.VAULT_ADDR }); + +export async function getVaultClient() { + const { auth } = await client.approleLogin({ + role_id: bootstrap.VAULT_ROLE_ID, + secret_id: bootstrap.VAULT_SECRET_ID, + }); + client.token = auth.client_token; + return client; +} +``` + +Document in README under "Secrets architecture": the Vault path, why Direct-Vault is required, and the lease/renewal strategy. + +--- + +## Forbidden Patterns (CI Lint Targets) + +The following patterns are forbidden in all Mosaic projects. CI lint SHOULD catch these automatically (implementation tracked separately). Agents MUST NOT introduce these patterns. + +### 1. Untagged fallback defaults for required values + +```yaml +# FORBIDDEN — required secret with silent fallback +environment: + - DB_PASSWORD=${DB_PASSWORD:-changeme} + - API_KEY=${API_KEY:-} + +# REQUIRED — fast-fail on missing required values +environment: + - DB_PASSWORD=${DB_PASSWORD:?DB_PASSWORD is required} + - API_KEY=${API_KEY:?API_KEY is required} + +# ALLOWED — true convenience default, tagged +environment: + - PORT=${PORT:-3000} # safe-default: non-secret, app works at any port +``` + +This applies to: `docker-compose.yml`, k8s manifests, Helm `values.yaml`, any env file committed to git. + +### 2. Vault KV calls in application source code (ESO-default projects) + +```python +# FORBIDDEN in ESO-default apps — direct Vault client in app source +import hvac +client = hvac.Client(url=os.environ['VAULT_ADDR']) +secret = client.secrets.kv.v2.read_secret_version(path='myapp/db') +``` + +ESO-default apps read env vars only. Direct-Vault clients belong only in apps with a documented dynamic-secrets justification in README. + +### 3. Hardcoded secrets or API keys in committed files + +```python +# FORBIDDEN — hardcoded credential +DB_PASSWORD = "supersecret123" +API_KEY = "sk-live-abc123" +``` + +No exceptions. CI lint must flag any string matching common secret patterns (`password`, `secret`, `api_key`, `token` assigned a literal non-env-var value). + +### 4. `.env` files in production deployment paths + +``` +# FORBIDDEN — .env file in a production deploy path +deploy/.env +k8s/.env +docker/.env + +# ALLOWED — local dev only +.env.example # template only, no real values +.env # local dev, must be in .gitignore +``` + +`.env` files are acceptable in local-dev contexts only and MUST be in `.gitignore`. They are forbidden in any path that a CI pipeline or production deployment process reads directly. -- 2.49.1 From e88a89f34d686dbeddf4204a9a0d922e56bf7898 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Fri, 22 May 2026 12:01:29 -0500 Subject: [PATCH 2/2] fix(framework): remediate Codex review findings in VAULT-SECRETS.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two should-fix findings from automated Codex review: 1. Vault KV v2 policy path — add explicit path for exact top-level `secret/data/k3s/` entry alongside the wildcard `/*` sub-path rule. Without the exact path, apps reading the top-level secret get permission denied from Vault KV v2 even with the wildcard. 2. Go envconfig example — remove unused `os` import from config.go snippet (os was only referenced in a comment). Move the main() usage to a separate clearly-labelled main.go block to make both snippets copy-paste compilable. Both fixes mirrored to duplicate path: guides/ <-> packages/mosaic/framework/guides/ Co-Authored-By: Claude Opus 4.7 --- guides/VAULT-SECRETS.md | 18 ++++++++++++++---- .../mosaic/framework/guides/VAULT-SECRETS.md | 18 ++++++++++++++---- 2 files changed, 28 insertions(+), 8 deletions(-) diff --git a/guides/VAULT-SECRETS.md b/guides/VAULT-SECRETS.md index 3692b36..9a797ad 100644 --- a/guides/VAULT-SECRETS.md +++ b/guides/VAULT-SECRETS.md @@ -355,7 +355,6 @@ package config import ( "fmt" - "os" "github.com/kelseyhightower/envconfig" ) @@ -373,10 +372,16 @@ func Load() (*Config, error) { } return &cfg, nil } +``` -// In main(): -// cfg, err := config.Load() -// if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) } +In your `main.go`: + +```go +cfg, err := config.Load() +if err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) +} ``` --- @@ -399,7 +404,12 @@ Use this pattern ONLY when a documented dynamic-secrets requirement applies (DB vault auth enable approle # Create a Vault policy for the app +# Note: KV v2 paths require both the exact path (for the top-level secret) and the +# wildcard (for sub-paths). Always include both to avoid permission denied errors. vault policy write -policy - <" { + capabilities = ["read"] +} path "secret/data/k3s//*" { capabilities = ["read"] } diff --git a/packages/mosaic/framework/guides/VAULT-SECRETS.md b/packages/mosaic/framework/guides/VAULT-SECRETS.md index 3692b36..9a797ad 100644 --- a/packages/mosaic/framework/guides/VAULT-SECRETS.md +++ b/packages/mosaic/framework/guides/VAULT-SECRETS.md @@ -355,7 +355,6 @@ package config import ( "fmt" - "os" "github.com/kelseyhightower/envconfig" ) @@ -373,10 +372,16 @@ func Load() (*Config, error) { } return &cfg, nil } +``` -// In main(): -// cfg, err := config.Load() -// if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) } +In your `main.go`: + +```go +cfg, err := config.Load() +if err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) +} ``` --- @@ -399,7 +404,12 @@ Use this pattern ONLY when a documented dynamic-secrets requirement applies (DB vault auth enable approle # Create a Vault policy for the app +# Note: KV v2 paths require both the exact path (for the top-level secret) and the +# wildcard (for sub-paths). Always include both to avoid permission denied errors. vault policy write -policy - <" { + capabilities = ["read"] +} path "secret/data/k3s//*" { capabilities = ["read"] } -- 2.49.1