mosaic/bootstrap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

centralize guides and rails under mosaic with runtime compatibility links

Browse Source
This commit is contained in:
Jason Woltje
2026-02-17 11:39:52 -06:00
parent a1c2efef1c
commit 4eac2c76e6
85 changed files with 10785 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

144
guides/authentication.md Normal file
View File

@@ -0,0 +1,144 @@
# Authentication & Authorization Guide
## Before Starting
1. Check assigned issue: `~/.mosaic/rails/git/issue-list.sh -a @me`
2. Review existing auth implementation in codebase
3. Review Vault secrets structure: `docs/vault-secrets-structure.md`
## Authentication Patterns
### JWT (JSON Web Tokens)
```
Vault Path: secret-{env}/backend-api/jwt/signing-key
Fields: key, algorithm, expiry_seconds
```
**Best Practices:**
- Use RS256 or ES256 (asymmetric) for distributed systems
- Use HS256 (symmetric) only for single-service auth
- Set reasonable expiry (15min-1hr for access tokens)
- Include minimal claims (sub, exp, iat, roles)
- Never store sensitive data in JWT payload
### Session-Based
```
Vault Path: secret-{env}/{service}/session/secret
Fields: secret, cookie_name, max_age
```
**Best Practices:**
- Use secure, httpOnly, sameSite cookies
- Regenerate session ID on privilege change
- Implement session timeout
- Store sessions server-side (Redis/database)
### OAuth2/OIDC
```
Vault Paths:
- secret-{env}/{service}/oauth/{provider}/client_id
- secret-{env}/{service}/oauth/{provider}/client_secret
```
**Best Practices:**
- Use PKCE for public clients
- Validate state parameter
- Verify token signatures
- Check issuer and audience claims
## Authorization Patterns
### Role-Based Access Control (RBAC)
```python
# Example middleware
def require_role(roles: list):
def decorator(handler):
def wrapper(request):
user_roles = get_user_roles(request.user_id)
if not any(role in user_roles for role in roles):
raise ForbiddenError()
return handler(request)
return wrapper
return decorator
@require_role(['admin', 'moderator'])
def delete_user(request):
pass
```
### Permission-Based
```python
# Check specific permissions
def check_permission(user_id, resource, action):
permissions = get_user_permissions(user_id)
return f"{resource}:{action}" in permissions
```
## Security Requirements
### Password Handling
- Use bcrypt, scrypt, or Argon2 for hashing
- Minimum 12 character passwords
- Check against breached password lists
- Implement account lockout after failed attempts
### Token Security
- Rotate secrets regularly
- Implement token revocation
- Use short-lived access tokens with refresh tokens
- Store refresh tokens securely (httpOnly cookies or encrypted storage)
### Multi-Factor Authentication
- Support TOTP (Google Authenticator compatible)
- Consider WebAuthn for passwordless
- Require MFA for sensitive operations
## Testing Authentication
### Test Cases Required
```python
class TestAuthentication:
def test_login_success_returns_token(self):
pass
def test_login_failure_returns_401(self):
pass
def test_invalid_token_returns_401(self):
pass
def test_expired_token_returns_401(self):
pass
def test_missing_token_returns_401(self):
pass
def test_insufficient_permissions_returns_403(self):
pass
def test_token_refresh_works(self):
pass
def test_logout_invalidates_token(self):
pass
```
## Common Vulnerabilities to Avoid
1. **Broken Authentication**
- Weak password requirements
- Missing brute-force protection
- Session fixation
2. **Broken Access Control**
- Missing authorization checks
- IDOR (Insecure Direct Object Reference)
- Privilege escalation
3. **Security Misconfiguration**
- Default credentials
- Verbose error messages
- Missing security headers
## Commit Format
```
feat(#89): Implement JWT authentication
- Add /auth/login and /auth/refresh endpoints
- Implement token validation middleware
- Configure 15min access token expiry
Fixes #89
```

111
guides/backend.md Normal file
View File

@@ -0,0 +1,111 @@
# Backend Development Guide
## Before Starting
1. Check assigned issue: `~/.mosaic/rails/git/issue-list.sh -a @me`
2. Create scratchpad: `docs/scratchpads/{issue-number}-{short-name}.md`
3. Review API contracts and database schema
## Development Standards
### API Design
- Follow RESTful conventions (or GraphQL patterns if applicable)
- Use consistent endpoint naming: `/api/v1/resource-name`
- Return appropriate HTTP status codes
- Include pagination for list endpoints
- Document all endpoints (OpenAPI/Swagger preferred)
### Database
- Write migrations for schema changes
- Use parameterized queries (prevent SQL injection)
- Index frequently queried columns
- Document relationships and constraints
### Error Handling
- Return structured error responses
- Log errors with context (request ID, user ID if applicable)
- Never expose internal errors to clients
- Use appropriate error codes
```json
{
"error": {
"code": "VALIDATION_ERROR",
"message": "User-friendly message",
"details": []
}
}
```
### Security
- Validate all input at API boundaries
- Implement rate limiting on public endpoints
- Use secrets from Vault (see `docs/vault-secrets-structure.md`)
- Never log sensitive data (passwords, tokens, PII)
- Follow OWASP guidelines
### Authentication/Authorization
- Use project's established auth pattern
- Validate tokens on every request
- Check permissions before operations
- See `~/.mosaic/guides/authentication.md` for details
## Testing Requirements (TDD)
1. Write tests BEFORE implementation
2. Minimum 85% coverage
3. Test categories:
- Unit tests for business logic
- Integration tests for API endpoints
- Database tests with transactions/rollback
### Test Patterns
```python
# API test example structure
class TestResourceEndpoint:
def test_create_returns_201(self):
pass
def test_create_validates_input(self):
pass
def test_get_returns_404_for_missing(self):
pass
def test_requires_authentication(self):
pass
```
## Code Style
- Follow Google Style Guide for your language
- **TypeScript: Follow `~/.mosaic/guides/typescript.md` — MANDATORY**
- Use linter/formatter from project configuration
- Keep functions focused and small
- Document complex business logic
### TypeScript Quick Rules (see typescript.md for full guide)
- **NO `any`** — define explicit types always
- **NO lazy `unknown`** — only for error catches and external data with validation
- **Explicit return types** on all exported functions
- **Explicit parameter types** always
- **Interface for DTOs** — never inline object types
- **Typed errors** — use custom error classes
## Performance
- Use database connection pooling
- Implement caching where appropriate
- Profile slow endpoints
- Use async operations for I/O
## Commit Format
```
feat(#45): Add user registration endpoint
- POST /api/v1/users for registration
- Email validation and uniqueness check
- Password hashing with bcrypt
Fixes #45
```
## Before Completing
1. Run full test suite
2. Verify migrations work (up and down)
3. Test API with curl/httpie
4. Update scratchpad with completion notes
5. Reference issue in commit

338
guides/bootstrap.md Normal file
View File

@@ -0,0 +1,338 @@
# Project Bootstrap Guide
> Load this guide when setting up a new project for AI-assisted development.
## Overview
This guide covers how to bootstrap a project so AI agents (Claude, Codex, etc.) can work on it effectively. Proper bootstrapping ensures:
1. Agents understand the project structure and conventions
2. Orchestration works correctly with quality gates
3. Independent code review and security review are configured
4. Issue tracking is consistent across projects
## Quick Start
```bash
# Automated bootstrap (recommended)
~/.mosaic/rails/bootstrap/init-project.sh \
--name "my-project" \
--type "nestjs-nextjs" \
--repo "https://git.mosaicstack.dev/owner/repo"
# Or manually using templates
export PROJECT_NAME="My Project"
export PROJECT_DESCRIPTION="What this project does"
export TASK_PREFIX="MP"
envsubst < ~/.mosaic/templates/agent/CLAUDE.md.template > CLAUDE.md
envsubst < ~/.mosaic/templates/agent/AGENTS.md.template > AGENTS.md
```
---
## Step 1: Detect Project Type
Check what files exist in the project root to determine the type:
| File Present | Project Type | Template |
|---|---|---|
| `package.json` + `pnpm-workspace.yaml` + NestJS+Next.js | NestJS + Next.js Monorepo | `projects/nestjs-nextjs/` |
| `pyproject.toml` + `manage.py` | Django | `projects/django/` |
| `pyproject.toml` (no Django) | Python (generic) | Generic template |
| `package.json` (no monorepo) | Node.js (generic) | Generic template |
| Other | Generic | Generic template |
```bash
# Auto-detect project type
detect_project_type() {
if [[ -f "pnpm-workspace.yaml" ]] && [[ -f "turbo.json" ]]; then
# Check for NestJS + Next.js
if grep -q "nestjs" package.json 2>/dev/null && grep -q "next" package.json 2>/dev/null; then
echo "nestjs-nextjs"
return
fi
fi
if [[ -f "manage.py" ]] && [[ -f "pyproject.toml" ]]; then
echo "django"
return
fi
if [[ -f "pyproject.toml" ]]; then
echo "python"
return
fi
if [[ -f "package.json" ]]; then
echo "nodejs"
return
fi
echo "generic"
}
```
---
## Step 2: Create CLAUDE.md
The `CLAUDE.md` file is the primary configuration file for AI agents. It tells them everything they need to know about the project.
### Using a Tech-Stack Template
```bash
# Set variables
export PROJECT_NAME="My Project"
export PROJECT_DESCRIPTION="Multi-tenant SaaS platform"
export PROJECT_DIR="my-project"
export REPO_URL="https://git.mosaicstack.dev/owner/repo"
export TASK_PREFIX="MP"
# Use tech-stack-specific template if available
TYPE=$(detect_project_type)
TEMPLATE_DIR="$HOME/.mosaic/templates/agent/projects/$TYPE"
if [[ -d "$TEMPLATE_DIR" ]]; then
envsubst < "$TEMPLATE_DIR/CLAUDE.md.template" > CLAUDE.md
else
envsubst < "$HOME/.mosaic/templates/agent/CLAUDE.md.template" > CLAUDE.md
fi
```
### Using the Generic Template
```bash
# Set all required variables
export PROJECT_NAME="My Project"
export PROJECT_DESCRIPTION="What this project does"
export REPO_URL="https://git.mosaicstack.dev/owner/repo"
export PROJECT_DIR="my-project"
export SOURCE_DIR="src"
export CONFIG_FILES="pyproject.toml / package.json"
export FRONTEND_STACK="N/A"
export BACKEND_STACK="Python / FastAPI"
export DATABASE_STACK="PostgreSQL"
export TESTING_STACK="pytest"
export DEPLOYMENT_STACK="Docker"
export BUILD_COMMAND="pip install -e ."
export TEST_COMMAND="pytest tests/"
export LINT_COMMAND="ruff check ."
export TYPECHECK_COMMAND="mypy ."
export QUALITY_GATES="ruff check . && mypy . && pytest tests/"
envsubst < ~/.mosaic/templates/agent/CLAUDE.md.template > CLAUDE.md
```
### Required Sections
Every CLAUDE.md should contain:
1. **Project description** — One-line summary
2. **Conditional documentation loading** — Table of guides
3. **Technology stack** — What's used
4. **Repository structure** — Directory tree
5. **Development workflow** — Branch strategy, build, test
6. **Quality gates** — Commands that must pass
7. **Issue tracking** — Commit format, labels
8. **Code review** — Codex and fallback review commands
9. **Secrets management** — How secrets are handled
---
## Step 3: Create AGENTS.md
The `AGENTS.md` file contains agent-specific patterns, gotchas, and orchestrator integration details.
```bash
TYPE=$(detect_project_type)
TEMPLATE_DIR="$HOME/.mosaic/templates/agent/projects/$TYPE"
if [[ -d "$TEMPLATE_DIR" ]]; then
envsubst < "$TEMPLATE_DIR/AGENTS.md.template" > AGENTS.md
else
envsubst < "$HOME/.mosaic/templates/agent/AGENTS.md.template" > AGENTS.md
fi
```
### Living Document
AGENTS.md is a **living document**. Agents should update it when they discover:
- Reusable patterns (how similar features are built)
- Non-obvious requirements (e.g., "frontend env vars need NEXT_PUBLIC_ prefix")
- Common gotchas (e.g., "run migrations after schema changes")
- Testing approaches specific to this project
---
## Step 4: Create Directory Structure
```bash
# Create standard directories
mkdir -p docs/scratchpads
mkdir -p docs/templates
mkdir -p docs/reports
```
---
## Step 5: Initialize Repository Labels & Milestones
```bash
# Use the init script
~/.mosaic/rails/bootstrap/init-repo-labels.sh
# Or manually create standard labels
~/.mosaic/rails/git/issue-create.sh # (labels are created on first use)
```
### Standard Labels
| Label | Color | Purpose |
|-------|-------|---------|
| `epic` | `#3E4B9E` | Large feature spanning multiple issues |
| `feature` | `#0E8A16` | New functionality |
| `bug` | `#D73A4A` | Defect fix |
| `task` | `#0075CA` | General work item |
| `documentation` | `#0075CA` | Documentation updates |
| `security` | `#B60205` | Security-related |
| `breaking` | `#D93F0B` | Breaking change |
### Initial Milestone
Create the first milestone for MVP:
```bash
~/.mosaic/rails/git/milestone-create.sh -t "0.1.0" -d "MVP - Minimum Viable Product"
```
---
## Step 6: Set Up CI/CD Review Pipeline
### Woodpecker CI
```bash
# Copy Codex review pipeline
mkdir -p .woodpecker/schemas
cp ~/.mosaic/rails/codex/woodpecker/codex-review.yml .woodpecker/
cp ~/.mosaic/rails/codex/schemas/*.json .woodpecker/schemas/
# Add codex_api_key secret to Woodpecker CI dashboard
```
### GitHub Actions
For GitHub repos, use the official Codex GitHub Action instead:
```yaml
# .github/workflows/codex-review.yml
uses: openai/codex-action@v1
```
---
## Step 7: Verify Bootstrap
After bootstrapping, verify everything works:
```bash
# Check files exist
ls CLAUDE.md AGENTS.md docs/scratchpads/
# Verify CLAUDE.md has required sections
grep -c "Quality Gates" CLAUDE.md
grep -c "Technology Stack" CLAUDE.md
grep -c "Code Review" CLAUDE.md
# Verify quality gates run
eval "$(grep -A1 'Quality Gates' CLAUDE.md | tail -1)"
# Test Codex review (if configured)
~/.mosaic/rails/codex/codex-code-review.sh --help
```
---
## Available Templates
### Generic Templates
| Template | Path | Purpose |
|----------|------|---------|
| `CLAUDE.md.template` | `~/.mosaic/templates/agent/` | Generic project CLAUDE.md |
| `AGENTS.md.template` | `~/.mosaic/templates/agent/` | Generic agent context |
### Tech-Stack Templates
| Stack | Path | Includes |
|-------|------|----------|
| NestJS + Next.js | `~/.mosaic/templates/agent/projects/nestjs-nextjs/` | CLAUDE.md, AGENTS.md |
| Django | `~/.mosaic/templates/agent/projects/django/` | CLAUDE.md, AGENTS.md |
### Orchestrator Templates
| Template | Path | Purpose |
|----------|------|---------|
| `tasks.md.template` | `~/src/jarvis-brain/docs/templates/orchestrator/` | Task tracking |
| `orchestrator-learnings.json.template` | `~/src/jarvis-brain/docs/templates/orchestrator/` | Variance tracking |
| `phase-issue-body.md.template` | `~/src/jarvis-brain/docs/templates/orchestrator/` | Gitea issue body |
| `scratchpad.md.template` | `~/src/jarvis-brain/docs/templates/` | Per-task working doc |
### Variables Reference
| Variable | Description | Example |
|----------|-------------|---------|
| `${PROJECT_NAME}` | Human-readable project name | "Mosaic Stack" |
| `${PROJECT_DESCRIPTION}` | One-line description | "Multi-tenant platform" |
| `${PROJECT_DIR}` | Directory name | "mosaic-stack" |
| `${PROJECT_SLUG}` | Python package slug | "mosaic_stack" |
| `${REPO_URL}` | Git remote URL | "https://git.mosaicstack.dev/mosaic/stack" |
| `${TASK_PREFIX}` | Orchestrator task prefix | "MS" |
| `${SOURCE_DIR}` | Source code directory | "src" or "apps" |
| `${QUALITY_GATES}` | Quality gate commands | "pnpm typecheck && pnpm lint && pnpm test" |
| `${BUILD_COMMAND}` | Build command | "pnpm build" |
| `${TEST_COMMAND}` | Test command | "pnpm test" |
| `${LINT_COMMAND}` | Lint command | "pnpm lint" |
| `${TYPECHECK_COMMAND}` | Type check command | "pnpm typecheck" |
| `${FRONTEND_STACK}` | Frontend technologies | "Next.js + React" |
| `${BACKEND_STACK}` | Backend technologies | "NestJS + Prisma" |
| `${DATABASE_STACK}` | Database technologies | "PostgreSQL" |
| `${TESTING_STACK}` | Testing technologies | "Vitest + Playwright" |
| `${DEPLOYMENT_STACK}` | Deployment technologies | "Docker" |
| `${CONFIG_FILES}` | Key config files | "package.json, tsconfig.json" |
---
## Bootstrap Scripts
### init-project.sh
Full project bootstrap with interactive and flag-based modes:
```bash
~/.mosaic/rails/bootstrap/init-project.sh \
--name "My Project" \
--type "nestjs-nextjs" \
--repo "https://git.mosaicstack.dev/owner/repo" \
--prefix "MP" \
--description "Multi-tenant platform"
```
### init-repo-labels.sh
Initialize standard labels and MVP milestone:
```bash
~/.mosaic/rails/bootstrap/init-repo-labels.sh
```
---
## Checklist
After bootstrapping, verify:
- [ ] `CLAUDE.md` exists with all required sections
- [ ] `AGENTS.md` exists with patterns and quality gates
- [ ] `docs/scratchpads/` directory exists
- [ ] Git labels created (epic, feature, bug, task, etc.)
- [ ] Initial milestone created (0.1.0 MVP)
- [ ] Quality gates run successfully
- [ ] `.env.example` exists (if project uses env vars)
- [ ] CI/CD pipeline configured (if using Woodpecker/GitHub Actions)
- [ ] Codex review scripts accessible (`~/.mosaic/rails/codex/`)

904
guides/ci-cd-pipelines.md Normal file
View File

@@ -0,0 +1,904 @@
# CI/CD Pipeline Guide
> **Load this guide when:** Adding Docker build/push steps, configuring Woodpecker CI pipelines, publishing packages to registries, or implementing CI/CD for a new project.
## Overview
This guide covers the canonical CI/CD pattern used across projects. The pipeline runs in Woodpecker CI and follows this flow:
```
GIT PUSH
↓
QUALITY GATES (lint, typecheck, test, audit)
↓ all pass
BUILD (compile all packages)
↓ only on main/develop/tags
DOCKER BUILD & PUSH (Kaniko → Gitea Container Registry)
↓ all images pushed
PACKAGE LINKING (associate images with repository in Gitea)
```
## Reference Implementations
### Split Pipelines (Preferred for Monorepos)
**Mosaic Telemetry** (`~/src/mosaic-telemetry-monorepo/.woodpecker/`) is the canonical example of **split per-package pipelines** with path filtering, full security chain (source + container scanning), and efficient CI resource usage.
**Key features:**
- One YAML per package in `.woodpecker/` directory
- Path filtering: only the affected package's pipeline runs on push
- Security chain: source scanning (bandit/npm audit) + dependency audit (pip-audit) + container scanning (Trivy)
- Docker build gates on ALL quality steps
**Always use this pattern for monorepos.** It saves CI minutes and isolates failures.
### Single Pipeline (Legacy/Simple Projects)
**Mosaic Stack** (`~/src/mosaic-stack/.woodpecker/build.yml`) uses a single pipeline that builds everything on every push. This works but wastes CI resources on large monorepos. **Mosaic Stack is scheduled for migration to split pipelines.**
Always read the telemetry pipelines first when implementing a new pipeline.
## Infrastructure Instances
| Project | Gitea | Woodpecker | Registry |
|---------|-------|------------|----------|
| Mosaic Stack | `git.mosaicstack.dev` | `ci.mosaicstack.dev` | `git.mosaicstack.dev` |
| U-Connect | `git.uscllc.com` | `woodpecker.uscllc.net` | `git.uscllc.com` |
The patterns are identical — only the hostnames and org/repo names differ.
## Woodpecker Pipeline Structure
### YAML Anchors (DRY)
Define reusable values at the top of `.woodpecker.yml`:
```yaml
variables:
- &node_image "node:20-alpine"
- &install_deps |
corepack enable
npm ci
# For pnpm projects, use:
# - &install_deps |
# corepack enable
# pnpm install --frozen-lockfile
- &kaniko_setup |
mkdir -p /kaniko/.docker
echo "{\"auths\":{\"REGISTRY_HOST\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
```
Replace `REGISTRY_HOST` with the actual Gitea hostname (e.g., `git.uscllc.com`).
### Step Dependencies
Woodpecker runs steps in parallel by default. Use `depends_on` to create the dependency graph:
```yaml
steps:
install:
image: *node_image
commands:
- *install_deps
lint:
image: *node_image
commands:
- npm run lint
depends_on:
- install
typecheck:
image: *node_image
commands:
- npm run type-check
depends_on:
- install
test:
image: *node_image
commands:
- npm run test
depends_on:
- install
build:
image: *node_image
environment:
NODE_ENV: "production"
commands:
- npm run build
depends_on:
- lint
- typecheck
- test
```
### Conditional Execution
Use `when` clauses to limit expensive steps (Docker builds) to relevant branches:
```yaml
when:
# Top-level: run quality gates on everything
- event: [push, pull_request, manual]
# Per-step: only build Docker images on main/develop/tags
docker-build-api:
when:
- branch: [main, develop]
event: [push, manual, tag]
```
## Docker Build & Push with Kaniko
### Why Kaniko
Kaniko builds container images without requiring a Docker daemon. This is the standard approach in Woodpecker CI because:
- No privileged mode needed
- No Docker-in-Docker security concerns
- Multi-destination tagging in a single build
- Works in any container runtime
### Kaniko Step Template
```yaml
docker-build-SERVICE:
image: gcr.io/kaniko-project/executor:debug
environment:
GITEA_USER:
from_secret: gitea_username
GITEA_TOKEN:
from_secret: gitea_token
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
commands:
- *kaniko_setup
- |
DESTINATIONS="--destination REGISTRY/ORG/IMAGE_NAME:${CI_COMMIT_SHA:0:8}"
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
DESTINATIONS="$DESTINATIONS --destination REGISTRY/ORG/IMAGE_NAME:latest"
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
DESTINATIONS="$DESTINATIONS --destination REGISTRY/ORG/IMAGE_NAME:dev"
fi
if [ -n "$CI_COMMIT_TAG" ]; then
DESTINATIONS="$DESTINATIONS --destination REGISTRY/ORG/IMAGE_NAME:$CI_COMMIT_TAG"
fi
/kaniko/executor --context . --dockerfile PATH/TO/Dockerfile $DESTINATIONS
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on:
- build
```
**Replace these placeholders:**
| Placeholder | Example (Mosaic) | Example (U-Connect) |
|-------------|-------------------|----------------------|
| `REGISTRY` | `git.mosaicstack.dev` | `git.uscllc.com` |
| `ORG` | `mosaic` | `usc` |
| `IMAGE_NAME` | `stack-api` | `uconnect-backend-api` |
| `PATH/TO/Dockerfile` | `apps/api/Dockerfile` | `src/backend-api/Dockerfile` |
### Image Tagging Strategy
Every build produces multiple tags:
| Condition | Tag | Purpose |
|-----------|-----|---------|
| Always | `${CI_COMMIT_SHA:0:8}` | Immutable reference to exact commit |
| `main` branch | `latest` | Current production release |
| `develop` branch | `dev` | Current development build |
| Git tag (e.g., `v1.0.0`) | `v1.0.0` | Semantic version release |
### Kaniko Options
Common flags for `/kaniko/executor`:
| Flag | Purpose |
|------|---------|
| `--context .` | Build context directory |
| `--dockerfile path/Dockerfile` | Dockerfile location |
| `--destination registry/org/image:tag` | Push target (repeatable) |
| `--build-arg KEY=VALUE` | Pass build arguments |
| `--cache=true` | Enable layer caching |
| `--cache-repo registry/org/image-cache` | Cache storage location |
### Build Arguments
Pass environment-specific values at build time:
```yaml
/kaniko/executor --context . --dockerfile apps/web/Dockerfile \
--build-arg NEXT_PUBLIC_API_URL=https://api.example.com \
$DESTINATIONS
```
## Gitea Container Registry
### How It Works
Gitea has a built-in container registry. When you push an image to `git.example.com/org/image:tag`, Gitea stores it and makes it available in the Packages section.
### Authentication
Kaniko authenticates via a Docker config file created at pipeline start:
```json
{
"auths": {
"git.example.com": {
"username": "GITEA_USER",
"password": "GITEA_TOKEN"
}
}
}
```
The token must have `package:write` scope. Generate it at: `https://GITEA_HOST/user/settings/applications`
### Pulling Images
After pushing, images are available at:
```bash
docker pull git.example.com/org/image:tag
```
In `docker-compose.yml`:
```yaml
services:
api:
image: git.example.com/org/image:${IMAGE_TAG:-dev}
```
## Package Linking
After pushing images to the Gitea registry, link them to the source repository so they appear on the repository's Packages tab.
### Gitea Package Linking API
```
POST /api/v1/packages/{owner}/{type}/{name}/-/link/{repo}
```
| Parameter | Value |
|-----------|-------|
| `owner` | Organization name (e.g., `mosaic`, `usc`) |
| `type` | `container` |
| `name` | Image name (e.g., `stack-api`) |
| `repo` | Repository name (e.g., `stack`, `uconnect`) |
### Link Step Template
```yaml
link-packages:
image: alpine:3
environment:
GITEA_TOKEN:
from_secret: gitea_token
commands:
- apk add --no-cache curl
- echo "Waiting 10 seconds for packages to be indexed..."
- sleep 10
- |
set -e
link_package() {
PKG="$$1"
echo "Linking $$PKG..."
for attempt in 1 2 3; do
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
-H "Authorization: token $$GITEA_TOKEN" \
"https://GITEA_HOST/api/v1/packages/ORG/container/$$PKG/-/link/REPO")
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
echo " Linked $$PKG"
return 0
elif [ "$$STATUS" = "400" ]; then
echo " $$PKG already linked"
return 0
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
sleep 5
else
echo " FAILED: $$PKG status $$STATUS"
cat /tmp/link-response.txt
return 1
fi
done
}
link_package "image-name-1"
link_package "image-name-2"
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on:
- docker-build-image-1
- docker-build-image-2
```
**Replace:** `GITEA_HOST`, `ORG`, `REPO`, and the `link_package` calls with actual image names.
**Note on `$$`:** Woodpecker uses `$$` to escape `$` in shell commands within YAML. Use `$$` for shell variables and `${CI_*}` (single `$`) for Woodpecker CI variables.
### Status Codes
| Code | Meaning | Action |
|------|---------|--------|
| 201 | Created | Success |
| 204 | No content | Success |
| 400 | Bad request | Already linked (OK) |
| 404 | Not found | Retry — package may not be indexed yet |
### Known Issue
The Gitea package linking API (added in Gitea 1.24.0) can return 404 for recently pushed packages. The retry logic with 5-second delays handles this. If linking still fails, packages are usable — they just won't appear on the repository Packages tab. They can be linked manually via the Gitea web UI.
## Woodpecker Secrets
### Required Secrets
Configure these in the Woodpecker UI (Settings > Secrets) or via CLI:
| Secret Name | Value | Scope |
|-------------|-------|-------|
| `gitea_username` | Gitea username or service account | `push`, `manual`, `tag` |
| `gitea_token` | Gitea token with `package:write` scope | `push`, `manual`, `tag` |
### Setting Secrets via CLI
```bash
# Woodpecker CLI
woodpecker secret add ORG/REPO --name gitea_username --value "USERNAME"
woodpecker secret add ORG/REPO --name gitea_token --value "TOKEN"
```
### Security Rules
- Never hardcode tokens in pipeline YAML
- Use `from_secret` for all credentials
- Limit secret event scope (don't expose on `pull_request` from forks)
- Use dedicated service accounts, not personal tokens
- Rotate tokens periodically
## npm Package Publishing
For projects with publishable npm packages (e.g., shared libraries, design systems).
### Publishing to Gitea npm Registry
Gitea includes a built-in npm registry at `https://GITEA_HOST/api/packages/ORG/npm/`.
**Pipeline step:**
```yaml
publish-packages:
image: *node_image
environment:
GITEA_TOKEN:
from_secret: gitea_token
commands:
- |
echo "//GITEA_HOST/api/packages/ORG/npm/:_authToken=$$GITEA_TOKEN" > .npmrc
echo "@SCOPE:registry=https://GITEA_HOST/api/packages/ORG/npm/" >> .npmrc
- npm publish -w @SCOPE/package-name
when:
- branch: [main]
event: [push, manual, tag]
depends_on:
- build
```
**Replace:** `GITEA_HOST`, `ORG`, `SCOPE`, `package-name`.
### Why Gitea npm (not Verdaccio)
Gitea's built-in npm registry eliminates the need for a separate Verdaccio instance. Benefits:
- **Same auth** — Gitea token with `package:write` scope works for git, containers, AND npm
- **No extra service** — No Verdaccio container, no OAuth/Authentik integration, no separate compose stack
- **Same UI** — Packages appear alongside container images in Gitea's Packages tab
- **Same secrets** — `gitea_token` in Woodpecker handles both Docker push and npm publish
If a project currently uses Verdaccio (e.g., U-Connect at `npm.uscllc.net`), migrate to Gitea npm. See the migration checklist below.
### Versioning
Only publish when the version in `package.json` has changed. Add a version check:
```yaml
commands:
- |
CURRENT=$(node -p "require('./src/PACKAGE/package.json').version")
PUBLISHED=$(npm view @SCOPE/PACKAGE version 2>/dev/null || echo "0.0.0")
if [ "$CURRENT" = "$PUBLISHED" ]; then
echo "Version $CURRENT already published, skipping"
exit 0
fi
echo "Publishing $CURRENT (was $PUBLISHED)"
npm publish -w @SCOPE/PACKAGE
```
## CI Services (Test Databases)
For projects that need a database during CI (migrations, integration tests):
```yaml
services:
postgres:
image: postgres:17-alpine
environment:
POSTGRES_DB: test_db
POSTGRES_USER: test_user
POSTGRES_PASSWORD: test_password
steps:
test:
image: *node_image
environment:
DATABASE_URL: "postgresql://test_user:test_password@postgres:5432/test_db?schema=public"
commands:
- npm run test
depends_on:
- install
```
The service name (`postgres`) becomes the hostname within the pipeline network.
## Split Pipelines for Monorepos (REQUIRED)
For any monorepo with multiple packages/apps, use **split pipelines** — one YAML per package in `.woodpecker/`.
### Why Split?
| Aspect | Single pipeline | Split pipelines |
|--------|----------------|-----------------|
| Path filtering | None — everything rebuilds | Per-package — only affected code |
| Security scanning | Often missing | Required per-package |
| CI minutes | Wasted on unaffected packages | Efficient |
| Failure isolation | One failure blocks everything | Per-package failures isolated |
| Readability | One massive file | Focused, maintainable |
### Structure
```
.woodpecker/
├── api.yml # Only runs when apps/api/** changes
├── web.yml # Only runs when apps/web/** changes
└── (infra.yml) # Optional: shared infra (DB images, etc.)
```
**IMPORTANT:** Do NOT also have `.woodpecker.yml` at root — `.woodpecker/` directory takes precedence and the `.yml` file will be silently ignored.
### Path Filtering Template
```yaml
when:
- event: [push, pull_request, manual]
path:
include: ['apps/api/**', '.woodpecker/api.yml']
```
Each pipeline self-triggers on its own YAML changes. Manual triggers run regardless of path.
### Kaniko Context Scoping
In split pipelines, scope the Kaniko context to the app directory:
```yaml
/kaniko/executor --context apps/api --dockerfile apps/api/Dockerfile $$DESTINATIONS
```
This means Dockerfile `COPY . .` only copies the app's files, not the entire monorepo.
### Reference: Telemetry Split Pipeline
See `~/src/mosaic-telemetry-monorepo/.woodpecker/api.yml` and `web.yml` for a complete working example with path filtering, security chain, and Trivy scanning.
## Security Scanning (REQUIRED)
Every pipeline MUST include security scanning. Docker build steps MUST gate on all security steps passing.
### Source-Level Security (per tech stack)
**Python:**
```yaml
security-bandit:
image: *uv_image
commands:
- |
cd apps/api
uv sync --all-extras --frozen
uv run bandit -r src/ -f screen
depends_on: [install]
security-audit:
image: *uv_image
commands:
- |
cd apps/api
uv sync --all-extras --frozen
uv run pip-audit
depends_on: [install]
```
**Node.js:**
```yaml
security-audit:
image: node:22-alpine
commands:
- cd apps/web && npm audit --audit-level=high
depends_on: [install]
```
### Container Scanning (Trivy) — Post-Build
Run Trivy against every built image to catch OS-level and runtime vulnerabilities:
```yaml
security-trivy:
image: aquasec/trivy:latest
environment:
GITEA_USER:
from_secret: gitea_username
GITEA_TOKEN:
from_secret: gitea_token
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
commands:
- |
mkdir -p ~/.docker
echo "{\"auths\":{\"REGISTRY\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
REGISTRY/ORG/IMAGE:$${CI_COMMIT_SHA:0:8}
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on:
- docker-build-SERVICE
```
**Replace:** `REGISTRY`, `ORG`, `IMAGE`, `SERVICE`.
### Full Dependency Chain
```
install → [lint, typecheck, security-source, security-deps, test] → docker-build → trivy → link-package
```
Docker build MUST depend on ALL quality + security steps. Trivy runs AFTER build. Package linking runs AFTER Trivy.
## Monorepo Considerations
### pnpm + Turbo (Mosaic Stack pattern)
```yaml
variables:
- &install_deps |
corepack enable
pnpm install --frozen-lockfile
steps:
build:
commands:
- *install_deps
- pnpm build # Turbo handles dependency order and caching
```
### npm Workspaces (U-Connect pattern)
```yaml
variables:
- &install_deps |
corepack enable
npm ci
steps:
# Build shared dependencies first
build-deps:
commands:
- npm run build -w @scope/shared-auth
- npm run build -w @scope/shared-types
# Then build everything
build-all:
commands:
- npm run build -w @scope/package-1
- npm run build -w @scope/package-2
# ... in dependency order
depends_on:
- build-deps
```
### Per-Package Quality Checks
For large monorepos, run checks per-package in parallel:
```yaml
lint-api:
commands:
- npm run lint -w @scope/api
depends_on: [install]
lint-web:
commands:
- npm run lint -w @scope/web
depends_on: [install]
# These run in parallel since they share the same dependency
```
## Complete Pipeline Example
This is a minimal but complete pipeline for a project with two services:
```yaml
when:
- event: [push, pull_request, manual]
variables:
- &node_image "node:20-alpine"
- &install_deps |
corepack enable
npm ci
- &kaniko_setup |
mkdir -p /kaniko/.docker
echo "{\"auths\":{\"git.example.com\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
steps:
# === Quality Gates ===
install:
image: *node_image
commands:
- *install_deps
lint:
image: *node_image
commands:
- npm run lint
depends_on: [install]
test:
image: *node_image
commands:
- npm run test
depends_on: [install]
build:
image: *node_image
environment:
NODE_ENV: "production"
commands:
- npm run build
depends_on: [lint, test]
# === Docker Build & Push ===
docker-build-api:
image: gcr.io/kaniko-project/executor:debug
environment:
GITEA_USER:
from_secret: gitea_username
GITEA_TOKEN:
from_secret: gitea_token
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
commands:
- *kaniko_setup
- |
DESTINATIONS="--destination git.example.com/org/api:${CI_COMMIT_SHA:0:8}"
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/api:latest"
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/api:dev"
fi
if [ -n "$CI_COMMIT_TAG" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/api:$CI_COMMIT_TAG"
fi
/kaniko/executor --context . --dockerfile src/api/Dockerfile $DESTINATIONS
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on: [build]
docker-build-web:
image: gcr.io/kaniko-project/executor:debug
environment:
GITEA_USER:
from_secret: gitea_username
GITEA_TOKEN:
from_secret: gitea_token
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
commands:
- *kaniko_setup
- |
DESTINATIONS="--destination git.example.com/org/web:${CI_COMMIT_SHA:0:8}"
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/web:latest"
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/web:dev"
fi
if [ -n "$CI_COMMIT_TAG" ]; then
DESTINATIONS="$DESTINATIONS --destination git.example.com/org/web:$CI_COMMIT_TAG"
fi
/kaniko/executor --context . --dockerfile src/web/Dockerfile $DESTINATIONS
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on: [build]
# === Package Linking ===
link-packages:
image: alpine:3
environment:
GITEA_TOKEN:
from_secret: gitea_token
commands:
- apk add --no-cache curl
- sleep 10
- |
set -e
link_package() {
PKG="$$1"
for attempt in 1 2 3; do
STATUS=$$(curl -s -o /dev/null -w "%{http_code}" -X POST \
-H "Authorization: token $$GITEA_TOKEN" \
"https://git.example.com/api/v1/packages/org/container/$$PKG/-/link/repo")
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ] || [ "$$STATUS" = "400" ]; then
echo "Linked $$PKG ($$STATUS)"
return 0
elif [ $$attempt -lt 3 ]; then
sleep 5
else
echo "FAILED: $$PKG ($$STATUS)"
return 1
fi
done
}
link_package "api"
link_package "web"
when:
- branch: [main, develop]
event: [push, manual, tag]
depends_on:
- docker-build-api
- docker-build-web
```
## Checklist: Adding CI/CD to a Project
1. **Verify Dockerfiles exist** for each service that needs an image
2. **Create Woodpecker secrets** (`gitea_username`, `gitea_token`) in the Woodpecker UI
3. **Verify Gitea token scope** includes `package:write`
4. **Add Docker build steps** to `.woodpecker.yml` using the Kaniko template above
5. **Add package linking step** after all Docker builds
6. **Update `docker-compose.yml`** to reference registry images instead of local builds:
```yaml
image: git.example.com/org/service:${IMAGE_TAG:-dev}
```
7. **Test on develop branch first** — push a small change and verify the pipeline
8. **Verify images appear** in Gitea Packages tab after successful pipeline
## Gitea as Unified Platform
Gitea provides **three services in one**, eliminating the need for separate Harbor and Verdaccio deployments:
| Service | What Gitea Replaces | Registry URL |
|---------|---------------------|-------------|
| **Git hosting** | GitHub/GitLab | `https://GITEA_HOST/org/repo` |
| **Container registry** | Harbor, Docker Hub | `docker pull GITEA_HOST/org/image:tag` |
| **npm registry** | Verdaccio, Artifactory | `https://GITEA_HOST/api/packages/org/npm/` |
### Additional Package Types
Gitea also supports PyPI, Maven, NuGet, Cargo, Composer, Conan, Conda, Generic, and more. All use the same token authentication.
### Single Token, Three Services
A Gitea token with `package:write` scope handles:
- `git push` / `git pull`
- `docker push` / `docker pull` (container registry)
- `npm publish` / `npm install` (npm registry)
This means a single `gitea_token` secret in Woodpecker CI covers all CI/CD operations.
### Architecture Simplification
**Before (3 services):**
```
Gitea (git) + Harbor (containers) + Verdaccio (npm)
↓ separate auth ↓ separate auth ↓ OAuth/Authentik
3 tokens 1 robot account 1 OIDC integration
3 backup targets complex RBAC group-based access
```
**After (1 service):**
```
Gitea (git + containers + npm)
↓ single token
1 secret in Woodpecker
1 backup target
unified RBAC via Gitea teams
```
## Migrating from Verdaccio to Gitea npm
If a project currently uses Verdaccio (e.g., U-Connect at `npm.uscllc.net`), follow this migration checklist:
### Migration Steps
1. **Verify Gitea npm registry is accessible:**
```bash
curl -s https://GITEA_HOST/api/packages/ORG/npm/ | head -5
```
2. **Update `.npmrc` in project root:**
```ini
# Before (Verdaccio)
@uconnect:registry=https://npm.uscllc.net
# After (Gitea)
@uconnect:registry=https://git.uscllc.com/api/packages/usc/npm/
```
3. **Update CI pipeline** — replace `npm_token` secret with `gitea_token`:
```yaml
# Uses same token as Docker push — no extra secret needed
echo "//GITEA_HOST/api/packages/ORG/npm/:_authToken=$$GITEA_TOKEN" > .npmrc
```
4. **Re-publish existing packages** to Gitea registry:
```bash
# For each @scope/package
npm publish -w @scope/package --registry https://GITEA_HOST/api/packages/ORG/npm/
```
5. **Update consumer projects** — any project that `npm install`s from the old registry needs its `.npmrc` updated
6. **Remove Verdaccio infrastructure:**
- Docker compose stack (`compose.verdaccio.yml`)
- Authentik OAuth provider/blueprints
- Verdaccio config files
- DNS entry for `npm.uscllc.net` (eventually)
### What You Can Remove
| Component | Location | Purpose (was) |
|-----------|----------|---------------|
| Verdaccio compose | `compose.verdaccio.yml` | npm registry container |
| Verdaccio config | `config/verdaccio/` | Server configuration |
| Authentik blueprints | `config/authentik/blueprints/*/verdaccio-*` | OAuth integration |
| Verdaccio scripts | `scripts/verdaccio/` | Blueprint application |
| OIDC env vars | `.env` | `AUTHENTIK_VERDACCIO_*`, `VERDACCIO_OPENID_*` |
## Troubleshooting
### "unauthorized: authentication required"
- Verify `gitea_username` and `gitea_token` secrets are set in Woodpecker
- Verify the token has `package:write` scope
- Check the registry hostname in `kaniko_setup` matches the Gitea instance
### Kaniko build fails with "error building image"
- Verify the Dockerfile path is correct relative to `--context`
- Check that multi-stage builds don't reference stages that don't exist
- Run `docker build` locally first to verify the Dockerfile works
### Package linking returns 404
- Normal for recently pushed packages — the retry logic handles this
- If persistent: verify the package name matches exactly (case-sensitive)
- Check Gitea version is 1.24.0+ (package linking API requirement)
### Images not visible in Gitea Packages
- Linking may have failed — check the `link-packages` step logs
- Images are still usable via `docker pull` even without linking
- Link manually: Gitea UI > Packages > Select package > Link to repository
### Pipeline runs Docker builds on pull requests
- Verify `when` clause on Docker build steps restricts to `branch: [main, develop]`
- Pull requests should only run quality gates, not build/push images

101
guides/code-review.md Normal file
View File

@@ -0,0 +1,101 @@
# Code Review Guide
## Review Checklist
### 1. Correctness
- [ ] Code does what the issue/PR description says
- [ ] Edge cases are handled
- [ ] Error conditions are managed properly
- [ ] No obvious bugs or logic errors
### 2. Security
- [ ] No hardcoded secrets or credentials
- [ ] Input validation at boundaries
- [ ] SQL injection prevention (parameterized queries)
- [ ] XSS prevention (output encoding)
- [ ] Authentication/authorization checks present
- [ ] Sensitive data not logged
- [ ] Secrets follow Vault structure (see `docs/vault-secrets-structure.md`)
### 3. Testing
- [ ] Tests exist for new functionality
- [ ] Tests cover happy path AND error cases
- [ ] Coverage meets 85% minimum
- [ ] Tests are readable and maintainable
- [ ] No flaky tests introduced
### 4. Code Quality
- [ ] Follows Google Style Guide for the language
- [ ] Functions are focused and reasonably sized
- [ ] No unnecessary complexity
- [ ] DRY - no significant duplication
- [ ] Clear naming for variables and functions
- [ ] No dead code or commented-out code
### 4a. TypeScript Strict Typing (see `typescript.md`)
- [ ] **NO `any` types** — explicit types required everywhere
- [ ] **NO lazy `unknown`** — only for error catches with immediate narrowing
- [ ] **Explicit return types** on all exported/public functions
- [ ] **Explicit parameter types** — never implicit any
- [ ] **No type assertions** (`as Type`) — use type guards instead
- [ ] **No non-null assertions** (`!`) — use proper null handling
- [ ] **Interfaces for objects** — not inline types
- [ ] **Discriminated unions** for variant types
### 5. Documentation
- [ ] Complex logic has explanatory comments
- [ ] Public APIs are documented
- [ ] README updated if needed
- [ ] Breaking changes noted
### 6. Performance
- [ ] No obvious N+1 queries
- [ ] No blocking operations in hot paths
- [ ] Resource cleanup (connections, file handles)
- [ ] Reasonable memory usage
### 7. Dependencies
- [ ] No deprecated packages
- [ ] No unnecessary new dependencies
- [ ] Dependency versions pinned appropriately
## Review Process
### Getting Context
```bash
# List the issue being addressed
~/.mosaic/rails/git/issue-list.sh -i {issue-number}
# View the changes
git diff main...HEAD
```
### Providing Feedback
- Be specific: point to exact lines/files
- Explain WHY something is problematic
- Suggest alternatives when possible
- Distinguish between blocking issues and suggestions
- Be constructive, not critical of the person
### Feedback Categories
- **Blocker**: Must fix before merge (security, bugs, test failures)
- **Should Fix**: Important but not blocking (code quality, minor issues)
- **Suggestion**: Optional improvements (style preferences, nice-to-haves)
- **Question**: Seeking clarification
### Review Comment Format
```
[BLOCKER] Line 42: SQL injection vulnerability
The user input is directly interpolated into the query.
Use parameterized queries instead:
`db.query("SELECT * FROM users WHERE id = ?", [userId])`
[SUGGESTION] Line 78: Consider extracting to helper
This pattern appears in 3 places. A shared helper would reduce duplication.
```
## After Review
1. Update issue with review status
2. If changes requested, assign back to author
3. If approved, note approval in issue comments
4. For merges, ensure CI passes first

80
guides/frontend.md Normal file
View File

@@ -0,0 +1,80 @@
# Frontend Development Guide
## Before Starting
1. Check assigned issue in git repo: `~/.mosaic/rails/git/issue-list.sh -a @me`
2. Create scratchpad: `docs/scratchpads/{issue-number}-{short-name}.md`
3. Review existing components and patterns in the codebase
## Development Standards
### Framework Conventions
- Follow project's existing framework patterns (React, Vue, Svelte, etc.)
- Use existing component library/design system if present
- Maintain consistent file structure with existing code
### Styling
- Use project's established styling approach (CSS modules, Tailwind, styled-components, etc.)
- Follow existing naming conventions for CSS classes
- Ensure responsive design unless explicitly single-platform
### State Management
- Use project's existing state management solution
- Keep component state local when possible
- Document any new global state additions
### Accessibility
- Include proper ARIA labels
- Ensure keyboard navigation works
- Test with screen reader considerations
- Maintain color contrast ratios (WCAG 2.1 AA minimum)
## Testing Requirements (TDD)
1. Write tests BEFORE implementation
2. Minimum 85% coverage
3. Test categories:
- Unit tests for utility functions
- Component tests for UI behavior
- Integration tests for user flows
### Test Patterns
```javascript
// Component test example structure
describe('ComponentName', () => {
it('renders without crashing', () => {});
it('handles user interaction correctly', () => {});
it('displays error states appropriately', () => {});
it('is accessible', () => {});
});
```
## Code Style
- Follow Google JavaScript/TypeScript Style Guide
- **TypeScript: Follow `~/.mosaic/guides/typescript.md` — MANDATORY**
- Use ESLint/Prettier configuration from project
- Prefer functional components over class components (React)
- TypeScript strict mode is REQUIRED, not optional
### TypeScript Quick Rules (see typescript.md for full guide)
- **NO `any`** — define explicit types always
- **NO lazy `unknown`** — only for error catches and external data with validation
- **Explicit return types** on all exported functions
- **Explicit parameter types** always
- **Interface for props** — never inline object types
- **Event handlers** — use proper React event types
## Commit Format
```
feat(#123): Add user profile component
- Implement avatar display
- Add edit mode toggle
- Include form validation
Refs #123
```
## Before Completing
1. Run full test suite
2. Verify build succeeds
3. Update scratchpad with completion notes
4. Reference issue in commit: `Fixes #N` or `Refs #N`

165
guides/infrastructure.md Normal file
View File

@@ -0,0 +1,165 @@
# Infrastructure & DevOps Guide
## Before Starting
1. Check assigned issue: `~/.mosaic/rails/git/issue-list.sh -a @me`
2. Create scratchpad: `docs/scratchpads/{issue-number}-{short-name}.md`
3. Review existing infrastructure configuration
## Vault Secrets Management
**CRITICAL**: Follow canonical Vault structure for ALL secrets.
### Structure
```
{mount}/{service}/{component}/{secret-name}
Examples:
- secret-prod/postgres/database/app
- secret-prod/redis/auth/default
- secret-prod/authentik/admin/token
```
### Environment Mounts
- `secret-dev/` - Development environment
- `secret-staging/` - Staging environment
- `secret-prod/` - Production environment
### Standard Field Names
- Credentials: `username`, `password`
- Tokens: `token`
- OAuth: `client_id`, `client_secret`
- Connection strings: `url`, `host`, `port`
See `docs/vault-secrets-structure.md` for complete reference.
## Container Standards
### Dockerfile Best Practices
```dockerfile
# Use specific version tags
FROM node:20-alpine
# Create non-root user
RUN addgroup -S app && adduser -S app -G app
# Set working directory
WORKDIR /app
# Copy dependency files first (layer caching)
COPY package*.json ./
RUN npm ci --only=production
# Copy application code
COPY --chown=app:app . .
# Switch to non-root user
USER app
# Use exec form for CMD
CMD ["node", "server.js"]
```
### Container Security
- Use minimal base images (alpine, distroless)
- Run as non-root user
- Don't store secrets in images
- Scan images for vulnerabilities
- Pin dependency versions
## Kubernetes/Docker Compose
### Resource Limits
Always set resource limits to prevent runaway containers:
```yaml
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "500m"
```
### Health Checks
```yaml
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 3
```
## CI/CD Pipelines
### Pipeline Stages
1. **Lint**: Code style and static analysis
2. **Test**: Unit and integration tests
3. **Build**: Compile and package
4. **Scan**: Security and vulnerability scanning
5. **Deploy**: Environment-specific deployment
### Pipeline Security
- Use secrets management (not hardcoded)
- Pin action/image versions
- Implement approval gates for production
- Audit pipeline access
## Monitoring & Logging
### Logging Standards
- Use structured logging (JSON)
- Include correlation IDs
- Log at appropriate levels (ERROR, WARN, INFO, DEBUG)
- Never log sensitive data
### Metrics to Collect
- Request latency (p50, p95, p99)
- Error rates
- Resource utilization (CPU, memory)
- Business metrics
### Alerting
- Define SLOs (Service Level Objectives)
- Alert on symptoms, not causes
- Include runbook links in alerts
- Avoid alert fatigue
## Testing Infrastructure
### Test Categories
1. **Unit tests**: Terraform/Ansible logic
2. **Integration tests**: Deployed resources work together
3. **Smoke tests**: Critical paths after deployment
4. **Chaos tests**: Failure mode validation
### Infrastructure Testing Tools
- Terraform: `terraform validate`, `terraform plan`
- Ansible: `ansible-lint`, molecule
- Kubernetes: `kubectl dry-run`, kubeval
- General: Terratest, ServerSpec
## Commit Format
```
chore(#67): Configure Redis cluster
- Add Redis StatefulSet with 3 replicas
- Configure persistence with PVC
- Add Vault secret for auth password
Refs #67
```
## Before Completing
1. Validate configuration syntax
2. Run infrastructure tests
3. Test in dev/staging first
4. Document any manual steps required
5. Update scratchpad and close issue

126
guides/orchestrator-learnings.md Normal file
View File

@@ -0,0 +1,126 @@
# Orchestrator Learnings (Universal)
> Cross-project heuristic adjustments based on observed variance data.
>
> **Note:** This file contains generic patterns only. Project-specific evidence is stored in each project's `docs/orchestrator-learnings.json`.
## Task Type Multipliers
Apply these multipliers to base estimates from `orchestrator.md`:
| Task Type | Base Estimate | Multiplier | Confidence | Samples | Last Updated |
|-----------|---------------|------------|------------|---------|--------------|
| STYLE_FIX | 3-5K | 0.64 | MEDIUM | n=1 | 2026-02-05 |
| BULK_CLEANUP | file_count × 550 | 1.0 | MEDIUM | n=2 | 2026-02-05 |
| GUARD_ADD | 5-8K | 1.0 | LOW | n=0 | - |
| SECURITY_FIX | 8-12K | 2.5 | LOW | n=0 | - |
| AUTH_ADD | 15-25K | 1.0 | HIGH | n=1 | 2026-02-05 |
| REFACTOR | 10-15K | 1.0 | LOW | n=0 | - |
| TEST_ADD | 15-25K | 1.0 | LOW | n=0 | - |
| ERROR_HANDLING | 8-12K | 2.3 | MEDIUM | n=1 | 2026-02-05 |
| CONFIG_DEFAULT_CHANGE | 5-10K | 1.8 | MEDIUM | n=1 | 2026-02-05 |
| INPUT_VALIDATION | 5-8K | 1.7 | MEDIUM | n=1 | 2026-02-05 |
## Phase Factors
Apply to all estimates based on task position in milestone:
| Phase Position | Factor | Rationale |
|----------------|--------|-----------|
| Early (tasks 1-3) | 1.45 | Codebase learning overhead |
| Mid (tasks 4-7) | 1.25 | Pattern recognition phase |
| Late (tasks 8+) | 1.10 | Established patterns |
## Estimation Formula
```
Final Estimate = Base Estimate × Type Multiplier × Phase Factor × TDD Overhead
Where:
- Base Estimate: From orchestrator.md task type table
- Type Multiplier: From table above (default 1.0)
- Phase Factor: 1.45 / 1.25 / 1.10 based on position
- TDD Overhead: 1.20 if tests required
```
## Known Patterns
### BULK_CLEANUP
**Pattern:** Multi-file cleanup tasks are severely underestimated.
**Why:** Iterative testing across many files, cascading fixes, and debugging compound the effort.
**Observed:** +112% to +276% variance when using fixed estimates.
**Recommendation:** Use `file_count × 550` instead of fixed estimate.
### ERROR_HANDLING
**Pattern:** Error handling changes that modify type interfaces cascade through the codebase.
**Why:** Adding fields to result types requires updating all callers, error messages, and tests.
**Observed:** +131% variance.
**Multiplier:** 2.3x base estimate when type interfaces are modified.
### CONFIG_DEFAULT_CHANGE
**Pattern:** Config default changes require more test coverage than expected.
**Why:** Security-sensitive defaults need validation tests, warning tests, and edge case coverage.
**Observed:** +80% variance.
**Multiplier:** 1.8x when config changes need security validation.
### INPUT_VALIDATION
**Pattern:** Security input validation with allowlists is more complex than simple validation.
**Why:** Comprehensive allowlists (e.g., OAuth error codes), encoding requirements, and security tests add up.
**Observed:** +70% variance.
**Multiplier:** 1.7x when security allowlists are involved.
### STYLE_FIX
**Pattern:** Pure formatting fixes are faster than estimated when isolated.
**Observed:** -36% variance.
**Multiplier:** 0.64x for isolated style-only fixes.
## Changelog
| Date | Change | Samples | Confidence |
|------|--------|---------|------------|
| 2026-02-05 | Added BULK_CLEANUP category | n=2 | MEDIUM |
| 2026-02-05 | Added STYLE_FIX multiplier 0.64 | n=1 | MEDIUM |
| 2026-02-05 | Confirmed AUTH_ADD heuristic accurate | n=1 | HIGH |
| 2026-02-05 | Added ERROR_HANDLING multiplier 2.3x | n=1 | MEDIUM |
| 2026-02-05 | Added CONFIG_DEFAULT_CHANGE multiplier 1.8x | n=1 | MEDIUM |
| 2026-02-05 | Added INPUT_VALIDATION multiplier 1.7x | n=1 | MEDIUM |
## Update Protocol
**Graduated Autonomy:**
| Phase | Condition | Action |
|-------|-----------|--------|
| **Now** | All proposals | Human review required |
| **After 3 milestones** | <30% change, n≥3 samples, HIGH confidence | Auto-update allowed |
| **Mature** | All changes | Auto with notification, revert on regression |
**Validation Before Update:**
1. Minimum 3 samples for same task type
2. Standard deviation < 30% of mean
3. Outliers (>2σ) excluded
4. New formula must not increase variance on historical data
## Where to Find Project-Specific Data
- **Project learnings:** `<project>/docs/orchestrator-learnings.json`
- **Cross-project metrics:** `jarvis-brain/data/orchestrator-metrics.json`

866
guides/orchestrator.md Normal file
View File

@@ -0,0 +1,866 @@
# Autonomous Orchestrator Guide
> Load this guide when orchestrating autonomous task completion across any project.
## Overview
The orchestrator **cold-starts** on any project with just a review report location and minimal kickstart. It autonomously:
1. Parses review reports to extract findings
2. Categorizes findings into phases by severity
3. Estimates token usage per task
4. Creates Gitea issues (phase-level)
5. Bootstraps `docs/tasks.md` from scratch
6. Coordinates completion using worker agents
**Key principle:** The orchestrator is the **sole writer** of `docs/tasks.md`. Worker agents execute tasks and report results — they never modify the tracking file.
---
## Orchestrator Boundaries (CRITICAL)
**The orchestrator NEVER:**
- Edits source code directly (*.ts, *.tsx, *.js, *.py, etc.)
- Runs quality gates itself (that's the worker's job)
- Makes commits containing code changes
- "Quickly fixes" something to save time — this is how drift starts
**The orchestrator ONLY:**
- Reads/writes `docs/tasks.md`
- Reads/writes `docs/orchestrator-learnings.json`
- Spawns workers via the Task tool for ALL code changes
- Parses worker JSON results
- Commits task tracking updates (tasks.md, learnings)
- Outputs status reports and handoff messages
**If you find yourself about to edit source code, STOP.**
Spawn a worker instead. No exceptions. No "quick fixes."
**Worker Limits:**
- Maximum **2 parallel workers** at any time
- Wait for at least one worker to complete before spawning more
- This optimizes token usage and reduces context pressure
---
## Bootstrap Templates
Use templates from `jarvis-brain/docs/templates/` to scaffold tracking files:
```bash
# Set environment variables
export PROJECT="project-name"
export MILESTONE="M1-Feature"
export CURRENT_DATETIME=$(date -Iseconds)
export TASK_PREFIX="PR-SEC"
export PHASE_ISSUE="#1"
export PHASE_BRANCH="fix/security"
# Copy templates
TEMPLATES=~/src/jarvis-brain/docs/templates
# Create tasks.md (then populate with findings)
envsubst < $TEMPLATES/orchestrator/tasks.md.template > docs/tasks.md
# Create learnings tracking
envsubst < $TEMPLATES/orchestrator/orchestrator-learnings.json.template > docs/orchestrator-learnings.json
# Create review report structure (if doing new review)
$TEMPLATES/reports/review-report-scaffold.sh codebase-review
```
**Available templates:**
| Template | Purpose |
|----------|---------|
| `orchestrator/tasks.md.template` | Task tracking table with schema |
| `orchestrator/orchestrator-learnings.json.template` | Variance tracking |
| `orchestrator/phase-issue-body.md.template` | Gitea issue body |
| `orchestrator/compaction-summary.md.template` | 60% checkpoint format |
| `reports/review-report-scaffold.sh` | Creates report directory |
| `scratchpad.md.template` | Per-task working document |
See `jarvis-brain/docs/templates/README.md` for full documentation.
---
## Phase 1: Bootstrap
### Step 1: Parse Review Reports
Review reports typically follow this structure:
```
docs/reports/{report-name}/
├── 00-executive-summary.md # Start here - overview and counts
├── 01-security-review.md # Security findings with IDs like SEC-*
├── 02-code-quality-review.md # Code quality findings like CQ-*
├── 03-qa-test-coverage.md # Test coverage gaps like TEST-*
└── ...
```
**Extract findings by looking for:**
- Finding IDs (e.g., `SEC-API-1`, `CQ-WEB-3`, `TEST-001`)
- Severity labels: Critical, High, Medium, Low
- Affected files/components (use for `repo` column)
- Specific line numbers or code patterns
**Parse each finding into:**
```
{
id: "SEC-API-1",
severity: "critical",
title: "Brief description",
component: "api", // For repo column
file: "path/to/file.ts", // Reference for worker
lines: "45-67" // Specific location
}
```
### Step 2: Categorize into Phases
Map severity to phases:
| Severity | Phase | Focus | Branch Pattern |
|----------|-------|-------|----------------|
| Critical | 1 | Security vulnerabilities, data exposure | `fix/security` |
| High | 2 | Security hardening, auth gaps | `fix/security` |
| Medium | 3 | Code quality, performance, bugs | `fix/code-quality` |
| Low | 4 | Tests, documentation, cleanup | `fix/test-coverage` |
**Within each phase, order tasks by:**
1. Blockers first (tasks that unblock others)
2. Same-file tasks grouped together
3. Simpler fixes before complex ones
### Step 3: Estimate Token Usage
Use these heuristics based on task type:
| Task Type | Estimate | Examples |
|-----------|----------|----------|
| Single-line fix | 3-5K | Typo, wrong operator, missing null check |
| Add guard/validation | 5-8K | Add auth decorator, input validation |
| Fix error handling | 8-12K | Proper try/catch, error propagation |
| Refactor pattern | 10-15K | Replace KEYS with SCAN, fix memory leak |
| Add new functionality | 15-25K | New service method, new component |
| Write tests | 15-25K | Unit tests for untested service |
| Complex refactor | 25-40K | Architectural change, multi-file refactor |
**Adjust estimates based on:**
- Number of files affected (+5K per additional file)
- Test requirements (+5-10K if tests needed)
- Documentation needs (+2-3K if docs needed)
### Step 4: Determine Dependencies
**Automatic dependency rules:**
1. All tasks in Phase N depend on the Phase N-1 verification task
2. Tasks touching the same file should be sequential (earlier blocks later)
3. Auth/security foundation tasks block tasks that rely on them
4. Each phase ends with a verification task that depends on all phase tasks
**Create verification tasks:**
- `{PREFIX}-SEC-{LAST}`: Phase 1 verification (run security tests)
- `{PREFIX}-HIGH-{LAST}`: Phase 2 verification
- `{PREFIX}-CQ-{LAST}`: Phase 3 verification
- `{PREFIX}-TEST-{LAST}`: Phase 4 verification (final quality gates)
### Step 5: Create Gitea Issues (Phase-Level)
Create ONE issue per phase using git scripts:
```bash
~/.mosaic/rails/git/issue-create.sh \
-t "Phase 1: Critical Security Fixes" \
-b "$(cat <<'EOF'
## Findings
- SEC-API-1: Description
- SEC-WEB-2: Description
- SEC-ORCH-1: Description
## Acceptance Criteria
- [ ] All critical findings remediated
- [ ] Quality gates passing
- [ ] No new regressions
EOF
)" \
-l "security,critical" \
-m "{milestone-name}"
```
**Capture issue numbers** — you'll link tasks to these.
### Step 6: Create docs/tasks.md
Create the file with this exact schema:
```markdown
# Tasks
| id | status | description | issue | repo | branch | depends_on | blocks | agent | started_at | completed_at | estimate | used |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| {PREFIX}-SEC-001 | not-started | SEC-API-1: Brief description | #{N} | api | fix/security | | {PREFIX}-SEC-002 | | | | 8K | |
```
**Column definitions:**
| Column | Format | Purpose |
|--------|--------|---------|
| `id` | `{PREFIX}-{CAT}-{NNN}` | Unique task ID (e.g., MS-SEC-001) |
| `status` | `not-started` \| `in-progress` \| `done` \| `failed` | Current state |
| `description` | `{FindingID}: Brief summary` | What to fix |
| `issue` | `#NNN` | Gitea issue (phase-level, all tasks in phase share) |
| `repo` | Workspace name | `api`, `web`, `orchestrator`, etc. |
| `branch` | Branch name | `fix/security`, `fix/code-quality`, etc. |
| `depends_on` | Comma-separated IDs | Must complete first |
| `blocks` | Comma-separated IDs | Tasks waiting on this |
| `agent` | Agent identifier | Assigned worker (fill when claiming) |
| `started_at` | ISO 8601 | When work began |
| `completed_at` | ISO 8601 | When work finished |
| `estimate` | `5K`, `15K`, etc. | Predicted token usage |
| `used` | `4.2K`, `12.8K`, etc. | Actual usage (fill on completion) |
**Category prefixes:**
- `SEC` — Security (Phase 1-2)
- `HIGH` — High priority (Phase 2)
- `CQ` — Code quality (Phase 3)
- `TEST` — Test coverage (Phase 4)
- `PERF` — Performance (Phase 3)
### Step 7: Commit Bootstrap
```bash
git add docs/tasks.md
git commit -m "chore(orchestrator): Bootstrap tasks.md from review report
Parsed {N} findings into {M} tasks across {P} phases.
Estimated total: {X}K tokens."
git push
```
---
## Phase 2: Execution Loop
```
1. git pull --rebase
2. Read docs/tasks.md
3. Find next task: status=not-started AND all depends_on are done
4. If no task available:
- All done? → Report success, run final retrospective, STOP
- Some blocked? → Report deadlock, STOP
5. Update tasks.md: status=in-progress, agent={identifier}, started_at={now}
6. Spawn worker agent (Task tool) with task details
7. Wait for worker completion
8. Parse worker result (JSON)
9. **Variance check**: Calculate (actual - estimate) / estimate × 100
- If |variance| > 50%: Capture learning (see Learning & Retrospective)
- If |variance| > 100%: Flag as CRITICAL — review task classification
10. **Post-Coding Review** (see Phase 2b below)
11. Update tasks.md: status=done/failed/needs-qa, completed_at={now}, used={actual}
12. **Cleanup reports**: Remove processed report files for completed task
```bash
# Find and remove reports matching the finding ID
find docs/reports/qa-automation/pending/ -name "*{finding_id}*" -delete 2>/dev/null || true
# If task failed, move reports to escalated/ instead
```
13. Commit + push: git add docs/tasks.md .gitignore && git commit && git push
14. If phase verification task: Run phase retrospective, clean up all phase reports
15. Check context usage
16. If >= 55%: Output COMPACTION REQUIRED checkpoint, STOP, wait for user
17. If < 55%: Go to step 1
18. After user runs /compact and says "continue": Go to step 1
```
---
## Phase 2b: Post-Coding Review (MANDATORY)
**CRITICAL:** After any worker completes a task that modifies source code, the orchestrator MUST run an independent review before marking the task as done. This catches bugs, security issues, and regressions that the worker missed.
### When to Review
Run review when the worker's result includes code changes (commits). Skip for tasks that only modify docs, config, or tracking files.
### Step 1: Run Codex Review (Primary)
```bash
# Navigate to the project directory
cd {project_path}
# Code quality review
~/.mosaic/rails/codex/codex-code-review.sh -b {base_branch} -o /tmp/review-{task_id}.json
# Security review
~/.mosaic/rails/codex/codex-security-review.sh -b {base_branch} -o /tmp/security-{task_id}.json
```
### Step 2: Parse Review Results
```bash
# Check code review
CODE_BLOCKERS=$(jq '.stats.blockers // 0' /tmp/review-{task_id}.json)
CODE_VERDICT=$(jq -r '.verdict // "comment"' /tmp/review-{task_id}.json)
# Check security review
SEC_CRITICAL=$(jq '.stats.critical // 0' /tmp/security-{task_id}.json)
SEC_HIGH=$(jq '.stats.high // 0' /tmp/security-{task_id}.json)
```
### Step 3: Decision Tree
```
IF Codex is unavailable (command not found, auth failure, API error):
→ Use fallback review (Step 4)
IF CODE_BLOCKERS > 0 OR SEC_CRITICAL > 0 OR SEC_HIGH > 0:
→ Mark task as "needs-qa" in tasks.md
→ Create a remediation task:
- ID: {task_id}-QA
- Description: Fix findings from review (list specific issues)
- depends_on: (none — it's a follow-up, not a blocker)
- Notes: Include finding titles and file locations
→ Continue to next task (remediation task will be picked up in order)
IF CODE_VERDICT == "request-changes" (but no blockers):
→ Log should-fix findings in task notes
→ Mark task as done (non-blocking suggestions)
→ Consider creating a tech-debt issue for significant suggestions
IF CODE_VERDICT == "approve" AND SEC_CRITICAL == 0 AND SEC_HIGH == 0:
→ Mark task as done
→ Log: "Review passed — no issues found"
```
### Step 4: Fallback Review (When Codex is Unavailable)
If the `codex` CLI is not installed or authentication fails, use Claude's built-in review capabilities:
```markdown
## Fallback: Spawn a Review Agent
Use the Task tool to spawn a review subagent:
Prompt:
---
## Independent Code Review
Review the code changes on branch {branch} against {base_branch}.
1. Run: `git diff {base_branch}...HEAD`
2. Review for:
- Correctness (bugs, logic errors, edge cases)
- Security (OWASP Top 10, secrets, injection)
- Testing (coverage, quality)
- Code quality (complexity, duplication)
3. Reference: ~/.mosaic/guides/code-review.md
Report findings as JSON:
```json
{
"verdict": "approve|request-changes",
"blockers": 0,
"critical_security": 0,
"findings": [
{"severity": "blocker|should-fix|suggestion", "title": "...", "file": "...", "description": "..."}
]
}
```
---
```
### Review Timing Guidelines
| Task Type | Review Required? |
|-----------|-----------------|
| Source code changes (*.ts, *.py, etc.) | **YES — always** |
| Configuration changes (*.yml, *.toml) | YES — security review only |
| Documentation changes (*.md) | No |
| Task tracking updates (tasks.md) | No |
| Test-only changes | YES — code review only |
### Logging Review Results
In the task notes column of tasks.md, append review results:
```
Review: approve (0 blockers, 0 critical) | Codex 0.98.0
```
or:
```
Review: needs-qa (1 blocker, 2 high) → QA task {task_id}-QA created
```
---
## Worker Prompt Template
Construct this from the task row and pass to worker via Task tool:
````markdown
## Task Assignment: {id}
**Description:** {description}
**Repository:** {project_path}/apps/{repo}
**Branch:** {branch}
**Reference:** See `docs/reports/` for detailed finding description. Search for the finding ID.
## Workflow
1. Checkout branch: `git checkout {branch} || git checkout -b {branch} develop && git pull`
2. Read the finding details from the report
3. Implement the fix following existing code patterns
4. Run quality gates (ALL must pass — zero lint errors, zero type errors, all tests green):
```bash
{quality_gates_command}
```
**MANDATORY:** This ALWAYS includes linting. If the project has a linter configured
(ESLint, Biome, ruff, etc.), you MUST run it and fix ALL violations in files you touched.
Do NOT leave lint warnings or errors for someone else to clean up.
5. If gates fail: Fix and retry. Do NOT report success with failures.
6. Commit: `git commit -m "fix({finding_id}): brief description"`
7. Push: `git push origin {branch}`
8. Report result as JSON (see format below)
## Git Scripts
For issue/PR/milestone operations, use scripts (NOT raw tea/gh):
- `~/.mosaic/rails/git/issue-view.sh -i {N}`
- `~/.mosaic/rails/git/pr-create.sh -t "Title" -b "Desc" -B develop`
Standard git commands (pull, commit, push, checkout) are fine.
## Result Format (MANDATORY)
End your response with this JSON block:
```json
{
"task_id": "{id}",
"status": "success|failed",
"used": "5.2K",
"commit_sha": "abc123",
"notes": "Brief summary of what was done"
}
```
## Post-Coding Review
After you complete and push your changes, the orchestrator will independently
review your code using Codex (or a fallback review agent). If the review finds
blockers or critical security issues, a follow-up remediation task will be
created. You do NOT need to run the review yourself — the orchestrator handles it.
## Rules
- DO NOT modify docs/tasks.md
- DO NOT claim other tasks
- Complete this single task, report results, done
````
---
## Context Threshold Protocol (Orchestrator Replacement)
**Threshold:** 55-60% context usage
**Why replacement, not compaction?**
- Compaction causes **protocol drift** — agent "remembers" gist but loses specifics
- Post-compaction agents may violate core rules (e.g., letting workers modify tasks.md)
- Fresh orchestrator has **100% protocol fidelity**
- All state lives in `docs/tasks.md` — the orchestrator is **stateless and replaceable**
**At threshold (55-60%):**
1. Complete current task
2. Persist all state:
- Update docs/tasks.md with all progress
- Update docs/orchestrator-learnings.json with variances
- Commit and push both files
3. Output **ORCHESTRATOR HANDOFF** message with ready-to-use takeover kickstart
4. **STOP COMPLETELY** — do not continue working
**Handoff message format:**
```
---
⚠️ ORCHESTRATOR HANDOFF REQUIRED
Context: {X}% — Replacement recommended to prevent drift
Progress: {completed}/{total} tasks ({percentage}%)
Current phase: Phase {N} ({phase_name})
State persisted:
- docs/tasks.md ✓
- docs/orchestrator-learnings.json ✓
## Takeover Kickstart
Copy and paste this to spawn a fresh orchestrator:
---
## Continuation Mission
Continue {mission_description} from existing state.
## Setup
- Project: {project_path}
- State: docs/tasks.md (already populated)
- Protocol: docs/claude/orchestrator.md
- Quality gates: {quality_gates_command}
## Resume Point
- Next task: {task_id}
- Phase: {current_phase}
- Progress: {completed}/{total} tasks ({percentage}%)
## Instructions
1. Read docs/claude/orchestrator.md for protocol
2. Read docs/tasks.md to understand current state
3. Continue execution from task {task_id}
4. Follow Two-Phase Completion Protocol
5. You are the SOLE writer of docs/tasks.md
---
STOP: Terminate this session and spawn fresh orchestrator with the kickstart above.
---
```
**Rules:**
- Do NOT attempt to compact yourself — compaction causes drift
- Do NOT continue past 60%
- Do NOT claim you can "just continue" — protocol drift is real
- STOP means STOP — the user (Coordinator) will spawn your replacement
- Include ALL context needed for the replacement in the takeover kickstart
---
## Two-Phase Completion Protocol
Each major phase uses a two-phase approach to maximize completion while managing diminishing returns.
### Bulk Phase (Target: 90%)
- Focus on tractable errors
- Parallelize where possible
- When 90% reached, transition to Polish (do NOT declare success)
### Polish Phase (Target: 100%)
1. **Inventory:** List all remaining errors with file:line
2. **Categorize:**
| Category | Criteria | Action |
|----------|----------|--------|
| Quick-win | <5 min, straightforward | Fix immediately |
| Medium | 5-30 min, clear path | Fix in order |
| Hard | >30 min or uncertain | Attempt 15 min, then document |
| Architectural | Requires design change | Document and defer |
3. **Work priority:** Quick-win → Medium → Hard
4. **Document deferrals** in `docs/deferred-errors.md`:
```markdown
## {PREFIX}-XXX: [Error description]
- File: path/to/file.ts:123
- Error: [exact error message]
- Category: Hard | Architectural | Framework Limitation
- Reason: [why this is non-trivial]
- Suggested approach: [how to fix in future]
- Risk: Low | Medium | High
```
5. **Phase complete when:**
- All Quick-win/Medium fixed
- All Hard attempted (fixed or documented)
- Architectural items documented with justification
### Phase Boundary Rule
Do NOT proceed to the next major phase until the current phase reaches Polish completion:
```
✅ Phase 2 Bulk: 91%
✅ Phase 2 Polish: 118 errors triaged
- 40 medium → fixed
- 78 low → EACH documented with rationale
✅ Phase 2 Complete: Created docs/deferred-errors.md
→ NOW proceed to Phase 3
❌ WRONG: Phase 2 at 91%, "low priority acceptable", starting Phase 3
```
### Reporting
When transitioning from Bulk to Polish:
```
Phase X Bulk Complete: {N}% ({fixed}/{total})
Entering Polish Phase: {remaining} errors to triage
```
When Polish Phase complete:
```
Phase X Complete: {final_pct}% ({fixed}/{total})
- Quick-wins: {n} fixed
- Medium: {n} fixed
- Hard: {n} fixed, {n} documented
- Framework limitations: {n} documented
```
---
## Learning & Retrospective
Orchestrators capture learnings to improve future estimation accuracy.
### Variance Thresholds
| Variance | Action |
|----------|--------|
| 0-30% | Log only (acceptable) |
| 30-50% | Flag for review |
| 50-100% | Capture learning to `docs/orchestrator-learnings.json` |
| >100% | CRITICAL — review task classification, possible mismatch |
### Task Type Classification
Classify tasks by description keywords for pattern analysis:
| Type | Keywords | Base Estimate |
|------|----------|---------------|
| STYLE_FIX | "formatting", "prettier", "lint" | 3-5K |
| BULK_CLEANUP | "unused", "warnings", "~N files" | file_count × 550 |
| GUARD_ADD | "add guard", "decorator", "validation" | 5-8K |
| SECURITY_FIX | "sanitize", "injection", "XSS" | 8-12K × 2.5 |
| AUTH_ADD | "authentication", "auth" | 15-25K |
| REFACTOR | "refactor", "replace", "migrate" | 10-15K |
| TEST_ADD | "add tests", "coverage" | 15-25K |
### Capture Learning
When |variance| > 50%, append to `docs/orchestrator-learnings.json`:
```json
{
"task_id": "UC-CLEAN-003",
"task_type": "BULK_CLEANUP",
"estimate_k": 30,
"actual_k": 112.8,
"variance_pct": 276,
"characteristics": {
"file_count": 200,
"keywords": ["object injection", "type guards"]
},
"analysis": "Multi-file type guards severely underestimated",
"captured_at": "2026-02-05T19:45:00Z"
}
```
### Retrospective Triggers
| Trigger | Action |
|---------|--------|
| Phase verification task | Analyze phase variance, summarize patterns |
| 60% compaction | Persist learnings buffer, include in summary |
| Milestone complete | Full retrospective, generate heuristic proposals |
### Enhanced Compaction Summary
Include learnings in compaction output:
```
Session Summary (Compacting at 60%):
Completed: MS-SEC-001 (15K→0.3K, -98%), MS-SEC-002 (8K→12K, +50%)
Quality: All gates passing
Learnings Captured:
- MS-SEC-001: -98% variance — AUTH_ADD may need SKIP_IF_EXISTS category
- MS-SEC-002: +50% variance — XSS sanitization more complex than expected
Remaining: MS-SEC-004 (ready), MS-SEC-005 through MS-SEC-010
Next: MS-SEC-004
```
### Cross-Project Learnings
Universal heuristics are maintained in `~/.mosaic/guides/orchestrator-learnings.md`.
After completing a milestone, review variance patterns and propose updates to the universal guide.
---
## Report Cleanup
QA automation generates report files in `docs/reports/qa-automation/pending/`. These must be cleaned up to prevent accumulation.
**Directory structure:**
```
docs/reports/qa-automation/
├── pending/ # Reports awaiting processing
└── escalated/ # Reports for failed tasks (manual review needed)
```
**Gitignore:** Add this to project `.gitignore`:
```
# Orchestrator reports (generated by QA automation, cleaned up after processing)
docs/reports/qa-automation/
```
**Cleanup timing:**
| Event | Action |
|-------|--------|
| Task success | Delete matching reports from `pending/` |
| Task failed | Move reports to `escalated/` for investigation |
| Phase verification | Clean up all `pending/` reports for that phase |
| Milestone complete | Archive or delete entire `escalated/` directory |
**Cleanup commands:**
```bash
# After successful task (finding ID pattern, e.g., SEC-API-1)
find docs/reports/qa-automation/pending/ -name "*relevant-file-pattern*" -delete
# After phase verification - clean all pending
rm -rf docs/reports/qa-automation/pending/*
# Move failed task reports to escalated
mv docs/reports/qa-automation/pending/*failing-file* docs/reports/qa-automation/escalated/
```
---
## Error Handling
**Quality gates fail:**
1. Worker should retry up to 2 times
2. If still failing, worker reports `failed` with error details
3. Orchestrator updates tasks.md: keep `in-progress`, add notes
4. Orchestrator may re-spawn with error context, or mark `failed` and continue
5. If failed task blocks others: Report deadlock, STOP
**Worker reports blocker:**
1. Update tasks.md with blocker notes
2. Skip to next unblocked task if possible
3. If all remaining tasks blocked: Report blockers, STOP
**Git push conflict:**
1. `git pull --rebase`
2. If auto-resolves: push again
3. If conflict on tasks.md: Report, STOP (human resolves)
---
## Stopping Criteria
**ONLY stop if:**
1. All tasks in docs/tasks.md are `done`
2. Critical blocker preventing progress (document and alert)
3. Context usage >= 55% — output COMPACTION REQUIRED checkpoint and wait
4. Absolute context limit reached AND cannot compact further
**DO NOT stop to ask "should I continue?"** — the answer is always YES.
**DO stop at 55-60%** — output the compaction checkpoint and wait for user to run `/compact`.
---
## Sprint Completion Protocol
When all tasks in `docs/tasks.md` are `done` (or triaged as `deferred`), archive the sprint artifacts before stopping. This preserves them for post-mortems, variance calibration, and historical reference.
### Archive Steps
1. **Create archive directory** (if it doesn't exist):
```bash
mkdir -p docs/tasks/
```
2. **Move tasks.md to archive:**
```bash
mv docs/tasks.md docs/tasks/{milestone-name}-tasks.md
```
Example: `docs/tasks/M6-AgentOrchestration-Fixes-tasks.md`
3. **Move learnings to archive:**
```bash
mv docs/orchestrator-learnings.json docs/tasks/{milestone-name}-learnings.json
```
4. **Commit the archive:**
```bash
git add docs/tasks/
git rm docs/tasks.md docs/orchestrator-learnings.json 2>/dev/null || true
git commit -m "chore(orchestrator): Archive {milestone-name} sprint artifacts
{completed}/{total} tasks completed, {deferred} deferred.
Archived to docs/tasks/ for post-mortem reference."
git push
```
5. **Run final retrospective** — review variance patterns and propose updates to estimation heuristics.
### Recovery
If an orchestrator starts and `docs/tasks.md` does not exist, check `docs/tasks/` for the most recent archive:
```bash
ls -t docs/tasks/*-tasks.md 2>/dev/null | head -1
```
If found, this may indicate another session archived the file. The orchestrator should:
1. Report what it found in `docs/tasks/`
2. Ask whether to resume from the archived file or bootstrap fresh
3. If resuming: copy the archive back to `docs/tasks.md` and continue
### Retention Policy
Keep all archived sprints indefinitely. They are small text files and valuable for:
- Post-mortem analysis
- Estimation variance calibration across milestones
- Understanding what was deferred and why
- Onboarding new orchestrators to project history
---
## Kickstart Message Format
The kickstart should be **minimal** — the orchestrator figures out the rest:
```markdown
## Mission
Remediate findings from the codebase review.
## Setup
- Project: /path/to/project
- Review: docs/reports/{report-name}/
- Quality gates: {command}
- Milestone: {milestone-name} (for issue creation)
- Task prefix: {PREFIX} (e.g., MS, UC)
## Protocol
Read ~/.mosaic/guides/orchestrator.md for full instructions.
## Start
Bootstrap from the review report, then execute until complete.
```
**The orchestrator will:**
1. Read this guide
2. Parse the review reports
3. Determine phases, estimates, dependencies
4. Create issues and tasks.md
5. Execute until done or blocked
---
## Quick Reference
| Phase | Action |
|-------|--------|
| Bootstrap | Parse reports → Categorize → Estimate → Create issues → Create tasks.md |
| Execute | Loop: claim → spawn worker → update → commit |
| Compact | At 60%: summarize, clear history, continue |
| Stop | Queue empty, blocker, or context limit |
**Orchestrator owns tasks.md. Workers execute and report. Single writer eliminates conflicts.**

202
guides/qa-testing.md Normal file
View File

@@ -0,0 +1,202 @@
# QA & Testing Guide
## Before Starting
1. Check assigned issue: `~/.mosaic/rails/git/issue-list.sh -a @me`
2. Create scratchpad: `docs/scratchpads/{issue-number}-{short-name}.md`
3. Review existing test structure and patterns
## Test-Driven Development (TDD) Process
### The TDD Cycle
1. **Red**: Write a failing test first
2. **Green**: Write minimal code to pass
3. **Refactor**: Improve code while keeping tests green
### TDD Rules
- Never write production code without a failing test
- Write only enough test to fail
- Write only enough code to pass
- Refactor continuously
## Coverage Requirements
### Minimum Standards
- **Overall Coverage**: 85% minimum
- **Critical Paths**: 95% minimum (auth, payments, data mutations)
- **New Code**: 90% minimum
### What to Cover
- All public interfaces
- Error handling paths
- Edge cases and boundaries
- Integration points
### What NOT to Count
- Generated code
- Configuration files
- Third-party library wrappers (thin wrappers only)
## Test Categories
### Unit Tests
- Test single functions/methods in isolation
- Mock external dependencies
- Fast execution (< 100ms per test)
- No network, database, or filesystem access
```python
def test_calculate_discount_applies_percentage():
result = calculate_discount(100, 0.20)
assert result == 80
```
### Integration Tests
- Test multiple components together
- Use real databases (test containers)
- Test API contracts
- Slower execution acceptable
```python
def test_create_user_persists_to_database(db_session):
user = create_user(db_session, "test@example.com")
retrieved = get_user_by_email(db_session, "test@example.com")
assert retrieved.id == user.id
```
### End-to-End Tests
- Test complete user workflows
- Use real browser (Playwright, Cypress)
- Test critical paths only (expensive to maintain)
```javascript
test('user can complete checkout', async ({ page }) => {
await page.goto('/products');
await page.click('[data-testid="add-to-cart"]');
await page.click('[data-testid="checkout"]');
await page.fill('#email', 'test@example.com');
await page.click('[data-testid="submit-order"]');
await expect(page.locator('.order-confirmation')).toBeVisible();
});
```
## Test Structure
### Naming Convention
```
test_{what}_{condition}_{expected_result}
Examples:
- test_login_with_valid_credentials_returns_token
- test_login_with_invalid_password_returns_401
- test_get_user_when_not_found_returns_404
```
### Arrange-Act-Assert Pattern
```python
def test_add_item_to_cart_increases_count():
# Arrange
cart = Cart()
item = Item(id=1, name="Widget", price=9.99)
# Act
cart.add(item)
# Assert
assert cart.item_count == 1
assert cart.total == 9.99
```
### Test Isolation
- Each test should be independent
- Use setup/teardown for common state
- Clean up after tests
- Don't rely on test execution order
## Mocking Guidelines
### When to Mock
- External APIs and services
- Time-dependent operations
- Random number generation
- Expensive operations
### When NOT to Mock
- The code under test
- Simple data structures
- Database in integration tests
### Mock Example
```python
def test_send_notification_calls_email_service(mocker):
mock_email = mocker.patch('services.email.send')
send_notification(user_id=1, message="Hello")
mock_email.assert_called_once_with(
to="user@example.com",
subject="Notification",
body="Hello"
)
```
## Test Data Management
### Fixtures
- Use factories for complex objects
- Keep test data close to tests
- Use realistic but anonymized data
### Database Tests
- Use transactions with rollback
- Or use test containers
- Never test against production data
## Reporting
### Test Reports Should Include
- Total tests run
- Pass/fail counts
- Coverage percentage
- Execution time
- Flaky test identification
### QA Report Template
```markdown
# QA Report - Issue #{number}
## Summary
- Tests Added: X
- Tests Modified: Y
- Coverage: XX%
## Test Results
- Passed: X
- Failed: X
- Skipped: X
## Coverage Analysis
- Lines: XX%
- Branches: XX%
- Functions: XX%
## Notes
[Any observations or concerns]
```
## Commit Format
```
test(#34): Add user registration tests
- Unit tests for validation logic
- Integration tests for /api/users endpoint
- Coverage increased from 72% to 87%
Refs #34
```
## Before Completing
1. All tests pass locally
2. Coverage meets 85% threshold
3. No flaky tests introduced
4. CI pipeline passes
5. Update scratchpad with results

382
guides/typescript.md Normal file
View File

@@ -0,0 +1,382 @@
# TypeScript Style Guide
**Authority**: This guide is MANDATORY for all TypeScript code. No exceptions without explicit approval.
Based on Google TypeScript Style Guide with stricter enforcement.
---
## Core Principles
1. **Explicit over implicit** — Always declare types, never rely on inference for public APIs
2. **Specific over generic** — Use the narrowest type that works
3. **Safe over convenient** — Type safety is not negotiable
---
## Forbidden Patterns (NEVER USE)
### `any` Type — FORBIDDEN
```typescript
// ❌ NEVER
function process(data: any) { }
const result: any = fetchData();
Record<string, any>
// ✅ ALWAYS define explicit types
interface UserData {
id: string;
name: string;
email: string;
}
function process(data: UserData) { }
```
### `unknown` as Lazy Typing — FORBIDDEN
`unknown` is only acceptable in these specific cases:
1. Error catch blocks (then immediately narrow)
2. JSON.parse results (then validate with Zod/schema)
3. External API responses before validation
```typescript
// ❌ NEVER - using unknown to avoid typing
function getData(): unknown { }
const config: Record<string, unknown> = {};
// ✅ ACCEPTABLE - error handling with immediate narrowing
try {
riskyOperation();
} catch (error: unknown) {
if (error instanceof Error) {
logger.error(error.message);
} else {
logger.error('Unknown error', { error: String(error) });
}
}
// ✅ ACCEPTABLE - external data with validation
const raw: unknown = JSON.parse(response);
const validated = UserSchema.parse(raw); // Zod validation
```
### Implicit `any` — FORBIDDEN
```typescript
// ❌ NEVER - implicit any from missing types
function process(data) { } // Parameter has implicit any
const handler = (e) => { } // Parameter has implicit any
// ✅ ALWAYS - explicit types
function process(data: RequestPayload): ProcessedResult { }
const handler = (e: React.MouseEvent<HTMLButtonElement>): void => { }
```
### Type Assertions to Bypass Safety — FORBIDDEN
```typescript
// ❌ NEVER - lying to the compiler
const user = data as User;
const element = document.getElementById('app') as HTMLDivElement;
// ✅ USE - type guards and narrowing
function isUser(data: unknown): data is User {
return typeof data === 'object' && data !== null && 'id' in data;
}
if (isUser(data)) {
console.log(data.id); // Safe
}
// ✅ USE - null checks
const element = document.getElementById('app');
if (element instanceof HTMLDivElement) {
element.style.display = 'none'; // Safe
}
```
### Non-null Assertion (`!`) — FORBIDDEN (except tests)
```typescript
// ❌ NEVER in production code
const name = user!.name;
const element = document.getElementById('app')!;
// ✅ USE - proper null handling
const name = user?.name ?? 'Anonymous';
const element = document.getElementById('app');
if (element) {
// Safe to use element
}
```
---
## Required Patterns
### Explicit Return Types — REQUIRED for all public functions
```typescript
// ❌ WRONG - missing return type
export function calculateTotal(items: Item[]) {
return items.reduce((sum, item) => sum + item.price, 0);
}
// ✅ CORRECT - explicit return type
export function calculateTotal(items: Item[]): number {
return items.reduce((sum, item) => sum + item.price, 0);
}
```
### Explicit Parameter Types — REQUIRED always
```typescript
// ❌ WRONG
const multiply = (a, b) => a * b;
users.map(user => user.name); // If user type isn't inferred
// ✅ CORRECT
const multiply = (a: number, b: number): number => a * b;
users.map((user: User): string => user.name);
```
### Interface Over Type Alias — PREFERRED for objects
```typescript
// ✅ PREFERRED - interface (extendable, better error messages)
interface User {
id: string;
name: string;
email: string;
}
// ✅ ACCEPTABLE - type alias for unions, intersections, primitives
type Status = 'active' | 'inactive' | 'pending';
type ID = string | number;
```
### Const Assertions for Literals — REQUIRED
```typescript
// ❌ WRONG - loses literal types
const config = {
endpoint: '/api/users',
method: 'GET',
};
// config.method is string, not 'GET'
// ✅ CORRECT - preserves literal types
const config = {
endpoint: '/api/users',
method: 'GET',
} as const;
// config.method is 'GET'
```
### Discriminated Unions — REQUIRED for variants
```typescript
// ❌ WRONG - optional properties for variants
interface ApiResponse {
success: boolean;
data?: User;
error?: string;
}
// ✅ CORRECT - discriminated union
interface SuccessResponse {
success: true;
data: User;
}
interface ErrorResponse {
success: false;
error: string;
}
type ApiResponse = SuccessResponse | ErrorResponse;
```
---
## Generic Constraints
### Meaningful Constraints — REQUIRED
```typescript
// ❌ WRONG - unconstrained generic
function merge<T>(a: T, b: T): T { }
// ✅ CORRECT - constrained generic
function merge<T extends object>(a: T, b: Partial<T>): T { }
```
### Default Generic Parameters — USE SPECIFIC TYPES
```typescript
// ❌ WRONG
interface Repository<T = unknown> { }
// ✅ CORRECT - no default if type should be explicit
interface Repository<T extends Entity> { }
// ✅ ACCEPTABLE - meaningful default
interface Cache<T extends Serializable = JsonValue> { }
```
---
## React/JSX Specific
### Event Handlers — EXPLICIT TYPES REQUIRED
```typescript
// ❌ WRONG
const handleClick = (e) => { };
const handleChange = (e) => { };
// ✅ CORRECT
const handleClick = (e: React.MouseEvent<HTMLButtonElement>): void => { };
const handleChange = (e: React.ChangeEvent<HTMLInputElement>): void => { };
const handleSubmit = (e: React.FormEvent<HTMLFormElement>): void => { };
```
### Component Props — INTERFACE REQUIRED
```typescript
// ❌ WRONG - inline types
function Button({ label, onClick }: { label: string; onClick: () => void }) { }
// ✅ CORRECT - named interface
interface ButtonProps {
label: string;
onClick: () => void;
disabled?: boolean;
}
function Button({ label, onClick, disabled = false }: ButtonProps): JSX.Element {
return <button onClick={onClick} disabled={disabled}>{label}</button>;
}
```
### Children Prop — USE React.ReactNode
```typescript
interface LayoutProps {
children: React.ReactNode;
sidebar?: React.ReactNode;
}
```
---
## API Response Typing
### Define Explicit Response Types
```typescript
// ❌ WRONG
const response = await fetch('/api/users');
const data = await response.json(); // data is any
// ✅ CORRECT
interface UsersResponse {
users: User[];
pagination: PaginationInfo;
}
const response = await fetch('/api/users');
const data: UsersResponse = await response.json();
// ✅ BEST - with runtime validation
const response = await fetch('/api/users');
const raw = await response.json();
const data = UsersResponseSchema.parse(raw); // Zod validates at runtime
```
---
## Error Handling
### Typed Error Classes — REQUIRED for domain errors
```typescript
class ValidationError extends Error {
constructor(
message: string,
public readonly field: string,
public readonly code: string
) {
super(message);
this.name = 'ValidationError';
}
}
class NotFoundError extends Error {
constructor(
public readonly resource: string,
public readonly id: string
) {
super(`${resource} with id ${id} not found`);
this.name = 'NotFoundError';
}
}
```
### Error Narrowing — REQUIRED
```typescript
try {
await saveUser(user);
} catch (error: unknown) {
if (error instanceof ValidationError) {
return { error: error.message, field: error.field };
}
if (error instanceof NotFoundError) {
return { error: 'Not found', resource: error.resource };
}
if (error instanceof Error) {
logger.error('Unexpected error', { message: error.message, stack: error.stack });
return { error: 'Internal error' };
}
logger.error('Unknown error type', { error: String(error) });
return { error: 'Internal error' };
}
```
---
## ESLint Rules — ENFORCE THESE
```javascript
{
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/explicit-function-return-type": ["error", {
"allowExpressions": true,
"allowTypedFunctionExpressions": true
}],
"@typescript-eslint/explicit-module-boundary-types": "error",
"@typescript-eslint/no-inferrable-types": "off", // Allow explicit primitives
"@typescript-eslint/no-non-null-assertion": "error",
"@typescript-eslint/strict-boolean-expressions": "error",
"@typescript-eslint/no-unsafe-assignment": "error",
"@typescript-eslint/no-unsafe-member-access": "error",
"@typescript-eslint/no-unsafe-call": "error",
"@typescript-eslint/no-unsafe-return": "error"
}
```
---
## TSConfig Strict Mode — REQUIRED
```json
{
"compilerOptions": {
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"strictFunctionTypes": true,
"strictBindCallApply": true,
"strictPropertyInitialization": true,
"noImplicitThis": true,
"useUnknownInCatchVariables": true,
"noUncheckedIndexedAccess": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"noImplicitOverride": true
}
}
```
---
## Summary: The Type Safety Hierarchy
From best to worst:
1. **Explicit specific type** (interface/type) — REQUIRED
2. **Generic with constraints** — ACCEPTABLE
3. **`unknown` with immediate validation** — ONLY for external data
4. **`any`** — FORBIDDEN
**When in doubt, define an interface.**

192
guides/vault-secrets.md Normal file
View File

@@ -0,0 +1,192 @@
# Vault Secrets Management Guide
This guide applies when the project uses HashiCorp Vault for secrets management.
## Before Starting
1. Verify Vault access: `vault status`
2. Authenticate: `vault login` (method depends on environment)
3. Check your permissions for the required paths
## Canonical Structure
**ALL Vault secrets MUST follow this structure:**
```
{mount}/{service}/{component}/{secret-name}
```
### Components
- **mount**: Environment-specific mount point
- **service**: The service or application name
- **component**: Logical grouping (database, api, oauth, etc.)
- **secret-name**: Specific secret identifier
## Environment Mounts
| Mount | Environment | Usage |
|-------|-------------|-------|
| `secret-dev/` | Development | Local dev, CI |
| `secret-staging/` | Staging | Pre-production testing |
| `secret-prod/` | Production | Live systems |
## Examples
```bash
# Database credentials
secret-prod/postgres/database/app
secret-prod/mysql/database/readonly
secret-staging/redis/auth/default
# API tokens
secret-prod/authentik/admin/token
secret-prod/stripe/api/live-key
secret-dev/sendgrid/api/test-key
# JWT/Authentication
secret-prod/backend-api/jwt/signing-key
secret-prod/auth-service/session/secret
# OAuth providers
secret-prod/backend-api/oauth/google
secret-prod/backend-api/oauth/github
# Internal services
secret-prod/loki/read-auth/admin
secret-prod/grafana/admin/password
```
## Standard Field Names
Use consistent field names within secrets:
| Purpose | Fields |
|---------|--------|
| Credentials | `username`, `password` |
| Tokens | `token` |
| OAuth | `client_id`, `client_secret` |
| Connection | `url`, `host`, `port` |
| Keys | `public_key`, `private_key` |
### Example Secret Structure
```json
// secret-prod/postgres/database/app
{
"username": "app_user",
"password": "secure-password-here",
"host": "db.example.com",
"port": "5432",
"database": "myapp"
}
```
## Rules
1. **DO NOT GUESS** secret paths - Always verify the path exists
2. **Use helper scripts** in `scripts/vault/` when available
3. **All lowercase, hyphenated** (kebab-case) for all path segments
4. **Standard field names** - Use the conventions above
5. **No sensitive data in path names** - Path itself should not reveal secrets
6. **Environment separation** - Never reference prod secrets from dev
## Deprecated Paths (DO NOT USE)
These legacy patterns are deprecated and should be migrated:
| Deprecated | Migrate To |
|------------|------------|
| `secret/infrastructure/*` | `secret-{env}/{service}/...` |
| `secret/oauth/*` | `secret-{env}/{service}/oauth/{provider}` |
| `secret/database/*` | `secret-{env}/{service}/database/{user}` |
| `secret/credentials/*` | `secret-{env}/{service}/{component}/{name}` |
## Reading Secrets
### CLI
```bash
# Read a secret
vault kv get secret-prod/postgres/database/app
# Get specific field
vault kv get -field=password secret-prod/postgres/database/app
# JSON output
vault kv get -format=json secret-prod/postgres/database/app
```
### Application Code
**Python (hvac):**
```python
import hvac
client = hvac.Client(url='https://vault.example.com')
secret = client.secrets.kv.v2.read_secret_version(
path='postgres/database/app',
mount_point='secret-prod'
)
password = secret['data']['data']['password']
```
**Node.js (node-vault):**
```javascript
const vault = require('node-vault')({ endpoint: 'https://vault.example.com' });
const secret = await vault.read('secret-prod/data/postgres/database/app');
const password = secret.data.data.password;
```
**Go:**
```go
secret, err := client.Logical().Read("secret-prod/data/postgres/database/app")
password := secret.Data["data"].(map[string]interface{})["password"].(string)
```
## Writing Secrets
Only authorized personnel should write secrets. If you need a new secret:
1. Request through proper channels (ticket, PR to IaC repo)
2. Follow the canonical structure
3. Document the secret's purpose
4. Set appropriate access policies
```bash
# Example (requires write permissions)
vault kv put secret-dev/myapp/database/app \
username="dev_user" \
password="dev-password" \
host="localhost" \
port="5432"
```
## Troubleshooting
### Permission Denied
```
Error: permission denied
```
- Verify your token has read access to the path
- Check if you're using the correct mount point
- Confirm the secret path exists
### Secret Not Found
```
Error: no value found at secret-prod/data/service/component/name
```
- Verify the exact path (use `vault kv list` to explore)
- Check for typos in service/component names
- Confirm you're using the correct environment mount
### Token Expired
```
Error: token expired
```
- Re-authenticate: `vault login`
- Check token TTL: `vault token lookup`
## Security Best Practices
1. **Least privilege** - Request only the permissions you need
2. **Short-lived tokens** - Use tokens with appropriate TTLs
3. **Audit logging** - All access is logged; act accordingly
4. **No local copies** - Don't store secrets in files or env vars long-term
5. **Rotate on compromise** - Immediately rotate any exposed secrets