refactor(ci): split monolithic pipeline into per-package pipelines
Replace single build.yml with split pipelines per the CI/CD guide: - api.yml: API with postgres, prisma, Trivy scan - web.yml: Web with Trivy scan - orchestrator.yml: Orchestrator with Trivy scan - coordinator.yml: Python with ruff/mypy/bandit/pip-audit/Trivy - infra.yml: postgres + openbao builds with Trivy Adds path filtering (only affected packages rebuild), Trivy container scanning for all images, and scoped per-package quality gates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,120 +1,142 @@
|
||||
# Woodpecker CI Configuration for Mosaic Stack
|
||||
|
||||
## Pipeline Architecture
|
||||
|
||||
Split per-package pipelines with path filtering. Only affected packages rebuild on push.
|
||||
|
||||
```
|
||||
.woodpecker/
|
||||
├── api.yml # @mosaic/api (NestJS)
|
||||
├── web.yml # @mosaic/web (Next.js)
|
||||
├── orchestrator.yml # @mosaic/orchestrator (NestJS)
|
||||
├── coordinator.yml # mosaic-coordinator (Python/FastAPI)
|
||||
├── infra.yml # postgres + openbao Docker images
|
||||
├── codex-review.yml # AI code/security review (PRs only)
|
||||
├── README.md
|
||||
└── schemas/
|
||||
├── code-review-schema.json
|
||||
└── security-review-schema.json
|
||||
```
|
||||
|
||||
## Path Filtering
|
||||
|
||||
| Pipeline | Triggers On |
|
||||
| ------------------ | --------------------------------------------------- |
|
||||
| `api.yml` | `apps/api/**`, `packages/**`, root configs |
|
||||
| `web.yml` | `apps/web/**`, `packages/**`, root configs |
|
||||
| `orchestrator.yml` | `apps/orchestrator/**`, `packages/**`, root configs |
|
||||
| `coordinator.yml` | `apps/coordinator/**` |
|
||||
| `infra.yml` | `docker/**` |
|
||||
| `codex-review.yml` | All PRs (no path filter) |
|
||||
|
||||
**Root configs** = `pnpm-lock.yaml`, `pnpm-workspace.yaml`, `turbo.json`, `package.json`
|
||||
|
||||
## Security Chain
|
||||
|
||||
Every pipeline follows the full security chain required by the CI/CD guide:
|
||||
|
||||
```
|
||||
source scanning (lint + pnpm audit / bandit + pip-audit)
|
||||
-> docker build (Kaniko)
|
||||
-> container scanning (Trivy: HIGH,CRITICAL)
|
||||
-> package linking (Gitea registry)
|
||||
```
|
||||
|
||||
Docker builds gate on ALL quality + security steps passing.
|
||||
|
||||
## Pipeline Dependency Graphs
|
||||
|
||||
### Node.js Apps (api, web, orchestrator)
|
||||
|
||||
```
|
||||
install -> [security-audit, lint, prisma-generate*]
|
||||
prisma-generate* -> [typecheck, prisma-migrate*]
|
||||
prisma-migrate* -> test
|
||||
[all quality gates] -> build -> docker-build -> trivy -> link
|
||||
```
|
||||
|
||||
_\*prisma steps: api.yml only_
|
||||
|
||||
### Coordinator (Python)
|
||||
|
||||
```
|
||||
install -> [ruff-check, mypy, security-bandit, security-pip-audit, test]
|
||||
[all quality gates] -> docker-build -> trivy -> link
|
||||
```
|
||||
|
||||
### Infrastructure
|
||||
|
||||
```
|
||||
[docker-build-postgres, docker-build-openbao]
|
||||
-> [trivy-postgres, trivy-openbao]
|
||||
-> link
|
||||
```
|
||||
|
||||
## Docker Images
|
||||
|
||||
| Image | Registry Path | Context |
|
||||
| ------------------ | ----------------------------------------------- | ------------------- |
|
||||
| stack-api | `git.mosaicstack.dev/mosaic/stack-api` | `.` (monorepo root) |
|
||||
| stack-web | `git.mosaicstack.dev/mosaic/stack-web` | `.` (monorepo root) |
|
||||
| stack-orchestrator | `git.mosaicstack.dev/mosaic/stack-orchestrator` | `.` (monorepo root) |
|
||||
| stack-coordinator | `git.mosaicstack.dev/mosaic/stack-coordinator` | `apps/coordinator` |
|
||||
| stack-postgres | `git.mosaicstack.dev/mosaic/stack-postgres` | `docker/postgres` |
|
||||
| stack-openbao | `git.mosaicstack.dev/mosaic/stack-openbao` | `docker/openbao` |
|
||||
|
||||
## Image Tagging
|
||||
|
||||
| Condition | Tag | Purpose |
|
||||
| ---------------- | -------------------------- | -------------------------- |
|
||||
| Always | `${CI_COMMIT_SHA:0:8}` | Immutable commit reference |
|
||||
| `main` branch | `latest` | Current production release |
|
||||
| `develop` branch | `dev` | Current development build |
|
||||
| Git tag | tag value (e.g., `v1.0.0`) | Semantic version release |
|
||||
|
||||
## Required Secrets
|
||||
|
||||
Configure in Woodpecker UI (Settings > Secrets):
|
||||
|
||||
| Secret | Scope | Purpose |
|
||||
| ---------------- | ----------------- | ------------------------------------------- |
|
||||
| `gitea_username` | push, manual, tag | Gitea registry auth |
|
||||
| `gitea_token` | push, manual, tag | Gitea registry auth (`package:write` scope) |
|
||||
| `codex_api_key` | pull_request | Codex AI reviews |
|
||||
|
||||
## Codex AI Review Pipeline
|
||||
|
||||
This directory contains the Codex AI review pipeline configuration for automated code and security reviews on pull requests.
|
||||
The `codex-review.yml` pipeline runs independently on all PRs:
|
||||
|
||||
### Setup
|
||||
- **Code review**: Correctness, code quality, testing, performance
|
||||
- **Security review**: OWASP Top 10, hardcoded secrets, injection flaws
|
||||
|
||||
1. **Add Codex API key to Woodpecker:**
|
||||
- Go to mosaic-stack repo at `https://ci.mosaicstack.dev`
|
||||
- Settings → Secrets
|
||||
- Add secret: `codex_api_key` with your OpenAI API key
|
||||
|
||||
2. **Enable the pipeline:**
|
||||
- The `codex-review.yml` pipeline will automatically run on all PRs
|
||||
- The main `.woodpecker.yml` handles primary CI tasks
|
||||
- This codex pipeline is independent and focused solely on reviews
|
||||
|
||||
### What Gets Reviewed
|
||||
|
||||
**Code Review (`code-review` step):**
|
||||
|
||||
- Correctness — logic errors, edge cases, error handling
|
||||
- Code Quality — complexity, duplication, naming
|
||||
- Testing — coverage, test quality
|
||||
- Performance — N+1 queries, blocking ops
|
||||
- Dependencies — deprecated packages
|
||||
- Documentation — comments, API docs
|
||||
|
||||
**Security Review (`security-review` step):**
|
||||
|
||||
- OWASP Top 10 vulnerabilities
|
||||
- Hardcoded secrets/credentials
|
||||
- Injection flaws (SQL, NoSQL, OS command)
|
||||
- XSS, CSRF, SSRF
|
||||
- Auth/authz gaps
|
||||
- Data exposure in logs
|
||||
|
||||
### Pipeline Behavior
|
||||
|
||||
- **Triggers:** Every pull request
|
||||
- **Runs:** Code review + Security review in parallel
|
||||
- **Fails if:**
|
||||
- Code review finds **blockers**
|
||||
- Security review finds **critical** or **high** severity issues
|
||||
- **Outputs:** Structured JSON results in CI logs
|
||||
Fails on blockers or critical/high severity security findings.
|
||||
|
||||
### Local Testing
|
||||
|
||||
Test the review scripts locally before pushing:
|
||||
|
||||
```bash
|
||||
# Code review of uncommitted changes
|
||||
~/.claude/scripts/codex/codex-code-review.sh --uncommitted
|
||||
|
||||
# Security review of uncommitted changes
|
||||
~/.claude/scripts/codex/codex-security-review.sh --uncommitted
|
||||
|
||||
# Code review against main branch
|
||||
~/.claude/scripts/codex/codex-code-review.sh -b main
|
||||
|
||||
# Security review and save JSON
|
||||
~/.claude/scripts/codex/codex-security-review.sh -b main -o security.json
|
||||
```
|
||||
|
||||
### Schema Files
|
||||
## Troubleshooting
|
||||
|
||||
The `schemas/` directory contains JSON schemas that enforce structured output from Codex:
|
||||
### "unauthorized: authentication required"
|
||||
|
||||
- `code-review-schema.json` — Defines output for code quality reviews
|
||||
- `security-review-schema.json` — Defines output for security reviews
|
||||
- Verify `gitea_username` and `gitea_token` secrets in Woodpecker
|
||||
- Verify token has `package:write` scope
|
||||
|
||||
These schemas ensure consistent, machine-readable findings that the CI pipeline can parse and fail on.
|
||||
### Trivy scan fails with HIGH/CRITICAL
|
||||
|
||||
### Integration with Main Pipeline
|
||||
- Check if the vulnerability is in the base image (not our code)
|
||||
- Add to `.trivyignore` if it's a known, accepted risk
|
||||
- Use `--ignore-unfixed` (already set) to skip unfixable CVEs
|
||||
|
||||
The main `.woodpecker.yml` in the repo root handles:
|
||||
### Package linking returns 404
|
||||
|
||||
- Type checking (TypeScript)
|
||||
- Linting (ESLint)
|
||||
- Unit tests (Vitest)
|
||||
- Integration tests (Playwright)
|
||||
- Docker image builds
|
||||
- Normal for recently pushed packages — retry logic handles this
|
||||
- If persistent: verify package name matches exactly (case-sensitive)
|
||||
|
||||
This `codex-review.yml` is independent and focuses solely on:
|
||||
### Pipeline runs Docker builds on pull requests
|
||||
|
||||
- AI-powered code quality review
|
||||
- AI-powered security vulnerability scanning
|
||||
|
||||
Both pipelines run in parallel on PRs.
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
**Pipeline fails with "codex: command not found"**
|
||||
|
||||
- Check that the node image in `codex-review.yml` matches a version with npm
|
||||
- Current: `node:22-slim`
|
||||
|
||||
**Pipeline fails with auth errors**
|
||||
|
||||
- Verify `codex_api_key` secret is set in Woodpecker
|
||||
- Test the key locally: `CODEX_API_KEY=<key> codex exec "test"`
|
||||
|
||||
**Pipeline passes but should fail**
|
||||
|
||||
- Check the failure conditions in `codex-review.yml`
|
||||
- Current thresholds: blockers, critical, or high findings
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
| ------------------------------------- | -------------------------------------- |
|
||||
| `codex-review.yml` | Codex AI review pipeline configuration |
|
||||
| `schemas/code-review-schema.json` | Code review output schema |
|
||||
| `schemas/security-review-schema.json` | Security review output schema |
|
||||
| `README.md` | This file |
|
||||
|
||||
## Parent CI Pipeline
|
||||
|
||||
The main `.woodpecker.yml` is located at the repository root and handles all build/test tasks.
|
||||
- Docker build steps have `when: branch: [main, develop]` guards
|
||||
- PRs only run quality gates, not Docker builds
|
||||
|
||||
216
.woodpecker/api.yml
Normal file
216
.woodpecker/api.yml
Normal file
@@ -0,0 +1,216 @@
|
||||
# API Pipeline - Mosaic Stack
|
||||
# Quality gates, build, and Docker publish for @mosaic/api
|
||||
#
|
||||
# Triggers on: apps/api/**, packages/**, root configs
|
||||
# Security chain: source audit + Trivy container scan
|
||||
|
||||
when:
|
||||
- event: [push, pull_request, manual]
|
||||
path:
|
||||
include:
|
||||
- "apps/api/**"
|
||||
- "packages/**"
|
||||
- "pnpm-lock.yaml"
|
||||
- "pnpm-workspace.yaml"
|
||||
- "turbo.json"
|
||||
- "package.json"
|
||||
- ".woodpecker/api.yml"
|
||||
|
||||
variables:
|
||||
- &node_image "node:20-alpine"
|
||||
- &install_deps |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
- &use_deps |
|
||||
corepack enable
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
environment:
|
||||
POSTGRES_DB: test_db
|
||||
POSTGRES_USER: test_user
|
||||
POSTGRES_PASSWORD: test_password
|
||||
|
||||
steps:
|
||||
# === Quality Gates ===
|
||||
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *install_deps
|
||||
|
||||
security-audit:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm audit --audit-level=high
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
lint:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" lint
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
prisma-generate:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" prisma:generate
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" typecheck
|
||||
depends_on:
|
||||
- prisma-generate
|
||||
|
||||
prisma-migrate:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
DATABASE_URL: "postgresql://test_user:test_password@postgres:5432/test_db?schema=public"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" prisma migrate deploy
|
||||
depends_on:
|
||||
- prisma-generate
|
||||
|
||||
test:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
DATABASE_URL: "postgresql://test_user:test_password@postgres:5432/test_db?schema=public"
|
||||
ENCRYPTION_KEY: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" test
|
||||
depends_on:
|
||||
- prisma-migrate
|
||||
|
||||
# === Build ===
|
||||
|
||||
build:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
NODE_ENV: "production"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm turbo build --filter=@mosaic/api
|
||||
depends_on:
|
||||
- lint
|
||||
- typecheck
|
||||
- test
|
||||
- security-audit
|
||||
|
||||
# === Docker Build & Push ===
|
||||
|
||||
docker-build-api:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-api:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/api/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# === Container Security Scan ===
|
||||
|
||||
security-trivy-api:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-api:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-api
|
||||
|
||||
# === Package Linking ===
|
||||
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " Linked $$PKG"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " $$PKG already linked"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " FAILED: $$PKG status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
link_package "stack-api"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- security-trivy-api
|
||||
@@ -1,357 +0,0 @@
|
||||
# Woodpecker CI Quality Enforcement Pipeline - Monorepo
|
||||
when:
|
||||
- event: [push, pull_request, manual]
|
||||
|
||||
variables:
|
||||
- &node_image "node:20-alpine"
|
||||
- &install_deps |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
- &use_deps |
|
||||
corepack enable
|
||||
# Kaniko base command setup
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
environment:
|
||||
POSTGRES_DB: test_db
|
||||
POSTGRES_USER: test_user
|
||||
POSTGRES_PASSWORD: test_password
|
||||
|
||||
steps:
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *install_deps
|
||||
|
||||
security-audit:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm audit --audit-level=high
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
lint:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm lint
|
||||
depends_on:
|
||||
- install
|
||||
when:
|
||||
- evaluate: 'CI_PIPELINE_EVENT != "pull_request" || CI_COMMIT_BRANCH != "main"'
|
||||
|
||||
prisma-generate:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" prisma:generate
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
prisma-migrate:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
DATABASE_URL: "postgresql://test_user:test_password@postgres:5432/test_db?schema=public"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/api" prisma migrate deploy
|
||||
depends_on:
|
||||
- prisma-generate
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm typecheck
|
||||
depends_on:
|
||||
- prisma-generate
|
||||
|
||||
test:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
DATABASE_URL: "postgresql://test_user:test_password@postgres:5432/test_db?schema=public"
|
||||
ENCRYPTION_KEY: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm test
|
||||
depends_on:
|
||||
- prisma-migrate
|
||||
|
||||
build:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
NODE_ENV: "production"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm build
|
||||
depends_on:
|
||||
- lint
|
||||
- typecheck
|
||||
- test
|
||||
- security-audit
|
||||
|
||||
# ======================
|
||||
# Docker Build & Push (main/develop only)
|
||||
# ======================
|
||||
# Requires secrets: gitea_username, gitea_token
|
||||
#
|
||||
# Tagging Strategy:
|
||||
# - Always: commit SHA (e.g., 658ec077)
|
||||
# - main branch: 'latest'
|
||||
# - develop branch: 'dev'
|
||||
# - git tags: version tag (e.g., v1.0.0)
|
||||
|
||||
# Build and push API image using Kaniko
|
||||
docker-build-api:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-api:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-api:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/api/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# Build and push Web image using Kaniko
|
||||
docker-build-web:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-web:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/web/Dockerfile --build-arg NEXT_PUBLIC_API_URL=https://api.mosaicstack.dev $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# Build and push Postgres image using Kaniko
|
||||
docker-build-postgres:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-postgres:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context docker/postgres --dockerfile docker/postgres/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# Build and push OpenBao image using Kaniko
|
||||
docker-build-openbao:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-openbao:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context docker/openbao --dockerfile docker/openbao/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# Build and push Orchestrator image using Kaniko
|
||||
docker-build-orchestrator:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-orchestrator:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/orchestrator/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# Build and push Coordinator image using Kaniko
|
||||
docker-build-coordinator:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-coordinator:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context apps/coordinator --dockerfile apps/coordinator/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# ======================
|
||||
# Link Packages to Repository
|
||||
# ======================
|
||||
# Links all Docker packages to the mosaic/stack repository
|
||||
# This makes packages visible on the repository page in Gitea
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- echo "Waiting 10 seconds for packages to be indexed in registry..."
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
|
||||
# Retry up to 3 times with 5 second delays
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " ✅ Linked $$PKG to stack"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " ✅ $$PKG already linked (OK)"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " ⏳ $$PKG not found yet, waiting 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " ❌ $$PKG link failed with status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
link_package "stack-api"
|
||||
link_package "stack-web"
|
||||
link_package "stack-postgres"
|
||||
link_package "stack-openbao"
|
||||
link_package "stack-orchestrator"
|
||||
link_package "stack-coordinator"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-api
|
||||
- docker-build-web
|
||||
- docker-build-postgres
|
||||
- docker-build-openbao
|
||||
- docker-build-orchestrator
|
||||
- docker-build-coordinator
|
||||
172
.woodpecker/coordinator.yml
Normal file
172
.woodpecker/coordinator.yml
Normal file
@@ -0,0 +1,172 @@
|
||||
# Coordinator Pipeline - Mosaic Stack
|
||||
# Quality gates, build, and Docker publish for mosaic-coordinator (Python)
|
||||
#
|
||||
# Triggers on: apps/coordinator/**
|
||||
# Security chain: bandit + pip-audit + Trivy container scan
|
||||
|
||||
when:
|
||||
- event: [push, pull_request, manual]
|
||||
path:
|
||||
include:
|
||||
- "apps/coordinator/**"
|
||||
- ".woodpecker/coordinator.yml"
|
||||
|
||||
variables:
|
||||
- &python_image "python:3.11-slim"
|
||||
- &activate_venv |
|
||||
cd apps/coordinator
|
||||
. venv/bin/activate
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
steps:
|
||||
# === Quality Gates ===
|
||||
|
||||
install:
|
||||
image: *python_image
|
||||
commands:
|
||||
- cd apps/coordinator
|
||||
- python -m venv venv
|
||||
- . venv/bin/activate
|
||||
- pip install --no-cache-dir -e ".[dev]"
|
||||
- pip install --no-cache-dir bandit pip-audit
|
||||
|
||||
ruff-check:
|
||||
image: *python_image
|
||||
commands:
|
||||
- *activate_venv
|
||||
- ruff check src/ tests/
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
mypy:
|
||||
image: *python_image
|
||||
commands:
|
||||
- *activate_venv
|
||||
- mypy src/
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
security-bandit:
|
||||
image: *python_image
|
||||
commands:
|
||||
- *activate_venv
|
||||
- bandit -r src/ -f screen
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
security-pip-audit:
|
||||
image: *python_image
|
||||
commands:
|
||||
- *activate_venv
|
||||
- pip-audit
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
test:
|
||||
image: *python_image
|
||||
commands:
|
||||
- *activate_venv
|
||||
- pytest
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
# === Docker Build & Push ===
|
||||
|
||||
docker-build-coordinator:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-coordinator:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-coordinator:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context apps/coordinator --dockerfile apps/coordinator/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- ruff-check
|
||||
- mypy
|
||||
- security-bandit
|
||||
- security-pip-audit
|
||||
- test
|
||||
|
||||
# === Container Security Scan ===
|
||||
|
||||
security-trivy-coordinator:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-coordinator:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-coordinator
|
||||
|
||||
# === Package Linking ===
|
||||
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " Linked $$PKG"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " $$PKG already linked"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " FAILED: $$PKG status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
link_package "stack-coordinator"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- security-trivy-coordinator
|
||||
160
.woodpecker/infra.yml
Normal file
160
.woodpecker/infra.yml
Normal file
@@ -0,0 +1,160 @@
|
||||
# Infrastructure Pipeline - Mosaic Stack
|
||||
# Docker build, Trivy scan, and publish for postgres + openbao images
|
||||
#
|
||||
# Triggers on: docker/**
|
||||
# No quality gates — infrastructure images (base image + config only)
|
||||
|
||||
when:
|
||||
- event: [push, manual, tag]
|
||||
path:
|
||||
include:
|
||||
- "docker/**"
|
||||
- ".woodpecker/infra.yml"
|
||||
|
||||
variables:
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
steps:
|
||||
# === Docker Build & Push ===
|
||||
|
||||
docker-build-postgres:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-postgres:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-postgres:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context docker/postgres --dockerfile docker/postgres/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
|
||||
docker-build-openbao:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-openbao:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-openbao:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context docker/openbao --dockerfile docker/openbao/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
|
||||
# === Container Security Scans ===
|
||||
|
||||
security-trivy-postgres:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-postgres:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-postgres
|
||||
|
||||
security-trivy-openbao:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-openbao:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-openbao
|
||||
|
||||
# === Package Linking ===
|
||||
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " Linked $$PKG"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " $$PKG already linked"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " FAILED: $$PKG status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
link_package "stack-postgres"
|
||||
link_package "stack-openbao"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- security-trivy-postgres
|
||||
- security-trivy-openbao
|
||||
185
.woodpecker/orchestrator.yml
Normal file
185
.woodpecker/orchestrator.yml
Normal file
@@ -0,0 +1,185 @@
|
||||
# Orchestrator Pipeline - Mosaic Stack
|
||||
# Quality gates, build, and Docker publish for @mosaic/orchestrator
|
||||
#
|
||||
# Triggers on: apps/orchestrator/**, packages/**, root configs
|
||||
# Security chain: source audit + Trivy container scan
|
||||
|
||||
when:
|
||||
- event: [push, pull_request, manual]
|
||||
path:
|
||||
include:
|
||||
- "apps/orchestrator/**"
|
||||
- "packages/**"
|
||||
- "pnpm-lock.yaml"
|
||||
- "pnpm-workspace.yaml"
|
||||
- "turbo.json"
|
||||
- "package.json"
|
||||
- ".woodpecker/orchestrator.yml"
|
||||
|
||||
variables:
|
||||
- &node_image "node:20-alpine"
|
||||
- &install_deps |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
- &use_deps |
|
||||
corepack enable
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
steps:
|
||||
# === Quality Gates ===
|
||||
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *install_deps
|
||||
|
||||
security-audit:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm audit --audit-level=high
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
lint:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/orchestrator" lint
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/orchestrator" typecheck
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
test:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/orchestrator" test
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
# === Build ===
|
||||
|
||||
build:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
NODE_ENV: "production"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm turbo build --filter=@mosaic/orchestrator
|
||||
depends_on:
|
||||
- lint
|
||||
- typecheck
|
||||
- test
|
||||
- security-audit
|
||||
|
||||
# === Docker Build & Push ===
|
||||
|
||||
docker-build-orchestrator:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-orchestrator:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-orchestrator:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/orchestrator/Dockerfile $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# === Container Security Scan ===
|
||||
|
||||
security-trivy-orchestrator:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-orchestrator:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-orchestrator
|
||||
|
||||
# === Package Linking ===
|
||||
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " Linked $$PKG"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " $$PKG already linked"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " FAILED: $$PKG status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
link_package "stack-orchestrator"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- security-trivy-orchestrator
|
||||
185
.woodpecker/web.yml
Normal file
185
.woodpecker/web.yml
Normal file
@@ -0,0 +1,185 @@
|
||||
# Web Pipeline - Mosaic Stack
|
||||
# Quality gates, build, and Docker publish for @mosaic/web
|
||||
#
|
||||
# Triggers on: apps/web/**, packages/**, root configs
|
||||
# Security chain: source audit + Trivy container scan
|
||||
|
||||
when:
|
||||
- event: [push, pull_request, manual]
|
||||
path:
|
||||
include:
|
||||
- "apps/web/**"
|
||||
- "packages/**"
|
||||
- "pnpm-lock.yaml"
|
||||
- "pnpm-workspace.yaml"
|
||||
- "turbo.json"
|
||||
- "package.json"
|
||||
- ".woodpecker/web.yml"
|
||||
|
||||
variables:
|
||||
- &node_image "node:20-alpine"
|
||||
- &install_deps |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
- &use_deps |
|
||||
corepack enable
|
||||
- &kaniko_setup |
|
||||
mkdir -p /kaniko/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
steps:
|
||||
# === Quality Gates ===
|
||||
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *install_deps
|
||||
|
||||
security-audit:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm audit --audit-level=high
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
lint:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/web" lint
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/web" typecheck
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
test:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm --filter "@mosaic/web" test
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
# === Build ===
|
||||
|
||||
build:
|
||||
image: *node_image
|
||||
environment:
|
||||
SKIP_ENV_VALIDATION: "true"
|
||||
NODE_ENV: "production"
|
||||
commands:
|
||||
- *use_deps
|
||||
- pnpm turbo build --filter=@mosaic/web
|
||||
depends_on:
|
||||
- lint
|
||||
- typecheck
|
||||
- test
|
||||
- security-audit
|
||||
|
||||
# === Docker Build & Push ===
|
||||
|
||||
docker-build-web:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- *kaniko_setup
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaic/stack-web:${CI_COMMIT_SHA:0:8}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:latest"
|
||||
elif [ "$CI_COMMIT_BRANCH" = "develop" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:dev"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaic/stack-web:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile apps/web/Dockerfile --build-arg NEXT_PUBLIC_API_URL=https://api.mosaicstack.dev $DESTINATIONS
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# === Container Security Scan ===
|
||||
|
||||
security-trivy-web:
|
||||
image: aquasec/trivy:latest
|
||||
environment:
|
||||
GITEA_USER:
|
||||
from_secret: gitea_username
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- |
|
||||
mkdir -p ~/.docker
|
||||
echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$$GITEA_USER\",\"password\":\"$$GITEA_TOKEN\"}}}" > ~/.docker/config.json
|
||||
trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed \
|
||||
git.mosaicstack.dev/mosaic/stack-web:$${CI_COMMIT_SHA:0:8}
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- docker-build-web
|
||||
|
||||
# === Package Linking ===
|
||||
|
||||
link-packages:
|
||||
image: alpine:3
|
||||
environment:
|
||||
GITEA_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sleep 10
|
||||
- |
|
||||
set -e
|
||||
link_package() {
|
||||
PKG="$$1"
|
||||
echo "Linking $$PKG..."
|
||||
for attempt in 1 2 3; do
|
||||
STATUS=$$(curl -s -o /tmp/link-response.txt -w "%{http_code}" -X POST \
|
||||
-H "Authorization: token $$GITEA_TOKEN" \
|
||||
"https://git.mosaicstack.dev/api/v1/packages/mosaic/container/$$PKG/-/link/stack")
|
||||
if [ "$$STATUS" = "201" ] || [ "$$STATUS" = "204" ]; then
|
||||
echo " Linked $$PKG"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "400" ]; then
|
||||
echo " $$PKG already linked"
|
||||
return 0
|
||||
elif [ "$$STATUS" = "404" ] && [ $$attempt -lt 3 ]; then
|
||||
echo " $$PKG not found yet, retrying in 5s (attempt $$attempt/3)..."
|
||||
sleep 5
|
||||
else
|
||||
echo " FAILED: $$PKG status $$STATUS"
|
||||
cat /tmp/link-response.txt
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
}
|
||||
link_package "stack-web"
|
||||
when:
|
||||
- branch: [main, develop]
|
||||
event: [push, manual, tag]
|
||||
depends_on:
|
||||
- security-trivy-web
|
||||
Reference in New Issue
Block a user