Co-authored-by: Jason Woltje <jason@diversecanvas.com> Co-committed-by: Jason Woltje <jason@diversecanvas.com>
305 lines
8.8 KiB
Markdown
305 lines
8.8 KiB
Markdown
---
|
|
name: setup-cicd
|
|
description: "Configure CI/CD Docker build, push, and package linking for a project. Use when adding Docker builds to a Woodpecker pipeline, setting up Gitea container registry, or implementing CI/CD for deployment. Triggers on: setup cicd, add docker builds, configure pipeline, add ci/cd, setup ci."
|
|
---
|
|
|
|
# CI/CD Pipeline Setup
|
|
|
|
Configure Docker build, registry push, and package linking for a Woodpecker CI pipeline using Kaniko and Gitea's container registry.
|
|
|
|
**Before starting:** Read `~/.config/mosaic/guides/CI-CD-PIPELINES.md` for deep background on the patterns used here.
|
|
|
|
**Reference implementation:** `~/src/mosaic-stack/.woodpecker.yml`
|
|
|
|
---
|
|
|
|
## The Job
|
|
|
|
1. Scan the current project for services, Dockerfiles, and registry info
|
|
2. Ask clarifying questions about what to build and how to name images
|
|
3. Generate Woodpecker YAML for Docker build/push/link steps
|
|
4. Provide secrets configuration commands
|
|
5. Output a verification checklist
|
|
|
|
**Important:** This skill generates YAML to *append* to an existing `.woodpecker.yml`, not replace it. The project should already have quality gate steps (lint, test, typecheck, build).
|
|
|
|
---
|
|
|
|
## Step 1: Project Scan
|
|
|
|
Run these scans and present results to the user:
|
|
|
|
### 1a. Detect registry info from git remote
|
|
|
|
```bash
|
|
# Extract Gitea host and org/repo from remote
|
|
REMOTE_URL=$(git remote get-url origin 2>/dev/null)
|
|
# Parse: https://git.example.com/org/repo.git -> host=git.example.com, org=org, repo=repo
|
|
```
|
|
|
|
Present:
|
|
- **Registry host:** (extracted from remote)
|
|
- **Organization:** (extracted from remote)
|
|
- **Repository:** (extracted from remote)
|
|
|
|
### 1b. Find all Dockerfiles
|
|
|
|
```bash
|
|
find . -name "Dockerfile" -o -name "Dockerfile.*" | grep -v node_modules | grep -v .git | sort
|
|
```
|
|
|
|
For each Dockerfile found, note:
|
|
- Path relative to project root
|
|
- Whether it's a dev variant (`Dockerfile.dev`) or production
|
|
- The service name (inferred from parent directory)
|
|
|
|
### 1c. Detect existing pipeline
|
|
|
|
```bash
|
|
cat .woodpecker.yml 2>/dev/null || cat .woodpecker/*.yml 2>/dev/null
|
|
```
|
|
|
|
Check:
|
|
- Does a `build` step exist? (Docker builds will depend on it)
|
|
- Are there already Docker build steps? (avoid duplicating)
|
|
- What's the existing dependency chain?
|
|
|
|
### 1d. Find publishable npm packages (if applicable)
|
|
|
|
```bash
|
|
# Find package.json files without "private": true
|
|
find . -name "package.json" -not -path "*/node_modules/*" -exec grep -L '"private": true' {} \;
|
|
```
|
|
|
|
### 1e. Present scan results
|
|
|
|
Show the user a summary table:
|
|
|
|
```
|
|
=== CI/CD Scan Results ===
|
|
Registry: git.example.com
|
|
Organization: org-name
|
|
Repository: repo-name
|
|
|
|
Dockerfiles Found:
|
|
1. src/backend-api/Dockerfile → backend-api
|
|
2. src/web-portal/Dockerfile → web-portal
|
|
3. src/ingest-api/Dockerfile → ingest-api
|
|
4. src/backend-api/Dockerfile.dev → (dev variant, skip)
|
|
|
|
Existing Pipeline: .woodpecker.yml
|
|
- Has build step: yes (build-all)
|
|
- Has Docker steps: no
|
|
|
|
Publishable npm Packages:
|
|
- @scope/schemas (src/schemas)
|
|
- @scope/design-system (src/design-system)
|
|
```
|
|
|
|
---
|
|
|
|
## Step 2: Clarifying Questions
|
|
|
|
Ask these questions with lettered options (user can respond "1A, 2B, 3C"):
|
|
|
|
```
|
|
1. Which Dockerfiles should be built in CI?
|
|
(Select all that apply — list found Dockerfiles with letters)
|
|
A. src/backend-api/Dockerfile (backend-api)
|
|
B. src/web-portal/Dockerfile (web-portal)
|
|
C. src/ingest-api/Dockerfile (ingest-api)
|
|
D. All of the above
|
|
E. Other: [specify]
|
|
|
|
2. Image naming convention?
|
|
A. {org}/{service} (e.g., usc/uconnect-backend-api) — Recommended
|
|
B. {org}/{repo}-{service} (e.g., usc/uconnect-backend-api)
|
|
C. Custom: [specify]
|
|
|
|
3. Do any services need build arguments?
|
|
A. No build args needed
|
|
B. Yes: [specify service:KEY=VALUE, e.g., web-portal:NEXT_PUBLIC_API_URL=https://api.example.com]
|
|
|
|
4. Which branches should trigger Docker builds?
|
|
A. main and develop (Recommended)
|
|
B. main only
|
|
C. Custom: [specify]
|
|
|
|
5. Should npm packages be published? (only if publishable packages found)
|
|
A. Yes, to Gitea npm registry
|
|
B. Yes, to custom registry: [specify URL]
|
|
C. No, skip npm publishing
|
|
```
|
|
|
|
---
|
|
|
|
## Step 3: Generate Pipeline YAML
|
|
|
|
### 3a. Add kaniko_setup anchor
|
|
|
|
If the project's `.woodpecker.yml` doesn't already have a `kaniko_setup` anchor in its `variables:` section, add it:
|
|
|
|
```bash
|
|
~/.config/mosaic/tools/cicd/generate-docker-steps.sh --kaniko-setup-only --registry REGISTRY_HOST
|
|
```
|
|
|
|
This outputs:
|
|
```yaml
|
|
# Kaniko base command setup
|
|
- &kaniko_setup |
|
|
mkdir -p /kaniko/.docker
|
|
echo "{\"auths\":{\"REGISTRY\":{\"username\":\"$GITEA_USER\",\"password\":\"$GITEA_TOKEN\"}}}" > /kaniko/.docker/config.json
|
|
```
|
|
|
|
Add this to the existing `variables:` block at the top of `.woodpecker.yml`.
|
|
|
|
### 3b. Generate Docker build/push/link steps
|
|
|
|
Use the generator script with the user's answers:
|
|
|
|
```bash
|
|
~/.config/mosaic/tools/cicd/generate-docker-steps.sh \
|
|
--registry REGISTRY \
|
|
--org ORG \
|
|
--repo REPO \
|
|
--service "SERVICE_NAME:DOCKERFILE_PATH" \
|
|
--service "SERVICE_NAME:DOCKERFILE_PATH" \
|
|
--branches "main,develop" \
|
|
--depends-on "BUILD_STEP_NAME" \
|
|
[--build-arg "SERVICE:KEY=VALUE"] \
|
|
[--npm-package "@scope/pkg:path" --npm-registry "URL"]
|
|
```
|
|
|
|
### 3c. Present generated YAML
|
|
|
|
Show the full YAML output to the user and ask for confirmation before appending to `.woodpecker.yml`.
|
|
|
|
### 3d. Append to pipeline
|
|
|
|
Append the generated YAML to the end of `.woodpecker.yml`. The kaniko_setup anchor goes in the `variables:` section.
|
|
|
|
---
|
|
|
|
## Step 4: Secrets Checklist
|
|
|
|
Present the required Woodpecker secrets and commands to configure them:
|
|
|
|
```
|
|
=== Required Woodpecker Secrets ===
|
|
|
|
Configure these at: https://WOODPECKER_HOST/repos/ORG/REPO/settings/secrets
|
|
|
|
1. gitea_username
|
|
Value: Your Gitea username or service account
|
|
Events: push, manual, tag
|
|
|
|
2. gitea_token
|
|
Value: Gitea token with package:write scope
|
|
Generate at: https://REGISTRY_HOST/user/settings/applications
|
|
Events: push, manual, tag
|
|
|
|
CLI commands:
|
|
woodpecker secret add ORG/REPO --name gitea_username --value "USERNAME"
|
|
woodpecker secret add ORG/REPO --name gitea_token --value "TOKEN"
|
|
```
|
|
|
|
If npm publishing was selected, also list:
|
|
```
|
|
3. npm_token (if using separate npm registry)
|
|
Value: npm registry auth token
|
|
Events: push, manual, tag
|
|
```
|
|
|
|
---
|
|
|
|
## Step 5: Verification Checklist
|
|
|
|
Present this checklist for the user to follow after implementation:
|
|
|
|
```
|
|
=== Verification Checklist ===
|
|
|
|
□ 1. Secrets configured in Woodpecker UI
|
|
→ gitea_username and gitea_token set
|
|
→ Token has package:write scope
|
|
|
|
□ 2. Update docker-compose.yml to use registry images
|
|
→ Change: build: ./src/backend-api
|
|
→ To: image: REGISTRY/ORG/SERVICE:${IMAGE_TAG:-dev}
|
|
|
|
□ 3. Push to develop branch and verify pipeline
|
|
→ Quality gates pass
|
|
→ Docker build steps run (only on main/develop)
|
|
→ Images appear in Gitea Packages
|
|
|
|
□ 4. Verify package linking
|
|
→ Images linked to repository on Gitea Packages tab
|
|
→ If linking fails (404), link manually via Gitea web UI
|
|
|
|
□ 5. Test image pull
|
|
→ docker login REGISTRY
|
|
→ docker pull REGISTRY/ORG/SERVICE:dev
|
|
|
|
□ 6. Commit the pipeline changes
|
|
→ git add .woodpecker.yml
|
|
→ git commit -m "feat: Add Docker build/push/link to CI pipeline"
|
|
```
|
|
|
|
---
|
|
|
|
## Example Session
|
|
|
|
```
|
|
User: /setup-cicd
|
|
|
|
Agent: Let me scan this project...
|
|
|
|
=== CI/CD Scan Results ===
|
|
Registry: git.uscllc.com
|
|
Organization: usc
|
|
Repository: uconnect
|
|
|
|
Dockerfiles Found:
|
|
1. src/backend-api/Dockerfile → backend-api
|
|
2. src/web-portal/Dockerfile → web-portal
|
|
3. src/ingest-api/Dockerfile → ingest-api
|
|
4. src/heartbeat-collector/Dockerfile → heartbeat-collector
|
|
5. src/field-agent/Dockerfile → field-agent
|
|
6. src/egress-api/Dockerfile → egress-api
|
|
|
|
Existing Pipeline: .woodpecker.yml
|
|
- Has build step: yes (build-all)
|
|
- Has Docker steps: no
|
|
|
|
I have a few questions:
|
|
|
|
1. Which Dockerfiles should be built in CI?
|
|
A. backend-api
|
|
B. web-portal
|
|
C. ingest-api
|
|
...
|
|
G. All of the above
|
|
|
|
User: 1G, 2A, 3A, 4A
|
|
|
|
Agent: Generating pipeline YAML...
|
|
[shows YAML]
|
|
Shall I append this to .woodpecker.yml?
|
|
|
|
User: Yes
|
|
|
|
Agent: Done. Here are the secrets you need to configure:
|
|
[shows secrets checklist]
|
|
[shows verification checklist]
|
|
```
|
|
|
|
---
|
|
|
|
## Notes
|
|
|
|
- The generator script handles `$$` escaping for Woodpecker shell variables automatically
|
|
- Package linking requires Gitea 1.24.0+ (the API endpoint was added in that version)
|
|
- If the project has no existing `.woodpecker.yml`, suggest running `init-project.sh` first to set up quality gates
|
|
- For the kaniko_setup anchor, the registry hostname must not include `https://` — just the bare hostname
|
|
- Build context defaults to `.` (project root) for Dockerfiles under `apps/`, `src/`, or `packages/`. For other locations (like `docker/postgres/`), the context is the Dockerfile's parent directory.
|