66 lines
4.1 KiB
Markdown
66 lines
4.1 KiB
Markdown
# RM-61 — CI contract exemption for #1000 teardown artifact
|
||
|
||
**Tracking:** RM-61 / issue #1000
|
||
**Branch:** `fix/rm-61-ci-contract-exemption`
|
||
**Owner:** `coder-mos1`
|
||
|
||
## Objective
|
||
|
||
Determine, by red-first provider controls, whether the `ci-postgres` pod-not-found teardown signature discriminates from a real PostgreSQL failure. Only if it discriminates may a named, bounded CI-contract exemption be implemented. The exemption must retire when #1000 is fixed; fixing #1000 is the closure path.
|
||
|
||
## Pre-registered kill criterion
|
||
|
||
If an injected real `ci-postgres` failure also yields `pods "wp-svc-<ULID>-ci-postgres" not found` as the service's provider-visible failure, the signature does not discriminate. Option B is unsafe; stop exemption implementation and fall to Option A (#1000).
|
||
|
||
## Plan
|
||
|
||
1. Capture full `-f json` records for the 11 supplied observations and state counts.
|
||
2. Run one startup-failure control using the real pgvector/PostgreSQL image with an invalid `initdb` argument.
|
||
3. Run one post-readiness crash control using real PostgreSQL, `pg_isready`, and a deliberate postmaster kill while a DB-dependent probe is active.
|
||
4. Compare the raw `ci-postgres` service record independently of failures in dependent steps.
|
||
5. Investigate runner/time/head clustering only as a hypothesis; never encode incidental correlates or retries into policy.
|
||
6. If and only if the controls discriminate, implement and test the exact exemption, document its two-way boundary, and track retirement at #1000.
|
||
|
||
## Budget
|
||
|
||
No explicit token cap supplied. Working estimate: 20K–30K tokens. Limit provider controls to the two pre-registered runs; no retries or re-roll policy.
|
||
|
||
## Initial evidence
|
||
|
||
Historical JSON saved locally under `.evidence/rm-61/` (not for commit). Supplied pipelines: 11 total. Child-step counts: five pipelines with 9 children and six with 10 children. Seven contain the `ci-postgres` pod-not-found failure (#2170, #2175, #2180, #2181, #2182, #2187, #2188); four do not (#2158, #2167, #2184, #2186). Every observed workflow reports `agent_id=44`, so the available JSON does not separate clean and artifact runs by runner. This refutes runner identity as a discriminator in the sampled record.
|
||
|
||
## Progress
|
||
|
||
- [x] Requirements and kill criterion recorded before control implementation.
|
||
- [x] Historical full-JSON records captured.
|
||
- [x] Startup-failure control observed terminal.
|
||
- [ ] Post-readiness crash control observed terminal.
|
||
- [ ] Discrimination verdict recorded.
|
||
- [ ] Conditional exemption implementation (only if verdict permits).
|
||
|
||
## Tests / evidence
|
||
|
||
### Control 1 — real startup failure
|
||
|
||
- Commit: `3931b0e29eb834914f7b17e4db7e221481d436fa`
|
||
- Pipeline: #2189, exact commit match.
|
||
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
|
||
- `ci-postgres`: `state=failure`, `exit_code=1`, `error=null`, with a five-second execution window.
|
||
- `test`: `state=failure`, `exit_code=1` after the readiness budget expired.
|
||
- Pipeline/workflow: terminal `failure`.
|
||
|
||
This control is red and its service record differs from #1000 (`exit_code=0` plus pod-not-found). It proves the startup-failure direction only. It does not settle the dangerous post-readiness crash/garbage-collection path.
|
||
|
||
### Control 2 setup attempt — invalid, excluded from evidence
|
||
|
||
- Commit: `9455cd6a2650b2b7e70f746c07933d96e5cb3d20`
|
||
- Pipeline: #2190, exact commit match.
|
||
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
|
||
- Service log: `/bin/sh: 0: -c requires an argument`.
|
||
- Root cause: Woodpecker service `commands` did not become the third `sh -c` argument. PostgreSQL never started, so this run is **not** the post-readiness crash control and provides no discrimination evidence.
|
||
- Focused remediation: place the script directly in the third `entrypoint` element and supply `PGPASSWORD` for the marker query. This is a control-fixture correction, not a retry of #1000 and not evidence for either verdict.
|
||
|
||
## Risks
|
||
|
||
The Woodpecker Kubernetes backend may garbage-collect both genuinely failed and successfully used service pods before reconciliation. If so, the provider-visible service signature cannot safely support Option B.
|