veronica
f0d2dd9920
Applies the document contract from
docs/plans/2026-08-20_stack-docs-flatten-and-alignment.md section 3, partially:
`kind` and `status` only. `parent` is deliberately held until the flatten in
section 4 lands, so that 127 documents do not have to be re-pointed by hand
when docs/fleet/NORTH_STAR.yaml moves to docs/NORTH_STAR.yaml.
Scope, measured on origin/next at 63069149:
127 live docs = all *.md under docs/ minus docs/archive/ minus docs/_old_structure/
104 stamped here
19 held operator judgement (plan section 9), worklist in the same PR
3 held the SUPERSEDED TASKS.md stamps, which cite the moving path
1 untouched docs/fleet/FLEET-DOCTRINE.md, already stamped in W1
Kinds applied: 54 guide, 34 record, 9 spec, 6 tracking, 1 projection.
Every row carries a confidence and a one-line rationale in the worklist.
Two collisions with the existing state, both flagged rather than resolved:
1. docs/README.md:150-160 already documents a front-matter convention
(title/type/audience/status/source_of_truth) with its own allowed values.
It is applied to 4 of 127 files. Its `status` vocabulary is
current|draft|deprecated|historical; the new contract's is active|superseded-by.
The key collides. This commit lets the new contract win and rewrites
`status: current` to `status: active` on those 4 files, keeping their other
legacy keys untouched. No code reads any of them: `git grep source_of_truth`
outside docs/ returns nothing. docs/README.md still prescribes the old
convention and is an operator row, so it is not edited here.
2. Two of the plan's 20 operator rows are YAML files, not markdown
(docs/fleet/examples/roster-v2.yaml, docs/openapi-tess.yaml), and the
contract's front-matter form has no defined meaning for a .yaml document.
That gap also applies to docs/fleet/NORTH_STAR.yaml, the source of truth
itself. Raised in the worklist.
A third row from the plan, docs/fleet/north-star.md, no longer exists: W1
renamed it to docs/fleet/FLEET-DOCTRINE.md.
Verification: 104/104 parse with the expected kind and status in front matter;
the check was shown to reject a wrong kind before it was trusted. The diff
removes 4 lines total, all of them `status: current`.
77 lines
6.9 KiB
Markdown
77 lines
6.9 KiB
Markdown
---
|
|
kind: record
|
|
status: active
|
|
---
|
|
|
|
# #1019 — Zero-timeout queue-guard harness race
|
|
|
|
- **Issue:** #1019 (parent status remains `believed-fixed, pending jarvis validation`; do not close)
|
|
- **Branch:** `fix/1019-ci-queue-timeout-harness`
|
|
- **Owner:** `be-coder-08`
|
|
- **Base:** `origin/main` at `5916aeefd6ed12bcac086c6834c7f6c4ae38e1bc`
|
|
- **Charter:** `/home/hermes/agent-work/tl-mosaic/CHARTER-1019-HARNESS-FIX.md`
|
|
|
|
## Objective
|
|
|
|
Make `test-ci-queue-wait-tristate.sh` deterministic without changing any asserted outcome. Remove the indiscriminate zero-timeout race, require every status-classification case to prove the provider was observed, and prove the harness-controlled virtual clock is active.
|
|
|
|
## Scope
|
|
|
|
- In scope: `packages/mosaic/framework/tools/git/test-ci-queue-wait-tristate.sh` only, plus this evidence scratchpad.
|
|
- Out of scope: guard parsers, D2/D3 behavior, installer/reseed staleness, PR #1060, and issue closure.
|
|
|
|
## Acceptance criteria
|
|
|
|
1. RED deterministically reproduces deadline pre-emption before the provider call.
|
|
2. Every case that intends status classification positively proves provider observation.
|
|
3. Pending observes `pending` before deterministic virtual-time expiration.
|
|
4. The virtual clock has a positive interception control; a broken-clock mutant makes the suite red.
|
|
5. The exact CI-base image passes the final harness repeatedly with zero failures.
|
|
6. Baseline gates, independent code/security review, exact-head CI, and coordinator-authorized squash merge pass.
|
|
|
|
## Plan
|
|
|
|
1. Add deterministic RED instrumentation for the known merge/provider-unreachable pre-emption.
|
|
2. Replace global `-t 0` with a nonzero timeout interpreted under an event-driven virtual clock; stub sleep without wall waiting.
|
|
3. Add provider-observation and virtual-clock positive controls without changing outcome assertions.
|
|
4. Run focused shell checks, repeat in exact CI-base image, baseline gates, and independent reviews.
|
|
5. Commit with both identity layers, queue-guard plus direct Woodpecker terminal-state verification, push, self-post PR, verify poster/head/CI, obtain coordinator merge authorization, then squash merge without closing #1019.
|
|
|
|
## Budget
|
|
|
|
- No explicit token cap supplied. Keep scope to one harness file and one scratchpad; stop/report at the charter's 60% context gate.
|
|
|
|
## Evidence
|
|
|
|
- RED, deterministic pre-provider expiry: `evidence/1019-harness-fix/red-pre-provider-expiry.log` — rc 1; merge/provider-unreachable got rc 124 instead of 75, omitted CANNOT_ASSERT, did not observe the status provider, and wrote no additional audit record (four named failures).
|
|
- GREEN host focused harness: `evidence/1019-harness-fix/green-host.log` — rc 0, all outcome classes passed.
|
|
- Load-bearing clock negative control: a temporary same-directory mutant replaced the virtual `date` body with `/bin/date`; `evidence/1019-harness-fix/red-clock-not-intercepted.log` — rc 1 with named `virtual clock interception did not run` failures. The mutant file was removed after the run.
|
|
- Exact CI-base repeat: `git.mosaicstack.dev/mosaicstack/stack/ci-base:latest`, repository mounted read-only, harness work under container `/tmp`; `evidence/1019-harness-fix/ci-image-repeat/summary.log` — **100 pass / 0 fail / 100 total**.
|
|
- Synchronization design: provider-status observation creates the event marker; virtual time is 1000 before the event and 1002 afterward. Pending alone reaches the stubbed no-op sleep and a post-observation deadline check. `-t 1` is uniquely load-bearing because removing it restores the 900-second default deadline at virtual time 1900, which 1002 does not cross. The numeric timeout is subject semantics under virtual time, not a wall-clock synchronization duration.
|
|
|
|
## Review remediation — semantic timeout vs. liveness bound
|
|
|
|
Security review found that virtual time remained at 1000 forever before provider observation and stubbed sleep never waited. A regression looping before the status endpoint—or blocking in the first provider call—therefore could prevent `run_guard` from returning, so the post-return provider assertion could never fire.
|
|
|
|
**General rule:** A timeout usually serves two purposes: semantics and liveness. Removing wall time from semantic synchronization can silently remove the only independent hang bound. Preserve deterministic virtual time for subject semantics, but provide a separately implemented real-clock liveness watchdog and prove that watchdog fires.
|
|
|
|
Remediation:
|
|
|
|
- Every guard subject invocation is launched by absolute `/usr/bin/python3` in a new session. Python's internal monotonic `wait(timeout=...)` provides real-clock liveness independently of PATH; expiry kills the entire isolated process group, so neither PATH-front shims nor a blocked provider descendant can retain the capture pipe.
|
|
- Watchdog expiry returns distinct harness rc 90 plus `FAIL HANG watchdog`, separate from subject timeout rc 124.
|
|
- A first attempt using absolute `/usr/bin/timeout -s KILL` passed on GNU coreutils but failed in the exact Alpine CI-base image: BusyBox killed the immediate wrapper while the guard/provider descendants survived and retained the command-substitution pipe. The process-group kill is therefore required behavior, not portability polish.
|
|
- A committed positive control hangs the branch-provider stub before the status endpoint. It must terminate through the watchdog, emit the hang-specific diagnostic, return rc 90, and prove the status provider was never reached.
|
|
- RED before remediation: a temporary ordinary-success mutant hung before provider observation; only an external control could kill the suite (rc 137), and there was no internal hang-specific diagnostic (`red-watchdog-absent.log`).
|
|
- The watchdog mutant/control is load-bearing: removing the internal watchdog leaves the control unable to produce its required rc 90 and diagnostic.
|
|
|
|
Post-review evidence:
|
|
|
|
- Host focused harness with process-group watchdog: rc 0 (`green-watchdog-process-group-host.log`).
|
|
- Exact Alpine CI-base focused harness with process-group watchdog: rc 0 (`green-watchdog-ci-image.log`).
|
|
- Hanging ordinary-success mutant: suite rc 1; success returned rc 90, emitted `FAIL HANG watchdog`, and loudly reported that provider/clock observation did not occur (`red-watchdog-fires.log`).
|
|
- Removed-`-t 1` mutant: suite rc 1; pending was terminated by the watchdog instead of producing `ASSERTED_NOT_READY`, proving the explicit timeout is load-bearing (`red-timeout-argument-removed.log`).
|
|
|
|
## 60% context hold
|
|
|
|
Stopped before baseline/review/commit as required by the charter. Remaining: inspect final diff, shell/static/baseline gates, independent code/security review, remediation if any, identity-bound commit/trailer verification, mandatory queue guard plus direct terminal Woodpecker `mosaic` enumeration, push, self-posted PR/provider poster read-back, exact-head terminal-green CI, coordinator merge authorization, squash merge, main CI verification, and leave #1019 unclosed as `believed-fixed, pending jarvis validation`.
|