ci/woodpecker/pr/ci Pipeline failed
Four blockers from adversarial review, and three are one defect wearing three hats: a test over the WHOLE command text deciding an ALLOW. That is the fail-open shape this file keeps rediscovering, and it had reached the break-glass itself. - Break-glass read POSITIONALLY. `case $CMD in *MOSAIC_WRAPPER_OVERRIDE=1*)` cleared the entire command if that string appeared anywhere in it, so quoting the override in a note, naming a variable after it, or writing =10 disabled the guard for the call sitting beside it. Now only leading NAME=value assignments count, exactly where the shell would honour one. Cost, pinned as a fixture: an override after `&&` no longer arms. - $HOME resolved once, and unset / empty / "/" refused. This was filed as a checkout-arm defect and is larger: under `set -u` the old file died at line 62 on EVERY command with HOME unset, exit 1, before the API arms or the APPROVE trap ran. A seat with no HOME (systemd unit, container, env -i) had no guard at all. A checkout whose question cannot be asked now blocks; the blast radius is asserted to be that one command shape and not the session. - Subresource refinement inverted. It asked whether a subresource appears anywhere in the command, so `gh api -X PATCH .../issues/1 -f body=cf-/pulls/2/files` was cleared on the strength of text in its own body. It now clears only when EVERY numbered-object occurrence carries a subresource. - -K/--config refused. curl reads the method, body, headers and URL from that file, so none of them are in the command: every write test read 0 and the call went through. An unreadable request is not a cleared one. mosaic-worktree: never close a git pipe early resolve_repo took the first porcelain line with `awk ... exit`, which closes the read end while git is still writing. git takes SIGPIPE, pipefail returns 141, and the function aborts SILENTLY — no message, no path, exit 141. It fires as a function of REPO SIZE: fine on three worktrees, reliable on seventy. Measured at 73 worktrees (10 KB of porcelain): rc=141, no output. The file already removed a `head -200` for this exact reason; the rule is now uniform. Evidence — every new case run against029af418, the tree before these fixes: test-wrapper-guard.sh 130/130 pass here; 15 FAIL against029af418test-mosaic-worktree-large-repo.sh 2/2 pass here; 2 FAIL against029af418(got rc=141 and empty output, the signature) No pre-existing fixture changed behaviour on the old guard, so the new cases are the whole delta. The size dependence is stubbed out rather than inherited: a test that ran against whatever repo it sits in would have PASSED on the broken tree.
307 lines
13 KiB
Bash
Executable File
307 lines
13 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# mosaic-worktree.sh — the only supported way to create and dispose of a git
|
|
# worktree on a fleet host.
|
|
#
|
|
# Why this exists as a helper and not as a rule: the rule already existed, in
|
|
# the framework's own words ("Big work → /var/tmp"), and 255 GB accumulated in
|
|
# $HOME across 842 directories anyway. Five placement conventions were live on
|
|
# one fleet host simultaneously. Every one was a decision an agent had to make,
|
|
# and a decision an agent has to make is a decision that drifts.
|
|
#
|
|
# So this script makes NO placement decision available. The caller supplies a
|
|
# branch name. Every path is DERIVED:
|
|
#
|
|
# main worktree <- git worktree list --porcelain (never cwd, which may
|
|
# itself already be a worktree)
|
|
# REPO_NAME <- basename of the main worktree
|
|
# REPO_PARENT <- dirname of the main worktree
|
|
# WT_ROOT <- $REPO_PARENT/$REPO_NAME-worktrees
|
|
# SLUG <- branch with '/' replaced by '-'
|
|
# WT_PATH <- $WT_ROOT/$SLUG
|
|
#
|
|
# The derivation puts the worktree on the same filesystem as the object store
|
|
# it shares, as a sibling of the repo, under one root per repo. Those are the
|
|
# properties that make the checkout cheap and — via `git worktree list` —
|
|
# enumerable, which is the only reason automated cleanup can ever be safe.
|
|
#
|
|
# Usage:
|
|
# mosaic-worktree.sh new <branch> [--from <base>] create (branch may exist)
|
|
# mosaic-worktree.sh path <branch> print derived path, no side effect
|
|
# mosaic-worktree.sh list this repo's worktrees + state
|
|
# mosaic-worktree.sh rm <branch> [--force] remove; refuses to lose work
|
|
# mosaic-worktree.sh gc [--apply] report/remove clean+pushed worktrees
|
|
#
|
|
# `rm` and `gc` refuse to delete a worktree with uncommitted changes, with
|
|
# commits absent from every remote, or holding ignored files that are not of the
|
|
# well-known regenerable kind (a `.env` is ignored so it is never committed,
|
|
# which is also why nothing else holds a copy). That check is by EVIDENCE, never
|
|
# by size or age. --force overrides it and is yours to type deliberately.
|
|
#
|
|
# Run from anywhere inside the repo, or pass --repo <path>.
|
|
|
|
set -euo pipefail
|
|
|
|
die() { printf 'mosaic-worktree: %s\n' "$*" >&2; exit 1; }
|
|
|
|
REPO_HINT=""
|
|
ARGS=()
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--repo) REPO_HINT="${2:-}"; shift 2 ;;
|
|
*) ARGS+=("$1"); shift ;;
|
|
esac
|
|
done
|
|
set -- "${ARGS[@]+"${ARGS[@]}"}"
|
|
|
|
CMD="${1:-}"
|
|
[ -n "$CMD" ] || die "no command. Try: new | path | list | rm | gc"
|
|
shift || true
|
|
|
|
# ---- mechanical derivation -------------------------------------------------
|
|
# The FIRST entry of `git worktree list --porcelain` is always the main
|
|
# worktree, regardless of which worktree we are standing in. Deriving from cwd
|
|
# would nest worktrees inside worktrees.
|
|
resolve_repo() {
|
|
local start="${REPO_HINT:-$PWD}"
|
|
git -C "$start" rev-parse --git-dir >/dev/null 2>&1 \
|
|
|| die "not inside a git repository: $start"
|
|
# Take the first entry WITHOUT closing the pipe early. `awk ... exit` on the
|
|
# first match closes the read end while git is still writing, git takes SIGPIPE,
|
|
# and under `set -euo pipefail` the command substitution returns 141 and this
|
|
# function aborts SILENTLY — no message, no worktree, and `new` exits 141 while
|
|
# printing nothing at all.
|
|
#
|
|
# Whether it happens depends on how much git still had to write when awk left,
|
|
# so the failure is a function of REPO SIZE: fine on a repo with three
|
|
# worktrees, reliably broken on one with seventy. That is backwards — the repos
|
|
# this helper exists to serve are exactly the ones that accumulated worktrees,
|
|
# and it silently did nothing on those while working everywhere it was tried.
|
|
# Measured on a repo with 73 worktrees (10 KB of porcelain): rc=141, no output.
|
|
#
|
|
# The file's own comment block below already names this class for `head -200`
|
|
# and removed that cap for the same reason. The `exit` here is the same defect
|
|
# in the same file, so the rule is now uniform: nothing in this script closes a
|
|
# git pipe early. Dropping `exit` costs one pass over a few KB.
|
|
MAIN_WT="$(git -C "$start" worktree list --porcelain | awk '/^worktree /&&!seen{print substr($0,10); seen=1}')"
|
|
[ -n "$MAIN_WT" ] || die "could not resolve the main worktree"
|
|
REPO_NAME="$(basename -- "$MAIN_WT")"
|
|
REPO_PARENT="$(dirname -- "$MAIN_WT")"
|
|
WT_ROOT="$REPO_PARENT/$REPO_NAME-worktrees"
|
|
}
|
|
|
|
slugify() { printf '%s' "$1" | tr '/' '-'; }
|
|
|
|
derive_path() {
|
|
local branch="$1"
|
|
[ -n "$branch" ] || die "branch name required"
|
|
printf '%s/%s' "$WT_ROOT" "$(slugify "$branch")"
|
|
}
|
|
|
|
# A worktree root under $HOME defeats the entire point: wrong filesystem, and
|
|
# $HOME is for configuration and state, not work products. Refuse rather than
|
|
# silently produce the layout we are trying to eliminate.
|
|
assert_not_home() {
|
|
local p="$1" home_real repo_real
|
|
home_real="$(cd "$HOME" && pwd -P)"
|
|
repo_real="$(cd "$(dirname -- "$p")" 2>/dev/null && pwd -P || dirname -- "$p")"
|
|
case "$repo_real/" in
|
|
"$home_real"/*)
|
|
die "refusing: derived path is under \$HOME ($p).
|
|
The repo itself lives under \$HOME, so its worktrees would too. Move the repo
|
|
to a work filesystem (e.g. /src/$REPO_NAME) and re-run. \$HOME holds
|
|
configuration, credentials, state and caches — not checkouts." ;;
|
|
esac
|
|
}
|
|
|
|
# ---- work-loss evidence ----------------------------------------------------
|
|
# Two independent questions, both answered from git, neither from size or age:
|
|
# dirty — anything uncommitted in the tree
|
|
# unpushed — commits reachable from HEAD that no remote ref contains
|
|
# precious — IGNORED files git will not mention and will not miss
|
|
#
|
|
# The third question is not obvious and was missed on the first pass. An
|
|
# independent reviewer demonstrated it in four commands: a pushed, clean
|
|
# worktree whose .gitignore covers `*.secret`, holding one `local.secret`.
|
|
# `git status --porcelain` is empty, `rev-list --count HEAD --not --remotes` is
|
|
# 0 — the evidence reads SAFE — and `git worktree remove` deletes the file. The
|
|
# same shape covers `.env`, credentials, scratch notes, downloaded fixtures:
|
|
# precisely the files that are ignored BECAUSE they must not be committed, which
|
|
# is also why nothing else is holding a copy.
|
|
#
|
|
# So ignored files count as work unless they are the well-known regenerable
|
|
# kind. Getting that set wrong is asymmetric: an over-broad list preserves a
|
|
# worktree that could have been reclaimed (cheap, visible, fixable by --force),
|
|
# an over-narrow one deletes the only copy of a secret (silent, permanent).
|
|
# The list stays short and conservative for that reason.
|
|
DISPOSABLE_RE='(^|/)(node_modules|\.venv|venv|__pycache__|\.mypy_cache|\.pytest_cache|\.ruff_cache|\.turbo|\.cache|\.parcel-cache|\.gradle|dist|build|out|target|coverage|\.next|\.nuxt|\.svelte-kit)(/|$)|\.(pyc|pyo|o|class)$'
|
|
|
|
# These three run under `set -euo pipefail` inside command substitution, which
|
|
# makes any nonzero exit ANYWHERE in the pipeline abort the calling function
|
|
# silently. Two ways that bites, one of which shipped:
|
|
#
|
|
# * `grep -v` exits 1 when it filters everything out. A worktree whose only
|
|
# ignored entry is `node_modules/` is exactly the SAFE case, and it made
|
|
# `rm` exit 1 with no message and no removal — found by review.
|
|
# * `head -200` closes the pipe, SIGPIPEs the producer, and turns a worktree
|
|
# with 201 dirty files into the same silent abort. Not reported; it is the
|
|
# same defect one step upstream, so the cap is gone. Counting is cheap;
|
|
# the cap only ever protected output that is now never printed.
|
|
#
|
|
# Every one of them therefore ends in a total, and every stage that can
|
|
# legitimately exit nonzero says so explicitly.
|
|
wt_dirty() {
|
|
local out
|
|
out="$(git -C "$1" status --porcelain 2>/dev/null || true)"
|
|
if [ -n "$out" ]; then printf '%s\n' "$out" | wc -l; else printf '0'; fi
|
|
}
|
|
|
|
wt_unpushed() { git -C "$1" rev-list --count HEAD --not --remotes 2>/dev/null || printf '?'; }
|
|
|
|
# Default --ignored (not =matching) so a 40k-file node_modules collapses to one
|
|
# directory entry instead of being enumerated and then discarded.
|
|
wt_precious() {
|
|
local ignored
|
|
ignored="$(git -C "$1" status --porcelain --ignored 2>/dev/null \
|
|
| awk '/^!! /{print substr($0,4)}' || true)"
|
|
[ -n "$ignored" ] || { printf '0'; return 0; }
|
|
printf '%s\n' "$ignored" | grep -Ecv "$DISPOSABLE_RE" || true
|
|
}
|
|
|
|
wt_state() {
|
|
local wt="$1" d u p
|
|
d="$(wt_dirty "$wt")"; u="$(wt_unpushed "$wt")"; p="$(wt_precious "$wt")"
|
|
if [ "$d" -eq 0 ] && [ "$u" = "0" ] && [ "$p" -eq 0 ]; then
|
|
printf 'SAFE\tclean; 0 unpushed; no ignored files worth keeping'
|
|
else
|
|
printf 'PRESERVE\t%s uncommitted; %s unpushed; %s ignored-but-not-disposable' "$d" "$u" "$p"
|
|
fi
|
|
}
|
|
|
|
# ---- commands --------------------------------------------------------------
|
|
cmd_path() { resolve_repo; derive_path "${1:-}"; echo; }
|
|
|
|
cmd_new() {
|
|
local branch="${1:-}" base=""
|
|
shift || true
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in --from) base="${2:-}"; shift 2 ;; *) die "unknown flag: $1" ;; esac
|
|
done
|
|
[ -n "$branch" ] || die "usage: mosaic-worktree.sh new <branch> [--from <base>]"
|
|
|
|
resolve_repo
|
|
local path; path="$(derive_path "$branch")"
|
|
assert_not_home "$path"
|
|
|
|
if [ -e "$path" ]; then
|
|
echo "exists: $path"
|
|
echo "(already checked out — reuse it, or 'rm' it first)"
|
|
return 0
|
|
fi
|
|
|
|
mkdir -p "$WT_ROOT"
|
|
|
|
# Existing branch -> check it out. New branch -> create from base (default:
|
|
# the remote's default branch if resolvable, else current HEAD).
|
|
if git -C "$MAIN_WT" show-ref --verify --quiet "refs/heads/$branch" \
|
|
|| git -C "$MAIN_WT" show-ref --verify --quiet "refs/remotes/origin/$branch"; then
|
|
git -C "$MAIN_WT" worktree add "$path" "$branch"
|
|
else
|
|
if [ -z "$base" ]; then
|
|
base="$(git -C "$MAIN_WT" symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || true)"
|
|
[ -n "$base" ] || base="HEAD"
|
|
fi
|
|
git -C "$MAIN_WT" worktree add -b "$branch" "$path" "$base"
|
|
fi
|
|
|
|
cat <<EOF
|
|
|
|
worktree: $path
|
|
branch: $branch
|
|
|
|
Removal is part of this task, not a later chore. When the work is pushed:
|
|
mosaic-worktree.sh rm $branch
|
|
EOF
|
|
}
|
|
|
|
cmd_list() {
|
|
resolve_repo
|
|
printf 'repo: %s\nroot: %s\n\n' "$MAIN_WT" "$WT_ROOT"
|
|
git -C "$MAIN_WT" worktree list --porcelain \
|
|
| awk '/^worktree /{print substr($0,10)}' \
|
|
| while read -r wt; do
|
|
[ "$wt" = "$MAIN_WT" ] && { printf '%-10s %s (main)\n' "-" "$wt"; continue; }
|
|
printf '%-10s %s\t%s\n' "$(wt_state "$wt" | cut -f1)" "$wt" "$(wt_state "$wt" | cut -f2)"
|
|
done
|
|
}
|
|
|
|
cmd_rm() {
|
|
local branch="${1:-}" force=0
|
|
shift || true
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in --force) force=1; shift ;; *) die "unknown flag: $1" ;; esac
|
|
done
|
|
[ -n "$branch" ] || die "usage: mosaic-worktree.sh rm <branch> [--force]"
|
|
|
|
resolve_repo
|
|
local path; path="$(derive_path "$branch")"
|
|
[ -d "$path" ] || die "no worktree at $path"
|
|
|
|
local d u p
|
|
d="$(wt_dirty "$path")"; u="$(wt_unpushed "$path")"; p="$(wt_precious "$path")"
|
|
if [ "$force" -eq 0 ] && { [ "$d" -ne 0 ] || [ "$u" != "0" ] || [ "$p" -ne 0 ]; }; then
|
|
die "refusing to remove $path
|
|
uncommitted files: $d
|
|
unpushed commits: $u
|
|
ignored, not disposable: $p
|
|
Commit and push first — that is the contract. Ignored files are counted because
|
|
git will neither report them nor miss them: a .env or a *.secret is ignored
|
|
precisely so it is never committed, which is also why nothing else holds a copy.
|
|
List them with: git -C $path status --porcelain --ignored | grep '^!!'
|
|
If this work is genuinely disposable, re-run with --force."
|
|
fi
|
|
|
|
# NB: ${force:+--force} would expand for force=0 too ("0" is non-empty).
|
|
if [ "$force" -eq 1 ]; then
|
|
git -C "$MAIN_WT" worktree remove --force "$path"
|
|
else
|
|
git -C "$MAIN_WT" worktree remove "$path"
|
|
fi
|
|
git -C "$MAIN_WT" worktree prune
|
|
echo "removed: $path"
|
|
rmdir "$WT_ROOT" 2>/dev/null || true
|
|
}
|
|
|
|
cmd_gc() {
|
|
local apply=0
|
|
[ "${1:-}" = "--apply" ] && apply=1
|
|
resolve_repo
|
|
git -C "$MAIN_WT" worktree prune
|
|
git -C "$MAIN_WT" worktree list --porcelain \
|
|
| awk '/^worktree /{print substr($0,10)}' \
|
|
| while read -r wt; do
|
|
[ "$wt" = "$MAIN_WT" ] && continue
|
|
local_state="$(wt_state "$wt")"
|
|
case "$local_state" in
|
|
SAFE*)
|
|
if [ "$apply" -eq 1 ]; then
|
|
git -C "$MAIN_WT" worktree remove "$wt" && echo "removed: $wt"
|
|
else
|
|
echo "reclaimable (clean + fully pushed): $wt"
|
|
fi ;;
|
|
*) echo "preserved: $wt [$(printf '%s' "$local_state" | cut -f2)]" ;;
|
|
esac
|
|
done
|
|
git -C "$MAIN_WT" worktree prune
|
|
[ "$apply" -eq 1 ] || echo $'\n(report only — re-run with --apply to remove the reclaimable ones)'
|
|
}
|
|
|
|
case "$CMD" in
|
|
new) cmd_new "$@" ;;
|
|
path) cmd_path "$@" ;;
|
|
list) cmd_list "$@" ;;
|
|
rm) cmd_rm "$@" ;;
|
|
gc) cmd_gc "$@" ;;
|
|
-h|--help|help) sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//' ;;
|
|
*) die "unknown command: $CMD (new | path | list | rm | gc)" ;;
|
|
esac
|