From b7a6179a58fc09329dcfba0f54f4de6aaa66b0aa Mon Sep 17 00:00:00 2001 From: fred Date: Sat, 15 Aug 2026 13:46:48 -0500 Subject: [PATCH] installer: harden the Node provisioning path against its own inputs Answers the review on #1228. Each item below was measured against the pre-change code, and where the review's stated consequence did not reproduce, that is recorded rather than repeated. BLOCKER -- `mapfile` is a Bash 4 builtin and macOS ships Bash 3.2, which this installer supports (node_platform names Darwin). newest_matching_file was therefore unavailable on macOS, and an empty answer is exactly what sends the uninstaller down its delete-the-destination branch. The lookup no longer renders candidates as text at all: the glob output is compared in-shell by mtime, via a stat helper that probes for GNU -c vs BSD -f once. That removes the Bash 4 dependency, the `ls | head` SIGPIPE failure, and the newline-splitting bug together, because all three came from turning filenames into lines. The function now distinguishes three outcomes instead of two: found, nothing matched, and could-not-tell. Callers act destructively on the answer, so the third case had to stop being indistinguishable from the second. The uninstaller leaves the file in place on an unanswerable lookup, and the manifest builder refuses to record a null backup it cannot vouch for. HIGH -- writing ~/.profile does not reach the shells that matter. A bash login shell reads the first of .bash_profile / .bash_login / .profile that exists and never looks at the rest, so on a host with either of the first two the entry was a silent no-op; a non-interactive remote zsh reads .zshenv and neither .zprofile nor .zshrc, which is what the previous version wrote; and a systemd --user unit reads no shell file at all, which is how a Mosaic agent seat starts. All four are now covered, with .bash_profile and .bash_login appended to only when they already exist -- creating one would itself start shadowing .profile. The systemd case is an environment.d drop-in. MEDIUM -- the checksum lookup interpolated the filename into a grep pattern. A Node tarball name is mostly dots, and a dot matches any character, so a manifest line for a different-but-regex-equivalent name was accepted as this file's checksum. Confirmed against the old function: it accepted the decoy. Filenames are now compared exactly, every line is read so a duplicate entry is refused rather than silently resolved, and the digest must look like a SHA-256. MEDIUM -- the PATH line is executed by every future shell that reads the file, and the directory was interpolated unescaped. A path containing shell syntax is now refused with a message instead of written. MEDIUM -- the idempotence check was an unanchored substring match, so a commented-out example of the same export made the installer skip the real entry. Reproduced against the old function, and now anchored with grep -Fqx. MEDIUM -- MOSAIC_NODE_DIST accepted any scheme. https:// and file:// only. The narrower point in the review stands and is not fixed by this: when the dist is overridden, the tarball and the checksum that vouches for it come from the same place, so the gate is integrity and not authenticity. HIGH, with a correction -- MOSAIC_NODE_VERSION is now validated before it becomes a path, but the review's specific consequence does not reproduce. `rm -rf` on a path ending in `..` is refused by rm itself, and a traversal version mangles the download URL so the run dies at curl long before the removal. Both were measured. The check is defence in depth and a clearer error, not a demonstrated hole being closed. Also removed a second `| head -1` in node_resolve_version, the same SIGPIPE shape as the one this PR already fixed, and the index result is validated before it becomes a path. Tests. The review was right that several existing cases passed on the unpatched code. The version-selection case now lists a higher major first and an older release of the right major after the right answer, so "first entry" and "last match" both fail it. The PATH case starts a real login shell and asks it to resolve node, rather than grepping for text the installer just wrote. The checksum-failure case asserts nothing survives, including the staging directory. New cases cover the empty manifest, the regex-equivalent decoy, the duplicate entry, the invalid version, the non-https dist, the shell-syntax path, the commented-out profile line, the .bash_profile shadow, and the environment.d drop-in. Each new case was run against the pre-change installer: the decoy, the commented-out line, the .bash_profile shadow and environment.d all go red there, which is the evidence that they test something. Bash 3.2 cannot be executed here, so the portability guard is a lint over install.sh for Bash 4 syntax. It is a weaker instrument than a run and is not claimed otherwise -- but every Bash 4 construct that has broken macOS in this file was added by someone who was not running it there either. test:installer passes. --- tools/install-newest-matching-file.test.sh | 83 +++++++- tools/install-node-provisioning.test.sh | 195 ++++++++++++++++- tools/install.sh | 235 ++++++++++++++++++--- 3 files changed, 463 insertions(+), 50 deletions(-) diff --git a/tools/install-newest-matching-file.test.sh b/tools/install-newest-matching-file.test.sh index acb07aa1..237157c7 100755 --- a/tools/install-newest-matching-file.test.sh +++ b/tools/install-newest-matching-file.test.sh @@ -1,15 +1,22 @@ #!/usr/bin/env bash -# Regression test for newest_matching_file() in tools/install.sh. +# Tests for newest_matching_file() in tools/install.sh. # -# The function used to pipe `ls -1t` into `head -1`. Under `set -o pipefail` head -# closes the pipe after the first line, ls dies on SIGPIPE, and the function returns -# 141 with no output -- so once a directory holds enough matches to fill a pipe -# buffer (~1600 names), "find the newest backup" starts failing the install instead -# of answering. Its callers assign it at top level under `set -e`, so a 141 aborts -# the run. +# The function answers one question -- "which is the most recent backup / tarball +# here?" -- and its callers act destructively on the answer. Three ways of getting it +# wrong have already been found, and each has a case below: # -# The large-population case is the whole point: with two or three files the old code -# passes, which is why this went unnoticed. +# * `ls -1t | head -1` returns 141 under `set -o pipefail` once the listing fills a +# pipe buffer (~1600 names), because head closes the pipe and ls takes SIGPIPE. +# Callers assign it at top level under `set -e`, so a 141 aborts the run. +# * `mapfile` is a Bash 4 builtin. macOS ships Bash 3.2 and the installer supports +# Darwin, so the whole lookup was unavailable there -- and an empty answer is what +# sends the uninstaller down its delete-the-destination branch. +# * Any line-based parse of `ls` splits a filename containing a newline into two +# wrong answers. +# +# The large-population and newline cases are the point: with two or three ordinary +# names every version of this function passes, which is why the first two went +# unnoticed. set -euo pipefail @@ -17,7 +24,9 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" TMP="$(mktemp -d "${TMPDIR:-/tmp}/mosaic-newest-match-test-XXXXXX")" trap 'rm -rf "$TMP"' EXIT -# Load just the function under test, with the same shell options install.sh runs under. +# Load the function under test and the mtime helper it depends on, with the same +# shell options install.sh runs under. +eval "$(sed -n '/^_MTIME_STYLE=/,/^}/p' "$ROOT/tools/install.sh")" eval "$(sed -n '/^newest_matching_file()/,/^}/p' "$ROOT/tools/install.sh")" POPULATED="$TMP/many" @@ -55,6 +64,30 @@ GOT="$(newest_matching_file "$SMALL" 'mosaicstack-gateway-*.tgz')" exit 1 } +echo "[test] a name containing a space is returned whole" +SPACED="$TMP/spaced" +mkdir -p "$SPACED" +: > "$SPACED/agents.md.mosaic-bak-one two" +GOT="$(newest_matching_file "$SPACED" 'agents.md.mosaic-bak-*')" +[[ "$GOT" == "$SPACED/agents.md.mosaic-bak-one two" ]] || { + echo "expected the spaced name intact, got '${GOT}'" >&2 + exit 1 +} + +echo "[test] a name containing a newline is returned whole, not split" +# The old `ls -1t` parse reported this file as two separate shorter names, neither of +# which exists -- so the caller saw a backup path that could not be restored. +NEWLINE="$TMP/newline" +mkdir -p "$NEWLINE" +WEIRD="$NEWLINE/agents.md.mosaic-bak-$(printf 'a\nb')" +: > "$WEIRD" +GOT="$(newest_matching_file "$NEWLINE" 'agents.md.mosaic-bak-*')" +[[ "$GOT" == "$WEIRD" ]] || { + echo "expected the newline-containing name intact, got '${GOT}'" >&2 + exit 1 +} +[[ -f "$GOT" ]] || { echo "the returned path does not name a real file" >&2; exit 1; } + echo "[test] no match is an empty answer, not an error" EMPTY="$TMP/none" mkdir -p "$EMPTY" @@ -71,4 +104,34 @@ RC=$? set -e [[ "$RC" -eq 0 && -z "$GOT" ]] || { echo "expected empty output and rc=0, got '${GOT}' rc=${RC}" >&2; exit 1; } +echo "[test] an unanswerable lookup fails loudly instead of reporting no match" +# This is the distinction the uninstaller depends on. "No backup exists" is licence to +# delete the destination; "I could not tell" must never reach that branch. +_MTIME_STYLE=none +set +e +GOT="$(newest_matching_file "$SMALL" 'mosaicstack-gateway-*.tgz')" +RC=$? +set -e +_MTIME_STYLE="" +[[ "$RC" -ne 0 ]] || { + echo "expected a non-zero rc when no mtime source is usable, got rc=0 output '${GOT}'" >&2 + exit 1 +} + +echo "[test] the installer uses no Bash 4 syntax" +# A lint, not an execution test: this host has no Bash 3.2 to run under. It is still +# the thing that stops the regression, because every Bash 4 construct that has broken +# macOS here was introduced by someone who never ran the script there either. +# Comments are stripped first -- the ones above name these constructs on purpose. +BASH4_HITS="$( + sed 's/#.*$//' "$ROOT/tools/install.sh" \ + | grep -nE '(^|[^[:alnum:]_])(mapfile|readarray)([^[:alnum:]_]|$)|declare[[:space:]]+-[a-zA-Z]*A|local[[:space:]]+-[a-zA-Z]*A|\$\{[A-Za-z_][A-Za-z0-9_]*(\^\^|,,)' \ + || true +)" +[[ -z "$BASH4_HITS" ]] || { + echo "tools/install.sh uses Bash 4+ syntax, which macOS's Bash 3.2 cannot run:" >&2 + echo "$BASH4_HITS" >&2 + exit 1 +} + echo "[test] newest_matching_file tests passed" diff --git a/tools/install-node-provisioning.test.sh b/tools/install-node-provisioning.test.sh index 8fa154d2..98d87fdb 100755 --- a/tools/install-node-provisioning.test.sh +++ b/tools/install-node-provisioning.test.sh @@ -42,8 +42,10 @@ case "$(uname -m)" in esac PLATFORM="${TEST_OS}-${TEST_ARCH}" -VERSION=v22.99.0 -OLD_VERSION=v20.99.0 +VERSION=v22.99.0 # the one that must be chosen +MID_VERSION=v22.50.0 # same major, older -- catches "take the last match" +OLD_VERSION=v20.99.0 # wrong major +NEWER_MAJOR=v24.99.0 # listed first -- catches "take the first entry" # ─── fixtures ───────────────────────────────────────────────────────────────── @@ -130,9 +132,14 @@ publish_release() { } publish_release "$VERSION" +publish_release "$MID_VERSION" publish_release "$OLD_VERSION" -# Newest-first, as nodejs.org publishes it. -printf '[{"version":"%s"},{"version":"%s"}]\n' "$VERSION" "$OLD_VERSION" > "$DIST/index.json" +publish_release "$NEWER_MAJOR" +# Newest-first, as nodejs.org publishes it. Every wrong entry is genuinely installable, +# so a resolver that picks one fails on the assertion rather than on a 404 -- the +# assertion is then about version selection and not about the fixture. +printf '[{"version":"%s"},{"version":"%s"},{"version":"%s"},{"version":"%s"}]\n' \ + "$NEWER_MAJOR" "$VERSION" "$MID_VERSION" "$OLD_VERSION" > "$DIST/index.json" # A PATH with the usual tools but no Node toolchain, so "a host with no Node" is # actually true on a developer machine and in CI, both of which have one installed. @@ -182,7 +189,12 @@ grep -qF -- "Installed Node ${VERSION}" <<<"$OUTPUT" [[ -x "${NODE_HOME}/${VERSION}/bin/node" ]] grep -qF -- "install -g @mosaicstack/mosaic@0.0.50-next.999" "$LOG" -echo "[test] the newest matching major is chosen, not merely the first published" +echo "[test] the newest release of the required major is chosen" +# The index lists a higher major first and an older release of the right major after +# the right answer, so "first entry" and "last match" both produce a wrong directory. +[[ -d "${NODE_HOME}/${VERSION}" ]] +[[ ! -d "${NODE_HOME}/${NEWER_MAJOR}" ]] +[[ ! -d "${NODE_HOME}/${MID_VERSION}" ]] [[ ! -d "${NODE_HOME}/${OLD_VERSION}" ]] echo "[test] future shells can find both Node and the CLI" @@ -193,11 +205,54 @@ grep -qF -- "export PATH=\"${PREFIX}/bin:\$PATH\"" "$HOME_DIR/.profile" grep -qF -- "export PATH=\"${NODE_HOME}/${VERSION}/bin:\$PATH\"" "$HOME_DIR/.bashrc" grep -qF -- "export PATH=\"${PREFIX}/bin:\$PATH\"" "$HOME_DIR/.bashrc" +echo "[test] a real login shell resolves node, not just the text of a profile line" +# Grepping the file only proves the installer wrote something. This starts an actual +# login shell against that HOME and asks it to find the binary. +RESOLVED="$(env -i HOME="$HOME_DIR" PATH="$NONODE_BIN" TERM=dumb bash -lc 'command -v node')" +[[ "$RESOLVED" == "${NODE_HOME}/${VERSION}/bin/node" ]] || { + echo "a login shell resolved node to '${RESOLVED}'" >&2 + exit 1 +} + +echo "[test] a systemd --user unit gets the same PATH, via environment.d" +# Units read no shell file at all, which is how a Mosaic agent seat starts. +ENVD="$HOME_DIR/.config/environment.d/50-mosaic-path.conf" +[[ -f "$ENVD" ]] || { echo "no environment.d drop-in was written" >&2; exit 1; } +grep -qF -- "PATH=${NODE_HOME}/${VERSION}/bin:\${PATH}" "$ENVD" +grep -qF -- "PATH=${PREFIX}/bin:\${PATH}" "$ENVD" + echo "[test] re-running reuses the Node it installed and does not duplicate PATH lines" OUTPUT="$(run_bare --cli --next --yes --no-auto-launch 2>&1)" grep -qF -- "from ${NODE_HOME}" <<<"$OUTPUT" [[ "$(grep -c 'export PATH=' "$HOME_DIR/.profile")" -eq 2 ]] [[ "$(grep -c 'export PATH=' "$HOME_DIR/.bashrc")" -eq 2 ]] +[[ "$(grep -c '^PATH=' "$ENVD")" -eq 2 ]] + +reset_home +echo "[test] a ~/.bash_profile does not silently swallow the PATH entry" +# A bash login shell reads the first of .bash_profile / .bash_login / .profile that +# exists and never looks at the rest. Writing only .profile is a no-op on such a host, +# and the failure is invisible until something cannot find node. +: > "$HOME_DIR/.bash_profile" +run_bare --cli --next --yes --no-auto-launch >/dev/null 2>&1 +RESOLVED="$(env -i HOME="$HOME_DIR" PATH="$NONODE_BIN" TERM=dumb bash -lc 'command -v node')" +[[ "$RESOLVED" == "${NODE_HOME}/${VERSION}/bin/node" ]] || { + echo "with a .bash_profile present, a login shell resolved node to '${RESOLVED}'" >&2 + exit 1 +} + +reset_home +echo "[test] a commented-out example does not count as the PATH entry already existing" +# The idempotence check used to be an unanchored substring match, so a line like this +# in a user's profile made the installer skip the real entry. +mkdir -p "$HOME_DIR" +printf '# export PATH="%s/%s/bin:$PATH"\n' "$NODE_HOME" "$VERSION" > "$HOME_DIR/.profile" +run_bare --cli --next --yes --no-auto-launch >/dev/null 2>&1 +[[ "$(grep -c '^export PATH=' "$HOME_DIR/.profile")" -eq 2 ]] || { + echo "expected two real export lines, found:" >&2 + cat "$HOME_DIR/.profile" >&2 + exit 1 +} reset_home echo "[test] --no-node-install refuses instead of installing" @@ -228,7 +283,13 @@ RC=$? set -e [[ "$RC" -ne 0 ]] grep -qF -- "failed checksum verification" <<<"$OUTPUT" +# Not just "no usable node": nothing at all may survive. An unpack that ran before +# verification, or a staging directory left behind, would still satisfy the weaker +# check while leaving unverified bytes on disk for the next run to adopt. [[ ! -x "${NODE_HOME}/${VERSION}/bin/node" ]] +[[ ! -e "${NODE_HOME}/${VERSION}" ]] +[[ ! -e "${NODE_HOME}/${VERSION}.partial" ]] +[[ ! -d "$NODE_HOME" ]] || [[ -z "$(ls -A "$NODE_HOME")" ]] publish_release "$VERSION" reset_home @@ -272,4 +333,128 @@ OUTPUT="$( grep -qF -- "older than the required >= 22" <<<"$OUTPUT" [[ -x "${NODE_HOME}/${VERSION}/bin/node" ]] +# ─── refusals: untrusted input that reaches a path or an exec ───────────────── + +reset_home +echo "[test] an empty checksum manifest is refused, not read as an empty digest" +: > "${DIST}/${VERSION}/SHASUMS256.txt" +set +e +OUTPUT="$(run_bare --cli --next --yes --no-auto-launch 2>&1)" +RC=$? +set -e +[[ "$RC" -ne 0 ]] +grep -qF -- "No checksum published" <<<"$OUTPUT" +[[ ! -e "${NODE_HOME}/${VERSION}" ]] +publish_release "$VERSION" + +reset_home +echo "[test] a manifest naming a regex-equivalent file does not vouch for this one" +# The lookup used to interpolate the filename into a grep pattern. A Node tarball name +# is mostly dots, and a dot matches any character, so this line -- which names a +# different file -- was accepted as this file's checksum. +DECOY="node-${VERSION}-${PLATFORM}Xtar.gz" +printf '%s %s\n' "$(printf '0%.0s' $(seq 1 64))" "$DECOY" > "${DIST}/${VERSION}/SHASUMS256.txt" +set +e +OUTPUT="$(run_bare --cli --next --yes --no-auto-launch 2>&1)" +RC=$? +set -e +[[ "$RC" -ne 0 ]] +grep -qF -- "No checksum published" <<<"$OUTPUT" +[[ ! -e "${NODE_HOME}/${VERSION}" ]] +publish_release "$VERSION" + +reset_home +echo "[test] a manifest listing the same file twice is refused rather than guessed at" +BASE="node-${VERSION}-${PLATFORM}.tar.gz" +GOOD="$(awk '{print $1}' "${DIST}/${VERSION}/SHASUMS256.txt")" +{ + printf '%s %s\n' "$GOOD" "$BASE" + printf '%s %s\n' "$(printf '0%.0s' $(seq 1 64))" "$BASE" +} > "${DIST}/${VERSION}/SHASUMS256.txt" +set +e +OUTPUT="$(run_bare --cli --next --yes --no-auto-launch 2>&1)" +RC=$? +set -e +[[ "$RC" -ne 0 ]] +grep -qF -- "refusing to guess" <<<"$OUTPUT" +[[ ! -e "${NODE_HOME}/${VERSION}" ]] +publish_release "$VERSION" + +echo "[test] a version string is checked before it becomes a path" +# MOSAIC_NODE_VERSION becomes a directory name under NODE_HOME, and that directory is +# later handed to `rm -rf`. This is defence in depth, and the honest scope should be +# recorded: the plain 'v..' case is separately refused by rm itself, and a traversal +# value breaks the download URL before the removal is reached. Measured, not assumed. +# What the check buys is that neither of those accidents is what is protecting us, and +# that a typo is refused with its own name on it rather than a curl error. +eval "$(sed -n '/^node_valid_version()/,/^}/p' "$ROOT/tools/install.sh")" +for good in v22.99.0 v0.0.0 v22.11.0 v100.0.1; do + node_valid_version "$good" || { echo "rejected a real version: ${good}" >&2; exit 1; } +done +for bad in 'v..' '..' 'v9.9.9/../../elsewhere' '/etc' 'v22' 'v22.1' '22.1.0' 'v22.1.0-rc1' '' 'v1.0.0 ' '$(id)'; do + ! node_valid_version "$bad" || { echo "accepted a bad version: '${bad}'" >&2; exit 1; } +done + +reset_home +echo "[test] a bad MOSAIC_NODE_VERSION is refused by name, before any download" +set +e +OUTPUT="$( + env -u npm_config_prefix \ + HOME="$HOME_DIR" MOSAIC_HOME="$MOSAIC_HOME_DIR" MOSAIC_PREFIX="$PREFIX" \ + MOSAIC_NO_COLOR=1 MOSAIC_NODE_HOME="$NODE_HOME" \ + MOSAIC_NODE_DIST="file://${DIST}" MOSAIC_NODE_VERSION="v9.9.9/../../elsewhere" \ + MOSAIC_TEST_REAL_NODE="$REAL_NODE" MOSAIC_TEST_NPM_LOG="$LOG" \ + MOSAIC_TEST_STATE="$STATE" PATH="$NONODE_BIN" \ + bash "$ROOT/tools/install.sh" --cli --next --yes --no-auto-launch 2>&1 +)" +RC=$? +set -e +[[ "$RC" -ne 0 ]] +grep -qF -- "MOSAIC_NODE_VERSION" <<<"$OUTPUT" +grep -qF -- "Downloading Node" <<<"$OUTPUT" && { + echo "the download started despite an invalid version" >&2 + exit 1 +} +[[ ! -d "$NODE_HOME" ]] + +reset_home +echo "[test] a download location with no transport integrity is refused" +set +e +OUTPUT="$( + env -u npm_config_prefix \ + HOME="$HOME_DIR" MOSAIC_HOME="$MOSAIC_HOME_DIR" MOSAIC_PREFIX="$PREFIX" \ + MOSAIC_NO_COLOR=1 MOSAIC_NODE_HOME="$NODE_HOME" \ + MOSAIC_NODE_DIST="http://example.invalid/dist" \ + MOSAIC_TEST_REAL_NODE="$REAL_NODE" MOSAIC_TEST_NPM_LOG="$LOG" \ + MOSAIC_TEST_STATE="$STATE" PATH="$NONODE_BIN" \ + bash "$ROOT/tools/install.sh" --cli --next --yes --no-auto-launch 2>&1 +)" +RC=$? +set -e +[[ "$RC" -ne 0 ]] +grep -qF -- "MOSAIC_NODE_DIST must be" <<<"$OUTPUT" +[[ ! -d "$NODE_HOME" ]] + +reset_home +echo "[test] a path containing shell syntax is not written into a profile" +# The PATH line is executed by every future shell that reads the file, so a directory +# holding $() or a quote would run there as code. +EVIL="$TMP/ev\$(touch $TMP/pwned)il" +set +e +env -u npm_config_prefix \ + HOME="$HOME_DIR" MOSAIC_HOME="$MOSAIC_HOME_DIR" MOSAIC_PREFIX="$EVIL" \ + MOSAIC_NO_COLOR=1 MOSAIC_NODE_HOME="$NODE_HOME" \ + MOSAIC_NODE_DIST="file://${DIST}" \ + MOSAIC_TEST_REAL_NODE="$REAL_NODE" MOSAIC_TEST_NPM_LOG="$LOG" \ + MOSAIC_TEST_STATE="$STATE" PATH="$NONODE_BIN" \ + bash "$ROOT/tools/install.sh" --cli --next --yes --no-auto-launch >/dev/null 2>&1 +set -e +if [[ -f "$HOME_DIR/.profile" ]]; then + grep -qF -- 'touch' "$HOME_DIR/.profile" && { + echo "a command substitution was written into .profile" >&2 + exit 1 + } +fi +[[ ! -e "$TMP/pwned" ]] || { echo "the embedded command ran" >&2; exit 1; } + echo "[test] installer node provisioning tests passed" diff --git a/tools/install.sh b/tools/install.sh index 251cc15b..978d0943 100755 --- a/tools/install.sh +++ b/tools/install.sh @@ -159,6 +159,43 @@ fi WORK_DIR="" EXTRACTED_DIR="" +# Modification time of one file, as an integer. GNU/BusyBox stat takes -c, BSD/macOS +# stat takes -f, and there is no flag both accept -- so probe once and remember. +_MTIME_STYLE="" +file_mtime() { + if [[ -z "$_MTIME_STYLE" ]]; then + if stat -c %Y . >/dev/null 2>&1; then + _MTIME_STYLE=gnu + elif stat -f %m . >/dev/null 2>&1; then + _MTIME_STYLE=bsd + else + _MTIME_STYLE=none + fi + fi + case "$_MTIME_STYLE" in + gnu) stat -c %Y -- "$1" 2>/dev/null ;; + bsd) stat -f %m -- "$1" 2>/dev/null ;; + *) return 1 ;; + esac +} + +# The most recently modified file in "$dir" matching "$pattern". +# +# Three separate contracts, and callers must tell them apart: +# rc=0 with output — this is the newest match +# rc=0, no output — the directory or the pattern matched nothing +# rc=1 — the answer could not be determined +# +# The third one exists because the uninstall path treats "no backup" as licence to +# delete the destination. A lookup that fails must never be mistaken for a lookup +# that succeeded and found nothing. +# +# The candidates come from a glob and are compared in-shell, never rendered as text. +# That is deliberate, and it closes three bugs at once: `mapfile` is a Bash 4 builtin +# and macOS ships Bash 3.2, which this installer supports (see node_platform); piping +# `ls` into `head` dies on SIGPIPE under `set -o pipefail` once the listing fills a +# pipe buffer, returning 141 with no output; and any line-based parse of `ls` splits a +# filename that contains a newline into two wrong answers. newest_matching_file() { local dir="$1" local pattern="$2" @@ -169,16 +206,17 @@ newest_matching_file() { matches=("$dir"/$pattern) shopt -u nullglob [[ "${#matches[@]}" -gt 0 ]] || return 0 - # Read the whole listing and take the first entry, rather than piping into - # `head -1`. Under `set -o pipefail`, head closes the pipe after one line, ls - # dies on SIGPIPE, and the function returns 141 -- so on the day the directory - # holds enough files to fill a pipe buffer, finding the newest one starts - # failing the install. Process substitution has no pipeline to fail. - local -a sorted=() - # shellcheck disable=SC2012 # Need portable mtime sorting across Linux/macOS. - mapfile -t sorted < <(ls -1t "${matches[@]}" 2>/dev/null) - [[ "${#sorted[@]}" -gt 0 ]] || return 0 - printf '%s\n' "${sorted[0]}" + + local newest="" newest_t="" candidate t + for candidate in "${matches[@]}"; do + t="$(file_mtime "$candidate")" || return 1 + [[ -n "$t" ]] || return 1 + if [[ -z "$newest_t" ]] || [[ "$t" -gt "$newest_t" ]]; then + newest="$candidate" + newest_t="$t" + fi + done + printf '%s\n' "$newest" } # ─── uninstall path ─────────────────────────────────────────────────────────── @@ -241,12 +279,17 @@ if [[ "$FLAG_UNINSTALL" == "true" ]]; then for dest in "${RUNTIME_DESTS[@]}"; do base="$(basename "$dest")" dir="$(dirname "$dest")" - # Find most recent backup + # Find most recent backup. A lookup that could not answer is not the same as + # "there is no backup": removing the destination on a failed lookup would destroy + # the file the backup exists to restore. backup="" + backup_lookup_ok=true if [[ -d "$dir" ]]; then - backup="$(newest_matching_file "$dir" "${base}.mosaic-bak-*")" + backup="$(newest_matching_file "$dir" "${base}.mosaic-bak-*")" || backup_lookup_ok=false fi - if [[ -n "$backup" ]] && [[ -f "$backup" ]]; then + if [[ "$backup_lookup_ok" != "true" ]]; then + echo " Skipped: $dest (could not check for a backup; left in place)" + elif [[ -n "$backup" ]] && [[ -f "$backup" ]]; then cp "$backup" "$dest" rm -f "$backup" echo " Restored: $dest" @@ -345,6 +388,27 @@ if [[ "${MOSAIC_NO_NODE_INSTALL:-0}" == "1" ]]; then FLAG_NO_NODE_INSTALL=true fi +# A Node version string is about to become a directory name under NODE_HOME, and that +# directory is passed to `rm -rf`. Nothing reaches a filesystem operation until it has +# matched this. `v..` is the case that matters: it resolves to NODE_HOME's parent. +node_valid_version() { + [[ "$1" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] +} + +# The download location is executable code. Refuse a scheme that carries no transport +# integrity at all, and say plainly what an override does and does not buy, since the +# tarball and the checksum that vouches for it then come from the same place. +case "$NODE_DIST" in + https://*) ;; + file://*) ;; + *) + if [[ -n "${MOSAIC_NODE_DIST:-}" ]]; then + fail "MOSAIC_NODE_DIST must be an https:// or file:// URL; got '${NODE_DIST}'" + exit 1 + fi + ;; +esac + node_major_of() { # Read the major from the binary rather than parsing `node --version` text, so a # build with a suffix (v22.1.0-nightly…) does not read as a different major. @@ -376,27 +440,67 @@ node_platform() { # Newest release of the wanted major. Resolved rather than pinned so a fresh install # picks up security releases; MOSAIC_NODE_VERSION pins it when reproducibility matters. node_resolve_version() { - local want="$1" index + local want="$1" index resolved if [[ -n "${MOSAIC_NODE_VERSION:-}" ]]; then + if ! node_valid_version "$MOSAIC_NODE_VERSION"; then + fail "MOSAIC_NODE_VERSION must look like v22.11.0; got '${MOSAIC_NODE_VERSION}'" + return 1 + fi printf '%s' "$MOSAIC_NODE_VERSION" return 0 fi index="$(curl -fsSL --retry 3 "${NODE_DIST}/index.json" 2>/dev/null)" || return 1 # index.json is newest-first, so the first match is the latest of that major. # grep/sed rather than a JSON parser because node is the thing we do not have yet. - printf '%s' "$index" \ - | grep -o "\"version\":\"v${want}\.[0-9]\+\.[0-9]\+\"" \ - | head -1 \ - | sed 's/.*"v/v/; s/"$//' + # No `| head -1` here: head closes the pipe, grep takes SIGPIPE, and under + # `set -o pipefail` the whole substitution returns 141 -- the bug already fixed in + # newest_matching_file. Take the first line in the shell instead. + local found + found="$(printf '%s' "$index" | grep -o "\"version\":\"v${want}\.[0-9]\+\.[0-9]\+\"")" || return 1 + found="${found%%$'\n'*}" + resolved="${found#\"version\":\"}" + resolved="${resolved%\"}" + # The index is remote input, and what comes out of it becomes a path. + [[ -n "$resolved" ]] || return 1 + node_valid_version "$resolved" || return 1 + printf '%s' "$resolved" } node_verify_checksum() { - local dir="$1" file="$2" expected - expected="$(grep " ${file}\$" "${dir}/SHASUMS256.txt" | awk '{print $1}')" - if [[ -z "$expected" ]]; then + local dir="$1" file="$2" expected="" line name matched=0 + local manifest="${dir}/SHASUMS256.txt" + + if [[ ! -f "$manifest" ]]; then + fail "No checksum manifest was downloaded for ${file}" + return 1 + fi + + # Compare filenames exactly rather than `grep " ${file}$"`. A Node tarball name is + # mostly dots, and in a regex a dot matches any character -- so a manifest line for + # a name that merely looks like this one would be accepted as this one's checksum. + # + # Every line is read, not just the first match: two entries for the same file mean + # the manifest is not trustworthy, and picking either one is a decision this code + # has no basis to make. + while IFS= read -r line || [[ -n "$line" ]]; do + name="${line#* }" + [[ "$name" == "$file" ]] || continue + expected="${line%% *}" + matched=$(( matched + 1 )) + done < "$manifest" + + if [[ "$matched" -eq 0 ]]; then fail "No checksum published for ${file}" return 1 fi + if [[ "$matched" -gt 1 ]]; then + fail "Checksum manifest lists ${file} ${matched} times; refusing to guess." + return 1 + fi + if [[ ! "$expected" =~ ^[0-9a-fA-F]{64}$ ]]; then + fail "Checksum for ${file} is not a SHA-256 digest: '${expected}'" + return 1 + fi local actual if command -v sha256sum &>/dev/null; then actual="$(sha256sum "${dir}/${file}" | awk '{print $1}')" @@ -452,6 +556,14 @@ node_install() { local version="$1" platform="$2" local dest="${NODE_HOME}/${version}" + # Re-checked here, not only where the version was resolved: `dest` is about to be + # handed to `rm -rf`, and this is the last place before that happens. A version of + # `..` would point the removal at NODE_HOME's parent. + if ! node_valid_version "$version"; then + fail "Refusing to install Node from an unexpected version string: '${version}'" + return 1 + fi + if [[ -x "${dest}/bin/node" ]]; then info "Reusing Node ${version} already at ${dest}" return 0 @@ -464,26 +576,59 @@ node_install() { return "$rc" } -# Put a directory on PATH for future shells, once. A user-local Node and a -# user-local npm prefix are only useful if the next shell can still find them, and +# Put a directory on PATH for future processes, once. A user-local Node and a +# user-local npm prefix are only useful if the next process can still find them, and # the installer used to do no more than warn about it. # -# Both the login profile and the interactive rc get the line, because neither one -# alone covers the shells that matter. Debian's ~/.bashrc returns early when the -# shell is not interactive, so a line there is invisible to `bash -lc`, to an ssh -# command, and to a systemd unit -- which is exactly how an agent seat starts. +# There is no one file that covers this. Each target below is the only thing that +# works for some way a user -- or an agent seat -- actually starts a process: +# +# ~/.profile POSIX login shells, and `bash -lc` when no bash-specific +# profile exists. +# ~/.bash_profile A bash login shell reads the first of these that exists and +# ~/.bash_login then never reads ~/.profile. On a host with one of them, +# writing only ~/.profile is a silent no-op. Appended to when +# present, never created -- creating one would itself start +# shadowing ~/.profile for everything else the user has there. +# ~/.bashrc Interactive non-login shells. Debian's returns early when the +# shell is not interactive, so it cannot stand in for a profile. +# ~/.zshenv Every zsh invocation, including `ssh host cmd`. A remote +# non-interactive zsh reads neither ~/.zprofile nor ~/.zshrc, +# which is what the previous version of this function wrote. +# environment.d systemd --user units, which read no shell file at all. A +# Mosaic agent seat starts as a unit, so this one is the point. persist_path_line() { local dir="$1" line rc wrote="" + + # This text is written into files that a future shell will execute, so a directory + # containing shell syntax would run there as code. Refuse rather than escape: such + # a path can only arrive through MOSAIC_NODE_HOME or MOSAIC_PREFIX, and a real + # install directory never needs these characters. + if [[ "$dir" =~ [\"\$\`\\] ]] || [[ "$dir" == *"'"* ]] || [[ "$dir" == *$'\n'* ]]; then + warn "Not adding ${dir} to PATH automatically: the path contains shell syntax." + dim " Put it on PATH by hand, or reinstall to a path without those characters." + return 0 + fi + line="export PATH=\"${dir}:\$PATH\"" - local files=() + local files=("$HOME/.profile") case "$(basename "${SHELL:-/bin/bash}")" in - zsh) files=("$HOME/.zprofile" "$HOME/.zshrc") ;; - *) files=("$HOME/.profile" "$HOME/.bashrc") ;; + zsh) + files+=("$HOME/.zshenv") + ;; + *) + files+=("$HOME/.bashrc") + if [[ -f "$HOME/.bash_profile" ]]; then files+=("$HOME/.bash_profile"); fi + if [[ -f "$HOME/.bash_login" ]]; then files+=("$HOME/.bash_login"); fi + ;; esac for rc in "${files[@]}"; do - if [[ -f "$rc" ]] && grep -Fq "$line" "$rc"; then + # -x anchors the match to a whole line. Without it, a commented-out example of + # this same export counts as already present and the real entry never gets + # written -- the failure then looks like the installer simply did nothing. + if [[ -f "$rc" ]] && grep -Fqx "$line" "$rc"; then continue fi { @@ -493,9 +638,21 @@ persist_path_line() { wrote+="${wrote:+, }${rc}" done + # systemd --user units inherit from the user manager, not from any shell. + local envd="$HOME/.config/environment.d" + local envd_file="$envd/50-mosaic-path.conf" + local envd_line="PATH=${dir}:\${PATH}" + if mkdir -p "$envd" 2>/dev/null; then + if [[ ! -f "$envd_file" ]] || ! grep -Fqx "$envd_line" "$envd_file"; then + printf '%s\n' "$envd_line" >> "$envd_file" + wrote+="${wrote:+, }${envd_file}" + fi + fi + if [[ -n "$wrote" ]]; then ok "Added ${dir} to PATH in ${wrote}" - dim " This shell: export PATH=\"${dir}:\$PATH\"" + dim " This shell: export PATH=\"${dir}:\$PATH\"" + dim " systemd --user: systemctl --user daemon-reload (or log in again)" fi } @@ -725,8 +882,10 @@ install_cli_from_source() { ( cd "$src/apps/gateway" && pnpm pack --pack-destination "$out_dir" ) 2>&1 | sed 's/^/ /' local cli_tgz gw_tgz - cli_tgz="$(newest_matching_file "$out_dir" 'mosaicstack-mosaic-*.tgz')" - gw_tgz="$(newest_matching_file "$out_dir" 'mosaicstack-gateway-*.tgz')" + # An unanswerable lookup becomes an empty path, which the -f guards below report + # properly. Nothing destructive happens on this path, so failing soft is safe here. + cli_tgz="$(newest_matching_file "$out_dir" 'mosaicstack-mosaic-*.tgz')" || cli_tgz="" + gw_tgz="$(newest_matching_file "$out_dir" 'mosaicstack-gateway-*.tgz')" || gw_tgz="" if [[ ! -f "$cli_tgz" ]]; then fail "CLI tarball was not produced by pnpm pack." @@ -1082,7 +1241,13 @@ if [[ "$FLAG_CHECK" == "false" ]]; then local base dir backup_path backup_val base="$(basename "$dest")" dir="$(dirname "$dest")" - backup_path="$(newest_matching_file "$dir" "${base}.mosaic-bak-*")" + # Recording null here would tell a later uninstall that no backup exists, and + # it would then delete the destination instead of restoring it. An unanswerable + # lookup must stop the manifest, not guess at it. + if ! backup_path="$(newest_matching_file "$dir" "${base}.mosaic-bak-*")"; then + fail "Could not determine the backup state of ${dest}; refusing to write a manifest." + return 1 + fi if [[ -n "$backup_path" ]]; then backup_val="\"$backup_path\"" else