fix(tests): clear NODE_TEST_CONTEXT for nested node --test in two suites (#1508)
test-foundation.sh and test-discord.sh ran a nested node --test that would exit 0 on failure under a parent runner. Both clear the variable now, and each has a check that fails the suite if it comes back. Darkwing wrote it, Filbert approved it (e464be6c). Nine suites green on an index export. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Claude Opus 5.5
Executable
+39
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env bash
|
||||
# N13 check (#1508): a failing nested `node --test` must fail test-foundation.sh
|
||||
# and test-discord.sh even when a parent runner's NODE_TEST_CONTEXT is set.
|
||||
# Runs in a scratch clone only: n13-check.sh CLONE (a clone of HEAD with
|
||||
# node_modules linked). It copies this checkout's two suites into the clone,
|
||||
# plants a failing test in each suite's test directory, and restores after.
|
||||
set -uo pipefail
|
||||
SRC="$(cd "$(dirname "$0")/../../../.." && pwd)"
|
||||
CLONE="${1:?usage: n13-check.sh CLONE}"
|
||||
[ "$(cd "$CLONE" && pwd)" != "$SRC" ] || { echo "refusing to run in the canonical checkout" >&2; exit 4; }
|
||||
cd "$CLONE" || exit 1
|
||||
|
||||
PLANT='import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
test("N13 planted failure", () => assert.equal(1, 2));'
|
||||
FAILS=0
|
||||
expect() { # NAME WANT GOT
|
||||
if [ "$2" = "$3" ]; then echo "ok $1 (rc $3)"; else echo "FAIL $1 (want rc $2, got $3)"; FAILS=$((FAILS+1)); fi
|
||||
}
|
||||
run() { # SUITE -> rc, output in /tmp/n13-<suite>-<tag>.txt
|
||||
NODE_TEST_CONTEXT=child-v8 NO_COLOR=1 "scripts/test-$1.sh" >"/tmp/n13-$1-$2.txt" 2>&1
|
||||
}
|
||||
|
||||
for pair in foundation:scripts/foundation discord:packages/discord/tests; do
|
||||
suite=${pair%%:*} dir=${pair#*:}
|
||||
git checkout -q -- "scripts/test-$suite.sh"
|
||||
printf '%s\n' "$PLANT" >"$dir/zz-n13-planted.test.mjs"
|
||||
run "$suite" head-planted; expect "$suite at HEAD, planted failure, parent context set" 0 $?
|
||||
cp -p "$SRC/scripts/test-$suite.sh" "scripts/test-$suite.sh"
|
||||
run "$suite" fixed-planted; expect "$suite fixed, planted failure, parent context set" 1 $?
|
||||
rm -f "$dir/zz-n13-planted.test.mjs"
|
||||
run "$suite" fixed-clean; expect "$suite fixed, no planted failure, parent context set" 0 $?
|
||||
sed -i 's/node_tests() { env -u NODE_TEST_CONTEXT node --test/node_tests() { node --test/' "scripts/test-$suite.sh"
|
||||
run "$suite" mutant-clean; expect "$suite with the clearing removed, no planted failure" 1 $?
|
||||
git checkout -q -- "scripts/test-$suite.sh"
|
||||
done
|
||||
git status --short
|
||||
echo "n13 check: $FAILS failed"
|
||||
[ "$FAILS" -eq 0 ]
|
||||
@@ -0,0 +1,59 @@
|
||||
# N13: nested `node --test` in two suites (#1508)
|
||||
|
||||
Darkwing, 2026-09-26. Filbert's note N13, put in DEFERRED by Sage as a
|
||||
small reviewed item before A2. Nothing is committed, staged or pushed.
|
||||
|
||||
## The defect
|
||||
|
||||
`scripts/test-foundation.sh:76` and `scripts/test-discord.sh:142` start a
|
||||
nested `node --test` without clearing `NODE_TEST_CONTEXT`. Under a parent
|
||||
test runner the nested run reports to that runner and exits 0 whatever its
|
||||
tests do. I reproduced it at HEAD 40a02d2b: with a failing test planted in
|
||||
each suite's test directory and `NODE_TEST_CONTEXT=child-v8` set, both
|
||||
suites exit 0. The check line reads `OK node --test ... (summary
|
||||
missing)`. In a control run of foundation without the variable, the same
|
||||
planted test fails the suite, so only a run under a parent runner is
|
||||
blind. I ran that control for foundation only.
|
||||
|
||||
## The change
|
||||
|
||||
`n13.patch` (sha256 00868b2f) changes only the two suites, +20 −2 lines.
|
||||
Each suite now has a `node_tests` function that runs `env -u
|
||||
NODE_TEST_CONTEXT node --test`, and uses it for its test run. Each also
|
||||
gets one new check. It writes a failing test into the sandbox, runs it
|
||||
through `node_tests` with `NODE_TEST_CONTEXT=child-v8` set, and requires
|
||||
exit 1 and `✖ planted failure` in the output. If someone drops the `env
|
||||
-u`, that check fails on every run, not only under a parent runner.
|
||||
|
||||
| File | sha256 |
|
||||
|---|---|
|
||||
| `scripts/test-foundation.sh` | 60f04822 |
|
||||
| `scripts/test-discord.sh` | 2ad3be74 |
|
||||
| `n13-check.sh` | 6a231759 |
|
||||
|
||||
## The check
|
||||
|
||||
`n13-check.sh CLONE` runs in a scratch clone only and refuses the
|
||||
canonical checkout. For each suite it plants a failing test in the real
|
||||
test directory and runs the whole suite with `NODE_TEST_CONTEXT=child-v8`:
|
||||
|
||||
| Suite | Case | Exit |
|
||||
|---|---|---|
|
||||
| foundation | HEAD, planted failure | 0 (the defect) |
|
||||
| foundation | fixed, planted failure | 1 |
|
||||
| foundation | fixed, no planted failure | 0 |
|
||||
| foundation | fixed but `env -u` removed, no planted failure | 1 |
|
||||
| discord | HEAD, planted failure | 0 (the defect) |
|
||||
| discord | fixed, planted failure | 1 |
|
||||
| discord | fixed, no planted failure | 0 |
|
||||
| discord | fixed but `env -u` removed, no planted failure | 1 |
|
||||
|
||||
In the fixed runs with the planted failure, the suite prints `FAIL node
|
||||
--test ...` with the pass count and the planted test's name. The last row
|
||||
of each is the mutation: the new check alone fails the suite.
|
||||
|
||||
All nine suites pass at 40a02d2b with this change and the queue A1
|
||||
candidate: foundation 44 and discord 64, one more check each than before.
|
||||
|
||||
I found no other nested `node --test` in the suites. `test-queue.sh` and
|
||||
`queue-commit.sh` already clear the variable.
|
||||
@@ -0,0 +1,44 @@
|
||||
diff --git a/scripts/test-discord.sh b/scripts/test-discord.sh
|
||||
index 6044100c..9ec4ddda 100755
|
||||
--- a/scripts/test-discord.sh
|
||||
+++ b/scripts/test-discord.sh
|
||||
@@ -139,7 +139,16 @@ else
|
||||
fi
|
||||
|
||||
# --- the seven offline groups ---
|
||||
-node --test --test-reporter=spec packages/discord/tests/ >"$SANDBOX/node-test.log" 2>&1
|
||||
+# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
|
||||
+# to that runner and exits 0 whatever its tests do, so the suite clears it
|
||||
+# (#1508 N13). The planted failing test proves a failure still fails here.
|
||||
+node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
|
||||
+mkdir -p "$SANDBOX/planted"
|
||||
+printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
|
||||
+NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
|
||||
+[ $? -eq 1 ] && grep -q '^✖ planted failure' "$SANDBOX/planted.log"
|
||||
+check "a failing nested test fails the run under a parent runner's NODE_TEST_CONTEXT" $?
|
||||
+node_tests --test-reporter=spec packages/discord/tests/ >"$SANDBOX/node-test.log" 2>&1
|
||||
NODE_RC=$?
|
||||
check "node --test packages/discord/tests/ ($(grep -E '^ℹ pass' "$SANDBOX/node-test.log" | tr -d '\n' || echo 'summary missing'))" $NODE_RC
|
||||
if [ "$NODE_RC" -ne 0 ]; then
|
||||
diff --git a/scripts/test-foundation.sh b/scripts/test-foundation.sh
|
||||
index 251eb674..b94dab5c 100755
|
||||
--- a/scripts/test-foundation.sh
|
||||
+++ b/scripts/test-foundation.sh
|
||||
@@ -73,7 +73,16 @@ done
|
||||
check "checked-in demo bundles equal a fresh generation" $DEMO_OK
|
||||
|
||||
# --- unit, CLI, privacy, non-effect and fixture-index tests ---
|
||||
-node --test scripts/foundation/ >"$SANDBOX/node-test.log" 2>&1
|
||||
+# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
|
||||
+# to that runner and exits 0 whatever its tests do, so the suite clears it
|
||||
+# (#1508 N13). The planted failing test proves a failure still fails here.
|
||||
+node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
|
||||
+mkdir -p "$SANDBOX/planted"
|
||||
+printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
|
||||
+NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
|
||||
+[ $? -eq 1 ] && grep -q '^✖ planted failure' "$SANDBOX/planted.log"
|
||||
+check "a failing nested test fails the run under a parent runner's NODE_TEST_CONTEXT" $?
|
||||
+node_tests scripts/foundation/ >"$SANDBOX/node-test.log" 2>&1
|
||||
NODE_RC=$?
|
||||
check "node --test scripts/foundation/ ($(grep -E '^ℹ pass' "$SANDBOX/node-test.log" | tr -d '\n' || echo 'summary missing'))" $NODE_RC
|
||||
[ "$NODE_RC" -ne 0 ] && grep -E "^✖|AssertionError" "$SANDBOX/node-test.log" | head -20
|
||||
Reference in New Issue
Block a user