feat(agent): interactive TUI launcher + identity + TOOLS.md (#35)
- scripts/agent.sh <name>: launches interactive pi TUI in the container with contracts + optional mission + agent identity + named session + optional workspace/tools; the Mosaic alternative to vanilla pi - pi adapter: MOSAIC_INTERACTIVE branch (clean TUI, no -p, no initial prompt); headless exec rebuilt via positional args (no word-splitting on the request); MOSAIC_AGENT_NAME optional in headless - loader: AGENT IDENTITY section when the launcher names the agent - compose: fixed command removed (request defaults live in run-agent.sh); MOSAIC_INTERACTIVE/MOSAIC_AGENT_NAME passthrough - docs/TOOLS.md: full on-demand tool reference; AGENTS.md routes to it - RELEASE -> 0.0.8 (container change); build verified Closes #35
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
# unslop-hook
|
||||
|
||||
Mechanical AI-tell enforcement for pi seats. Anti-drift gate for the writing
|
||||
standard in SYSTEM.md / ms-unslop: prose distribution alone decays over long
|
||||
sessions; this check cannot forget.
|
||||
|
||||
- `lists.json`: committed machine source for every list the checker enforces:
|
||||
words, phrases, punct rules, regex patterns, density thresholds. Each entry
|
||||
carries provenance (`ms-unslop:<pattern id>` or `system-md`), the mention
|
||||
convention, and the documented divergence of the density gate from
|
||||
SYSTEM.md's outright em-dash ban. Edit lists here, not in code.
|
||||
- `unslop-check.js`: dependency-free checker (node CLI + module) driven by
|
||||
lists.json. Loads and schema-validates the lists on first use and hard-fails
|
||||
closed: empty, unparseable, or invalid lists throw. Detects banned vocabulary,
|
||||
chatbot/sycophancy phrases, filler phrases, em/en dashes, curly quotes,
|
||||
`not just X but Y`. Strips fenced and inline code first, so quoted code is
|
||||
never flagged. Exit 0 clean, 1 violations, 2 gate broken (lists unreadable,
|
||||
never a clean verdict).
|
||||
- `extension.ts`: pi extension. `message_end` checks finalized assistant text
|
||||
and notifies the operator (TUI/RPC). `before_agent_start` reads the most
|
||||
recent assistant reply from the session file and, if it carries tells,
|
||||
injects a correction notice the model sees on its next turn. `/unslop`
|
||||
reports session stats. Violation state lives in the session file, so the
|
||||
injection path survives restart, resume, fork, and reload (an in-memory
|
||||
pending flag was measured dead across print-mode turns, 2026-08-19). A
|
||||
broken lists.json fails closed: checks stop, `broken_lists` /
|
||||
`skipped_broken` events log the reason, operator notified once, seat keeps
|
||||
running.
|
||||
- `test-unslop-check.js`: unit tests with red and green controls.
|
||||
|
||||
## Use
|
||||
|
||||
```bash
|
||||
node test-unslop-check.js # suite
|
||||
node unslop-check.js <file> # CLI check
|
||||
UNSLOP_LISTS=<path> node unslop-check.js <file> # alt lists location
|
||||
pi -e ~/.mosaic/tools/unslop-hook/extension.ts # ad-hoc load
|
||||
# deploy: copy dir to ~/.pi/agent/extensions/unslop-hook/ or seat .pi
|
||||
# equivalent, or list it in settings.json "extensions"
|
||||
```
|
||||
|
||||
Env: `MOSAIC_UNSLOP_HOOK=0` disables. `MOSAIC_UNSLOP_LOG=<path>` appends JSONL
|
||||
events (loaded / flagged / notice_injected / checked / broken_lists /
|
||||
skipped_broken) for headless evidence. `UNSLOP_LISTS=<path>` overrides the
|
||||
lists.json location for both CLI and extension.
|
||||
|
||||
## Verified here (2026-08-19)
|
||||
|
||||
- Unit suite 24/24 (8 behavioral, 11 loader/CLI, 5 review follow-up), red and
|
||||
green controls both exercised, including exit-2 on broken lists and on an
|
||||
unreadable input file (S3).
|
||||
- CLI: slop file exit 1, clean file exit 0, broken lists exit 2 with the fault
|
||||
named on stderr.
|
||||
- Extension, healthy path (print mode, zai/glm-5.3:low): startup probe loads
|
||||
lists.json, reply checked clean.
|
||||
- Extension, broken-lists path (print mode): `broken_lists` at startup,
|
||||
`skipped_broken` per turn, seat survives, reply still delivered.
|
||||
- Earlier live evidence (pre-C1, inline lists): forced-slop turn flagged;
|
||||
fresh-process follow-up injected the notice and the reply came back clean;
|
||||
full TUI trial (notify line, injection, /unslop stats) on session vision-unslop.
|
||||
- Log evidence in session scratchpad.
|
||||
|
||||
## Limits
|
||||
|
||||
- `/unslop` command not tested headless (print mode has no command surface);
|
||||
it is a thin stats wrapper.
|
||||
- En dash flag fires on typographic ranges too (2–3); acceptable for fleet
|
||||
prose, revisit if it noisifies technical writing.
|
||||
- Notice injection is a nudger, not a blocker. Output already streamed to the
|
||||
user stays as-is; correction lands on the next turn.
|
||||
- A broken lists.json latches for the session: repairing the file mid-session
|
||||
does not revive checks until the seat restarts. Acceptable for an advisory
|
||||
gate (review S1).
|
||||
- The fail-closed operator notification requires a UI. Print-mode sessions
|
||||
log `skipped_broken` but notify nobody (review S2).
|
||||
- The word/phrase lists are the mechanical subset of ms-unslop only, keyed to
|
||||
pattern ids in lists.json. Style judgments (voice, rhythm, structure) stay in
|
||||
the skill, not the gate.
|
||||
|
||||
## Promotion path
|
||||
|
||||
Stack issue (A4): checker shared as the single source for a matching Claude
|
||||
Code Stop-hook script; lists versioned beside SYSTEM.md contract text.
|
||||
@@ -0,0 +1,163 @@
|
||||
// unslop-hook — pi extension wrapper around unslop-check.js.
|
||||
// Detects mechanical AI tells in finalized assistant messages and injects a
|
||||
// correction notice the model sees on its next turn. Anti-drift enforcement for
|
||||
// SYSTEM.md / ms-unslop; prose distribution alone decays, this cannot forget.
|
||||
//
|
||||
// Deploy: copy dir to ~/.pi/agent/extensions/unslop-hook/ (or seat .pi equivalent),
|
||||
// or add this file's dir to settings.json "extensions".
|
||||
// Test: pi -e <abs path>/extension.ts
|
||||
// Off: MOSAIC_UNSLOP_HOOK=0
|
||||
// Log: MOSAIC_UNSLOP_LOG=/path/to/log.jsonl (JSONL events; headless evidence)
|
||||
// Broken: lists.json missing/empty/invalid → the checker throws; checks are
|
||||
// skipped, logged as skipped_broken, and the operator is notified once.
|
||||
// Never silently pass while the lists cannot load (fail closed).
|
||||
//
|
||||
// Design note: violation state lives in the SESSION FILE, not memory. At
|
||||
// before_agent_start we read the most recent assistant text message from
|
||||
// ctx.sessionManager and check it there. That survives process restarts, resume,
|
||||
// fork, and reload — an in-memory pending flag measured dead on 2026-08-19 when
|
||||
// a print-mode second turn never injected.
|
||||
import { appendFileSync } from "node:fs";
|
||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||
import { checkText } from "./unslop-check.js";
|
||||
|
||||
interface Finding {
|
||||
rule: string;
|
||||
detail: string;
|
||||
count: number;
|
||||
}
|
||||
|
||||
interface MessageEntry {
|
||||
type: "message";
|
||||
id: string;
|
||||
message: { role?: string; content?: unknown };
|
||||
}
|
||||
|
||||
function assistantText(entry: unknown): string | null {
|
||||
const e = entry as Partial<MessageEntry>;
|
||||
if (e?.type !== "message") return null;
|
||||
const msg = e.message;
|
||||
if (msg?.role !== "assistant" || !Array.isArray(msg.content)) return null;
|
||||
const text = msg.content
|
||||
.filter((b): b is { type: "text"; text: string } =>
|
||||
typeof b === "object" && b !== null && (b as { type?: string }).type === "text")
|
||||
.map((b) => b.text ?? "")
|
||||
.join("\n");
|
||||
return text.trim() ? text : null; // tool-call-only assistant messages return null
|
||||
}
|
||||
|
||||
export default function (pi: ExtensionAPI) {
|
||||
if (process.env.MOSAIC_UNSLOP_HOOK === "0") return;
|
||||
|
||||
const LOG = process.env.MOSAIC_UNSLOP_LOG;
|
||||
const log = (ev: Record<string, unknown>) => {
|
||||
if (LOG) appendFileSync(LOG, JSON.stringify({ ts: Date.now(), ...ev }) + "\n");
|
||||
};
|
||||
|
||||
// Entry ids we have already injected a notice for. In-memory only: after a
|
||||
// restart the same entry may inject once more, which re-anchors the style
|
||||
// after a context loss. That is wanted, not a bug.
|
||||
const injectedFor = new Set<string>();
|
||||
let turnsChecked = 0;
|
||||
let turnsFlagged = 0;
|
||||
const histogram = new Map<string, number>();
|
||||
|
||||
// Fail-closed path for a broken lists.json. A checker that cannot load its
|
||||
// lists must never be read as "everything passed": checks stop, the skip is
|
||||
// logged each turn, and the operator is notified once.
|
||||
let broken: string | null = null;
|
||||
let brokenNotified = false;
|
||||
const reportBroken = (ctx: { hasUI?: boolean } | undefined, where: string) => {
|
||||
log({ ev: "skipped_broken", where, reason: broken });
|
||||
if (!brokenNotified && ctx?.hasUI) {
|
||||
ctx.ui.notify(`unslop gate BROKEN: ${broken}. Fix tools/unslop-hook/lists.json; no clean verdicts until then.`, "error");
|
||||
brokenNotified = true;
|
||||
}
|
||||
};
|
||||
const safeCheck = (text: string): ReturnType<typeof checkText> | null => {
|
||||
if (broken) return null;
|
||||
try {
|
||||
return checkText(text);
|
||||
} catch (e) {
|
||||
broken = String((e as Error).message);
|
||||
log({ ev: "broken_lists", reason: broken });
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
pi.on("session_start", async (event, _ctx) => {
|
||||
log({ ev: "loaded", reason: event.reason });
|
||||
try {
|
||||
checkText(""); // probe: load+validate lists at startup, not mid-conversation
|
||||
} catch (e) {
|
||||
broken = String((e as Error).message);
|
||||
log({ ev: "broken_lists", reason: broken, at: "startup" });
|
||||
}
|
||||
});
|
||||
|
||||
pi.on("message_end", async (event, ctx) => {
|
||||
if ((event.message as { role?: string }).role !== "assistant") return;
|
||||
const text = assistantText({ type: "message", id: "", message: event.message });
|
||||
if (text === null) return;
|
||||
|
||||
const result = safeCheck(text);
|
||||
if (result === null) {
|
||||
reportBroken(ctx, "message_end");
|
||||
return;
|
||||
}
|
||||
turnsChecked++;
|
||||
if (result.clean) {
|
||||
log({ ev: "checked", clean: true, turn: turnsChecked, charsChecked: result.charsChecked });
|
||||
return;
|
||||
}
|
||||
turnsFlagged++;
|
||||
for (const f of result.findings) histogram.set(f.rule, (histogram.get(f.rule) ?? 0) + 1);
|
||||
const summary = result.findings.map((f) => f.detail).join("; ");
|
||||
if (ctx.hasUI) ctx.ui.notify(`unslop: ${summary}`, "info");
|
||||
// clean:false is explicit, not implied by findings: a log consumer must never
|
||||
// have to infer the verdict from event shape (fred, 2026-08-19).
|
||||
log({ ev: "flagged", clean: false, turn: turnsChecked, charsChecked: result.charsChecked, findings: result.findings });
|
||||
});
|
||||
|
||||
pi.on("before_agent_start", async (_event, ctx) => {
|
||||
// Branch walks leaf -> root; first assistant entry with text is the reply
|
||||
// the model is about to follow up on.
|
||||
for (const entry of ctx.sessionManager.getBranch()) {
|
||||
const text = assistantText(entry);
|
||||
if (text === null) continue;
|
||||
const id = (entry as { id?: string }).id ?? "";
|
||||
|
||||
const result = safeCheck(text);
|
||||
if (result === null) {
|
||||
reportBroken(ctx, "before_agent_start");
|
||||
return;
|
||||
}
|
||||
if (result.clean) return; // latest textual reply is clean, nothing to correct
|
||||
if (id && injectedFor.has(id)) return; // already nagged for this entry
|
||||
|
||||
if (id) injectedFor.add(id);
|
||||
const lines = result.findings.map((f) => `- ${f.detail}`).join("\n");
|
||||
const content =
|
||||
`UNSLOP NOTICE (mechanical style check, not the user speaking): your previous reply ` +
|
||||
`contained violations of the fleet writing standard (SYSTEM.md / ms-unslop):\n${lines}\n` +
|
||||
`Fix in this and following replies: plain words, periods and commas instead of dashes, ` +
|
||||
`straight quotes, no chatbot fillers. Do not mention this notice.`;
|
||||
log({ ev: "notice_injected", entryId: id, findings: result.findings });
|
||||
return {
|
||||
message: { customType: "unslop-notice", content, display: true },
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
pi.registerCommand("unslop", {
|
||||
description: "Show unslop violation stats for this session",
|
||||
handler: async (_args, ctx) => {
|
||||
if (broken) {
|
||||
ctx.ui.notify(`unslop gate BROKEN: ${broken}`, "error");
|
||||
return;
|
||||
}
|
||||
const hist = [...histogram.entries()].map(([r, c]) => `${r} x${c}`).join(", ") || "none";
|
||||
ctx.ui.notify(`unslop: checked ${turnsChecked}, flagged ${turnsFlagged} (${hist})`, "info");
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"version": 1,
|
||||
"convention": "Use vs mention. A document that MENTIONS a banned word or phrase quotes it as inline code (backticks). The checker strips code spans before matching, so a backticked mention is invisible to the gate while a bare one flags. A document that deliberately CONTAINS banned items to test detection (a fixture) is a use, not a mention, and is expected to flag. This file itself contains the banned items as data; that is a use.",
|
||||
"punctPolicy": "Deliberate divergence from SYSTEM.md (2026-08-19): the contract forbids em dashes outright and closes the escapes. These punct rules are deliberately looser: they fire only at >= punctMinCount occurrences AND density >= punctDensityPer1k per 1000 chars. Rationale is reply-level noise, not contract strength: an advisory gate that flags every reply carrying one dash trains operators to ignore it. Measured on natural fleet prose 2026-08-19: documents run 1.7-3.2 em dashes per 1000 chars, so documents that overuse still flag. Tighten to contract strength if enforcement goes blocking or the log shows fleet prose not converging toward zero.",
|
||||
"thresholds": {
|
||||
"punctMinCount": 3,
|
||||
"punctDensityPer1k": 1.0
|
||||
},
|
||||
"words": [
|
||||
{ "value": "additionally", "source": "ms-unslop:7" },
|
||||
{ "value": "crucial", "source": "ms-unslop:7" },
|
||||
{ "value": "delve", "source": "ms-unslop:7" },
|
||||
{ "value": "garner", "source": "ms-unslop:7" },
|
||||
{ "value": "interplay", "source": "ms-unslop:7" },
|
||||
{ "value": "intricate", "source": "ms-unslop:7" },
|
||||
{ "value": "pivotal", "source": "ms-unslop:7" },
|
||||
{ "value": "showcase", "source": "ms-unslop:7" },
|
||||
{ "value": "tapestry", "source": "ms-unslop:7" },
|
||||
{ "value": "testament", "source": "ms-unslop:7" },
|
||||
{ "value": "underscore", "source": "ms-unslop:7" },
|
||||
{ "value": "vibrant", "source": "ms-unslop:7" },
|
||||
{ "value": "utilize", "source": "ms-unslop:31" },
|
||||
{ "value": "leverage", "source": "ms-unslop:31" },
|
||||
{ "value": "facilitate", "source": "ms-unslop:31" },
|
||||
{ "value": "load-bearing", "source": "system-md" }
|
||||
],
|
||||
"phrases": [
|
||||
{ "value": "worth stating plainly", "source": "system-md" },
|
||||
{ "value": "here's the honest truth", "source": "system-md" },
|
||||
{ "value": "heres the honest truth", "source": "system-md", "note": "apostrophe-OMITTED renderings only; ASCII and curly-apostrophe forms match the main entry because the checker normalizes U+2019/U+2018 to ASCII before phrase matching" },
|
||||
{ "value": "the real tension", "source": "system-md" },
|
||||
{ "value": "carry the argument", "source": "system-md" },
|
||||
{ "value": "in order to", "source": "ms-unslop:23" },
|
||||
{ "value": "due to the fact that", "source": "ms-unslop:23" },
|
||||
{ "value": "it is important to note", "source": "ms-unslop:23" },
|
||||
{ "value": "i hope this helps", "source": "ms-unslop:20" },
|
||||
{ "value": "let me know if", "source": "ms-unslop:20" },
|
||||
{ "value": "of course!", "source": "ms-unslop:20" },
|
||||
{ "value": "certainly!", "source": "ms-unslop:20" },
|
||||
{ "value": "found the smoking gun", "source": "ms-unslop:20" },
|
||||
{ "value": "happy to help", "source": "ms-unslop:20", "note": "extension of the named pattern set" },
|
||||
{ "value": "great question", "source": "ms-unslop:22" },
|
||||
{ "value": "absolutely right", "source": "ms-unslop:22" },
|
||||
{ "value": "excellent question", "source": "ms-unslop:22", "note": "extension of the named pattern set" }
|
||||
],
|
||||
"punct": [
|
||||
{ "value": "em", "label": "em dash", "chars": ["\u2014"], "source": "ms-unslop:13+system-md" },
|
||||
{ "value": "en", "label": "en dash", "chars": ["\u2013"], "source": "ms-unslop:13" },
|
||||
{ "value": "curly", "label": "curly quote/apostrophe", "chars": ["\u201c", "\u201d", "\u2018", "\u2019"], "source": "ms-unslop:19" }
|
||||
],
|
||||
"patterns": [
|
||||
{ "value": "not-just-but", "regex": "not just\\s+[^.!?]{0,80}?\\s+but", "flags": "gi", "detail": "not just X but Y", "source": "ms-unslop:9" }
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,228 @@
|
||||
"use strict";
|
||||
// Tests for unslop-check.js. Run: node test-unslop-check.js
|
||||
// Exit 0 = all pass. Cases include a red control (slop must fail) and a green
|
||||
// control (clean prose must pass) per evidence discipline.
|
||||
|
||||
const assert = require("node:assert");
|
||||
const fs = require("node:fs");
|
||||
const os = require("node:os");
|
||||
const path = require("node:path");
|
||||
const { spawnSync } = require("node:child_process");
|
||||
const { checkText, stripCode, loadLists } = require("./unslop-check.js");
|
||||
|
||||
const SLOP = `Certainly! Let me delve into the evolving tapestry of database technology — it’s truly “pivotal” — and intricate.
|
||||
In order to understand it — we should leverage this interplay of systems — deeply. I hope this helps!`;
|
||||
|
||||
const CLEAN = `The loader parses the file and validates each row. Rows that fail are logged
|
||||
and skipped. We measured a range from 1 to 10 seconds. Use "straight quotes" and
|
||||
commas, not dashes. That is the whole finding.`;
|
||||
|
||||
// Code-stripping control: banned words inside code must not count.
|
||||
const WITH_CODE = [
|
||||
"The config uses `utilize=false` internally.",
|
||||
"```",
|
||||
"delve tapestry — pivotal",
|
||||
"```",
|
||||
"The config file sets one flag. It is parsed at startup.",
|
||||
].join("\n");
|
||||
|
||||
const results = [];
|
||||
function t(name, fn) {
|
||||
try { fn(); results.push([name, true]); } catch (e) { results.push([name, false]); console.error(`FAIL ${name}: ${e.message}`); }
|
||||
}
|
||||
|
||||
t("slop fixture is flagged (red control)", () => {
|
||||
const r = checkText(SLOP);
|
||||
assert.ok(!r.clean, "slop must not be clean");
|
||||
const details = r.findings.map((f) => f.detail).join("; ");
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("delve")), `delve missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("tapestry")), `tapestry missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("pivotal")), `pivotal missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("em dash")), `em dash missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("curly")), `curly missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("in order to")), `in order to missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("i hope this helps")), `chatbot phrase missing: ${details}`);
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("certainly")), `certainly missing: ${details}`);
|
||||
});
|
||||
|
||||
t("clean fixture passes (green control)", () => {
|
||||
const r = checkText(CLEAN);
|
||||
assert.deepStrictEqual(r.findings, [], `unexpected findings: ${JSON.stringify(r.findings)}`);
|
||||
});
|
||||
|
||||
t("numeric range is not a false range flag", () => {
|
||||
const r = checkText(CLEAN);
|
||||
assert.ok(!r.findings.some((f) => f.rule === "pattern"), "must not flag numeric ranges");
|
||||
});
|
||||
|
||||
t("code blocks and inline code are stripped", () => {
|
||||
const r = checkText(WITH_CODE);
|
||||
assert.deepStrictEqual(r.findings, [], `code leaked into check: ${JSON.stringify(r.findings)}`);
|
||||
});
|
||||
|
||||
t("stripCode removes fenced and inline code", () => {
|
||||
const s = stripCode("a `x — y` b\n```\ndelve\n```\nc");
|
||||
assert.ok(!s.includes("delve"), "fenced code not stripped");
|
||||
assert.ok(!s.includes("—"), "inline code not stripped");
|
||||
assert.ok(s.includes("a") && s.includes("b") && s.includes("c"), "prose lost");
|
||||
});
|
||||
|
||||
t("not-just-but pattern is detected", () => {
|
||||
const r = checkText("This is not just a cache but a coordination layer.");
|
||||
assert.ok(r.findings.some((f) => f.rule === "pattern"), "pattern missed");
|
||||
});
|
||||
|
||||
t("light dash use is not flagged (below threshold)", () => {
|
||||
const prose =
|
||||
"The loader parses each row and validates it against the schema. Rows that fail " +
|
||||
"are logged — with their line numbers — and skipped. The operator reviews the log " +
|
||||
"daily and reconciles the rejects against the source system by hand, which takes " +
|
||||
"a few minutes and has never once produced a discrepancy worth acting on.";
|
||||
const r = checkText(prose);
|
||||
assert.ok(!r.findings.some((f) => f.detail.includes("dash")), "2 dashes in ~330 chars must not flag");
|
||||
});
|
||||
|
||||
t("dash overuse is flagged (above threshold)", () => {
|
||||
const r = checkText("One — two — three — four. That is the whole sentence.");
|
||||
assert.ok(r.findings.some((f) => f.detail.includes("em dash")), "4 dashes in 50 chars must flag");
|
||||
});
|
||||
|
||||
// ── C1: lists.json machine source ──────────────────────────────────────
|
||||
// The committed lists are the single source of truth; these tests pin the
|
||||
// file's validity, its shape, and the loader's fail-closed behavior.
|
||||
|
||||
const tmpdir = fs.mkdtempSync(path.join(os.tmpdir(), "unslop-c1-"));
|
||||
const tmp = (n) => path.join(tmpdir, n);
|
||||
|
||||
function brokenVariant(mutate) {
|
||||
const l = JSON.parse(JSON.stringify(loadLists()));
|
||||
mutate(l);
|
||||
return l;
|
||||
}
|
||||
|
||||
function writeTmp(name, data) {
|
||||
const f = tmp(name);
|
||||
fs.writeFileSync(f, typeof data === "string" ? data : JSON.stringify(data));
|
||||
return f;
|
||||
}
|
||||
|
||||
t("lists.json (the real file) validates and is pinned in size", () => {
|
||||
const l = loadLists();
|
||||
assert.strictEqual(l.version, 1);
|
||||
// Counts pin the migration: 16 words, 17 phrases, 3 punct, 1 pattern moved
|
||||
// from the old inline constants. Changing a count means changing this test
|
||||
// too, consciously.
|
||||
assert.strictEqual(l.words.length, 16, "word count drifted");
|
||||
assert.strictEqual(l.phrases.length, 17, "phrase count drifted");
|
||||
assert.strictEqual(l.punct.length, 3, "punct count drifted");
|
||||
assert.strictEqual(l.patterns.length, 1, "pattern count drifted");
|
||||
assert.ok(l.convention.length > 50, "mention convention must be present");
|
||||
assert.ok(l.punctPolicy.length > 50, "punct divergence policy must be present");
|
||||
for (const e of [...l.words, ...l.phrases, ...l.punct, ...l.patterns]) {
|
||||
assert.ok(e.source && e.source.trim(), `entry missing source: ${JSON.stringify(e)}`);
|
||||
}
|
||||
});
|
||||
|
||||
t("loader rejects an empty file", () => {
|
||||
const f = writeTmp("empty.json", "");
|
||||
assert.throws(() => loadLists(f), /empty file/);
|
||||
});
|
||||
|
||||
t("loader rejects unparseable JSON", () => {
|
||||
const f = writeTmp("bad.json", "{nope");
|
||||
assert.throws(() => loadLists(f), /unparseable/);
|
||||
});
|
||||
|
||||
t("loader rejects a missing file", () => {
|
||||
assert.throws(() => loadLists(tmp("does-not-exist.json")), /cannot read/);
|
||||
});
|
||||
|
||||
t("loader rejects missing keys", () => {
|
||||
const f = writeTmp("nokeys.json", { version: 1 });
|
||||
assert.throws(() => loadLists(f), /missing key/);
|
||||
});
|
||||
|
||||
t("loader rejects an emptied word list", () => {
|
||||
const f = writeTmp("emptywords.json", brokenVariant((l) => { l.words = []; }));
|
||||
assert.throws(() => loadLists(f), /words must be a non-empty array/);
|
||||
});
|
||||
|
||||
t("loader rejects entries without provenance", () => {
|
||||
const f = writeTmp("nosource.json", brokenVariant((l) => { delete l.phrases[0].source; }));
|
||||
assert.throws(() => loadLists(f), /source/);
|
||||
});
|
||||
|
||||
t("loader rejects duplicate values", () => {
|
||||
const f = writeTmp("dup.json", brokenVariant((l) => { l.words.push({ ...l.words[0] }); }));
|
||||
assert.throws(() => loadLists(f), /duplicate/);
|
||||
});
|
||||
|
||||
t("loader rejects a non-compiling pattern regex", () => {
|
||||
const f = writeTmp("badregex.json", brokenVariant((l) => { l.patterns[0].regex = "("; }));
|
||||
assert.throws(() => loadLists(f), /does not compile/);
|
||||
});
|
||||
|
||||
t("CLI exits 2 on broken lists (red control)", () => {
|
||||
const f = writeTmp("cli-broken.json", "");
|
||||
const r = spawnSync(process.execPath, [path.join(__dirname, "unslop-check.js")], {
|
||||
input: "some prose",
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, UNSLOP_LISTS: f },
|
||||
});
|
||||
assert.strictEqual(r.status, 2, `expected exit 2, got ${r.status} (stderr: ${r.stderr})`);
|
||||
assert.ok(r.stderr.includes("lists.json invalid"), `stderr must name the fault: ${r.stderr}`);
|
||||
});
|
||||
|
||||
t("CLI honors UNSLOP_LISTS for a valid file (green control)", () => {
|
||||
const r = spawnSync(process.execPath, [path.join(__dirname, "unslop-check.js")], {
|
||||
input: "plain prose with no tells at all",
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, UNSLOP_LISTS: path.join(__dirname, "lists.json") },
|
||||
});
|
||||
assert.strictEqual(r.status, 0, `expected exit 0, got ${r.status} (stderr: ${r.stderr})`);
|
||||
});
|
||||
|
||||
// ── Review follow-up (rev-code-01, 2026-08-19): F1, F2, F3, S3 ────────
|
||||
|
||||
t("loader rejects non-finite thresholds (F1)", () => {
|
||||
// Infinity cannot round-trip JSON.stringify, so the fixture is a raw string
|
||||
// edit of the real file — exactly the hand-edit that produced the finding.
|
||||
const real = fs.readFileSync(path.join(__dirname, "lists.json"), "utf8");
|
||||
const f = writeTmp("inf-threshold.json", real.replace('"punctMinCount": 3', '"punctMinCount": 1e999'));
|
||||
assert.ok(real !== fs.readFileSync(f, "utf8") || !real.includes('"punctMinCount": 3'), "fixture mutation did not apply; test is vacuous");
|
||||
assert.throws(() => loadLists(f), /finite/);
|
||||
const f2 = writeTmp("inf-density.json", real.replace('"punctDensityPer1k": 1.0', '"punctDensityPer1k": 1e999'));
|
||||
assert.throws(() => loadLists(f2), /finite/);
|
||||
});
|
||||
|
||||
t("curly-apostrophe phrase rendering is flagged (F2 red control)", () => {
|
||||
const r = checkText("Here\u2019s the honest truth about the deploy.");
|
||||
assert.ok(!r.clean, "curly apostrophe must not defeat phrase matching");
|
||||
assert.ok(r.findings.some((x) => x.detail.includes("here's the honest truth")), `main entry must match, got: ${JSON.stringify(r.findings)}`);
|
||||
});
|
||||
|
||||
t("curly apostrophes still fire the punct rule alongside phrases (F2 ordering)", () => {
|
||||
// Normalization for phrases must not eat the punct signal: four curly
|
||||
// quotes in short text must flag punct, not only the phrase.
|
||||
const r = checkText("It\u2019s \u2019one\u2019 \u2019two\u2019 \u2019three\u2019 \u2019four\u2019 done.");
|
||||
assert.ok(r.findings.some((f) => f.rule === "punct"), `punct must fire on original text: ${JSON.stringify(r.findings)}`);
|
||||
});
|
||||
|
||||
t("loader rejects non-lowercase phrase values (F3)", () => {
|
||||
const f = writeTmp("cap-phrase.json", brokenVariant((l) => { l.phrases[0].value = "Worth Stating Plainly"; }));
|
||||
assert.throws(() => loadLists(f), /lowercase/);
|
||||
});
|
||||
|
||||
t("CLI exits 2 on unreadable input file (S3)", () => {
|
||||
const r = spawnSync(process.execPath, [path.join(__dirname, "unslop-check.js"), tmp("definitely-absent.txt")], {
|
||||
encoding: "utf8",
|
||||
});
|
||||
assert.strictEqual(r.status, 2, `expected exit 2, got ${r.status} (stderr: ${r.stderr})`);
|
||||
assert.ok(r.stderr.includes("cannot read input"), `stderr must name the fault: ${r.stderr}`);
|
||||
});
|
||||
|
||||
let failed = 0;
|
||||
for (const [name, ok] of results) { console.log(`${ok ? "PASS" : "FAIL"} ${name}`); if (!ok) failed++; }
|
||||
console.log(`${results.length - failed}/${results.length} passed`);
|
||||
try { fs.rmSync(tmpdir, { recursive: true, force: true }); } catch {}
|
||||
process.exit(failed ? 1 : 0);
|
||||
@@ -0,0 +1,181 @@
|
||||
#!/usr/bin/env node
|
||||
"use strict";
|
||||
// unslop-check — mechanical AI-tell checker (ms-unslop subset + SYSTEM.md phrase bans).
|
||||
// Plain JS, no deps, so pi extensions (jiti) and Claude Code hook scripts (node CLI)
|
||||
// share one implementation.
|
||||
//
|
||||
// The lists live in lists.json beside this file: committed machine source with
|
||||
// per-entry provenance (which ms-unslop pattern or SYSTEM.md rule each entry
|
||||
// mechanizes), the mention convention, and the punct thresholds. The loader
|
||||
// hard-fails closed: an empty, unparseable, or schema-invalid lists.json throws,
|
||||
// and the CLI exits 2 so a broken gate is never mistaken for a clean verdict.
|
||||
//
|
||||
// CLI: node unslop-check.js <file> (or stdin)
|
||||
// exit 0 = clean, exit 1 = violations found (findings printed as JSON),
|
||||
// exit 2 = gate broken (lists.json missing/empty/invalid; error on stderr).
|
||||
// Env: UNSLOP_LISTS=<path> overrides the lists.json location (testing; reuse by
|
||||
// other harnesses sharing this file).
|
||||
//
|
||||
// Provenance note (2026-08-19): the inline lists this file carried before C1
|
||||
// moved to lists.json unchanged — 16 words, 17 phrases, 3 punct rules, 1 pattern.
|
||||
// The suite pins those counts; a list edit without a test edit is a drift signal.
|
||||
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
|
||||
function stripCode(text) {
|
||||
// Fenced blocks (``` or ~~~), then inline code spans. Code is quoted material,
|
||||
// not the agent's prose style. This is also the mention convention: a banned
|
||||
// item quoted as inline code is a mention and must not flag (see lists.json).
|
||||
return text
|
||||
.replace(/```[\s\S]*?```/g, " ")
|
||||
.replace(/~~~[\s\S]*?~~~/g, " ")
|
||||
.replace(/`[^`\n]*`/g, " ");
|
||||
}
|
||||
|
||||
// ── lists.json loading and validation ─────────────────────────────────────────
|
||||
|
||||
function validateLists(data) {
|
||||
const fail = (why) => { throw new Error(`lists.json invalid: ${why}`); };
|
||||
if (typeof data !== "object" || data === null || Array.isArray(data)) fail("top level must be an object");
|
||||
for (const k of ["version", "convention", "punctPolicy", "thresholds", "words", "phrases", "punct", "patterns"]) {
|
||||
if (!(k in data)) fail(`missing key: ${k}`);
|
||||
}
|
||||
if (typeof data.version !== "number" || data.version < 1) fail("version must be a number >= 1");
|
||||
for (const k of ["convention", "punctPolicy"]) {
|
||||
if (typeof data[k] !== "string" || !data[k].trim()) fail(`${k} must be a non-empty string`);
|
||||
}
|
||||
const th = data.thresholds;
|
||||
if (typeof th !== "object" || th === null) fail("thresholds must be an object");
|
||||
// Number.isFinite, not just typeof: JSON.parse of 1e999 yields Infinity, which
|
||||
// passes typeof-number and would silently disable the punct gate (review F1).
|
||||
if (!Number.isFinite(th.punctMinCount) || th.punctMinCount < 1) fail("thresholds.punctMinCount must be a finite number >= 1");
|
||||
if (!Number.isFinite(th.punctDensityPer1k) || !(th.punctDensityPer1k > 0)) fail("thresholds.punctDensityPer1k must be a finite number > 0");
|
||||
|
||||
const seen = new Set();
|
||||
const checkEntries = (arr, kind, extra) => {
|
||||
if (!Array.isArray(arr) || arr.length === 0) fail(`${kind} must be a non-empty array`);
|
||||
arr.forEach((e, i) => {
|
||||
const at = `${kind}[${i}]`;
|
||||
if (typeof e !== "object" || e === null) fail(`${at} must be an object`);
|
||||
if (typeof e.value !== "string" || !e.value.trim()) fail(`${at}.value must be a non-empty string`);
|
||||
if (typeof e.source !== "string" || !e.source.trim()) fail(`${at}.source must be a non-empty string (pattern id or system-md)`);
|
||||
if (extra) extra(e, at, fail);
|
||||
if (seen.has(`${kind}:${e.value}`)) fail(`duplicate ${kind} value: ${e.value}`);
|
||||
seen.add(`${kind}:${e.value}`);
|
||||
});
|
||||
};
|
||||
checkEntries(data.words, "words");
|
||||
checkEntries(data.phrases, "phrases", (e, at, fail) => {
|
||||
// Phrase matching splits a lowercased haystack, so an uppercase letter in a
|
||||
// phrase value is a silently dead rule (review F3). Reject, do not silently
|
||||
// normalize: list edits should fail loud (D-a).
|
||||
if (e.value !== e.value.toLowerCase()) fail(`${at}.value must be lowercase; phrase matching lowercases the haystack: ${e.value}`);
|
||||
});
|
||||
checkEntries(data.punct, "punct", (e, at, fail) => {
|
||||
if (typeof e.label !== "string" || !e.label.trim()) fail(`${at}.label must be a non-empty string`);
|
||||
if (!Array.isArray(e.chars) || e.chars.length === 0 || !e.chars.every((c) => typeof c === "string" && c.length === 1)) {
|
||||
fail(`${at}.chars must be a non-empty array of single-char strings`);
|
||||
}
|
||||
});
|
||||
checkEntries(data.patterns, "patterns", (e, at, fail) => {
|
||||
if (typeof e.regex !== "string" || !e.regex.trim()) fail(`${at}.regex must be a non-empty string`);
|
||||
if (typeof e.flags !== "string") fail(`${at}.flags must be a string`);
|
||||
if (typeof e.detail !== "string" || !e.detail.trim()) fail(`${at}.detail must be a non-empty string`);
|
||||
try { new RegExp(e.regex, e.flags); } catch (err) { fail(`${at}.regex does not compile: ${err.message}`); }
|
||||
});
|
||||
return data;
|
||||
}
|
||||
|
||||
let cache = null;
|
||||
function loadLists(filePath) {
|
||||
if (cache && !filePath) return cache;
|
||||
const p = filePath || process.env.UNSLOP_LISTS || path.join(__dirname, "lists.json");
|
||||
let raw;
|
||||
try {
|
||||
raw = fs.readFileSync(p, "utf8");
|
||||
} catch (e) {
|
||||
throw new Error(`lists.json invalid: cannot read ${p}: ${e.message}`);
|
||||
}
|
||||
if (!raw.trim()) throw new Error(`lists.json invalid: empty file: ${p}`);
|
||||
let data;
|
||||
try {
|
||||
data = JSON.parse(raw);
|
||||
} catch (e) {
|
||||
throw new Error(`lists.json invalid: unparseable JSON: ${e.message}`);
|
||||
}
|
||||
const validated = validateLists(data);
|
||||
if (!filePath) cache = validated;
|
||||
return validated;
|
||||
}
|
||||
|
||||
// ── checker ───────────────────────────────────────────────────────────────────
|
||||
|
||||
const escapeRegex = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
|
||||
function checkText(raw) {
|
||||
const lists = loadLists();
|
||||
const text = stripCode(String(raw));
|
||||
// Phrase haystack: lowercased, then curly apostrophes normalized to ASCII.
|
||||
// This must be a SEPARATE string from `text`: punct counting reads the
|
||||
// original, so curly quotes still fire the punct rule (review F2 ordering).
|
||||
const phraseHay = text.toLowerCase().replace(/[\u2018\u2019]/g, "'");
|
||||
const findings = [];
|
||||
|
||||
for (const w of lists.words) {
|
||||
const re = new RegExp("\\b" + escapeRegex(w.value) + "\\b", "gi");
|
||||
const count = (text.match(re) || []).length;
|
||||
if (count > 0) findings.push({ rule: "word", detail: `banned word "${w.value}" x${count}`, count });
|
||||
}
|
||||
|
||||
for (const p of lists.phrases) {
|
||||
const count = phraseHay.split(p.value).length - 1;
|
||||
if (count > 0) findings.push({ rule: "phrase", detail: `phrase "${p.value}" x${count}`, count });
|
||||
}
|
||||
|
||||
// Density-gated punctuation. The deliberate divergence from SYSTEM.md's
|
||||
// outright em-dash ban is documented in lists.json punctPolicy, not only here.
|
||||
for (const pc of lists.punct) {
|
||||
let count = 0;
|
||||
for (const ch of pc.chars) count += text.split(ch).length - 1;
|
||||
if (count < lists.thresholds.punctMinCount) continue;
|
||||
if (count / Math.max(text.length, 1) * 1000 < lists.thresholds.punctDensityPer1k) continue;
|
||||
findings.push({ rule: "punct", detail: `${pc.label} x${count} (density-gated)`, count });
|
||||
}
|
||||
|
||||
for (const pt of lists.patterns) {
|
||||
const flags = pt.flags.includes("g") ? pt.flags : pt.flags + "g";
|
||||
const m = text.match(new RegExp(pt.regex, flags));
|
||||
const count = m ? m.length : 0;
|
||||
if (count > 0) findings.push({ rule: "pattern", detail: `"${pt.detail}" x${count}`, count });
|
||||
}
|
||||
|
||||
return { clean: findings.length === 0, findings, charsChecked: text.length };
|
||||
}
|
||||
|
||||
module.exports = { checkText, stripCode, loadLists, validateLists };
|
||||
|
||||
if (require.main === module) {
|
||||
let input;
|
||||
try {
|
||||
input = process.argv[2] ? fs.readFileSync(process.argv[2], "utf8") : fs.readFileSync(0, "utf8");
|
||||
} catch (e) {
|
||||
// An unreadable input must not exit 1: that is the violations code, and a
|
||||
// wrapper keying on rc alone would report slop-free for a file it never
|
||||
// read (review S3).
|
||||
console.error(`unslop-check: cannot read input: ${e.message}`);
|
||||
process.exit(2);
|
||||
}
|
||||
let result;
|
||||
try {
|
||||
result = checkText(input);
|
||||
} catch (e) {
|
||||
if (String(e.message).startsWith("lists.json invalid")) {
|
||||
console.error(`unslop-check: ${e.message}`);
|
||||
process.exit(2);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
process.exit(result.clean ? 0 : 1);
|
||||
}
|
||||
Reference in New Issue
Block a user