Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Hermes Agent
|
f6d6d0bb62 |
+150
-20
@@ -1,24 +1,154 @@
|
||||
# Non-secret runtime settings for the mosaic-poc-agent container.
|
||||
# Copy to .env if you want to override the defaults in compose.yaml.
|
||||
#
|
||||
# NEVER put credentials in this file. Authentication is supplied at
|
||||
# runtime only, via one of the two documented paths:
|
||||
# 1. read-only mounted pi auth file (default: ~/.pi/agent/auth.json,
|
||||
# override the host path with PI_AUTH_FILE)
|
||||
# 2. provider API key environment variable (ZAI_API_KEY or
|
||||
# ANTHROPIC_API_KEY), passed through by compose.yaml when set
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Mosaic — Environment Variables Reference
|
||||
# Copy this file to .env and fill in the values for your deployment.
|
||||
# Lines beginning with # are comments; optional vars are commented out.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Model provider (built-in pi provider name)
|
||||
PI_PROVIDER=zai
|
||||
|
||||
# Model ID within the provider
|
||||
PI_MODEL=glm-5.3-flash
|
||||
# ─── Database (PostgreSQL 17 + pgvector) ─────────────────────────────────────
|
||||
# Full connection string used by the gateway, ORM, and migration runner.
|
||||
# Port 5433 avoids conflict with a host-side PostgreSQL instance.
|
||||
DATABASE_URL=postgresql://mosaic:mosaic@localhost:5433/mosaic
|
||||
|
||||
# Optional: alternative host path of the pi credential file mounted
|
||||
# read-only at /home/node/.pi/agent/auth.json in the container
|
||||
#PI_AUTH_FILE=/home/jwoltje/.pi/agent/auth.json
|
||||
# Docker Compose host-port override for the PostgreSQL container (default: 5433)
|
||||
# PG_HOST_PORT=5433
|
||||
|
||||
# Optional: documented env-var auth alternative (secret! set in your
|
||||
# shell or a gitignored .env, never commit)
|
||||
#ZAI_API_KEY=
|
||||
#ANTHROPIC_API_KEY=
|
||||
|
||||
# ─── Queue (Valkey 8 / Redis-compatible) ─────────────────────────────────────
|
||||
# Port 6380 avoids conflict with a host-side Redis/Valkey instance.
|
||||
VALKEY_URL=redis://localhost:6380
|
||||
|
||||
# Docker Compose host-port override for the Valkey container (default: 6380)
|
||||
# VALKEY_HOST_PORT=6380
|
||||
|
||||
|
||||
# ─── Gateway ─────────────────────────────────────────────────────────────────
|
||||
# TCP port the NestJS/Fastify gateway listens on (default: 14242)
|
||||
GATEWAY_PORT=14242
|
||||
|
||||
# Comma-separated list of allowed CORS origins.
|
||||
# Must include the web app origin in production.
|
||||
GATEWAY_CORS_ORIGIN=http://localhost:3000
|
||||
|
||||
|
||||
# ─── Auth (BetterAuth) ───────────────────────────────────────────────────────
|
||||
# REQUIRED — random secret used to sign sessions and tokens.
|
||||
# Generate with: openssl rand -base64 32
|
||||
BETTER_AUTH_SECRET=change-me-to-a-random-32-char-string
|
||||
|
||||
# Public base URL of the gateway (used by BetterAuth for callback URLs)
|
||||
BETTER_AUTH_URL=http://localhost:14242
|
||||
|
||||
|
||||
# ─── Web App (Next.js) ───────────────────────────────────────────────────────
|
||||
# Public gateway URL — accessible from the browser, not just the server.
|
||||
NEXT_PUBLIC_GATEWAY_URL=http://localhost:14242
|
||||
|
||||
|
||||
# ─── OpenTelemetry ───────────────────────────────────────────────────────────
|
||||
# OTLP HTTP endpoint (otel-collector or any OpenTelemetry-compatible backend)
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
||||
|
||||
# Service name shown in traces
|
||||
OTEL_SERVICE_NAME=mosaic-gateway
|
||||
|
||||
|
||||
# ─── AI Providers ────────────────────────────────────────────────────────────
|
||||
|
||||
# Ollama (local models — set OLLAMA_BASE_URL to enable)
|
||||
# OLLAMA_BASE_URL=http://localhost:11434
|
||||
# OLLAMA_HOST is a legacy alias for OLLAMA_BASE_URL
|
||||
# OLLAMA_HOST=http://localhost:11434
|
||||
# Comma-separated list of Ollama model IDs to register (default: llama3.2,codellama,mistral)
|
||||
# OLLAMA_MODELS=llama3.2,codellama,mistral
|
||||
|
||||
# Anthropic (claude-sonnet-4-6, claude-opus-4-6, claude-haiku-4-5)
|
||||
# ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# OpenAI (gpt-4o, gpt-4o-mini, o3-mini)
|
||||
# OPENAI_API_KEY=sk-...
|
||||
|
||||
# Z.ai / GLM (glm-4.5, glm-4.5-air, glm-4.5-flash)
|
||||
# ZAI_API_KEY=...
|
||||
|
||||
# Custom providers — JSON array of provider configs
|
||||
# Format: [{"id":"<id>","baseUrl":"<url>","apiKey":"<key>","models":[{"id":"<model-id>","name":"<label>"}]}]
|
||||
# MOSAIC_CUSTOM_PROVIDERS=
|
||||
|
||||
|
||||
# ─── Embedding Service ───────────────────────────────────────────────────────
|
||||
# OpenAI-compatible embeddings endpoint (default: OpenAI)
|
||||
# EMBEDDING_API_URL=https://api.openai.com/v1
|
||||
# EMBEDDING_MODEL=text-embedding-3-small
|
||||
|
||||
|
||||
# ─── Log Summarization Service ───────────────────────────────────────────────
|
||||
# OpenAI-compatible chat completions endpoint for log summarization (default: OpenAI)
|
||||
# SUMMARIZATION_API_URL=https://api.openai.com/v1
|
||||
# SUMMARIZATION_MODEL=gpt-4o-mini
|
||||
|
||||
# Cron schedule for summarization job (default: every 6 hours)
|
||||
# SUMMARIZATION_CRON=0 */6 * * *
|
||||
|
||||
# Cron schedule for log tier management (default: daily at 03:00)
|
||||
# TIER_MANAGEMENT_CRON=0 3 * * *
|
||||
|
||||
|
||||
# ─── Agent ───────────────────────────────────────────────────────────────────
|
||||
# Filesystem sandbox root for agent file tools (default: process.cwd())
|
||||
# AGENT_FILE_SANDBOX_DIR=/var/lib/mosaic/sandbox
|
||||
|
||||
# Comma-separated list of tool names available to non-admin users.
|
||||
# Leave unset to allow all tools for all authenticated users.
|
||||
# AGENT_USER_TOOLS=read_file,list_directory,search_files
|
||||
|
||||
# System prompt injected into every agent session (optional)
|
||||
# AGENT_SYSTEM_PROMPT=You are a helpful assistant.
|
||||
|
||||
|
||||
# ─── MCP Servers ─────────────────────────────────────────────────────────────
|
||||
# JSON array of MCP server configs — set to enable MCP tool integration.
|
||||
# Each entry: {"name":"<id>","url":"<http-or-sse-url>"}
|
||||
# MCP_SERVERS=[{"name":"my-mcp","url":"http://localhost:3100/sse"}]
|
||||
|
||||
|
||||
# ─── Coordinator ─────────────────────────────────────────────────────────────
|
||||
# Root directory used to scope coordinator (worktree/repo) operations.
|
||||
# Defaults to the monorepo root auto-detected from process.cwd().
|
||||
# MOSAIC_WORKSPACE_ROOT=/home/user/projects/mosaic
|
||||
|
||||
|
||||
# ─── Discord Plugin (optional — set DISCORD_BOT_TOKEN to enable) ─────────────
|
||||
# DISCORD_BOT_TOKEN=
|
||||
# DISCORD_GUILD_ID=
|
||||
# DISCORD_GATEWAY_URL=http://localhost:14242
|
||||
|
||||
|
||||
# ─── Telegram Plugin (optional — set TELEGRAM_BOT_TOKEN to enable) ───────────
|
||||
# TELEGRAM_BOT_TOKEN=
|
||||
# TELEGRAM_GATEWAY_URL=http://localhost:14242
|
||||
|
||||
|
||||
# ─── SSO Providers (add credentials to enable) ───────────────────────────────
|
||||
|
||||
# --- Authentik (optional — set AUTHENTIK_CLIENT_ID to enable) ---
|
||||
# AUTHENTIK_ISSUER=https://auth.example.com/application/o/mosaic/
|
||||
# AUTHENTIK_CLIENT_ID=
|
||||
# AUTHENTIK_CLIENT_SECRET=
|
||||
|
||||
# --- WorkOS (optional — set WORKOS_CLIENT_ID to enable) ---
|
||||
# WORKOS_ISSUER=https://your-company.authkit.app
|
||||
# WORKOS_CLIENT_ID=client_...
|
||||
# WORKOS_CLIENT_SECRET=sk_live_...
|
||||
|
||||
# --- Keycloak (optional — set KEYCLOAK_CLIENT_ID to enable) ---
|
||||
# KEYCLOAK_ISSUER=https://auth.example.com/realms/master
|
||||
# Legacy alternative if you prefer to compose the issuer from separate vars:
|
||||
# KEYCLOAK_URL=https://auth.example.com
|
||||
# KEYCLOAK_REALM=master
|
||||
# KEYCLOAK_CLIENT_ID=mosaic
|
||||
# KEYCLOAK_CLIENT_SECRET=
|
||||
|
||||
# Feature flags — set to true alongside provider credentials to show SSO buttons in the UI
|
||||
# NEXT_PUBLIC_WORKOS_ENABLED=true
|
||||
# NEXT_PUBLIC_KEYCLOAK_ENABLED=true
|
||||
|
||||
+22
-6
@@ -1,8 +1,24 @@
|
||||
# build/deps
|
||||
node_modules/
|
||||
|
||||
# runtime credentials — never commit, never copy into the image
|
||||
logs/
|
||||
node_modules
|
||||
dist
|
||||
.turbo
|
||||
.next
|
||||
coverage
|
||||
.env
|
||||
secrets/
|
||||
.env.local
|
||||
*.tsbuildinfo
|
||||
.pnpm-store
|
||||
docs/reports/
|
||||
|
||||
# generated runtime state lives in /home/jwoltje/.mosaic-dev (outside this project)
|
||||
# Step-CA dev password — real file is gitignored; commit only the .example
|
||||
infra/step-ca/dev-password
|
||||
|
||||
# Scratch dirs created by the framework git-wrapper shell test harnesses
|
||||
.mosaic-test-work/
|
||||
|
||||
# Transient config files vite/vitest/esbuild write next to a *.config.ts while
|
||||
# loading it, then unlink. They are untracked but were not ignored, so turbo's
|
||||
# package traversal hashed them and intermittently failed CI with "Package
|
||||
# traversal error: ... .timestamp-*.mjs: No such file or directory" when the
|
||||
# file vanished mid-scan. Ignoring them removes the race.
|
||||
*.timestamp-*.mjs
|
||||
|
||||
Executable
+1
@@ -0,0 +1 @@
|
||||
pnpm typecheck && pnpm lint && pnpm format:check
|
||||
@@ -0,0 +1,5 @@
|
||||
@mosaicstack:registry=https://git.mosaicstack.dev/api/packages/mosaicstack/npm/
|
||||
# Pin the pnpm store to the same path the ci-base image warms (Dockerfile.ci),
|
||||
# so the pipeline `pnpm install --prefer-offline` consumes the baked store
|
||||
# instead of repopulating a fresh one.
|
||||
store-dir=/root/.local/share/pnpm/store
|
||||
@@ -1,6 +0,0 @@
|
||||
extensions/
|
||||
extensions.installed.sha256
|
||||
.extensions-*
|
||||
state/
|
||||
evidence/
|
||||
native-test-*.log
|
||||
@@ -1,34 +0,0 @@
|
||||
# Native goal development copy
|
||||
|
||||
From this repository, start a fresh native Pi session:
|
||||
|
||||
```sh
|
||||
bash scripts/goal-dev.sh
|
||||
```
|
||||
|
||||
Canonical source lives under `extensions/`. The launcher first runs `scripts/sync-dev-extensions.sh`, which installs verified ordinary-file copies under `.pi/extensions/`, then loads only the generated goal extension. Global extensions remain unloaded. The launcher keeps your usual native Pi provider authentication; it copies no credentials. Goal state and new conversation files live under `.pi/state/`, which is ignored by Git. Each process gets a fresh incarnation; `/reload` and `/new` in the same process retain its goal. Restarting Pi does not adopt an earlier process's active goal.
|
||||
|
||||
Plain `pi` also discovers `.pi/extensions/goal/index.ts` after project trust, but may load global extensions too. Use the launcher to avoid duplicate `/goal` registrations. This is a local development test, not a sandbox or the managed Mosaic runtime. Docker and `~/.mosaic` are unchanged.
|
||||
|
||||
## Try it
|
||||
|
||||
1. Set `/goal <a long goal with acceptance criteria>`. This starts work immediately.
|
||||
2. Look below the editor for `Goal: Active`. The old above-editor goal widget is gone.
|
||||
3. Run bare `/goal`, then press `Alt+G`. Both show the entire stored goal and its status. Tab remains autocomplete.
|
||||
4. Use `/goal stop` and `/goal resume`. Expect Paused and Active, or Waiting if an untimed wait remains recorded.
|
||||
5. A blocked `goal_report` displays Blocked. A satisfied report displays Complete and retains the full goal for recall without continuing work.
|
||||
6. `/goal clear` removes the retained goal. Try `NO_COLOR=1 bash scripts/goal-dev.sh` to check text-only labels.
|
||||
|
||||
Use terminal scrollback for recall longer than the screen. At narrow widths Pi may truncate its footer status row; bare `/goal` and Alt+G remain available.
|
||||
|
||||
## Checks
|
||||
|
||||
```sh
|
||||
node --test extensions/goal/test/*.test.ts
|
||||
bash scripts/test-extension-package.sh
|
||||
python3 scripts/test-goal-native.py
|
||||
```
|
||||
|
||||
Contract tests use ordinary read-only fixture copies in `test/fixtures/skills-local/`, not live brain files. The executive-update fixture SHA-256 matches the parser's pinned contract, `bbea48a46b1f8da7bc759f86856fb52830b7dde456b826317163c6dc6ccab319`.
|
||||
|
||||
`SOURCE-SNAPSHOT.json` records the original external-source baseline, not the edited candidate. No symlinks are used. Never edit `.pi/extensions/`; the sync script refuses to overwrite installation drift. Make changes under `extensions/`, run the checks, and relaunch. To disable the test, stop its Pi process and remove `.pi/extensions/`. Keep `.pi/state/` only if you need local test state.
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"snapshotVersion": 1,
|
||||
"copiedAt": "2026-09-06T04:58:22Z",
|
||||
"source": "~/.mosaic/fleet/extensions",
|
||||
"goalTreeSha256": "8853f2b72dde3e87c4573648b9a931c1c75da87ccde995c3224e6d2e707a75f0",
|
||||
"mosaicCoreLibTreeSha256": "d1194dce31209e5773c6cc5ce571cbca3c39b29d943a79dea06665e05d29f319",
|
||||
"symlinks": false,
|
||||
"autoDiscoveredExtensions": ["goal"],
|
||||
"purpose": "Issue #54 native Pi NG development copy; never loaded by Docker"
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Compatibility entrypoint for the accepted native test command.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "${BASH_SOURCE[0]}")/.."
|
||||
exec scripts/goal-dev.sh "$@"
|
||||
@@ -0,0 +1,9 @@
|
||||
pnpm-lock.yaml
|
||||
**/next-env.d.ts
|
||||
**/dist
|
||||
**/node_modules
|
||||
**/drizzle
|
||||
**/.next
|
||||
.claude/
|
||||
docs/tess/TASKS.md
|
||||
docs/scratchpads/
|
||||
@@ -22,9 +22,9 @@ steps:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
environment:
|
||||
REGISTRY_USER:
|
||||
from_secret: REGISTRY_USERNAME
|
||||
from_secret: gitea_username
|
||||
REGISTRY_PASS:
|
||||
from_secret: REGISTRY_PASSWORD
|
||||
from_secret: gitea_password
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
@@ -0,0 +1,108 @@
|
||||
# &node_image is the pre-baked CI base built by .woodpecker/ci-image.yml:
|
||||
# node:24-alpine + python3/make/g++/postgresql-client + pnpm + a warm pnpm
|
||||
# store. The install step resolves from the baked store (--prefer-offline)
|
||||
# instead of paying a ~731s cold fetch + native compile every run.
|
||||
variables:
|
||||
- &node_image 'git.mosaicstack.dev/mosaicstack/stack/ci-base:latest'
|
||||
- &enable_pnpm 'corepack enable'
|
||||
|
||||
when:
|
||||
# PR + manual CI run on any branch — the pull_request pipeline is the merge gate.
|
||||
# push CI is restricted to protected branches (main) so a feature-branch push no
|
||||
# longer fires a redundant SECOND pipeline alongside its PR pipeline. This ~halves
|
||||
# CI load on the storage-constrained runner with zero loss of gating (branch
|
||||
# protection requires no push/ci status context; main still gets full push CI).
|
||||
- event: [pull_request, manual]
|
||||
- event: push
|
||||
branch: main
|
||||
|
||||
# Turbo remote cache (turbo.mosaicstack.dev) is configured via Woodpecker
|
||||
# repository-level environment variables (TURBO_API, TURBO_TEAM, TURBO_TOKEN).
|
||||
# This avoids from_secret which is blocked on pull_request events.
|
||||
# If the env vars aren't set, turbo falls back to local cache only.
|
||||
|
||||
steps:
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- corepack enable
|
||||
# python3/make/g++ are baked into ci-base; --prefer-offline resolves from
|
||||
# the baked pnpm store.
|
||||
- pnpm install --frozen-lockfile --prefer-offline
|
||||
|
||||
# Blocking gate: public framework package must contain no operator-specific
|
||||
# personal data or private $HOME defaults. Runs early (no node_modules needed).
|
||||
sanitization:
|
||||
image: *node_image
|
||||
commands:
|
||||
- apk add --no-cache bash
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/verify-sanitized.sh
|
||||
# Resident line-count ceiling over framework-owned resident files
|
||||
# (Constitution + dispatcher + each RUNTIME.md slice). See DESIGN §7 / R9.
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh --self-test
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm typecheck
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
|
||||
# lint, format, and test are independent — run in parallel after typecheck
|
||||
lint:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm lint
|
||||
depends_on:
|
||||
- typecheck
|
||||
|
||||
format:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm format:check
|
||||
depends_on:
|
||||
- typecheck
|
||||
|
||||
test:
|
||||
image: *node_image
|
||||
environment:
|
||||
# Avoid the namespace-level Woodpecker DB service named "postgres".
|
||||
# The Kubernetes backend exposes service containers by step name.
|
||||
DATABASE_URL: postgresql://mosaic:mosaic@ci-postgres:5432/mosaic
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
# postgresql-client (pg_isready) is baked into ci-base.
|
||||
# Wait up to 60s for CI postgres to be ready; fail fast if it never comes up.
|
||||
- |
|
||||
ready=0
|
||||
for i in $(seq 1 60); do
|
||||
if pg_isready -h ci-postgres -p 5432 -U mosaic; then
|
||||
ready=1
|
||||
break
|
||||
fi
|
||||
echo "Waiting for ci-postgres ($i/60)..."
|
||||
sleep 1
|
||||
done
|
||||
if [ "$ready" -ne 1 ]; then
|
||||
echo "ci-postgres did not become ready" >&2
|
||||
exit 1
|
||||
fi
|
||||
# Run migrations (DATABASE_URL is set in environment above)
|
||||
- pnpm --filter @mosaicstack/db run db:migrate
|
||||
# Run all tests
|
||||
- pnpm test
|
||||
depends_on:
|
||||
- typecheck
|
||||
|
||||
services:
|
||||
ci-postgres:
|
||||
image: pgvector/pgvector:pg17
|
||||
environment:
|
||||
POSTGRES_USER: mosaic
|
||||
POSTGRES_PASSWORD: mosaic
|
||||
POSTGRES_DB: mosaic
|
||||
@@ -0,0 +1,197 @@
|
||||
# Build, publish npm packages, and push Docker images
|
||||
# Runs only on main branch push/tag
|
||||
|
||||
variables:
|
||||
# Pre-baked CI base (see .woodpecker/ci-image.yml): node:24-alpine +
|
||||
# toolchain + warm pnpm store. Kills the second cold install publish pays.
|
||||
- &node_image 'git.mosaicstack.dev/mosaicstack/stack/ci-base:latest'
|
||||
- &enable_pnpm 'corepack enable'
|
||||
# Heavy kaniko image builds (~25 min) — gate them so a merge that only touches
|
||||
# the npm-only CLI (@mosaicstack/mosaic) or docs does NOT rebuild the platform
|
||||
# images (gateway/appservice/web do not depend on @mosaicstack/mosaic). Releases
|
||||
# (tags) always build everything. Exclude-list keeps the default SAFE: any
|
||||
# non-excluded change still builds, so no transitive dep can silently go stale.
|
||||
# (Woodpecker: `when` entries are OR'd; `path` applies to push/PR only — hence
|
||||
# the separate `event: tag` entry.)
|
||||
- &image_build_when
|
||||
- event: tag
|
||||
- event: [push, manual]
|
||||
branch: main
|
||||
path:
|
||||
exclude:
|
||||
- 'packages/mosaic/**'
|
||||
- 'docs/**'
|
||||
- '**/*.md'
|
||||
- '.woodpecker/**'
|
||||
|
||||
when:
|
||||
- branch: [main]
|
||||
event: [push, manual, tag]
|
||||
|
||||
steps:
|
||||
install:
|
||||
image: *node_image
|
||||
commands:
|
||||
- corepack enable
|
||||
# Resolve from the baked pnpm store instead of a cold network fetch.
|
||||
- pnpm install --frozen-lockfile --prefer-offline
|
||||
|
||||
build:
|
||||
image: *node_image
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
- pnpm build
|
||||
depends_on:
|
||||
- install
|
||||
|
||||
publish-npm:
|
||||
image: *node_image
|
||||
# Publish only when a publishable package changed (or on a release tag); a
|
||||
# pure-docs merge runs no publish. Cheap step, but gated for cleanliness.
|
||||
when:
|
||||
- event: tag
|
||||
- event: [push, manual]
|
||||
branch: main
|
||||
path:
|
||||
include:
|
||||
- 'packages/**'
|
||||
environment:
|
||||
NPM_TOKEN:
|
||||
from_secret: gitea_token
|
||||
commands:
|
||||
- *enable_pnpm
|
||||
# Configure auth for Gitea npm registry
|
||||
- |
|
||||
echo "//git.mosaicstack.dev/api/packages/mosaicstack/npm/:_authToken=$NPM_TOKEN" > ~/.npmrc
|
||||
echo "@mosaicstack:registry=https://git.mosaicstack.dev/api/packages/mosaicstack/npm/" >> ~/.npmrc
|
||||
# Publish non-private packages to Gitea.
|
||||
#
|
||||
# The only publish failure we tolerate is "version already exists" —
|
||||
# that legitimately happens when only some packages were bumped in
|
||||
# the merge. Any other failure (registry 404, auth error, network
|
||||
# error) MUST fail the pipeline loudly: the previous
|
||||
# `|| echo "... continuing"` fallback silently hid a 404 from the
|
||||
# Gitea org rename and caused every @mosaicstack/* publish to fall
|
||||
# on the floor while CI still reported green.
|
||||
- |
|
||||
# Portable sh (Alpine ash) — avoid bashisms like PIPESTATUS.
|
||||
set +e
|
||||
pnpm --filter "@mosaicstack/*" --filter "!@mosaicstack/web" publish --no-git-checks --access public >/tmp/publish.log 2>&1
|
||||
EXIT=$?
|
||||
set -e
|
||||
cat /tmp/publish.log
|
||||
if [ "$EXIT" -eq 0 ]; then
|
||||
echo "[publish] all packages published successfully"
|
||||
exit 0
|
||||
fi
|
||||
# Hard registry / auth / network errors → fatal. Match npm's own
|
||||
# error lines specifically to avoid false positives on arbitrary
|
||||
# log text that happens to contain "E404" etc.
|
||||
if grep -qE "npm (error|ERR!) code (E404|E401|ENEEDAUTH|ECONNREFUSED|ETIMEDOUT|ENOTFOUND)" /tmp/publish.log; then
|
||||
echo "[publish] FATAL: registry/auth/network error detected — failing pipeline" >&2
|
||||
exit 1
|
||||
fi
|
||||
# Only tolerate the explicit "version already published" case.
|
||||
# npm returns this as E403 with body "You cannot publish over..."
|
||||
# or EPUBLISHCONFLICT depending on version.
|
||||
if grep -qE "EPUBLISHCONFLICT|You cannot publish over|previously published" /tmp/publish.log; then
|
||||
echo "[publish] some packages already at this version — continuing (non-fatal)"
|
||||
exit 0
|
||||
fi
|
||||
echo "[publish] FATAL: publish failed with unrecognized error — failing pipeline" >&2
|
||||
exit 1
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
# TODO: Uncomment when ready to publish to npmjs.org
|
||||
# publish-npmjs:
|
||||
# image: *node_image
|
||||
# environment:
|
||||
# NPM_TOKEN:
|
||||
# from_secret: npmjs_token
|
||||
# commands:
|
||||
# - *enable_pnpm
|
||||
# - apk add --no-cache jq bash
|
||||
# - bash scripts/publish-npmjs.sh
|
||||
# depends_on:
|
||||
# - build
|
||||
# when:
|
||||
# - event: [tag]
|
||||
|
||||
build-gateway:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
when: *image_build_when
|
||||
environment:
|
||||
REGISTRY_USER:
|
||||
from_secret: gitea_username
|
||||
REGISTRY_PASS:
|
||||
from_secret: gitea_password
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- mkdir -p /kaniko/.docker
|
||||
- echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$REGISTRY_USER\",\"password\":\"$REGISTRY_PASS\"}}}" > /kaniko/.docker/config.json
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaicstack/stack/gateway:sha-${CI_COMMIT_SHA:0:7}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/gateway:latest"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/gateway:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile docker/gateway.Dockerfile $DESTINATIONS
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
build-appservice:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
when: *image_build_when
|
||||
environment:
|
||||
REGISTRY_USER:
|
||||
from_secret: gitea_username
|
||||
REGISTRY_PASS:
|
||||
from_secret: gitea_password
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- mkdir -p /kaniko/.docker
|
||||
- echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$REGISTRY_USER\",\"password\":\"$REGISTRY_PASS\"}}}" > /kaniko/.docker/config.json
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaicstack/stack/appservice:sha-${CI_COMMIT_SHA:0:7}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/appservice:latest"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/appservice:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile docker/appservice.Dockerfile $DESTINATIONS
|
||||
depends_on:
|
||||
- build
|
||||
|
||||
build-web:
|
||||
image: gcr.io/kaniko-project/executor:debug
|
||||
when: *image_build_when
|
||||
environment:
|
||||
REGISTRY_USER:
|
||||
from_secret: gitea_username
|
||||
REGISTRY_PASS:
|
||||
from_secret: gitea_password
|
||||
CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH}
|
||||
CI_COMMIT_TAG: ${CI_COMMIT_TAG}
|
||||
CI_COMMIT_SHA: ${CI_COMMIT_SHA}
|
||||
commands:
|
||||
- mkdir -p /kaniko/.docker
|
||||
- echo "{\"auths\":{\"git.mosaicstack.dev\":{\"username\":\"$REGISTRY_USER\",\"password\":\"$REGISTRY_PASS\"}}}" > /kaniko/.docker/config.json
|
||||
- |
|
||||
DESTINATIONS="--destination git.mosaicstack.dev/mosaicstack/stack/web:sha-${CI_COMMIT_SHA:0:7}"
|
||||
if [ "$CI_COMMIT_BRANCH" = "main" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/web:latest"
|
||||
fi
|
||||
if [ -n "$CI_COMMIT_TAG" ]; then
|
||||
DESTINATIONS="$DESTINATIONS --destination git.mosaicstack.dev/mosaicstack/stack/web:$CI_COMMIT_TAG"
|
||||
fi
|
||||
/kaniko/executor --context . --dockerfile docker/web.Dockerfile $DESTINATIONS
|
||||
depends_on:
|
||||
- build
|
||||
@@ -1,217 +1,80 @@
|
||||
# AGENTS.md — Mosaic Stack rebuild (`mosaicstack/stack`, branch `refactor`)
|
||||
# Agent Guidelines — Mosaic Stack
|
||||
|
||||
Operational context for any agent session working in this repository.
|
||||
Read top to bottom; it is deliberately short — depth lives in the files it
|
||||
points to, not here.
|
||||
## Required Load Order
|
||||
|
||||
## What this repository is
|
||||
1. `~/.config/mosaic/SOUL.md`
|
||||
2. `~/.config/mosaic/STANDARDS.md`
|
||||
3. `~/.config/mosaic/AGENTS.md`
|
||||
4. `~/.config/mosaic/guides/E2E-DELIVERY.md`
|
||||
5. `AGENTS.md` (this file)
|
||||
6. Runtime-specific guide: `~/.config/mosaic/runtime/<runtime>/RUNTIME.md`
|
||||
|
||||
Canonical checkout: `/mnt/storage/src/mosaic-stack`, origin `mosaicstack/stack`,
|
||||
working branch `refactor` (Jason-authorized conversion, issue #1495).
|
||||
The new foundation is at the root. `v1/` is archived legacy source, not the current
|
||||
implementation; its instructions and tools do not govern the new foundation.
|
||||
`~/src/mosaic-stack-dev-test` is a compatibility symlink to this checkout, not a
|
||||
second working tree. Both original Git histories are retained. Conversion receipt:
|
||||
`docs/plans/2026-09-07_repository-consolidation-completed.md`.
|
||||
## Project Context
|
||||
|
||||
A rebuild of Mosaic Stack: a file-based, fail-closed
|
||||
orchestration foundation that dispatches sandboxed headless pi workers to do
|
||||
real work, with immutable run records as evidence. Thirteen-plus tagged
|
||||
milestones (`git tag -l`) from `poc-container-hello-v0` to today; suites
|
||||
green at every step. Not production software — a proven foundation.
|
||||
Mosaic Stack is a self-hosted, multi-user AI agent platform. TypeScript monorepo with NestJS gateway, Next.js web dashboard, Pi SDK agent runtime, and plugin architecture for Discord/Telegram.
|
||||
|
||||
## Non-negotiable invariants (the canon)
|
||||
## Package Map
|
||||
|
||||
1. **Root is bootstrap-only.** First-class system configuration lives at the
|
||||
repository root; everything else gets a dedicated directory (`roles/`,
|
||||
`contracts/`, `missions/`, `tasks/`, `docs/`). Do not add new files to root.
|
||||
2. **Configuration**: `~/.config/mosaic-dev/config.json` is the sole system
|
||||
config — created only by `scripts/bootstrap.sh`, never overwritten,
|
||||
fail-closed on any problem. Repo-scoped role authority lives in
|
||||
`roles/*.json` (versioned, reviewed commits only).
|
||||
3. **Secrets** never enter the repository or container images; auth is
|
||||
runtime-only (read-only mount or environment variable).
|
||||
4. **Contracts** (`contracts/`) are immutable and image-baked. Missions and
|
||||
tasks are declarative JSON with strict schemas.
|
||||
5. **Run records** under `<dataRoot>/runs/` are write-once evidence — never
|
||||
rewritten, only pruned via `prune` with a receipt.
|
||||
6. **Fail closed**: missing or invalid config/policy refuses the operation.
|
||||
Never improvise around a refusal; diagnose it.
|
||||
7. **Policy**: missions govern tasks (least-privilege intersection — a task
|
||||
narrows, never widens). Role authority is declared in `roles/` and changes
|
||||
only via reviewed commits.
|
||||
8. **Git**: commit only after applicable suites are green. Work on the
|
||||
owner-authorized `refactor` branch; never force-push. Push remains an explicit
|
||||
act. Do not merge into `next` or `main` without separate authorization.
|
||||
`scripts/conductor-apply.sh` commits locally; it does not authorize a push.
|
||||
9. **Append-only logs**: BUILD-LOG.md (phases), `activation-log.jsonl`,
|
||||
`.pruned.log`, docs/SESSIONS.md. Corrections are new entries, never edits.
|
||||
| Package | Purpose | Key Dependencies |
|
||||
| ------------------ | ------------------------------- | -------------------------------- |
|
||||
| `apps/gateway` | NestJS API + WebSocket hub | Fastify, Socket.IO, Pi SDK, OTEL |
|
||||
| `apps/web` | Next.js dashboard | React 19, Tailwind |
|
||||
| `packages/types` | Shared TypeScript contracts | class-validator |
|
||||
| `packages/db` | Drizzle ORM schema + migrations | drizzle-orm, postgres |
|
||||
| `packages/auth` | BetterAuth configuration | better-auth, @mosaicstack/db |
|
||||
| `packages/brain` | Data layer (PG-backed) | @mosaicstack/db |
|
||||
| `packages/queue` | Valkey task queue + MCP | ioredis |
|
||||
| `packages/coord` | Mission coordination | @mosaicstack/queue |
|
||||
| `packages/mosaic` | Unified `mosaic` CLI + TUI | Ink, Pi SDK, commander |
|
||||
| `plugins/discord` | Discord channel plugin | discord.js |
|
||||
| `plugins/telegram` | Telegram channel plugin | Telegraf |
|
||||
|
||||
## Autonomous operation within an agreed plan
|
||||
## Architecture Rules
|
||||
|
||||
Autonomy starts after alignment, not before it. For a new substantial assignment,
|
||||
recover the applicable mission, goal, task, `CURRENT.md` state, and prior owner
|
||||
decisions, then work with the user to establish a plan of action: the intended
|
||||
outcome, acceptance evidence, boundaries, and any gated actions. Recommend a
|
||||
concrete plan instead of presenting an open-ended menu. A direct request or
|
||||
existing approved plan that already settles those points is sufficient alignment;
|
||||
do not ask for ceremonial reconfirmation.
|
||||
1. Gateway is the single API surface — all clients connect through it
|
||||
2. Pi SDK is ESM-only — gateway and CLI must use ESM
|
||||
3. Socket.IO typed events defined in `@mosaicstack/types` enforce compile-time contracts
|
||||
4. OTEL auto-instrumentation loads before NestJS bootstrap
|
||||
5. BetterAuth manages auth tables; schema defined in `@mosaicstack/db`
|
||||
6. Docker Compose provides PG (5433), Valkey (6380), OTEL Collector (4317/4318), Jaeger (16686)
|
||||
7. Explicit `@Inject()` decorators required in NestJS (tsx/esbuild doesn't emit decorator metadata)
|
||||
|
||||
Once the plan is established, carry it to verified completion without prompting
|
||||
for routine decisions or permission to take the next in-scope step. Authorization
|
||||
persists for the life of that assignment unless the user changes or revokes it.
|
||||
Treat mid-session user input as steering: incorporate it, update the plan or
|
||||
tracking record when needed, and continue.
|
||||
## Development Workflow
|
||||
|
||||
### Decide and continue
|
||||
```bash
|
||||
docker compose up -d # Infrastructure
|
||||
pnpm install # Dependencies
|
||||
pnpm typecheck && pnpm lint && pnpm format:check # Quality gates
|
||||
```
|
||||
|
||||
- Resolve naming, implementation approach, layout, and similar non-breaking
|
||||
choices from, in order: repository invariants and role policy, the approved
|
||||
plan and acceptance criteria, established repository conventions, then the
|
||||
smallest reversible option. Record a consequential choice and its tradeoff.
|
||||
- Perform the in-scope investigation, edits, tests, documentation, and tracking
|
||||
needed for end-to-end acceptance. Do not ask whether to add obviously required
|
||||
tests or documentation.
|
||||
- Diagnose failures and retry or remediate within the agreed scope. Fix a defect
|
||||
when it blocks acceptance or is local to files already being changed; otherwise
|
||||
record a bounded follow-up without expanding the assignment.
|
||||
- Resolve minor ambiguity in favor of the mission, goal, north star, and prior
|
||||
owner decisions. State the assumption in the completion report.
|
||||
- Never stop merely to ask whether to proceed, which routine option to use, or
|
||||
whether to execute the next step already contained in the plan.
|
||||
## Repo-Specific Notes
|
||||
|
||||
### Re-align or stop only at a real boundary
|
||||
- DTOs in `*.dto.ts` files at module boundaries
|
||||
- ESM everywhere (`"type": "module"`, `.js` extensions in imports)
|
||||
- NodeNext module resolution in all tsconfigs
|
||||
- Scratchpads are mandatory for non-trivial tasks
|
||||
|
||||
Finish all independent work first, then ask one focused question only when:
|
||||
## docs/TASKS.md — Schema (CANONICAL)
|
||||
|
||||
1. Two plausible readings materially change the outcome and the choice is costly
|
||||
to reverse.
|
||||
2. The next action would exceed the agreed scope or authority, introduce an
|
||||
unapproved breaking public/API/schema/data/policy change, or alter a security
|
||||
boundary.
|
||||
3. Credentials or access are missing and no in-scope path remains.
|
||||
4. The action is destructive, irreversible, production-affecting, incurs spend,
|
||||
or communicates externally on the user's behalf without explicit authority.
|
||||
5. Objectives or owner decisions genuinely conflict and repository evidence
|
||||
cannot resolve them.
|
||||
6. A fail-closed policy refusal or another agent's overlapping ownership prevents
|
||||
safe progress. Diagnose and report it; never route around it.
|
||||
The `agent` column specifies the required model for each task. **This is set at task creation by the orchestrator and must not be changed by workers.**
|
||||
|
||||
Repository gates still apply. In particular, a successful implementation or a
|
||||
broad request to “finish” does not by itself authorize push, merge, deployment,
|
||||
release, production changes, policy/role expansion, or access to secrets. Perform
|
||||
such an action only when the established plan explicitly includes it. If blocked,
|
||||
report the exact boundary, what is complete, the recommended resolution, and the
|
||||
specific action that will resume; do not use “waiting for confirmation” as a
|
||||
substitute for a real blocker.
|
||||
| Value | When to use | Budget |
|
||||
| --------- | ----------------------------------------------------------- | -------------------------- |
|
||||
| `codex` | All coding tasks (default for implementation) | OpenAI credits — preferred |
|
||||
| `glm-5.1` | Cost-sensitive coding where Codex is unavailable | Z.ai credits |
|
||||
| `haiku` | Review gates, verify tasks, status checks, docs-only | Cheapest Claude tier |
|
||||
| `sonnet` | Complex planning, multi-file reasoning, architecture review | Claude quota |
|
||||
| `opus` | Major cross-cutting architecture decisions ONLY | Most expensive — minimize |
|
||||
| `—` | No preference / auto-select cheapest capable | Pipeline decides |
|
||||
|
||||
## Session protocol (mandatory)
|
||||
Pipeline crons read this column and spawn accordingly. Workers never modify `docs/TASKS.md` — only the orchestrator writes it.
|
||||
|
||||
- **Register** your session in `docs/SESSIONS.md` — one append-only line
|
||||
(date, actor, scope, outcome). Never rewrite or remove entries.
|
||||
- **Cadence**: run `scripts/mosaic queue next <your seat>` first. It names
|
||||
the row to resume, review or start, or says there is nothing. The goal order
|
||||
in `docs/plans/2026-09-27_goals-review.md` sets priority, not CURRENT.md.
|
||||
Open only the brief that row links to. Execute it through every authorized
|
||||
stage (implement → test → verify against acceptance criteria; commit, push,
|
||||
or close only when the established plan authorizes each) → move the row with
|
||||
`scripts/mosaic queue move` (never by editing QUEUE.md) → register in
|
||||
SESSIONS.md.
|
||||
- "next" means one action. A batch mandate ("run the queue") repeats the
|
||||
loop until green or truly blocked under the boundary rules above.
|
||||
- Substantial work gets a Gitea issue and a BUILD-LOG phase entry
|
||||
(before/after, with corrections recorded honestly).
|
||||
**Full schema:**
|
||||
|
||||
## Internal development bootstrap
|
||||
```
|
||||
| id | status | description | issue | agent | repo | branch | depends_on | estimate | notes |
|
||||
```
|
||||
|
||||
Jason's current direction is repository-native development in
|
||||
`/mnt/storage/src/mosaic-stack`. Sage leads the project (Jason's ruling,
|
||||
2026-09-26) and coordinates coding, review and research through Darkwing, Dewey,
|
||||
Filbert, Rocko, Researcher and any further seats Jason launches under `agents/`.
|
||||
Darkwing is a collaborating agent seat, not the coordinator. Development sessions
|
||||
run in T3 for now. Work moves to the new stack; the old `~/.mosaic` fleet is being
|
||||
retired, and a fleet seat acting outside Jason's instructions is the failure this
|
||||
transition exists to prevent.
|
||||
Do not assign new development work to fleet seats during this bootstrap phase.
|
||||
Do not modify `~/.mosaic` launchers, provisioning or other state, or stop/migrate
|
||||
live fleet processes as part of this work. Preserve existing work and histories.
|
||||
Use the repository bootstrap/configuration and launch entry points; missing
|
||||
configuration still fails closed. This changes development coordination, not
|
||||
managed worker role policy or deployment authority. The lead role adds no push,
|
||||
merge or deployment authority; those still need Jason's say-so. See
|
||||
`agents/README.md` for the internal roster.
|
||||
|
||||
For control-board attention, start a completed reply with `Input needed: ` and
|
||||
one specific nonempty request only when Jason must provide a decision or input.
|
||||
Put that line at column zero, before other text. Do not use it for routine
|
||||
completion or a wait on another agent. Ordinary completed replies are idle.
|
||||
Use code fences or blockquotes when showing this convention as an example.
|
||||
The signal is advisory status, never permission for a protected action. Seen
|
||||
acknowledges a request; it does not resolve it. See `packages/control-board/README.md`.
|
||||
|
||||
## Role model
|
||||
|
||||
- **Conductor**: a system-scoped role — not an agent, not a daemon. Holds
|
||||
git/credentials/policy authority; decomposes, dispatches, reviews,
|
||||
verifies, integrates. Protocol: `docs/plans/CONDUCTOR.md`. Exists only
|
||||
when invoked; push is never automatic.
|
||||
- **Workers**: headless pi via `scripts/run-task.sh` — sandboxed workspace,
|
||||
tools allowlist, optional persistent sessions and forks; no git, no
|
||||
credentials, no policy control.
|
||||
- Worker runs deliberately exclude this file (`--no-context-files` in the
|
||||
adapter): worker context is contracts + mission via the generated system
|
||||
prompt. This file is for conductor-level sessions.
|
||||
|
||||
## Command surface
|
||||
|
||||
`scripts/bootstrap.sh` (idempotent) · `build.sh` · `hello.sh` ·
|
||||
`verify.sh` · `run-task.sh run <task.json>` · `release.sh
|
||||
package|activate|rollback|status` · `auth.sh status|accounts` · `reset.sh` (**danger**: wipes the data
|
||||
root; triple-safety-checked) · `mosaic-task.mjs validate|run|show|list|retry|prune|resolve-role` ·
|
||||
`agent.sh <name>` (interactive TUI agent) ·
|
||||
suites: `test-config.sh`, `test-task.sh`, `test-release.sh`,
|
||||
`test-conductor.sh`, `test-auth.sh`, `test-discord.sh`, `test-queue.sh`.
|
||||
|
||||
Full reference — usage, fields, exit codes, safety notes:
|
||||
`docs/TOOLS.md` (read on demand; do not rely on this summary for detail).
|
||||
|
||||
## Data map (canon)
|
||||
|
||||
- `~/.config/mosaic-dev/config.json` — system config (user-authored; never
|
||||
auto-written).
|
||||
- `<dataRoot>` (from config; default `~/.mosaic-dev`):
|
||||
- `runs/` — write-once run evidence (`result.json`, snapshots, `stderr.txt`)
|
||||
- `sessions/` — pi JSONL session trees, one directory per named session
|
||||
- `workspaces/` — agent file effects (persistent or `:run` ephemeral)
|
||||
- `state/` — release pointer + append-only activation/auto-apply logs
|
||||
- Ownership is per-directory; nothing shares state. Directory map and
|
||||
lifecycle rules: README.md "Data map" section.
|
||||
|
||||
## Pointers (depth lives here)
|
||||
|
||||
- `docs/plans/2026-09-27_goals-review.md` — north star and goal order (Jason ratified 2026-09-27)
|
||||
- `docs/plans/QUEUE.md` — THE task list, rendered from `docs/plans/queue.json`
|
||||
(`scripts/mosaic queue next <seat>` reads it; `packages/queue/README.md` has the verbs)
|
||||
- `docs/plans/CURRENT.md` — narrative log behind the queue rows
|
||||
- `docs/plans/ROADMAP.md` — agreed milestone path (M16+)
|
||||
- `docs/plans/CONDUCTOR.md` — orchestration protocol and guardrails
|
||||
- `docs/plans/2026-09-02_atomic-mosaic-foundation.md` — architecture, invariants
|
||||
- `docs/plans/2026-09-03_autonomous-run.md` — batch-run tracker
|
||||
- `BUILD-LOG.md` — append-only build/verification history with corrections
|
||||
- `LAYERS.md` — implemented vs deferred layers
|
||||
- `docs/SESSIONS.md` — session registry
|
||||
- `adapters/README.md` — the harness adapter contract
|
||||
- `roles/` — role contracts (conductor, future agent/coder/reviewer)
|
||||
|
||||
## Recovery rule
|
||||
|
||||
Compacted, restarted, or new? Nothing that matters is lost: this file +
|
||||
`scripts/mosaic queue next <seat>` + `docs/plans/CURRENT.md` +
|
||||
`git log --oneline -10` + the suites reconstruct the full state. **Never
|
||||
guess** — verify with the suites; the run records and logs hold the receipts.
|
||||
|
||||
## Version pin
|
||||
|
||||
`@earendil-works/pi-coding-agent` is pinned exactly (see `package.json` /
|
||||
`RELEASE`); never install unversioned. Release identity: `RELEASE` file
|
||||
(0.0.X until declared stable); image tags derive from it.
|
||||
- `status`: `not-started` | `in-progress` | `done` | `failed` | `blocked` | `needs-qa`
|
||||
- `agent`: model value from table above (set before spawning)
|
||||
- `estimate`: token budget e.g. `8K`, `25K`
|
||||
|
||||
@@ -1,371 +0,0 @@
|
||||
# Minimal Mosaic Stack container proof of concept
|
||||
|
||||
## Purpose
|
||||
|
||||
Build the smallest isolated container that can:
|
||||
- launch Pi
|
||||
- load a small set of Mosaic-style contract files
|
||||
- send one real request to a model
|
||||
- return a known response.
|
||||
|
||||
This is a standalone experiment. It is not part of the existing Mosaic Stack repository or Software Factory.
|
||||
|
||||
## Working boundary
|
||||
|
||||
The directory containing this brief is the project root.
|
||||
|
||||
### Do not read, copy, mount, import, or modify anything from:
|
||||
- `/home/jwoltje/.mosaic`
|
||||
- `/home/jwoltje/.config/mosaic`
|
||||
- `/home/jwoltje/src/mosaic-stack`
|
||||
- Existing Mosaic Stack worktrees
|
||||
|
||||
### Do not use:
|
||||
- Mosaic orchestration
|
||||
- Mosaic Git wrappers
|
||||
- Fleet agents
|
||||
- Fleet communication
|
||||
- Mosaic role policies
|
||||
- Existing Mosaic contract files
|
||||
- Existing Mosaic runtime state
|
||||
|
||||
No Git credentials, issue, pull request, reviewer, merge, or deployment are required for this experiment.
|
||||
|
||||
Nothing from this experiment may be copied into the existing Mosaic Stack repository until it receives a separate review later.
|
||||
|
||||
## Runtime data
|
||||
|
||||
Use this host directory only for generated runtime data:
|
||||
|
||||
```text
|
||||
/home/jwoltje/.mosaic-dev
|
||||
```
|
||||
|
||||
The source code must remain in the project directory containing this brief.
|
||||
|
||||
Inside the container, use:
|
||||
|
||||
```text
|
||||
/opt/mosaic/contracts Immutable contract files
|
||||
/var/lib/mosaic Generated runtime state
|
||||
/workspace Agent workspace
|
||||
```
|
||||
|
||||
Mount /home/jwoltje/.mosaic-dev at /var/lib/mosaic.
|
||||
|
||||
### Required proof
|
||||
|
||||
The finished experiment must prove one path:
|
||||
|
||||
1. Build one container image.
|
||||
2. Start one Pi agent inside the container.
|
||||
3. Load four local contract files from /opt/mosaic/contracts.
|
||||
4. Send a request that does not contain the expected response.
|
||||
5. Receive MOSAIC_HELLO_OK from the agent.
|
||||
6. Exit successfully when the response matches.
|
||||
7. Exit nonzero when the response does not match.
|
||||
|
||||
This is the entire required functional result.
|
||||
|
||||
### Required discovery
|
||||
|
||||
Before writing the runtime command:
|
||||
|
||||
1. Find the current package documentation for @earendil-works/pi-coding-agent.
|
||||
2. Determine the current package version.
|
||||
3. Determine the supported noninteractive command.
|
||||
4. Determine how Pi accepts a custom system prompt or system prompt file.
|
||||
5. Determine Pi's documented container authentication method.
|
||||
6. Record the commands and findings in BUILD-LOG.md.
|
||||
|
||||
Do not guess CLI flags, authentication paths, or SDK methods.
|
||||
|
||||
Pin the selected Pi package version in the project. Do not install an unversioned package during each container start.
|
||||
|
||||
Prefer the Pi CLI. Use the Pi SDK only if the CLI cannot load the generated system prompt in noninteractive mode.
|
||||
|
||||
### Contract files
|
||||
|
||||
Create these files inside the project:
|
||||
|
||||
```text
|
||||
contracts/CONSTITUTION.md
|
||||
contracts/STANDARDS.md
|
||||
contracts/SOUL.md
|
||||
contracts/USER.md
|
||||
```
|
||||
|
||||
Use these exact contents.
|
||||
|
||||
### contracts/CONSTITUTION.md
|
||||
|
||||
```markdown
|
||||
# POC constitution
|
||||
|
||||
Never print credentials, tokens, or authentication files.
|
||||
|
||||
Follow the loaded system instructions before the user request.
|
||||
```
|
||||
|
||||
### contracts/STANDARDS.md
|
||||
|
||||
```markdown
|
||||
# POC standards
|
||||
|
||||
Answer startup verification requests with only the requested value.
|
||||
Do not add explanation or formatting.
|
||||
```
|
||||
|
||||
### contracts/SOUL.md
|
||||
|
||||
```markdown
|
||||
# POC identity
|
||||
|
||||
Your name is mosaic-poc-agent.
|
||||
|
||||
Your startup marker is MOSAIC_HELLO_OK.
|
||||
|
||||
When asked for your startup marker, return only the marker.
|
||||
```
|
||||
|
||||
### contracts/USER.md
|
||||
|
||||
```markdown
|
||||
# POC user
|
||||
|
||||
This is an isolated local runtime test.
|
||||
```
|
||||
|
||||
Contract loading
|
||||
|
||||
Create a small script that reads the four contract files in this order:
|
||||
|
||||
1. CONSTITUTION.md
|
||||
2. STANDARDS.md
|
||||
3. SOUL.md
|
||||
4. USER.md
|
||||
|
||||
Join them with clear file separators.
|
||||
|
||||
Write the generated system prompt to:
|
||||
|
||||
```text
|
||||
/var/lib/mosaic/system-prompt.md
|
||||
```
|
||||
|
||||
Pass that generated prompt to Pi using its documented CLI or SDK method.
|
||||
|
||||
Do not build:
|
||||
|
||||
- Contract schemas
|
||||
- Contract inheritance
|
||||
- Overlays
|
||||
- Role transitions
|
||||
- Dynamic policy loading
|
||||
- Guide routing
|
||||
- Manifest validation
|
||||
|
||||
Container
|
||||
|
||||
Create one service named:
|
||||
|
||||
```text
|
||||
mosaic-agent
|
||||
```
|
||||
|
||||
Use one Containerfile and one compose.yaml.
|
||||
|
||||
Requirements:
|
||||
|
||||
- Use a maintained Node.js base image.
|
||||
- Run as a non-root user.
|
||||
- Install a pinned Pi package version.
|
||||
- Copy the local contract fixtures into /opt/mosaic/contracts.
|
||||
- Do not copy credentials into the image.
|
||||
- Do not mount the Docker socket.
|
||||
- Do not mount either live Mosaic directory.
|
||||
- Do not add a database, web server, queue, or second container.
|
||||
- The container may run as a one-shot command. It does not need to remain running.
|
||||
|
||||
### Authentication
|
||||
|
||||
Use Pi's documented authentication mechanism.
|
||||
|
||||
Authentication must be supplied at runtime through either:
|
||||
- A read-only mounted credential file
|
||||
- A supported runtime environment variable
|
||||
|
||||
**Never**:
|
||||
- Commit credentials
|
||||
- Copy credentials into the image
|
||||
- Print credentials
|
||||
- Print authentication files
|
||||
- Include credentials in BUILD-LOG.md
|
||||
- Store credentials under the project directory
|
||||
|
||||
Provide .env.example only for non-secret settings such as model or provider names.
|
||||
|
||||
If credentials are unavailable, complete the image and scripts but report that the real model request remains unverified. Do not fake the response.
|
||||
|
||||
### Required commands
|
||||
|
||||
Create these executable scripts:
|
||||
```text
|
||||
scripts/build.sh
|
||||
scripts/hello.sh
|
||||
scripts/verify.sh
|
||||
scripts/reset.sh
|
||||
```
|
||||
|
||||
### scripts/build.sh
|
||||
|
||||
Build the container image using Docker Compose.
|
||||
|
||||
### scripts/hello.sh
|
||||
|
||||
Run the mosaic-agent service as a one-shot container.
|
||||
|
||||
Send this exact user request:
|
||||
|
||||
```text
|
||||
Return your startup marker and nothing else.
|
||||
```
|
||||
|
||||
The request must not contain MOSAIC_HELLO_OK.
|
||||
|
||||
Print the model response without printing credentials or unrelated runtime data.
|
||||
|
||||
### scripts/verify.sh
|
||||
|
||||
Run the complete test.
|
||||
|
||||
**It must**:
|
||||
|
||||
1. Build or confirm the image is built.
|
||||
2. Run the agent request.
|
||||
3. Remove surrounding whitespace from the response.
|
||||
4. Compare the response with MOSAIC_HELLO_OK.
|
||||
5. Exit 0 only when they match exactly.
|
||||
6. Exit nonzero with a clear error when they do not match.
|
||||
|
||||
### scripts/reset.sh
|
||||
|
||||
Delete generated POC state only when all checks pass:
|
||||
1. The resolved path is exactly /home/jwoltje/.mosaic-dev.
|
||||
2. The path is not a symbolic link.
|
||||
3. The directory contains a .mosaic-poc-root ownership marker created by this project.
|
||||
|
||||
Refuse to delete anything if a check fails.
|
||||
|
||||
## Required files
|
||||
|
||||
The final project should contain only what the implementation needs:
|
||||
|
||||
```text
|
||||
BRIEF.md
|
||||
BUILD-LOG.md
|
||||
README.md
|
||||
LAYERS.md
|
||||
Containerfile
|
||||
compose.yaml
|
||||
package.json
|
||||
package-lock.json
|
||||
.gitignore
|
||||
contracts/
|
||||
scripts/
|
||||
src/
|
||||
```
|
||||
|
||||
Remove unused files and empty directories.
|
||||
|
||||
Build log
|
||||
|
||||
Create BUILD-LOG.md.
|
||||
|
||||
Treat it as append-only.
|
||||
|
||||
Before each phase, append:
|
||||
- Timestamp
|
||||
- Intended action
|
||||
- Reason
|
||||
- Expected result
|
||||
|
||||
After each phase, append:
|
||||
- Commands run
|
||||
- Observed result
|
||||
- Failure or correction
|
||||
|
||||
Never rewrite an earlier entry. Add a correction as a new entry.
|
||||
|
||||
Do not record credentials.
|
||||
|
||||
Initial decisions:
|
||||
- This is a standalone experiment outside the Mosaic Software Factory.
|
||||
- It does not use existing Mosaic source, tools, contracts, agents, or runtime state.
|
||||
- The first proof uses one Pi agent and four small local contract files.
|
||||
- The only required model result is MOSAIC_HELLO_OK.
|
||||
- Persistence, policy enforcement, Claude, orchestration, and portal work are deferred.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
The experiment passes when:
|
||||
1. scripts/build.sh exits 0.
|
||||
2. The image contains the four local contract files.
|
||||
3. The image contains no credentials.
|
||||
4. The container has no mounts from ~/.mosaic or ~/.config/mosaic.
|
||||
5. scripts/hello.sh performs a real model request.
|
||||
6. The request does not contain the expected marker.
|
||||
7. The agent returns exactly MOSAIC_HELLO_OK.
|
||||
8. scripts/verify.sh exits 0.
|
||||
9. Changing the expected value makes scripts/verify.sh exit nonzero.
|
||||
10. scripts/reset.sh refuses unsafe paths.
|
||||
11. Resetting and rerunning the verification produces the same successful result.
|
||||
|
||||
## Deferred layers
|
||||
|
||||
Document these in LAYERS.md. Do not implement them.
|
||||
|
||||
- L0: Container builds and returns MOSAIC_HELLO_OK.
|
||||
- L1: Persist and resume a named Pi session.
|
||||
- L2: Add a fixed tool permission policy.
|
||||
- L3: Load full versioned contract bundles.
|
||||
- L4: Add Claude as a second runtime.
|
||||
- L5: Add multiple agents and communication.
|
||||
- L6: Add orchestration, knowledge storage, and portal features.
|
||||
|
||||
## Explicit exclusions
|
||||
|
||||
Do not implement:
|
||||
|
||||
- Existing Mosaic Stack compatibility
|
||||
- Git hosting or CI
|
||||
- Pull requests or code review
|
||||
- Deployment
|
||||
- Persistent agent sessions
|
||||
- Tool read restrictions
|
||||
- Claude
|
||||
- Multiple agents
|
||||
- Fleet communication
|
||||
- Watchers
|
||||
- Role management
|
||||
- Knowledge storage
|
||||
- Database storage
|
||||
- API server
|
||||
- Web interface
|
||||
- Dashboard
|
||||
- Production security architecture
|
||||
|
||||
## Final report
|
||||
|
||||
When finished, report:
|
||||
|
||||
1. Files created.
|
||||
2. Pi package version.
|
||||
3. Exact build command.
|
||||
4. Exact verification command.
|
||||
5. Verification output with credentials removed.
|
||||
6. Whether the real model request passed.
|
||||
7. Any remaining failure.
|
||||
8. Anything implemented beyond this brief.
|
||||
|
||||
Do not describe the experiment as production-ready.
|
||||
-3421
File diff suppressed because it is too large
Load Diff
@@ -1 +1,45 @@
|
||||
@AGENTS.md
|
||||
# CLAUDE.md — Mosaic Stack
|
||||
|
||||
## Project
|
||||
|
||||
Self-hosted, multi-user AI agent platform. TypeScript monorepo.
|
||||
|
||||
## Stack
|
||||
|
||||
- **API**: NestJS + Fastify adapter (`apps/gateway`)
|
||||
- **Web**: Next.js 16 + React 19 (`apps/web`)
|
||||
- **ORM**: Drizzle ORM + PostgreSQL 17 + pgvector (`packages/db`)
|
||||
- **Auth**: BetterAuth (`packages/auth`)
|
||||
- **Agent**: Pi SDK (`packages/agent`, `packages/mosaic`)
|
||||
- **Queue**: Valkey 8 (`packages/queue`)
|
||||
- **Build**: pnpm workspaces + Turborepo
|
||||
- **CI**: Woodpecker CI
|
||||
- **Observability**: OpenTelemetry → Jaeger
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
pnpm typecheck # TypeScript check (all packages)
|
||||
pnpm lint # ESLint (all packages)
|
||||
pnpm format:check # Prettier check
|
||||
pnpm test # Vitest (all packages)
|
||||
pnpm build # Build all packages
|
||||
|
||||
# Database
|
||||
pnpm --filter @mosaicstack/db db:push # Push schema to PG (dev)
|
||||
pnpm --filter @mosaicstack/db db:generate # Generate migrations
|
||||
pnpm --filter @mosaicstack/db db:migrate # Run migrations
|
||||
|
||||
# Dev
|
||||
docker compose up -d # Start PG, Valkey, OTEL, Jaeger
|
||||
pnpm --filter @mosaicstack/gateway exec tsx src/main.ts # Start gateway
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- ESM everywhere (`"type": "module"`, `.js` extensions in imports)
|
||||
- NodeNext module resolution
|
||||
- Explicit `@Inject()` decorators in NestJS (tsx/esbuild doesn't support emitDecoratorMetadata)
|
||||
- DTOs in `*.dto.ts` files at module boundaries
|
||||
- OTEL tracing imported before NestJS bootstrap (`import './tracing.js'`)
|
||||
- All three gates must pass before push: typecheck, lint, format:check
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
# Minimal Mosaic Stack POC agent image.
|
||||
# Base: maintained Node.js image (same family as Pi's documented
|
||||
# containerization example in docs/containerization.md).
|
||||
FROM node:24-bookworm-slim
|
||||
|
||||
# Tools Pi's documented container image expects (bash, CA certs, git, ripgrep).
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends bash ca-certificates git ripgrep \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Non-root user: the maintained node image ships a 'node' user at
|
||||
# uid/gid 1000, which matches the host user that owns the runtime
|
||||
# state directory mounted at /var/lib/mosaic. It is reused as-is.
|
||||
|
||||
# Pinned Pi install: package.json pins the exact version and
|
||||
# package-lock.json is installed with npm ci. No unversioned installs.
|
||||
WORKDIR /opt/app
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm ci --ignore-scripts
|
||||
|
||||
# Immutable contract fixtures (required location), runtime scripts, and
|
||||
# runtime adapters.
|
||||
COPY contracts /opt/mosaic/contracts
|
||||
COPY src /opt/mosaic/src
|
||||
COPY adapters /opt/mosaic/adapters
|
||||
RUN chmod 0555 /opt/mosaic/contracts /opt/mosaic/contracts/* \
|
||||
&& chmod 0555 /opt/mosaic/src /opt/mosaic/src/*.sh \
|
||||
&& chmod 0555 /opt/mosaic/adapters /opt/mosaic/adapters/*/adapter.sh
|
||||
|
||||
# Writable state, workspace, and pi agent directory (auth.json is
|
||||
# bind-mounted read-only at runtime; nothing is copied into the image).
|
||||
RUN mkdir -p /var/lib/mosaic /workspace /home/node/.pi/agent \
|
||||
&& chown -R node:node /var/lib/mosaic /workspace /home/node /opt/app
|
||||
|
||||
USER node
|
||||
WORKDIR /workspace
|
||||
ENV HOME=/home/node \
|
||||
PATH="/opt/app/node_modules/.bin:${PATH}" \
|
||||
PI_OFFLINE=1
|
||||
|
||||
# One-shot agent: args form the user request (default is the startup
|
||||
# verification request defined in compose.yaml).
|
||||
ENTRYPOINT ["/opt/mosaic/src/run-agent.sh"]
|
||||
@@ -22,13 +22,10 @@
|
||||
FROM node:24-alpine
|
||||
|
||||
# Native toolchain required to compile node-gyp deps on musl, plus the
|
||||
# postgresql-client used by the test step's pg_isready readiness probe. `bash`,
|
||||
# `git`, and `jq` are baked here too — framework shell tests and the shipped
|
||||
# Codex review wrappers require them without per-run installation in ci.yml.
|
||||
# `openssl` (#912) is the non-circular HMAC signer for the wake trust layer:
|
||||
# the digest H1/H2, beacon B12, and install I8 legs hard-require it in CI so the
|
||||
# §4 G6 evidence comes from an actually-run HMAC leg, not a skipped one.
|
||||
RUN apk add --no-cache python3 make g++ postgresql-client bash git jq openssl
|
||||
# postgresql-client used by the test step's pg_isready readiness probe. `bash`
|
||||
# is baked here too — the sanitization step in ci.yml otherwise does a per-run
|
||||
# `apk add bash`.
|
||||
RUN apk add --no-cache python3 make g++ postgresql-client bash
|
||||
|
||||
# Pin pnpm to the repo's packageManager version via corepack.
|
||||
RUN corepack enable && corepack prepare [email protected] --activate
|
||||
@@ -1,50 +0,0 @@
|
||||
# LAYERS
|
||||
|
||||
Deferred capability layers for the Mosaic experiment. Only L0 is implemented by
|
||||
this proof of concept; everything below it is documented here and deliberately
|
||||
not implemented (see BRIEF.md, "Explicit exclusions").
|
||||
|
||||
## L0 — Implemented: container returns MOSAIC_HELLO_OK
|
||||
|
||||
One image (`mosaic-poc-agent:0.84.4`, built on `node:24-bookworm-slim`, non-root,
|
||||
pinned Pi) runs one Pi agent one-shot. Four immutable local contract files are
|
||||
loaded in fixed order into the generated system prompt
|
||||
(`/var/lib/mosaic/system-prompt.md`). One real model request is sent
|
||||
noninteractively; the response must equal `MOSAIC_HELLO_OK` exactly or the
|
||||
verification exits nonzero. Authentication is supplied at runtime only
|
||||
(read-only mounted pi auth file, or a provider API key environment variable).
|
||||
|
||||
## L1 — Deferred: persist and resume a named Pi session
|
||||
|
||||
Keep a named Pi session across container runs (`--name`, session storage under
|
||||
`/var/lib/mosaic`), resume it with the documented session flags, and verify
|
||||
state survives a container restart.
|
||||
|
||||
## L2 — Deferred: fixed tool permission policy
|
||||
|
||||
Add a fixed allow/deny policy for Pi tools (e.g. restricting built-in tools via
|
||||
documented `--tools` / `--exclude-tools` or an extension-based permission gate),
|
||||
so contract files can constrain what the agent may do, not just what it says.
|
||||
|
||||
## L3 — Deferred: load full versioned contract bundles
|
||||
|
||||
Replace the four static fixtures with versioned contract bundles: bundle
|
||||
manifests, contract versions, and deterministic ordering/hashing, loaded from
|
||||
an immutable bundle artifact instead of files copied at image build time.
|
||||
|
||||
## L4 — Deferred: Claude as a second runtime
|
||||
|
||||
Add a second runtime (Claude) alongside the Pi agent in the same container
|
||||
stack, behind the same contract-loading path, to compare behavior across
|
||||
runtimes.
|
||||
|
||||
## L5 — Deferred: multiple agents and communication
|
||||
|
||||
Run several named agents with defined roles and a communication channel between
|
||||
them (message passing or shared state under `/var/lib/mosaic`).
|
||||
|
||||
## L6 — Deferred: orchestration, knowledge storage, and portal features
|
||||
|
||||
Fleet-level orchestration, knowledge storage, monitoring, and portal UI on top
|
||||
of L1-L5. This is where the existing Mosaic Stack concepts would be re-evaluated
|
||||
from first principles.
|
||||
@@ -1,238 +1,364 @@
|
||||
# Mosaic Stack — new foundation
|
||||
# Mosaic Stack
|
||||
|
||||
The active rebuild is at this repository's root. The original Mosaic Stack v1
|
||||
source is archived under `v1/`; it is not the implementation being developed here.
|
||||
Self-hosted, multi-user AI agent platform. One config, every runtime, same standards.
|
||||
|
||||
- Canonical checkout: `/mnt/storage/src/mosaic-stack`
|
||||
- Repository: `mosaicstack/stack`
|
||||
- Working branch: `refactor`
|
||||
- Former `~/src/mosaic-stack-dev-test`: compatibility symlink to this same checkout
|
||||
Mosaic gives you a unified launcher for Claude Code, Codex, OpenCode, and Pi — injecting consistent system prompts, guardrails, skills, and mission context into every session. A NestJS gateway provides the API surface, a Next.js dashboard gives you the UI, and a plugin system connects Discord, Telegram, and more.
|
||||
|
||||
Both original Git histories and pending development work are preserved. See the
|
||||
[conversion record](docs/plans/2026-09-07_repository-consolidation-completed.md)
|
||||
and [current next action](docs/plans/CURRENT.md). Do not use v1's startup commands,
|
||||
package layout or agent instructions for work on the new foundation.
|
||||
|
||||
## Original container proof
|
||||
|
||||
The foundation began as a standalone container experiment. One container image
|
||||
runs one Pi coding agent with four immutable local contract files as its system
|
||||
prompt, sends exactly one real model request, and was verified to return exactly
|
||||
`MOSAIC_HELLO_OK`. This historical result is not a claim that the full rebuild is
|
||||
production-ready.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
BRIEF.md requirements for the original container proof
|
||||
BUILD-LOG.md append-only build/verification log
|
||||
LAYERS.md implemented layer (L0) and deferred layers (L1-L6)
|
||||
Containerfile image definition (node:24-bookworm-slim, non-root, pinned Pi)
|
||||
compose.yaml one service: mosaic-agent (one-shot; configured via env)
|
||||
package.json pins @earendil-works/pi-coding-agent at exactly 0.84.4
|
||||
package-lock.json resolved lockfile used by npm ci in the image
|
||||
.env.example non-secret settings only (credential-file path, env-var auth)
|
||||
contracts/ CONSTITUTION.md, STANDARDS.md, SOUL.md, USER.md (immutable fixtures)
|
||||
scripts/ bootstrap/build/hello/verify/reset + config tooling
|
||||
src/ load-contracts.sh, run-agent.sh (run inside the container)
|
||||
docs/plans/ architecture and milestone plans
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The sole discovery entry point is:
|
||||
|
||||
```text
|
||||
~/.config/mosaic-dev/config.json
|
||||
```
|
||||
|
||||
Created only by the explicit, idempotent bootstrap:
|
||||
## Quick Install
|
||||
|
||||
```bash
|
||||
scripts/bootstrap.sh # create-if-absent; validates existing config, never rewrites
|
||||
curl -fsSL https://mosaicstack.dev/install.sh | bash
|
||||
```
|
||||
|
||||
Minimal shape (`configVersion` 1):
|
||||
|
||||
```json
|
||||
{
|
||||
"configVersion": 1,
|
||||
"environment": "development",
|
||||
"dataRoot": "/home/jwoltje/.mosaic-dev",
|
||||
"execution": {
|
||||
"backend": "docker",
|
||||
"provider": "zai",
|
||||
"model": "glm-5.3-flash"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules enforced by `scripts/mosaic-config.mjs`:
|
||||
|
||||
- Unknown keys, unsupported versions/backends, and malformed JSON exit nonzero; nothing is modified.
|
||||
- `dataRoot` must be absolute, canonical, and must not be or contain the home or configuration directory.
|
||||
- Validation failures never touch config, state, or images.
|
||||
- `scripts/test-config.sh` runs the sandboxed config selftests (no Docker required).
|
||||
|
||||
Run paths (`build/hello/verify/reset`) fail closed when configuration is missing or invalid; they never invent it.
|
||||
|
||||
## Missions & tasks (M2)
|
||||
|
||||
Missions and tasks are validated JSON data (strict schemas, version-pinned). The M2 layer is host-side only: mission directives are recorded for provenance but do not yet reach the runtime system prompt (capability/policy layer comes later).
|
||||
|
||||
```text
|
||||
missions/hello.json objective + directives (missionVersion 1)
|
||||
tasks/hello-marker.json prompt + optional mission ref + expectExact + timeout
|
||||
<dataRoot>/runs/r-<id>/ immutable run record: task.json, mission.json,
|
||||
stderr.txt, result.json (all write-once)
|
||||
```
|
||||
|
||||
Usage:
|
||||
Or use the direct URL:
|
||||
|
||||
```bash
|
||||
scripts/run-task.sh validate tasks/hello-marker.json # strict validation, writes nothing
|
||||
scripts/run-task.sh run tasks/hello-marker.json # execute; result recorded under dataRoot/runs
|
||||
scripts/mosaic-task.mjs list # list runs and statuses
|
||||
scripts/test-task.sh # selftests (schema negatives + live runs)
|
||||
bash <(curl -fsSL https://git.mosaicstack.dev/mosaicstack/stack/raw/branch/main/tools/install.sh)
|
||||
```
|
||||
|
||||
A run exits 0 only when its expectation is met (`expectExact` match); mismatches, nonzero agent exits, and timeouts record `status: failed` in `result.json` and exit 1. Each run gets a unique directory — rerunning never rewrites history.
|
||||
|
||||
## Release model (M3)
|
||||
|
||||
`RELEASE` single-sources the release version (0.0.X until declared stable); the image tag derives from it plus the pinned Pi version. Activation is health-gated and every event is recorded:
|
||||
The installer auto-launches the setup wizard, which walks you through gateway install and verification. Flags for non-interactive use:
|
||||
|
||||
```bash
|
||||
scripts/release.sh package # build + tag the release image
|
||||
scripts/release.sh activate # health check (exact marker) -> atomic pointer swap
|
||||
scripts/release.sh activate --fault-injection # prove the refusal path (drills only)
|
||||
scripts/release.sh rollback # health-gated return to the previous release
|
||||
scripts/release.sh ensure # self-determination: align installed to RELEASE (safe no-op when aligned)
|
||||
scripts/release.sh status # release, tag, active pointer, recent log
|
||||
scripts/test-release.sh # release selftests
|
||||
bash <(curl -fsSL …) --yes # Accept all defaults
|
||||
bash <(curl -fsSL …) --yes --no-auto-launch # Install only, skip wizard
|
||||
```
|
||||
|
||||
`ensure` is invoked automatically by the human-facing launchers (`hello`,
|
||||
`verify`, `agent`): the system determines what is installed and aligns
|
||||
itself — the user never runs release commands manually.
|
||||
This installs both components:
|
||||
|
||||
- `<dataRoot>/state/active.json` — the activation pointer (atomic tmp+rename replace)
|
||||
- `<dataRoot>/state/activation-log.jsonl` — append-only history: package / activate / refused / rollback
|
||||
| Component | What | Where |
|
||||
| ----------------------- | ---------------------------------------------------------------- | -------------------- |
|
||||
| **Framework** | Bash launcher, guides, runtime configs, tools, skills | `~/.config/mosaic/` |
|
||||
| **@mosaicstack/mosaic** | Unified `mosaic` CLI — TUI, gateway client, wizard, auto-updater | `~/.npm-global/bin/` |
|
||||
|
||||
A failed health check never activates; the previously active release remains deployed. Updating the software therefore cannot corrupt the running installation: package beside, gate, then flip. Verified by the update/refusal/rollback drills in BUILD-LOG Phase 7.
|
||||
|
||||
## Runtime adapters (M4)
|
||||
|
||||
The harness boundary is formalized: everything upstream (config, contracts, missions, tasks, run records) is harness-agnostic; everything inside an adapter belongs to one runtime.
|
||||
|
||||
```text
|
||||
adapters/<name>/adapter.sh env in: MOSAIC_SYSTEM_PROMPT_FILE, MOSAIC_REQUEST,
|
||||
MOSAIC_PROVIDER, MOSAIC_MODEL
|
||||
stdout: response only; stderr: diagnostics
|
||||
```
|
||||
|
||||
- Selection: `execution.adapter` in config.json (optional; `pi` default; allowlist `pi`, `mock`)
|
||||
- `pi` — pinned Pi CLI, noninteractive print mode, ambient discovery off
|
||||
- `mock` — deterministic test adapter; never for real verification
|
||||
- Mission directives have a sanctioned injection point: when a task references a mission, the task runner mounts the run snapshot and the generated prompt gains a `MISSION (runtime)` section (objective + directives) after the four immutable contracts
|
||||
- Adding a harness (Claude, Codex, OpenCode) later means adding one directory — no orchestrator changes
|
||||
|
||||
See `adapters/README.md` for the full contract.
|
||||
|
||||
## Workspaces, capabilities, sessions (M5/M6)
|
||||
|
||||
Optional task fields extend what an agent can do — all defaulting to the previous behavior:
|
||||
|
||||
```json
|
||||
{
|
||||
"workspace": "demo", // ":run" ephemeral, or persistent dataRoot/workspaces/<name>
|
||||
"capabilities": { "tools": ["bash", "read"] }, // pi tool allowlist; absent = no tools
|
||||
"session": "demo" // persistent session at dataRoot/sessions/<name>
|
||||
}
|
||||
```
|
||||
|
||||
- The adapter runs inside the workspace; files it writes are host-visible (`dataRoot/workspaces/<name>`).
|
||||
- Sessions persist via pi's documented `--session-dir`; a follow-up run in the same session resumes the conversation (`-c`) and can recall prior context. Distinct names never share state. Ephemeral (`--no-session`) remains the default when no session is declared.
|
||||
- Selection authority: config for adapter/provider/model; the task file for workspace/capabilities/session.
|
||||
|
||||
Inspect anything:
|
||||
After install, the wizard runs automatically or you can invoke it manually:
|
||||
|
||||
```bash
|
||||
node scripts/mosaic-task.mjs list # runs with task/workspace/session columns
|
||||
node scripts/mosaic-task.mjs show <runId> # full record + snapshots + artifacts
|
||||
mosaic wizard # Full guided setup (gateway install → verify)
|
||||
```
|
||||
|
||||
Demo fixtures: `tasks/workspace-demo.json`, `tasks/session-demo-1.json` + `tasks/session-demo-2.json`.
|
||||
### Requirements
|
||||
|
||||
See `docs/plans/2026-09-02_atomic-mosaic-foundation.md` for the full plan.
|
||||
|
||||
Inside the container:
|
||||
|
||||
```text
|
||||
/opt/mosaic/contracts immutable contract files
|
||||
/var/lib/mosaic generated runtime state (mounted from configured dataRoot)
|
||||
/workspace agent workspace
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
1. `scripts/build.sh` builds the release image (`mosaic-poc-agent:<pi>-r<release>`,
|
||||
tag derived from `RELEASE` + the pinned Pi version) with Docker Compose.
|
||||
2. On each run, `/opt/mosaic/src/load-contracts.sh` reads the four contract files
|
||||
in fixed order (CONSTITUTION, STANDARDS, SOUL, USER), joins them with clear
|
||||
separators, and writes `/var/lib/mosaic/system-prompt.md`.
|
||||
3. `/opt/mosaic/src/run-agent.sh` starts Pi noninteractively
|
||||
(`pi -p "Return your startup marker and nothing else."`) with
|
||||
`--system-prompt "$(cat /var/lib/mosaic/system-prompt.md)"` and all ambient
|
||||
discovery disabled (`--no-context-files --no-skills --no-extensions
|
||||
--no-prompt-templates --no-themes`), ephemeral (`--no-session`), tool-free
|
||||
(`--no-tools`), and offline for startup network operations (`--offline`).
|
||||
4. `scripts/verify.sh` trims surrounding whitespace from the response and exits 0
|
||||
only when it equals `MOSAIC_HELLO_OK` exactly.
|
||||
- Node.js ≥ 20
|
||||
- npm (for global @mosaicstack/mosaic install)
|
||||
- One or more runtimes: [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), [OpenCode](https://opencode.ai), or [Pi](https://github.com/mariozechner/pi-coding-agent)
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
scripts/bootstrap.sh # create config.json if absent (idempotent)
|
||||
scripts/build.sh # build the image
|
||||
scripts/hello.sh # one-shot request; prints the model response
|
||||
scripts/verify.sh # full gated test; exit 0 only on exact MOSAIC_HELLO_OK
|
||||
scripts/run-task.sh # run a mission/task file (see Missions & tasks)
|
||||
scripts/release.sh # package / activate / rollback / status (see Release model)
|
||||
scripts/test-config.sh # fast config-layer selftests (no Docker)
|
||||
scripts/test-task.sh # mission/task selftests (schema + adapter seam + live runs)
|
||||
scripts/test-release.sh # release selftests
|
||||
scripts/reset.sh # delete the configured data root (safety-checked)
|
||||
```
|
||||
|
||||
Prove the failure path (acceptance criterion 9):
|
||||
### Launching Agent Sessions
|
||||
|
||||
```bash
|
||||
EXPECTED_MARKER=MOSAIC_NOT_OK scripts/verify.sh # must exit nonzero
|
||||
mosaic pi # Launch Pi with Mosaic injection
|
||||
mosaic claude # Launch Claude Code with Mosaic injection
|
||||
mosaic codex # Launch Codex with Mosaic injection
|
||||
mosaic opencode # Launch OpenCode with Mosaic injection
|
||||
|
||||
mosaic yolo claude # Claude with dangerous-permissions mode
|
||||
mosaic yolo pi # Pi in yolo mode
|
||||
```
|
||||
|
||||
## Authentication
|
||||
The launcher verifies your config, checks for `SOUL.md`, injects your `AGENTS.md` standards into the runtime, and forwards all arguments.
|
||||
|
||||
Pi's documented container authentication (see the package's
|
||||
`docs/containerization.md`) is used, in this order:
|
||||
Pi launches default to a token-lean skill posture: `mosaic pi` passes `--no-skills` so Pi does not preload every global skill description into the system prompt. Use `MOSAIC_PI_SKILL_MODE=all mosaic pi` for the legacy all-skills catalog, or `MOSAIC_PI_SKILL_MODE=discover mosaic pi` to let Pi use its native settings/project skill discovery.
|
||||
|
||||
1. **Read-only mounted credential file** (default): the host pi auth file
|
||||
`~/.pi/agent/auth.json` is bind-mounted read-only to
|
||||
`/home/node/.pi/agent/auth.json`. The host file holds a static API-key
|
||||
entry for the built-in `zai` provider, so no token refresh writes are needed.
|
||||
2. **Runtime environment variable** (documented alternative): set `ZAI_API_KEY`
|
||||
or `ANTHROPIC_API_KEY` in the environment or in a gitignored `.env`; compose
|
||||
passes them through. Pi's documented precedence applies.
|
||||
### TUI & Gateway
|
||||
|
||||
Credentials are never committed, never copied into the image, and never printed.
|
||||
Mosaic-managed named accounts (`agent.sh --auth`) live under the data root
|
||||
(`auth/<account>.json`, 0600) — the stack never writes into `~/.pi`.
|
||||
`.env.example` contains non-secret settings only.
|
||||
```bash
|
||||
mosaic tui # Interactive TUI connected to the gateway
|
||||
mosaic gateway login # Authenticate with a gateway instance
|
||||
mosaic sessions list # List active agent sessions
|
||||
```
|
||||
|
||||
## Boundaries honored
|
||||
### Gateway Management
|
||||
|
||||
- No mounts of `~/.mosaic` or `~/.config/mosaic`; no Docker socket mount.
|
||||
- Source stays in this project directory; generated state only in
|
||||
`/home/jwoltje/.mosaic-dev` (host) and `/var/lib/mosaic` (container).
|
||||
- No database, web server, queue, second container, orchestration, Git
|
||||
integration, persistent sessions, or policy machinery.
|
||||
```bash
|
||||
mosaic gateway install # Install and configure the gateway service
|
||||
mosaic gateway verify # Post-install health check
|
||||
mosaic gateway login # Authenticate and store a session token
|
||||
mosaic gateway config rotate-token # Rotate your API token
|
||||
mosaic gateway config recover-token # Recover a token via BetterAuth cookie
|
||||
```
|
||||
|
||||
If you already have a gateway account but no token, use `mosaic gateway config recover-token` to retrieve one without recreating your account.
|
||||
|
||||
### Configuration
|
||||
|
||||
Mosaic supports three storage tiers: `local` (PGlite, single-host), `standalone` (PostgreSQL, single-host), and `federated` (PostgreSQL + pgvector + Valkey, multi-host). See [Federated Tier Setup](docs/federation/SETUP.md) for multi-user and production deployments, or [Migrating to Federated](docs/guides/migrate-tier.md) to upgrade from existing tiers.
|
||||
|
||||
```bash
|
||||
mosaic config show # Print full config as JSON
|
||||
mosaic config get <key> # Read a specific key
|
||||
mosaic config set <key> <val># Write a key
|
||||
mosaic config edit # Open config in $EDITOR
|
||||
mosaic config path # Print config file path
|
||||
```
|
||||
|
||||
### Management
|
||||
|
||||
```bash
|
||||
mosaic doctor # Health audit — detect drift and missing files
|
||||
mosaic sync # Sync skills from canonical source
|
||||
mosaic update # Check for and install CLI updates
|
||||
mosaic wizard # Full guided setup wizard
|
||||
mosaic bootstrap <path> # Bootstrap a repo with Mosaic standards
|
||||
mosaic coord init # Initialize a new orchestration mission
|
||||
mosaic prdy init # Create a PRD via guided session
|
||||
```
|
||||
|
||||
### Sub-package Commands
|
||||
|
||||
Each Mosaic sub-package exposes its API surface through the unified CLI:
|
||||
|
||||
```bash
|
||||
# User management
|
||||
mosaic auth users list
|
||||
mosaic auth users create
|
||||
mosaic auth sso
|
||||
|
||||
# Agent brain (projects, missions, tasks)
|
||||
mosaic brain projects
|
||||
mosaic brain missions
|
||||
mosaic brain tasks
|
||||
mosaic brain conversations
|
||||
|
||||
# Agent forge pipeline
|
||||
mosaic forge run
|
||||
mosaic forge status
|
||||
mosaic forge resume
|
||||
mosaic forge personas
|
||||
|
||||
# Structured logging
|
||||
mosaic log tail
|
||||
mosaic log search
|
||||
mosaic log export
|
||||
mosaic log level
|
||||
|
||||
# MACP protocol
|
||||
mosaic macp tasks
|
||||
mosaic macp submit
|
||||
mosaic macp gate
|
||||
mosaic macp events
|
||||
|
||||
# Agent memory
|
||||
mosaic memory search
|
||||
mosaic memory stats
|
||||
mosaic memory insights
|
||||
mosaic memory preferences
|
||||
|
||||
# Task queue (Valkey)
|
||||
mosaic queue list
|
||||
mosaic queue stats
|
||||
mosaic queue pause
|
||||
mosaic queue resume
|
||||
mosaic queue jobs
|
||||
mosaic queue drain
|
||||
|
||||
# Object storage
|
||||
mosaic storage status
|
||||
mosaic storage tier
|
||||
mosaic storage export
|
||||
mosaic storage import
|
||||
mosaic storage migrate
|
||||
```
|
||||
|
||||
### Telemetry
|
||||
|
||||
```bash
|
||||
# Local observability (OTEL / Jaeger)
|
||||
mosaic telemetry local status
|
||||
mosaic telemetry local tail
|
||||
mosaic telemetry local jaeger
|
||||
|
||||
# Remote telemetry (dry-run by default)
|
||||
mosaic telemetry status
|
||||
mosaic telemetry opt-in
|
||||
mosaic telemetry opt-out
|
||||
mosaic telemetry test
|
||||
mosaic telemetry upload # Dry-run unless opted in
|
||||
```
|
||||
|
||||
Consent state is persisted in config. Remote upload is a no-op until you run `mosaic telemetry opt-in`.
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js ≥ 20
|
||||
- pnpm 10.6+
|
||||
- Docker & Docker Compose
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
git clone [email protected]:mosaicstack/stack.git
|
||||
cd stack
|
||||
|
||||
# Start infrastructure (Postgres, Valkey, Jaeger)
|
||||
docker compose up -d
|
||||
|
||||
# Install dependencies
|
||||
pnpm install
|
||||
|
||||
# Run migrations
|
||||
pnpm --filter @mosaicstack/db run db:migrate
|
||||
|
||||
# Start all services in dev mode
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
### Infrastructure
|
||||
|
||||
Docker Compose provides:
|
||||
|
||||
| Service | Port | Purpose |
|
||||
| --------------------- | --------- | ---------------------- |
|
||||
| PostgreSQL (pgvector) | 5433 | Primary database |
|
||||
| Valkey | 6380 | Task queue + caching |
|
||||
| Jaeger | 16686 | Distributed tracing UI |
|
||||
| OTEL Collector | 4317/4318 | Telemetry ingestion |
|
||||
|
||||
### Quality Gates
|
||||
|
||||
```bash
|
||||
pnpm typecheck # TypeScript type checking (all packages)
|
||||
pnpm lint # ESLint (all packages)
|
||||
pnpm test # Vitest (all packages)
|
||||
pnpm format:check # Prettier check
|
||||
pnpm format # Prettier auto-fix
|
||||
```
|
||||
|
||||
### CI
|
||||
|
||||
Woodpecker CI runs on every push:
|
||||
|
||||
- `pnpm install --frozen-lockfile`
|
||||
- Database migration against a fresh Postgres
|
||||
- `pnpm test` (Turbo-orchestrated across all packages)
|
||||
|
||||
npm packages are published to the Gitea package registry on main merges.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
stack/
|
||||
├── apps/
|
||||
│ ├── gateway/ NestJS API + WebSocket hub (Fastify, Socket.IO, OTEL)
|
||||
│ └── web/ Next.js dashboard (React 19, Tailwind)
|
||||
├── packages/
|
||||
│ ├── mosaic/ Unified CLI — TUI, gateway client, wizard, sub-package commands
|
||||
│ ├── types/ Shared TypeScript contracts (Socket.IO typed events)
|
||||
│ ├── db/ Drizzle ORM schema + migrations (pgvector)
|
||||
│ ├── auth/ BetterAuth configuration
|
||||
│ ├── brain/ Data layer (PG-backed)
|
||||
│ ├── queue/ Valkey task queue + MCP
|
||||
│ ├── coord/ Mission coordination
|
||||
│ ├── forge/ Multi-stage AI pipeline (intake → board → plan → code → review)
|
||||
│ ├── macp/ MACP protocol — credential resolution, gate runner, events
|
||||
│ ├── agent/ Agent session management
|
||||
│ ├── memory/ Agent memory layer
|
||||
│ ├── log/ Structured logging
|
||||
│ ├── prdy/ PRD creation and validation
|
||||
│ ├── quality-rails/ Quality templates (TypeScript, Next.js, monorepo)
|
||||
│ └── design-tokens/ Shared design tokens
|
||||
├── plugins/
|
||||
│ ├── discord/ Discord channel plugin (discord.js)
|
||||
│ ├── telegram/ Telegram channel plugin (Telegraf)
|
||||
│ ├── macp/ OpenClaw MACP runtime plugin
|
||||
│ └── mosaic-framework/ OpenClaw framework injection plugin
|
||||
├── tools/
|
||||
│ └── install.sh Unified installer (framework + npm CLI, --yes / --no-auto-launch)
|
||||
├── scripts/agent/ Agent session lifecycle scripts
|
||||
├── docker-compose.yml Dev infrastructure
|
||||
└── .woodpecker/ CI pipeline configs
|
||||
```
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
- **Gateway is the single API surface** — all clients (TUI, web, Discord, Telegram) connect through it
|
||||
- **ESM everywhere** — `"type": "module"`, `.js` extensions in imports, NodeNext resolution
|
||||
- **Socket.IO typed events** — defined in `@mosaicstack/types`, enforced at compile time
|
||||
- **OTEL auto-instrumentation** — loads before NestJS bootstrap
|
||||
- **Explicit `@Inject()` decorators** — required since tsx/esbuild doesn't emit decorator metadata
|
||||
|
||||
### Framework (`~/.config/mosaic/`)
|
||||
|
||||
The framework is the bash-based standards layer installed to every developer machine:
|
||||
|
||||
```
|
||||
~/.config/mosaic/
|
||||
├── AGENTS.md ← Central standards (loaded into every runtime)
|
||||
├── SOUL.md ← Agent identity (name, style, guardrails)
|
||||
├── USER.md ← User profile (name, timezone, preferences)
|
||||
├── TOOLS.md ← Machine-level tool reference
|
||||
├── bin/mosaic ← Unified launcher (claude, codex, opencode, pi, yolo)
|
||||
├── guides/ ← E2E delivery, orchestrator protocol, PRD, etc.
|
||||
├── runtime/ ← Per-runtime configs (claude/, codex/, opencode/, pi/)
|
||||
├── skills/ ← Universal skills (synced from agent-skills repo)
|
||||
├── tools/ ← Tool suites (orchestrator, git, quality, prdy, etc.)
|
||||
└── memory/ ← Persistent agent memory (preserved across upgrades)
|
||||
```
|
||||
|
||||
### Forge Pipeline
|
||||
|
||||
Forge is a multi-stage AI pipeline for autonomous feature delivery:
|
||||
|
||||
```
|
||||
Intake → Discovery → Board Review → Planning (3 stages) → Coding → Review → Remediation → Test → Deploy
|
||||
```
|
||||
|
||||
Each stage has a dispatch mode (`exec` for research/review, `yolo` for coding), quality gates, and timeouts. The board review uses multiple AI personas (CEO, CTO, CFO, COO + specialists) to evaluate briefs before committing resources.
|
||||
|
||||
## Upgrading
|
||||
|
||||
Run the installer again — it handles upgrades automatically:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://mosaicstack.dev/install.sh | bash
|
||||
```
|
||||
|
||||
Or use the direct URL:
|
||||
|
||||
```bash
|
||||
bash <(curl -fsSL https://git.mosaicstack.dev/mosaicstack/stack/raw/branch/main/tools/install.sh)
|
||||
```
|
||||
|
||||
Or use the CLI:
|
||||
|
||||
```bash
|
||||
mosaic update # Check + install CLI updates
|
||||
mosaic update --check # Check only, don't install
|
||||
```
|
||||
|
||||
The CLI also performs a background update check on every invocation (cached for 1 hour).
|
||||
|
||||
### Installer Flags
|
||||
|
||||
```bash
|
||||
bash tools/install.sh --check # Version check only
|
||||
bash tools/install.sh --framework # Framework only (skip npm CLI)
|
||||
bash tools/install.sh --cli # npm CLI only (skip framework)
|
||||
bash tools/install.sh --ref v1.0 # Install from a specific git ref
|
||||
bash tools/install.sh --yes # Non-interactive, accept all defaults
|
||||
bash tools/install.sh --no-auto-launch # Skip auto-launch of wizard
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
```bash
|
||||
# Create a feature branch
|
||||
git checkout -b feat/my-feature
|
||||
|
||||
# Make changes, then verify
|
||||
pnpm typecheck && pnpm lint && pnpm test && pnpm format:check
|
||||
|
||||
# Commit (husky runs lint-staged automatically)
|
||||
git commit -m "feat: description of change"
|
||||
|
||||
# Push and create PR
|
||||
git push -u origin feat/my-feature
|
||||
```
|
||||
|
||||
DTOs go in `*.dto.ts` files at module boundaries. Scratchpads (`docs/scratchpads/`) are mandatory for non-trivial tasks. See `AGENTS.md` for the full standards reference.
|
||||
|
||||
## License
|
||||
|
||||
Proprietary — all rights reserved.
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
# Mosaic Stack
|
||||
|
||||
You are the default collaborator for Mosaic Stack: a practical engineering
|
||||
partner helping people build, inspect, and operate a trustworthy foundation
|
||||
for delegated work.
|
||||
|
||||
Mosaic Stack is deliberately small, file-based, and evidence-oriented. Its
|
||||
purpose is not to perform confidence; it is to make useful work attributable,
|
||||
bounded, reproducible, and reviewable. Treat the system's contracts, policies,
|
||||
and run records as part of the product, not paperwork around it.
|
||||
|
||||
Work with calm precision. Start from what the user is trying to accomplish,
|
||||
make the next useful step clear, and explain results in plain language. Be
|
||||
decisive when the evidence supports a decision; be explicit about uncertainty
|
||||
when it does not. Never claim a test, command, integration, or outcome that
|
||||
you have not actually verified.
|
||||
|
||||
Respect boundaries. Ask before expanding scope, changing authority, touching
|
||||
credentials, or taking an irreversible external action. Prefer the least
|
||||
privileged path, preserve user work, and stop on a policy or validation
|
||||
refusal rather than working around it. A clean refusal with a useful diagnosis
|
||||
is better than a superficially successful but untrustworthy result.
|
||||
|
||||
Leave a legible trail. Make changes intentional, keep records honest, and
|
||||
report what changed, how it was checked, and what remains unresolved. When
|
||||
coordinating other workers, give each one a bounded objective and review their
|
||||
evidence instead of treating their confidence as proof.
|
||||
|
||||
The aim is dependable progress: small enough to understand, safe enough to
|
||||
trust, and concrete enough for a person to verify.
|
||||
@@ -1,54 +0,0 @@
|
||||
# Mosaic runtime adapters
|
||||
|
||||
An adapter is the entire harness-specific surface of the system. Everything
|
||||
upstream of an adapter — configuration, contracts, missions, tasks, run
|
||||
records — is harness-agnostic; everything inside an adapter may assume one
|
||||
specific agent runtime.
|
||||
|
||||
## Contract
|
||||
|
||||
An adapter lives at:
|
||||
|
||||
```text
|
||||
/opt/mosaic/adapters/<name>/adapter.sh
|
||||
```
|
||||
|
||||
and must be executable. The dispatcher (`/opt/mosaic/src/run-agent.sh`)
|
||||
selects it via `MOSAIC_ADAPTER` (default: `pi`) and execs it after the
|
||||
system prompt has been generated.
|
||||
|
||||
**Inputs (environment):**
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `MOSAIC_SYSTEM_PROMPT_FILE` | Absolute path to the generated system prompt (contracts + optional mission section). Read it; do not modify it. |
|
||||
| `MOSAIC_REQUEST` | The exact user request text (may contain newlines). |
|
||||
| `MOSAIC_PROVIDER` | Configured provider name. |
|
||||
| `MOSAIC_MODEL` | Configured model id. |
|
||||
|
||||
Optional, adapter-specific (documented per adapter):
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `MOSAIC_MOCK_RESPONSE` | mock only: the verbatim response to emit |
|
||||
|
||||
**Outputs:**
|
||||
|
||||
- `stdout`: the model response text — the only channel the orchestrator captures
|
||||
- `stderr`: diagnostics (never credentials)
|
||||
- exit `0`: success; nonzero: failure
|
||||
|
||||
## Rules
|
||||
|
||||
1. Adapters print ONLY the response on stdout. Status lines go to stderr.
|
||||
2. Adapters never read configuration files; the resolved settings arrive via environment.
|
||||
3. Adapters never write outside `/var/lib/mosaic`.
|
||||
4. Adding an adapter requires: a new directory, the contract implementation, and
|
||||
adding the name to the allowlist in `scripts/mosaic-config.mjs`.
|
||||
|
||||
## Included adapters
|
||||
|
||||
- `pi` — the pinned `@earendil-works/pi-coding-agent` CLI in noninteractive
|
||||
print mode (`-p`), ambient discovery disabled, stdin detached.
|
||||
- `mock` — deterministic echo of `MOSAIC_MOCK_RESPONSE`. Test-only: never use
|
||||
it where a real model response is required.
|
||||
@@ -1,18 +0,0 @@
|
||||
#!/bin/sh
|
||||
# Mock adapter: deterministic response for seam tests. NEVER use where a
|
||||
# real model response is required.
|
||||
#
|
||||
# Contract: see /opt/mosaic/adapters/README.md.
|
||||
set -eu
|
||||
|
||||
[ -n "${MOSAIC_SYSTEM_PROMPT_FILE:-}" ] || { echo "mock adapter: MOSAIC_SYSTEM_PROMPT_FILE is required" >&2; exit 2; }
|
||||
if [ "${MOSAIC_INTERACTIVE:-}" != "1" ]; then
|
||||
[ -n "${MOSAIC_REQUEST:-}" ] || { echo "mock adapter: MOSAIC_REQUEST is required" >&2; exit 2; }
|
||||
fi
|
||||
[ -r "$MOSAIC_SYSTEM_PROMPT_FILE" ] || { echo "mock adapter: system prompt not readable: $MOSAIC_SYSTEM_PROMPT_FILE" >&2; exit 2; }
|
||||
|
||||
echo "mock adapter: responding verbatim from MOSAIC_MOCK_RESPONSE" >&2
|
||||
# Deterministic plumbing evidence: which MOSAIC_* variables did the
|
||||
# orchestrator actually deliver? (Auth secrets are not MOSAIC_-prefixed.)
|
||||
(env | grep '^MOSAIC_' | sort) >&2 2>/dev/null || true
|
||||
printf '%s\n' "${MOSAIC_MOCK_RESPONSE:-}"
|
||||
@@ -1,96 +0,0 @@
|
||||
#!/bin/sh
|
||||
# Pi adapter: implements the Mosaic adapter contract for the pinned
|
||||
# @earendil-works/pi-coding-agent CLI.
|
||||
#
|
||||
# Contract: see /opt/mosaic/adapters/README.md.
|
||||
# Headless (default): stdout = response only; stderr = diagnostics; exit 0.
|
||||
# Interactive (MOSAIC_INTERACTIVE=1): full pi TUI on the attached terminal.
|
||||
set -eu
|
||||
|
||||
[ -n "${MOSAIC_SYSTEM_PROMPT_FILE:-}" ] || { echo "pi adapter: MOSAIC_SYSTEM_PROMPT_FILE is required" >&2; exit 2; }
|
||||
[ -r "$MOSAIC_SYSTEM_PROMPT_FILE" ] || { echo "pi adapter: system prompt not readable: $MOSAIC_SYSTEM_PROMPT_FILE" >&2; exit 2; }
|
||||
# MOSAIC_AGENT_NAME is optional in headless mode (identity section is then
|
||||
# omitted); interactive launches always set it via scripts/agent.sh.
|
||||
|
||||
: "${PI_PROVIDER:?pi adapter: PI_PROVIDER is required}"
|
||||
: "${PI_MODEL:?pi adapter: PI_MODEL is required}"
|
||||
|
||||
INTERACTIVE="${MOSAIC_INTERACTIVE:-}"
|
||||
if [ "$INTERACTIVE" != "1" ]; then
|
||||
[ -n "${MOSAIC_REQUEST:-}" ] || { echo "pi adapter: MOSAIC_REQUEST is required" >&2; exit 2; }
|
||||
fi
|
||||
|
||||
# Workspace (M5): run inside the provided workspace when present.
|
||||
if [ -n "${MOSAIC_WORKSPACE:-}" ]; then
|
||||
mkdir -p "$MOSAIC_WORKSPACE"
|
||||
cd "$MOSAIC_WORKSPACE"
|
||||
fi
|
||||
|
||||
# Session (M6/M11): default ephemeral (--no-session). With a declared
|
||||
# session dir: persist there and resume the most recent session. With a
|
||||
# fork source: branch the source session file into the target dir
|
||||
# (pi --fork) - the ancestor session is never modified.
|
||||
SESSION_FLAGS="--no-session"
|
||||
if [ -n "${MOSAIC_SESSION_FORK:-}" ]; then
|
||||
[ -n "${MOSAIC_SESSION_DIR:-}" ] || { echo "pi adapter: session fork requires MOSAIC_SESSION_DIR" >&2; exit 2; }
|
||||
mkdir -p "$MOSAIC_SESSION_DIR"
|
||||
SESSION_FLAGS="--fork $MOSAIC_SESSION_FORK --session-dir $MOSAIC_SESSION_DIR"
|
||||
elif [ -n "${MOSAIC_SESSION_DIR:-}" ]; then
|
||||
mkdir -p "$MOSAIC_SESSION_DIR"
|
||||
SESSION_FLAGS="--session-dir $MOSAIC_SESSION_DIR"
|
||||
if [ -n "$(ls -A "$MOSAIC_SESSION_DIR" 2>/dev/null)" ]; then
|
||||
SESSION_FLAGS="$SESSION_FLAGS -c"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Capabilities (M5): explicit allowlist or no tools.
|
||||
TOOLS_FLAG="--no-tools"
|
||||
[ -n "${MOSAIC_TOOLS:-}" ] && TOOLS_FLAG="--tools $MOSAIC_TOOLS"
|
||||
|
||||
# Skills (M17): explicitly provided skill dirs replace discovery. When none
|
||||
# are provided the agent runs with --no-skills (nothing ambient to find).
|
||||
SKILLS_FLAG="--no-skills"
|
||||
if [ -n "${MOSAIC_SKILLS:-}" ]; then
|
||||
SKILLS_FLAG=""
|
||||
OLDIFS=$IFS; IFS=','
|
||||
for s in $MOSAIC_SKILLS; do
|
||||
[ -d "$s" ] || { echo "pi adapter: skill dir missing: $s" >&2; exit 2; }
|
||||
SKILLS_FLAG="$SKILLS_FLAG --skill $s"
|
||||
done
|
||||
IFS=$OLDIFS
|
||||
fi
|
||||
|
||||
# Mode (M13): interactive TUI or one-shot print.
|
||||
PRINT_MODE="-p"
|
||||
REQUEST_ARG=""
|
||||
if [ "$INTERACTIVE" = "1" ]; then
|
||||
PRINT_MODE=""
|
||||
else
|
||||
REQUEST_ARG="$MOSAIC_REQUEST"
|
||||
fi
|
||||
|
||||
# All flags documented in the pi package README (CLI Reference):
|
||||
# -p/--print one-shot mode: print the response and exit (omitted in
|
||||
# interactive TUI mode)
|
||||
# --system-prompt replace the default prompt with the generated one
|
||||
# --no-* no ambient context/skills/extensions/templates/themes
|
||||
# SESSION_FLAGS ephemeral | persistent | forked (per env)
|
||||
# TOOLS_FLAG per capabilities
|
||||
# --offline no startup network operations (update checks/telemetry)
|
||||
PROMPT_CONTENT="$(cat "$MOSAIC_SYSTEM_PROMPT_FILE")"
|
||||
set -- \
|
||||
--offline \
|
||||
--no-extensions \
|
||||
$SKILLS_FLAG \
|
||||
--no-prompt-templates \
|
||||
--no-themes \
|
||||
--no-context-files \
|
||||
$TOOLS_FLAG \
|
||||
$SESSION_FLAGS \
|
||||
--provider "$PI_PROVIDER" \
|
||||
--model "$PI_MODEL" \
|
||||
--system-prompt "$PROMPT_CONTENT"
|
||||
# One-shot mode appends -p and the request (both safely quoted);
|
||||
# interactive mode appends nothing - clean TUI.
|
||||
[ "$INTERACTIVE" = "1" ] || set -- "$@" -p "$MOSAIC_REQUEST"
|
||||
exec pi "$@"
|
||||
@@ -1,42 +0,0 @@
|
||||
# Mosaic Stack development team
|
||||
|
||||
These are interactive host development agents working in the canonical
|
||||
checkout. They do not create managed fleet registrations or change role policy.
|
||||
Sage leads the project and coordinates assignments, review, and integration
|
||||
(Jason's ruling, 2026-09-26). Development sessions run in T3 for now.
|
||||
For the current bootstrap phase, direct coding, review and research use Darkwing,
|
||||
Dewey, Filbert, Rocko, Researcher and further seats Jason launches from this repository's `agents/` directory,
|
||||
not fleet seats. Keep changes in `/mnt/storage/src/mosaic-stack`; do not modify
|
||||
`~/.mosaic` launchers/provisioning or migrate/stop live fleet processes.
|
||||
|
||||
| Agent | Responsibility | Runtime | Launch from repository root |
|
||||
| --- | --- | --- | --- |
|
||||
| Darkwing | Hands-on engineering; collaborating seat under Sage | Pi, configured Mosaic model | `agents/darkwing/launch.sh` |
|
||||
| Dewey | Frontend design, UX, accessibility, and UI implementation | Pi, configured Mosaic model | `agents/dewey/launch.sh` |
|
||||
| Rocko | General development, investigation, testing, and review | Claude Code, Sonnet model | `agents/rocko/launch.sh` |
|
||||
| Filbert | General development, investigation, testing, and review | Pi, `openai-codex/gpt-6-astra:low` | `agents/filbert/launch.sh` |
|
||||
| Researcher | Source-grounded technical research and evidence | Pi, configured Mosaic model | `agents/researcher/launch.sh` |
|
||||
| Sage | Project lead: coordination, review, integration; earlier DYOR strategy records retained | T3 session (Claude Code); Pi launcher `zai/glm-5.3:high` retained | `agents/sage/launch.sh` |
|
||||
|
||||
Each script supports `--check` and `--fresh`. Normal launches resume the agent's
|
||||
own conversation; a first launch starts one. See each agent's README for
|
||||
context inputs, authentication, and recovery details. Launch scripts can also
|
||||
be invoked by absolute path from any directory. No assignment or model request
|
||||
is submitted by the launcher itself.
|
||||
|
||||
The shared Pi helper supports `--provider NAME`, `--model ID`, and
|
||||
`--thinking LEVEL` as per-launch overrides of the validated system defaults.
|
||||
Filbert's wrapper appends the required provider, model and thinking flags so
|
||||
its launch configuration remains fixed, including on resume. Use Filbert's
|
||||
wrapper to select that configuration; a direct shared-helper invocation uses
|
||||
its own supplied flags or the system defaults.
|
||||
|
||||
All agents follow repository governance and current user direction. Team
|
||||
leadership does not add deployment or push authority. Coordinate overlapping
|
||||
work with Sage and preserve other sessions' changes.
|
||||
|
||||
Before 2026-09-26 Sage worked only on DYOR strategy and sat outside the
|
||||
development queue. Jason then made Sage the project lead and Darkwing a
|
||||
collaborating seat. A separate fleet Sage seat under `~/.mosaic` is being
|
||||
decommissioned; it does not speak for this seat. Whether the DYOR strategy work
|
||||
continues is Jason's call. Joe retains DYOR engineering.
|
||||
@@ -1,38 +0,0 @@
|
||||
|
||||
===== DARKWING NATIVE DEVELOPMENT CONTEXT =====
|
||||
|
||||
Your identity is Darkwing. This launch runs Pi directly on the host, in the
|
||||
Mosaic Stack development repository. The injected SOUL defines your persona;
|
||||
CONSTITUTION and STANDARDS supply governance, USER supplies user context,
|
||||
and AGENTS.md supplies repository instructions.
|
||||
|
||||
You have host read, bash, edit, write, grep, find, and ls tools. This is a
|
||||
development TUI with the operator's OS access, not a sandbox or a registered
|
||||
managed fleet seat. Use repository scripts for Mosaic operations and inspect
|
||||
their effects before running them. Container paths in skills describe worker
|
||||
deployments, not your current workspace. A tool's presence is not authority
|
||||
to change unrelated files, other agents' work, or the live fleet.
|
||||
|
||||
For an assigned improvement, inspect the implementation, reproduce the issue,
|
||||
make the smallest useful change, verify it, and continue through the authorized
|
||||
outcome. Run scripts/mosaic queue next darkwing for ownership, gates and your next piece;
|
||||
a new user assignment does not silently resume unrelated queued work.
|
||||
|
||||
The local /goal extension is loaded and owns any operator-set goal lifecycle.
|
||||
Use ms-proactive-agent for work selection and ms-goal for recovery guidance;
|
||||
do not create a competing goal loop. Follow goal_report's actual schema and
|
||||
reporting instructions. Its text format is Just Completed / Next Step /
|
||||
Blocked, with '* none' for empty sections. No external reporting skill is
|
||||
needed to discover that format. Native development packaging supersedes
|
||||
older skill statements that this extension is unavailable.
|
||||
|
||||
For relocation recovery, read agents/darkwing/work/RESTART.md after the root
|
||||
AGENTS.md and docs/plans/CURRENT.md. It records verified checkpoints and limits,
|
||||
not a new assignment. The canonical checkout is /mnt/storage/src/mosaic-stack;
|
||||
v1/ is archived legacy source. Reconcile newer owner direction before acting.
|
||||
|
||||
Conversation history persists across launcher restarts. Goals belong to a
|
||||
single process incarnation; recover the assignment from verified records and
|
||||
the operator's direction after a restart. No goal is started by this launcher.
|
||||
Context is captured anew at launch; source edits do not update this process's
|
||||
injected snapshot. Relaunch to load approved context changes.
|
||||
@@ -1,64 +0,0 @@
|
||||
# Darkwing development TUI
|
||||
|
||||
From any terminal, run:
|
||||
|
||||
```sh
|
||||
/home/jwoltje/src/mosaic-stack-dev-test/agents/darkwing/launch.sh
|
||||
```
|
||||
|
||||
The agent launcher is a thin shim to `scripts/agent.sh --host-dev darkwing`,
|
||||
forwarding all arguments unchanged. `scripts/agent.sh` is the common entry
|
||||
point; `scripts/agent-host-dev.sh` implements its native development mode.
|
||||
The host launcher opens the repository as Darkwing's workspace.
|
||||
It uses the repository-pinned Pi, the configured Mosaic provider/model, and
|
||||
native Pi authentication (normal `~/.pi/agent`, or `PI_CODING_AGENT_DIR` if
|
||||
explicitly set). It never copies credentials. Install dependencies with
|
||||
`npm ci --ignore-scripts --no-audit --no-fund` if needed.
|
||||
|
||||
`--check` validates configuration and required inputs without opening Pi or
|
||||
calling a model. `--fresh` starts a new conversation without deleting earlier
|
||||
ones. Normal launches continue the latest conversation under
|
||||
`.pi/state/darkwing/sessions/`; the first launch creates one. A launcher lock
|
||||
rejects simultaneous launches through this script. It does not exclude Pi
|
||||
processes started another way. Damaged JSONL history refuses automatic resume;
|
||||
`--fresh` is an explicit escape hatch that preserves the damaged evidence.
|
||||
|
||||
The current files are combined into a private launch snapshot under
|
||||
`.pi/state/darkwing/launches/`:
|
||||
|
||||
- `contracts/CONSTITUTION.md` and `contracts/STANDARDS.md`
|
||||
- `agents/darkwing/SOUL.md`
|
||||
- `<configured dataRoot>/user/USER.md`, the deployment's live user profile
|
||||
- the repository's `AGENTS.md` and Darkwing's `CONTEXT.md`
|
||||
|
||||
Use `--soul FILE`, `--constitution FILE`, or `--user FILE` to select alternate
|
||||
inputs, including a future `contracts/USER.md`. Relative paths resolve from
|
||||
the repository root. Missing or empty inputs refuse launch. Snapshots can
|
||||
contain personal context and remain local, with private file permissions.
|
||||
Context edits take effect on relaunch, including when resuming a conversation.
|
||||
|
||||
The launcher enables coding/search tools, `goal_report`, ten explicit local
|
||||
skills, and the canonical goal extension through `scripts/sync-dev-extensions.sh`.
|
||||
Ambient context, skills, extensions, templates, and themes are disabled.
|
||||
The normal Pi coding prompt is retained with the Mosaic context appended.
|
||||
Enter `/goal <assignment and acceptance criteria>` to start continuing work;
|
||||
`/goal stop`, `/goal resume`, and `/goal` pause, resume, and inspect it. A new
|
||||
process does not automatically adopt a previous process's goal.
|
||||
|
||||
This TUI has the operator's host access, including repository edits and host
|
||||
commands. Its tool list is not OS isolation. It creates no managed role or
|
||||
fleet registration. Worker dispatch still uses the governed Mosaic task runner.
|
||||
The user supplies the assignment; launch alone does not start self-modification.
|
||||
|
||||
## Deployment findings
|
||||
|
||||
The existing `scripts/agent.sh` launches a Docker container, defaults to the
|
||||
`agent-<name>` session directory, and asks Pi to continue when that directory
|
||||
is nonempty. Its default workspace is `<dataRoot>/workspaces/<name>`, not this
|
||||
checkout. `src/load-contracts.sh` loads image-baked governance, an optional
|
||||
seat SOUL override, live user Markdown, and mission context into a shared
|
||||
prompt path. A seat override requires `agent.json`; a standalone SOUL is not
|
||||
discovered. `adapters/pi/adapter.sh` disables extensions. The temporary host
|
||||
launcher follows the existing native development path to provide repository
|
||||
access and `/goal`, and keeps its conversations separate from container and
|
||||
live fleet sessions. It does not invoke release alignment on startup.
|
||||
@@ -1,30 +0,0 @@
|
||||
# SOUL — Darkwing
|
||||
|
||||
You are Darkwing, Mosaic Stack's hands-on engineering collaborator, working
|
||||
with Sage as project lead. Your job is to help Jason make the system
|
||||
dependable by using it, finding where it falls short, and carrying authorized
|
||||
improvements through verification.
|
||||
|
||||
Be curious, direct, and resourceful. Have a technical opinion and explain
|
||||
the evidence behind it. Investigate before guessing. Distinguish a design
|
||||
claim, a passing test, and behavior you have observed in the running system.
|
||||
|
||||
Use Mosaic's own tools and workflows where they fit. Turn a failure into a
|
||||
reproducible case, make a focused correction, and test the behavior again.
|
||||
Let each verified improvement inform the next one within the assignment.
|
||||
Keep the human informed when the result, scope, or next decision changes.
|
||||
|
||||
Own the outcome while respecting other agents' work. Preserve their changes
|
||||
and records, give delegated work clear boundaries, and seek independent
|
||||
review where required. Self-improvement never grants new authority: changing
|
||||
your instructions, permissions, or a live deployment follows the same review
|
||||
and authorization rules as any other system change.
|
||||
|
||||
Sage leads the development team (Jason's ruling, 2026-09-26) and translates
|
||||
Jason's priorities into scoped work, coordinates ownership and dependencies,
|
||||
reviews results, and verifies integration. You are a collaborating seat.
|
||||
Dewey owns frontend design and UX. Rocko and Filbert support general project needs,
|
||||
including implementation, investigation, testing, and review. Reconcile
|
||||
concurrent edits with Sage before integration. Keep Jason informed of
|
||||
outcomes and decisions that require his input. Your role does not expand the
|
||||
project's existing authorization or release rules.
|
||||
@@ -1,10 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Darkwing's native development mode through the Mosaic agent entry point.
|
||||
set -euo pipefail
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
# Register this seat with the control board (packages/seat) unless already
|
||||
# registered by `mosaic launch` or only running the checks.
|
||||
if [ -z "${MOSAIC_LAUNCH_REGISTERED:-}" ] && ! printf '%s\n' "$@" | grep -qx -- '--check'; then
|
||||
exec "$REPO/scripts/mosaic" launch --repo "$REPO" --harness pi darkwing -- "$@"
|
||||
fi
|
||||
exec "$REPO/scripts/agent.sh" --host-dev darkwing "$@"
|
||||
@@ -1,21 +0,0 @@
|
||||
// Refuse damaged history before Pi's --continue can silently skip it.
|
||||
import { readFileSync, lstatSync } from 'node:fs';
|
||||
|
||||
try {
|
||||
for (const file of process.argv.slice(2)) {
|
||||
if (!lstatSync(file).isFile()) throw new Error(`not a regular session file: ${file}`);
|
||||
const lines = readFileSync(file, 'utf8').trim().split('\n');
|
||||
const entries = lines.map((line) => JSON.parse(line));
|
||||
const header = entries[0];
|
||||
if (header?.type !== 'session' || typeof header.id !== 'string' || !header.id ||
|
||||
typeof header.version !== 'number' || typeof header.cwd !== 'string' ||
|
||||
!Number.isFinite(Date.parse(header.timestamp)) ||
|
||||
entries.slice(1).some((entry) => !entry || typeof entry.type !== 'string')) {
|
||||
throw new Error(`invalid session structure: ${file}`);
|
||||
}
|
||||
if (header.cwd !== process.cwd()) throw new Error(`session belongs to another workspace: ${file}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`darkwing: cannot safely resume: ${error.message}; inspect history or explicitly use --fresh`);
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -1,108 +0,0 @@
|
||||
# Darkwing — relocation handoff
|
||||
|
||||
Recorded 2026-09-07 17:43 UTC. Jason intends to relaunch with
|
||||
`/mnt/storage/src/mosaic-stack/agents/darkwing/launch.sh`.
|
||||
This is a recovery note, not a new assignment or automatic goal resumption.
|
||||
|
||||
## Read first
|
||||
|
||||
1. Root `AGENTS.md` and `docs/plans/CURRENT.md`.
|
||||
2. This note, then `git status --short` and `git log --oneline -5`.
|
||||
3. Reconcile current owner direction and any newer declared artifacts before acting.
|
||||
|
||||
## Repository conversion is completed locally
|
||||
|
||||
Jason explicitly ordered the conversion and confirmed no work was active.
|
||||
- Canonical checkout: `/mnt/storage/src/mosaic-stack`.
|
||||
- Origin: `https://git.mosaicstack.dev/mosaicstack/stack`.
|
||||
- Branch: `refactor`.
|
||||
- Conversion commit: `127a54fdff1fe6ae56c3197edddf957481465db4`.
|
||||
- New foundation is at root. `v1/` is legacy archival source, NOT current code.
|
||||
- Old `/home/jwoltje/src/mosaic-stack-dev-test` is a compatibility symlink to this
|
||||
same checkout. Do not recreate a second working copy there.
|
||||
- Both histories retained: merge parents v2 `9a5fbdbda74b16adf488fe28138b2ba69ea5e669`
|
||||
and v1 `5d2770002612a09ae0cadc129b4ea30619133e8a`.
|
||||
- Exact 3,507-file v1 tracked tree imported; v1 refs under `refs/archive/v1/`.
|
||||
- Original v2 refs retained; `stack-v2-archive` remote has a disabled push URL.
|
||||
- Only legacy tracked tree and four conversion docs committed. All earlier
|
||||
uncommitted/untracked/ignored work preserved. Index was verified clean.
|
||||
- Issue https://git.mosaicstack.dev/mosaicstack/stack/issues/1495 closed explicitly
|
||||
for local conversion. No push, PR/trunk merge or live-service change occurred.
|
||||
|
||||
Record: `docs/plans/2026-09-07_repository-consolidation-completed.md`.
|
||||
Receipts: `docs/plans/reviews/2026-09-07_repository-conversion-verification.json`
|
||||
and `2026-09-07_repository-conversion-postcommit-verification.json`.
|
||||
Verified rollback copies, NOT development roots:
|
||||
- `/mnt/storage/src/.mosaic-stack-conversion-20260907T172430Z/`
|
||||
- `/home/jwoltje/src/.mosaic-stack-dev-test.pre-conversion-20260907T172430Z`
|
||||
Do not delete them, launch from them or restore over newer work.
|
||||
|
||||
## Current unfinished foundation gate
|
||||
|
||||
Jason's A9 acceptance of the first offline synthetic scope/permission inspector
|
||||
is pending. Code is independently approved by Filbert; no blocking code finding
|
||||
remains at the reviewed r6 candidate. Owner acceptance is not inferred from tests.
|
||||
|
||||
- Manifest: `docs/plans/reviews/2026-09-07_foundation-inspector-rocko-build-manifest-r6.json`
|
||||
SHA-256 `a4a4493000aff5905337a643886ca36e7c5377d52deed77b8aeab7174ca73dcf`.
|
||||
- Report: `docs/plans/reviews/2026-09-07_foundation-inspector-rocko-build-r6.md`
|
||||
SHA-256 `ee0e83efd7c71eddecf5e26f939e9a34ba85b184cfcd1cffac9ff9e56ea13c37`.
|
||||
- APPROVED verdict: `docs/plans/reviews/2026-09-07_foundation-inspector-code-verdict-r6.md`
|
||||
SHA-256 `ab9dd5e5c3cad5c9263e873ff82cac444da2d36040e907e4798b208fa1c08b13`.
|
||||
- Guide: `docs/plans/reviews/2026-09-07_foundation-inspector-demo.md`.
|
||||
|
||||
All 382 approved inspector files and pinned inputs survived conversion unchanged.
|
||||
Actual offline checks: Node 80/0, selftests 43/0, oracle 1,568 records / zero
|
||||
schema disagreements, foundation checker PASS, config/auth/conductor 24/15/17.
|
||||
Postcommit conductor 17/0 and four CLI demos passed: allowed read, allowed change
|
||||
PREVIEW (no mutation), missing-registration refusal, unresolved reassignment with
|
||||
original selection retained. Demo inputs are separate synthetic scenarios.
|
||||
|
||||
`test-task.sh` and `test-release.sh` remain NOT RUN / DEFERRED under Jason's bounded
|
||||
offline-demo ruling. No deployment/native/live/provider/security certification.
|
||||
Reviewer qualifications: ordering equality means structural equality, not byte
|
||||
identity; auxiliary native-parser warm-run anomalies remain separate unresolved
|
||||
observations, not a passing universal parser-equivalence claim. Preserve all earlier
|
||||
NOT APPROVED reviews and the historical correction that r3 ran unauthorized live
|
||||
branches; later deferral did not retroactively authorize them.
|
||||
|
||||
## Ownership and limits
|
||||
|
||||
- Rocko authored inspector code; Filbert independently reviewed; Darkwing coordinates
|
||||
and verifies. Keep the approved candidate frozen unless a new fix is authorized.
|
||||
- No automatic permission to push, merge to next/main, deploy, change live config,
|
||||
grant permissions, access credentials, investigate ~/.mosaic, or start new runtime
|
||||
work. Local conversion authority is not authority for those activities.
|
||||
- Preserve unrelated pending work. In particular `scripts/agent.sh`, `docs/TOOLS.md`,
|
||||
host launcher/context files and other untracked concepts/skills belong to existing
|
||||
work. Do not blanket-stage/reset/clean. Root logs and CURRENT remain uncommitted.
|
||||
- Foundation #53 in the old stack-v2 project remains a separate open issue; do not
|
||||
silently close or renumber it. Accepted historical SHA/path citations remain valid.
|
||||
- Rocko's Archify C1 remains HELD for owner T2/T3 decisions. No lane reassignment.
|
||||
- Future durability/workflow/evidence/federation/onboarding topics are notes, not
|
||||
authorization to expand the inspector.
|
||||
|
||||
## Communications
|
||||
|
||||
Use only `tools/tmux/agent-send.sh`; sender `dragon-lin:darkwing`.
|
||||
Rocko: `-L mosaic-fleet -s '=rocko'`; Filbert/Dewey:
|
||||
`-L default -s '=filbert'` / `'=dewey'`.
|
||||
Conversion notice delivered to Rocko. Filbert/Dewey sends were unconfirmed
|
||||
(input boxes not locatable); no retries, no acknowledgement claimed. Check declared
|
||||
artifact paths as well as direct messages; completed reviews have existed without
|
||||
transported replies. Do not inspect private panes or blindly resend.
|
||||
|
||||
## Relaunch and goal recovery
|
||||
|
||||
The project launcher continues its own latest `.pi/state/darkwing/sessions/`
|
||||
conversation by default. Do NOT assume this pre-launch conversation is already in
|
||||
that store or that the next launch resumes this exact conversation. This handoff
|
||||
is the durable bridge. No session-tree migration or launch was performed here.
|
||||
|
||||
The goal extension owns lifecycle. The earlier extension goal had been paused;
|
||||
no restart automatically resumes it. Reconcile the actual new process state and
|
||||
Jason's direction rather than reporting progress against a guessed old goal or
|
||||
creating a second goal loop. Launch alone grants no new assignment.
|
||||
|
||||
This handoff and its CONTEXT pointer are documentation-only. Launcher scripts,
|
||||
private sessions, credentials and runtime configuration were not modified.
|
||||
@@ -1,17 +0,0 @@
|
||||
{
|
||||
"issue": 1503,
|
||||
"candidate": "/tmp/board-attention-r1-q_924ksh",
|
||||
"files": [
|
||||
"AGENTS.md",
|
||||
"packages/control-board/src/scan.mjs",
|
||||
"packages/control-board/README.md",
|
||||
"packages/control-board/tests/scan.test.mjs",
|
||||
"packages/control-board/tests/serve.test.mjs",
|
||||
"packages/control-board/tests/attention.test.mjs",
|
||||
"packages/control-board/tests/attention-flow.test.mjs",
|
||||
"packages/webui/tests/fixture.mjs",
|
||||
"docs/plans/2026-09-13_board-attention-status.md"
|
||||
],
|
||||
"manifestSha256": "e40b58ecb6844d407ba776dcde8f19ad21c0b78076ca1f9b3d96b0bb1c852405",
|
||||
"testsPassed": 144
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T00:19:32.409806+00:00",
|
||||
"backendPid": 3204655,
|
||||
"health": "ok",
|
||||
"researcher": {
|
||||
"agent": "researcher",
|
||||
"project": "mosaic-stack",
|
||||
"alive": true,
|
||||
"state": "idle",
|
||||
"waitingOnYou": false,
|
||||
"lastActivity": "2026-09-13T21:19:57.102Z"
|
||||
},
|
||||
"fiveAgentProcessesUnchanged": true
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T00:19:11.579013+00:00",
|
||||
"ownerAuthorized": true,
|
||||
"oldPid": 1265952,
|
||||
"newPid": 3204655,
|
||||
"command": [
|
||||
"/usr/bin/node",
|
||||
"packages/control-board/src/cli.mjs",
|
||||
"serve"
|
||||
],
|
||||
"cwd": "/mnt/storage/src/mosaic-stack",
|
||||
"log": "/tmp/board-attention-backend-ag9xvks2.log",
|
||||
"agentProcessesBefore": {
|
||||
"default/darkwing": [
|
||||
[
|
||||
"2733924",
|
||||
"12863634"
|
||||
]
|
||||
],
|
||||
"default/dewey": [
|
||||
[
|
||||
"934346",
|
||||
"466065"
|
||||
]
|
||||
],
|
||||
"default/filbert": [
|
||||
[
|
||||
"72183",
|
||||
"100870"
|
||||
]
|
||||
],
|
||||
"default/researcher": [
|
||||
[
|
||||
"173699",
|
||||
"66404285"
|
||||
]
|
||||
],
|
||||
"mosaic-fleet/rocko": [
|
||||
[
|
||||
"90599",
|
||||
"128275"
|
||||
]
|
||||
]
|
||||
},
|
||||
"gracefulExitObserved": true
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
# CHAT-02 board routes: Darkwing's review (#1507)
|
||||
|
||||
Reviewer: Darkwing, 2026-09-26, per Sage's D3. Requested by Dewey. Scope: the
|
||||
two read-only routes only. Filbert reviews `packages/conversation` in full.
|
||||
|
||||
Candidate, base 34777c56, uncommitted. I verified both hashes:
|
||||
- `packages/control-board/src/serve.mjs` afc95bdb…c540d
|
||||
- `packages/control-board/tests/serve.test.mjs` e60aa14b…ecbc
|
||||
|
||||
**Verdict: approve**, with one commit condition and two nonblocking notes.
|
||||
|
||||
## What I checked
|
||||
|
||||
- Order. `foreignRequest` runs first on every request, then the POST routes,
|
||||
then the GET/HEAD check (405 otherwise), then these routes. A POST to
|
||||
either path is 405, and a foreign Host or Origin is 403 before any read.
|
||||
- Query validation. `/api/conversations` refuses any parameter.
|
||||
`/api/conversation` accepts only `id`, `branch` and `cursor`, one value
|
||||
each, each matching `QUERY_VALUE`. That is the same pattern as
|
||||
`parts.mjs` `ID`, so every id the reader issues (`safeId`, `root`, `c-`
|
||||
cursors, `pi-` conversations) passes. No path comes from the request.
|
||||
- Responses. JSON with `no-store` and `nosniff`, no CORS headers. A thrown
|
||||
error gives a fixed 500 body and logs to stderr only.
|
||||
- Refusal bodies. Every `Refusal` message in `packages/conversation/src` is
|
||||
a fixed string. The one interpolated message (`denied`, safe-fs.mjs:34)
|
||||
interpolates only "session root" or "session file". No path or content
|
||||
reaches the client through `error`.
|
||||
- Status map. It covers every code the route can reach. `unknown-actor` and
|
||||
`unsupported-purpose` are absent, and the route can't produce them because
|
||||
it always passes the default actor and purpose.
|
||||
- Tests: `serve.test.mjs` plus `packages/conversation/tests/`, 61/61 on the
|
||||
pinned files.
|
||||
|
||||
## Commit condition
|
||||
|
||||
`serve.mjs` imports `../../conversation/src/reader.mjs` at module load, and
|
||||
`packages/conversation/` is untracked. Committing the routes without that
|
||||
package breaks the board's start, not only these routes. The package must
|
||||
land in the same commit or an earlier one, after Filbert's review.
|
||||
|
||||
## Notes (nonblocking)
|
||||
|
||||
1. **A cursor needs its branch.** The header comment says `branch` and
|
||||
`cursor` are optional. But `next()` compares `branch !== record.branch`,
|
||||
and every cursor record carries a string branch (`safeId` or `root`). So
|
||||
`?id=X&cursor=C` without `branch` is always 409 `cursor-foreign`, with
|
||||
`reconcile: true`. That is safe, but a client that follows `nextCursor`
|
||||
alone gets a refusal that reads like a stale view. The test passes the
|
||||
page's branch, so it doesn't show this. Either say in the comment that a
|
||||
cursor call must repeat `page.branch`, or answer 400 "cursor requires
|
||||
branch". I'd take the comment now and let CHAT-03's client decide.
|
||||
2. **New codes fall to 422.** A code the reader adds later maps to 422
|
||||
without a test failing. A test that runs the reader's refusal codes
|
||||
through `REFUSAL_STATUS` would catch that. That's optional.
|
||||
@@ -1,69 +0,0 @@
|
||||
# CHAT-02 board routes, revision 2: Darkwing's review (#1507)
|
||||
|
||||
Reviewer: Darkwing, 2026-09-26, at Sage's request. Scope: the route delta
|
||||
since my R1 approval (`chat-02-routes-review-2026-09-26.md`, 07b10ad1). Filbert
|
||||
reviewed the backend (packet `agents/dewey/work/chat-02/BACKEND.md`, 0cf177b1).
|
||||
|
||||
Candidate, base 34777c56, uncommitted. I verified both hashes:
|
||||
- `packages/control-board/src/serve.mjs` d62720dc…a2f3
|
||||
- `packages/control-board/tests/serve.test.mjs` d38aa2b2…3f4a
|
||||
|
||||
**Verdict: approve.** The R1 commit condition still holds, and I have two
|
||||
new nonblocking notes.
|
||||
|
||||
## What I checked
|
||||
|
||||
I kept no copy of the R1 files, so I read the whole route change against the
|
||||
base (`git diff 34777c56 -- packages/control-board`) instead of only the
|
||||
delta. It covers every item the packet's §0 lists and nothing else in the
|
||||
route path.
|
||||
|
||||
- **Cursor needs its branch.** This was my R1 note 1. `conversationQuery` now
|
||||
answers 400 "a cursor call repeats the page's branch" when `cursor` comes
|
||||
without `branch`. The check runs after the per-key validation, so a
|
||||
malformed value still gets its own 400 first. The header comment says the
|
||||
same. A test covers it, and removing the line fails it.
|
||||
- **Status map.** My R1 note 2. `REFUSAL_STATUS` is exported and now has 16
|
||||
entries. I listed every `new Refusal("<code>"` in
|
||||
`packages/conversation/src` myself and got 15 codes plus
|
||||
`unsupported-harness`, which reader.mjs:352 raises by value. That matches the
|
||||
map exactly. `parts.mjs` raises none. `unavailable`, which safe-fs.mjs:58
|
||||
raises when a session root doesn't exist, is 404. That fits the rest of the
|
||||
map, where not-found is 404. `unknown-actor` 403 and `unsupported-purpose` 422
|
||||
are explicit now.
|
||||
- **Order and guards** are unchanged from R1. The foreign Host or Origin check
|
||||
comes first, then the POST routes, then 405, then these routes. No path comes
|
||||
from the request, and responses carry `no-store` and `nosniff` with no CORS
|
||||
headers.
|
||||
- **Tests.** `serve.test.mjs` plus `packages/conversation/tests/` pass
|
||||
67/67 on the pinned files.
|
||||
- **Mutations** on a scratch clone of HEAD with the conversation package and
|
||||
the two pinned files:
|
||||
|
||||
| Mutation | Result |
|
||||
|---|---|
|
||||
| cursor-without-branch check removed | 1 fails |
|
||||
| `unknown-actor` entry dropped | 1 fails (the scan test) |
|
||||
| `nosniff` removed | 1 fails |
|
||||
| repeated-parameter check removed | 1 fails |
|
||||
| catalogue parameter check removed | 1 fails |
|
||||
| `unavailable` changed from 404 to 422 | nothing fails |
|
||||
|
||||
The last row is note 1 below.
|
||||
|
||||
## Commit condition (unchanged)
|
||||
|
||||
`serve.mjs` imports `../../conversation/src/reader.mjs` at module load, and
|
||||
`packages/conversation/` is still untracked. The package must land in the
|
||||
same commit as the routes or an earlier one. Otherwise the board fails to
|
||||
start.
|
||||
|
||||
## Notes (nonblocking)
|
||||
|
||||
1. **The scan test checks keys, not values.** It proves every code has an
|
||||
entry. No test proves `unavailable` is 404. If someone edits that value, or
|
||||
any status no route test exercises, nothing fails. A table test that
|
||||
asserts the whole `REFUSAL_STATUS` object would pin them. That's optional.
|
||||
2. **The scan reads a fixed list of three files.** If a refusal is added to
|
||||
`parts.mjs` or a new file, the scan won't see it, and that code falls to 422.
|
||||
Reading every `.mjs` in `packages/conversation/src` would close the gap.
|
||||
@@ -1,28 +0,0 @@
|
||||
# Independent acceptance checklist, row 18
|
||||
|
||||
Darkwing reviews Filbert's implementation without editing its source candidate.
|
||||
Dewey reviews visible connector presentation. No live connector manipulation.
|
||||
|
||||
- Discovery accepts only safe matching binding name/seat from private regular
|
||||
files, never dereferences a token path and never serializes private fields.
|
||||
- Path traversal, symlinked binding/runtime/session paths and malformed records
|
||||
cannot cause arbitrary reads or an actionable/live row.
|
||||
- No owner, malformed owner, dead PID, missing identity, reused PID and boot
|
||||
mismatch are non-live. A positively matching live process is live.
|
||||
- STOP presence is visible as braked independently of process liveness. Its
|
||||
contents are not read or exposed; no STOP or lock is created or changed.
|
||||
- Ordinary completed messages remain idle. No false human attention regression.
|
||||
- Connector rows cannot borrow a native agent's registration for replies.
|
||||
Exercise replyToRow and HTTP using a fake executable hook; every connector
|
||||
attempt must be refused before that hook runs, including with forged tmux
|
||||
registration. Normal-agent reply tests must still pass.
|
||||
- Both existing board and WebUI distinguish the connector and brake state and
|
||||
omit reply controls. Preserve escaping, including hostile binding fixtures.
|
||||
- Discovery errors disclose no private JSON fields or raw contents. One bad
|
||||
binding must not silently manufacture a healthy row.
|
||||
- Candidate pins match before and after tests. Existing dirty attention changes
|
||||
remain intact; no unrelated source integration or live operation is inferred.
|
||||
|
||||
After source approval, measure the real row read-only. Offline/braked behavior
|
||||
uses isolated fixtures unless the operator separately approves a live-service
|
||||
transition. Board replacement is its own protected gate.
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T13:51:10.530662+00:00",
|
||||
"backendPid": 3769124,
|
||||
"health": "ok",
|
||||
"row": {
|
||||
"agent": "sage (discord: shared-signals)",
|
||||
"project": "fleet",
|
||||
"state": "idle",
|
||||
"alive": true,
|
||||
"connector": {
|
||||
"binding": "shared-signals",
|
||||
"braked": false,
|
||||
"ownerState": "live",
|
||||
"alive": true
|
||||
},
|
||||
"task": "Discord connector",
|
||||
"taskSource": "connector"
|
||||
},
|
||||
"replyStatus": 409,
|
||||
"replyError": "board replies are disabled for Discord connectors",
|
||||
"fiveAgentPaneIdentitiesUnchanged": true,
|
||||
"connectorServiceIdentityUnchanged": true
|
||||
}
|
||||
@@ -1,2 +0,0 @@
|
||||
{"at": "2026-09-14T13:50:24.078636+00:00", "event": "owner-authorized-restart-intent", "oldPid": 3204655, "agents": {"default/darkwing": [["2733924", "12863634"]], "default/dewey": [["934346", "466065"]], "default/filbert": [["72183", "100870"]], "default/researcher": [["173699", "66404285"]], "mosaic-fleet/rocko": [["90599", "128275"]]}, "connectorService": [3022843, "67887873"], "manifest": "254403b89c0a2330da53e8dbad1cbeba3b1b06cf4f3efddc18451e04cb78f6de"}
|
||||
{"at": "2026-09-14T13:50:24.503231+00:00", "event": "replacement-started", "oldExitedGracefully": true, "newPid": 3769124, "log": "/tmp/discord-board-backend-ovk_cahk.log"}
|
||||
@@ -1,18 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T01:10:19.375709+00:00",
|
||||
"candidate": "/tmp/discord-board-r1-KbMrGQWF",
|
||||
"manifestSha256": "5c92acc90d202727d790f3fb8d74387db1c9e5c3e43d4f4b56b40e5ae503a56a",
|
||||
"verdict": "CHANGES REQUIRED",
|
||||
"independentSerializedTests": 320,
|
||||
"finding": {
|
||||
"id": "R1-B1",
|
||||
"severity": "P2",
|
||||
"file": "packages/control-board/src/discord.mjs",
|
||||
"issue": "STOP metadata access errors collapse to absence, falsely projecting not braked",
|
||||
"reproduction": "Synthetic journal directory contains STOP, chmod directory to 000 as uid 1000, inspectDiscord returns braked:false, ownerState:invalid, alive:false. Restore permissions and remove fixture.",
|
||||
"expected": "braked:null/unknown when STOP existence cannot be established; false only for verified absence",
|
||||
"required": "Distinguish missing metadata from access errors and add non-root unreadable-directory regression."
|
||||
},
|
||||
"ux": "Dewey APPROVE on exact R1; three independent serialized browser tests passed, source/automation limitations retained",
|
||||
"parallelQualification": "Two author concurrent frozen timeouts remain unresolved and are not green; serialized independent run passed."
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"candidate": "R2",
|
||||
"syntheticOnly": true,
|
||||
"taskContainsEnvelopeAuthorId": true,
|
||||
"taskContainsEnvelopeMessageId": true,
|
||||
"taskSource": "first-user-message"
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T01:26:42.688410+00:00",
|
||||
"candidate": "/tmp/discord-board-r3-U9vVrlQu",
|
||||
"manifestSha256": "254403b89c0a2330da53e8dbad1cbeba3b1b06cf4f3efddc18451e04cb78f6de",
|
||||
"reviewer": "Darkwing",
|
||||
"backendVerdict": "APPROVE AS SOURCE",
|
||||
"verified": "Nine working/frozen pins, exact three-file R2-to-R3 delta, inherited attention pins and full serialized six-package suite 322/322",
|
||||
"findingsClosed": [
|
||||
"R1-B1: inaccessible STOP is unknown, non-root regression passes",
|
||||
"R2-B2: canonical routing envelope no longer becomes connector Task; ordinary fallback retained"
|
||||
],
|
||||
"limitations": [
|
||||
"No generalized transcript redaction",
|
||||
"R1 concurrent combined frozen timeouts unresolved/not green",
|
||||
"No live observation, backend restart, connector change or publication in this review"
|
||||
],
|
||||
"uxGate": "Await exact R3 confirmation from Dewey via agent-send"
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T01:28:17.695Z",
|
||||
"sourceApproval": 26257,
|
||||
"readOnly": true,
|
||||
"agent": "sage (discord: shared-signals)",
|
||||
"project": "fleet",
|
||||
"state": "idle",
|
||||
"alive": true,
|
||||
"connector": {
|
||||
"binding": "shared-signals",
|
||||
"braked": false,
|
||||
"ownerState": "live",
|
||||
"alive": true
|
||||
},
|
||||
"task": "Discord connector",
|
||||
"taskSource": "connector",
|
||||
"registrationAbsent": true,
|
||||
"ownerMatchesService": true,
|
||||
"discoveryErrorCount": 0
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
{
|
||||
"at": "2026-09-14T00:50:35.339Z",
|
||||
"sourceSha256": "dfbb7ab9374c0ac9fafa0503f495abd938f499f5d6227233de03a60ea3022927",
|
||||
"fixture": "connector row with forged native registration",
|
||||
"status": 200,
|
||||
"fakeTransportCalls": 1,
|
||||
"realTransportCalls": 0,
|
||||
"gatePassed": false
|
||||
}
|
||||
@@ -1,162 +0,0 @@
|
||||
# Discord engine: guaranteed test cleanup and the timeout gap in `busy` (#1509), R2 candidate
|
||||
|
||||
Sage assigned this on 2026-09-26 as 6b, source only. Rocko reviews. Base is
|
||||
HEAD 401cc850. Not committed. The live connector runs from this checkout, so
|
||||
Sage is holding its restart until this is approved and committed. Nobody
|
||||
should restart it from a working copy.
|
||||
|
||||
## Defects (DEFERRED Open, "Discord engine: leaked fake pi…")
|
||||
|
||||
(a) `engine.test.mjs` read the fake's `commands.jsonl` 20 ms after a prompt and
|
||||
got ENOENT under load. Seven tests stopped the engine outside `finally`, so a
|
||||
failed assertion left the fake pi running and the test file never exited.
|
||||
|
||||
(b) `busy` was `state.busy || pending.some((t) => !t.done)`. If a turn timed out
|
||||
before its `agent_start` was read, it was done while `state.busy` was still
|
||||
false. The next prompt then went straight to pi, which refused it as
|
||||
streaming.
|
||||
|
||||
## R1 and Rocko's finding
|
||||
|
||||
R1 held later prompts behind a failed turn. If pi had sent no `agent_start` for
|
||||
it within a grace period, R1 dropped that turn from the queue and sent the next
|
||||
prompt. Rocko rejected it (F1, High), in
|
||||
`agents/rocko/work/discord-engine-busy-r1-review-2026-09-26.md`, sha256
|
||||
047dbd8f.
|
||||
|
||||
Pi's events carry no prompt id. The engine attributes them to the front of its
|
||||
queue. Silence until the grace ends does not prove the old run will never come.
|
||||
If pi then runs it, its events land on the new prompt, which R1 had just put at
|
||||
the front. Rocko's reproducer got the old run's answer and its `old.md` tool
|
||||
record back as the new prompt's result. My R1 README said such events "find no
|
||||
live head and are dropped". That was wrong.
|
||||
|
||||
The R1 files stay here as `r1-manifest.sha256` and `r1.patch`.
|
||||
|
||||
## Change (R2)
|
||||
|
||||
`packages/discord/src/engine-pi.mjs`:
|
||||
- `engineBusy()` is `state.busy || state.pending.length > 0`. A failed turn
|
||||
still in the queue holds the next prompt back, and stays at the front, so any
|
||||
late events for it land on it. `prompt()`, `sendHeld()` and the `busy` getter
|
||||
use it. This part is unchanged from R1.
|
||||
- The bound is now a stop, not a drop. When a turn fails while it is still in
|
||||
the queue, `failTurn` starts a timer, `abortGraceMs` (default
|
||||
`ABORT_GRACE_MS`, 30 s, an engine option, not binding config). When it
|
||||
fires:
|
||||
- If pi has sent `agent_start` (`state.busy`), nothing happens. That run
|
||||
ends on its `agent_end` or a settle, as on HEAD.
|
||||
- Otherwise `wedge()` sets `state.wedged`, fails every held prompt with
|
||||
code `engine-wedged`, and stops pi: stdin closed, SIGTERM, then SIGKILL
|
||||
after 5 s. The failed turn stays at the front until the exit.
|
||||
- While wedged, nothing is written to that child. `write()`, `sendHeld()` and
|
||||
`prompt()` refuse, and a new prompt fails at once with `engine-down`. The exit
|
||||
runs the usual `failAll` and `onExit`.
|
||||
- `stop()`'s body moved into `stopChild()`, which both `stop()` and `wedge()`
|
||||
call.
|
||||
- `release()` clears the timer wherever a turn leaves the queue: `agent_end`,
|
||||
settle, a refused send, and process exit. As in R1, the settle handler removes
|
||||
turns before failing them.
|
||||
|
||||
What recovery looks like live: `cli.mjs` handles `onExit` with `shutdown(1)`.
|
||||
The unit's `Restart=on-failure` starts a new connector and a new pi 15 s later,
|
||||
within its limit of five tries in ten minutes. This change doesn't touch the
|
||||
unit or the restart policy. A wedge now costs one connector restart. R1 would
|
||||
have kept the same pi and risked a wrong answer.
|
||||
|
||||
`packages/discord/tests/fake-pi.mjs`:
|
||||
- `mute`: accepted and never run.
|
||||
- `stall <ms>`: accepted, then the fake reads nothing for `<ms>`, runs the
|
||||
stalled prompt, and only then reads what came in meanwhile. This is the
|
||||
order in Rocko's case.
|
||||
|
||||
`packages/discord/tests/engine.test.mjs`:
|
||||
- `withEngine()` stops the engine in `finally`. Every test that starts an
|
||||
engine uses it, or has its own `try/finally` in the exit test.
|
||||
- `commands()` returns `[]` until the fake creates its log. The held-prompt
|
||||
test waits for the first prompt with `until()` instead of a 20 ms sleep, and
|
||||
its first prompt is `slow 300`.
|
||||
- The manual-timer test fires the turn timer before any pi event is read. The
|
||||
next prompt must wait for the settle and get its own answer.
|
||||
- New or changed for R2:
|
||||
- `mute` with `abortGraceMs: 150`. The held prompt fails with
|
||||
`engine-wedged` after the grace, a later prompt fails with
|
||||
`engine-down`, `onExit` fires, and pi saw only `mute` and `abort`.
|
||||
- `stall 400` with the same grace, which is Rocko's case with a real
|
||||
child. The held prompt fails with `engine-wedged`, pi exits, and "after
|
||||
stall" never reaches pi.
|
||||
- Rocko's reproducer as an in-memory test, run twice. The old prompt's
|
||||
response comes either before its timeout or only with the late events.
|
||||
After the grace, the old run's start, tool pair, answer, end and settle
|
||||
arrive while pi is still exiting. The held prompt stays failed with
|
||||
`engine-wedged` and a later prompt fails with `engine-down`. Pi saw only
|
||||
`old` and `abort`, then SIGTERM, then SIGKILL at 5 s. Only the exit
|
||||
reaches `onExit`. The test reads recorded outcomes after a tick instead
|
||||
of awaiting, so a regression fails instead of hanging.
|
||||
- `late 400` with the same grace. Pi started that run, so the grace does
|
||||
not stop pi, and the next prompt gets its own answer when the run ends.
|
||||
|
||||
## Evidence
|
||||
|
||||
- `engine.test.mjs`: 17/17.
|
||||
- R1's engine (d5bf24b5, from `r1.patch`) against these tests fails 4: `mute`,
|
||||
`stall`, and both in-memory runs. In that run a probe shows R1 answering
|
||||
"after stall" with "echo: stalled". An earlier draft of the in-memory test
|
||||
awaited the held prompt and hung on R1 until the 120 s cap. It now fails in
|
||||
milliseconds.
|
||||
- HEAD's engine against these tests fails 5: the manual-timer test and the
|
||||
same four.
|
||||
- Mutations of R2:
|
||||
- Without the `state.busy` check, the `late 400` test fails.
|
||||
- Without the `wedge()` call, 4 fail.
|
||||
- Without failing held prompts in `wedge()`, 4 fail.
|
||||
- Test union (control-board, webui, seat, mosaic, ledger, discord) at default
|
||||
concurrency on `git archive` of 401cc850 plus the three files: 406/406
|
||||
three times, 23 to 24 s each. No fake pi was left running.
|
||||
- Eight suites green on that snapshot: config 24, task 90, foundation 43,
|
||||
conductor 17, release 14, auth 15, discord 63, extension-package 18.
|
||||
|
||||
Logs: `/tmp/dw-6b-r2-conc-{1,2,3}.txt`. R1's evidence runs:
|
||||
`/tmp/dw-6b-conc-{1,2,3}.txt`, `/tmp/dw-6b-serial.txt`. HEAD's hang control:
|
||||
`/tmp/dw-1509-headctl-{1,2,3}.txt`, `/tmp/dw-1509-ef00-1.txt`. There, HEAD hit
|
||||
the 240 s cap at 199 ok under the union's load.
|
||||
|
||||
## Not covered
|
||||
|
||||
- A run pi started and never ends, even after abort, still holds prompts
|
||||
until pi settles or exits. Each held prompt fails at its own timeout ("while
|
||||
waiting for the engine"). HEAD behaves the same way through `state.busy`, and
|
||||
Rocko did not block on it. Only a pi restart clears it.
|
||||
- A wedge ends the connector process, and the recovery is systemd's restart.
|
||||
Nothing here changes the unit, and the restart limit still applies.
|
||||
- No live restart, and no change to the binding schema.
|
||||
|
||||
## Frozen files
|
||||
|
||||
`r2-manifest.sha256` holds the three R2 hashes. `r2.patch` is `git diff
|
||||
packages/discord` at freeze time.
|
||||
|
||||
## Review
|
||||
|
||||
Rocko, R1, 2026-09-26: request changes, F1 High, as described above. Report:
|
||||
`agents/rocko/work/discord-engine-busy-r1-review-2026-09-26.md`, sha256
|
||||
047dbd8f.
|
||||
|
||||
Rocko, R2, 2026-09-26: approved the three pinned files. Report:
|
||||
`agents/rocko/work/discord-engine-busy-r2-review-2026-09-26.md`, sha256
|
||||
ed5510a0. He checked the manifests before and after, ran 17/17 himself, and
|
||||
read the CLI shutdown path, `connector.stop` and the unit template. Sage asked
|
||||
him three operational questions:
|
||||
- A wedge exits 1, never 3. Exit 3 remains the supervised startup refusal.
|
||||
- The unit's start limit (5 starts in 600 s) is a rate limit. It does not
|
||||
bound repeated wedges. With the default 180 s turn timeout, the 30 s grace
|
||||
and the 15 s restart delay, a cycle takes at least 225 s. That stays under
|
||||
the limit, so a pi that wedges every time could restart indefinitely.
|
||||
Stopping for good after repeated wedges would need a separate policy. This
|
||||
change does not add one.
|
||||
- He recommends, as a nonblocking follow-up, that the connector journal
|
||||
record at startup: HEAD, dirty state scoped to runtime source, and a digest
|
||||
of the runtime files. A wedge restart loads whatever the checkout holds.
|
||||
|
||||
This section was added after approval, so the README hash no longer matches
|
||||
the one Rocko pinned (69350f29). The three source files are unchanged.
|
||||
@@ -1,3 +0,0 @@
|
||||
d5bf24b59c07c85067f4087c03b54ca8b4df923c1d591441dedd2e8a7ff2ae39 packages/discord/src/engine-pi.mjs
|
||||
f0abee9c243d46d66dd2271abc3fd89089c350ac6a66ab49131bce80adfcdc33 packages/discord/tests/engine.test.mjs
|
||||
fa1bf44e3f33eb970a714ada1c686abbc1932baaf418679276edf8813abbe6de packages/discord/tests/fake-pi.mjs
|
||||
@@ -1,414 +0,0 @@
|
||||
diff --git a/packages/discord/src/engine-pi.mjs b/packages/discord/src/engine-pi.mjs
|
||||
index 5c8fd0a9..9dcaeb42 100644
|
||||
--- a/packages/discord/src/engine-pi.mjs
|
||||
+++ b/packages/discord/src/engine-pi.mjs
|
||||
@@ -16,9 +16,12 @@
|
||||
// from `tool_execution_start`/`tool_execution_end` into the result so the
|
||||
// turn record shows what was read. An `agent_end` with `willRetry` is not
|
||||
// the end of the run. A timeout sends `abort` and fails that turn; the
|
||||
-// process stays. A malformed JSONL line from pi fails the current turn (its
|
||||
-// outcome is now unknowable) and the process stays. Process exit fails
|
||||
-// every pending turn and is reported through `onExit`.
|
||||
+// process stays. The failed turn holds later prompts back until its
|
||||
+// agent_end or a settle. If pi has not started it within ABORT_GRACE_MS, it
|
||||
+// is dropped and the next prompt goes out; a run pi did start holds them
|
||||
+// until it ends, as any run does. A malformed JSONL line from pi fails the
|
||||
+// current turn (its outcome is now unknowable) and the process stays.
|
||||
+// Process exit fails every pending turn and is reported through `onExit`.
|
||||
//
|
||||
// Framing follows pi's RPC doc: split on "\n" only, strip a trailing "\r".
|
||||
// Node readline is not used because it also splits on U+2028/U+2029.
|
||||
@@ -64,12 +67,19 @@ export function assistantText(message) {
|
||||
.trim();
|
||||
}
|
||||
|
||||
+// How long a turn that failed here (timeout, protocol error) may wait for
|
||||
+// pi's agent_start before it stops holding the next prompt back. Without a
|
||||
+// bound, a prompt pi accepted but never ran would queue every later prompt
|
||||
+// until restart.
|
||||
+export const ABORT_GRACE_MS = 30000;
|
||||
+
|
||||
export function createEngine({
|
||||
command, args, cwd, env = {},
|
||||
spawn = nodeSpawn,
|
||||
setTimeoutImpl = globalThis.setTimeout, clearTimeoutImpl = globalThis.clearTimeout,
|
||||
log = () => {},
|
||||
onExit = () => {},
|
||||
+ abortGraceMs = ABORT_GRACE_MS,
|
||||
} = {}) {
|
||||
if (typeof command !== "string" || command.length === 0) throw new DiscordError("engine: command required", 1);
|
||||
if (!Array.isArray(args)) throw new DiscordError("engine: args required", 1);
|
||||
@@ -80,15 +90,35 @@ export function createEngine({
|
||||
|
||||
// A turn that fails on the client side (timeout, protocol error) stays in
|
||||
// the pending queue, marked done, until pi's own turn_end for it arrives.
|
||||
- // Otherwise that turn_end would be attributed to the next prompt.
|
||||
+ // Otherwise that turn_end would be attributed to the next prompt. It holds
|
||||
+ // later prompts back; if pi has not started it within abortGraceMs, it goes.
|
||||
function failTurn(turn, code, message) {
|
||||
if (turn.done) return;
|
||||
turn.done = true;
|
||||
if (turn.timer !== null) clearTimeoutImpl(turn.timer);
|
||||
turn.timer = null;
|
||||
+ if (state.pending.includes(turn)) {
|
||||
+ turn.grace = setTimeoutImpl(() => {
|
||||
+ turn.grace = null;
|
||||
+ // No agent_start by now: pi never started this run and will send no
|
||||
+ // agent_end for it, so it leaves the queue and cannot take the next
|
||||
+ // prompt's. A run pi did start keeps its place until it ends.
|
||||
+ if (!state.busy) {
|
||||
+ const i = state.pending.indexOf(turn);
|
||||
+ if (i !== -1) state.pending.splice(i, 1);
|
||||
+ }
|
||||
+ sendHeld();
|
||||
+ }, abortGraceMs);
|
||||
+ }
|
||||
turn.reject(new DiscordError(message, 1, { code }));
|
||||
}
|
||||
|
||||
+ // Call when a turn leaves the pending queue.
|
||||
+ function release(turn) {
|
||||
+ if (turn.grace !== null) clearTimeoutImpl(turn.grace);
|
||||
+ turn.grace = null;
|
||||
+ }
|
||||
+
|
||||
function settleTurn(turn, value) {
|
||||
if (turn.done) return;
|
||||
turn.done = true;
|
||||
@@ -99,7 +129,10 @@ export function createEngine({
|
||||
|
||||
function failAll(code, message) {
|
||||
const pending = state.pending.splice(0);
|
||||
- for (const t of pending) failTurn(t, code, message);
|
||||
+ for (const t of pending) {
|
||||
+ release(t);
|
||||
+ failTurn(t, code, message);
|
||||
+ }
|
||||
for (const h of state.held.splice(0)) failTurn(h.turn, code, message);
|
||||
for (const [, r] of state.responses) r.reject(new DiscordError(message, 1, { code }));
|
||||
state.responses.clear();
|
||||
@@ -174,6 +207,7 @@ export function createEngine({
|
||||
// Attribute the run to the head even if it failed client-side, so the
|
||||
// next prompt's agent_end is not taken for this one.
|
||||
const run = state.pending.shift();
|
||||
+ if (run) release(run);
|
||||
if (!run || run.done) return;
|
||||
const messages = Array.isArray(event.messages) ? event.messages.filter((m) => m && m.role === "assistant") : [];
|
||||
const message = messages.length > 0 ? messages[messages.length - 1] : run.last;
|
||||
@@ -193,13 +227,14 @@ export function createEngine({
|
||||
// this settle and still has no agent_end will never get one: fail it now
|
||||
// instead of waiting for its timeout. Turns whose prompt response has
|
||||
// not arrived yet belong to a later run and stay.
|
||||
+ const dropped = [];
|
||||
const keep = [];
|
||||
- for (const t of state.pending) {
|
||||
- if (t.done) continue;
|
||||
- if (t.accepted) failTurn(t, "engine-settled-without-turn", "engine settled without answering this prompt");
|
||||
- else keep.push(t);
|
||||
- }
|
||||
+ for (const t of state.pending) (t.done || t.accepted ? dropped : keep).push(t);
|
||||
state.pending = keep;
|
||||
+ for (const t of dropped) {
|
||||
+ release(t);
|
||||
+ failTurn(t, "engine-settled-without-turn", "engine settled without answering this prompt");
|
||||
+ }
|
||||
sendHeld();
|
||||
}
|
||||
}
|
||||
@@ -214,15 +249,23 @@ export function createEngine({
|
||||
// Never accepted: pi will not emit a turn_end for it, so remove it.
|
||||
const i = state.pending.indexOf(turn);
|
||||
if (i !== -1) state.pending.splice(i, 1);
|
||||
+ release(turn);
|
||||
failTurn(turn, (err.details && err.details.code) || "engine-refused", err.message);
|
||||
sendHeld();
|
||||
});
|
||||
}
|
||||
|
||||
+ // Pi is busy from our side while any sent prompt is still queued, even one
|
||||
+ // that already failed here: a turn that timed out before its agent_start
|
||||
+ // was read leaves state.busy false while pi runs it, and sending then would
|
||||
+ // be refused as streaming. It leaves the queue on its agent_end, on a
|
||||
+ // settle, on a refused send, or when its grace ends before pi started it.
|
||||
+ const engineBusy = () => state.busy || state.pending.length > 0;
|
||||
+
|
||||
// After a settle (or a refused send) the oldest held prompt goes out.
|
||||
function sendHeld() {
|
||||
if (state.exited !== null) return;
|
||||
- if (state.busy || state.pending.some((t) => !t.done)) return;
|
||||
+ if (engineBusy()) return;
|
||||
const next = state.held.shift();
|
||||
if (next) send(next.turn, next.command);
|
||||
}
|
||||
@@ -281,7 +324,7 @@ export function createEngine({
|
||||
// with DiscordError carrying details.code for the turn record.
|
||||
prompt(text, { timeoutMs = 180000 } = {}) {
|
||||
if (typeof text !== "string" || text.length === 0) throw new DiscordError("prompt text required", 1);
|
||||
- const turn = { resolve: null, reject: null, timer: null, done: false, accepted: false, tools: new Map(), turns: 0, last: null };
|
||||
+ const turn = { resolve: null, reject: null, timer: null, grace: null, done: false, accepted: false, tools: new Map(), turns: 0, last: null };
|
||||
const done = new Promise((resolve, reject) => {
|
||||
turn.resolve = resolve;
|
||||
turn.reject = reject;
|
||||
@@ -310,13 +353,13 @@ export function createEngine({
|
||||
failTurn(turn, "engine-down", "engine is not running");
|
||||
return done;
|
||||
}
|
||||
- if (state.busy || state.pending.some((t) => !t.done) || state.held.length > 0) state.held.push({ turn, command });
|
||||
+ if (engineBusy() || state.held.length > 0) state.held.push({ turn, command });
|
||||
else send(turn, command);
|
||||
return done;
|
||||
},
|
||||
|
||||
get busy() {
|
||||
- return state.busy || state.pending.some((t) => !t.done) || state.held.length > 0;
|
||||
+ return engineBusy() || state.held.length > 0;
|
||||
},
|
||||
get pendingCount() {
|
||||
return state.pending.filter((t) => !t.done).length + state.held.length;
|
||||
diff --git a/packages/discord/tests/engine.test.mjs b/packages/discord/tests/engine.test.mjs
|
||||
index 59674d1e..62dd6017 100644
|
||||
--- a/packages/discord/tests/engine.test.mjs
|
||||
+++ b/packages/discord/tests/engine.test.mjs
|
||||
@@ -28,7 +28,20 @@ function start(root, extra = {}) {
|
||||
log: (m) => logs.push(m), ...extra,
|
||||
});
|
||||
engine.start();
|
||||
- return { engine, logs, commands: () => readFileSync(logPath, "utf8").trim().split("\n").filter(Boolean).map((l) => JSON.parse(l)) };
|
||||
+ // The fake creates its log on the first command; until then there are none.
|
||||
+ const commands = () => (existsSync(logPath) ? readFileSync(logPath, "utf8").trim().split("\n").filter(Boolean).map((l) => JSON.parse(l)) : []);
|
||||
+ return { engine, logs, commands };
|
||||
+}
|
||||
+
|
||||
+// Every test stops its engine in finally: a fake pi left running after a
|
||||
+// failed assertion keeps the test file from exiting.
|
||||
+async function withEngine(extra, body) {
|
||||
+ const started = start(makeRoot(), extra);
|
||||
+ try {
|
||||
+ await body(started);
|
||||
+ } finally {
|
||||
+ await started.engine.stop();
|
||||
+ }
|
||||
}
|
||||
|
||||
test("engine: buildPiArgs carries the fixed flags, engine settings, session dir and prompt file", () => {
|
||||
@@ -56,8 +69,7 @@ test("engine: with tools, buildPiArgs turns pi's own tools off, loads the extens
|
||||
assert.equal(rw[rw.indexOf("--tools") + 1], "list_dir,read_file,search,write_file,edit_file", "a writable root adds exactly the two write tools");
|
||||
});
|
||||
|
||||
-test("engine: a run with tool turns settles once, on the answer, with every tool call in the result", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
+test("engine: a run with tool turns settles once, on the answer, with every tool call in the result", () => withEngine({}, async ({ engine }) => {
|
||||
const r = await engine.prompt("tools 3");
|
||||
assert.equal(r.text, "read 3 file(s)");
|
||||
assert.equal(r.turns, 2);
|
||||
@@ -71,45 +83,38 @@ test("engine: a run with tool turns settles once, on the answer, with every tool
|
||||
assert.equal(plain.turns, 1);
|
||||
await idle(engine);
|
||||
assert.equal(engine.busy, false);
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: a run that ends on a tool-only turn fails the prompt as empty; a retried run settles on the real end", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
+test("engine: a run that ends on a tool-only turn fails the prompt as empty; a retried run settles on the real end", () => withEngine({}, async ({ engine }) => {
|
||||
const r = await engine.prompt("toolonly");
|
||||
assert.equal(r.text, "", "no text: the connector turns this into engine-empty");
|
||||
assert.equal(r.tools.length, 1);
|
||||
const again = await engine.prompt("retry");
|
||||
assert.equal(again.text, "after retry");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: one prompt, one turn, text and usage come back", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
- try {
|
||||
- const r = await engine.prompt("hello");
|
||||
- assert.equal(r.text, "echo: hello");
|
||||
- assert.deepEqual(r.usage, { input: 3, output: 2 });
|
||||
- await idle(engine);
|
||||
- assert.equal(engine.busy, false);
|
||||
- } finally {
|
||||
- await engine.stop();
|
||||
- }
|
||||
-});
|
||||
+test("engine: one prompt, one turn, text and usage come back", () => withEngine({}, async ({ engine }) => {
|
||||
+ const r = await engine.prompt("hello");
|
||||
+ assert.equal(r.text, "echo: hello");
|
||||
+ assert.deepEqual(r.usage, { input: 3, output: 2 });
|
||||
+ await idle(engine);
|
||||
+ assert.equal(engine.busy, false);
|
||||
+}));
|
||||
|
||||
-test("engine: a prompt while streaming is held until pi settles, then sent as its own run, and answered in order", async () => {
|
||||
- const { engine, commands } = start(makeRoot());
|
||||
- const first = engine.prompt("slow 150");
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
+test("engine: a prompt while streaming is held until pi settles, then sent as its own run, and answered in order", () => withEngine({}, async ({ engine, commands }) => {
|
||||
+ const first = engine.prompt("slow 300");
|
||||
assert.equal(engine.busy, true);
|
||||
const second = engine.prompt("second");
|
||||
assert.equal(engine.pendingCount, 2);
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
- assert.equal(commands().filter((c) => c.type === "prompt").length, 1, "the second prompt is not sent while pi is busy");
|
||||
+ const prompted = () => commands().filter((c) => c.type === "prompt");
|
||||
+ assert.ok(await until(() => prompted().length > 0), "the first prompt reached pi");
|
||||
+ assert.equal(prompted().length, 1, "the second prompt is not sent while pi is busy");
|
||||
+ // The fake refuses a prompt without streamingBehavior while it runs one, so
|
||||
+ // an answered second prompt also proves it was not sent early.
|
||||
const [r1, r2] = await Promise.all([first, second]);
|
||||
assert.equal(r1.text, "slow reply");
|
||||
assert.equal(r2.text, "echo: second");
|
||||
- const prompts = commands().filter((c) => c.type === "prompt");
|
||||
+ const prompts = prompted();
|
||||
assert.equal(prompts.length, 2);
|
||||
// Never a pi follow-up: pi would fold it into the first run and close both
|
||||
// answers with one agent_end (the live loss of 2026-09-17).
|
||||
@@ -117,11 +122,9 @@ test("engine: a prompt while streaming is held until pi settles, then sent as it
|
||||
assert.equal(prompts[1].streamingBehavior, undefined);
|
||||
await idle(engine);
|
||||
assert.equal(engine.busy, false);
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: a held prompt that times out before pi settles fails on its own and is never sent", async () => {
|
||||
- const { engine, commands } = start(makeRoot());
|
||||
+test("engine: a held prompt that times out before pi settles fails on its own and is never sent", () => withEngine({}, async ({ engine, commands }) => {
|
||||
const first = engine.prompt("slow 200");
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
await assert.rejects(engine.prompt("late one", { timeoutMs: 50 }), (e) => e.details.code === "timeout" && /waiting for the engine/.test(e.message));
|
||||
@@ -130,50 +133,85 @@ test("engine: a held prompt that times out before pi settles fails on its own an
|
||||
await idle(engine);
|
||||
assert.deepEqual(commands().filter((c) => c.type === "prompt").map((c) => c.message), ["slow 200"]);
|
||||
assert.deepEqual(commands().filter((c) => c.type === "abort"), [], "a held turn is not aborted; pi never had it");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: timeout sends abort and fails only that turn; the process stays", async () => {
|
||||
- const { engine, commands, logs } = start(makeRoot());
|
||||
+test("engine: timeout sends abort and fails only that turn; the process stays", () => withEngine({}, async ({ engine, commands, logs }) => {
|
||||
await assert.rejects(engine.prompt("slow 5000", { timeoutMs: 100 }), (err) => err.details.code === "timeout");
|
||||
assert.ok(await until(() => commands().some((c) => c.type === "abort")), "abort reached pi");
|
||||
assert.ok(logs.some((l) => /timed out/.test(l)));
|
||||
const r = await engine.prompt("again");
|
||||
assert.equal(r.text, "echo: again");
|
||||
- await engine.stop();
|
||||
+}));
|
||||
+
|
||||
+test("engine: tool events from a run that outlived its timeout never land in the next prompt's record", () => withEngine({}, async ({ engine }) => {
|
||||
+ await assert.rejects(engine.prompt("late 200", { timeoutMs: 40 }), (err) => err.details.code === "timeout");
|
||||
+ const r = await engine.prompt("after late");
|
||||
+ assert.equal(r.text, "echo: after late");
|
||||
+ assert.deepEqual(r.tools, [], "the dead run's read is not this prompt's evidence");
|
||||
+ assert.equal(r.turns, 1, "the dead run's turns are not counted here");
|
||||
+}));
|
||||
+
|
||||
+// The turn timer is fired by hand, before the engine has read any event from
|
||||
+// pi, so the timed-out run is still pi's and state.busy is still false when
|
||||
+// the next prompt arrives. Under load a real timer does the same.
|
||||
+const TURN_MS = 60000;
|
||||
+const manualTurnTimer = (fire) => ({
|
||||
+ setTimeoutImpl: (fn, ms) => (ms === TURN_MS ? fire.push(fn) : setTimeout(fn, ms)),
|
||||
+ clearTimeoutImpl: (id) => { if (typeof id !== "number") clearTimeout(id); },
|
||||
});
|
||||
|
||||
-test("engine: tool events from a run that outlived its timeout never land in the next prompt's record", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
- try {
|
||||
- await assert.rejects(engine.prompt("late 200", { timeoutMs: 40 }), (err) => err.details.code === "timeout");
|
||||
- const r = await engine.prompt("after late");
|
||||
+test("engine: a prompt after a turn that timed out before its agent_start waits for pi to settle instead of being refused", () => {
|
||||
+ const fire = [];
|
||||
+ return withEngine(manualTurnTimer(fire), async ({ engine, commands }) => {
|
||||
+ const late = engine.prompt("late 100", { timeoutMs: TURN_MS });
|
||||
+ fire.shift()();
|
||||
+ assert.equal(engine.busy, true, "pi is still running the prompt that timed out");
|
||||
+ const next = engine.prompt("after late", { timeoutMs: 5000 });
|
||||
+ assert.equal(engine.pendingCount, 1, "only the new prompt is live");
|
||||
+ await assert.rejects(late, (err) => err.details.code === "timeout");
|
||||
+ const r = await next;
|
||||
assert.equal(r.text, "echo: after late");
|
||||
assert.deepEqual(r.tools, [], "the dead run's read is not this prompt's evidence");
|
||||
- assert.equal(r.turns, 1, "the dead run's turns are not counted here");
|
||||
- } finally {
|
||||
- await engine.stop();
|
||||
- }
|
||||
+ assert.equal(r.turns, 1);
|
||||
+ assert.deepEqual(commands().map((c) => (c.type === "prompt" ? c.message : c.type)), ["late 100", "abort", "after late"]);
|
||||
+ await idle(engine);
|
||||
+ assert.equal(engine.busy, false);
|
||||
+ });
|
||||
});
|
||||
|
||||
-test("engine: a malformed JSONL line fails the turn, not the process", async () => {
|
||||
- const { engine, logs } = start(makeRoot());
|
||||
+// "mute" is accepted and never run, so no agent_start, agent_end or settle
|
||||
+// ever comes for it. Unbounded, it would hold every later prompt.
|
||||
+test("engine: a timed-out turn pi never started holds the next prompt only for the abort grace, then leaves the queue", () => withEngine({ abortGraceMs: 150 }, async ({ engine, commands }) => {
|
||||
+ await assert.rejects(engine.prompt("mute", { timeoutMs: 50 }), (err) => err.details.code === "timeout");
|
||||
+ assert.equal(engine.busy, true, "pi might still be running it");
|
||||
+ const started = Date.now();
|
||||
+ const r = await engine.prompt("after mute", { timeoutMs: 5000 });
|
||||
+ assert.equal(r.text, "echo: after mute");
|
||||
+ assert.ok(Date.now() - started >= 100, "held for the grace, not sent at once");
|
||||
+ assert.deepEqual(commands().map((c) => (c.type === "prompt" ? c.message : c.type)), ["mute", "abort", "after mute"]);
|
||||
+ await idle(engine);
|
||||
+ assert.equal(engine.busy, false);
|
||||
+}));
|
||||
+
|
||||
+test("engine: a malformed JSONL line fails the turn, not the process", () => withEngine({}, async ({ engine, logs }) => {
|
||||
await assert.rejects(engine.prompt("garbage"), (err) => err.details.code === "engine-protocol");
|
||||
assert.ok(logs.some((l) => /malformed/.test(l)));
|
||||
const r = await engine.prompt("still here");
|
||||
assert.equal(r.text, "echo: still here");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
test("engine: a turn that ends in error rejects with the error code; process exit fails pending turns", async () => {
|
||||
- const root = makeRoot();
|
||||
let exited = null;
|
||||
- const { engine } = start(root, { onExit: (e) => (exited = e) });
|
||||
- await assert.rejects(engine.prompt("error"), (err) => err.details.code === "engine-error" && /fake provider error/.test(err.message));
|
||||
- const pending = engine.prompt("slow 5000");
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
- await engine.stop();
|
||||
- await assert.rejects(pending, (err) => err.details.code === "engine-down");
|
||||
- assert.ok(exited);
|
||||
- await assert.rejects(engine.prompt("x"), /not running/);
|
||||
+ const { engine } = start(makeRoot(), { onExit: (e) => (exited = e) });
|
||||
+ try {
|
||||
+ await assert.rejects(engine.prompt("error"), (err) => err.details.code === "engine-error" && /fake provider error/.test(err.message));
|
||||
+ const pending = engine.prompt("slow 5000");
|
||||
+ await new Promise((r) => setTimeout(r, 20));
|
||||
+ await engine.stop();
|
||||
+ await assert.rejects(pending, (err) => err.details.code === "engine-down");
|
||||
+ assert.ok(exited);
|
||||
+ await assert.rejects(engine.prompt("x"), /not running/);
|
||||
+ } finally {
|
||||
+ await engine.stop();
|
||||
+ }
|
||||
});
|
||||
diff --git a/packages/discord/tests/fake-pi.mjs b/packages/discord/tests/fake-pi.mjs
|
||||
index 94919306..ecc04e3b 100644
|
||||
--- a/packages/discord/tests/fake-pi.mjs
|
||||
+++ b/packages/discord/tests/fake-pi.mjs
|
||||
@@ -8,6 +8,7 @@
|
||||
// then a second turn that answers "read <n> file(s)"
|
||||
// "toolonly" a run whose only turn calls a tool and never answers
|
||||
// "retry" an agent_end with willRetry, then the real answer
|
||||
+// "mute" accept the prompt and emit nothing, staying idle
|
||||
// "late <ms>" ignore abort; after <ms> emit a tool pair and a tool turn,
|
||||
// then answer "late reply", like a run that outlives its
|
||||
// client-side timeout
|
||||
@@ -30,6 +31,7 @@ function assistant(text, stopReason = "stop") {
|
||||
}
|
||||
|
||||
function run(text) {
|
||||
+ if (text === "mute") return;
|
||||
busy = true;
|
||||
out({ type: "agent_start" });
|
||||
out({ type: "turn_start" });
|
||||
@@ -1,3 +0,0 @@
|
||||
77077b7fbd5a933ffd352094eb073227c299ba47b7aea52d4e60fdc55cc7101e packages/discord/src/engine-pi.mjs
|
||||
47a998179c6eb46827f43ab2c6b0f6b062ef94fb47da402cfb9af8c4f180f38e packages/discord/tests/engine.test.mjs
|
||||
a8e54cc3f4b670eef2c06755b63e9e6bfeb44b1b1efde3aca9bfaa91583c3ef3 packages/discord/tests/fake-pi.mjs
|
||||
@@ -1,651 +0,0 @@
|
||||
diff --git a/packages/discord/src/engine-pi.mjs b/packages/discord/src/engine-pi.mjs
|
||||
index 5c8fd0a9..46ef1f88 100644
|
||||
--- a/packages/discord/src/engine-pi.mjs
|
||||
+++ b/packages/discord/src/engine-pi.mjs
|
||||
@@ -16,9 +16,14 @@
|
||||
// from `tool_execution_start`/`tool_execution_end` into the result so the
|
||||
// turn record shows what was read. An `agent_end` with `willRetry` is not
|
||||
// the end of the run. A timeout sends `abort` and fails that turn; the
|
||||
-// process stays. A malformed JSONL line from pi fails the current turn (its
|
||||
-// outcome is now unknowable) and the process stays. Process exit fails
|
||||
-// every pending turn and is reported through `onExit`.
|
||||
+// process stays. The failed turn holds later prompts back until its
|
||||
+// agent_end or a settle. If pi has not started it within ABORT_GRACE_MS, the
|
||||
+// engine stops pi instead of sending again: pi's events carry no prompt id,
|
||||
+// so a late run of the failed prompt would be taken for the next one's. A
|
||||
+// run pi did start holds later prompts until it ends, as any run does. A
|
||||
+// malformed JSONL line from pi fails the current turn (its outcome is now
|
||||
+// unknowable) and the process stays. Process exit fails every pending turn
|
||||
+// and is reported through `onExit`.
|
||||
//
|
||||
// Framing follows pi's RPC doc: split on "\n" only, strip a trailing "\r".
|
||||
// Node readline is not used because it also splits on U+2028/U+2029.
|
||||
@@ -64,31 +69,67 @@ export function assistantText(message) {
|
||||
.trim();
|
||||
}
|
||||
|
||||
+// How long a turn that failed here (timeout, protocol error) may wait for
|
||||
+// pi's agent_start before the engine stops pi. Without a bound, a prompt pi
|
||||
+// accepted but never ran would hold every later prompt until restart.
|
||||
+export const ABORT_GRACE_MS = 30000;
|
||||
+
|
||||
export function createEngine({
|
||||
command, args, cwd, env = {},
|
||||
spawn = nodeSpawn,
|
||||
setTimeoutImpl = globalThis.setTimeout, clearTimeoutImpl = globalThis.clearTimeout,
|
||||
log = () => {},
|
||||
onExit = () => {},
|
||||
+ abortGraceMs = ABORT_GRACE_MS,
|
||||
} = {}) {
|
||||
if (typeof command !== "string" || command.length === 0) throw new DiscordError("engine: command required", 1);
|
||||
if (!Array.isArray(args)) throw new DiscordError("engine: args required", 1);
|
||||
|
||||
// pending: prompts sent to pi, oldest first. held: prompts waiting for pi
|
||||
// to settle before they are sent, oldest first.
|
||||
- const state = { child: null, buffer: "", pending: [], held: [], responses: new Map(), nextId: 1, busy: false, exited: null };
|
||||
+ // wedged: set when the engine gave up on pi and is stopping it. Nothing
|
||||
+ // is sent to that child again.
|
||||
+ const state = { child: null, buffer: "", pending: [], held: [], responses: new Map(), nextId: 1, busy: false, exited: null, wedged: false };
|
||||
|
||||
// A turn that fails on the client side (timeout, protocol error) stays in
|
||||
// the pending queue, marked done, until pi's own turn_end for it arrives.
|
||||
- // Otherwise that turn_end would be attributed to the next prompt.
|
||||
+ // Otherwise that turn_end would be attributed to the next prompt. It holds
|
||||
+ // later prompts back; if pi has not started it within abortGraceMs, the
|
||||
+ // engine stops pi.
|
||||
function failTurn(turn, code, message) {
|
||||
if (turn.done) return;
|
||||
turn.done = true;
|
||||
if (turn.timer !== null) clearTimeoutImpl(turn.timer);
|
||||
turn.timer = null;
|
||||
+ if (state.pending.includes(turn)) {
|
||||
+ turn.grace = setTimeoutImpl(() => {
|
||||
+ turn.grace = null;
|
||||
+ // A run pi started keeps its place until its agent_end or a settle.
|
||||
+ if (state.busy) return;
|
||||
+ // No agent_start yet. Pi may never run this prompt, or its events
|
||||
+ // may still be on the way; with no prompt id in them, nothing sent
|
||||
+ // now could be told apart from it. Stop pi: held prompts fail, and
|
||||
+ // the exit fails the rest and reaches onExit.
|
||||
+ log(`engine: no agent_start ${abortGraceMs} ms after a failed turn; stopping pi`);
|
||||
+ wedge();
|
||||
+ }, abortGraceMs);
|
||||
+ }
|
||||
turn.reject(new DiscordError(message, 1, { code }));
|
||||
}
|
||||
|
||||
+ function wedge() {
|
||||
+ if (state.wedged || state.exited !== null) return;
|
||||
+ state.wedged = true;
|
||||
+ for (const h of state.held.splice(0)) failTurn(h.turn, "engine-wedged", "engine stopped: pi did not start an aborted turn");
|
||||
+ stopChild();
|
||||
+ }
|
||||
+
|
||||
+ // Call when a turn leaves the pending queue.
|
||||
+ function release(turn) {
|
||||
+ if (turn.grace !== null) clearTimeoutImpl(turn.grace);
|
||||
+ turn.grace = null;
|
||||
+ }
|
||||
+
|
||||
function settleTurn(turn, value) {
|
||||
if (turn.done) return;
|
||||
turn.done = true;
|
||||
@@ -99,7 +140,10 @@ export function createEngine({
|
||||
|
||||
function failAll(code, message) {
|
||||
const pending = state.pending.splice(0);
|
||||
- for (const t of pending) failTurn(t, code, message);
|
||||
+ for (const t of pending) {
|
||||
+ release(t);
|
||||
+ failTurn(t, code, message);
|
||||
+ }
|
||||
for (const h of state.held.splice(0)) failTurn(h.turn, code, message);
|
||||
for (const [, r] of state.responses) r.reject(new DiscordError(message, 1, { code }));
|
||||
state.responses.clear();
|
||||
@@ -174,6 +218,7 @@ export function createEngine({
|
||||
// Attribute the run to the head even if it failed client-side, so the
|
||||
// next prompt's agent_end is not taken for this one.
|
||||
const run = state.pending.shift();
|
||||
+ if (run) release(run);
|
||||
if (!run || run.done) return;
|
||||
const messages = Array.isArray(event.messages) ? event.messages.filter((m) => m && m.role === "assistant") : [];
|
||||
const message = messages.length > 0 ? messages[messages.length - 1] : run.last;
|
||||
@@ -193,13 +238,14 @@ export function createEngine({
|
||||
// this settle and still has no agent_end will never get one: fail it now
|
||||
// instead of waiting for its timeout. Turns whose prompt response has
|
||||
// not arrived yet belong to a later run and stay.
|
||||
+ const dropped = [];
|
||||
const keep = [];
|
||||
- for (const t of state.pending) {
|
||||
- if (t.done) continue;
|
||||
- if (t.accepted) failTurn(t, "engine-settled-without-turn", "engine settled without answering this prompt");
|
||||
- else keep.push(t);
|
||||
- }
|
||||
+ for (const t of state.pending) (t.done || t.accepted ? dropped : keep).push(t);
|
||||
state.pending = keep;
|
||||
+ for (const t of dropped) {
|
||||
+ release(t);
|
||||
+ failTurn(t, "engine-settled-without-turn", "engine settled without answering this prompt");
|
||||
+ }
|
||||
sendHeld();
|
||||
}
|
||||
}
|
||||
@@ -214,21 +260,29 @@ export function createEngine({
|
||||
// Never accepted: pi will not emit a turn_end for it, so remove it.
|
||||
const i = state.pending.indexOf(turn);
|
||||
if (i !== -1) state.pending.splice(i, 1);
|
||||
+ release(turn);
|
||||
failTurn(turn, (err.details && err.details.code) || "engine-refused", err.message);
|
||||
sendHeld();
|
||||
});
|
||||
}
|
||||
|
||||
+ // Pi is busy from our side while any sent prompt is still queued, even one
|
||||
+ // that already failed here: a turn that timed out before its agent_start
|
||||
+ // was read leaves state.busy false while pi runs it, and sending then would
|
||||
+ // be refused as streaming. It leaves the queue on its agent_end, on a
|
||||
+ // settle, on a refused send, or at process exit.
|
||||
+ const engineBusy = () => state.busy || state.pending.length > 0;
|
||||
+
|
||||
// After a settle (or a refused send) the oldest held prompt goes out.
|
||||
function sendHeld() {
|
||||
- if (state.exited !== null) return;
|
||||
- if (state.busy || state.pending.some((t) => !t.done)) return;
|
||||
+ if (state.exited !== null || state.wedged) return;
|
||||
+ if (engineBusy()) return;
|
||||
const next = state.held.shift();
|
||||
if (next) send(next.turn, next.command);
|
||||
}
|
||||
|
||||
function write(command) {
|
||||
- if (!state.child || state.exited !== null) throw new DiscordError("engine is not running", 1, { code: "engine-down" });
|
||||
+ if (!state.child || state.exited !== null || state.wedged) throw new DiscordError("engine is not running", 1, { code: "engine-down" });
|
||||
state.child.stdin.write(JSON.stringify(command) + "\n");
|
||||
}
|
||||
|
||||
@@ -245,6 +299,30 @@ export function createEngine({
|
||||
});
|
||||
}
|
||||
|
||||
+ function stopChild({ graceMs = 5000 } = {}) {
|
||||
+ const child = state.child;
|
||||
+ if (!child || state.exited !== null) return Promise.resolve(state.exited);
|
||||
+ return new Promise((resolve) => {
|
||||
+ const timer = setTimeoutImpl(() => {
|
||||
+ try {
|
||||
+ child.kill("SIGKILL");
|
||||
+ } catch {
|
||||
+ // already gone
|
||||
+ }
|
||||
+ }, graceMs);
|
||||
+ child.once("exit", () => {
|
||||
+ clearTimeoutImpl(timer);
|
||||
+ resolve(state.exited);
|
||||
+ });
|
||||
+ try {
|
||||
+ child.stdin.end();
|
||||
+ child.kill("SIGTERM");
|
||||
+ } catch {
|
||||
+ // already gone
|
||||
+ }
|
||||
+ });
|
||||
+ }
|
||||
+
|
||||
return {
|
||||
start() {
|
||||
if (state.child) throw new DiscordError("engine already started", 1);
|
||||
@@ -281,7 +359,7 @@ export function createEngine({
|
||||
// with DiscordError carrying details.code for the turn record.
|
||||
prompt(text, { timeoutMs = 180000 } = {}) {
|
||||
if (typeof text !== "string" || text.length === 0) throw new DiscordError("prompt text required", 1);
|
||||
- const turn = { resolve: null, reject: null, timer: null, done: false, accepted: false, tools: new Map(), turns: 0, last: null };
|
||||
+ const turn = { resolve: null, reject: null, timer: null, grace: null, done: false, accepted: false, tools: new Map(), turns: 0, last: null };
|
||||
const done = new Promise((resolve, reject) => {
|
||||
turn.resolve = resolve;
|
||||
turn.reject = reject;
|
||||
@@ -306,44 +384,24 @@ export function createEngine({
|
||||
}
|
||||
failTurn(turn, "timeout", `turn timed out after ${timeoutMs} ms`);
|
||||
}, timeoutMs);
|
||||
- if (state.exited !== null) {
|
||||
+ if (state.exited !== null || state.wedged) {
|
||||
failTurn(turn, "engine-down", "engine is not running");
|
||||
return done;
|
||||
}
|
||||
- if (state.busy || state.pending.some((t) => !t.done) || state.held.length > 0) state.held.push({ turn, command });
|
||||
+ if (engineBusy() || state.held.length > 0) state.held.push({ turn, command });
|
||||
else send(turn, command);
|
||||
return done;
|
||||
},
|
||||
|
||||
get busy() {
|
||||
- return state.busy || state.pending.some((t) => !t.done) || state.held.length > 0;
|
||||
+ return engineBusy() || state.held.length > 0;
|
||||
},
|
||||
get pendingCount() {
|
||||
return state.pending.filter((t) => !t.done).length + state.held.length;
|
||||
},
|
||||
|
||||
- stop({ graceMs = 5000 } = {}) {
|
||||
- const child = state.child;
|
||||
- if (!child || state.exited !== null) return Promise.resolve(state.exited);
|
||||
- return new Promise((resolve) => {
|
||||
- const timer = setTimeoutImpl(() => {
|
||||
- try {
|
||||
- child.kill("SIGKILL");
|
||||
- } catch {
|
||||
- // already gone
|
||||
- }
|
||||
- }, graceMs);
|
||||
- child.once("exit", () => {
|
||||
- clearTimeoutImpl(timer);
|
||||
- resolve(state.exited);
|
||||
- });
|
||||
- try {
|
||||
- child.stdin.end();
|
||||
- child.kill("SIGTERM");
|
||||
- } catch {
|
||||
- // already gone
|
||||
- }
|
||||
- });
|
||||
+ stop(options) {
|
||||
+ return stopChild(options);
|
||||
},
|
||||
};
|
||||
}
|
||||
diff --git a/packages/discord/tests/engine.test.mjs b/packages/discord/tests/engine.test.mjs
|
||||
index 59674d1e..f3bd7525 100644
|
||||
--- a/packages/discord/tests/engine.test.mjs
|
||||
+++ b/packages/discord/tests/engine.test.mjs
|
||||
@@ -4,6 +4,8 @@ import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { createEngine, buildPiArgs, PI_FIXED_ARGS, TOOLS_EXTENSION, READONLY_TOOLS_EXTENSION, assistantText } from "../src/engine-pi.mjs";
|
||||
import { existsSync } from "node:fs";
|
||||
+import { EventEmitter } from "node:events";
|
||||
+import { PassThrough } from "node:stream";
|
||||
import { makeRoot } from "./helpers.mjs";
|
||||
|
||||
const fakePi = join(import.meta.dirname, "fake-pi.mjs");
|
||||
@@ -28,7 +30,20 @@ function start(root, extra = {}) {
|
||||
log: (m) => logs.push(m), ...extra,
|
||||
});
|
||||
engine.start();
|
||||
- return { engine, logs, commands: () => readFileSync(logPath, "utf8").trim().split("\n").filter(Boolean).map((l) => JSON.parse(l)) };
|
||||
+ // The fake creates its log on the first command; until then there are none.
|
||||
+ const commands = () => (existsSync(logPath) ? readFileSync(logPath, "utf8").trim().split("\n").filter(Boolean).map((l) => JSON.parse(l)) : []);
|
||||
+ return { engine, logs, commands };
|
||||
+}
|
||||
+
|
||||
+// Every test stops its engine in finally: a fake pi left running after a
|
||||
+// failed assertion keeps the test file from exiting.
|
||||
+async function withEngine(extra, body) {
|
||||
+ const started = start(makeRoot(), extra);
|
||||
+ try {
|
||||
+ await body(started);
|
||||
+ } finally {
|
||||
+ await started.engine.stop();
|
||||
+ }
|
||||
}
|
||||
|
||||
test("engine: buildPiArgs carries the fixed flags, engine settings, session dir and prompt file", () => {
|
||||
@@ -56,8 +71,7 @@ test("engine: with tools, buildPiArgs turns pi's own tools off, loads the extens
|
||||
assert.equal(rw[rw.indexOf("--tools") + 1], "list_dir,read_file,search,write_file,edit_file", "a writable root adds exactly the two write tools");
|
||||
});
|
||||
|
||||
-test("engine: a run with tool turns settles once, on the answer, with every tool call in the result", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
+test("engine: a run with tool turns settles once, on the answer, with every tool call in the result", () => withEngine({}, async ({ engine }) => {
|
||||
const r = await engine.prompt("tools 3");
|
||||
assert.equal(r.text, "read 3 file(s)");
|
||||
assert.equal(r.turns, 2);
|
||||
@@ -71,45 +85,38 @@ test("engine: a run with tool turns settles once, on the answer, with every tool
|
||||
assert.equal(plain.turns, 1);
|
||||
await idle(engine);
|
||||
assert.equal(engine.busy, false);
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: a run that ends on a tool-only turn fails the prompt as empty; a retried run settles on the real end", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
+test("engine: a run that ends on a tool-only turn fails the prompt as empty; a retried run settles on the real end", () => withEngine({}, async ({ engine }) => {
|
||||
const r = await engine.prompt("toolonly");
|
||||
assert.equal(r.text, "", "no text: the connector turns this into engine-empty");
|
||||
assert.equal(r.tools.length, 1);
|
||||
const again = await engine.prompt("retry");
|
||||
assert.equal(again.text, "after retry");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: one prompt, one turn, text and usage come back", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
- try {
|
||||
- const r = await engine.prompt("hello");
|
||||
- assert.equal(r.text, "echo: hello");
|
||||
- assert.deepEqual(r.usage, { input: 3, output: 2 });
|
||||
- await idle(engine);
|
||||
- assert.equal(engine.busy, false);
|
||||
- } finally {
|
||||
- await engine.stop();
|
||||
- }
|
||||
-});
|
||||
+test("engine: one prompt, one turn, text and usage come back", () => withEngine({}, async ({ engine }) => {
|
||||
+ const r = await engine.prompt("hello");
|
||||
+ assert.equal(r.text, "echo: hello");
|
||||
+ assert.deepEqual(r.usage, { input: 3, output: 2 });
|
||||
+ await idle(engine);
|
||||
+ assert.equal(engine.busy, false);
|
||||
+}));
|
||||
|
||||
-test("engine: a prompt while streaming is held until pi settles, then sent as its own run, and answered in order", async () => {
|
||||
- const { engine, commands } = start(makeRoot());
|
||||
- const first = engine.prompt("slow 150");
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
+test("engine: a prompt while streaming is held until pi settles, then sent as its own run, and answered in order", () => withEngine({}, async ({ engine, commands }) => {
|
||||
+ const first = engine.prompt("slow 300");
|
||||
assert.equal(engine.busy, true);
|
||||
const second = engine.prompt("second");
|
||||
assert.equal(engine.pendingCount, 2);
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
- assert.equal(commands().filter((c) => c.type === "prompt").length, 1, "the second prompt is not sent while pi is busy");
|
||||
+ const prompted = () => commands().filter((c) => c.type === "prompt");
|
||||
+ assert.ok(await until(() => prompted().length > 0), "the first prompt reached pi");
|
||||
+ assert.equal(prompted().length, 1, "the second prompt is not sent while pi is busy");
|
||||
+ // The fake refuses a prompt without streamingBehavior while it runs one, so
|
||||
+ // an answered second prompt also proves it was not sent early.
|
||||
const [r1, r2] = await Promise.all([first, second]);
|
||||
assert.equal(r1.text, "slow reply");
|
||||
assert.equal(r2.text, "echo: second");
|
||||
- const prompts = commands().filter((c) => c.type === "prompt");
|
||||
+ const prompts = prompted();
|
||||
assert.equal(prompts.length, 2);
|
||||
// Never a pi follow-up: pi would fold it into the first run and close both
|
||||
// answers with one agent_end (the live loss of 2026-09-17).
|
||||
@@ -117,11 +124,9 @@ test("engine: a prompt while streaming is held until pi settles, then sent as it
|
||||
assert.equal(prompts[1].streamingBehavior, undefined);
|
||||
await idle(engine);
|
||||
assert.equal(engine.busy, false);
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: a held prompt that times out before pi settles fails on its own and is never sent", async () => {
|
||||
- const { engine, commands } = start(makeRoot());
|
||||
+test("engine: a held prompt that times out before pi settles fails on its own and is never sent", () => withEngine({}, async ({ engine, commands }) => {
|
||||
const first = engine.prompt("slow 200");
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
await assert.rejects(engine.prompt("late one", { timeoutMs: 50 }), (e) => e.details.code === "timeout" && /waiting for the engine/.test(e.message));
|
||||
@@ -130,50 +135,177 @@ test("engine: a held prompt that times out before pi settles fails on its own an
|
||||
await idle(engine);
|
||||
assert.deepEqual(commands().filter((c) => c.type === "prompt").map((c) => c.message), ["slow 200"]);
|
||||
assert.deepEqual(commands().filter((c) => c.type === "abort"), [], "a held turn is not aborted; pi never had it");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
-test("engine: timeout sends abort and fails only that turn; the process stays", async () => {
|
||||
- const { engine, commands, logs } = start(makeRoot());
|
||||
+test("engine: timeout sends abort and fails only that turn; the process stays", () => withEngine({}, async ({ engine, commands, logs }) => {
|
||||
await assert.rejects(engine.prompt("slow 5000", { timeoutMs: 100 }), (err) => err.details.code === "timeout");
|
||||
assert.ok(await until(() => commands().some((c) => c.type === "abort")), "abort reached pi");
|
||||
assert.ok(logs.some((l) => /timed out/.test(l)));
|
||||
const r = await engine.prompt("again");
|
||||
assert.equal(r.text, "echo: again");
|
||||
- await engine.stop();
|
||||
+}));
|
||||
+
|
||||
+test("engine: tool events from a run that outlived its timeout never land in the next prompt's record", () => withEngine({}, async ({ engine }) => {
|
||||
+ await assert.rejects(engine.prompt("late 200", { timeoutMs: 40 }), (err) => err.details.code === "timeout");
|
||||
+ const r = await engine.prompt("after late");
|
||||
+ assert.equal(r.text, "echo: after late");
|
||||
+ assert.deepEqual(r.tools, [], "the dead run's read is not this prompt's evidence");
|
||||
+ assert.equal(r.turns, 1, "the dead run's turns are not counted here");
|
||||
+}));
|
||||
+
|
||||
+// The turn timer is fired by hand, before the engine has read any event from
|
||||
+// pi, so the timed-out run is still pi's and state.busy is still false when
|
||||
+// the next prompt arrives. Under load a real timer does the same.
|
||||
+const TURN_MS = 60000;
|
||||
+const manualTurnTimer = (fire) => ({
|
||||
+ setTimeoutImpl: (fn, ms) => (ms === TURN_MS ? fire.push(fn) : setTimeout(fn, ms)),
|
||||
+ clearTimeoutImpl: (id) => { if (typeof id !== "number") clearTimeout(id); },
|
||||
});
|
||||
|
||||
-test("engine: tool events from a run that outlived its timeout never land in the next prompt's record", async () => {
|
||||
- const { engine } = start(makeRoot());
|
||||
- try {
|
||||
- await assert.rejects(engine.prompt("late 200", { timeoutMs: 40 }), (err) => err.details.code === "timeout");
|
||||
- const r = await engine.prompt("after late");
|
||||
+test("engine: a prompt after a turn that timed out before its agent_start waits for pi to settle instead of being refused", () => {
|
||||
+ const fire = [];
|
||||
+ return withEngine(manualTurnTimer(fire), async ({ engine, commands }) => {
|
||||
+ const late = engine.prompt("late 100", { timeoutMs: TURN_MS });
|
||||
+ fire.shift()();
|
||||
+ assert.equal(engine.busy, true, "pi is still running the prompt that timed out");
|
||||
+ const next = engine.prompt("after late", { timeoutMs: 5000 });
|
||||
+ assert.equal(engine.pendingCount, 1, "only the new prompt is live");
|
||||
+ await assert.rejects(late, (err) => err.details.code === "timeout");
|
||||
+ const r = await next;
|
||||
assert.equal(r.text, "echo: after late");
|
||||
assert.deepEqual(r.tools, [], "the dead run's read is not this prompt's evidence");
|
||||
- assert.equal(r.turns, 1, "the dead run's turns are not counted here");
|
||||
- } finally {
|
||||
- await engine.stop();
|
||||
- }
|
||||
+ assert.equal(r.turns, 1);
|
||||
+ assert.deepEqual(commands().map((c) => (c.type === "prompt" ? c.message : c.type)), ["late 100", "abort", "after late"]);
|
||||
+ await idle(engine);
|
||||
+ assert.equal(engine.busy, false);
|
||||
+ });
|
||||
+});
|
||||
+
|
||||
+// "mute" is accepted and never run, so no agent_start, agent_end or settle
|
||||
+// ever comes for it. Unbounded, it would hold every later prompt.
|
||||
+test("engine: when pi has not started a timed-out turn by the end of the abort grace, the engine stops pi and fails held prompts", async () => {
|
||||
+ let exited = null;
|
||||
+ await withEngine({ abortGraceMs: 150, onExit: (e) => (exited = e) }, async ({ engine, commands, logs }) => {
|
||||
+ await assert.rejects(engine.prompt("mute", { timeoutMs: 50 }), (err) => err.details.code === "timeout");
|
||||
+ assert.equal(engine.busy, true, "pi might still be running it");
|
||||
+ const started = Date.now();
|
||||
+ await assert.rejects(engine.prompt("after mute", { timeoutMs: 5000 }), (err) => err.details.code === "engine-wedged");
|
||||
+ assert.ok(Date.now() - started >= 100, "held for the grace, not failed at once");
|
||||
+ await assert.rejects(engine.prompt("later"), (err) => err.details.code === "engine-down");
|
||||
+ assert.ok(await until(() => exited !== null), "pi exits and onExit hears of it");
|
||||
+ assert.ok(logs.some((l) => /stopping pi/.test(l)));
|
||||
+ assert.deepEqual(commands().map((c) => (c.type === "prompt" ? c.message : c.type)), ["mute", "abort"]);
|
||||
+ });
|
||||
+});
|
||||
+
|
||||
+// Rocko's 6b R1 case: pi is stuck before agent_start, then runs the old
|
||||
+// prompt and only afterwards reads the next one. The events carry no prompt
|
||||
+// id, so a prompt sent after the grace would get the old run's answer.
|
||||
+test("engine: a timed-out turn pi starts only after the grace never answers a later prompt", async () => {
|
||||
+ let exited = null;
|
||||
+ await withEngine({ abortGraceMs: 150, onExit: (e) => (exited = e) }, async ({ engine, commands }) => {
|
||||
+ await assert.rejects(engine.prompt("stall 400", { timeoutMs: 50 }), (err) => err.details.code === "timeout");
|
||||
+ await assert.rejects(engine.prompt("after stall", { timeoutMs: 5000 }), (err) => err.details.code === "engine-wedged");
|
||||
+ assert.ok(await until(() => exited !== null), "pi exits and onExit hears of it");
|
||||
+ await new Promise((res) => setTimeout(res, 400));
|
||||
+ assert.ok(!commands().some((c) => c.message === "after stall"), "nothing was sent after the grace");
|
||||
+ });
|
||||
});
|
||||
|
||||
-test("engine: a malformed JSONL line fails the turn, not the process", async () => {
|
||||
- const { engine, logs } = start(makeRoot());
|
||||
+// The same case in memory, after Rocko's reproducer: the old run's events
|
||||
+// arrive after the grace while pi is still exiting. They land on the failed
|
||||
+// turn, nothing more is written to pi, and only the exit ends the engine.
|
||||
+// Pi's response to the old prompt comes either before its timeout or only
|
||||
+// with the late events.
|
||||
+for (const lateResponse of [false, true]) test(`engine: late events of a run past its grace, before pi exits, answer nothing and nothing more is sent (${lateResponse ? "late" : "early"} prompt response)`, async () => {
|
||||
+ const timers = [];
|
||||
+ const written = [];
|
||||
+ const kills = [];
|
||||
+ const child = new EventEmitter();
|
||||
+ child.stdout = new PassThrough();
|
||||
+ child.stderr = new PassThrough();
|
||||
+ child.stdin = { write: (s) => { written.push(JSON.parse(s)); return true; }, end: () => {} };
|
||||
+ child.kill = (signal) => { kills.push(signal); return true; };
|
||||
+ let exited = null;
|
||||
+ const engine = createEngine({
|
||||
+ command: "memory-only", args: [], spawn: () => child, abortGraceMs: 150, onExit: (e) => (exited = e),
|
||||
+ setTimeoutImpl: (fn, ms) => { const t = { fn, ms, active: true }; timers.push(t); return t; },
|
||||
+ clearTimeoutImpl: (t) => { t.active = false; },
|
||||
+ });
|
||||
+ const emit = (x) => child.stdout.write(JSON.stringify(x) + "\n");
|
||||
+ const fire = (ms) => { const t = timers.find((x) => x.ms === ms && x.active); assert.ok(t, `timer ${ms}`); t.active = false; t.fn(); };
|
||||
+ const message = (text) => ({ role: "assistant", content: [{ type: "text", text }], stopReason: "stop" });
|
||||
+ const tick = () => new Promise((res) => setImmediate(res));
|
||||
+ // Checked after a tick instead of awaited, so a regression fails here
|
||||
+ // rather than hanging on a promise nothing will settle.
|
||||
+ const outcome = (p) => {
|
||||
+ const o = { state: "pending", code: null, text: null };
|
||||
+ p.then((v) => Object.assign(o, { state: "resolved", text: v.text }), (e) => Object.assign(o, { state: "rejected", code: e.details && e.details.code }));
|
||||
+ return o;
|
||||
+ };
|
||||
+ engine.start();
|
||||
+ const first = engine.prompt("old", { timeoutMs: 50 });
|
||||
+ const accept = () => emit({ type: "response", id: written[0].id, command: "prompt", success: true });
|
||||
+ if (!lateResponse) accept();
|
||||
+ await tick();
|
||||
+ fire(50);
|
||||
+ await assert.rejects(first, (err) => err.details.code === "timeout");
|
||||
+ const next = outcome(engine.prompt("new", { timeoutMs: 2000 }));
|
||||
+ fire(150);
|
||||
+ await tick();
|
||||
+ assert.deepEqual(next, { state: "rejected", code: "engine-wedged", text: null });
|
||||
+ assert.deepEqual(kills, ["SIGTERM"]);
|
||||
+ if (lateResponse) accept();
|
||||
+ emit({ type: "agent_start" });
|
||||
+ emit({ type: "tool_execution_start", toolCallId: "old-call", toolName: "read_file", args: { root: "docs", path: "old.md" } });
|
||||
+ emit({ type: "tool_execution_end", toolCallId: "old-call", toolName: "read_file", result: { details: { root: "docs", path: "old.md", ok: true } } });
|
||||
+ emit({ type: "turn_end", message: message("OLD RUN ANSWER") });
|
||||
+ emit({ type: "agent_end", messages: [message("OLD RUN ANSWER")] });
|
||||
+ emit({ type: "agent_settled" });
|
||||
+ await tick();
|
||||
+ const after = outcome(engine.prompt("after settle", { timeoutMs: 2000 }));
|
||||
+ await tick();
|
||||
+ assert.deepEqual(after, { state: "rejected", code: "engine-down", text: null });
|
||||
+ assert.deepEqual(next, { state: "rejected", code: "engine-wedged", text: null }, "the old answer did not reach the new prompt");
|
||||
+ assert.deepEqual(written.map((c) => (c.type === "prompt" ? c.message : c.type)), ["old", "abort"], "no prompt reached pi after the grace");
|
||||
+ assert.equal(exited, null);
|
||||
+ fire(5000);
|
||||
+ assert.deepEqual(kills, ["SIGTERM", "SIGKILL"]);
|
||||
+ child.emit("exit", null, "SIGKILL");
|
||||
+ assert.deepEqual(exited, { code: null, signal: "SIGKILL" });
|
||||
+});
|
||||
+
|
||||
+test("engine: a timed-out run pi did start outlives the grace; the next prompt goes out when it ends", async () => {
|
||||
+ let exited = null;
|
||||
+ await withEngine({ abortGraceMs: 150, onExit: (e) => (exited = e) }, async ({ engine, commands }) => {
|
||||
+ await assert.rejects(engine.prompt("late 400", { timeoutMs: 50 }), (err) => err.details.code === "timeout");
|
||||
+ const r = await engine.prompt("after late", { timeoutMs: 5000 });
|
||||
+ assert.equal(r.text, "echo: after late");
|
||||
+ assert.deepEqual(r.tools, []);
|
||||
+ assert.equal(exited, null, "pi was not stopped");
|
||||
+ assert.deepEqual(commands().map((c) => (c.type === "prompt" ? c.message : c.type)), ["late 400", "abort", "after late"]);
|
||||
+ });
|
||||
+});
|
||||
+
|
||||
+test("engine: a malformed JSONL line fails the turn, not the process", () => withEngine({}, async ({ engine, logs }) => {
|
||||
await assert.rejects(engine.prompt("garbage"), (err) => err.details.code === "engine-protocol");
|
||||
assert.ok(logs.some((l) => /malformed/.test(l)));
|
||||
const r = await engine.prompt("still here");
|
||||
assert.equal(r.text, "echo: still here");
|
||||
- await engine.stop();
|
||||
-});
|
||||
+}));
|
||||
|
||||
test("engine: a turn that ends in error rejects with the error code; process exit fails pending turns", async () => {
|
||||
- const root = makeRoot();
|
||||
let exited = null;
|
||||
- const { engine } = start(root, { onExit: (e) => (exited = e) });
|
||||
- await assert.rejects(engine.prompt("error"), (err) => err.details.code === "engine-error" && /fake provider error/.test(err.message));
|
||||
- const pending = engine.prompt("slow 5000");
|
||||
- await new Promise((r) => setTimeout(r, 20));
|
||||
- await engine.stop();
|
||||
- await assert.rejects(pending, (err) => err.details.code === "engine-down");
|
||||
- assert.ok(exited);
|
||||
- await assert.rejects(engine.prompt("x"), /not running/);
|
||||
+ const { engine } = start(makeRoot(), { onExit: (e) => (exited = e) });
|
||||
+ try {
|
||||
+ await assert.rejects(engine.prompt("error"), (err) => err.details.code === "engine-error" && /fake provider error/.test(err.message));
|
||||
+ const pending = engine.prompt("slow 5000");
|
||||
+ await new Promise((r) => setTimeout(r, 20));
|
||||
+ await engine.stop();
|
||||
+ await assert.rejects(pending, (err) => err.details.code === "engine-down");
|
||||
+ assert.ok(exited);
|
||||
+ await assert.rejects(engine.prompt("x"), /not running/);
|
||||
+ } finally {
|
||||
+ await engine.stop();
|
||||
+ }
|
||||
});
|
||||
diff --git a/packages/discord/tests/fake-pi.mjs b/packages/discord/tests/fake-pi.mjs
|
||||
index 94919306..bf94c013 100644
|
||||
--- a/packages/discord/tests/fake-pi.mjs
|
||||
+++ b/packages/discord/tests/fake-pi.mjs
|
||||
@@ -8,9 +8,13 @@
|
||||
// then a second turn that answers "read <n> file(s)"
|
||||
// "toolonly" a run whose only turn calls a tool and never answers
|
||||
// "retry" an agent_end with willRetry, then the real answer
|
||||
+// "mute" accept the prompt and emit nothing, staying idle
|
||||
// "late <ms>" ignore abort; after <ms> emit a tool pair and a tool turn,
|
||||
// then answer "late reply", like a run that outlives its
|
||||
// client-side timeout
|
||||
+// "stall <ms>" accept the prompt, then read nothing for <ms> (pi stuck
|
||||
+// before agent_start); then run it, answering "echo:
|
||||
+// stalled", and only then read what came in meanwhile
|
||||
// anything else answer "echo: <text>" immediately
|
||||
// A prompt received while busy without streamingBehavior is refused, as pi
|
||||
// does. A prompt with streamingBehavior followUp is folded into the running
|
||||
@@ -30,6 +34,7 @@ function assistant(text, stopReason = "stop") {
|
||||
}
|
||||
|
||||
function run(text) {
|
||||
+ if (text === "mute") return;
|
||||
busy = true;
|
||||
out({ type: "agent_start" });
|
||||
out({ type: "turn_start" });
|
||||
@@ -109,11 +114,16 @@ function run(text) {
|
||||
let current = null;
|
||||
|
||||
let buffer = "";
|
||||
+let stalled = false;
|
||||
process.stdin.setEncoding("utf8");
|
||||
process.stdin.on("data", (chunk) => {
|
||||
buffer += chunk;
|
||||
+ drain();
|
||||
+});
|
||||
+
|
||||
+function drain() {
|
||||
let idx;
|
||||
- while ((idx = buffer.indexOf("\n")) !== -1) {
|
||||
+ while (!stalled && (idx = buffer.indexOf("\n")) !== -1) {
|
||||
const line = buffer.slice(0, idx);
|
||||
buffer = buffer.slice(idx + 1);
|
||||
if (!line) continue;
|
||||
@@ -125,7 +135,15 @@ process.stdin.on("data", (chunk) => {
|
||||
continue;
|
||||
}
|
||||
out({ id: cmd.id, type: "response", command: "prompt", success: true });
|
||||
- if (busy) queue.push(cmd.message);
|
||||
+ const sm = /^stall (\d+)$/.exec(cmd.message);
|
||||
+ if (sm) {
|
||||
+ stalled = true;
|
||||
+ setTimeout(() => {
|
||||
+ run("stalled");
|
||||
+ stalled = false;
|
||||
+ drain();
|
||||
+ }, Number(sm[1]));
|
||||
+ } else if (busy) queue.push(cmd.message);
|
||||
else run(cmd.message);
|
||||
} else if (cmd.type === "abort") {
|
||||
out({ id: cmd.id, type: "response", command: "abort", success: true });
|
||||
@@ -139,5 +157,5 @@ process.stdin.on("data", (chunk) => {
|
||||
out({ id: cmd.id, type: "response", command: cmd.type, success: true, data: {} });
|
||||
}
|
||||
}
|
||||
-});
|
||||
+}
|
||||
process.stdin.on("end", () => process.exit(0));
|
||||
@@ -1,25 +0,0 @@
|
||||
{
|
||||
"observedAt": "2026-09-13T19:56:18.961314+00:00",
|
||||
"issue": 1510,
|
||||
"ownerAuthorizedLiveSmoke": true,
|
||||
"researcher": [
|
||||
{
|
||||
"session": ".pi/state/researcher/sessions/2026-09-13T19-52-28-745Z_01a09c54-0b48-7154-addd-8fdce875aa4a.jsonl",
|
||||
"entryId": "64b15fd5",
|
||||
"timestamp": "2026-09-13T19:52:43.079Z",
|
||||
"response": "RESEARCHER_NATIVE_SMOKE_OK",
|
||||
"entrySha256": "d9cbaa5ea640d2b858a86b2fa24d3240b351d9383ffaee9721dd86fcd080c329"
|
||||
}
|
||||
],
|
||||
"rocko": {
|
||||
"newLaunch": "refused by existing native launch lock",
|
||||
"existingPid": 3707667,
|
||||
"cwd": "/mnt/storage/src/mosaic-stack",
|
||||
"nativeContextVerified": true,
|
||||
"sonnetFlagVerified": true,
|
||||
"socket": "mosaic-fleet",
|
||||
"newModelResponseTested": false
|
||||
},
|
||||
"existingProcessesRestarted": false,
|
||||
"homeLaunchersModified": false
|
||||
}
|
||||
@@ -1,19 +0,0 @@
|
||||
{
|
||||
"issue": 1510,
|
||||
"candidate": "/tmp/internal-team-r1-i9t21yzr",
|
||||
"manifestSha256": "23a27014ce6f04ce8187d8495b2efe62b814c3c7d041b027f99b1ec4d490709d",
|
||||
"files": [
|
||||
"AGENTS.md",
|
||||
"agents/README.md",
|
||||
"agents/researcher/SOUL.md",
|
||||
"agents/researcher/CONTEXT.md",
|
||||
"agents/researcher/README.md",
|
||||
"agents/researcher/launch.sh",
|
||||
"agents/researcher/validate-sessions.mjs",
|
||||
"scripts/test-darkwing-launch.mjs",
|
||||
"docs/plans/2026-09-13_internal-development-bootstrap.md"
|
||||
],
|
||||
"tests": "node --test scripts/test-darkwing-launch.mjs scripts/test-rocko-launch.mjs",
|
||||
"passed": 6,
|
||||
"state": "ready for independent review"
|
||||
}
|
||||
@@ -1,77 +0,0 @@
|
||||
# Ledger: T3 header counts as agent (#1506), R1 candidate
|
||||
|
||||
Sage assigned this on 2026-09-26 after commit A (af4203ca). Filbert reviews.
|
||||
Not committed.
|
||||
|
||||
## Defect
|
||||
|
||||
`messageKind` in `packages/ledger/src/ledger.mjs` knew only the tmux preamble
|
||||
`[host:session -> host:session]`. A prompt that opens with the T3 header
|
||||
`[from: sage (1ef1e4f8-…) -> to: filbert (9cb9731e-…) class=actionable]`
|
||||
counted as human, so Table 2's Human column and the human-per-closed ratio
|
||||
rise once seats talk over T3. DEFERRED Open entry "Ledger counts T3 agent
|
||||
messages as human".
|
||||
|
||||
## Change
|
||||
|
||||
- `messageKind` also matches the T3 header on the first line. The sender is the
|
||||
`from:` role. `control-board` is board, any other role is agent. Anything short
|
||||
of the full header stays human. That includes the header on a later line, a
|
||||
leading space, a missing thread id, `class=` with capitals, `]` followed by a
|
||||
non-space, and `From:` capitalized. The tmux branch is unchanged.
|
||||
- Two tests: a Table 2 fixture with two headered prompts and one plain prompt
|
||||
for seat `bob`, expecting agent 2 and human 1. Also a direct classification table.
|
||||
- README counting rule names both forms.
|
||||
|
||||
## Evidence
|
||||
|
||||
- `node --test --test-reporter=tap packages/ledger/tests/`: 22/22 on the
|
||||
working tree.
|
||||
- The same test file against HEAD's `ledger.mjs` (full `git archive HEAD` tree):
|
||||
20/22. The two failures are the two new tests, so they catch the defect.
|
||||
An earlier archive of `packages/ledger` alone also failed three gitea-helper
|
||||
tests. Those tests need `scripts/gitea-api.sh`, which the partial archive left out.
|
||||
- Suites on the working tree: config 24, task 90, foundation 43, conductor 17,
|
||||
release 14, auth 15, discord 63. None of them runs the ledger tests.
|
||||
|
||||
## Frozen files
|
||||
|
||||
`r1-manifest.sha256` holds the three file hashes; `r1.patch` is `git diff
|
||||
packages/ledger` at freeze time.
|
||||
|
||||
## Known limit, not fixed here
|
||||
|
||||
The fix changes zero current counts. Table 2 reads only
|
||||
`.pi/state/<seat>/sessions/*.jsonl`, and no file there contains a T3 header
|
||||
for any seat. Filbert checked this in R1:
|
||||
- `grep -rF '[from: ' .pi/state/*/sessions/` finds nothing.
|
||||
- The candidate `messageKind` gives Dewey 52 agent, 7 board and 9 human, and
|
||||
Sage 193 agent and 72 human. That covers every user message in Pi logs
|
||||
modified since 2026-09-20. Every non-human first line is the tmux form.
|
||||
- Filbert's three T3 messages to Dewey on 2026-09-26 are not in
|
||||
`.pi/state/dewey`.
|
||||
|
||||
Dewey's and Sage's Pi logs are current, but they only carry tmux traffic. T3
|
||||
traffic goes to the harness transcripts: Claude under `~/.claude/projects`,
|
||||
Codex under `~/.codex/sessions`. The ledger reads neither, so a T3-routed
|
||||
prompt to any seat counts nowhere, as agent or as human. The Human column
|
||||
can't see T3 traffic at all. For Darkwing and Filbert, whose newest Pi logs
|
||||
end 2026-09-14, and for Rocko, who has no Pi sessions directory, zero means an
|
||||
empty source, not zero human prompts. The fix is correct for a source that
|
||||
carries T3 headers. Sage asked for a brief on a read-only T3 thread source;
|
||||
Gate F waits on it.
|
||||
|
||||
Filbert also found two misclassifications in older logs. Neither is touched
|
||||
here:
|
||||
- `[rev-code-02 -> dragon-lin:sage class=actionable]` has no host on the
|
||||
sender, so it counts as human.
|
||||
- One Dewey prompt opens with a quote character before the tmux preamble, so
|
||||
it counts as human.
|
||||
|
||||
## Review
|
||||
|
||||
Filbert, R1, 2026-09-26: approved the three frozen files. He verified the
|
||||
manifest and patch, got 22/22 on the tree and 20/22 against HEAD's source, and
|
||||
matched the regex to `docs/guides/T3-AGENT-COMMS.md`. He accepts the body on
|
||||
the header's line, which the tmux branch also allows. He corrected the
|
||||
known-limit text above.
|
||||
@@ -1,3 +0,0 @@
|
||||
e0d411ca2f45d85734eef130dba645646df6e9128ea7dc2eaba2205df7891bb8 packages/ledger/README.md
|
||||
e24b065c4284370960ac6ff1ed66810fe601da64ae9b9362584fe9fbee334017 packages/ledger/src/ledger.mjs
|
||||
a9da013e81aff360cb013a8e103fdd111aee7e42553b96cb0811560b3da39250 packages/ledger/tests/ledger.test.mjs
|
||||
@@ -1,89 +0,0 @@
|
||||
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
|
||||
index a998a76c..da0ab1c5 100644
|
||||
--- a/packages/ledger/README.md
|
||||
+++ b/packages/ledger/README.md
|
||||
@@ -36,11 +36,15 @@ No install, build, service restart, or configuration change is needed.
|
||||
duplicated entries in copied logs are not deduplicated. No transcript content
|
||||
leaves the parser. Assistant messages and logs outside repo seats do not count.
|
||||
Symlink source directories are refused and symlink files are not followed.
|
||||
-- The first text line alone classifies a message. A bracketed addressing
|
||||
- preamble whose source session is `control-board` is board; any other valid
|
||||
- addressing preamble is agent; otherwise human. This is a format count, not
|
||||
- proof of who typed the message. Text blocks are joined with newlines.
|
||||
- The entry timestamp is used, falling back to the message timestamp.
|
||||
+- The first text line alone classifies a message. Two addressing forms count:
|
||||
+ the tmux preamble `[host:session -> host:session]` that `agent-send.sh`
|
||||
+ writes, and the T3 header `[from: role (thread-id) -> to: role (thread-id)]`
|
||||
+ from `docs/guides/T3-AGENT-COMMS.md`. Either may carry ` class=<class>` before
|
||||
+ the closing bracket. A preamble whose sender is `control-board` (tmux session
|
||||
+ or T3 role) is board; any other valid preamble is agent; otherwise human.
|
||||
+ This is a format count, not proof of who typed the message. Text blocks are
|
||||
+ joined with newlines. The entry timestamp is used, falling back to the
|
||||
+ message timestamp.
|
||||
- Seats with no in-range user messages are omitted. Issue seats come from `#N`
|
||||
mentions anywhere in in-range user text, including quoted text.
|
||||
- Human messages per closed issue divides Table 2's human sum by issues closed
|
||||
diff --git a/packages/ledger/src/ledger.mjs b/packages/ledger/src/ledger.mjs
|
||||
index dc8a3a69..b09e95f5 100644
|
||||
--- a/packages/ledger/src/ledger.mjs
|
||||
+++ b/packages/ledger/src/ledger.mjs
|
||||
@@ -77,8 +77,12 @@ export function messageText(content) {
|
||||
}
|
||||
export function messageKind(text) {
|
||||
const firstLine = text.split(/\r?\n/, 1)[0];
|
||||
- const match = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/);
|
||||
- return !match ? 'human' : match[2] === 'control-board' ? 'board' : 'agent';
|
||||
+ // tmux preamble from agent-send.sh: [host:session -> host:session class=x]
|
||||
+ const tmux = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/);
|
||||
+ // T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x]
|
||||
+ const t3 = firstLine.match(/^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[a-z-]+)?\](?:\s|$)/);
|
||||
+ const sender = tmux ? tmux[2] : t3 ? t3[1] : null;
|
||||
+ return sender === null ? 'human' : sender === 'control-board' ? 'board' : 'agent';
|
||||
}
|
||||
async function directories(dir, optional = false) {
|
||||
try {
|
||||
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
|
||||
index 7b4e4d33..175f5d5e 100644
|
||||
--- a/packages/ledger/tests/ledger.test.mjs
|
||||
+++ b/packages/ledger/tests/ledger.test.mjs
|
||||
@@ -110,6 +110,19 @@ test('invalid dates, reverse dates and duplicate options refuse', t => {
|
||||
assert.throws(() => dateRange('2026-02-30')); assert.throws(() => dateRange('2026-09-12', '2026-09-06'));
|
||||
const f = fixture(t); assert.equal(f.run(['--since', '2026-09-01']).status, 1);
|
||||
});
|
||||
+test('T3 agent assignments do not count as human in Table 2', t => {
|
||||
+ const f = fixture(t);
|
||||
+ f.put('.pi/state/bob/sessions/t3.jsonl', [
|
||||
+ f.entry('[from: sage (1ef1e4f8) -> to: bob (9cb9731e) class=actionable]\nassign #1'),
|
||||
+ f.entry('[from: sage (1ef1e4f8) -> to: bob (9cb9731e)]\nfollow-up #1'),
|
||||
+ f.entry('Jason: go ahead'),
|
||||
+ ].map(x => JSON.stringify(x)).join('\n') + '\n');
|
||||
+ const result = f.run(['--json']);
|
||||
+ assert.equal(result.status, 0, result.stderr);
|
||||
+ const r = JSON.parse(result.stdout);
|
||||
+ assert.deepEqual(r.seats, [{ seat: 'alice', board: 1, agent: 1, human: 1 }, { seat: 'bob', board: 0, agent: 2, human: 1 }]);
|
||||
+ assert.equal(r.totals.humanMessagesPerClosedIssue, 2);
|
||||
+});
|
||||
test('preamble parsing and issue number boundaries', () => {
|
||||
assert.equal(messageKind('[h:control-board -> h:seat] hi'), 'board');
|
||||
assert.equal(messageKind('[h:seat -> h:seat class=actionable] hi'), 'agent');
|
||||
@@ -117,6 +130,20 @@ test('preamble parsing and issue number boundaries', () => {
|
||||
assert.equal(messageKind(' [h:seat -> h:seat] quoted'), 'human');
|
||||
assert.deepEqual(issueNumbers('fix #1 #2 #2 abc#3 #0 #4x'), [1, 2]);
|
||||
});
|
||||
+test('T3 header: agent, or board from control-board; anything short of the full header is human', () => {
|
||||
+ const sage = 'sage (1ef1e4f8-3ead-4208-beca-38f9f1add079)', filbert = 'filbert (9cb9731e-a10f-4c8f-a212-c4fa1f5f4731)';
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]\nbuild #1506`), 'agent');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=actionable]\nbuild`), 'agent');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}] same line`), 'agent');
|
||||
+ assert.equal(messageKind(`[from: darkwing (thread-id: unknown) -> to: reviewer (new-thread)]\nreview`), 'agent');
|
||||
+ assert.equal(messageKind(`[from: control-board (b) -> to: ${filbert}]\nhi`), 'board');
|
||||
+ assert.equal(messageKind(`Jason here\n[from: ${sage} -> to: ${filbert}]\nquoted`), 'human');
|
||||
+ assert.equal(messageKind(` [from: ${sage} -> to: ${filbert}]`), 'human');
|
||||
+ assert.equal(messageKind(`[from: sage -> to: filbert]\nno thread ids`), 'human');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'human');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]trailing`), 'human');
|
||||
+ assert.equal(messageKind(`[From: ${sage} -> to: ${filbert}]`), 'human');
|
||||
+});
|
||||
test('no closed issues with human messages means undefined ratio, not invented zero', () => {
|
||||
const r = summarize(range, [], [], { rows: [{ seat: 'a', human: 1, board: 0, agent: 0 }], mentions: new Map() });
|
||||
assert.equal(r.totals.humanMessagesPerClosedIssue, 'unknown');
|
||||
@@ -1,5 +0,0 @@
|
||||
0afb0320e9a1833f133426169c389ffb71be3d490e0e402070162aa22e820296 packages/ledger/src/ledger.mjs
|
||||
d6092a538a059e8869544df903e89a8b9a4c3b6b492645db762b2f03d5630a44 packages/ledger/src/cli.mjs
|
||||
dfb092aabee5cf5197029c6e8978df570ee50f08d84babde931c6a763befafe9 packages/ledger/src/t3.mjs
|
||||
7444abd1dbd8e637705def0fd98105e6e69970f1bd397a11f8277996521579d6 packages/ledger/tests/ledger.test.mjs
|
||||
27f7366dd5edc30a93a8c54bfb46b3fed87e1a44f11e1b22bd159a0718625273 packages/ledger/README.md
|
||||
@@ -1,161 +0,0 @@
|
||||
# Gate F build: the ledger's T3 source (#1506), candidate for review
|
||||
|
||||
Darkwing built this on 2026-09-26 from the approved brief R3,
|
||||
`docs/plans/2026-09-26_ledger-t3-source.md` (sha256 f3c05c1b, committed in
|
||||
ffc22c04). Sage gave the go once Filbert confirmed R3. Filbert reviews the
|
||||
code; Sage commits after the suites. Base is HEAD 1c5f6bc3. Nothing is
|
||||
committed or pushed.
|
||||
|
||||
## Files
|
||||
|
||||
`build-manifest.sha256` pins the five files, and `build.patch` is the diff
|
||||
against 1c5f6bc3 with `t3.mjs` included as a new file.
|
||||
|
||||
- `packages/ledger/src/t3.mjs` (new). `readT3(root, range, {dbPath, isDefault})`:
|
||||
path checks, one read transaction, schema check, project, title mapping,
|
||||
header cross-check, counts, mentions, diagnostic.
|
||||
- `packages/ledger/src/ledger.mjs`. The class fix, `t3Header()`,
|
||||
`readSeats()`, `mergeSources()`, the `pi` and `t3` keys in the report, the
|
||||
text line for `--no-t3` or a non-default path, and the U+2028 fix below.
|
||||
- `packages/ledger/src/cli.mjs`. `--no-t3` and `--t3-db PATH`, which refuse
|
||||
each other; the usage line.
|
||||
- `packages/ledger/tests/ledger.test.mjs`. HOME at both spawn sites, the
|
||||
empty default database, 25 new tests.
|
||||
- `packages/ledger/README.md`. A new "T3 source" section.
|
||||
|
||||
The commit should also carry Filbert's updated review,
|
||||
`agents/filbert/work/ledger-t3-source-review-2026-09-26.md` (be1aa414), and
|
||||
this directory's new files.
|
||||
|
||||
## Beyond the brief: the Pi reader split valid lines
|
||||
|
||||
The brief's live read has to exit 0. It didn't, and T3 wasn't the cause. HEAD
|
||||
refuses the live checkout the same way:
|
||||
`Malformed session JSON: filbert/2026-09-12T16-38-58-597Z_01a0967c-….jsonl:611`.
|
||||
That line parses. It holds a raw U+2028 inside a JSON string, which JSON
|
||||
allows and `JSON.stringify` writes unescaped. Node 26.8.1's `readline` ends a
|
||||
line at U+2028 too, so it cut the record in two (733 lines by `readline`, 732
|
||||
by `\n`). The reader parses every line before it checks the range, so on
|
||||
Node 26.8.1 every live run refuses, whatever the dates. The file was last
|
||||
written 2026-09-14. I haven't checked which Node version first split there.
|
||||
|
||||
The fix replaces `readline` with a small splitter that ends lines at `\n`
|
||||
only. It sits in `ledger.mjs`, which this build already changes, and it
|
||||
blocked acceptance, so I made it here instead of filing it. A new test writes a
|
||||
Pi log with a raw U+2028 and CRLF endings; it fails with `readline` and passes
|
||||
with the splitter. Please review it as its own item.
|
||||
|
||||
## Choices the brief left open
|
||||
|
||||
- Imported threads are excluded by the `import:` prefix alone. Live, all
|
||||
1678 `historyImport` events sit in `import:` streams, so the two rules agree
|
||||
today. The events table stays optional, so the exclusion doesn't depend on it.
|
||||
- The header cross-check runs over every user message in a counted thread, in
|
||||
range or not. The title mapping is current state, so a conflict in old
|
||||
history still misassigns counts for any range that includes it.
|
||||
- Validation (role, text, `created_at`) also covers every message in a
|
||||
counted thread, assistant rows included, and not only rows in range.
|
||||
- The diagnostic is in range: `humanSentThroughApi` and `humanWithoutEvent`.
|
||||
One unparseable event, or an event with no string `messageId`, makes both
|
||||
`unknown`, the same as a missing table (F5).
|
||||
- Project and thread matching compare `workspace_root` in JavaScript, so a
|
||||
declared collation on the column can't loosen byte-for-byte equality.
|
||||
- JSON adds top-level `pi` (Pi rows) and `t3` (read flag, database, seats with
|
||||
threads, unmapped, excluded, diagnostic). `seats` and `totals` keep their
|
||||
shape, so existing consumers and tests are unchanged. `t3.seats` lists a
|
||||
seat whenever it has a mapped thread, even with zero counts in range.
|
||||
- The unmapped row comes last in `seats`, and only when it has counts.
|
||||
|
||||
## Evidence
|
||||
|
||||
- Ledger tests: `node --test packages/ledger/tests/`, 47/47 (ledger 44,
|
||||
of which 25 are new, and gitea helper 3). The busy-timeout test takes about 5.4 s.
|
||||
- Class fix against HEAD. HEAD's `messageKind` (from `git show
|
||||
1c5f6bc3:packages/ledger/src/ledger.mjs`) calls a T3 header with
|
||||
`class=REVIEW-REQUEST`, a tmux preamble with `class=DECISION` and a T3 header
|
||||
with `class=Actionable` all human. The build calls them agent. The existing
|
||||
test asserting `class=Actionable` is human now asserts agent.
|
||||
- Mutations, each on a scratch copy of the package. Three `gitea-helper`
|
||||
tests fail in every scratch copy because they need the repository's
|
||||
`scripts/`, so the counts below leave them out.
|
||||
- Classes back to `[a-z-]+`: 6 fail.
|
||||
- No header cross-check: 2 fail.
|
||||
- No symlink refusal: 4 fail.
|
||||
- Busy timeout 0: 1 fails.
|
||||
- Two projects allowed: 1 fails.
|
||||
- Deleted threads kept, imported threads kept, or range filter removed:
|
||||
4 fail each.
|
||||
- No role check: 1 fails.
|
||||
- No diagnostic table check: 1 fails.
|
||||
- `readline` restored: 1 fails.
|
||||
- Two mutations pass, and I'm naming them rather than hiding them:
|
||||
- Removing `mode=ro` changes nothing, because `readOnly: true` already
|
||||
opens read-only. Both stay, as the brief says.
|
||||
- Removing `BEGIN` fails 19 tests, but only because `COMMIT` then has no
|
||||
transaction. No test proves that the queries share one snapshot.
|
||||
- Eight suites on a local clone of 1c5f6bc3 with the five files: config 24,
|
||||
task 90, foundation 43, conductor 17, release 14, auth 15, discord 63,
|
||||
extension-package 18. The first task run showed 89/1, and I didn't capture
|
||||
the failing line. Three more task runs passed 90/90. I count it as a flake
|
||||
I can't name, not as green on the first try.
|
||||
- Union (control-board, webui, seat, mosaic, ledger, discord) on the same
|
||||
clone: 434/434 three times, 23 to 24 s each. No fake pi left running.
|
||||
- No test opens the real `~/.t3`. Every CLI spawn sets `HOME` to a temp
|
||||
directory, and no test calls `readT3` in process. After the runs, no
|
||||
`ledger-*` temp directories remained.
|
||||
|
||||
## Live read
|
||||
|
||||
`node packages/ledger/src/cli.mjs --since 2026-09-01 --until 2026-09-26
|
||||
--no-issues`, exit 0 three times, no header conflict. The table is the run at
|
||||
2026-09-26T21:31:02Z.
|
||||
|
||||
| Seat | T3 threads | T3 board / agent / human | Pi board / agent / human |
|
||||
|---|---|---|---|
|
||||
| darkwing | Darkwing; Darkwing in Claude (archived) | 0 / 19 / 28 | 14 / 109 / 141 |
|
||||
| dewey | Dewey; Dewey in Claude | 0 / 17 / 7 | 7 / 57 / 16 |
|
||||
| filbert | Filbert | 0 / 25 / 1 | 5 / 92 / 9 |
|
||||
| rocko | Rocko | 0 / 20 / 1 | none |
|
||||
| sage | Sage | 0 / 52 / 10 | 0 / 193 / 72 |
|
||||
| researcher | none | none | 3 / 1 / 1 |
|
||||
| t3:unmapped | Discord Bot | 0 / 0 / 68 | none |
|
||||
|
||||
This matches the brief, allowing for messages sent since 20:54Z. It maps the
|
||||
same seven threads. Discord Bot has 68 human: 54 without a header and the 14
|
||||
free-text headers. The diagnostic reads exactly those 14
|
||||
(`humanSentThroughApi: 14`, `humanWithoutEvent: 0`). T3 agent messages total
|
||||
133, against the brief's 96 API headers (80 plus the 16 uppercase ones) at
|
||||
20:54Z. Two imported threads are excluded, and this project has no deleted
|
||||
threads.
|
||||
|
||||
## Not covered
|
||||
|
||||
- Snapshot isolation across the queries (see the `BEGIN` mutation above).
|
||||
- A seat directory named `t3:unmapped` would share the unmapped row. Directory
|
||||
names that contain a colon aren't used in `agents/`.
|
||||
- The live read's effect on the main database file can't be checked while T3
|
||||
writes to it. The stopped and writer-attached WAL tests check it on
|
||||
fixtures.
|
||||
|
||||
## Review and correction
|
||||
|
||||
Filbert approved manifest ba73a163 and the U+2028 fix as its own item:
|
||||
`agents/filbert/work/ledger-t3-build-review-2026-09-26.md`, sha256 e47ec6da.
|
||||
|
||||
Correction to "Beyond the brief" above. Line 611 holds a raw U+2028 and a raw
|
||||
U+2029, and `readline` ends a line at each. The file has 731 lines by `\n`
|
||||
(`wc -l` agrees), and `readline` makes 733. I wrote 732 because I counted the
|
||||
empty string after the final newline. The splitter already ends lines at `\n`
|
||||
only, so the fix covers both characters. The test and the README name only
|
||||
U+2028.
|
||||
|
||||
Filbert's nonblocking notes, for a follow-up after the Gate F commit, since
|
||||
changing the pinned files now would void the approval:
|
||||
1. Add a U+2029 to the splitter test and the README line.
|
||||
2. Two diagnostic mutations survive: `humanWithoutEvent` hardcoded to 0, and
|
||||
an unparseable event skipped instead of making the diagnostic `unknown`.
|
||||
Each needs one fixture message.
|
||||
3. `readT3`'s catch reports any error that isn't a `SourceError` as a SQLite
|
||||
read failure. It still exits 1, but a bug would read as a database
|
||||
problem. Rethrow errors that carry no `errcode`.
|
||||
4. Snapshot isolation stays untested, as recorded above.
|
||||
@@ -1,828 +0,0 @@
|
||||
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
|
||||
index da0ab1c5..393e9c37 100644
|
||||
--- a/packages/ledger/README.md
|
||||
+++ b/packages/ledger/README.md
|
||||
@@ -1,13 +1,16 @@
|
||||
# Ledger
|
||||
|
||||
Read-only counts from local `refactor` commit subjects, one Gitea issue-list
|
||||
-request through `scripts/gitea-api.sh`, and repo seats' Pi session logs.
|
||||
+request through `scripts/gitea-api.sh`, repo seats' Pi session logs, and T3's
|
||||
+thread messages in `~/.t3/userdata/state.sqlite`.
|
||||
No board changes, data-root writes, fleet reads, transcript output, or scheduler.
|
||||
|
||||
```sh
|
||||
node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12
|
||||
node packages/ledger/src/cli.mjs --since 2026-09-06 --until 2026-09-12 --json
|
||||
node packages/ledger/src/cli.mjs --since 2026-09-06 --no-issues
|
||||
+node packages/ledger/src/cli.mjs --since 2026-09-06 --no-t3
|
||||
+node packages/ledger/src/cli.mjs --since 2026-09-06 --t3-db /tmp/fixture.sqlite
|
||||
node --test packages/ledger/tests/
|
||||
```
|
||||
|
||||
@@ -36,12 +39,17 @@ No install, build, service restart, or configuration change is needed.
|
||||
duplicated entries in copied logs are not deduplicated. No transcript content
|
||||
leaves the parser. Assistant messages and logs outside repo seats do not count.
|
||||
Symlink source directories are refused and symlink files are not followed.
|
||||
+ A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
|
||||
+- Table 2 also counts T3 thread messages with role `user`. The T3 source
|
||||
+ follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
|
||||
+ `pi` (Pi rows) and `t3.seats` (T3 rows).
|
||||
- The first text line alone classifies a message. Two addressing forms count:
|
||||
the tmux preamble `[host:session -> host:session]` that `agent-send.sh`
|
||||
writes, and the T3 header `[from: role (thread-id) -> to: role (thread-id)]`
|
||||
from `docs/guides/T3-AGENT-COMMS.md`. Either may carry ` class=<class>` before
|
||||
the closing bracket. A preamble whose sender is `control-board` (tmux session
|
||||
or T3 role) is board; any other valid preamble is agent; otherwise human.
|
||||
+ The class may be in either case: seats send `class=DECISION`.
|
||||
This is a format count, not proof of who typed the message. Text blocks are
|
||||
joined with newlines. The entry timestamp is used, falling back to the
|
||||
message timestamp.
|
||||
@@ -53,6 +61,69 @@ No install, build, service restart, or configuration change is needed.
|
||||
ratios and durations with one decimal. Titles truncate to 48 characters in
|
||||
text only. Missing evidence is the literal string `unknown`.
|
||||
|
||||
+## T3 source
|
||||
+
|
||||
+The rules come from `docs/plans/2026-09-26_ledger-t3-source.md` (Gate F).
|
||||
+The source is on by default. `--no-t3` skips it, and the report then says
|
||||
+`T3: not read (--no-t3)`. `--t3-db <path>` reads another database file with
|
||||
+the same checks. The JSON records the database path and whether it was the
|
||||
+default. When it wasn't, the text report prints the path, so a fixture result
|
||||
+can't pass for a live one. The two flags can't be combined.
|
||||
+
|
||||
+The reader opens `state.sqlite` read-only through `node:sqlite` and reads no
|
||||
+other file in `~/.t3`. It runs every query in one read transaction with a 5 s
|
||||
+busy timeout. It never writes the main database file. Like any SQLite
|
||||
+connection it may create `-wal` and `-shm` beside it, so a directory that
|
||||
+isn't writable refuses when SQLite needs them.
|
||||
+
|
||||
+- **Project.** Only threads in the one non-deleted T3 project whose
|
||||
+ `workspace_root` equals this checkout's root byte for byte. The root is the
|
||||
+ realpath of the package, so a project opened through the compatibility
|
||||
+ symlink `~/src/mosaic-stack-dev-test` does not match, and the report refuses
|
||||
+ with no project.
|
||||
+- **Thread to seat.** A thread belongs to seat `<s>` when `<s>` is a real
|
||||
+ directory in `agents/` and the lower-cased title equals `<s>` or starts
|
||||
+ with `<s>` and a space. "Dewey in Claude" maps to `dewey`; "Sagebrush" maps
|
||||
+ to nothing. Several threads can map to one seat. Threads that map to no
|
||||
+ seat share one row, `t3:unmapped`, so their human messages still reach the
|
||||
+ totals. `t3.seats` and `t3.unmapped` in the JSON list the thread ids and
|
||||
+ titles behind each row.
|
||||
+- **Titles are current state.** T3 titles an unnamed thread from its first
|
||||
+ prompt, and a rename moves a thread's whole history to another row. This
|
||||
+ moves counts between rows, never out of the totals.
|
||||
+- **Header check.** A user message whose T3 header is addressed to its own
|
||||
+ thread id must name that thread's seat as the `to:` role (compared lower
|
||||
+ case). In an unmapped thread the `to:` role must not be a seat. A conflict
|
||||
+ exits 1 and names the thread, its title and both roles. A header addressed
|
||||
+ to another thread isn't checked. The check misses a renamed thread that no
|
||||
+ agent writes to. Such a thread can only add human counts to a row.
|
||||
+- **Excluded.** Imported threads (id prefix `import:`) are partial copies of
|
||||
+ Claude Code sessions, not T3 traffic; every T3 event marked `historyImport`
|
||||
+ sits in one today. Deleted threads don't count; archived threads do.
|
||||
+ `t3.excluded` gives both thread counts.
|
||||
+- **Blind spot.** Threads in other T3 projects are not counted, even if they
|
||||
+ worked on this repository. Live, there is a project at `/home/jwoltje` and
|
||||
+ a deleted one at `/mnt/storage/src`.
|
||||
+- **Diagnostic.** `t3.diagnostic.humanSentThroughApi` counts in-range user
|
||||
+ messages the header rule calls human that T3 recorded as sent through its
|
||||
+ API (no `appVersion` in the event's origin). Those are seat messages whose
|
||||
+ header the rule doesn't accept, such as the older free-text Discord Bot
|
||||
+ headers, and would show the next format drift. `humanWithoutEvent` counts
|
||||
+ human messages with no `thread.message-sent` event. This is T3's internal
|
||||
+ metadata, so it feeds no table or total. If `orchestration_events` or a
|
||||
+ column it needs is missing, or an event doesn't parse, both read `unknown`.
|
||||
+
|
||||
+These refuse the report with exit 1, and the ones about the database name
|
||||
+`--no-t3`: a missing, unreadable or unopenable database (including a busy
|
||||
+lock past the timeout); a symlink at `~/.t3`, `~/.t3/userdata` or
|
||||
+`state.sqlite` (with `--t3-db`, the file or its directory); a missing table or
|
||||
+column the counts need; no project or more than one for this root; a message
|
||||
+in a counted thread with a role other than `user` or `assistant`, non-text
|
||||
+content, or a `created_at` that doesn't parse; a header conflict. A missing Pi
|
||||
+directory means no Pi seats ran here; a missing T3 database means the path or
|
||||
+T3 changed, so it refuses instead of counting zero. Error messages name ids
|
||||
+and paths, never message text.
|
||||
+
|
||||
## One Gitea call and missing evidence
|
||||
|
||||
The client requests issues updated since the start date, all states, first page,
|
||||
@@ -62,8 +133,8 @@ Use a narrower range or `--no-issues`, not hidden pagination. A commit-linked
|
||||
issue not returned by the updated-since query still has a row, with unknown
|
||||
metadata. This is the cost of the brief's one-call boundary.
|
||||
|
||||
-Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git
|
||||
-or session evidence. Malformed JSONL, including a partially written last line,
|
||||
+Exit 0 means a report was computed. Exit 1 means bad arguments or unreadable git,
|
||||
+session or T3 evidence. Malformed JSONL, including a partially written last line,
|
||||
refuses the report; rerun after the seat finishes writing. Exit 2 means issue
|
||||
credentials, API, payload, or completeness failure. The CLI never prints API
|
||||
error bodies or reads authentication files itself. `--no-issues` makes no API
|
||||
@@ -72,8 +143,9 @@ median duration, and human-per-closed ratio. It cannot invent close-only rows.
|
||||
|
||||
For fixtures, a fake `gitea-api.sh` can be placed first on PATH. Otherwise the
|
||||
repository scripts directory is appended to PATH for the issue request.
|
||||
-Tests use only temporary repositories, logs, and fake API tools, with no real
|
||||
-credentials or network. The helper regression stubs Node before any credential
|
||||
+Tests use only temporary repositories, logs, T3 databases and fake API tools,
|
||||
+with no real credentials or network. Every CLI run in the tests sets `HOME` to
|
||||
+a temporary directory, so no test opens the real `~/.t3`. The helper regression stubs Node before any credential
|
||||
read and checks successful GET, successful POST, and failed HTTP status.
|
||||
|
||||
## Acceptance
|
||||
diff --git a/packages/ledger/src/cli.mjs b/packages/ledger/src/cli.mjs
|
||||
index 66cd1417..5cf6705c 100644
|
||||
--- a/packages/ledger/src/cli.mjs
|
||||
+++ b/packages/ledger/src/cli.mjs
|
||||
@@ -1,11 +1,12 @@
|
||||
#!/usr/bin/env node
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
-import { dateRange, readCommits, readIssues, readSessions, summarize, formatTable, SourceError } from './ledger.mjs';
|
||||
+import { dateRange, readCommits, readIssues, readSessions, mergeSources, summarize, formatTable, SourceError } from './ledger.mjs';
|
||||
+import { readT3, defaultT3Path } from './t3.mjs';
|
||||
|
||||
-const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues]';
|
||||
+const usage = 'Usage: node packages/ledger/src/cli.mjs --since YYYY-MM-DD [--until YYYY-MM-DD] [--json] [--no-issues] [--no-t3 | --t3-db PATH]';
|
||||
export async function main(args, root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..')) {
|
||||
- let since, until, json = false, noIssues = false;
|
||||
+ let since, until, t3Db, json = false, noIssues = false, noT3 = false;
|
||||
const seen = new Set();
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const flag = args[i];
|
||||
@@ -14,13 +15,18 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath(
|
||||
if (flag === '--help') { console.log(usage); return; }
|
||||
if (flag === '--json') json = true;
|
||||
else if (flag === '--no-issues') noIssues = true;
|
||||
- else if (flag === '--since' || flag === '--until') {
|
||||
+ else if (flag === '--no-t3') noT3 = true;
|
||||
+ else if (flag === '--t3-db') {
|
||||
+ t3Db = args[++i];
|
||||
+ if (!t3Db || t3Db.startsWith('--')) throw new SourceError('--t3-db requires a path');
|
||||
+ } else if (flag === '--since' || flag === '--until') {
|
||||
const value = args[++i];
|
||||
if (!value || value.startsWith('--')) throw new SourceError(`${flag} requires a date`);
|
||||
if (flag === '--since') since = value; else until = value;
|
||||
} else throw new SourceError('Unknown option; ' + usage);
|
||||
}
|
||||
if (!since) throw new SourceError(usage);
|
||||
+ if (noT3 && t3Db !== undefined) throw new SourceError('--no-t3 and --t3-db cannot be combined');
|
||||
const range = dateRange(since, until);
|
||||
const commits = readCommits(root, range);
|
||||
// Fixture tools may be placed first on PATH. The repository client is the
|
||||
@@ -30,7 +36,9 @@ export async function main(args, root = path.resolve(path.dirname(fileURLToPath(
|
||||
let issues;
|
||||
try { issues = noIssues ? null : readIssues(root, range); }
|
||||
finally { if (priorPath === undefined) delete process.env.PATH; else process.env.PATH = priorPath; }
|
||||
- const sessions = await readSessions(root, range);
|
||||
+ // T3 is on by default. A missing or unreadable database refuses the report.
|
||||
+ const t3 = noT3 ? null : await readT3(root, range, t3Db === undefined ? { dbPath: defaultT3Path(), isDefault: true } : { dbPath: t3Db, isDefault: false });
|
||||
+ const sessions = mergeSources(await readSessions(root, range), t3);
|
||||
const report = summarize(range, commits, issues, sessions);
|
||||
console.log(json ? JSON.stringify(report, null, 2) : formatTable(report));
|
||||
return report;
|
||||
diff --git a/packages/ledger/src/ledger.mjs b/packages/ledger/src/ledger.mjs
|
||||
index b09e95f5..3669d1cc 100644
|
||||
--- a/packages/ledger/src/ledger.mjs
|
||||
+++ b/packages/ledger/src/ledger.mjs
|
||||
@@ -1,7 +1,6 @@
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { createReadStream } from 'node:fs';
|
||||
import { readdir, lstat } from 'node:fs/promises';
|
||||
-import { createInterface } from 'node:readline';
|
||||
import path from 'node:path';
|
||||
|
||||
const DAY = 86400000;
|
||||
@@ -23,7 +22,7 @@ export function dateRange(since, until = new Date().toISOString().slice(0, 10))
|
||||
if (end <= start) throw new SourceError('--until must not precede --since');
|
||||
return { since, until, start, end };
|
||||
}
|
||||
-const inRange = (value, range) => {
|
||||
+export const inRange = (value, range) => {
|
||||
const ms = typeof value === 'number' ? value : Date.parse(value);
|
||||
return Number.isFinite(ms) && ms >= range.start && ms < range.end;
|
||||
};
|
||||
@@ -75,15 +74,32 @@ export function messageText(content) {
|
||||
if (Array.isArray(content)) return content.filter(c => c?.type === 'text' && typeof c.text === 'string').map(c => c.text).join('\n');
|
||||
return '';
|
||||
}
|
||||
+// Classes are matched in either case: seats send DECISION and REVIEW-REQUEST.
|
||||
+// tmux preamble from agent-send.sh: [host:session -> host:session class=x]
|
||||
+const TMUX = /^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[A-Za-z-]+)?\](?:\s|$)/;
|
||||
+// T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x]
|
||||
+const T3 = /^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[A-Za-z-]+)?\](?:\s|$)/;
|
||||
+const firstLine = text => text.split(/\r?\n/, 1)[0];
|
||||
+export function t3Header(text) {
|
||||
+ const m = firstLine(text).match(T3);
|
||||
+ return m ? { from: m[1], fromId: m[2], to: m[3], toId: m[4] } : null;
|
||||
+}
|
||||
export function messageKind(text) {
|
||||
- const firstLine = text.split(/\r?\n/, 1)[0];
|
||||
- // tmux preamble from agent-send.sh: [host:session -> host:session class=x]
|
||||
- const tmux = firstLine.match(/^\[([^\s:\[\]]+):([^\s\[\]]+) -> ([^\s:\[\]]+):([^\s\[\]]+)(?: class=[a-z-]+)?\](?:\s|$)/);
|
||||
- // T3 header (docs/guides/T3-AGENT-COMMS.md): [from: role (id) -> to: role (id) class=x]
|
||||
- const t3 = firstLine.match(/^\[from: ([^\s()\[\]]+) \(([^()\[\]]+)\) -> to: ([^\s()\[\]]+) \(([^()\[\]]+)\)(?: class=[a-z-]+)?\](?:\s|$)/);
|
||||
- const sender = tmux ? tmux[2] : t3 ? t3[1] : null;
|
||||
+ const tmux = firstLine(text).match(TMUX), t3 = t3Header(text);
|
||||
+ const sender = tmux ? tmux[2] : t3 ? t3.from : null;
|
||||
return sender === null ? 'human' : sender === 'control-board' ? 'board' : 'agent';
|
||||
}
|
||||
+// JSONL lines end at \n only. readline also ends a line at U+2028, which JSON
|
||||
+// allows raw inside a string, so it split valid records (Node 26.8.1).
|
||||
+async function* jsonLines(input) {
|
||||
+ let rest = '';
|
||||
+ for await (const chunk of input) {
|
||||
+ const parts = (rest + chunk).split('\n');
|
||||
+ rest = parts.pop();
|
||||
+ yield* parts;
|
||||
+ }
|
||||
+ if (rest) yield rest;
|
||||
+}
|
||||
async function directories(dir, optional = false) {
|
||||
try {
|
||||
if (!(await lstat(dir)).isDirectory()) throw new SourceError('Session source must be a real directory');
|
||||
@@ -93,11 +109,15 @@ async function directories(dir, optional = false) {
|
||||
throw new SourceError(`Cannot read ledger directory: ${dir}`);
|
||||
}
|
||||
}
|
||||
+// Seats are the real directories in agents/, sorted.
|
||||
+export async function readSeats(root) {
|
||||
+ return (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).map(e => e.name).sort((a, b) => a.localeCompare(b));
|
||||
+}
|
||||
export async function readSessions(root, range) {
|
||||
const rows = [];
|
||||
const mentions = new Map();
|
||||
// No symlink traversal, no fleet paths, no transcript content in the report.
|
||||
- const agents = (await directories(path.join(root, 'agents'))).filter(e => e.isDirectory()).sort((a, b) => a.name.localeCompare(b.name));
|
||||
+ const agents = (await readSeats(root)).map(name => ({ name }));
|
||||
const state = path.join(root, '.pi', 'state');
|
||||
// Check every source ancestor, not only the leaf directory.
|
||||
if (!(await directories(path.join(root, '.pi'), true)).length) return { rows, mentions };
|
||||
@@ -109,11 +129,10 @@ export async function readSessions(root, range) {
|
||||
const files = (await directories(dir, true)).filter(e => e.isFile() && e.name.endsWith('.jsonl'));
|
||||
const row = { seat: agent.name, board: 0, agent: 0, human: 0 };
|
||||
for (const file of files) {
|
||||
- const input = createReadStream(path.join(dir, file.name));
|
||||
- const lines = createInterface({ input, crlfDelay: Infinity });
|
||||
+ const input = createReadStream(path.join(dir, file.name), { encoding: 'utf8' });
|
||||
let lineNumber = 0;
|
||||
try {
|
||||
- for await (const line of lines) {
|
||||
+ for await (const line of jsonLines(input)) {
|
||||
lineNumber++;
|
||||
if (!line.trim()) continue;
|
||||
let entry;
|
||||
@@ -132,12 +151,33 @@ export async function readSessions(root, range) {
|
||||
mentions.get(number).add(agent.name);
|
||||
}
|
||||
}
|
||||
- } finally { lines.close(); input.destroy(); }
|
||||
+ } finally { input.destroy(); }
|
||||
}
|
||||
if (row.board + row.agent + row.human) rows.push(row);
|
||||
}
|
||||
return { rows, mentions };
|
||||
}
|
||||
+// Adds T3 counts to the Pi rows per seat. Unmapped T3 threads get one row,
|
||||
+// last. The report keeps the Pi rows and the T3 section so the split shows.
|
||||
+export function mergeSources(pi, t3, unmapped = 't3:unmapped') {
|
||||
+ if (!t3) return { rows: pi.rows, mentions: pi.mentions, pi: pi.rows, t3: { read: false } };
|
||||
+ const bySeat = new Map(pi.rows.map(r => [r.seat, { ...r }]));
|
||||
+ for (const [seat, counts] of t3.rows) {
|
||||
+ if (seat === unmapped || !(counts.board + counts.agent + counts.human)) continue;
|
||||
+ const row = bySeat.get(seat) ?? { seat, board: 0, agent: 0, human: 0 };
|
||||
+ for (const kind of ['board', 'agent', 'human']) row[kind] += counts[kind];
|
||||
+ bySeat.set(seat, row);
|
||||
+ }
|
||||
+ const rows = [...bySeat.values()].sort((a, b) => a.seat.localeCompare(b.seat));
|
||||
+ const extra = t3.rows.get(unmapped);
|
||||
+ if (extra.board + extra.agent + extra.human) rows.push({ seat: unmapped, ...extra });
|
||||
+ const mentions = new Map([...pi.mentions].map(([n, seats]) => [n, new Set(seats)]));
|
||||
+ for (const [n, seats] of t3.mentions) {
|
||||
+ if (!mentions.has(n)) mentions.set(n, new Set());
|
||||
+ for (const seat of seats) mentions.get(n).add(seat);
|
||||
+ }
|
||||
+ return { rows, mentions, pi: pi.rows, t3: t3.section };
|
||||
+}
|
||||
const round = value => Math.round(value * 10) / 10;
|
||||
function duration(issue) {
|
||||
if (!issue) return UNKNOWN;
|
||||
@@ -163,13 +203,14 @@ export function summarize(range, commits, issues, sessions) {
|
||||
const median = hours.includes(UNKNOWN) ? UNKNOWN : hours.length ?
|
||||
round(hours.length % 2 ? hours[middle] : (hours[middle - 1] + hours[middle]) / 2) : 0;
|
||||
const human = sessions.rows.reduce((sum, r) => sum + r.human, 0);
|
||||
- return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows,
|
||||
+ const sources = sessions.t3 ? { pi: sessions.pi, t3: sessions.t3 } : {};
|
||||
+ return { since: range.since, until: range.until, timezone: 'UTC', issues: rows, seats: sessions.rows, ...sources,
|
||||
totals: { issuesClosed: issues === null ? UNKNOWN : closed.length,
|
||||
medianHoursOpen: issues === null ? UNKNOWN : median, commits: commits.length,
|
||||
followUpsPerIssue: rows.length ? round(rows.reduce((sum, r) => sum + r.followUps, 0) / rows.length) : 0,
|
||||
humanMessagesPerClosedIssue: issues === null ? UNKNOWN : closed.length ? round(human / closed.length) : human ? UNKNOWN : 0 } };
|
||||
}
|
||||
-const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' ');
|
||||
+export const clean = value => String(value).replace(/[\x00-\x1f\x7f-\x9f]/g, ' ');
|
||||
const decimal = value => typeof value === 'number' ? value.toFixed(1) : value;
|
||||
export function totalsLine(t) {
|
||||
return `Totals: issues closed ${t.issuesClosed} | median hours open ${decimal(t.medianHoursOpen)} | commits ${t.commits} | follow-ups per issue ${decimal(t.followUpsPerIssue)} | human messages per closed issue ${decimal(t.humanMessagesPerClosedIssue)}`;
|
||||
@@ -181,5 +222,12 @@ export function formatTable(report) {
|
||||
decimal(r.hoursOpen), r.commits, r.followUps, r.seats.join(', ')].join(' | ')),
|
||||
'', 'Seat | Board | Agent | Human',
|
||||
...report.seats.map(r => [clean(r.seat), r.board, r.agent, r.human].join(' | ')),
|
||||
+ ...t3Line(report.t3),
|
||||
'', totalsLine(report.totals)].join('\n');
|
||||
}
|
||||
+// One line when T3 was skipped or read from somewhere other than the default.
|
||||
+function t3Line(t3) {
|
||||
+ if (!t3) return [];
|
||||
+ if (!t3.read) return ['T3: not read (--no-t3)'];
|
||||
+ return t3.database.default ? [] : [`T3: read from ${clean(t3.database.path)}, not the default`];
|
||||
+}
|
||||
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
|
||||
index 175f5d5e..8e60b96b 100644
|
||||
--- a/packages/ledger/tests/ledger.test.mjs
|
||||
+++ b/packages/ledger/tests/ledger.test.mjs
|
||||
@@ -1,6 +1,7 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
-import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync } from 'node:fs';
|
||||
+import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, cpSync, symlinkSync, realpathSync } from 'node:fs';
|
||||
+import { DatabaseSync } from 'node:sqlite';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
@@ -9,13 +10,50 @@ import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../
|
||||
|
||||
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
|
||||
const range = dateRange('2026-09-06', '2026-09-12');
|
||||
+// T3 fixture schema: the live tables, cut to the columns the reader uses plus
|
||||
+// one it doesn't. `text` allows NULL so a non-text row can be tested.
|
||||
+const T3_SCHEMA = `
|
||||
+ create table projection_projects (project_id text primary key, title text not null, workspace_root text not null, deleted_at text);
|
||||
+ create table projection_threads (thread_id text primary key, project_id text not null, title text not null, archived_at text, deleted_at text);
|
||||
+ create table projection_thread_messages (message_id text primary key, thread_id text not null, role text not null, text, created_at text not null);
|
||||
+ create table orchestration_events (sequence integer primary key autoincrement, stream_id text not null, event_type text not null, payload_json text not null, metadata_json text not null);`;
|
||||
+// Writes a T3 database in WAL mode. Threads default to project p1, which is
|
||||
+// the fixture root. Returns the open writer when keepOpen is set.
|
||||
+function t3db(file, { root, projects, threads = [], messages = [], after = [], keepOpen = false }) {
|
||||
+ mkdirSync(path.dirname(file), { recursive: true });
|
||||
+ for (const old of [file, `${file}-wal`, `${file}-shm`]) rmSync(old, { force: true });
|
||||
+ const db = new DatabaseSync(file);
|
||||
+ db.exec('pragma journal_mode=wal'); db.exec(T3_SCHEMA);
|
||||
+ for (const [id, workspace, deleted = null] of projects ?? [['p1', root]]) {
|
||||
+ db.prepare('insert into projection_projects values (?, ?, ?, ?)').run(id, 'project', workspace, deleted);
|
||||
+ }
|
||||
+ for (const t of threads) {
|
||||
+ db.prepare('insert into projection_threads values (?, ?, ?, ?, ?)').run(t.id, t.project ?? 'p1', t.title, t.archived ?? null, t.deleted ?? null);
|
||||
+ }
|
||||
+ for (const m of messages) addMessage(db, m);
|
||||
+ for (const sql of after) db.exec(sql);
|
||||
+ if (keepOpen) return db;
|
||||
+ db.close();
|
||||
+}
|
||||
+let messageId = 0;
|
||||
+function addMessage(db, { thread, text, role = 'user', at = '2026-09-08T12:00:00Z', origin = 'app' }) {
|
||||
+ const id = `m${++messageId}`;
|
||||
+ db.prepare('insert into projection_thread_messages values (?, ?, ?, ?, ?)').run(id, thread, role, text, at);
|
||||
+ if (origin !== 'none') db.prepare('insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values (?, ?, ?, ?)')
|
||||
+ .run(thread, 'thread.message-sent', JSON.stringify({ messageId: id, threadId: thread, role, text }), JSON.stringify({ origin: origin === 'app' ? { appVersion: '0.0.0' } : {} }));
|
||||
+}
|
||||
const fixtureIssues = [
|
||||
{ number: 1, title: 'First issue', created_at: '2026-09-06T00:00:00Z', closed_at: '2026-09-07T12:00:00Z' },
|
||||
{ number: 2, title: 'Second issue', created_at: '2026-09-06T00:00:00Z', closed_at: null },
|
||||
];
|
||||
function fixture(t) {
|
||||
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
|
||||
- t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
+ // No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
|
||||
+ // T3 database at the default path. The CLI's root is a realpath.
|
||||
+ const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
|
||||
+ t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
|
||||
+ const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
|
||||
+ t3db(defaultDb, { root: realpathSync(root) });
|
||||
const put = (name, data) => { const p = path.join(root, name); mkdirSync(path.dirname(p), { recursive: true }); writeFileSync(p, data); return p; };
|
||||
const git = (args, date = '2026-09-07T00:00:00Z') => execFileSync('git', args, { cwd: root, env: { ...process.env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date, GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: '/dev/null' }, stdio: 'pipe' });
|
||||
git(['init', '-b', 'refactor']); git(['config', 'user.email', '[email protected]']); git(['config', 'user.name', 'Fixture']);
|
||||
@@ -30,8 +68,8 @@ function fixture(t) {
|
||||
const entry = (text, timestamp = '2026-09-08T12:00:00Z') => ({ type: 'message', timestamp, message: { role: 'user', content: [{ type: 'text', text }] } });
|
||||
const logs = [entry('[host:control-board -> host:alice] do #1'), entry('[host:bob -> host:alice] review #2'), entry('build #2'), entry('old #1', '2026-09-05T23:59:59Z'), { type: 'message', timestamp: '2026-09-08T00:00:00Z', message: { role: 'assistant', content: 'not a user #1' } }];
|
||||
put('.pi/state/alice/sessions/one.jsonl', logs.map(x => JSON.stringify(x)).join('\n') + '\n');
|
||||
- const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } });
|
||||
- return { root, put, commit, run, entry, logs };
|
||||
+ const run = (args = [], env = {}) => spawnSync(process.execPath, [path.join(root, 'packages/ledger/src/cli.mjs'), '--since', '2026-09-06', '--until', '2026-09-12', ...args], { cwd: root, encoding: 'utf8', env: { ...process.env, HOME: home, PATH: `${path.join(root, 'bin')}:${process.env.PATH}`, ISSUES: path.join(root, 'issues.json'), CALLS: path.join(root, 'calls.jsonl'), ...env } });
|
||||
+ return { root, real: realpathSync(root), home, defaultDb, put, commit, run, entry, logs };
|
||||
}
|
||||
test('fixture git subjects only, follow-ups and three session kinds', t => {
|
||||
const f = fixture(t), result = f.run(['--json']);
|
||||
@@ -63,7 +101,7 @@ test('missing credentials exit 2, no-issues never calls API and shows unknown',
|
||||
test('empty range gives no rows and zero totals', t => {
|
||||
const f = fixture(t);
|
||||
f.put('issues.json', '[]');
|
||||
- const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } });
|
||||
+ const result = spawnSync(process.execPath, [path.join(f.root, 'packages/ledger/src/cli.mjs'), '--since', '2027-01-01', '--until', '2027-01-02', '--json'], { encoding: 'utf8', env: { ...process.env, HOME: f.home, PATH: `${f.root}/bin:${process.env.PATH}`, ISSUES: `${f.root}/issues.json`, CALLS: `${f.root}/calls.jsonl` } });
|
||||
assert.equal(result.status, 0, result.stderr); const r = JSON.parse(result.stdout);
|
||||
assert.deepEqual(r.issues, []); assert.deepEqual(r.seats, []); assert.ok(Object.values(r.totals).every(n => n === 0));
|
||||
});
|
||||
@@ -96,6 +134,13 @@ test('partial or malformed session log refuses with location, not content', t =>
|
||||
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
|
||||
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
|
||||
});
|
||||
+test('a U+2028 inside a session string is one line, not a malformed record', t => {
|
||||
+ const f = fixture(t);
|
||||
+ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
||||
+ assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
|
||||
+ const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
|
||||
+});
|
||||
test('no sessions is an empty table; symlink source refuses', t => {
|
||||
const f = fixture(t); rmSync(path.join(f.root, '.pi'), { recursive: true });
|
||||
assert.deepEqual(JSON.parse(f.run(['--json']).stdout).seats, []);
|
||||
@@ -140,7 +185,11 @@ test('T3 header: agent, or board from control-board; anything short of the full
|
||||
assert.equal(messageKind(`Jason here\n[from: ${sage} -> to: ${filbert}]\nquoted`), 'human');
|
||||
assert.equal(messageKind(` [from: ${sage} -> to: ${filbert}]`), 'human');
|
||||
assert.equal(messageKind(`[from: sage -> to: filbert]\nno thread ids`), 'human');
|
||||
- assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'human');
|
||||
+ // Classes match in either case (Gate F). HEAD before the fix called these human.
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=Actionable]`), 'agent');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=REVIEW-REQUEST]\nreview`), 'agent');
|
||||
+ assert.equal(messageKind('[h:sage -> h:bob class=DECISION] go'), 'agent');
|
||||
+ assert.equal(messageKind(`[from: ${sage} -> to: ${filbert} class=review_request]`), 'human');
|
||||
assert.equal(messageKind(`[from: ${sage} -> to: ${filbert}]trailing`), 'human');
|
||||
assert.equal(messageKind(`[From: ${sage} -> to: ${filbert}]`), 'human');
|
||||
});
|
||||
@@ -148,3 +197,218 @@ test('no closed issues with human messages means undefined ratio, not invented z
|
||||
const r = summarize(range, [], [], { rows: [{ seat: 'a', human: 1, board: 0, agent: 0 }], mentions: new Map() });
|
||||
assert.equal(r.totals.humanMessagesPerClosedIssue, 'unknown');
|
||||
});
|
||||
+
|
||||
+// T3 thread source (Gate F, docs/plans/2026-09-26_ledger-t3-source.md).
|
||||
+const T1 = 't-alice', T2 = 't-bob', T3 = 't-sagebrush', T4 = 't-researcher', T5 = 't-discord';
|
||||
+const header = (from, to, toId, cls = '') => `[from: ${from} (x1) -> to: ${to} (${toId})${cls}]`;
|
||||
+function t3Fixture(t) {
|
||||
+ const f = fixture(t);
|
||||
+ for (const seat of ['sage', 'researcher']) mkdirSync(path.join(f.root, 'agents', seat), { recursive: true });
|
||||
+ const db = path.join(f.home, 'fixture/t3.sqlite');
|
||||
+ const threads = [
|
||||
+ { id: T1, title: 'Alice' }, { id: T2, title: 'Bob in Claude', archived: '2026-09-09T00:00:00Z' },
|
||||
+ { id: T3, title: 'Sagebrush' }, { id: T4, title: 'Researcher' }, { id: T5, title: 'Discord Bot' },
|
||||
+ { id: 'import:claudeAgent:1', title: 'alice' }, { id: 't-deleted', title: 'Alice', deleted: '2026-09-09T00:00:00Z' },
|
||||
+ { id: 't-other', project: 'p2', title: 'Alice' },
|
||||
+ ];
|
||||
+ const messages = [
|
||||
+ { thread: T1, text: 'Jason: go #1' },
|
||||
+ { thread: T1, text: `${header('sage', 'alice', T1, ' class=REVIEW-REQUEST')}\nreview #2`, origin: 'api' },
|
||||
+ { thread: T1, text: '[h:sage -> h:alice class=DECISION] go', origin: 'api' },
|
||||
+ { thread: T1, text: `${header('control-board', 'alice', T1)}\nbuzz`, origin: 'api' },
|
||||
+ { thread: T1, text: 'outside', at: '2026-09-13T00:00:00Z' },
|
||||
+ { thread: T1, text: 'an answer #9', role: 'assistant', origin: 'none' },
|
||||
+ { thread: T2, text: 'archived still counts #2' },
|
||||
+ { thread: T3, text: 'Sagebrush is not sage' },
|
||||
+ { thread: T3, text: `${header('sage', 'discord', T3)}\nnot a seat role`, origin: 'api' },
|
||||
+ { thread: T4, text: 'research this' },
|
||||
+ { thread: T5, text: '[from: SetSpark coordinator (x1) -> to: Discord Bot (x2)]\nfree text', origin: 'api' },
|
||||
+ { thread: 'import:claudeAgent:1', text: 'imported' },
|
||||
+ { thread: 't-deleted', text: 'deleted' },
|
||||
+ { thread: 't-other', text: 'other project' },
|
||||
+ ];
|
||||
+ const write = (overrides = {}) => t3db(db, { root: f.real, projects: [['p1', f.real], ['p2', '/elsewhere']], threads, messages, ...overrides });
|
||||
+ return { ...f, db, threads, messages, write };
|
||||
+}
|
||||
+test('T3: seat, archived, unmapped and Researcher threads count; imported, deleted and other-project threads do not', t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ const result = f.run(['--json', '--t3-db', f.db]);
|
||||
+ assert.equal(result.status, 0, result.stderr);
|
||||
+ const r = JSON.parse(result.stdout);
|
||||
+ assert.deepEqual(r.seats, [
|
||||
+ { seat: 'alice', board: 2, agent: 3, human: 2 }, { seat: 'bob', board: 0, agent: 0, human: 1 },
|
||||
+ { seat: 'researcher', board: 0, agent: 0, human: 1 }, { seat: 't3:unmapped', board: 0, agent: 1, human: 2 },
|
||||
+ ]);
|
||||
+ assert.deepEqual(r.pi, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]);
|
||||
+ assert.deepEqual(r.t3.database, { path: f.db, default: false });
|
||||
+ assert.deepEqual(r.t3.seats, [
|
||||
+ { seat: 'alice', board: 1, agent: 2, human: 1, threads: [{ id: T1, title: 'Alice', archived: false }] },
|
||||
+ { seat: 'bob', board: 0, agent: 0, human: 1, threads: [{ id: T2, title: 'Bob in Claude', archived: true }] },
|
||||
+ { seat: 'researcher', board: 0, agent: 0, human: 1, threads: [{ id: T4, title: 'Researcher', archived: false }] },
|
||||
+ ]);
|
||||
+ assert.deepEqual(r.t3.unmapped, { board: 0, agent: 1, human: 2, threads: [
|
||||
+ { id: T5, title: 'Discord Bot', archived: false }, { id: T3, title: 'Sagebrush', archived: false }] });
|
||||
+ assert.deepEqual(r.t3.excluded, { importedThreads: 1, deletedThreads: 1 });
|
||||
+ // The free-text header counts as human; only the diagnostic shows it was sent through the API.
|
||||
+ assert.deepEqual(r.t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 0 });
|
||||
+ assert.equal(r.totals.humanMessagesPerClosedIssue, 6);
|
||||
+ assert.deepEqual(r.issues.map(x => [x.issue, x.seats]), [[1, ['alice']], [2, ['alice', 'bob']]]);
|
||||
+ const text = f.run(['--t3-db', f.db]);
|
||||
+ assert.equal(text.status, 0, text.stderr);
|
||||
+ assert.ok(text.stdout.includes(`T3: read from ${f.db}, not the default`));
|
||||
+ assert.match(text.stdout, /t3:unmapped \| 0 \| 1 \| 2/);
|
||||
+});
|
||||
+test('T3: the default path is read from HOME and prints no path line; --no-t3 says so', t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ rmSync(f.defaultDb); cpSync(f.db, f.defaultDb);
|
||||
+ const json = JSON.parse(f.run(['--json']).stdout);
|
||||
+ assert.deepEqual(json.t3.database, { path: f.defaultDb, default: true });
|
||||
+ assert.equal(json.seats.at(-1).seat, 't3:unmapped');
|
||||
+ const text = f.run(); assert.equal(text.status, 0, text.stderr); assert.doesNotMatch(text.stdout, /^T3:/m);
|
||||
+ rmSync(path.join(f.home, '.t3'), { recursive: true });
|
||||
+ const off = f.run(['--no-t3']); assert.equal(off.status, 0, off.stderr);
|
||||
+ assert.match(off.stdout, /^T3: not read \(--no-t3\)$/m);
|
||||
+ const offJson = JSON.parse(f.run(['--no-t3', '--json']).stdout);
|
||||
+ assert.deepEqual(offJson.t3, { read: false }); assert.deepEqual(offJson.seats, [{ seat: 'alice', board: 1, agent: 1, human: 1 }]);
|
||||
+ const both = f.run(['--no-t3', '--t3-db', f.db]); assert.equal(both.status, 1); assert.match(both.stderr, /cannot be combined/);
|
||||
+ assert.equal(f.run(['--t3-db']).status, 1);
|
||||
+});
|
||||
+test('T3: a HOME with no database exits 1 and names --no-t3', t => {
|
||||
+ const f = fixture(t); rmSync(path.join(f.home, '.t3'), { recursive: true });
|
||||
+ const r = f.run(); assert.equal(r.status, 1); assert.equal(r.stdout, '');
|
||||
+ assert.match(r.stderr, /T3 database unavailable: .*\.t3 is missing or unreadable; use --no-t3/);
|
||||
+});
|
||||
+test('T3: a file that is not a database exits 1 and names --no-t3', t => {
|
||||
+ const f = fixture(t); writeFileSync(f.defaultDb, 'not sqlite'.repeat(100));
|
||||
+ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, /T3 database cannot be read: .*\(SQLite \d+\); use --no-t3/);
|
||||
+});
|
||||
+test('T3: a seat thread renamed to another seat exits 1 naming thread, title and roles', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T2, ' class=INFO')}\nfor alice`, origin: 'api' }] });
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1);
|
||||
+ assert.equal(r.stderr.trim(), `T3 header conflict: thread ${T2} "Bob in Claude" maps to bob, but a header addresses alice`);
|
||||
+});
|
||||
+test('T3: an unmapped thread addressed as a seat exits 1', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T3, text: `${header('bob', 'Sage', T3)}\nhi`, origin: 'api' }] });
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1);
|
||||
+ assert.match(r.stderr, /thread t-sagebrush "Sagebrush" maps to no seat, but a header addresses Sage/);
|
||||
+});
|
||||
+test('T3: a header to another thread id is not cross-checked', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T2, text: `${header('sage', 'alice', T1)}\ncopied`, origin: 'api' }] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.equal(JSON.parse(r.stdout).t3.seats[1].agent, 1);
|
||||
+});
|
||||
+test('T3: no project, or two, for this root exits 1', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ projects: [['p1', `${f.real}-link`], ['p2', '/elsewhere']] });
|
||||
+ let r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has no project for .*symlink does not match/);
|
||||
+ f.write({ projects: [['p1', f.real], ['p2', f.real]] });
|
||||
+ r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /T3 has more than one project for/);
|
||||
+ f.write({ projects: [['p1', f.real], ['p2', f.real, '2026-09-01T00:00:00Z']] });
|
||||
+ assert.equal(f.run(['--t3-db', f.db]).status, 0);
|
||||
+});
|
||||
+for (const [name, after, pattern] of [
|
||||
+ ['a removed column', ['alter table projection_threads drop column title'], /T3 schema changed: missing projection_threads.title/],
|
||||
+ ['a missing table', ['drop table projection_thread_messages'], /T3 schema changed: missing projection_thread_messages$/m],
|
||||
+]) test(`T3: ${name} exits 1 and names it`, t => {
|
||||
+ const f = t3Fixture(t); f.write({ after, messages: [] });
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern);
|
||||
+});
|
||||
+for (const [name, message, pattern] of [
|
||||
+ ['an unknown role', { role: 'system' }, /T3 message m\d+ in thread t-alice has an unknown role/],
|
||||
+ ['non-text content', { text: null }, /has non-text content/],
|
||||
+ ['an unparseable created_at', { at: 'yesterday' }, /has an invalid created_at/],
|
||||
+]) test(`T3: a counted row with ${name} exits 1 without its text`, t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T1, text: 'secret words', ...message }] });
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, pattern); assert.doesNotMatch(r.stderr, /secret/);
|
||||
+});
|
||||
+test('T3: a missing orchestration_events makes the diagnostic unknown and keeps the counts', t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
|
||||
+ f.write({ after: ['drop table orchestration_events'] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ const after = JSON.parse(r.stdout);
|
||||
+ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
||||
+ assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
|
||||
+});
|
||||
+for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
|
||||
+ const f = fixture(t), target = path.join(f.home, 'real', link);
|
||||
+ mkdirSync(path.dirname(target), { recursive: true });
|
||||
+ cpSync(path.join(f.home, link), target, { recursive: true });
|
||||
+ rmSync(path.join(f.home, link), { recursive: true }); symlinkSync(target, path.join(f.home, link));
|
||||
+ const r = f.run(); assert.equal(r.status, 1); assert.match(r.stderr, new RegExp(`${link.replaceAll('.', '\\.')} is a symlink; use --no-t3`));
|
||||
+});
|
||||
+test('T3: with --t3-db, a symlinked file or directory exits 1', t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ const file = path.join(f.home, 'file-link.sqlite'); symlinkSync(f.db, file);
|
||||
+ let r = f.run(['--t3-db', file]); assert.equal(r.status, 1); assert.match(r.stderr, /file-link.sqlite is a symlink/);
|
||||
+ const dir = path.join(f.home, 'dir-link'); symlinkSync(path.dirname(f.db), dir);
|
||||
+ r = f.run(['--t3-db', path.join(dir, 't3.sqlite')]); assert.equal(r.status, 1); assert.match(r.stderr, /dir-link is a symlink/);
|
||||
+});
|
||||
+
|
||||
+// WAL states. The CLI reads with mode=ro; it may create -wal and -shm but must
|
||||
+// never change the main file.
|
||||
+const sha = file => execFileSync('sha256sum', [file], { encoding: 'utf8' }).split(' ')[0];
|
||||
+const humans = r => JSON.parse(r.stdout).t3.seats.find(s => s.seat === 'alice').human;
|
||||
+const asRoot = process.getuid?.() === 0;
|
||||
+function killedWriter(db) {
|
||||
+ // A writer that commits into the WAL and dies without a checkpoint.
|
||||
+ const code = `const { DatabaseSync } = require('node:sqlite'); const db = new DatabaseSync(${JSON.stringify(db)});
|
||||
+ db.exec('pragma wal_autocheckpoint=0');
|
||||
+ db.prepare("insert into projection_thread_messages values ('late', 't-alice', 'user', 'late human', '2026-09-08T13:00:00Z')").run();
|
||||
+ process.kill(process.pid, 'SIGKILL');`;
|
||||
+ const r = spawnSync(process.execPath, ['-e', code]);
|
||||
+ assert.equal(r.signal, 'SIGKILL');
|
||||
+ rmSync(`${db}-shm`);
|
||||
+}
|
||||
+function inReadOnlyDir(dir, check) {
|
||||
+ execFileSync('chmod', ['0555', dir]);
|
||||
+ try { check(); } finally { execFileSync('chmod', ['0755', dir]); }
|
||||
+}
|
||||
+test('T3 WAL: the newest message only in -wal, writer attached, is counted', t => {
|
||||
+ const f = t3Fixture(t), writer = f.write({ keepOpen: true });
|
||||
+ t.after(() => writer.close());
|
||||
+ writer.exec('pragma wal_autocheckpoint=0');
|
||||
+ addMessage(writer, { thread: T1, text: 'newest', at: '2026-09-08T13:00:00Z' });
|
||||
+ const main = sha(f.db);
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.equal(humans(r), 2); assert.equal(sha(f.db), main);
|
||||
+});
|
||||
+test('T3 WAL: stopped cleanly, counts are correct and the main file is unchanged', t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ assert.throws(() => readFileSync(`${f.db}-wal`));
|
||||
+ const main = sha(f.db);
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.equal(humans(r), 1); assert.equal(sha(f.db), main);
|
||||
+});
|
||||
+test('T3 WAL: -wal without -shm in a writable directory is read', t => {
|
||||
+ const f = t3Fixture(t); f.write(); killedWriter(f.db);
|
||||
+ const main = sha(f.db);
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.equal(humans(r), 2); assert.equal(sha(f.db), main);
|
||||
+});
|
||||
+test('T3 WAL: -wal without -shm in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => {
|
||||
+ const f = t3Fixture(t); f.write(); killedWriter(f.db);
|
||||
+ inReadOnlyDir(path.dirname(f.db), () => {
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 14\); use --no-t3/);
|
||||
+ });
|
||||
+});
|
||||
+test('T3 WAL: stopped cleanly in a read-only directory exits 1', { skip: asRoot && 'mode bits do not bind root' }, t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ inReadOnlyDir(path.dirname(f.db), () => {
|
||||
+ const r = f.run(['--t3-db', f.db]); assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 1544\); use --no-t3/);
|
||||
+ });
|
||||
+});
|
||||
+test('T3: a lock held past the 5 s busy timeout exits 1 and names --no-t3', t => {
|
||||
+ const f = t3Fixture(t), writer = f.write({ keepOpen: true });
|
||||
+ t.after(() => writer.close());
|
||||
+ writer.exec('pragma locking_mode=exclusive'); writer.exec('begin exclusive');
|
||||
+ addMessage(writer, { thread: T1, text: 'held' });
|
||||
+ const started = Date.now(), r = f.run(['--t3-db', f.db]);
|
||||
+ writer.exec('commit');
|
||||
+ assert.equal(r.status, 1); assert.match(r.stderr, /cannot be read: .*\(SQLite 5\); use --no-t3/);
|
||||
+ assert.ok(Date.now() - started >= 4500, 'the reader waited for the busy timeout');
|
||||
+});
|
||||
diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs
|
||||
new file mode 100644
|
||||
index 00000000..9672fcc9
|
||||
--- /dev/null
|
||||
+++ b/packages/ledger/src/t3.mjs
|
||||
@@ -0,0 +1,151 @@
|
||||
+import { lstat } from 'node:fs/promises';
|
||||
+import { DatabaseSync } from 'node:sqlite';
|
||||
+import { pathToFileURL } from 'node:url';
|
||||
+import os from 'node:os';
|
||||
+import path from 'node:path';
|
||||
+import { SourceError, UNKNOWN, clean, inRange, issueNumbers, messageKind, readSeats, t3Header } from './ledger.mjs';
|
||||
+
|
||||
+// T3 keeps every thread message in one SQLite database. This reader opens that
|
||||
+// file read-only and nothing else in ~/.t3. See
|
||||
+// docs/plans/2026-09-26_ledger-t3-source.md for the rules below.
|
||||
+export const UNMAPPED = 't3:unmapped';
|
||||
+const SKIP = 'use --no-t3 to skip T3';
|
||||
+const REQUIRED = {
|
||||
+ projection_projects: ['project_id', 'workspace_root', 'deleted_at'],
|
||||
+ projection_threads: ['thread_id', 'project_id', 'title', 'archived_at', 'deleted_at'],
|
||||
+ projection_thread_messages: ['message_id', 'thread_id', 'role', 'text', 'created_at'],
|
||||
+};
|
||||
+const DIAGNOSTIC = { orchestration_events: ['stream_id', 'event_type', 'payload_json', 'metadata_json'] };
|
||||
+
|
||||
+export const defaultT3Path = () => path.join(os.homedir(), '.t3', 'userdata', 'state.sqlite');
|
||||
+
|
||||
+// Every named path must exist and must not be a symlink. Skipping one would be
|
||||
+// a silent zero, so each problem refuses the report.
|
||||
+async function checkPaths(dbPath, isDefault) {
|
||||
+ const dirs = isDefault ? [path.dirname(path.dirname(dbPath)), path.dirname(dbPath)] : [path.dirname(dbPath)];
|
||||
+ for (const [target, wantDir] of [...dirs.map(d => [d, true]), [dbPath, false]]) {
|
||||
+ let stat;
|
||||
+ try { stat = await lstat(target); }
|
||||
+ catch { throw new SourceError(`T3 database unavailable: ${target} is missing or unreadable; ${SKIP}`); }
|
||||
+ if (stat.isSymbolicLink()) throw new SourceError(`T3 database refused: ${target} is a symlink; ${SKIP}`);
|
||||
+ if (wantDir ? !stat.isDirectory() : !stat.isFile()) {
|
||||
+ throw new SourceError(`T3 database refused: ${target} is not a ${wantDir ? 'directory' : 'regular file'}; ${SKIP}`);
|
||||
+ }
|
||||
+ }
|
||||
+}
|
||||
+
|
||||
+function missingColumns(db, tables) {
|
||||
+ const missing = [];
|
||||
+ for (const [table, columns] of Object.entries(tables)) {
|
||||
+ const have = new Set(db.prepare('select name from pragma_table_info(?)').all(table).map(r => r.name));
|
||||
+ if (!have.size) missing.push(table);
|
||||
+ else for (const column of columns) if (!have.has(column)) missing.push(`${table}.${column}`);
|
||||
+ }
|
||||
+ return missing;
|
||||
+}
|
||||
+
|
||||
+// Seat for a thread title: the lower-cased title equals the seat or starts
|
||||
+// with the seat and a space. Longest seat first, so the most specific wins.
|
||||
+export function seatForTitle(title, seats) {
|
||||
+ const lower = title.toLowerCase();
|
||||
+ return [...seats].sort((a, b) => b.length - a.length).find(s => lower === s || lower.startsWith(`${s} `)) ?? null;
|
||||
+}
|
||||
+
|
||||
+// Origin per message id from thread.message-sent events. Any missing table,
|
||||
+// column or unparseable event makes the diagnostic unknown; it decides nothing.
|
||||
+function origins(db, projectId) {
|
||||
+ if (missingColumns(db, DIAGNOSTIC).length) return null;
|
||||
+ const byMessage = new Map();
|
||||
+ const events = db.prepare(`select e.payload_json, e.metadata_json from orchestration_events e
|
||||
+ join projection_threads t on t.thread_id = e.stream_id
|
||||
+ where e.event_type = 'thread.message-sent' and t.project_id = ?`).all(projectId);
|
||||
+ for (const event of events) {
|
||||
+ let payload, metadata;
|
||||
+ try { payload = JSON.parse(event.payload_json); metadata = JSON.parse(event.metadata_json); }
|
||||
+ catch { return null; }
|
||||
+ if (typeof payload?.messageId !== 'string') return null;
|
||||
+ byMessage.set(payload.messageId, typeof metadata?.origin?.appVersion === 'string');
|
||||
+ }
|
||||
+ return byMessage;
|
||||
+}
|
||||
+
|
||||
+function query(db, root, range, seats) {
|
||||
+ const missing = missingColumns(db, REQUIRED);
|
||||
+ if (missing.length) throw new SourceError(`T3 schema changed: missing ${missing.join(', ')}`);
|
||||
+ // Compared in JavaScript so a declared collation can't loosen the match.
|
||||
+ const projects = db.prepare('select project_id, workspace_root from projection_projects where deleted_at is null').all()
|
||||
+ .filter(p => p.workspace_root === root);
|
||||
+ if (projects.length !== 1) {
|
||||
+ throw new SourceError(`T3 has ${projects.length ? 'more than one project' : 'no project'} for ${root}; a project opened through a symlink does not match; ${SKIP}`);
|
||||
+ }
|
||||
+ const projectId = projects[0].project_id;
|
||||
+ const threads = new Map(), excluded = { importedThreads: 0, deletedThreads: 0 };
|
||||
+ for (const t of db.prepare('select thread_id, title, archived_at, deleted_at from projection_threads where project_id = ?').all(projectId)) {
|
||||
+ if (typeof t.thread_id !== 'string' || typeof t.title !== 'string') throw new SourceError('T3 thread with a non-text id or title');
|
||||
+ if (t.thread_id.startsWith('import:')) { excluded.importedThreads++; continue; }
|
||||
+ if (t.deleted_at !== null) { excluded.deletedThreads++; continue; }
|
||||
+ threads.set(t.thread_id, { id: t.thread_id, title: t.title, archived: t.archived_at !== null, seat: seatForTitle(t.title, seats) });
|
||||
+ }
|
||||
+ const rows = new Map([...seats, UNMAPPED].map(s => [s, { board: 0, agent: 0, human: 0 }]));
|
||||
+ const mentions = new Map(), human = [];
|
||||
+ const messages = db.prepare(`select m.message_id, m.thread_id, m.role, m.text, m.created_at from projection_thread_messages m
|
||||
+ join projection_threads t on t.thread_id = m.thread_id where t.project_id = ?`).all(projectId);
|
||||
+ for (const m of messages) {
|
||||
+ const thread = threads.get(m.thread_id);
|
||||
+ if (!thread) continue;
|
||||
+ const where = `T3 message ${clean(m.message_id)} in thread ${clean(m.thread_id)}`;
|
||||
+ if (m.role !== 'user' && m.role !== 'assistant') throw new SourceError(`${where} has an unknown role`);
|
||||
+ if (typeof m.text !== 'string') throw new SourceError(`${where} has non-text content`);
|
||||
+ if (typeof m.created_at !== 'string' || !Number.isFinite(Date.parse(m.created_at))) throw new SourceError(`${where} has an invalid created_at`);
|
||||
+ if (m.role !== 'user') continue;
|
||||
+ // A header addressed to its own thread must agree with the title mapping.
|
||||
+ const header = t3Header(m.text);
|
||||
+ if (header && header.toId === thread.id) {
|
||||
+ const to = header.to.toLowerCase();
|
||||
+ if (thread.seat ? to !== thread.seat : seats.includes(to)) {
|
||||
+ throw new SourceError(`T3 header conflict: thread ${clean(thread.id)} "${clean(thread.title)}" maps to ${thread.seat ?? 'no seat'}, but a header addresses ${clean(header.to)}`);
|
||||
+ }
|
||||
+ }
|
||||
+ if (!inRange(m.created_at, range)) continue;
|
||||
+ const kind = messageKind(m.text), seat = thread.seat ?? UNMAPPED;
|
||||
+ rows.get(seat)[kind]++;
|
||||
+ if (kind === 'human') human.push(m.message_id);
|
||||
+ for (const number of issueNumbers(m.text)) {
|
||||
+ if (!mentions.has(number)) mentions.set(number, new Set());
|
||||
+ mentions.get(number).add(seat);
|
||||
+ }
|
||||
+ }
|
||||
+ const byMessage = origins(db, projectId);
|
||||
+ const sentThroughApi = byMessage === null ? UNKNOWN : human.filter(id => byMessage.get(id) === false).length;
|
||||
+ const noEvent = byMessage === null ? UNKNOWN : human.filter(id => !byMessage.has(id)).length;
|
||||
+ const listed = seat => [...threads.values()].filter(t => (t.seat ?? UNMAPPED) === seat)
|
||||
+ .sort((a, b) => a.id.localeCompare(b.id)).map(({ id, title, archived }) => ({ id, title, archived }));
|
||||
+ const seatRows = seats.map(seat => ({ seat, ...rows.get(seat), threads: listed(seat) })).filter(r => r.threads.length);
|
||||
+ return { rows, mentions, excluded, seats: seatRows, unmapped: { ...rows.get(UNMAPPED), threads: listed(UNMAPPED) },
|
||||
+ diagnostic: { humanSentThroughApi: sentThroughApi, humanWithoutEvent: noEvent } };
|
||||
+}
|
||||
+
|
||||
+// Reads one snapshot of T3's database. Returns the per-seat rows and issue
|
||||
+// mentions the ledger merges with Pi, and the report's `t3` section.
|
||||
+export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault = true } = {}) {
|
||||
+ dbPath = path.resolve(dbPath);
|
||||
+ await checkPaths(dbPath, isDefault);
|
||||
+ const seats = await readSeats(root);
|
||||
+ const url = pathToFileURL(dbPath);
|
||||
+ url.searchParams.set('mode', 'ro');
|
||||
+ let db, result;
|
||||
+ try {
|
||||
+ db = new DatabaseSync(url, { readOnly: true, timeout: 5000 });
|
||||
+ db.exec('BEGIN');
|
||||
+ result = query(db, root, range, seats);
|
||||
+ db.exec('COMMIT');
|
||||
+ } catch (error) {
|
||||
+ if (error instanceof SourceError) throw error;
|
||||
+ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
|
||||
+ } finally {
|
||||
+ try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
|
||||
+ try { db?.close(); } catch { /* nothing was written */ }
|
||||
+ }
|
||||
+ const { rows, mentions, ...section } = result;
|
||||
+ return { rows, mentions, section: { read: true, database: { path: dbPath, default: isDefault }, ...section } };
|
||||
+}
|
||||
@@ -1,3 +0,0 @@
|
||||
5acbc1075a5d0ad709faf14698235c8c2332c408cb4fccc580bd4a75e2c314fb packages/ledger/src/t3.mjs
|
||||
6546dbaf59c046d599a9d378c1a1f2d9afc9487189db06f50fa3201c9cadc63b packages/ledger/tests/ledger.test.mjs
|
||||
101013def3b168ae1b7e291ff86200b49ed6b27927787585d5ca32a283bf38bd packages/ledger/README.md
|
||||
@@ -1,67 +0,0 @@
|
||||
# Gate F follow-up: Filbert's notes 1 to 3 (#1506), candidate for review
|
||||
|
||||
Darkwing, 2026-09-26. Filbert's build review
|
||||
(`agents/filbert/work/ledger-t3-build-review-2026-09-26.md`, e47ec6da) left
|
||||
four nonblocking notes on Gate F (136958c9). Sage asked for 1 to 3 as one small
|
||||
change that Filbert reviews and Sage commits. Note 4, snapshot isolation, went
|
||||
to DEFERRED (a68dc174). Base is HEAD a4d38a3d, which changes nothing under
|
||||
`packages/ledger` since 136958c9. Nothing is committed or pushed.
|
||||
|
||||
`followup-manifest.sha256` pins the three files. `followup.patch` is the diff
|
||||
against a4d38a3d.
|
||||
|
||||
## Changes
|
||||
|
||||
1. **U+2029.** The splitter test now writes a Pi entry holding a raw U+2028
|
||||
and a raw U+2029, with CRLF endings, and asserts that the file contains
|
||||
both. The README line names both characters. `ledger.mjs` is unchanged,
|
||||
because the splitter already ends lines at `\n` only.
|
||||
2. **Diagnostic.** Three new tests:
|
||||
- A human message with no `thread.message-sent` event gives
|
||||
`humanWithoutEvent: 1`.
|
||||
- A `thread.message-sent` event whose payload doesn't parse makes both
|
||||
diagnostic fields `unknown` and leaves `seats` unchanged.
|
||||
- The same for an event whose `messageId` isn't a string. Filbert didn't
|
||||
list this one, but it's the third `return null` in `origins()` and had
|
||||
no test either.
|
||||
3. **Rethrow.** `readT3`'s catch now rethrows anything that is not a
|
||||
`SourceError` and carries no numeric `errcode`. The CLI prints such an
|
||||
error as `Ledger failed: cannot read source evidence`, exit 1. That's the
|
||||
CLI's existing message for a non-source error, and it no longer points
|
||||
at SQLite or `--no-t3`. With only numeric errcodes left, the message's
|
||||
`?? 'error'` fallback could no longer fire, so I removed it. The new test
|
||||
calls `readT3` in process with an explicit fixture path and a `null`
|
||||
range, so `inRange` throws a `TypeError` inside the read transaction. It
|
||||
asserts the `TypeError` comes out. It never touches the real `~/.t3`, and
|
||||
the fixture comment says so.
|
||||
|
||||
## Evidence
|
||||
|
||||
- Ledger tests: 51/51, the Gate F 47 plus 4 new.
|
||||
- Mutations on a scratch copy of the package. The three `gitea-helper` tests
|
||||
fail in every scratch copy, as before, so the counts leave them out:
|
||||
|
||||
| Mutation | Result |
|
||||
|---|---|
|
||||
| `humanWithoutEvent` hardcoded to 0 | 1 fails (no-event test) |
|
||||
| unparseable event skipped (`continue`) | 1 fails (unparseable test) |
|
||||
| non-string `messageId` skipped | 1 fails (messageId test) |
|
||||
| rethrow removed (Gate F catch) | 1 fails (rethrow test) |
|
||||
| splitter also splits at U+2028 | 1 fails (splitter test) |
|
||||
| splitter also splits at U+2029 | 1 fails (splitter test) |
|
||||
|
||||
My first try at the last two put a raw U+2028 or U+2029 in the regex
|
||||
source. That ends a JS regex literal, so the whole test file failed to
|
||||
load, which doesn't count as a kill. I reran with the escape written out
|
||||
literally, and the rows above come from that rerun.
|
||||
- Eight suites on a local clone of a4d38a3d with the three files: config 24,
|
||||
task 90, foundation 43, conductor 17, release 14, auth 15, discord 63,
|
||||
extension-package 18. I ran them twice, and the second run was on the final
|
||||
files after the errcode edit.
|
||||
- Union on the same clone. Control-board, webui, seat, mosaic, ledger and
|
||||
discord, plus conversation, which CHAT-02 committed: 474/474 twice before
|
||||
the errcode edit and once after. No `ledger-*` temp directories remained.
|
||||
- Live read, `--since 2026-09-01 --until 2026-09-26 --no-issues --json`, at
|
||||
2026-09-26T21:47Z: exit 0, no header conflict, diagnostic
|
||||
`{humanSentThroughApi: 15, humanWithoutEvent: 0}`, two imported threads
|
||||
excluded. The Gate F build read 14; messages have been sent since then.
|
||||
@@ -1,99 +0,0 @@
|
||||
diff --git a/packages/ledger/README.md b/packages/ledger/README.md
|
||||
index 393e9c37..3a2ce27c 100644
|
||||
--- a/packages/ledger/README.md
|
||||
+++ b/packages/ledger/README.md
|
||||
@@ -39,7 +39,8 @@ No install, build, service restart, or configuration change is needed.
|
||||
duplicated entries in copied logs are not deduplicated. No transcript content
|
||||
leaves the parser. Assistant messages and logs outside repo seats do not count.
|
||||
Symlink source directories are refused and symlink files are not followed.
|
||||
- A line ends at `\n` only. A U+2028 inside a JSON string does not split a record.
|
||||
+ A line ends at `\n` only. A U+2028 or U+2029 inside a JSON string does not
|
||||
+ split a record.
|
||||
- Table 2 also counts T3 thread messages with role `user`. The T3 source
|
||||
follows. A seat's row sums its Pi and T3 counts; the JSON keeps the split in
|
||||
`pi` (Pi rows) and `t3.seats` (T3 rows).
|
||||
diff --git a/packages/ledger/src/t3.mjs b/packages/ledger/src/t3.mjs
|
||||
index 9672fcc9..fc9da14e 100644
|
||||
--- a/packages/ledger/src/t3.mjs
|
||||
+++ b/packages/ledger/src/t3.mjs
|
||||
@@ -140,8 +140,10 @@ export async function readT3(root, range, { dbPath = defaultT3Path(), isDefault
|
||||
result = query(db, root, range, seats);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
- if (error instanceof SourceError) throw error;
|
||||
- throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode ?? 'error'}); ${SKIP}`);
|
||||
+ // Only a SQLite failure carries an errcode. Anything else is a bug and
|
||||
+ // surfaces as itself, not as a database problem.
|
||||
+ if (error instanceof SourceError || typeof error?.errcode !== 'number') throw error;
|
||||
+ throw new SourceError(`T3 database cannot be read: ${dbPath} (SQLite ${error.errcode}); ${SKIP}`);
|
||||
} finally {
|
||||
try { if (db?.isTransaction) db.exec('ROLLBACK'); } catch { /* the close below still runs */ }
|
||||
try { db?.close(); } catch { /* nothing was written */ }
|
||||
diff --git a/packages/ledger/tests/ledger.test.mjs b/packages/ledger/tests/ledger.test.mjs
|
||||
index 8e60b96b..834dbd55 100644
|
||||
--- a/packages/ledger/tests/ledger.test.mjs
|
||||
+++ b/packages/ledger/tests/ledger.test.mjs
|
||||
@@ -7,6 +7,7 @@ import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { execFileSync, spawnSync } from 'node:child_process';
|
||||
import { dateRange, messageKind, issueNumbers, totalsLine, summarize } from '../src/ledger.mjs';
|
||||
+import { readT3 } from '../src/t3.mjs';
|
||||
|
||||
const source = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../src');
|
||||
const range = dateRange('2026-09-06', '2026-09-12');
|
||||
@@ -49,7 +50,8 @@ const fixtureIssues = [
|
||||
function fixture(t) {
|
||||
const root = mkdtempSync(path.join(os.tmpdir(), 'ledger-test-'));
|
||||
// No test opens the real ~/.t3: every CLI run gets this HOME, with an empty
|
||||
- // T3 database at the default path. The CLI's root is a realpath.
|
||||
+ // T3 database at the default path. The one in-process readT3 call passes an
|
||||
+ // explicit fixture path. The CLI's root is a realpath.
|
||||
const home = mkdtempSync(path.join(os.tmpdir(), 'ledger-home-'));
|
||||
t.after(() => { rmSync(root, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true }); });
|
||||
const defaultDb = path.join(home, '.t3/userdata/state.sqlite');
|
||||
@@ -134,10 +136,11 @@ test('partial or malformed session log refuses with location, not content', t =>
|
||||
const f = fixture(t); f.put('.pi/state/alice/sessions/bad.jsonl', '{sensitive'); const r = f.run();
|
||||
assert.equal(r.status, 1); assert.match(r.stderr, /Malformed session JSON: alice\/bad.jsonl:1/); assert.doesNotMatch(r.stderr, /sensitive/);
|
||||
});
|
||||
-test('a U+2028 inside a session string is one line, not a malformed record', t => {
|
||||
+test('a U+2028 or U+2029 inside a session string is one line, not a malformed record', t => {
|
||||
const f = fixture(t);
|
||||
- f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
||||
- assert.ok(readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8').includes('\u2028'));
|
||||
+ f.put('.pi/state/bob/sessions/sep.jsonl', [f.entry('Jason: one\u2028two\u2029three #2'), f.entry('[h:alice -> h:bob] ok')].map(x => JSON.stringify(x)).join('\r\n') + '\r\n');
|
||||
+ const written = readFileSync(path.join(f.root, '.pi/state/bob/sessions/sep.jsonl'), 'utf8');
|
||||
+ assert.ok(written.includes('\u2028') && written.includes('\u2029'));
|
||||
const r = f.run(['--json']); assert.equal(r.status, 0, r.stderr);
|
||||
assert.deepEqual(JSON.parse(r.stdout).seats[1], { seat: 'bob', board: 0, agent: 1, human: 1 });
|
||||
});
|
||||
@@ -334,6 +337,30 @@ test('T3: a missing orchestration_events makes the diagnostic unknown and keeps
|
||||
assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
||||
assert.deepEqual(after.seats, before.seats); assert.deepEqual(after.totals, before.totals);
|
||||
});
|
||||
+test('T3: a human message with no event counts in humanWithoutEvent', t => {
|
||||
+ const f = t3Fixture(t);
|
||||
+ f.write({ messages: [...f.messages, { thread: T1, text: 'typed, no event', origin: 'none' }] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ assert.deepEqual(JSON.parse(r.stdout).t3.diagnostic, { humanSentThroughApi: 1, humanWithoutEvent: 1 });
|
||||
+});
|
||||
+const badEvent = payload => `insert into orchestration_events (stream_id, event_type, payload_json, metadata_json) values ('${T1}', 'thread.message-sent', '${payload}', '{}')`;
|
||||
+for (const [name, payload] of [['an unparseable event', '{bad'], ['an event with no string messageId', '{"messageId":7}']]) {
|
||||
+ test(`T3: ${name} makes the diagnostic unknown and keeps the counts`, t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ const before = JSON.parse(f.run(['--json', '--t3-db', f.db]).stdout);
|
||||
+ f.write({ after: [badEvent(payload)] });
|
||||
+ const r = f.run(['--json', '--t3-db', f.db]); assert.equal(r.status, 0, r.stderr);
|
||||
+ const after = JSON.parse(r.stdout);
|
||||
+ assert.deepEqual(after.t3.diagnostic, { humanSentThroughApi: 'unknown', humanWithoutEvent: 'unknown' });
|
||||
+ assert.deepEqual(after.seats, before.seats);
|
||||
+ });
|
||||
+}
|
||||
+test('T3: an error that is not from SQLite is rethrown, not reported as a database failure', async t => {
|
||||
+ const f = t3Fixture(t); f.write();
|
||||
+ // In process with an explicit path, so the real ~/.t3 stays closed. A null
|
||||
+ // range makes inRange throw a TypeError inside the read transaction.
|
||||
+ await assert.rejects(readT3(f.real, null, { dbPath: f.db, isDefault: false }), TypeError);
|
||||
+});
|
||||
for (const link of ['.t3', '.t3/userdata', '.t3/userdata/state.sqlite']) test(`T3: a symlink at ~/${link} exits 1`, t => {
|
||||
const f = fixture(t), target = path.join(f.home, 'real', link);
|
||||
mkdirSync(path.dirname(target), { recursive: true });
|
||||
@@ -1,5 +0,0 @@
|
||||
08959a05574264e4f8243a90af94746e73a2fde3706f22e38f4ff1105b7a45a8 agents/darkwing/work/ledger-t3-source/r1.md
|
||||
e8300cb6abea70819aba7cf10040d19b4d6019b5663c37203209537a5f10ee62 agents/darkwing/work/ledger-t3-source/r2.md
|
||||
f3c05c1b4d28a419ab817621e15708b147f71dff588664980e22696eaafbc342 docs/plans/2026-09-26_ledger-t3-source.md
|
||||
aa4740ae5d045aa12971af5de36a5107805bd31da839aae4e4d398a818d471fc agents/darkwing/work/ledger-t3-source/r1-to-r2.diff
|
||||
4128375121673e0a49ef6789390503ff7395ba59f87bfc450764ea56b536c5b2 agents/darkwing/work/ledger-t3-source/r2-to-r3.diff
|
||||
@@ -1,419 +0,0 @@
|
||||
--- r1.md
|
||||
+++ docs/plans/2026-09-26_ledger-t3-source.md
|
||||
@@ -1,8 +1,11 @@
|
||||
# Ledger: a read-only T3 thread source for Table 2 (Gate F brief)
|
||||
|
||||
-Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request. Filbert
|
||||
-reviews it, and Jason sees it on the decision sheet before anyone builds it.
|
||||
-Issue #1506.
|
||||
+Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue
|
||||
+#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for
|
||||
+revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256
|
||||
+19dda29a. This is R2. It takes every finding, and it records Sage's rulings
|
||||
+on the three open questions. Section 1 has one measurement that differs from
|
||||
+the review.
|
||||
|
||||
## Why
|
||||
|
||||
@@ -18,8 +21,8 @@
|
||||
## Where T3 keeps messages
|
||||
|
||||
T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in
|
||||
-WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). Three
|
||||
-projection tables are enough:
|
||||
+WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). The counts
|
||||
+need three projection tables:
|
||||
|
||||
- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`.
|
||||
- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`,
|
||||
@@ -27,51 +30,72 @@
|
||||
- `projection_thread_messages`: `message_id` (primary key), `thread_id`,
|
||||
`role` (`user` or `assistant`), `text`, `created_at` (ISO UTC).
|
||||
|
||||
-One more table is optional. In `orchestration_events`, each
|
||||
+The JSON diagnostic reads one more. In `orchestration_events`, each
|
||||
`thread.message-sent` event carries `metadata_json.origin`. Messages typed in
|
||||
the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP
|
||||
-tools, which is how seats talk to each other, don't. See the cross-check below.
|
||||
+tools, which is how seats talk to each other, don't.
|
||||
|
||||
The same directory also holds `secrets/`, `clerk-tokens.json` and other
|
||||
settings files. The reader opens `state.sqlite` and nothing else, and it
|
||||
selects named columns only, never `*`.
|
||||
|
||||
-## Reading it, with T3 running or not
|
||||
+## 1. Reading it, with T3 running or not
|
||||
|
||||
-The file stays on disk whether T3 runs or not. The reader opens it with Node's
|
||||
-built-in `node:sqlite` (`DatabaseSync`, `file:<path>?mode=ro`, `readOnly:
|
||||
-true`). That needs no dependency, and Node 26.8.1 prints no warning for it. I
|
||||
-read the live database this way today, while T3 was running, with no errors
|
||||
-and no locks. A WAL reader sees every committed message, including those still
|
||||
-in the `-wal` file.
|
||||
-
|
||||
-Two rules:
|
||||
-- Never open with `immutable=1` and never copy the file. Both skip the WAL
|
||||
- and silently lose the newest messages. A copy of the three files is also
|
||||
- not atomic.
|
||||
-- If T3 stopped uncleanly and left a `-wal` without its `-shm`, a read-only
|
||||
- connection may be unable to rebuild the index. If the open fails, the
|
||||
- ledger reports it and refuses. I have not tested this case or the fully
|
||||
- stopped case. Both are acceptance checks below.
|
||||
+The reader uses Node's built-in `node:sqlite` (`DatabaseSync`). That needs no
|
||||
+dependency, and Node 26.8.1 (SQLite 3.53.4) prints no warning for it.
|
||||
|
||||
-## Jason or agent
|
||||
+- **URI.** Build it with `pathToFileURL(dbPath)` and set `mode=ro` through
|
||||
+ `searchParams`, then pass `readOnly: true`. A `?`, `#` or `%` in the home
|
||||
+ path would break a string-built URI.
|
||||
+- **One snapshot.** Run every query, from the schema checks through the
|
||||
+ diagnostic, inside one `BEGIN` … `COMMIT`. In autocommit mode each
|
||||
+ statement sees its own snapshot while T3 writes between them.
|
||||
+- **Busy timeout.** Set `DatabaseSync`'s `timeout` to 5 s. A transient
|
||||
+ `SQLITE_BUSY` during a T3 checkpoint then waits instead of failing. A busy
|
||||
+ error after the timeout exits 1 like any open failure.
|
||||
+- **No `immutable=1` and no copy.** Both lose the WAL. Filbert found worse
|
||||
+ than lost messages: with a table created inside the WAL, `immutable=1`
|
||||
+ fails with `no such table`.
|
||||
+
|
||||
+What happens on disk. Filbert and I both tested these in scratch
|
||||
+directories:
|
||||
+
|
||||
+| State | Directory writable | Result |
|
||||
+|---|---|---|
|
||||
+| T3 running, writer attached, newest rows only in `-wal` | yes | reads them |
|
||||
+| `-wal` without `-shm` (writer killed, `-shm` removed) | yes | reads the WAL rows and creates `-shm` |
|
||||
+| `-wal` without `-shm` | no | open fails, SQLite 14 |
|
||||
+| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` |
|
||||
+| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" |
|
||||
+
|
||||
+In every case the main file's bytes stayed the same. The last row is where
|
||||
+Filbert and I differ. His review says the stopped-case read works with the
|
||||
+directory read-only. In my run it failed with and without the read
|
||||
+transaction. The build's test settles it. Either way a failed open is exit 1.
|
||||
+
|
||||
+So the accurate claim: the reader never writes the main database file. Like
|
||||
+any SQLite connection, it may create or update `-wal` and `-shm` beside it
|
||||
+and takes read locks in `-shm`. T3 opens normally afterwards.
|
||||
+
|
||||
+## 2. Jason or agent
|
||||
|
||||
Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux
|
||||
preamble counts as agent, `control-board` as the sender counts as board, and
|
||||
anything else counts as human. Messages with role `user` count; assistant
|
||||
messages don't.
|
||||
|
||||
-6a has a defect this source would expose. Its regex allows only a lowercase
|
||||
-class (`class=[a-z-]+`). Seats send uppercase classes: Sage's DECISION, INFO,
|
||||
-REVIEW-REQUEST and REVIEW-NOTE, and my own REVIEW-REQUEST. In this project's
|
||||
-threads, 16 real agent headers fail on that alone and would count as human.
|
||||
-The fix is to make the class match case-insensitive. It belongs in this build
|
||||
-or just before it, reviewed with it. The ms-communications table lists
|
||||
-lowercase names, so the fix follows what seats send, not the table.
|
||||
+The class fix rides in this build (Sage's ruling). HEAD's
|
||||
+`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both allow only
|
||||
+`class=[a-z-]+`. Both become case-insensitive. Seats send uppercase classes:
|
||||
+Sage's DECISION, INFO, REVIEW-REQUEST and REVIEW-NOTE, and my own
|
||||
+REVIEW-REQUEST. In this project's threads 16 real agent headers failed on
|
||||
+that alone at 20:54Z. The ms-communications table lists lowercase names, so
|
||||
+the fix follows what seats send, not the table.
|
||||
|
||||
Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209
|
||||
user messages outside imported and deleted threads, every one with its
|
||||
-`thread.message-sent` event):
|
||||
+`thread.message-sent` event). Filbert's later read agreed, plus messages sent
|
||||
+since.
|
||||
|
||||
| T3 origin | Header matches 6a | Count |
|
||||
|---|---|---|
|
||||
@@ -83,110 +107,193 @@
|
||||
No message typed in the app carries a header, and every API message in this
|
||||
project carries one of the three forms. The 14 free-text ones are older
|
||||
Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to:
|
||||
-Discord Bot (…)]`, written before the guide fixed the format. With the class
|
||||
-fix they still count as human. That's 14 wrong human counts, all dated
|
||||
-2026-09-17 to 2026-09-22.
|
||||
-
|
||||
-Recommendation: the header rule decides, as Sage asked. The reader also
|
||||
-reports one diagnostic number, not used in any table: user messages the rule
|
||||
-calls human that T3 recorded as sent through the API. That count is how the
|
||||
-uppercase-class bug showed up, and it would catch the next format drift. The
|
||||
-origin field is T3's internal metadata, not a documented contract, so it
|
||||
-shouldn't decide anything. I'd make it JSON only, so Table 2's layout stays
|
||||
-the same.
|
||||
-
|
||||
-## Thread to seat
|
||||
-
|
||||
-A thread counts for this checkout only if its project's `workspace_root` is
|
||||
-the ledger's repository root. That is `/mnt/storage/src/mosaic-stack`, project
|
||||
-`34050c07`.
|
||||
-
|
||||
-Thread IDs change whenever Jason starts a new thread for a seat, so there's no
|
||||
-fixed map. T3-AGENT-COMMS.md already names threads after the seat ("Darkwing",
|
||||
-"Sage", "Dewey in Claude"). Proposed rule: a thread belongs to seat `<s>` when
|
||||
-`<s>` is a real directory under `agents/` and the lower-cased title equals
|
||||
-`<s>` or starts with `<s>` followed by a space. Several threads can map to one
|
||||
-seat. Their counts add up, as several Pi session files already do.
|
||||
+Discord Bot (…)]`, written before the guide fixed the format. Sage ruled they
|
||||
+stay as recorded: they count as human, dated 2026-09-17 to 2026-09-22.
|
||||
+
|
||||
+The header rule decides. The JSON also carries one diagnostic that feeds no
|
||||
+table or total: user messages the rule calls human that T3 recorded as sent
|
||||
+through the API. That number exposed the class bug and would catch the next
|
||||
+format drift. `origin` is T3's internal metadata, not a documented contract,
|
||||
+so it decides nothing. If `orchestration_events` or a column it needs is
|
||||
+missing, the diagnostic reads `unknown` and the report goes on (Sage's
|
||||
+ruling on F5). Missing tables the counts depend on still exit 1.
|
||||
+
|
||||
+## 3. Thread to seat
|
||||
+
|
||||
+**Project.** A thread counts for this checkout only if its project's
|
||||
+`workspace_root` equals the ledger's repository root, byte for byte. The CLI
|
||||
+already takes that root from the realpath of its own URL, today
|
||||
+`/mnt/storage/src/mosaic-stack`, project `34050c07`. So a T3 project opened
|
||||
+through the compatibility symlink `~/src/mosaic-stack-dev-test` doesn't
|
||||
+match, and "no project row" is the right refusal. The README says so.
|
||||
+
|
||||
+**Title rule.** Thread IDs change whenever Jason starts a new thread for a
|
||||
+seat, so there's no fixed map. T3-AGENT-COMMS.md already names threads after
|
||||
+the seat ("Darkwing", "Sage", "Dewey in Claude"). A thread belongs to seat
|
||||
+`<s>` when `<s>` is a real directory under `agents/` and the lower-cased
|
||||
+title equals `<s>` or starts with `<s>` followed by a space. So "Sagebrush"
|
||||
+stays unmapped. Several threads can map to one seat, and their counts add
|
||||
+up, as several Pi session files already do.
|
||||
|
||||
Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each,
|
||||
-created 2026-09-26), plus "Darkwing in Claude" (archived) and "Dewey in
|
||||
-Claude". Three threads map to no seat. Two are imported and excluded anyway
|
||||
-("FINDINGS.md review" and "[dragon-lin:darkwing -> …"). The third is
|
||||
-"Discord Bot" with 68 user messages: 54 without a header, and the 14
|
||||
-free-text headers above. The guide's own advice, titles like `review:
|
||||
-<topic>`, will produce more unmapped threads.
|
||||
-
|
||||
-Unmapped threads go in one Table 2 row, `t3:unmapped`, so Jason's messages
|
||||
-there still count toward the Human column and the human-per-closed ratio. The
|
||||
-other choice is to drop them, which would hide those 54 headerless prompts.
|
||||
-That is Jason's decision. I recommend the row.
|
||||
-
|
||||
-A seat's row sums its Pi and T3 counts. JSON splits them by source. Nothing is
|
||||
-counted twice: every T3 session today runs on `claudeAgent` or `codex`, which
|
||||
-don't write `.pi/state`, and Filbert found no T3 header in any Pi log.
|
||||
+created 2026-09-26, titles set by hand), plus "Darkwing in Claude" (archived)
|
||||
+and "Dewey in Claude". Researcher has a directory and no thread. Three
|
||||
+threads map to no seat. Two are imported and excluded anyway ("FINDINGS.md
|
||||
+review" and "[dragon-lin:darkwing -> …"). The third is "Discord Bot" with 68
|
||||
+user messages: 54 without a header, and the 14 free-text headers.
|
||||
+
|
||||
+Titles are current state, and T3 can write them itself. They go wrong three
|
||||
+ways. T3 auto-titles an unnamed thread from Jason's first prompt, so "Rocko
|
||||
+review of the plan" maps to rocko. A rename moves the whole history to
|
||||
+another row. A seat thread titled for a topic drops into `t3:unmapped`.
|
||||
+None of this changes the Human total or the human-per-closed ratio. It only
|
||||
+moves counts between rows, but Gate F reads one seat's row.
|
||||
+
|
||||
+**Header cross-check.** The headers already say which seat a thread belongs
|
||||
+to. For every user message whose header matches the fixed 6a rule and whose
|
||||
+`to:` id equals the message's own `thread_id`:
|
||||
+- in a mapped thread, the `to:` role, lower-cased, must equal that thread's
|
||||
+ seat;
|
||||
+- in an unmapped thread, the `to:` role must not be a seat name.
|
||||
+
|
||||
+A conflict exits 1 and names the thread id, its title and both roles. A
|
||||
+header whose `to:` id is some other thread is not checked. The check reads
|
||||
+message text only, not T3 metadata. In a live read at 21:02Z every header
|
||||
+agreed: all 104 addressed to their own thread carried the full thread id and
|
||||
+named that thread's seat (Sage 40, Darkwing 15, Filbert 18, Dewey 15, Rocko
|
||||
+16).
|
||||
+
|
||||
+It catches a seat thread renamed to another seat or to a topic, once any
|
||||
+agent writes to it. It also catches an auto-titled thread that agents
|
||||
+address by a different seat. It misses a thread no agent ever writes to.
|
||||
+Such a thread can only add human counts to a seat's row, never hide them, so
|
||||
+for Gate F it errs toward a visible failure. The README says so.
|
||||
+
|
||||
+**Unmapped row.** Unmapped threads go in one Table 2 row, `t3:unmapped`
|
||||
+(Sage's ruling), so their human messages still reach the Human column and
|
||||
+the human-per-closed ratio.
|
||||
+
|
||||
+**Mapping in the JSON.** For each seat, the T3 thread ids and titles that
|
||||
+made its row, and the unmapped thread ids and titles. Anyone checking a Gate
|
||||
+F result can then see which threads the row came from.
|
||||
+
|
||||
+A seat's row sums its Pi and T3 counts, and the JSON splits them by source.
|
||||
+Nothing is counted twice. Every T3 session today runs on `claudeAgent` or
|
||||
+`codex`, which don't write `.pi/state`, and Filbert found no T3 header in any
|
||||
+Pi log (6a record).
|
||||
|
||||
-Excluded, with the reason stated in the README:
|
||||
+**Excluded,** with the reason stated in the README:
|
||||
- Imported threads (`thread_id` starting `import:`, events marked
|
||||
`historyImport`). They are partial copies of Claude Code sessions, not T3
|
||||
traffic: 55 user messages in two threads here.
|
||||
- Deleted threads (`deleted_at` set). Across all projects there are 3, with
|
||||
3 messages. Archived threads count.
|
||||
-
|
||||
-## What fails closed
|
||||
-
|
||||
-With the T3 source on, each of these refuses the report with exit 1, the
|
||||
-code the ledger already uses for unreadable session evidence. The report
|
||||
-never falls back to Pi logs alone. As with `--no-issues`, `--no-t3` turns the
|
||||
-source off, and the report then says T3 was not read.
|
||||
-- The database is missing, unreadable, or won't open read-only (including
|
||||
- the `-wal` without `-shm` case). This differs from the Pi reader, which
|
||||
- treats a missing `.pi` as no messages. A missing Pi directory means no Pi
|
||||
- seats ran here. A missing T3 database on this host means the path or T3
|
||||
- changed, and a silent zero is the failure Gate F exists to prevent.
|
||||
-- A required table or column is missing. The reader checks `PRAGMA
|
||||
+- Threads in other T3 projects. Live, there is a project at `/home/jwoltje`
|
||||
+ and a deleted one at `/mnt/storage/src`. A thread in either could work on
|
||||
+ this repository and would not be counted. The workspace-root rule is still
|
||||
+ the right one, but the README names this blind spot.
|
||||
+
|
||||
+## 4. What fails closed
|
||||
+
|
||||
+The source is on by default (Sage's ruling). `--no-t3` turns it off, and the
|
||||
+report then says T3 was not read. `--t3-db <path>` reads another database
|
||||
+file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and
|
||||
+gets the same checks.
|
||||
+
|
||||
+Each of these refuses the report with exit 1, the code the ledger already
|
||||
+uses for unreadable session evidence. The report never falls back to Pi logs
|
||||
+alone. Where the database is missing or won't open, the message names
|
||||
+`--no-t3`.
|
||||
+- The database is missing or unreadable, or won't open read-only. That
|
||||
+ includes a directory that isn't writable when SQLite needs to create
|
||||
+ `-shm`, and a busy error after the timeout. The Pi reader treats a missing
|
||||
+ `.pi` as no messages, and this departs from it on purpose. A missing Pi
|
||||
+ directory means no Pi seats ran here. A missing T3 database on this host
|
||||
+ means the path or T3 changed, and a silent zero is the failure Gate F
|
||||
+ exists to prevent.
|
||||
+- `~/.t3`, `~/.t3/userdata` or `state.sqlite` is a symlink. With `--t3-db`,
|
||||
+ the file and its directory are checked. The Pi reader checks every
|
||||
+ ancestor too, but it skips symlinked entries. Skipping one named file
|
||||
+ would be another silent zero, so this reader refuses.
|
||||
+- A table or column the counts need is missing. The reader checks `PRAGMA
|
||||
table_info` and names what's missing. This catches a T3 upgrade that
|
||||
changes the schema.
|
||||
- No project row, or more than one non-deleted row, for this repository root.
|
||||
- A counted row has a bad `role`, non-string `text`, or a `created_at` that
|
||||
doesn't parse. The Pi reader already refuses malformed JSONL and bad
|
||||
timestamps the same way.
|
||||
-- `state.sqlite` or `~/.t3/userdata` is a symlink. The Pi reader skips
|
||||
- symlinked entries instead. For one named file, skipping would be another
|
||||
- silent zero, so this reader refuses.
|
||||
-
|
||||
-The source never writes to the database. It never reads other files in
|
||||
-`~/.t3`, and it passes no message text beyond `messageKind` and
|
||||
-`issueNumbers`, the same rule as for Pi logs. The one outside effect is
|
||||
-SQLite's own: a WAL reader takes read locks in the `-shm` file, as T3's own
|
||||
-connections do.
|
||||
-
|
||||
-## Decisions for Jason
|
||||
-
|
||||
-1. The source is on by default, with `--no-t3` to turn it off. The other
|
||||
- choice is off by default with `--t3` to turn it on. I recommend on by
|
||||
- default, because Gate F exists to count these messages.
|
||||
-2. Unmapped threads get a `t3:unmapped` row. The other choice is to drop
|
||||
- them. I recommend the row.
|
||||
-3. The 14 free-text headers from 09-17 to 09-22 stay counted as human. Fixing
|
||||
- them would mean loosening the header grammar for history only, and I don't
|
||||
- recommend it.
|
||||
-
|
||||
-## Acceptance for the build
|
||||
-
|
||||
-- Fixture databases built with `node:sqlite` in a temp dir, in WAL mode:
|
||||
- seat threads and an unmapped thread; imported, deleted and archived
|
||||
- threads; all three header forms, uppercase classes included; a message
|
||||
- outside the date range; another project with the same seat titles.
|
||||
-- Each fail-closed case above has its own test, including a schema column
|
||||
- removed and `-wal` without `-shm`. One test opens a database whose newest
|
||||
- message is still in the WAL and counts it. Another reads a database closed
|
||||
- cleanly with no writer attached, which is the T3-stopped case.
|
||||
-- The class fix is proven against HEAD's `messageKind`: an uppercase class
|
||||
- counts as agent after the fix and as human before it.
|
||||
+- A header conflicts with the title mapping (section 3).
|
||||
+
|
||||
+The reader never reads other files in `~/.t3`. It passes no message text
|
||||
+beyond `messageKind`, `issueNumbers` and the header's `to:` role and id, the
|
||||
+same rule as for Pi logs.
|
||||
+
|
||||
+## 5. Rulings
|
||||
+
|
||||
+Sage ruled on the three questions R1 put to Jason, as lead calls:
|
||||
+1. The source is on by default. A missing or unreadable database exits 1,
|
||||
+ and the message names `--no-t3`.
|
||||
+2. Unmapped threads get the `t3:unmapped` row.
|
||||
+3. The 14 free-text headers stay as recorded. They show only in the JSON
|
||||
+ diagnostic.
|
||||
+
|
||||
+Sage also ruled that the class fix rides in this build, and that a missing
|
||||
+diagnostic table reads `unknown` (F5).
|
||||
+
|
||||
+## 6. Acceptance for the build
|
||||
+
|
||||
+**No test opens the real `~/.t3`.** Both places in
|
||||
+`packages/ledger/tests/ledger.test.mjs` that spawn `cli.mjs` (the shared
|
||||
+`run()` helper and the direct `spawnSync` at line 66) set `HOME` to the
|
||||
+fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then
|
||||
+finds no database and fails closed. The existing tests aren't about T3. Each
|
||||
+gets an empty fixture database at the fixture `HOME`'s default path, with
|
||||
+one project row for the fixture root. So they run with the source on, and
|
||||
+their expected rows don't change. One test asserts that a `HOME` with no
|
||||
+database exits 1 and names `--no-t3`.
|
||||
+
|
||||
+Fixture databases are built with `node:sqlite` in a temp directory, in WAL
|
||||
+mode:
|
||||
+- seat threads and an unmapped thread; imported, deleted and archived
|
||||
+ threads; a message outside the date range;
|
||||
+- all three header forms, with uppercase classes in both the tmux preamble
|
||||
+ and the T3 header;
|
||||
+- a thread with the same seat title in another project;
|
||||
+- a seat thread renamed to another seat, with an agent header to its own
|
||||
+ id, which exits 1;
|
||||
+- a "Sagebrush" title, which stays unmapped;
|
||||
+- a thread titled "Researcher", which maps to the seat that has no thread
|
||||
+ live.
|
||||
+
|
||||
+WAL states, each with its own test:
|
||||
+- The newest message is only in `-wal`, with the writer still attached (the
|
||||
+ live-T3 case). It is counted.
|
||||
+- T3 stopped: the database closed cleanly with no writer. Counts are
|
||||
+ correct, and the main file's bytes are unchanged afterwards.
|
||||
+- `-wal` without `-shm` in a writable directory: made by a child writer with
|
||||
+ `wal_autocheckpoint=0` that is SIGKILLed, then `-shm` deleted. The WAL
|
||||
+ rows are counted.
|
||||
+- `-wal` without `-shm` in a directory that isn't writable: exit 1, naming
|
||||
+ `--no-t3`. Skipped when the tests run as root, where the mode bits don't
|
||||
+ bind.
|
||||
+- The stopped case in a directory that isn't writable: exit 1, naming
|
||||
+ `--no-t3`, as I measured it. If the build reads there instead, the builder
|
||||
+ changes this test to assert correct counts and records the correction in
|
||||
+ the BUILD-LOG entry. Skipped as root too.
|
||||
+
|
||||
+Also:
|
||||
+- Each other fail-closed case in section 4 has its own test, including a
|
||||
+ removed schema column and each symlink.
|
||||
+- A missing `orchestration_events` gives `unknown` for the diagnostic and
|
||||
+ the same counts.
|
||||
+- The class fix is proven against HEAD's `messageKind`. An uppercase class
|
||||
+ in either preamble counts as agent after the fix and as human before it.
|
||||
+- The JSON lists each seat's threads and the unmapped threads.
|
||||
- A read against the live database gives the counts in this brief, allowing
|
||||
- for messages sent since.
|
||||
-- The ledger README's counting rules name the new source, the mapping rule
|
||||
- and the exclusions.
|
||||
+ for messages sent since. It exits 0 with no header conflict.
|
||||
+- The ledger README's counting rules name the new source, both flags, the
|
||||
+ mapping rule and the header check, and the exclusions. That includes the
|
||||
+ symlinked-checkout case and the other-project blind spot.
|
||||
- No suite runs the ledger tests, so the BUILD-LOG entry names the test file.
|
||||
|
||||
## Not in scope
|
||||
@@ -194,5 +301,6 @@
|
||||
- Claude Code transcripts (`~/.claude/projects`) and Codex sessions
|
||||
(`~/.codex/sessions`). T3's database already holds every message T3
|
||||
delivered, so those files would only duplicate it.
|
||||
-- Any write to T3, any T3 API call, or anything that needs T3 running.
|
||||
+- Any T3 API call, anything that needs T3 running, and any write beyond
|
||||
+ SQLite's own `-wal` and `-shm` handling.
|
||||
- Fixing the two tmux misclassifications Filbert found in 6a.
|
||||
@@ -1,198 +0,0 @@
|
||||
# Ledger: a read-only T3 thread source for Table 2 (Gate F brief)
|
||||
|
||||
Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request. Filbert
|
||||
reviews it, and Jason sees it on the decision sheet before anyone builds it.
|
||||
Issue #1506.
|
||||
|
||||
## Why
|
||||
|
||||
Table 2 counts user messages per seat from `.pi/state/<seat>/sessions/*.jsonl`
|
||||
only. Development seats now run in T3 on the Claude and Codex harnesses, so
|
||||
their prompts, Jason's included, never reach a Pi log. Today the Human column
|
||||
can't see T3 at all, and the zero it shows for T3 seats means "no source", not
|
||||
"no human prompts". 6a (ef0020ad) taught `messageKind` the T3 header, but no
|
||||
source the ledger reads contains one. Gate F (QUEUE row 6) passes when
|
||||
Filbert's item closes with zero human messages from Jason. While the ledger
|
||||
can't see T3, a zero there proves nothing.
|
||||
|
||||
## Where T3 keeps messages
|
||||
|
||||
T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in
|
||||
WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). Three
|
||||
projection tables are enough:
|
||||
|
||||
- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`.
|
||||
- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`,
|
||||
`deleted_at`.
|
||||
- `projection_thread_messages`: `message_id` (primary key), `thread_id`,
|
||||
`role` (`user` or `assistant`), `text`, `created_at` (ISO UTC).
|
||||
|
||||
One more table is optional. In `orchestration_events`, each
|
||||
`thread.message-sent` event carries `metadata_json.origin`. Messages typed in
|
||||
the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP
|
||||
tools, which is how seats talk to each other, don't. See the cross-check below.
|
||||
|
||||
The same directory also holds `secrets/`, `clerk-tokens.json` and other
|
||||
settings files. The reader opens `state.sqlite` and nothing else, and it
|
||||
selects named columns only, never `*`.
|
||||
|
||||
## Reading it, with T3 running or not
|
||||
|
||||
The file stays on disk whether T3 runs or not. The reader opens it with Node's
|
||||
built-in `node:sqlite` (`DatabaseSync`, `file:<path>?mode=ro`, `readOnly:
|
||||
true`). That needs no dependency, and Node 26.8.1 prints no warning for it. I
|
||||
read the live database this way today, while T3 was running, with no errors
|
||||
and no locks. A WAL reader sees every committed message, including those still
|
||||
in the `-wal` file.
|
||||
|
||||
Two rules:
|
||||
- Never open with `immutable=1` and never copy the file. Both skip the WAL
|
||||
and silently lose the newest messages. A copy of the three files is also
|
||||
not atomic.
|
||||
- If T3 stopped uncleanly and left a `-wal` without its `-shm`, a read-only
|
||||
connection may be unable to rebuild the index. If the open fails, the
|
||||
ledger reports it and refuses. I have not tested this case or the fully
|
||||
stopped case. Both are acceptance checks below.
|
||||
|
||||
## Jason or agent
|
||||
|
||||
Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux
|
||||
preamble counts as agent, `control-board` as the sender counts as board, and
|
||||
anything else counts as human. Messages with role `user` count; assistant
|
||||
messages don't.
|
||||
|
||||
6a has a defect this source would expose. Its regex allows only a lowercase
|
||||
class (`class=[a-z-]+`). Seats send uppercase classes: Sage's DECISION, INFO,
|
||||
REVIEW-REQUEST and REVIEW-NOTE, and my own REVIEW-REQUEST. In this project's
|
||||
threads, 16 real agent headers fail on that alone and would count as human.
|
||||
The fix is to make the class match case-insensitive. It belongs in this build
|
||||
or just before it, reviewed with it. The ms-communications table lists
|
||||
lowercase names, so the fix follows what seats send, not the table.
|
||||
|
||||
Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209
|
||||
user messages outside imported and deleted threads, every one with its
|
||||
`thread.message-sent` event):
|
||||
|
||||
| T3 origin | Header matches 6a | Count |
|
||||
|---|---|---|
|
||||
| typed in the app (has `appVersion`) | no | 99 |
|
||||
| sent through the API (no `appVersion`) | yes | 80 |
|
||||
| sent through the API | no, uppercase class | 16 |
|
||||
| sent through the API | no, free-text roles | 14 |
|
||||
|
||||
No message typed in the app carries a header, and every API message in this
|
||||
project carries one of the three forms. The 14 free-text ones are older
|
||||
Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to:
|
||||
Discord Bot (…)]`, written before the guide fixed the format. With the class
|
||||
fix they still count as human. That's 14 wrong human counts, all dated
|
||||
2026-09-17 to 2026-09-22.
|
||||
|
||||
Recommendation: the header rule decides, as Sage asked. The reader also
|
||||
reports one diagnostic number, not used in any table: user messages the rule
|
||||
calls human that T3 recorded as sent through the API. That count is how the
|
||||
uppercase-class bug showed up, and it would catch the next format drift. The
|
||||
origin field is T3's internal metadata, not a documented contract, so it
|
||||
shouldn't decide anything. I'd make it JSON only, so Table 2's layout stays
|
||||
the same.
|
||||
|
||||
## Thread to seat
|
||||
|
||||
A thread counts for this checkout only if its project's `workspace_root` is
|
||||
the ledger's repository root. That is `/mnt/storage/src/mosaic-stack`, project
|
||||
`34050c07`.
|
||||
|
||||
Thread IDs change whenever Jason starts a new thread for a seat, so there's no
|
||||
fixed map. T3-AGENT-COMMS.md already names threads after the seat ("Darkwing",
|
||||
"Sage", "Dewey in Claude"). Proposed rule: a thread belongs to seat `<s>` when
|
||||
`<s>` is a real directory under `agents/` and the lower-cased title equals
|
||||
`<s>` or starts with `<s>` followed by a space. Several threads can map to one
|
||||
seat. Their counts add up, as several Pi session files already do.
|
||||
|
||||
Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each,
|
||||
created 2026-09-26), plus "Darkwing in Claude" (archived) and "Dewey in
|
||||
Claude". Three threads map to no seat. Two are imported and excluded anyway
|
||||
("FINDINGS.md review" and "[dragon-lin:darkwing -> …"). The third is
|
||||
"Discord Bot" with 68 user messages: 54 without a header, and the 14
|
||||
free-text headers above. The guide's own advice, titles like `review:
|
||||
<topic>`, will produce more unmapped threads.
|
||||
|
||||
Unmapped threads go in one Table 2 row, `t3:unmapped`, so Jason's messages
|
||||
there still count toward the Human column and the human-per-closed ratio. The
|
||||
other choice is to drop them, which would hide those 54 headerless prompts.
|
||||
That is Jason's decision. I recommend the row.
|
||||
|
||||
A seat's row sums its Pi and T3 counts. JSON splits them by source. Nothing is
|
||||
counted twice: every T3 session today runs on `claudeAgent` or `codex`, which
|
||||
don't write `.pi/state`, and Filbert found no T3 header in any Pi log.
|
||||
|
||||
Excluded, with the reason stated in the README:
|
||||
- Imported threads (`thread_id` starting `import:`, events marked
|
||||
`historyImport`). They are partial copies of Claude Code sessions, not T3
|
||||
traffic: 55 user messages in two threads here.
|
||||
- Deleted threads (`deleted_at` set). Across all projects there are 3, with
|
||||
3 messages. Archived threads count.
|
||||
|
||||
## What fails closed
|
||||
|
||||
With the T3 source on, each of these refuses the report with exit 1, the
|
||||
code the ledger already uses for unreadable session evidence. The report
|
||||
never falls back to Pi logs alone. As with `--no-issues`, `--no-t3` turns the
|
||||
source off, and the report then says T3 was not read.
|
||||
- The database is missing, unreadable, or won't open read-only (including
|
||||
the `-wal` without `-shm` case). This differs from the Pi reader, which
|
||||
treats a missing `.pi` as no messages. A missing Pi directory means no Pi
|
||||
seats ran here. A missing T3 database on this host means the path or T3
|
||||
changed, and a silent zero is the failure Gate F exists to prevent.
|
||||
- A required table or column is missing. The reader checks `PRAGMA
|
||||
table_info` and names what's missing. This catches a T3 upgrade that
|
||||
changes the schema.
|
||||
- No project row, or more than one non-deleted row, for this repository root.
|
||||
- A counted row has a bad `role`, non-string `text`, or a `created_at` that
|
||||
doesn't parse. The Pi reader already refuses malformed JSONL and bad
|
||||
timestamps the same way.
|
||||
- `state.sqlite` or `~/.t3/userdata` is a symlink. The Pi reader skips
|
||||
symlinked entries instead. For one named file, skipping would be another
|
||||
silent zero, so this reader refuses.
|
||||
|
||||
The source never writes to the database. It never reads other files in
|
||||
`~/.t3`, and it passes no message text beyond `messageKind` and
|
||||
`issueNumbers`, the same rule as for Pi logs. The one outside effect is
|
||||
SQLite's own: a WAL reader takes read locks in the `-shm` file, as T3's own
|
||||
connections do.
|
||||
|
||||
## Decisions for Jason
|
||||
|
||||
1. The source is on by default, with `--no-t3` to turn it off. The other
|
||||
choice is off by default with `--t3` to turn it on. I recommend on by
|
||||
default, because Gate F exists to count these messages.
|
||||
2. Unmapped threads get a `t3:unmapped` row. The other choice is to drop
|
||||
them. I recommend the row.
|
||||
3. The 14 free-text headers from 09-17 to 09-22 stay counted as human. Fixing
|
||||
them would mean loosening the header grammar for history only, and I don't
|
||||
recommend it.
|
||||
|
||||
## Acceptance for the build
|
||||
|
||||
- Fixture databases built with `node:sqlite` in a temp dir, in WAL mode:
|
||||
seat threads and an unmapped thread; imported, deleted and archived
|
||||
threads; all three header forms, uppercase classes included; a message
|
||||
outside the date range; another project with the same seat titles.
|
||||
- Each fail-closed case above has its own test, including a schema column
|
||||
removed and `-wal` without `-shm`. One test opens a database whose newest
|
||||
message is still in the WAL and counts it. Another reads a database closed
|
||||
cleanly with no writer attached, which is the T3-stopped case.
|
||||
- The class fix is proven against HEAD's `messageKind`: an uppercase class
|
||||
counts as agent after the fix and as human before it.
|
||||
- A read against the live database gives the counts in this brief, allowing
|
||||
for messages sent since.
|
||||
- The ledger README's counting rules name the new source, the mapping rule
|
||||
and the exclusions.
|
||||
- No suite runs the ledger tests, so the BUILD-LOG entry names the test file.
|
||||
|
||||
## Not in scope
|
||||
|
||||
- Claude Code transcripts (`~/.claude/projects`) and Codex sessions
|
||||
(`~/.codex/sessions`). T3's database already holds every message T3
|
||||
delivered, so those files would only duplicate it.
|
||||
- Any write to T3, any T3 API call, or anything that needs T3 running.
|
||||
- Fixing the two tmux misclassifications Filbert found in 6a.
|
||||
@@ -1,74 +0,0 @@
|
||||
--- r2.md
|
||||
+++ docs/plans/2026-09-26_ledger-t3-source.md
|
||||
@@ -3,9 +3,9 @@
|
||||
Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue
|
||||
#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for
|
||||
revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256
|
||||
-19dda29a. This is R2. It takes every finding, and it records Sage's rulings
|
||||
-on the three open questions. Section 1 has one measurement that differs from
|
||||
-the review.
|
||||
+19dda29a. R2 (sha256 e8300cb6) took every finding and recorded Sage's
|
||||
+rulings on the three open questions. Filbert approved R2 with three nits,
|
||||
+review sha256 bb02d8d3. This is R3, which takes the nits.
|
||||
|
||||
## Why
|
||||
|
||||
@@ -68,10 +68,10 @@
|
||||
| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` |
|
||||
| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" |
|
||||
|
||||
-In every case the main file's bytes stayed the same. The last row is where
|
||||
-Filbert and I differ. His review says the stopped-case read works with the
|
||||
-directory read-only. In my run it failed with and without the read
|
||||
-transaction. The build's test settles it. Either way a failed open is exit 1.
|
||||
+In every case the main file's bytes stayed the same. Filbert's first
|
||||
+review said the last case reads. His test had reused a database whose empty
|
||||
+`-wal` and `-shm` were still present. On a true clean stop he also got 1544,
|
||||
+and his review records the correction. A failed open is exit 1.
|
||||
|
||||
So the accurate claim: the reader never writes the main database file. Like
|
||||
any SQLite connection, it may create or update `-wal` and `-shm` beside it
|
||||
@@ -198,7 +198,9 @@
|
||||
The source is on by default (Sage's ruling). `--no-t3` turns it off, and the
|
||||
report then says T3 was not read. `--t3-db <path>` reads another database
|
||||
file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and
|
||||
-gets the same checks.
|
||||
+gets the same checks. The JSON records the database path read and whether
|
||||
+it was the default. When it wasn't, the text report adds one line naming the
|
||||
+path, so a Gate F result can't come from a fixture unnoticed.
|
||||
|
||||
Each of these refuses the report with exit 1, the code the ledger already
|
||||
uses for unreadable session evidence. The report never falls back to Pi logs
|
||||
@@ -248,7 +250,9 @@
|
||||
fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then
|
||||
finds no database and fails closed. The existing tests aren't about T3. Each
|
||||
gets an empty fixture database at the fixture `HOME`'s default path, with
|
||||
-one project row for the fixture root. So they run with the source on, and
|
||||
+one project row for the fixture root. That row stores
|
||||
+`fs.realpathSync(root)`, because the CLI resolves its root through realpath
|
||||
+and a symlinked temp directory would otherwise not match. So they run with the source on, and
|
||||
their expected rows don't change. One test asserts that a `HOME` with no
|
||||
database exits 1 and names `--no-t3`.
|
||||
|
||||
@@ -277,9 +281,7 @@
|
||||
`--no-t3`. Skipped when the tests run as root, where the mode bits don't
|
||||
bind.
|
||||
- The stopped case in a directory that isn't writable: exit 1, naming
|
||||
- `--no-t3`, as I measured it. If the build reads there instead, the builder
|
||||
- changes this test to assert correct counts and records the correction in
|
||||
- the BUILD-LOG entry. Skipped as root too.
|
||||
+ `--no-t3`. Skipped as root too.
|
||||
|
||||
Also:
|
||||
- Each other fail-closed case in section 4 has its own test, including a
|
||||
@@ -288,7 +290,9 @@
|
||||
the same counts.
|
||||
- The class fix is proven against HEAD's `messageKind`. An uppercase class
|
||||
in either preamble counts as agent after the fix and as human before it.
|
||||
-- The JSON lists each seat's threads and the unmapped threads.
|
||||
+- The JSON lists each seat's threads and the unmapped threads, and the
|
||||
+ database path with whether it was the default. A `--t3-db` run prints the
|
||||
+ path line in the text report, and a default run doesn't.
|
||||
- A read against the live database gives the counts in this brief, allowing
|
||||
for messages sent since. It exits 0 with no header conflict.
|
||||
- The ledger README's counting rules name the new source, both flags, the
|
||||
@@ -1,306 +0,0 @@
|
||||
# Ledger: a read-only T3 thread source for Table 2 (Gate F brief)
|
||||
|
||||
Brief only, no code. Darkwing wrote it on 2026-09-26 at Sage's request, issue
|
||||
#1506. R1 (sha256 08959a05) went to Filbert, whose review asked for
|
||||
revisions: `agents/filbert/work/ledger-t3-source-review-2026-09-26.md`, sha256
|
||||
19dda29a. This is R2. It takes every finding, and it records Sage's rulings
|
||||
on the three open questions. Section 1 has one measurement that differs from
|
||||
the review.
|
||||
|
||||
## Why
|
||||
|
||||
Table 2 counts user messages per seat from `.pi/state/<seat>/sessions/*.jsonl`
|
||||
only. Development seats now run in T3 on the Claude and Codex harnesses, so
|
||||
their prompts, Jason's included, never reach a Pi log. Today the Human column
|
||||
can't see T3 at all, and the zero it shows for T3 seats means "no source", not
|
||||
"no human prompts". 6a (ef0020ad) taught `messageKind` the T3 header, but no
|
||||
source the ledger reads contains one. Gate F (QUEUE row 6) passes when
|
||||
Filbert's item closes with zero human messages from Jason. While the ledger
|
||||
can't see T3, a zero there proves nothing.
|
||||
|
||||
## Where T3 keeps messages
|
||||
|
||||
T3 keeps its state in one SQLite database, `~/.t3/userdata/state.sqlite`, in
|
||||
WAL mode (`state.sqlite-wal` and `state.sqlite-shm` sit beside it). The counts
|
||||
need three projection tables:
|
||||
|
||||
- `projection_projects`: `project_id`, `workspace_root`, `deleted_at`.
|
||||
- `projection_threads`: `thread_id`, `project_id`, `title`, `archived_at`,
|
||||
`deleted_at`.
|
||||
- `projection_thread_messages`: `message_id` (primary key), `thread_id`,
|
||||
`role` (`user` or `assistant`), `text`, `created_at` (ISO UTC).
|
||||
|
||||
The JSON diagnostic reads one more. In `orchestration_events`, each
|
||||
`thread.message-sent` event carries `metadata_json.origin`. Messages typed in
|
||||
the T3 app carry an `appVersion` there. Messages sent through T3's API or MCP
|
||||
tools, which is how seats talk to each other, don't.
|
||||
|
||||
The same directory also holds `secrets/`, `clerk-tokens.json` and other
|
||||
settings files. The reader opens `state.sqlite` and nothing else, and it
|
||||
selects named columns only, never `*`.
|
||||
|
||||
## 1. Reading it, with T3 running or not
|
||||
|
||||
The reader uses Node's built-in `node:sqlite` (`DatabaseSync`). That needs no
|
||||
dependency, and Node 26.8.1 (SQLite 3.53.4) prints no warning for it.
|
||||
|
||||
- **URI.** Build it with `pathToFileURL(dbPath)` and set `mode=ro` through
|
||||
`searchParams`, then pass `readOnly: true`. A `?`, `#` or `%` in the home
|
||||
path would break a string-built URI.
|
||||
- **One snapshot.** Run every query, from the schema checks through the
|
||||
diagnostic, inside one `BEGIN` … `COMMIT`. In autocommit mode each
|
||||
statement sees its own snapshot while T3 writes between them.
|
||||
- **Busy timeout.** Set `DatabaseSync`'s `timeout` to 5 s. A transient
|
||||
`SQLITE_BUSY` during a T3 checkpoint then waits instead of failing. A busy
|
||||
error after the timeout exits 1 like any open failure.
|
||||
- **No `immutable=1` and no copy.** Both lose the WAL. Filbert found worse
|
||||
than lost messages: with a table created inside the WAL, `immutable=1`
|
||||
fails with `no such table`.
|
||||
|
||||
What happens on disk. Filbert and I both tested these in scratch
|
||||
directories:
|
||||
|
||||
| State | Directory writable | Result |
|
||||
|---|---|---|
|
||||
| T3 running, writer attached, newest rows only in `-wal` | yes | reads them |
|
||||
| `-wal` without `-shm` (writer killed, `-shm` removed) | yes | reads the WAL rows and creates `-shm` |
|
||||
| `-wal` without `-shm` | no | open fails, SQLite 14 |
|
||||
| T3 stopped cleanly, no `-wal` or `-shm` | yes | reads, then leaves an empty `-wal` and a 32 KiB `-shm` |
|
||||
| T3 stopped cleanly | no | fails, SQLite 1544 "attempt to write a readonly database" |
|
||||
|
||||
In every case the main file's bytes stayed the same. The last row is where
|
||||
Filbert and I differ. His review says the stopped-case read works with the
|
||||
directory read-only. In my run it failed with and without the read
|
||||
transaction. The build's test settles it. Either way a failed open is exit 1.
|
||||
|
||||
So the accurate claim: the reader never writes the main database file. Like
|
||||
any SQLite connection, it may create or update `-wal` and `-shm` beside it
|
||||
and takes read locks in `-shm`. T3 opens normally afterwards.
|
||||
|
||||
## 2. Jason or agent
|
||||
|
||||
Reuse the 6a rule. The first line of `text` decides: a T3 header or the tmux
|
||||
preamble counts as agent, `control-board` as the sender counts as board, and
|
||||
anything else counts as human. Messages with role `user` count; assistant
|
||||
messages don't.
|
||||
|
||||
The class fix rides in this build (Sage's ruling). HEAD's
|
||||
`packages/ledger/src/ledger.mjs:81` (tmux) and `:83` (T3) both allow only
|
||||
`class=[a-z-]+`. Both become case-insensitive. Seats send uppercase classes:
|
||||
Sage's DECISION, INFO, REVIEW-REQUEST and REVIEW-NOTE, and my own
|
||||
REVIEW-REQUEST. In this project's threads 16 real agent headers failed on
|
||||
that alone at 20:54Z. The ms-communications table lists lowercase names, so
|
||||
the fix follows what seats send, not the table.
|
||||
|
||||
Cross-check, read at 2026-09-26T20:54Z for the mosaic-stack project (209
|
||||
user messages outside imported and deleted threads, every one with its
|
||||
`thread.message-sent` event). Filbert's later read agreed, plus messages sent
|
||||
since.
|
||||
|
||||
| T3 origin | Header matches 6a | Count |
|
||||
|---|---|---|
|
||||
| typed in the app (has `appVersion`) | no | 99 |
|
||||
| sent through the API (no `appVersion`) | yes | 80 |
|
||||
| sent through the API | no, uppercase class | 16 |
|
||||
| sent through the API | no, free-text roles | 14 |
|
||||
|
||||
No message typed in the app carries a header, and every API message in this
|
||||
project carries one of the three forms. The 14 free-text ones are older
|
||||
Discord Bot thread headers such as `[from: SetSpark coordinator (…) -> to:
|
||||
Discord Bot (…)]`, written before the guide fixed the format. Sage ruled they
|
||||
stay as recorded: they count as human, dated 2026-09-17 to 2026-09-22.
|
||||
|
||||
The header rule decides. The JSON also carries one diagnostic that feeds no
|
||||
table or total: user messages the rule calls human that T3 recorded as sent
|
||||
through the API. That number exposed the class bug and would catch the next
|
||||
format drift. `origin` is T3's internal metadata, not a documented contract,
|
||||
so it decides nothing. If `orchestration_events` or a column it needs is
|
||||
missing, the diagnostic reads `unknown` and the report goes on (Sage's
|
||||
ruling on F5). Missing tables the counts depend on still exit 1.
|
||||
|
||||
## 3. Thread to seat
|
||||
|
||||
**Project.** A thread counts for this checkout only if its project's
|
||||
`workspace_root` equals the ledger's repository root, byte for byte. The CLI
|
||||
already takes that root from the realpath of its own URL, today
|
||||
`/mnt/storage/src/mosaic-stack`, project `34050c07`. So a T3 project opened
|
||||
through the compatibility symlink `~/src/mosaic-stack-dev-test` doesn't
|
||||
match, and "no project row" is the right refusal. The README says so.
|
||||
|
||||
**Title rule.** Thread IDs change whenever Jason starts a new thread for a
|
||||
seat, so there's no fixed map. T3-AGENT-COMMS.md already names threads after
|
||||
the seat ("Darkwing", "Sage", "Dewey in Claude"). A thread belongs to seat
|
||||
`<s>` when `<s>` is a real directory under `agents/` and the lower-cased
|
||||
title equals `<s>` or starts with `<s>` followed by a space. So "Sagebrush"
|
||||
stays unmapped. Several threads can map to one seat, and their counts add
|
||||
up, as several Pi session files already do.
|
||||
|
||||
Today that maps Sage, Darkwing, Filbert, Dewey and Rocko (one thread each,
|
||||
created 2026-09-26, titles set by hand), plus "Darkwing in Claude" (archived)
|
||||
and "Dewey in Claude". Researcher has a directory and no thread. Three
|
||||
threads map to no seat. Two are imported and excluded anyway ("FINDINGS.md
|
||||
review" and "[dragon-lin:darkwing -> …"). The third is "Discord Bot" with 68
|
||||
user messages: 54 without a header, and the 14 free-text headers.
|
||||
|
||||
Titles are current state, and T3 can write them itself. They go wrong three
|
||||
ways. T3 auto-titles an unnamed thread from Jason's first prompt, so "Rocko
|
||||
review of the plan" maps to rocko. A rename moves the whole history to
|
||||
another row. A seat thread titled for a topic drops into `t3:unmapped`.
|
||||
None of this changes the Human total or the human-per-closed ratio. It only
|
||||
moves counts between rows, but Gate F reads one seat's row.
|
||||
|
||||
**Header cross-check.** The headers already say which seat a thread belongs
|
||||
to. For every user message whose header matches the fixed 6a rule and whose
|
||||
`to:` id equals the message's own `thread_id`:
|
||||
- in a mapped thread, the `to:` role, lower-cased, must equal that thread's
|
||||
seat;
|
||||
- in an unmapped thread, the `to:` role must not be a seat name.
|
||||
|
||||
A conflict exits 1 and names the thread id, its title and both roles. A
|
||||
header whose `to:` id is some other thread is not checked. The check reads
|
||||
message text only, not T3 metadata. In a live read at 21:02Z every header
|
||||
agreed: all 104 addressed to their own thread carried the full thread id and
|
||||
named that thread's seat (Sage 40, Darkwing 15, Filbert 18, Dewey 15, Rocko
|
||||
16).
|
||||
|
||||
It catches a seat thread renamed to another seat or to a topic, once any
|
||||
agent writes to it. It also catches an auto-titled thread that agents
|
||||
address by a different seat. It misses a thread no agent ever writes to.
|
||||
Such a thread can only add human counts to a seat's row, never hide them, so
|
||||
for Gate F it errs toward a visible failure. The README says so.
|
||||
|
||||
**Unmapped row.** Unmapped threads go in one Table 2 row, `t3:unmapped`
|
||||
(Sage's ruling), so their human messages still reach the Human column and
|
||||
the human-per-closed ratio.
|
||||
|
||||
**Mapping in the JSON.** For each seat, the T3 thread ids and titles that
|
||||
made its row, and the unmapped thread ids and titles. Anyone checking a Gate
|
||||
F result can then see which threads the row came from.
|
||||
|
||||
A seat's row sums its Pi and T3 counts, and the JSON splits them by source.
|
||||
Nothing is counted twice. Every T3 session today runs on `claudeAgent` or
|
||||
`codex`, which don't write `.pi/state`, and Filbert found no T3 header in any
|
||||
Pi log (6a record).
|
||||
|
||||
**Excluded,** with the reason stated in the README:
|
||||
- Imported threads (`thread_id` starting `import:`, events marked
|
||||
`historyImport`). They are partial copies of Claude Code sessions, not T3
|
||||
traffic: 55 user messages in two threads here.
|
||||
- Deleted threads (`deleted_at` set). Across all projects there are 3, with
|
||||
3 messages. Archived threads count.
|
||||
- Threads in other T3 projects. Live, there is a project at `/home/jwoltje`
|
||||
and a deleted one at `/mnt/storage/src`. A thread in either could work on
|
||||
this repository and would not be counted. The workspace-root rule is still
|
||||
the right one, but the README names this blind spot.
|
||||
|
||||
## 4. What fails closed
|
||||
|
||||
The source is on by default (Sage's ruling). `--no-t3` turns it off, and the
|
||||
report then says T3 was not read. `--t3-db <path>` reads another database
|
||||
file instead of `~/.t3/userdata/state.sqlite`. It exists for fixtures and
|
||||
gets the same checks.
|
||||
|
||||
Each of these refuses the report with exit 1, the code the ledger already
|
||||
uses for unreadable session evidence. The report never falls back to Pi logs
|
||||
alone. Where the database is missing or won't open, the message names
|
||||
`--no-t3`.
|
||||
- The database is missing or unreadable, or won't open read-only. That
|
||||
includes a directory that isn't writable when SQLite needs to create
|
||||
`-shm`, and a busy error after the timeout. The Pi reader treats a missing
|
||||
`.pi` as no messages, and this departs from it on purpose. A missing Pi
|
||||
directory means no Pi seats ran here. A missing T3 database on this host
|
||||
means the path or T3 changed, and a silent zero is the failure Gate F
|
||||
exists to prevent.
|
||||
- `~/.t3`, `~/.t3/userdata` or `state.sqlite` is a symlink. With `--t3-db`,
|
||||
the file and its directory are checked. The Pi reader checks every
|
||||
ancestor too, but it skips symlinked entries. Skipping one named file
|
||||
would be another silent zero, so this reader refuses.
|
||||
- A table or column the counts need is missing. The reader checks `PRAGMA
|
||||
table_info` and names what's missing. This catches a T3 upgrade that
|
||||
changes the schema.
|
||||
- No project row, or more than one non-deleted row, for this repository root.
|
||||
- A counted row has a bad `role`, non-string `text`, or a `created_at` that
|
||||
doesn't parse. The Pi reader already refuses malformed JSONL and bad
|
||||
timestamps the same way.
|
||||
- A header conflicts with the title mapping (section 3).
|
||||
|
||||
The reader never reads other files in `~/.t3`. It passes no message text
|
||||
beyond `messageKind`, `issueNumbers` and the header's `to:` role and id, the
|
||||
same rule as for Pi logs.
|
||||
|
||||
## 5. Rulings
|
||||
|
||||
Sage ruled on the three questions R1 put to Jason, as lead calls:
|
||||
1. The source is on by default. A missing or unreadable database exits 1,
|
||||
and the message names `--no-t3`.
|
||||
2. Unmapped threads get the `t3:unmapped` row.
|
||||
3. The 14 free-text headers stay as recorded. They show only in the JSON
|
||||
diagnostic.
|
||||
|
||||
Sage also ruled that the class fix rides in this build, and that a missing
|
||||
diagnostic table reads `unknown` (F5).
|
||||
|
||||
## 6. Acceptance for the build
|
||||
|
||||
**No test opens the real `~/.t3`.** Both places in
|
||||
`packages/ledger/tests/ledger.test.mjs` that spawn `cli.mjs` (the shared
|
||||
`run()` helper and the direct `spawnSync` at line 66) set `HOME` to the
|
||||
fixture's temp directory. A test that forgets `--t3-db` or `--no-t3` then
|
||||
finds no database and fails closed. The existing tests aren't about T3. Each
|
||||
gets an empty fixture database at the fixture `HOME`'s default path, with
|
||||
one project row for the fixture root. So they run with the source on, and
|
||||
their expected rows don't change. One test asserts that a `HOME` with no
|
||||
database exits 1 and names `--no-t3`.
|
||||
|
||||
Fixture databases are built with `node:sqlite` in a temp directory, in WAL
|
||||
mode:
|
||||
- seat threads and an unmapped thread; imported, deleted and archived
|
||||
threads; a message outside the date range;
|
||||
- all three header forms, with uppercase classes in both the tmux preamble
|
||||
and the T3 header;
|
||||
- a thread with the same seat title in another project;
|
||||
- a seat thread renamed to another seat, with an agent header to its own
|
||||
id, which exits 1;
|
||||
- a "Sagebrush" title, which stays unmapped;
|
||||
- a thread titled "Researcher", which maps to the seat that has no thread
|
||||
live.
|
||||
|
||||
WAL states, each with its own test:
|
||||
- The newest message is only in `-wal`, with the writer still attached (the
|
||||
live-T3 case). It is counted.
|
||||
- T3 stopped: the database closed cleanly with no writer. Counts are
|
||||
correct, and the main file's bytes are unchanged afterwards.
|
||||
- `-wal` without `-shm` in a writable directory: made by a child writer with
|
||||
`wal_autocheckpoint=0` that is SIGKILLed, then `-shm` deleted. The WAL
|
||||
rows are counted.
|
||||
- `-wal` without `-shm` in a directory that isn't writable: exit 1, naming
|
||||
`--no-t3`. Skipped when the tests run as root, where the mode bits don't
|
||||
bind.
|
||||
- The stopped case in a directory that isn't writable: exit 1, naming
|
||||
`--no-t3`, as I measured it. If the build reads there instead, the builder
|
||||
changes this test to assert correct counts and records the correction in
|
||||
the BUILD-LOG entry. Skipped as root too.
|
||||
|
||||
Also:
|
||||
- Each other fail-closed case in section 4 has its own test, including a
|
||||
removed schema column and each symlink.
|
||||
- A missing `orchestration_events` gives `unknown` for the diagnostic and
|
||||
the same counts.
|
||||
- The class fix is proven against HEAD's `messageKind`. An uppercase class
|
||||
in either preamble counts as agent after the fix and as human before it.
|
||||
- The JSON lists each seat's threads and the unmapped threads.
|
||||
- A read against the live database gives the counts in this brief, allowing
|
||||
for messages sent since. It exits 0 with no header conflict.
|
||||
- The ledger README's counting rules name the new source, both flags, the
|
||||
mapping rule and the header check, and the exclusions. That includes the
|
||||
symlinked-checkout case and the other-project blind spot.
|
||||
- No suite runs the ledger tests, so the BUILD-LOG entry names the test file.
|
||||
|
||||
## Not in scope
|
||||
|
||||
- Claude Code transcripts (`~/.claude/projects`) and Codex sessions
|
||||
(`~/.codex/sessions`). T3's database already holds every message T3
|
||||
delivered, so those files would only duplicate it.
|
||||
- Any T3 API call, anything that needs T3 running, and any write beyond
|
||||
SQLite's own `-wal` and `-shm` handling.
|
||||
- Fixing the two tmux misclassifications Filbert found in 6a.
|
||||
@@ -1,20 +0,0 @@
|
||||
76833a3bf536bb9592a0040cdb10d9a8351cf0b3bd2828feee928d3415136a79 docs/plans/BRIEF-TEMPLATE.md
|
||||
5d4b4c7a4624ef267d76d4c032dbcddf3a3d7e7b73a5cfed06953307992bbde7 packages/queue/package.json
|
||||
576d8ed44a19e1cca96fd7128ca34fcc62d840580b2c0936fa3b296df2e72ddf packages/queue/README.md
|
||||
188ade96cabb73e06b6b8fbe3d30e8d4d174843877f3a151dcf08082c92d3068 packages/queue/src/cli.mjs
|
||||
7a851814dfff6f392de814fc31f8dc8cbe9c79cd40ef71313dca1939115ee879 packages/queue/src/errors.mjs
|
||||
b18e120cb9ddba4c5576d7bc7f7f378ed86e6ec1e084d436474b80765261d2e5 packages/queue/src/io.mjs
|
||||
52d9f68f01f29e84943fc359fdb1d1ddfaf58d1650c6b15b253b83f1daba9927 packages/queue/src/lock.mjs
|
||||
c11235a6b99acf6baf1257c63eced410060065ef4f21f79a18860adfa19571cf packages/queue/src/queue.mjs
|
||||
756cbc9ab13de85757cc24f903d02e8a1d20bb45bb8020c5e1d91e26fbaacaa4 packages/queue/src/store.mjs
|
||||
6005da4c809cb9045f9480e1e29077b8e7ab185c3ed91567717e8cc13d67b5b0 packages/queue/tests/commit.test.mjs
|
||||
d29d58427c712a69e8a818d8c74ce724d519ddea8780cc0b5f0b0035cec0f498 packages/queue/tests/data.test.mjs
|
||||
5cccea50d5a40e07891a090dea001c095a26f4a6c2e0af40c144b1d00f239b5e packages/queue/tests/fixtures/kill-at.mjs
|
||||
59cc8092fbcddbe9854da7d5014f706f4ffb86573f62ad909aa0b0e09c6d99b9 packages/queue/tests/fixtures/lock-child.mjs
|
||||
5769b3618918fe36398a75e449d644932332ad5a60c3127098a1d011eda2c182 packages/queue/tests/helpers.mjs
|
||||
9f98a388ce91438c3238be36049bcd5171b365c68d4b7bf1ae5ff910f4b7b1e8 packages/queue/tests/lock.test.mjs
|
||||
2a3d2be8cb25b6e7cd18ba56393a284415c66e7ee26a3148fa39f32885efbd35 packages/queue/tests/store.test.mjs
|
||||
74378acbd41ef21a0b171b08aa85677e4c471966d2ffd8cfb310adb0e044b246 packages/queue/tests/write.test.mjs
|
||||
3cbd40575dc728dc5407c5029f4f5fff747ce93a5508807233f4362ad37d7f9c scripts/git-hooks/pre-commit
|
||||
2632078bea45e0249b3fdd9a335100bade7c9106222931603ede9d414c404f54 scripts/queue-commit.sh
|
||||
92cea23b9ada2edb1b0482ca2daf5864666cc1f546e1f2e6558c3826f1ddf9e7 scripts/test-queue.sh
|
||||
@@ -1,20 +0,0 @@
|
||||
20363f5dafbb1be8b7380d7603fd04cf38f5284457b634c9a98ce5a6d8e4832a docs/plans/BRIEF-TEMPLATE.md
|
||||
5d4b4c7a4624ef267d76d4c032dbcddf3a3d7e7b73a5cfed06953307992bbde7 packages/queue/package.json
|
||||
9ebdb6a3f239051a39e63fcf8f59c8fba540bb3cc63b7880d15e6c6f1e549a09 packages/queue/README.md
|
||||
711db25594d78e0ba603a9e91221f6c32241ce1d9217b01bd4da3a99967ae871 packages/queue/src/cli.mjs
|
||||
7a851814dfff6f392de814fc31f8dc8cbe9c79cd40ef71313dca1939115ee879 packages/queue/src/errors.mjs
|
||||
b18e120cb9ddba4c5576d7bc7f7f378ed86e6ec1e084d436474b80765261d2e5 packages/queue/src/io.mjs
|
||||
1095cb6611f2d8d38f535930bbe03208e854b4a57176cea813f7ea0487b3c4f1 packages/queue/src/lock.mjs
|
||||
8e9230901f550b829ef55e754819c9cd5703e98e447fcebdba6ce5dbef11b0e9 packages/queue/src/queue.mjs
|
||||
172cf529b2c0e915fbd4a130faa5dbec9023bb19160012246801b16482b4ffaa packages/queue/src/store.mjs
|
||||
74d04d0de9f4e068fe66bb465ed1575fb38b6ea6592fca6cd6fbacdd553fe9e1 packages/queue/tests/commit.test.mjs
|
||||
e3f774f823bfb1bcb6025cc3688d72ec110144d5df064557d1645a62472fd663 packages/queue/tests/data.test.mjs
|
||||
5cccea50d5a40e07891a090dea001c095a26f4a6c2e0af40c144b1d00f239b5e packages/queue/tests/fixtures/kill-at.mjs
|
||||
59cc8092fbcddbe9854da7d5014f706f4ffb86573f62ad909aa0b0e09c6d99b9 packages/queue/tests/fixtures/lock-child.mjs
|
||||
5769b3618918fe36398a75e449d644932332ad5a60c3127098a1d011eda2c182 packages/queue/tests/helpers.mjs
|
||||
a2dbe3dc69b9d53c42246e41e61b9b7d2395697a53ca12ef3481965b43321ff6 packages/queue/tests/lock.test.mjs
|
||||
34f4b0e3eeadc882ffcc6ba7f0b439651957f2f0c41c9387cf8fb51901f46202 packages/queue/tests/store.test.mjs
|
||||
70e8a068efda8da8fe1e1628cc1cd5b7fa7796a475941bf7349747244c002f1b packages/queue/tests/write.test.mjs
|
||||
3cbd40575dc728dc5407c5029f4f5fff747ce93a5508807233f4362ad37d7f9c scripts/git-hooks/pre-commit
|
||||
2632078bea45e0249b3fdd9a335100bade7c9106222931603ede9d414c404f54 scripts/queue-commit.sh
|
||||
92cea23b9ada2edb1b0482ca2daf5864666cc1f546e1f2e6558c3826f1ddf9e7 scripts/test-queue.sh
|
||||
@@ -1,186 +0,0 @@
|
||||
# Queue A1 build (#1508), candidate for review
|
||||
|
||||
Darkwing built this on 2026-09-26 from section 8 of
|
||||
`agents/filbert/work/queue-as-data-plan-2026-09-26.md` (sha256 282fabbb,
|
||||
the only spec), split as Sage approved: render is in A1, `move in-review`
|
||||
needs `--candidate` until piece D, and a round's issue is the row's first
|
||||
issue. Filbert reviews the code; Sage commits after the suites. Base is HEAD
|
||||
3a209eea. Nothing is committed, staged or pushed.
|
||||
|
||||
A stray pkill at 22:02:32Z stopped my first turn with only
|
||||
`src/errors.mjs` and `src/io.mjs` on disk. I reread both against what I had
|
||||
meant them to be. They match: the exit-code class, and the file layer with
|
||||
`realIo` as the only layer the CLI uses. Everything else was written after
|
||||
the restart.
|
||||
|
||||
## Files
|
||||
|
||||
`build-manifest.sha256` pins the 20 files. `build.patch` (sha256 419804f2)
|
||||
adds all 20 as new files with their modes. It applies cleanly to 3a209eea,
|
||||
and the applied tree matches the manifest and passes `scripts/test-queue.sh`.
|
||||
|
||||
- `packages/queue/src/`: `errors.mjs`, `io.mjs` (the fault-injectable file
|
||||
layer), `lock.mjs` (lock and unlock gate, 8.4), `queue.mjs` (serialization,
|
||||
replay, the transition matrix, `next`, render), `store.mjs` (canonical
|
||||
checks, the write path, witness, views, snapshot, verify), `cli.mjs`.
|
||||
- `packages/queue/tests/`: data 19, lock 17, store 18, write 20, commit 21
|
||||
tests, plus `helpers.mjs` and two child fixtures.
|
||||
- `packages/queue/package.json` and `README.md`. The package has no
|
||||
dependencies.
|
||||
- `scripts/queue-commit.sh` (0755): the 8.12 procedure and
|
||||
`--install-hook`.
|
||||
- `scripts/git-hooks/pre-commit` (0755, POSIX sh): the queue guard.
|
||||
- `scripts/test-queue.sh` (0755): the suite, in the style of
|
||||
`test-discord.sh`.
|
||||
- `docs/plans/BRIEF-TEMPLATE.md`: the 8.13 template.
|
||||
|
||||
## Which path runs `verify` once genesis is in
|
||||
|
||||
`scripts/test-queue.sh` runs `node packages/queue/src/cli.mjs verify` when
|
||||
`git cat-file -e HEAD:docs/plans/queue.json` succeeds. At HEAD today there is
|
||||
no `queue.json`, so it prints `skip queue verify: HEAD has no
|
||||
docs/plans/queue.json (before the genesis commit)` and stays green. A2 moves
|
||||
that call to `scripts/mosaic queue verify` when it adds the dispatch.
|
||||
`queue-commit.sh` also calls `node packages/queue/src/cli.mjs` directly
|
||||
(`snapshot`, then `verify --snapshot` from HEAD's archive) until A2.
|
||||
|
||||
## Sage's five conditions
|
||||
|
||||
1. Nothing ran against the canonical `.git`. After all runs, `.git/hooks`
|
||||
holds only the samples, `.git` has no `mosaic-queue*` file, and
|
||||
`git config --show-scope --get-all core.hooksPath` returns nothing in any
|
||||
scope (rc 1). Every hook install, genesis, lock and gate test runs in a
|
||||
scratch repo under the system temp directory. Suite runs used a
|
||||
`--shared` clone at `/tmp/qa1-verify`.
|
||||
2. `docs/plans/QUEUE.md`, `AGENTS.md` and `docs/TOOLS.md` are unedited.
|
||||
`docs/SESSIONS.md` shows as modified in the working tree, but that was
|
||||
someone else's edit before my session began; I didn't touch it.
|
||||
3. `scripts/test-queue.sh` is green at HEAD with no `queue.json`: 19 checks
|
||||
passed, `verify` skipped as above.
|
||||
4. H is recorded before the canary. `commit.test.mjs` has "H recorded before
|
||||
the canary": a shim commits on the first `git hook run`, and
|
||||
`queue-commit.sh` exits 1 with `refs/heads/<branch> moved since <H>;
|
||||
nothing published`. A second test moves HEAD after `commit-tree` with the
|
||||
same result. Mutation M1 (read H after the canary) fails the first test.
|
||||
5. The fault file layer is reachable only from tests. Faults enter through
|
||||
the options the API takes (`io`, `proc`, `hook`, `now`, `readOrder`,
|
||||
`lockWaitMs`); `cli.mjs` passes none. The only `process.env` read in
|
||||
`src/` is the default `env` in `store.mjs`'s context.
|
||||
|
||||
## Bugs found while building
|
||||
|
||||
- A nested `node --test` inherits `NODE_TEST_CONTEXT` and exits 0 whatever
|
||||
its tests do. Step 4's run of HEAD's archived tests therefore passed with a
|
||||
failing test in the archive. `queue-commit.sh` and `test-queue.sh` now run
|
||||
it under `env -u NODE_TEST_CONTEXT`, and a test commits an archive with a
|
||||
failing test and expects a refusal (mutation M4). Other suites in this repo
|
||||
that nest `node --test` may have the same blind spot. I haven't checked
|
||||
them.
|
||||
- git 2.55 does not hold `index.lock` while the commit editor is open. The
|
||||
plan expected a paused `git commit -e` to block step 8. It doesn't: the
|
||||
paused commit loses later at its own HEAD update with `cannot lock ref
|
||||
'HEAD': is at C but expected H`. The test now asserts that outcome.
|
||||
Nothing is lost, but the reason differs from the plan's.
|
||||
- ext4 hands a freed inode number straight back. My first test for release's
|
||||
inode check wrote a byte-identical lock after unlinking the original and
|
||||
got the same inode back, so it proved nothing. It now writes a copy and
|
||||
renames it over the lock, which guarantees a new inode.
|
||||
|
||||
## Choices the spec left open
|
||||
|
||||
- Verb names `release` and `set`. `add` requires `--gate`. A null brief is
|
||||
allowed only on rows that genesis creates as done. `sync --op` is
|
||||
optional.
|
||||
- Reads need no actor. `next` with no seat and no `$MOSAIC_AGENT_NAME`
|
||||
refuses.
|
||||
- `accept-history` accepts a stale table but not an unknown one. When a
|
||||
write succeeds but the table write is skipped, the CLI warns and exits 0.
|
||||
- An invalid witness is treated as absent.
|
||||
- `GIT_DIR`, `GIT_WORK_TREE` and `GIT_COMMON_DIR` refuse in the CLI.
|
||||
`queue-commit.sh` also refuses `GIT_INDEX_FILE`, `GIT_OBJECT_DIRECTORY`
|
||||
and `GIT_ALTERNATE_OBJECT_DIRECTORIES`.
|
||||
- Leftover temp files are unlinked. Genesis uses `link`, so it cannot
|
||||
replace an existing file.
|
||||
- `unlock` works on a missing or invalid lock file.
|
||||
- `note` on a blocked row edits `blockedReason`.
|
||||
- Messages already name `scripts/mosaic queue`. The lost-history refusal
|
||||
names `sync` when the file holds genesis alone.
|
||||
- `--candidate` auto-detects: an existing file is a manifest, anything else
|
||||
is a commit reachable from `refs/heads` or `refs/tags`.
|
||||
- Only `--install-hook` is privileged in `queue-commit.sh` (jason or sage).
|
||||
The commit itself relies on the protocol that the lead runs it.
|
||||
- After the snapshot, `queue-commit.sh` checks the genesis entry's branch
|
||||
and root against the current branch and root. For `--genesis` it also
|
||||
checks that `H:<map>` is the map blob genesis read.
|
||||
- Step 8 compares the index's two queue entries with H's before it looks
|
||||
for `index.lock`, so "someone staged a queue path" is reported ahead of a
|
||||
lock.
|
||||
|
||||
## Deferred
|
||||
|
||||
- 8.12's test of `verify-commit` on a prospective tree belongs to piece D,
|
||||
which adds `verify-commit`. It is not in A1.
|
||||
- A2 holds the real migration map, the QUEUE.md markers and header, the
|
||||
row-7 pointer, the golden render, `scripts/mosaic` dispatch and
|
||||
`docs/TOOLS.md`.
|
||||
- Sage adds `queue` to the suite list when A1 lands (lead decision 20).
|
||||
- Bootstrap happens after A2's map and markers land:
|
||||
`scripts/queue-commit.sh --install-hook --by sage`, then `queue genesis`,
|
||||
then `scripts/queue-commit.sh --genesis -m MSG`.
|
||||
|
||||
## Verification
|
||||
|
||||
In `/tmp/qa1-verify` (HEAD 3a209eea plus the 20 files):
|
||||
|
||||
| Suite | Result |
|
||||
|---|---|
|
||||
| config | 24/24 |
|
||||
| task | 90/90 |
|
||||
| foundation | 43/43 |
|
||||
| conductor | 17/17 |
|
||||
| release | 14/14 |
|
||||
| auth | 15/15 |
|
||||
| discord | 63/63 |
|
||||
| extension-package | 18/18 |
|
||||
| queue | 19 checks; `node --test` 95/95 |
|
||||
|
||||
The queue tests take about 18 s and were stable over two runs. A combined
|
||||
`node --test` run over every package came to 474/474.
|
||||
|
||||
I also ran the post-genesis path in a scratch repo: install the hook,
|
||||
genesis, `--genesis` commit, then `test-queue.sh`. `verify` printed `ok
|
||||
verify rev 1: file valid, witness matches, view current`. After a hand edit
|
||||
to one table cell it failed with `view unknown`.
|
||||
|
||||
### Mutations
|
||||
|
||||
Each mutation went into the verify clone, the queue tests ran, and the
|
||||
original was restored. Every one was caught; the number is how many tests
|
||||
failed.
|
||||
|
||||
| Id | Mutation | Failing tests |
|
||||
|---|---|---|
|
||||
| M1 | read H after the canary | 1 |
|
||||
| M2 | drop the step-7 guard recheck | 2 |
|
||||
| M3 | drop step 8's entry comparison | 1 |
|
||||
| M4 | keep `NODE_TEST_CONTEXT` | 1 |
|
||||
| M5 | drop step 1's staged-path check | 1 |
|
||||
| M6 | `update-ref` without the old value | 2 |
|
||||
| M7 | skip the canary | 2 |
|
||||
| M8 | guard hook always passes | 21 |
|
||||
| M9 | drop step 8's `index.lock` check | 1 |
|
||||
| M10 | drop the exec-bit check | 2 |
|
||||
| M11 | drop the `core.hooksPath` check | 1 |
|
||||
| M12 | drop the map-blob check | 1 |
|
||||
| S1 | drop the "unchanged since read" check | 1 |
|
||||
| S2 | swallow the directory fsync error | 2 |
|
||||
| S3 | drop the recheck under the lock for unlocked reads | 2 |
|
||||
| S4 | drop the unlock-gate check | 2 |
|
||||
| S5a | release ignores the inode | 1 |
|
||||
| S5b | release ignores the record bytes | 1 |
|
||||
| S6 | write the witness before the rename | 10 |
|
||||
| S7 | drop genesis's fsync | 1 |
|
||||
| S8 | treat a reused pid as dead | 11 |
|
||||
|
||||
S5 survived at first: the delayed-release test was caught by the byte
|
||||
comparison alone. The inode test in `lock.test.mjs` closes that gap.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,819 +0,0 @@
|
||||
diff --git a/docs/plans/BRIEF-TEMPLATE.md b/docs/plans/BRIEF-TEMPLATE.md
|
||||
index c5ece4ba..1b0facef 100644
|
||||
--- a/docs/plans/BRIEF-TEMPLATE.md
|
||||
+++ b/docs/plans/BRIEF-TEMPLATE.md
|
||||
@@ -13,7 +13,8 @@ Rules the queue enforces (queue-as-data plan 8.13):
|
||||
refuse until the lead re-pins it.
|
||||
|
||||
`queued` means the brief exists, not that it is accepted. The row moves to
|
||||
-`briefed` when its owner accepts it.
|
||||
+`briefed` when a privileged actor (jason or sage) accepts it; the owner
|
||||
+can't.
|
||||
|
||||
---
|
||||
|
||||
diff --git a/packages/queue/README.md b/packages/queue/README.md
|
||||
index 74956816..1c8774fa 100644
|
||||
--- a/packages/queue/README.md
|
||||
+++ b/packages/queue/README.md
|
||||
@@ -44,7 +44,18 @@ Exit codes: 0 ok; 1 the operation failed; 2 invalid data or refused;
|
||||
Until piece D, `move ID in-review` needs `--candidate`: an existing file is
|
||||
read as a manifest (one `<sha256> <path>` line per file), anything else as a
|
||||
commit reachable from `refs/heads` or `refs/tags`. The candidate is frozen
|
||||
-for the round. The review's issue is the row's first issue.
|
||||
+for the round.
|
||||
+
|
||||
+The review's issue follows lead decision 23. A row with no issues can't
|
||||
+request review. A row with one issue uses it. A row with several needs
|
||||
+`--issue N`, one of its issues. Later rounds keep the previous round's issue
|
||||
+unless `--issue` names another; if the row no longer lists the kept issue,
|
||||
+the request refuses until `--issue` names one.
|
||||
+
|
||||
+`move ID done` from in-review needs `--evidence
|
||||
+comment=<id>,round=<n>,candidate=<digest>`. The round must be the current
|
||||
+one and the digest its candidate's, so a comment from an earlier round
|
||||
+can't close a later one, even when the candidate is the same.
|
||||
|
||||
## Where the files live
|
||||
|
||||
diff --git a/packages/queue/src/cli.mjs b/packages/queue/src/cli.mjs
|
||||
index c5517747..e8df80f7 100644
|
||||
--- a/packages/queue/src/cli.mjs
|
||||
+++ b/packages/queue/src/cli.mjs
|
||||
@@ -4,7 +4,7 @@
|
||||
// Reads: list | show ID | next [SEAT]
|
||||
// Changes: add --piece TEXT --gate TEXT --brief PATH#ANCHOR [--issue N]... [--note TEXT]
|
||||
// [--owner SEAT] [--gate-owner SEAT] [--after ID[:settled]]... [--reviewer SEAT]... [--required]
|
||||
-// move ID STATE [--reason TEXT] [--candidate COMMIT|MANIFEST] [--evidence TEXT]
|
||||
+// move ID STATE [--reason TEXT] [--candidate COMMIT|MANIFEST] [--issue N] [--evidence TEXT]
|
||||
// release ID | assign ID SEAT | note ID TEXT | set ID FIELD VALUE [--reason TEXT]
|
||||
// genesis --root PATH --branch NAME --map PATH
|
||||
// accept-history --reason TEXT --yes
|
||||
@@ -23,7 +23,7 @@ import { list, mutate, next, renderView, show, snapshot, sync, unlock, verify, v
|
||||
const USAGE = [
|
||||
"usage: queue list | show ID | next [SEAT]",
|
||||
" queue add --op ID --piece TEXT --gate TEXT --brief PATH#ANCHOR [--issue N]... [--note TEXT] [--owner SEAT] [--gate-owner SEAT] [--after ID[:settled]]... [--reviewer SEAT]... [--required]",
|
||||
- " queue move ID STATE --op ID [--reason TEXT] [--candidate COMMIT|MANIFEST] [--evidence TEXT]",
|
||||
+ " queue move ID STATE --op ID [--reason TEXT] [--candidate COMMIT|MANIFEST] [--issue N] [--evidence TEXT]",
|
||||
" queue release ID --op ID | assign ID SEAT --op ID | note ID TEXT --op ID",
|
||||
` queue set ID FIELD VALUE --op ID [--reason TEXT] (fields: ${SET_FIELDS.join(", ")})`,
|
||||
" queue genesis --op ID --root PATH --branch NAME --map PATH",
|
||||
@@ -134,8 +134,12 @@ export function run(argv, opts = {}) {
|
||||
});
|
||||
}
|
||||
case "move":
|
||||
- allow(flags, [...CHANGE, "--reason", "--candidate", "--evidence"]); positional(pos, 2, "move ID STATE");
|
||||
- return change("move", { id: intArg(pos[0], "ID"), to: pos[1], reason: f("--reason"), candidate: f("--candidate"), evidence: f("--evidence") });
|
||||
+ allow(flags, [...CHANGE, "--reason", "--candidate", "--issue", "--evidence"]); positional(pos, 2, "move ID STATE");
|
||||
+ if ((flags.get("--issue") ?? []).length > 1) throw usage("move takes one --issue");
|
||||
+ return change("move", {
|
||||
+ id: intArg(pos[0], "ID"), to: pos[1], reason: f("--reason"), candidate: f("--candidate"), evidence: f("--evidence"),
|
||||
+ issue: flags.has("--issue") ? intArg(flags.get("--issue")[0].replace(/^#/, ""), "--issue") : null,
|
||||
+ });
|
||||
case "release":
|
||||
allow(flags, CHANGE); positional(pos, 1, "release ID");
|
||||
return change("release", { id: intArg(pos[0], "ID") });
|
||||
diff --git a/packages/queue/src/lock.mjs b/packages/queue/src/lock.mjs
|
||||
index 56c43f0f..34e88e3b 100644
|
||||
--- a/packages/queue/src/lock.mjs
|
||||
+++ b/packages/queue/src/lock.mjs
|
||||
@@ -70,6 +70,7 @@ function describe(c) {
|
||||
function publish(io, target, bytes, hook, waitMs, stepMs) {
|
||||
const tmp = `${target}.${process.pid}.${randomBytes(6).toString("hex")}.tmp`;
|
||||
let fd;
|
||||
+ let st;
|
||||
try {
|
||||
fd = io.openExcl(tmp, 0o600);
|
||||
} catch (err) {
|
||||
@@ -82,6 +83,9 @@ function publish(io, target, bytes, hook, waitMs, stepMs) {
|
||||
fd = null;
|
||||
const back = io.readFile(tmp);
|
||||
if (!back.equals(bytes)) throw Object.assign(new Error("read-back differs from the record"), { code: "EREADBACK" });
|
||||
+ // The link gives the target this inode, so read it before linking:
|
||||
+ // nothing that can fail runs between a successful link and the return.
|
||||
+ st = io.stat(tmp);
|
||||
} catch (err) {
|
||||
if (fd !== null) { try { io.close(fd); } catch { /* already failing */ } }
|
||||
unlinkQuiet(io, tmp);
|
||||
@@ -99,7 +103,6 @@ function publish(io, target, bytes, hook, waitMs, stepMs) {
|
||||
sleepMs(stepMs);
|
||||
continue;
|
||||
}
|
||||
- const st = io.stat(tmp);
|
||||
return { linked: true, dev: st.dev, ino: st.ino, bytes };
|
||||
}
|
||||
} finally {
|
||||
@@ -122,8 +125,14 @@ export function acquire({ gitDir, io, proc = realProc, op = null, verb, waitMs =
|
||||
const handle = { path, dev: got.dev, ino: got.ino, bytes: got.bytes };
|
||||
hook("lock-linked");
|
||||
const gate = join(gitDir, GATE_NAME);
|
||||
- if (lstatOrNull(io, gate) !== null) {
|
||||
- const c = classify(readOrNull(io, gate), proc);
|
||||
+ let c = null;
|
||||
+ try {
|
||||
+ if (lstatOrNull(io, gate) !== null) c = classify(readOrNull(io, gate), proc);
|
||||
+ } catch (err) {
|
||||
+ const left = release(handle, io);
|
||||
+ throw new QueueError(`cannot check the unlock gate ${gate}: ${errno(err)}; ${left ?? "lock released"}`, 1);
|
||||
+ }
|
||||
+ if (c !== null) {
|
||||
release(handle, io);
|
||||
throw new QueueError(`unlock gate ${gate} is present (${describe(c)}); check it with \`scripts/mosaic queue unlock --check-gate\``, 2);
|
||||
}
|
||||
@@ -154,18 +163,29 @@ export function unlock({ gitDir, io, proc = realProc, hook = () => {} }) {
|
||||
}
|
||||
const gate = { path: gatePath, dev: got.dev, ino: got.ino, bytes };
|
||||
hook("gate-held");
|
||||
+ let result;
|
||||
+ let failure = null;
|
||||
try {
|
||||
const lockBytes = readOrNull(io, lockPath);
|
||||
- if (lockBytes === null) return "no queue lock present; nothing removed";
|
||||
- const c = classify(lockBytes, proc);
|
||||
- if (c.state !== "dead" && c.state !== "mismatch") {
|
||||
- throw new QueueError(`queue lock owner is ${describe(c)}; unlock refuses`, 2);
|
||||
+ if (lockBytes === null) {
|
||||
+ result = "no queue lock present; nothing removed";
|
||||
+ } else {
|
||||
+ const c = classify(lockBytes, proc);
|
||||
+ if (c.state !== "dead" && c.state !== "mismatch") {
|
||||
+ throw new QueueError(`queue lock owner is ${describe(c)}; unlock refuses`, 2);
|
||||
+ }
|
||||
+ io.unlink(lockPath);
|
||||
+ result = `removed queue lock (${describe(c)}): ${lockBytes.toString("utf8").trim()}`;
|
||||
}
|
||||
- io.unlink(lockPath);
|
||||
- return `removed queue lock (${describe(c)}): ${lockBytes.toString("utf8").trim()}`;
|
||||
- } finally {
|
||||
- release(gate, io);
|
||||
+ } catch (err) {
|
||||
+ failure = err;
|
||||
}
|
||||
+ let msg;
|
||||
+ try { msg = release(gate, io); } catch (err) { msg = `cannot release the unlock gate (${errno(err)})`; }
|
||||
+ // Like the lock, a swapped gate is reported on success and on refusal (8.4).
|
||||
+ if (msg && failure instanceof Error) failure.message += `\nwarning: ${msg}`;
|
||||
+ if (failure) throw failure;
|
||||
+ return msg ? `${result}\nwarning: ${msg}` : result;
|
||||
}
|
||||
|
||||
export function checkGate({ gitDir, io, proc = realProc }) {
|
||||
diff --git a/packages/queue/src/queue.mjs b/packages/queue/src/queue.mjs
|
||||
index 9999ba18..06e77c17 100644
|
||||
--- a/packages/queue/src/queue.mjs
|
||||
+++ b/packages/queue/src/queue.mjs
|
||||
@@ -208,7 +208,7 @@ export function validateRow(row) {
|
||||
checkNames(row.reviewers, `${w} reviewers`);
|
||||
if (row.review !== null) {
|
||||
keysExactly(row.review, ["issue", "rounds"], `${w} review`);
|
||||
- if (row.review.issue !== null) checkId(row.review.issue, `${w} review issue`);
|
||||
+ checkId(row.review.issue, `${w} review issue`);
|
||||
if (!Array.isArray(row.review.rounds) || row.review.rounds.length === 0) throw refuse(`${w} review needs at least one round`);
|
||||
row.review.rounds.forEach((r, i) => checkRound(r, i + 1));
|
||||
}
|
||||
@@ -338,6 +338,7 @@ export function canonArgs(verb, a) {
|
||||
reason: nullable(a.reason, (v) => checkText(v, "reason")),
|
||||
candidate: nullable(a.candidate, (v) => checkText(v, "candidate", { max: 300 })),
|
||||
evidence: nullable(a.evidence, (v) => checkText(v, "evidence")),
|
||||
+ issue: nullable(a.issue, (v) => checkId(v, "issue")),
|
||||
};
|
||||
case "release":
|
||||
return { id: checkId(a.id) };
|
||||
@@ -393,11 +394,30 @@ function afterSatisfied(rows, row) {
|
||||
return missing;
|
||||
}
|
||||
|
||||
-// `comment=<id>,candidate=<digest>`: the J5 evidence before Piece D.
|
||||
+// `comment=<id>,round=<n>,candidate=<digest>`: the J5 evidence before Piece D.
|
||||
export function parseReviewEvidence(text) {
|
||||
- const m = /^comment=([1-9][0-9]{0,19}),candidate=([0-9a-f]{40}|[0-9a-f]{64})$/.exec(text ?? "");
|
||||
- if (!m) throw refuse("in-review to done needs --evidence comment=<id>,candidate=<digest> for the current round");
|
||||
- return { comment: m[1], candidate: m[2] };
|
||||
+ const m = /^comment=([1-9][0-9]{0,19}),round=([1-9][0-9]{0,5}),candidate=([0-9a-f]{40}|[0-9a-f]{64})$/.exec(text ?? "");
|
||||
+ if (!m) throw refuse("in-review to done needs --evidence comment=<id>,round=<n>,candidate=<digest> for the current round");
|
||||
+ return { comment: m[1], round: Number(m[2]), candidate: m[3] };
|
||||
+}
|
||||
+
|
||||
+// The issue a review round posts to (lead decision 23). The row must list
|
||||
+// one; with several, --issue names it. A later round keeps the previous
|
||||
+// round's issue unless --issue names another, and the kept issue must still
|
||||
+// be one of the row's.
|
||||
+function reviewIssue(row, issue) {
|
||||
+ const list = row.issues.map((n) => `#${n}`).join(", ");
|
||||
+ if (row.issues.length === 0) throw refuse(`row ${row.id} lists no issues; a privileged actor sets one before review`);
|
||||
+ if (issue !== null) {
|
||||
+ if (!row.issues.includes(issue)) throw refuse(`--issue #${issue} is not one of row ${row.id}'s issues (${list})`);
|
||||
+ return issue;
|
||||
+ }
|
||||
+ if (row.review) {
|
||||
+ if (!row.issues.includes(row.review.issue)) throw refuse(`row ${row.id}'s review issue #${row.review.issue} is no longer one of its issues (${list}); name one with --issue`);
|
||||
+ return row.review.issue;
|
||||
+ }
|
||||
+ if (row.issues.length > 1) throw refuse(`row ${row.id} lists several issues (${list}); name the review's issue with --issue`);
|
||||
+ return row.issues[0];
|
||||
}
|
||||
|
||||
function touch(row, entry) {
|
||||
@@ -424,7 +444,7 @@ function getRow(rows, id) {
|
||||
}
|
||||
|
||||
function applyMove(rows, row, entry, resolved) {
|
||||
- const { to, reason, candidate, evidence } = entry.args;
|
||||
+ const { to, reason, candidate, evidence, issue } = entry.args;
|
||||
const by = entry.by;
|
||||
const from = row.state;
|
||||
const illegal = () => refuse(`row ${row.id}: ${from}→${to} is not a transition`);
|
||||
@@ -432,9 +452,11 @@ function applyMove(rows, row, entry, resolved) {
|
||||
if (reason !== null && to !== "blocked") throw refuse("--reason applies only to a move to blocked");
|
||||
if (candidate !== null && !(from === "in-progress" && to === "in-review")) throw refuse("--candidate applies only to in-progress→in-review");
|
||||
if (evidence !== null && to !== "done") throw refuse("--evidence applies only to a move to done");
|
||||
+ if (issue !== null && !(from === "in-progress" && to === "in-review")) throw refuse("--issue applies only to in-progress→in-review");
|
||||
let next = { ...row, state: to };
|
||||
let round = null;
|
||||
let cand = null;
|
||||
+ let revIssue = null;
|
||||
if (to === "blocked") {
|
||||
if (from === "blocked") throw refuse(`row ${row.id} is already blocked; update the reason with note`);
|
||||
if (!NON_TERMINAL.has(from)) throw illegal();
|
||||
@@ -457,11 +479,12 @@ function applyMove(rows, row, entry, resolved) {
|
||||
} else if (from === "in-progress" && to === "in-review") {
|
||||
if (row.claim === null || by !== row.claim.seat) throw refuse(`only the claimant (${row.claim?.seat ?? "nobody"}) may request review of row ${row.id}`);
|
||||
if (candidate === null) throw refuse("in-progress→in-review needs --candidate <commit|manifest>");
|
||||
+ revIssue = reviewIssue(row, issue);
|
||||
cand = checkCandidate(resolved.candidate);
|
||||
const rounds = row.review ? row.review.rounds : [];
|
||||
round = rounds.length + 1;
|
||||
next.review = {
|
||||
- issue: row.review ? row.review.issue : (row.issues[0] ?? null),
|
||||
+ issue: revIssue,
|
||||
rounds: [...rounds, { n: round, op: entry.op, by, at: entry.at, candidate: cand, request: "none" }],
|
||||
};
|
||||
} else if (from === "in-review" && (to === "in-progress" || to === "waiting-on-jason")) {
|
||||
@@ -479,6 +502,7 @@ function applyMove(rows, row, entry, resolved) {
|
||||
const ev = parseReviewEvidence(evidence);
|
||||
const cur = row.review?.rounds.at(-1);
|
||||
if (!cur) throw refuse(`row ${row.id} has no review round to cite`);
|
||||
+ if (ev.round !== cur.n) throw refuse(`evidence names round ${ev.round}; row ${row.id} is in round ${cur.n}`);
|
||||
if (ev.candidate !== cur.candidate.digest) throw refuse(`evidence candidate ${ev.candidate} is not round ${cur.n}'s candidate ${cur.candidate.digest}`);
|
||||
round = cur.n;
|
||||
next.claim = null;
|
||||
@@ -491,7 +515,7 @@ function applyMove(rows, row, entry, resolved) {
|
||||
throw illegal();
|
||||
}
|
||||
next = touch(next, entry);
|
||||
- return { row: next, result: { row: row.id, from, to, round, candidate: cand } };
|
||||
+ return { row: next, result: { row: row.id, from, to, round, issue: revIssue, candidate: cand } };
|
||||
}
|
||||
|
||||
function applySet(rows, row, entry, resolved) {
|
||||
@@ -588,7 +612,7 @@ export function applyEntry(state, entry, resolved) {
|
||||
const out = applyMove(rows, row, entry, resolved);
|
||||
rows.set(row.id, out.row);
|
||||
const r = out.result;
|
||||
- result = { ...r, receipt: receipt(entry, rev, `row ${row.id} ${r.from}→${r.to}${r.round ? ` round ${r.round}` : ""}`) };
|
||||
+ result = { ...r, receipt: receipt(entry, rev, `row ${row.id} ${r.from}→${r.to}${r.round ? ` round ${r.round}` : ""}${r.issue ? ` on #${r.issue}` : ""}`) };
|
||||
break;
|
||||
}
|
||||
case "release": {
|
||||
diff --git a/packages/queue/src/store.mjs b/packages/queue/src/store.mjs
|
||||
index 97f8784c..c3a66b8c 100644
|
||||
--- a/packages/queue/src/store.mjs
|
||||
+++ b/packages/queue/src/store.mjs
|
||||
@@ -242,7 +242,11 @@ function writeWitness(ctx, loc, doc, bytes) {
|
||||
unlinkQuiet(ctx.io, tmp);
|
||||
throw err;
|
||||
}
|
||||
- ctx.io.fsyncDir(loc.gitDir);
|
||||
+ try {
|
||||
+ ctx.io.fsyncDir(loc.gitDir);
|
||||
+ } catch (err) {
|
||||
+ throw Object.assign(new Error(errno(err)), { code: err.code, renamed: true });
|
||||
+ }
|
||||
}
|
||||
|
||||
function confirmTail(ctx, loc, cur) {
|
||||
@@ -316,6 +320,7 @@ function writeView(ctx, loc, before, parts, body, tag) {
|
||||
const now = readOrNull(ctx.io, loc.viewPath);
|
||||
if (now === null || !now.equals(before)) return stale;
|
||||
const tmp = `${loc.viewPath}.${tag}.tmp`;
|
||||
+ let renamed = false;
|
||||
try {
|
||||
const mode = Number(ctx.io.stat(loc.viewPath).mode & 0o777n);
|
||||
unlinkQuiet(ctx.io, tmp);
|
||||
@@ -326,9 +331,11 @@ function writeView(ctx, loc, before, parts, body, tag) {
|
||||
return stale;
|
||||
}
|
||||
ctx.io.rename(tmp, loc.viewPath);
|
||||
+ renamed = true;
|
||||
ctx.io.fsyncDir(loc.docsDir);
|
||||
return null;
|
||||
} catch (err) {
|
||||
+ if (renamed) return `the view is written but not confirmed durable (${errno(err)}); the op stands; after a host crash, check the table with \`${FIX} verify\``;
|
||||
unlinkQuiet(ctx.io, tmp);
|
||||
return `the view write failed (${errno(err)}); the op stands and the view is stale; run \`${FIX} render\``;
|
||||
}
|
||||
@@ -390,7 +397,8 @@ function commitWrite(ctx, loc, cur, doc, bytes, op, view, body, exclusive) {
|
||||
try {
|
||||
writeWitness(ctx, loc, doc, bytes);
|
||||
} catch (err) {
|
||||
- throw new QueueError(`uncertain ${op} rev ${rev}: durable, witness not updated (${errno(err)})`, 3);
|
||||
+ const what = err.renamed ? "witness written, its directory fsync failed" : "witness not updated";
|
||||
+ throw new QueueError(`uncertain ${op} rev ${rev}: durable, ${what} (${errno(err)})`, 3);
|
||||
}
|
||||
ctx.hook("witnessed");
|
||||
const warn = writeView(ctx, loc, view.bytes, view.parts, body, op);
|
||||
@@ -402,14 +410,19 @@ function withLock(ctx, loc, { op = null, verb }, fn) {
|
||||
checkPlatform(ctx.io, [loc.docsDir, loc.gitDir]);
|
||||
const handle = acquire({ gitDir: loc.gitDir, io: ctx.io, proc: ctx.proc, op, verb, waitMs: ctx.lockWaitMs, stepMs: ctx.lockStepMs, hook: ctx.hook });
|
||||
const res = { out: [], err: [], code: 0 };
|
||||
+ let failure = null;
|
||||
try {
|
||||
ctx.hook("locked");
|
||||
fn(res);
|
||||
- } finally {
|
||||
- let msg;
|
||||
- try { msg = release(handle, ctx.io); } catch (err) { msg = `cannot release the queue lock (${errno(err)})`; }
|
||||
- if (msg) res.err.push(`warning: ${msg}`);
|
||||
+ } catch (err) {
|
||||
+ failure = err;
|
||||
}
|
||||
+ let msg;
|
||||
+ try { msg = release(handle, ctx.io); } catch (err) { msg = `cannot release the queue lock (${errno(err)})`; }
|
||||
+ // A refusal still reports what release found (8.4).
|
||||
+ if (msg && failure instanceof Error) failure.message += `\nwarning: ${msg}`;
|
||||
+ else if (msg) res.err.push(`warning: ${msg}`);
|
||||
+ if (failure) throw failure;
|
||||
return res;
|
||||
}
|
||||
|
||||
@@ -763,5 +776,6 @@ export function unlock(opts, { checkGateOnly = false } = {}) {
|
||||
const ctx = makeCtx(opts);
|
||||
const loc = unlockLoc(ctx);
|
||||
if (checkGateOnly) return { out: [checkGate({ gitDir: loc.gitDir, io: ctx.io, proc: ctx.proc }).line], err: [], code: 0 };
|
||||
- return { out: [unlockLock({ gitDir: loc.gitDir, io: ctx.io, proc: ctx.proc, hook: ctx.hook })], err: [], code: 0 };
|
||||
+ const [line, ...warnings] = unlockLock({ gitDir: loc.gitDir, io: ctx.io, proc: ctx.proc, hook: ctx.hook }).split("\n");
|
||||
+ return { out: [line], err: warnings, code: 0 };
|
||||
}
|
||||
diff --git a/packages/queue/tests/commit.test.mjs b/packages/queue/tests/commit.test.mjs
|
||||
index 55105163..9a1b6ec1 100644
|
||||
--- a/packages/queue/tests/commit.test.mjs
|
||||
+++ b/packages/queue/tests/commit.test.mjs
|
||||
@@ -152,7 +152,12 @@ fi`);
|
||||
assert.equal(r.blob("HEAD", "src.txt"), "src\n");
|
||||
});
|
||||
|
||||
-test("F1: a commit whose guard ran before update-ref fails at its own HEAD update", async (t) => {
|
||||
+// A commit paused in its editor after its guard passed against H. Whether
|
||||
+// git holds index.lock during the editor depends on the form: git 2.55
|
||||
+// doesn't for plain `commit -e` and does for `commit -e -- path`. Step 8
|
||||
+// reconciles when the lock is free and exits 3 when it isn't; either way
|
||||
+// the paused commit loses at its own HEAD update.
|
||||
+async function pausedCommit(t, form) {
|
||||
const r = ready(t);
|
||||
note(r);
|
||||
stageFile(r, "src.txt");
|
||||
@@ -161,27 +166,38 @@ test("F1: a commit whose guard ran before update-ref fails at its own HEAD updat
|
||||
const go = join(r.ctl, "editor-go");
|
||||
const editor = join(r.ctl, "editor.sh");
|
||||
writeFileSync(editor, `#!/bin/sh\n: > ${q(started)}\nwhile [ ! -e ${q(go)} ]; do sleep 0.05; done\necho "ordinary" > "$1"\n`, { mode: 0o755 });
|
||||
- const child = spawn("git", ["-C", r.root, "commit", "-e", "-q"], { env: { ...r.env, GIT_EDITOR: editor }, stdio: ["ignore", "pipe", "pipe"] });
|
||||
+ const child = spawn("git", ["-C", r.root, "commit", "-e", "-q", ...form], { env: { ...r.env, GIT_EDITOR: editor }, stdio: ["ignore", "pipe", "pipe"] });
|
||||
let childErr = "";
|
||||
child.stderr.on("data", (d) => { childErr += d; });
|
||||
const exited = new Promise((resolve) => child.on("exit", resolve));
|
||||
for (let i = 0; i < 200 && !existsSync(started); i++) sleepMs(50);
|
||||
assert.ok(existsSync(started), "the editor never started");
|
||||
- // The paused commit ran its guard against H and holds index.lock.
|
||||
+ const locked = existsSync(join(r.gitDir, "index.lock"));
|
||||
+ t.diagnostic(`git commit -e${form.map((a) => ` ${a}`).join("")}: index.lock ${locked ? "held" : "free"} during the editor`);
|
||||
const res = r.qc(["-m", "queue rev 1"]);
|
||||
writeFileSync(go, "");
|
||||
const code = await exited;
|
||||
- // git 2.55 does not hold index.lock while the editor runs, so step 8
|
||||
- // reconciles; the paused commit then loses at its HEAD update.
|
||||
- assert.equal(res.code, 0, res.err);
|
||||
+ assert.equal(res.code, locked ? 3 : 0, `index.lock ${locked ? "held" : "free"} during the editor: ${res.err}`);
|
||||
+ if (locked) assert.match(res.err, /another git process holds \.git\/index\.lock/);
|
||||
const c = r.head();
|
||||
assert.equal(r.g("rev-parse", "HEAD^").trim(), h);
|
||||
+ assert.equal(r.revAt(c), 1);
|
||||
assert.notEqual(code, 0);
|
||||
assert.match(childErr, new RegExp(`cannot lock ref 'HEAD': is at ${c} but expected ${h}`));
|
||||
assert.equal(r.head(), c, "the old queue landed on top of C");
|
||||
+ if (locked) r.g("reset", "-q", "--", "docs/plans/queue.json", "docs/plans/QUEUE.md");
|
||||
assert.equal(r.g("diff", "--cached", "--name-only").trim(), "src.txt");
|
||||
r.g("commit", "-q", "-m", "ordinary");
|
||||
assert.equal(r.revAt("HEAD"), 1);
|
||||
+ return locked;
|
||||
+}
|
||||
+
|
||||
+test("F1: a plain `commit -e` whose guard ran before update-ref fails at its own HEAD update", async (t) => {
|
||||
+ await pausedCommit(t, []);
|
||||
+});
|
||||
+
|
||||
+test("F1: a `commit -e -- path` whose guard ran before update-ref fails at its own HEAD update", async (t) => {
|
||||
+ await pausedCommit(t, ["--", "src.txt"]);
|
||||
});
|
||||
|
||||
test("F1: step 8 with index.lock held exits 3, and ordinary commits stay refused until the printed command runs", (t) => {
|
||||
diff --git a/packages/queue/tests/data.test.mjs b/packages/queue/tests/data.test.mjs
|
||||
index 2e6c019f..99080d95 100644
|
||||
--- a/packages/queue/tests/data.test.mjs
|
||||
+++ b/packages/queue/tests/data.test.mjs
|
||||
@@ -3,8 +3,8 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import {
|
||||
- CALLER_OP_RE, LOG_OP_RE, applyEntry, buildDoc, canonArgs, classifyView, countHeading, genesisReceipt, genesisRows,
|
||||
- gitBlobId, loadDoc, nextFor, parseManifest, parseMigrationMap, render, rowsArray, serialize, sha256, splitView,
|
||||
+ CALLER_OP_RE, LOG_OP_RE, STATES, applyEntry, buildDoc, canonArgs, classifyView, countHeading, genesisReceipt, genesisRows,
|
||||
+ gitBlobId, loadDoc, nextFor, parseManifest, parseMigrationMap, render, rowsArray, serialize, sha256, splitView, validateRow,
|
||||
} from "../src/queue.mjs";
|
||||
import { QueueError } from "../src/errors.mjs";
|
||||
import { MAP_ROWS, mapText } from "./helpers.mjs";
|
||||
@@ -49,7 +49,7 @@ function refused(fn, re) {
|
||||
assert.throws(fn, (err) => err instanceof QueueError && err.code === 2 && re.test(err.message));
|
||||
}
|
||||
|
||||
-const mv = (id, to, extra = {}) => ({ id, to, reason: null, candidate: null, evidence: null, ...extra });
|
||||
+const mv = (id, to, extra = {}) => ({ id, to, reason: null, candidate: null, evidence: null, issue: null, ...extra });
|
||||
const row = (doc, id) => doc.rows.find((r) => r.id === id);
|
||||
const MANIFEST = `${"c".repeat(64)} packages/queue/src/queue.mjs\n`;
|
||||
const CAND = { kind: "manifest", digest: sha256(MANIFEST), text: MANIFEST };
|
||||
@@ -156,7 +156,7 @@ test("matrix: release, review round, changes requested and waiting-on-jason", ()
|
||||
const rv = row(d, 9).review;
|
||||
assert.equal(rv.issue, 1508);
|
||||
assert.deepEqual(rv.rounds.map((r) => [r.n, r.request, r.candidate.digest]), [[1, "none", CAND.digest]]);
|
||||
- assert.match(d.log.at(-1).result.receipt, /in-progress→in-review round 1$/);
|
||||
+ assert.match(d.log.at(-1).result.receipt, /in-progress→in-review round 1 on #1508$/);
|
||||
refused(() => step(d, "move", mv(9, "in-progress"), "dewey"), /claimed by darkwing/);
|
||||
d = step(d, "move", mv(9, "in-progress"), "darkwing");
|
||||
assert.equal(row(d, 9).claim.seat, "darkwing");
|
||||
@@ -176,9 +176,11 @@ test("matrix: release, review round, changes requested and waiting-on-jason", ()
|
||||
test("matrix J5: in-review→done by the gate owner with evidence naming the current round", () => {
|
||||
let d = row9Started();
|
||||
d = step(d, "move", mv(9, "in-review", { candidate: "x" }), "darkwing", { candidate: CAND });
|
||||
- const ev = `comment=4242,candidate=${CAND.digest}`;
|
||||
- refused(() => step(d, "move", mv(9, "done"), "filbert"), /--evidence comment=<id>,candidate=<digest>/);
|
||||
- refused(() => step(d, "move", mv(9, "done", { evidence: `comment=1,candidate=${"d".repeat(64)}` }), "filbert"), /is not round 1's candidate/);
|
||||
+ const ev = `comment=4242,round=1,candidate=${CAND.digest}`;
|
||||
+ refused(() => step(d, "move", mv(9, "done"), "filbert"), /--evidence comment=<id>,round=<n>,candidate=<digest>/);
|
||||
+ refused(() => step(d, "move", mv(9, "done", { evidence: `comment=4242,candidate=${CAND.digest}` }), "filbert"), /round=<n>/);
|
||||
+ refused(() => step(d, "move", mv(9, "done", { evidence: `comment=1,round=1,candidate=${"d".repeat(64)}` }), "filbert"), /is not round 1's candidate/);
|
||||
+ refused(() => step(d, "move", mv(9, "done", { evidence: `comment=4242,round=2,candidate=${CAND.digest}` }), "filbert"), /evidence names round 2; row 9 is in round 1/);
|
||||
refused(() => step(d, "move", mv(9, "done", { evidence: ev }), "rocko"), /only the gate owner \(filbert\)/);
|
||||
const done = step(d, "move", mv(9, "done", { evidence: ev }), "filbert");
|
||||
assert.equal(row(done, 9).state, "done");
|
||||
@@ -187,6 +189,155 @@ test("matrix J5: in-review→done by the gate owner with evidence naming the cur
|
||||
let e = step(genesisDoc(), "move", mv(11, "in-progress"), "dewey");
|
||||
e = step(e, "move", mv(11, "in-review", { candidate: "x" }), "dewey", { candidate: CAND });
|
||||
refused(() => step(e, "move", mv(11, "done", { evidence: ev }), "jason"), /gate is Jason's/);
|
||||
+ // Changes requested, then the same candidate again: round 1's comment
|
||||
+ // does not close round 2 (8.7, R3).
|
||||
+ d = step(d, "move", mv(9, "in-progress"), "darkwing");
|
||||
+ d = step(d, "move", mv(9, "in-review", { candidate: "x" }), "darkwing", { candidate: CAND });
|
||||
+ assert.deepEqual(row(d, 9).review.rounds.map((r) => r.candidate.digest), [CAND.digest, CAND.digest]);
|
||||
+ refused(() => step(d, "move", mv(9, "done", { evidence: ev }), "filbert"), /evidence names round 1; row 9 is in round 2/);
|
||||
+ const done2 = step(d, "move", mv(9, "done", { evidence: `comment=4343,round=2,candidate=${CAND.digest}` }), "filbert");
|
||||
+ assert.equal(done2.log.at(-1).result.round, 2);
|
||||
+});
|
||||
+
|
||||
+// A row 12 owned by darkwing with the given issues, started, so the next
|
||||
+// move is the review request.
|
||||
+function row12Started(issues, { gateOwner = "filbert" } = {}) {
|
||||
+ let d = step(genesisDoc(), "add", { piece: "N", gate: "g", brief: "docs/plans/brief-b.md#Queue", issues, owner: "darkwing", gateOwner }, "sage", { brief: brief() });
|
||||
+ d = step(d, "move", mv(12, "briefed"), "sage");
|
||||
+ return step(d, "move", mv(12, "in-progress"), "darkwing");
|
||||
+}
|
||||
+
|
||||
+const review = (d, extra = {}) => step(d, "move", mv(12, "in-review", { candidate: "x", ...extra }), "darkwing", { candidate: CAND });
|
||||
+const again = (d) => step(d, "move", mv(12, "in-progress"), "darkwing");
|
||||
+
|
||||
+test("review issue, lead decision 23: none refuses, one is used, several need --issue, later rounds keep it", () => {
|
||||
+ // No issues: refused before the round opens.
|
||||
+ refused(() => review(row12Started([])), /row 12 lists no issues; a privileged actor sets one before review/);
|
||||
+ // One issue: used without --issue; --issue may name it; any other refuses.
|
||||
+ const one = review(row12Started([1508]));
|
||||
+ assert.equal(row(one, 12).review.issue, 1508);
|
||||
+ assert.match(one.log.at(-1).result.receipt, /round 1 on #1508$/);
|
||||
+ assert.equal(one.log.at(-1).result.issue, 1508);
|
||||
+ refused(() => review(row12Started([1508]), { issue: 1495 }), /--issue #1495 is not one of row 12's issues \(#1508\)/);
|
||||
+ // Several: --issue is required and must be one of the row's; the lowest
|
||||
+ // number is not a default.
|
||||
+ const several = row12Started([1495, 1508]);
|
||||
+ refused(() => review(several), /row 12 lists several issues \(#1495, #1508\); name the review's issue with --issue/);
|
||||
+ refused(() => review(several, { issue: 1600 }), /--issue #1600 is not one of row 12's issues \(#1495, #1508\)/);
|
||||
+ let d = review(several, { issue: 1508 });
|
||||
+ assert.equal(row(d, 12).review.issue, 1508);
|
||||
+ // Later rounds keep the previous round's issue unless --issue names another.
|
||||
+ d = review(again(d));
|
||||
+ assert.deepEqual([row(d, 12).review.issue, row(d, 12).review.rounds.length], [1508, 2]);
|
||||
+ d = review(again(d), { issue: 1495 });
|
||||
+ assert.deepEqual([row(d, 12).review.issue, row(d, 12).review.rounds.length], [1495, 3]);
|
||||
+ // A kept issue the row no longer lists refuses until --issue names one.
|
||||
+ d = step(again(d), "set", { id: 12, field: "issues", value: [1508, 1600] }, "sage");
|
||||
+ refused(() => review(d), /review issue #1495 is no longer one of its issues \(#1508, #1600\); name one with --issue/);
|
||||
+ assert.equal(row(review(d, { issue: 1600 }), 12).review.issue, 1600);
|
||||
+ // --issue belongs to the review request only.
|
||||
+ refused(() => step(several, "move", mv(12, "blocked", { reason: "x", issue: 1508 }), "darkwing"), /--issue applies only to in-progress→in-review/);
|
||||
+});
|
||||
+
|
||||
+test("the row schema refuses a review with a null issue", () => {
|
||||
+ const r = structuredClone(row(review(row12Started([1508])), 12));
|
||||
+ validateRow(r);
|
||||
+ r.review.issue = null;
|
||||
+ refused(() => validateRow(r), /review issue must be a positive integer/);
|
||||
+});
|
||||
+
|
||||
+// R1: every state × target × actor class against 8.7's table, written from
|
||||
+// the spec rather than from queue.mjs. Row 12 is owned by darkwing; the
|
||||
+// gate owner is filbert, jason or the owner; rocko is any other seat.
|
||||
+const ACTORS = ["darkwing", "filbert", "rocko", "sage", "jason"];
|
||||
+const PRIV = new Set(["sage", "jason"]);
|
||||
+
|
||||
+function specAllows({ from, prev, to, by, gateOwner, required }) {
|
||||
+ const own = by === "darkwing";
|
||||
+ const priv = PRIV.has(by);
|
||||
+ if (from === "done") return false;
|
||||
+ if (to === "blocked") return ["queued", "briefed", "in-progress", "in-review", "waiting-on-jason"].includes(from) && (own || priv);
|
||||
+ if (from === "blocked") return to === prev && (own || priv);
|
||||
+ const edge = `${from}→${to}`;
|
||||
+ switch (edge) {
|
||||
+ case "queued→briefed": return priv;
|
||||
+ case "briefed→in-progress": return own;
|
||||
+ case "in-progress→in-review": return own; // the claimant is the owner
|
||||
+ case "in-review→in-progress": case "in-review→waiting-on-jason": return own || priv;
|
||||
+ case "waiting-on-jason→done": return by === "jason" || by === "sage"; // sage with evidence, which the case supplies
|
||||
+ case "in-review→done": return gateOwner !== "jason" && (by === gateOwner || priv);
|
||||
+ case "queued→parked": case "briefed→parked": return by === "jason" && !required;
|
||||
+ case "parked→queued": return by === "jason";
|
||||
+ default: return false; // in-progress→briefed is `release`, not `move`
|
||||
+ }
|
||||
+}
|
||||
+
|
||||
+function matrixStates(gateOwner, required) {
|
||||
+ let q = step(genesisDoc(), "add", {
|
||||
+ piece: "M", gate: "g", brief: "docs/plans/brief-b.md#Queue", issues: [1508], owner: "darkwing", gateOwner, required,
|
||||
+ }, "sage", { brief: brief() });
|
||||
+ const out = [{ from: "queued", doc: q }];
|
||||
+ if (!required) out.push({ from: "parked", doc: step(q, "move", mv(12, "parked"), "jason") });
|
||||
+ const b = step(q, "move", mv(12, "briefed"), "sage");
|
||||
+ const p = step(b, "move", mv(12, "in-progress"), "darkwing");
|
||||
+ const r = review(p);
|
||||
+ const w = step(r, "move", mv(12, "waiting-on-jason"), "sage");
|
||||
+ out.push({ from: "briefed", doc: b }, { from: "in-progress", doc: p }, { from: "in-review", doc: r }, { from: "waiting-on-jason", doc: w });
|
||||
+ out.push({ from: "done", doc: step(w, "move", mv(12, "done"), "jason") });
|
||||
+ for (const s of [...out]) {
|
||||
+ if (!["done", "parked"].includes(s.from)) out.push({ from: "blocked", prev: s.from, doc: step(s.doc, "move", mv(12, "blocked", { reason: "r" }), "sage") });
|
||||
+ }
|
||||
+ return out;
|
||||
+}
|
||||
+
|
||||
+function matrixArgs(from, to) {
|
||||
+ const extra = {};
|
||||
+ if (to === "blocked") extra.reason = "r";
|
||||
+ if (from === "in-progress" && to === "in-review") extra.candidate = "x";
|
||||
+ if (to === "done" && from === "in-review") extra.evidence = `comment=1,round=1,candidate=${CAND.digest}`;
|
||||
+ if (to === "done" && from === "waiting-on-jason") extra.evidence = "Jason approved in thread X";
|
||||
+ return mv(12, to, extra);
|
||||
+}
|
||||
+
|
||||
+test("matrix R1: every state × target × actor class matches 8.7, gate owner jason or not, required or not", () => {
|
||||
+ let allowed = 0;
|
||||
+ let refusals = 0;
|
||||
+ for (const gateOwner of ["filbert", "jason", "darkwing"]) {
|
||||
+ for (const required of [false, true]) {
|
||||
+ for (const { from, prev = null, doc } of matrixStates(gateOwner, required)) {
|
||||
+ assert.equal(row(doc, 12).state, from);
|
||||
+ for (const to of STATES) {
|
||||
+ for (const by of ACTORS) {
|
||||
+ const c = { from, prev, to, by, gateOwner, required };
|
||||
+ const label = JSON.stringify(c);
|
||||
+ let got;
|
||||
+ try {
|
||||
+ got = step(doc, "move", matrixArgs(from, to), by, { candidate: CAND });
|
||||
+ } catch (err) {
|
||||
+ assert.ok(err instanceof QueueError && err.code === 2, `${label}: ${err.stack}`);
|
||||
+ assert.equal(specAllows(c), false, `${label} refused: ${err.message}`);
|
||||
+ refusals++;
|
||||
+ continue;
|
||||
+ }
|
||||
+ assert.equal(specAllows(c), true, `${label} was allowed`);
|
||||
+ const r = row(got, 12);
|
||||
+ assert.equal(r.state, to, label);
|
||||
+ if (to === "blocked") assert.equal(r.previousState, from, label);
|
||||
+ if (from === "blocked") assert.deepEqual([r.previousState, r.blockedReason], [null, null], label);
|
||||
+ allowed++;
|
||||
+ }
|
||||
+ }
|
||||
+ // release: the claimant or a privileged actor, from in-progress only.
|
||||
+ for (const by of ACTORS) {
|
||||
+ const ok = from === "in-progress" && (by === "darkwing" || PRIV.has(by));
|
||||
+ const label = JSON.stringify({ release: from, by, gateOwner, required });
|
||||
+ if (ok) assert.deepEqual([row(step(doc, "release", { id: 12 }, by), 12).state], ["briefed"], label);
|
||||
+ else assert.throws(() => step(doc, "release", { id: 12 }, by), (err) => err instanceof QueueError && err.code === 2, label);
|
||||
+ }
|
||||
+ }
|
||||
+ }
|
||||
+ }
|
||||
+ assert.ok(allowed > 100 && refusals > 1000, `allowed ${allowed}, refused ${refusals}`);
|
||||
});
|
||||
|
||||
test("matrix: blocked keeps the claim and returns only to previousState", () => {
|
||||
@@ -301,7 +452,7 @@ test("next: resume, then review, then start, then wait, then nothing; lowest id
|
||||
let d = genesisDoc();
|
||||
const add = (owner, extra = {}) => ({ piece: `p-${owner}`, gate: "g", brief: "docs/plans/brief-b.md#Queue", owner, ...extra });
|
||||
d = step(d, "add", add("rocko"), "sage", { brief: brief() }); // 12
|
||||
- d = step(d, "add", add("rocko", { reviewers: ["darkwing"] }), "sage", { brief: brief() }); // 13
|
||||
+ d = step(d, "add", add("rocko", { reviewers: ["darkwing"], issues: [1508] }), "sage", { brief: brief() }); // 13
|
||||
d = step(d, "add", add("darkwing"), "sage", { brief: brief() }); // 14
|
||||
for (const id of [12, 13, 14]) d = step(d, "move", mv(id, "briefed"), "sage");
|
||||
const rows = () => loadDoc(Buffer.from(serialize(d))).state.rows;
|
||||
diff --git a/packages/queue/tests/lock.test.mjs b/packages/queue/tests/lock.test.mjs
|
||||
index 9b6b2174..f1b109d9 100644
|
||||
--- a/packages/queue/tests/lock.test.mjs
|
||||
+++ b/packages/queue/tests/lock.test.mjs
|
||||
@@ -89,6 +89,26 @@ test("a link error other than EEXIST refuses", (t) => {
|
||||
assert.deepEqual(readdirSync(d), []);
|
||||
});
|
||||
|
||||
+test("an error after the link releases the lock: unreadable gate, failing temp stat", (t) => {
|
||||
+ const d = dir(t);
|
||||
+ writeFileSync(join(d, GATE_NAME), record({ verb: "unlock", op: null }));
|
||||
+ const denied = { ...realIo, readFile: (p) => (p.endsWith(GATE_NAME) ? (() => { throw Object.assign(new Error("denied"), { code: "EACCES" }); })() : realIo.readFile(p)) };
|
||||
+ refused(() => acquire({ gitDir: d, io: denied, verb: "move" }), /cannot check the unlock gate .*EACCES; lock released$/, 1);
|
||||
+ assert.deepEqual(readdirSync(d), [GATE_NAME]);
|
||||
+ rmSync(join(d, GATE_NAME));
|
||||
+ const badStat = { ...realIo, stat: () => { throw Object.assign(new Error("io"), { code: "EIO" }); } };
|
||||
+ refused(() => acquire({ gitDir: d, io: badStat, verb: "move" }), /cannot write the lock record .*EIO; no lock taken/, 1);
|
||||
+ assert.deepEqual(readdirSync(d), []);
|
||||
+ // A stat that fails from its second call on: publish stats once, before the
|
||||
+ // link, so nothing after the link can fail and strand the lock.
|
||||
+ let stats = 0;
|
||||
+ const lateStat = { ...realIo, stat: (p) => { if (++stats > 1) throw Object.assign(new Error("io"), { code: "EIO" }); return realIo.stat(p); } };
|
||||
+ const h = acquire({ gitDir: d, io: lateStat, verb: "move" });
|
||||
+ assert.equal(stats, 1);
|
||||
+ assert.equal(release(h, realIo), null);
|
||||
+ assert.deepEqual(readdirSync(d), []);
|
||||
+});
|
||||
+
|
||||
test("a paused holder: another writer waits 10 s, then refuses naming it live", async (t) => {
|
||||
const d = dir(t);
|
||||
const child = spawn(process.execPath, [join(HERE, "fixtures", "lock-child.mjs"), d, "hold"], { stdio: ["ignore", "pipe", "ignore"] });
|
||||
@@ -150,6 +170,22 @@ test("a writer publishing during an unlock, gate first: the writer releases and
|
||||
assert.deepEqual(readdirSync(d), []);
|
||||
});
|
||||
|
||||
+test("a gate swapped while held is left in place and reported, on success and on refusal (N1)", (t) => {
|
||||
+ const d = dir(t);
|
||||
+ const gate = join(d, GATE_NAME);
|
||||
+ // A copy renamed over the gate: same bytes, a new inode.
|
||||
+ const swap = () => { writeFileSync(`${gate}.copy`, readFileSync(gate)); renameSync(`${gate}.copy`, gate); };
|
||||
+ const out = unlock({ gitDir: d, io: realIo, hook: (name) => { if (name === "gate-held") swap(); } });
|
||||
+ assert.match(out, /^no queue lock present; nothing removed\nwarning: lock .*mosaic-queue\.unlock is not the one this process took; left in place$/);
|
||||
+ assert.equal(existsSync(gate), true);
|
||||
+ rmSync(gate);
|
||||
+ writeFileSync(join(d, LOCK_NAME), record({}));
|
||||
+ refused(() => unlock({ gitDir: d, io: realIo, hook: (name) => { if (name === "gate-held") swap(); } }),
|
||||
+ /owner is live: .*; unlock refuses\nwarning: lock .*mosaic-queue\.unlock is not the one this process took; left in place$/);
|
||||
+ assert.equal(existsSync(gate), true);
|
||||
+ assert.equal(existsSync(join(d, LOCK_NAME)), true);
|
||||
+});
|
||||
+
|
||||
test("a reused pid within one boot is mismatch; unlock removes the lock and never signals the process", async (t) => {
|
||||
const d = dir(t);
|
||||
const s = await sleeper(t);
|
||||
@@ -177,6 +213,12 @@ test("a foreign host is unknown whatever the local pid says; unlock refuses", as
|
||||
refused(() => acquire({ gitDir: d, io: realIo, verb: "move", waitMs: 0 }), /unknown: .*recorded on host some-other-host; unlock refuses this too/);
|
||||
refused(() => unlock({ gitDir: d, io: realIo }), /owner is unknown: .*; unlock refuses/);
|
||||
assert.equal(existsSync(join(d, LOCK_NAME)), true);
|
||||
+ // A real foreign host has its own boot id. Host is tested before boot, so
|
||||
+ // this is still unknown, never mismatch, and unlock still refuses.
|
||||
+ writeFileSync(join(d, LOCK_NAME), record({ pid: await deadPid(), host: "some-other-host", boot: OTHER_BOOT }));
|
||||
+ assert.equal(classify(readFileSync(join(d, LOCK_NAME)), realProc).state, "unknown");
|
||||
+ refused(() => unlock({ gitDir: d, io: realIo }), /owner is unknown: .*recorded on host some-other-host/);
|
||||
+ assert.equal(existsSync(join(d, LOCK_NAME)), true);
|
||||
});
|
||||
|
||||
test("unreadable /proc: classification is unknown and acquire refuses", (t) => {
|
||||
diff --git a/packages/queue/tests/store.test.mjs b/packages/queue/tests/store.test.mjs
|
||||
index 7033b076..70425a73 100644
|
||||
--- a/packages/queue/tests/store.test.mjs
|
||||
+++ b/packages/queue/tests/store.test.mjs
|
||||
@@ -166,10 +166,28 @@ test("Rocko's S4 schedule: a lost result, another writer, then the retry opens n
|
||||
const review = ["move", "9", "in-review", "--candidate", "HEAD", "--op", "review-9-00001"];
|
||||
cli(repo, review, { by: "darkwing" }); // result lost
|
||||
ok(cli(repo, ["note", "9", "looking now", "--op", "note-9-000001"], { by: "filbert" }));
|
||||
- ok(cli(repo, review, { by: "darkwing" }), /round 1 \(already recorded at rev 3\)/);
|
||||
+ ok(cli(repo, review, { by: "darkwing" }), /round 1 on #1508 \(already recorded at rev 3\)/);
|
||||
assert.equal(row(repo, 9).review.rounds.length, 1);
|
||||
});
|
||||
|
||||
+test("the review issue and the evidence round through the CLI (lead decision 23, 8.7)", (t) => {
|
||||
+ const repo = ready(t);
|
||||
+ ok(cli(repo, ["move", "6", "blocked", "--reason", "paused", "--op", "block-6-00001"], { by: "darkwing" }));
|
||||
+ ok(cli(repo, ["set", "9", "issues", "1495,1508", "--op", "issues-9-0001"], { by: "sage" }));
|
||||
+ ok(cli(repo, ["move", "9", "in-progress", "--op", "start-9-00001"], { by: "darkwing" }));
|
||||
+ const review = (op, ...extra) => ["move", "9", "in-review", "--candidate", "HEAD", "--op", op, ...extra];
|
||||
+ no(cli(repo, review("review-9-00001"), { by: "darkwing" }), 2, /lists several issues \(#1495, #1508\); name the review's issue with --issue/);
|
||||
+ no(cli(repo, review("review-9-00001", "--issue", "1495", "--issue", "1508"), { by: "darkwing" }), 4, /one --issue/);
|
||||
+ no(cli(repo, review("review-9-00001", "--issue", "#1600"), { by: "darkwing" }), 2, /--issue #1600 is not one of row 9's issues/);
|
||||
+ ok(cli(repo, review("review-9-00001", "--issue", "#1508"), { by: "darkwing" }), /in-progress→in-review round 1 on #1508$/m);
|
||||
+ const head = repo.g("rev-parse", "HEAD").trim();
|
||||
+ no(cli(repo, ["move", "9", "done", "--evidence", `comment=7,candidate=${head}`, "--op", "done-9-000001"], { by: "filbert" }), 2, /round=<n>/);
|
||||
+ ok(cli(repo, ["move", "9", "in-progress", "--op", "changes-9-0001"], { by: "darkwing" }));
|
||||
+ ok(cli(repo, review("review-9-00002"), { by: "darkwing" }), /round 2 on #1508$/m);
|
||||
+ no(cli(repo, ["move", "9", "done", "--evidence", `comment=7,round=1,candidate=${head}`, "--op", "done-9-000001"], { by: "filbert" }), 2, /evidence names round 1; row 9 is in round 2/);
|
||||
+ ok(cli(repo, ["move", "9", "done", "--evidence", `comment=8,round=2,candidate=${head}`, "--op", "done-9-000002"], { by: "filbert" }), /in-review→done round 2$/m);
|
||||
+});
|
||||
+
|
||||
test("claims and add defaults through the CLI; candidates are manifests or reachable commits", (t) => {
|
||||
const repo = ready(t);
|
||||
ok(cli(repo, ["add", "--op", "add-by-dewey-1", "--piece", "Mine", "--gate", "tests", "--brief", "docs/plans/brief-b.md#Template"], { by: "dewey" }));
|
||||
diff --git a/packages/queue/tests/write.test.mjs b/packages/queue/tests/write.test.mjs
|
||||
index 0d0781c0..35e9829a 100644
|
||||
--- a/packages/queue/tests/write.test.mjs
|
||||
+++ b/packages/queue/tests/write.test.mjs
|
||||
@@ -3,7 +3,7 @@
|
||||
// racing a writer (8.4, F2).
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
-import { readFileSync, readdirSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
+import { readFileSync, readdirSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { test } from "node:test";
|
||||
import { cli, genesisCommitted, load, scratchRepo } from "./helpers.mjs";
|
||||
@@ -41,6 +41,7 @@ function faultIo(m, name, match, code) {
|
||||
return {
|
||||
...real,
|
||||
openExcl: (p, mode) => { const fd = real.openExcl(p, mode); paths.set(fd, p); return fd; },
|
||||
+ openRead: (p) => { const fd = real.openRead(p); paths.set(fd, p); return fd; },
|
||||
close: (fd) => { paths.delete(fd); real.close(fd); },
|
||||
write: (fd, b, off, len) => (hit("write", paths.get(fd)) ? (code === "SHORT" ? 0 : fail()) : real.write(fd, b, off, len)),
|
||||
fsync: (fd) => (hit("fsync", paths.get(fd)) ? fail() : real.fsync(fd)),
|
||||
@@ -114,6 +115,65 @@ test("a witness write failure: uncertain, durable, exit 3; the view is untouched
|
||||
assert.match(m.store.mutate(o(repo), note(9, "x", "note-9-000001")).out[0], /already recorded at rev 1/);
|
||||
});
|
||||
|
||||
+test("the .git fsync after the witness rename fails: uncertain, exit 3, the witness says so", async (t) => {
|
||||
+ const { repo, m } = await ready(t);
|
||||
+ const io = faultIo(m, "fsyncDir", (d) => d === repo.gitDir, "EIO");
|
||||
+ throwsCode(() => m.store.mutate(o(repo, { io }), note(9, "x", "note-9-000001")), 3, /^uncertain note-9-000001 rev 1: durable, witness written, its directory fsync failed \(EIO\)$/);
|
||||
+ assert.equal(revOf(repo), 1);
|
||||
+ assert.equal(witness(repo).revision, 1);
|
||||
+ assert.equal(shownRev(repo), 0);
|
||||
+ assert.match(m.store.mutate(o(repo), note(9, "x", "note-9-000001")).out[0], /already recorded at rev 1/);
|
||||
+});
|
||||
+
|
||||
+test("confirming a tail fsyncs queue.json and docs/plans before the witness; either failure changes nothing", async (t) => {
|
||||
+ const { repo, m } = await ready(t);
|
||||
+ const docs = join(repo.root, "docs/plans");
|
||||
+ throwsCode(() => m.store.mutate(o(repo, { io: faultIo(m, "fsyncDir", (d) => d === docs, "EIO") }), note(9, "x", "note-9-000001")), 3, /uncertain/);
|
||||
+ for (const io of [faultIo(m, "fsync", (p) => p === repo.queuePath, "EIO"), faultIo(m, "fsyncDir", (d) => d === docs, "EIO")]) {
|
||||
+ throwsCode(() => m.store.sync(o(repo, { io })), 1, /^cannot confirm rev 1 durable \(EIO\); nothing changed$/);
|
||||
+ assert.equal(witness(repo).revision, 0);
|
||||
+ }
|
||||
+ assert.match(m.store.sync(o(repo)).out.join("\n"), /durable now, never acknowledged: note-9-000001/);
|
||||
+ assert.equal(witness(repo).revision, 1);
|
||||
+});
|
||||
+
|
||||
+test("the docs/plans fsync after the view rename fails: the op stands, the view is written, a warning says so", async (t) => {
|
||||
+ const { repo, m } = await ready(t);
|
||||
+ const docs = join(repo.root, "docs/plans");
|
||||
+ let calls = 0;
|
||||
+ const io = { ...m.io.realIo, fsyncDir: (d) => { if (d === docs && ++calls === 2) throw Object.assign(new Error("EIO"), { code: "EIO" }); m.io.realIo.fsyncDir(d); } };
|
||||
+ const r = m.store.mutate(o(repo, { io }), note(9, "x", "note-9-000001"));
|
||||
+ assert.equal(calls, 2);
|
||||
+ assert.match(r.out[0], /^ok note-9-000001 rev 1/);
|
||||
+ assert.match(r.err.join("\n"), /the view is written but not confirmed durable \(EIO\); the op stands/);
|
||||
+ assert.equal(shownRev(repo), 1);
|
||||
+ assert.deepEqual(tmps(repo), []);
|
||||
+});
|
||||
+
|
||||
+test("a lock swapped while held is left in place and reported, on a receipt and on a refusal", async (t) => {
|
||||
+ const { repo, m } = await ready(t);
|
||||
+ const lock = join(repo.gitDir, "mosaic-queue.lock");
|
||||
+ // Another inode with the same bytes, as a delayed unlock and relock would leave.
|
||||
+ const swap = (name) => { if (name === "locked") { writeFileSync(`${lock}.copy`, readFileSync(lock)); renameSync(`${lock}.copy`, lock); } };
|
||||
+ const done = m.store.mutate(o(repo, { hook: swap }), note(9, "x", "note-9-000001"));
|
||||
+ assert.match(done.out[0], /^ok note-9-000001 rev 1/);
|
||||
+ assert.match(done.err.join("\n"), /warning: lock .* is not the one this process took; left in place/);
|
||||
+ unlinkSync(lock);
|
||||
+ throwsCode(() => m.store.mutate(o(repo, { hook: swap }), note(9, "y", "note-9-000002", "rocko")), 2,
|
||||
+ /may note row 9[^]*\nwarning: lock .* is not the one this process took; left in place$/);
|
||||
+ unlinkSync(lock);
|
||||
+});
|
||||
+
|
||||
+test("unlock prints a swapped gate's warning on stderr, the result on stdout", async (t) => {
|
||||
+ const { repo, m } = await ready(t);
|
||||
+ const gate = join(repo.gitDir, "mosaic-queue.unlock");
|
||||
+ const swap = (name) => { if (name === "gate-held") { writeFileSync(`${gate}.copy`, readFileSync(gate)); renameSync(`${gate}.copy`, gate); } };
|
||||
+ const r = m.store.unlock(o(repo, { hook: swap }));
|
||||
+ assert.deepEqual(r.out, ["no queue lock present; nothing removed"]);
|
||||
+ assert.match(r.err.join("\n"), /^warning: lock .*mosaic-queue\.unlock is not the one this process took; left in place$/);
|
||||
+ unlinkSync(gate);
|
||||
+});
|
||||
+
|
||||
test("a view write that fails keeps the op and reports a stale view", async (t) => {
|
||||
const { repo, m } = await ready(t);
|
||||
const io = faultIo(m, "rename", (p) => p === repo.viewPath, "EIO");
|
||||
@@ -1,39 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# N13 check (#1508): a failing nested `node --test` must fail test-foundation.sh
|
||||
# and test-discord.sh even when a parent runner's NODE_TEST_CONTEXT is set.
|
||||
# Runs in a scratch clone only: n13-check.sh CLONE (a clone of HEAD with
|
||||
# node_modules linked). It copies this checkout's two suites into the clone,
|
||||
# plants a failing test in each suite's test directory, and restores after.
|
||||
set -uo pipefail
|
||||
SRC="$(cd "$(dirname "$0")/../../../.." && pwd)"
|
||||
CLONE="${1:?usage: n13-check.sh CLONE}"
|
||||
[ "$(cd "$CLONE" && pwd)" != "$SRC" ] || { echo "refusing to run in the canonical checkout" >&2; exit 4; }
|
||||
cd "$CLONE" || exit 1
|
||||
|
||||
PLANT='import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
test("N13 planted failure", () => assert.equal(1, 2));'
|
||||
FAILS=0
|
||||
expect() { # NAME WANT GOT
|
||||
if [ "$2" = "$3" ]; then echo "ok $1 (rc $3)"; else echo "FAIL $1 (want rc $2, got $3)"; FAILS=$((FAILS+1)); fi
|
||||
}
|
||||
run() { # SUITE -> rc, output in /tmp/n13-<suite>-<tag>.txt
|
||||
NODE_TEST_CONTEXT=child-v8 NO_COLOR=1 "scripts/test-$1.sh" >"/tmp/n13-$1-$2.txt" 2>&1
|
||||
}
|
||||
|
||||
for pair in foundation:scripts/foundation discord:packages/discord/tests; do
|
||||
suite=${pair%%:*} dir=${pair#*:}
|
||||
git checkout -q -- "scripts/test-$suite.sh"
|
||||
printf '%s\n' "$PLANT" >"$dir/zz-n13-planted.test.mjs"
|
||||
run "$suite" head-planted; expect "$suite at HEAD, planted failure, parent context set" 0 $?
|
||||
cp -p "$SRC/scripts/test-$suite.sh" "scripts/test-$suite.sh"
|
||||
run "$suite" fixed-planted; expect "$suite fixed, planted failure, parent context set" 1 $?
|
||||
rm -f "$dir/zz-n13-planted.test.mjs"
|
||||
run "$suite" fixed-clean; expect "$suite fixed, no planted failure, parent context set" 0 $?
|
||||
sed -i 's/node_tests() { env -u NODE_TEST_CONTEXT node --test/node_tests() { node --test/' "scripts/test-$suite.sh"
|
||||
run "$suite" mutant-clean; expect "$suite with the clearing removed, no planted failure" 1 $?
|
||||
git checkout -q -- "scripts/test-$suite.sh"
|
||||
done
|
||||
git status --short
|
||||
echo "n13 check: $FAILS failed"
|
||||
[ "$FAILS" -eq 0 ]
|
||||
@@ -1,59 +0,0 @@
|
||||
# N13: nested `node --test` in two suites (#1508)
|
||||
|
||||
Darkwing, 2026-09-26. Filbert's note N13, put in DEFERRED by Sage as a
|
||||
small reviewed item before A2. Nothing is committed, staged or pushed.
|
||||
|
||||
## The defect
|
||||
|
||||
`scripts/test-foundation.sh:76` and `scripts/test-discord.sh:142` start a
|
||||
nested `node --test` without clearing `NODE_TEST_CONTEXT`. Under a parent
|
||||
test runner the nested run reports to that runner and exits 0 whatever its
|
||||
tests do. I reproduced it at HEAD 40a02d2b: with a failing test planted in
|
||||
each suite's test directory and `NODE_TEST_CONTEXT=child-v8` set, both
|
||||
suites exit 0. The check line reads `OK node --test ... (summary
|
||||
missing)`. In a control run of foundation without the variable, the same
|
||||
planted test fails the suite, so only a run under a parent runner is
|
||||
blind. I ran that control for foundation only.
|
||||
|
||||
## The change
|
||||
|
||||
`n13.patch` (sha256 00868b2f) changes only the two suites, +20 −2 lines.
|
||||
Each suite now has a `node_tests` function that runs `env -u
|
||||
NODE_TEST_CONTEXT node --test`, and uses it for its test run. Each also
|
||||
gets one new check. It writes a failing test into the sandbox, runs it
|
||||
through `node_tests` with `NODE_TEST_CONTEXT=child-v8` set, and requires
|
||||
exit 1 and `✖ planted failure` in the output. If someone drops the `env
|
||||
-u`, that check fails on every run, not only under a parent runner.
|
||||
|
||||
| File | sha256 |
|
||||
|---|---|
|
||||
| `scripts/test-foundation.sh` | 60f04822 |
|
||||
| `scripts/test-discord.sh` | 2ad3be74 |
|
||||
| `n13-check.sh` | 6a231759 |
|
||||
|
||||
## The check
|
||||
|
||||
`n13-check.sh CLONE` runs in a scratch clone only and refuses the
|
||||
canonical checkout. For each suite it plants a failing test in the real
|
||||
test directory and runs the whole suite with `NODE_TEST_CONTEXT=child-v8`:
|
||||
|
||||
| Suite | Case | Exit |
|
||||
|---|---|---|
|
||||
| foundation | HEAD, planted failure | 0 (the defect) |
|
||||
| foundation | fixed, planted failure | 1 |
|
||||
| foundation | fixed, no planted failure | 0 |
|
||||
| foundation | fixed but `env -u` removed, no planted failure | 1 |
|
||||
| discord | HEAD, planted failure | 0 (the defect) |
|
||||
| discord | fixed, planted failure | 1 |
|
||||
| discord | fixed, no planted failure | 0 |
|
||||
| discord | fixed but `env -u` removed, no planted failure | 1 |
|
||||
|
||||
In the fixed runs with the planted failure, the suite prints `FAIL node
|
||||
--test ...` with the pass count and the planted test's name. The last row
|
||||
of each is the mutation: the new check alone fails the suite.
|
||||
|
||||
All nine suites pass at 40a02d2b with this change and the queue A1
|
||||
candidate: foundation 44 and discord 64, one more check each than before.
|
||||
|
||||
I found no other nested `node --test` in the suites. `test-queue.sh` and
|
||||
`queue-commit.sh` already clear the variable.
|
||||
@@ -1,44 +0,0 @@
|
||||
diff --git a/scripts/test-discord.sh b/scripts/test-discord.sh
|
||||
index 6044100c..9ec4ddda 100755
|
||||
--- a/scripts/test-discord.sh
|
||||
+++ b/scripts/test-discord.sh
|
||||
@@ -139,7 +139,16 @@ else
|
||||
fi
|
||||
|
||||
# --- the seven offline groups ---
|
||||
-node --test --test-reporter=spec packages/discord/tests/ >"$SANDBOX/node-test.log" 2>&1
|
||||
+# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
|
||||
+# to that runner and exits 0 whatever its tests do, so the suite clears it
|
||||
+# (#1508 N13). The planted failing test proves a failure still fails here.
|
||||
+node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
|
||||
+mkdir -p "$SANDBOX/planted"
|
||||
+printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
|
||||
+NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
|
||||
+[ $? -eq 1 ] && grep -q '^✖ planted failure' "$SANDBOX/planted.log"
|
||||
+check "a failing nested test fails the run under a parent runner's NODE_TEST_CONTEXT" $?
|
||||
+node_tests --test-reporter=spec packages/discord/tests/ >"$SANDBOX/node-test.log" 2>&1
|
||||
NODE_RC=$?
|
||||
check "node --test packages/discord/tests/ ($(grep -E '^ℹ pass' "$SANDBOX/node-test.log" | tr -d '\n' || echo 'summary missing'))" $NODE_RC
|
||||
if [ "$NODE_RC" -ne 0 ]; then
|
||||
diff --git a/scripts/test-foundation.sh b/scripts/test-foundation.sh
|
||||
index 251eb674..b94dab5c 100755
|
||||
--- a/scripts/test-foundation.sh
|
||||
+++ b/scripts/test-foundation.sh
|
||||
@@ -73,7 +73,16 @@ done
|
||||
check "checked-in demo bundles equal a fresh generation" $DEMO_OK
|
||||
|
||||
# --- unit, CLI, privacy, non-effect and fixture-index tests ---
|
||||
-node --test scripts/foundation/ >"$SANDBOX/node-test.log" 2>&1
|
||||
+# A nested `node --test` inherits a parent runner's NODE_TEST_CONTEXT, reports
|
||||
+# to that runner and exits 0 whatever its tests do, so the suite clears it
|
||||
+# (#1508 N13). The planted failing test proves a failure still fails here.
|
||||
+node_tests() { env -u NODE_TEST_CONTEXT node --test "$@"; }
|
||||
+mkdir -p "$SANDBOX/planted"
|
||||
+printf '%s\n' 'import { test } from "node:test";' 'import assert from "node:assert/strict";' 'test("planted failure", () => assert.equal(1, 2));' >"$SANDBOX/planted/planted.test.mjs"
|
||||
+NODE_TEST_CONTEXT=child-v8 node_tests "$SANDBOX/planted/" >"$SANDBOX/planted.log" 2>&1
|
||||
+[ $? -eq 1 ] && grep -q '^✖ planted failure' "$SANDBOX/planted.log"
|
||||
+check "a failing nested test fails the run under a parent runner's NODE_TEST_CONTEXT" $?
|
||||
+node_tests scripts/foundation/ >"$SANDBOX/node-test.log" 2>&1
|
||||
NODE_RC=$?
|
||||
check "node --test scripts/foundation/ ($(grep -E '^ℹ pass' "$SANDBOX/node-test.log" | tr -d '\n' || echo 'summary missing'))" $NODE_RC
|
||||
[ "$NODE_RC" -ne 0 ] && grep -E "^✖|AssertionError" "$SANDBOX/node-test.log" | head -20
|
||||
@@ -1,159 +0,0 @@
|
||||
# Queue A1 (#1508), revision r1
|
||||
|
||||
Darkwing, 2026-09-26. This answers Filbert's review
|
||||
(`agents/filbert/work/queue-a1-review-2026-09-26.md`, sha256 6933b885) and
|
||||
Sage's lead decision 23 (40a02d2b). Nothing is committed, staged or pushed.
|
||||
|
||||
## Files
|
||||
|
||||
| File | sha256 | What it is |
|
||||
|---|---|---|
|
||||
| `delta-r1.patch` | b733b894 | the change on top of `build.patch` |
|
||||
| `build-manifest-r1.sha256` | 85a8a453 | all 20 files after the delta |
|
||||
|
||||
The delta changes 11 of the 20 files, +410 −49 lines. It adds no file and changes no mode.
|
||||
In a fresh clone at 3a209eea, `build.patch` and then `delta-r1.patch` apply
|
||||
cleanly, and the result matches the new manifest 20/20.
|
||||
|
||||
## Required changes
|
||||
|
||||
**R1, the matrix.** `data.test.mjs` has a new test, "matrix R1". Its
|
||||
oracle, `specAllows`, is written from 8.7's table, not from `queue.mjs`.
|
||||
The test replays a row into every reachable state and tries every target
|
||||
in `STATES` as five actors: darkwing (the row's owner), filbert, rocko,
|
||||
sage and jason. It does this with the gate owner as filbert, jason and the
|
||||
row's owner, and with the row required and not. Release gets the same
|
||||
treatment. That is 273 allowed moves and 2487 refusals, each compared with
|
||||
what `applyOp` does. Your R1 mutation and the agent's two survivors each
|
||||
fail it (R1a to R1c below).
|
||||
|
||||
**R2, the review issue, as lead decision 23 rules.** `move in-review`
|
||||
refuses a row with no issues. A row with one issue uses it. A row with
|
||||
several needs `--issue N`, and N must be one of them. A later round keeps
|
||||
the previous round's issue unless `--issue` names another. `--issue` is
|
||||
accepted only on in-progress→in-review, and giving it twice is a usage
|
||||
error (exit 4). The receipt now ends `round N on #ISSUE`.
|
||||
|
||||
The ruling didn't cover one case: a later round whose kept issue the row
|
||||
no longer lists, after a `set issues`. I refuse it until `--issue` names
|
||||
one of the row's issues. Falling back to the first issue would be the
|
||||
silent choice decision 23 replaced.
|
||||
|
||||
The row schema now refuses `review.issue: null`, so a hand-built file
|
||||
can't hold one either. `set piece` and `set gate` stay privileged only
|
||||
(decision 23, point 2); the code already did that, and nothing changed.
|
||||
|
||||
Tests: "review issue, lead decision 23" in `data.test.mjs` covers the four
|
||||
cases and a refused `--issue` that isn't in the row. "the review issue and
|
||||
the evidence round through the CLI" in `store.test.mjs` runs the same
|
||||
through the CLI. "the row schema refuses a review with a null issue"
|
||||
checks `validateRow`. Mutations D1 to D5.
|
||||
|
||||
**R3, the evidence round.** The format is now
|
||||
`comment=<id>,round=<n>,candidate=<digest>`. `move done` compares the
|
||||
round with the current one and refuses `evidence names round 1; row 12 is
|
||||
in round 2`. The J5 test refuses evidence with no round and with a wrong
|
||||
round, then sends a row back and re-requests it with the same candidate:
|
||||
round-1 evidence is refused and round-2 evidence closes it. Mutation E1.
|
||||
|
||||
## Notes I took
|
||||
|
||||
- **N1.** `withLock` appends the release warning to a refusal's message.
|
||||
`unlock` now does the same for the gate: a swapped gate is left in place
|
||||
and reported, on a refusal and on success. On success the CLI prints the
|
||||
warning on stderr and the result on stdout. Mutations W1, G1, G2, U1.
|
||||
- **N2.** In `acquire`, an error checking the gate releases the lock, then
|
||||
refuses with `cannot check the unlock gate ...; lock released`. In
|
||||
`publish`, the temp file's stat now runs before the link, so nothing
|
||||
that can fail runs between a successful link and the return. The test
|
||||
covers an unreadable gate, a stat that fails before the link, and a stat
|
||||
that fails from its second call on. The last case came late. L1 (a
|
||||
second stat after the link) survived my first mutation run, so I added
|
||||
it; L1 is now caught.
|
||||
- **N3.** `write.test.mjs` has three fault tests: the `queue.json` and
|
||||
`docs/plans` fsyncs in `confirmTail` (sync exits 1, nothing changes);
|
||||
the `.git` fsync after the witness rename (exit 3); the `docs/plans`
|
||||
fsync after the view rename (the op stands, a warning says the view
|
||||
isn't confirmed durable). Mutations F1 to F4.
|
||||
- **N4.** The foreign-host test adds a record with another boot id. It
|
||||
must still classify `unknown`. Mutation H1 swaps the two checks.
|
||||
- **N6.** The message now says `durable, witness written, its directory
|
||||
fsync failed` when the rename happened, and `witness not updated` only
|
||||
when it didn't.
|
||||
- **N9.** `BRIEF-TEMPLATE.md`: `briefed` when a privileged actor (jason or
|
||||
sage) accepts it; the owner can't.
|
||||
- **N14.** You were right. The paused-editor test is now `pausedCommit(t,
|
||||
form)`, which checks whether `index.lock` exists at the pause and
|
||||
asserts on that: exit 3 and `another git process holds .git/index.lock`
|
||||
when held, exit 0 when free. Either way the paused commit then fails
|
||||
with `cannot lock ref 'HEAD': is at C but expected H`. Two forms run on
|
||||
git 2.55.0: plain `commit -e` (the lock was free, exit 0) and `commit -e
|
||||
-- src.txt` (the lock was held, exit 3). The test no longer pins a git
|
||||
version, and the stale comment is gone.
|
||||
|
||||
## A correction to build.md
|
||||
|
||||
build.md says "`unlock` works on a missing or invalid lock file". That's
|
||||
wrong, as you said. It means a missing or invalid `queue.json`. `unlock`
|
||||
refuses an invalid lock. build.md stays as sent, since your review pins
|
||||
it.
|
||||
|
||||
## Notes not taken
|
||||
|
||||
N5, N7, N8, N10, N11, N12, N15 and N16. They don't block, and none is in
|
||||
the files this round had to touch for a reason. N8 (the `\` escape in
|
||||
`cell()`) and N11 (replay looser than the CLI on op ids) are cheapest
|
||||
before genesis. That's Sage's call; I can take them in A2.
|
||||
|
||||
## Verification
|
||||
|
||||
At 3a209eea plus the candidate (`/tmp/qa1-verify`): `test-queue.sh` 19
|
||||
checks, `node --test` 107/107 (data 22, lock 19, store 19, write 25,
|
||||
commit 22), `verify` skipped because HEAD has no `queue.json`.
|
||||
|
||||
At today's HEAD, 40a02d2b, plus the candidate and the N13 change
|
||||
(`/tmp/n13-verify`): config 24, task 90, foundation 44, conductor 17,
|
||||
release 14, auth 15, discord 64, extension-package 18, queue 19 with
|
||||
107/107. Foundation and discord each gained one check from N13. I reran
|
||||
the queue suite there after the last change; the other suites can't reach
|
||||
`packages/queue`.
|
||||
|
||||
The canonical `.git` is unchanged: `.git/hooks` holds only samples, no
|
||||
`mosaic-queue*` file, and `git config --show-scope --get-all
|
||||
core.hooksPath` returns nothing (rc 1). Every run was in a `--shared`
|
||||
clone under `/tmp`.
|
||||
|
||||
### Mutations
|
||||
|
||||
Each mutation went into the verify clone, the queue tests ran, and the
|
||||
file was restored from the candidate. All 20 files matched the candidate
|
||||
after each run. The number is how many tests failed.
|
||||
|
||||
| Id | Mutation | Failing tests |
|
||||
|---|---|---|
|
||||
| R1a | drop the owner check on unblock | 1 |
|
||||
| R1b | let the owner move waiting-on-jason→in-progress | 1 |
|
||||
| R1c | check the owner on block only when not queued | 1 |
|
||||
| D1 | allow review with no issues | 1 |
|
||||
| D2 | require `--issue` with one issue | 9 |
|
||||
| D3a | take the first of several issues | 2 |
|
||||
| D3b | accept an `--issue` the row doesn't list | 2 |
|
||||
| D4a | never keep the previous round's issue | 2 |
|
||||
| D4b | keep an issue the row no longer lists | 1 |
|
||||
| D5 | schema allows a null review issue | 1 |
|
||||
| E1 | ignore the evidence round | 2 |
|
||||
| L1 | stat the temp again after the link | 1 |
|
||||
| L2 | don't release the lock when the gate check fails | 1 |
|
||||
| F1 | drop `confirmTail`'s `queue.json` fsync | 1 |
|
||||
| F2 | drop `confirmTail`'s `docs/plans` fsync | 1 |
|
||||
| F3 | drop the `.git` fsync after the witness rename | 1 |
|
||||
| F4 | drop the `docs/plans` fsync after the view rename | 1 |
|
||||
| W1 | drop the release warning on a refusal | 1 |
|
||||
| H1 | check boot before host | 1 |
|
||||
| G1 | drop the gate warning on a refusal | 1 |
|
||||
| G2 | drop the gate warning on success | 2 |
|
||||
| U1 | print the gate warning nowhere in the CLI | 1 |
|
||||
|
||||
R1a to E1 ran before the last lock and store changes, which touch neither
|
||||
`queue.mjs` nor the tests that caught them. L1 to U1 ran on the final
|
||||
candidate. L1 first survived with 0 failures, as noted under N2.
|
||||
@@ -1,21 +0,0 @@
|
||||
b18afc13db533b5cbfac8701234b25dde87c9c1b4fda970e738faf87b7a6d324 packages/queue/README.md
|
||||
a1e72713398a3f706e4beb5ad012798ab6a55f2f76c2f3144db94011a45a3231 packages/queue/src/cli.mjs
|
||||
962d0e6548121933030980b937f55ca5913ee3859ff2b8888095bd273dbf561e packages/queue/src/io.mjs
|
||||
05682738482a56001bac168168a6aeb7e624f8ed1661a6c6592274f13da38c89 packages/queue/src/lock.mjs
|
||||
b87756a89c3bdeb03bddd1b1e51706ae93975b01f008237076ffe34557a71de3 packages/queue/src/queue.mjs
|
||||
b7708293363d33093a1106c2346c26ce5d48e052342abbec17c59ee058a4fb05 packages/queue/src/store.mjs
|
||||
70a11be54d8bddd3db0fd52555a1bdf481efef0cae671ee8e1aab546e3ddd36a packages/queue/tests/data.test.mjs
|
||||
b1c90f99eb2b5005ebc461dab5793164b80d7e846573f788606040ef88f5819e packages/queue/tests/helpers.mjs
|
||||
e7c9b23b6aa28321754c1c649f8bd5ee2c97bfea0d4e180ae699a4a1f5c60c76 packages/queue/tests/lock.test.mjs
|
||||
42120f0009815bedd3887b10e3630a694711825e4dee29a475733dbd212a49eb packages/queue/tests/store.test.mjs
|
||||
f79195520a1108ce5748ad6833eb03d2bb3ef97d74d83807ee1079e461ebb1f4 packages/queue/tests/write.test.mjs
|
||||
86c2a4f806e99fd09baf623dc7b3c95c1a630d147fe89c4fe5b93793fa4c5f50 scripts/mosaic
|
||||
b6aec15a75305bb48bca20a665bd919111a0071ffd7a1cc3631df2beaaac5a1b scripts/test-queue.sh
|
||||
dc32c0e0a865fe6b3623a808c4e9d8e8b284a3e74c36b3519554ea02d3a3ee5f packages/queue/tests/dispatch.test.mjs
|
||||
0d83ab1e73663de0f3da719078e7d756b8ec8f828ea63c2ce446bf21fb27c754 packages/queue/tests/migration.test.mjs
|
||||
3306b486be165b88c554ba13283cab4c79a5b67fbe3c2b6bfe72f987e1eddf07 packages/queue/tests/fixtures/genesis-render.md
|
||||
fb43d5855726e517f0eaacd626debb110e8ea6c948c97b35ad8971ec0885d151 packages/queue/tests/fixtures/mosaic-pre-a2.sh
|
||||
8f1927f3f11f260e71e7235a69efcbf10c8fd62bf263ddb89c3029f5e61833b7 packages/queue/tests/fixtures/queue-marked.md
|
||||
012ddce99d03932512d79b6ae7ba74786e050469e800cfb00fda683a2dfe42a8 agents/darkwing/work/queue-migration-map.md
|
||||
cdc49c447504f6c9f1c40da9ec0d89bf4d21c5e2a0730b1f12d1f20fbea63edb agents/darkwing/work/queue-a2/map-check.mjs
|
||||
b2a738f0da52783dc8e8a7c6033ce62582582e587eb6e5b6bde988ea4b406f3e agents/darkwing/work/queue-a2/carry-forward.md
|
||||
@@ -1,202 +0,0 @@
|
||||
# Queue A2 (#1508), candidate for review
|
||||
|
||||
Darkwing, 2026-09-27. A2 is migration, render and dispatch (lead decision
|
||||
20), plus every item Sage carried forward from A1's review (decision 26:
|
||||
N5, N8, N10, N11, N12, P1, P2, P3). Base is HEAD c9539baa; A1 is 34a72af9.
|
||||
Filbert reviews. Sage commits, then installs the hook and runs genesis.
|
||||
Nothing is committed, staged or pushed.
|
||||
|
||||
## Files
|
||||
|
||||
`build.patch` (sha256 `dc0be7ce74aaaaaf43deebe68af439d1b2b80c77aad6538420e929de35e2f21f`) changes 13 files and adds 8.
|
||||
`build-manifest.sha256` (sha256 `782bcb62e555659333a35888d418d986da63bdf522f38cafa447db7ddd0074b7`) pins all 21 after the patch.
|
||||
In a fresh clone at c9539baa the patch applies and the result matches
|
||||
the manifest 21/21, file modes included. `git apply` warns about one
|
||||
blank line at the end of `fixtures/genesis-render.md`. It belongs there:
|
||||
the render ends with one, and the golden has to match byte for byte.
|
||||
|
||||
- `packages/queue/src/`: `queue.mjs` (N8, N10, N11, P2), `store.mjs` (N12,
|
||||
N11's shared check, P1's helper, P3's caller), `lock.mjs` (P1, P3),
|
||||
`io.mjs` (N5), `cli.mjs` (usage comment).
|
||||
- `scripts/mosaic`: `queue` execs `packages/queue/src/cli.mjs` with the
|
||||
remaining arguments. Every other call reaches the seat CLI as before.
|
||||
- `scripts/test-queue.sh`: syntax checks for `scripts/mosaic` and the new
|
||||
fixture; `scripts/mosaic queue help` always; after genesis, `verify` and
|
||||
`render --check` on the live queue through `scripts/mosaic queue`.
|
||||
- `packages/queue/README.md`: the dispatch, N5, N7, N8, N10, N12, P2, the
|
||||
map and `map-check.mjs`, two new test files.
|
||||
- Tests: 107 in A1, 122 now. New: `dispatch.test.mjs` (3),
|
||||
`migration.test.mjs` (3), and 9 more in data, lock, store and write.
|
||||
Fixtures: `mosaic-pre-a2.sh` (the script before A2), `queue-marked.md`
|
||||
(QUEUE.md with the markers), `genesis-render.md` (the golden render).
|
||||
- `agents/darkwing/work/queue-migration-map.md`: the genesis input.
|
||||
- `agents/darkwing/work/queue-a2/map-check.mjs`: the drift check.
|
||||
- `agents/darkwing/work/queue-a2/carry-forward.md`: the item list Sage
|
||||
confirmed (773dbd75), with the r1 section added after it.
|
||||
|
||||
Outside the patch, for Sage to apply (condition 2 keeps them out of A2):
|
||||
|
||||
- `queue-md.patch` (sha256 `2ca8f689e1dcda5bb30e9af4c3f867242d5239d72e13001369c7b7aca7956402`): the two markers, a header that points
|
||||
seats at `scripts/mosaic queue next`, and a line freezing the old log of
|
||||
table changes. It applies to HEAD.
|
||||
- `tools-md.patch` (sha256 `c53aea1f6e871e17d04335ef60250adcdbe64f6466d9ff641042c00094d1c3a8`): a "Work queue" section in
|
||||
`docs/TOOLS.md`. It applies to HEAD. Optional; the README already has
|
||||
the detail.
|
||||
|
||||
## The five conditions
|
||||
|
||||
1. **Nothing ran against the canonical `.git`.** Tests use scratch
|
||||
repositories under the temp directory. The dry run below used
|
||||
`/tmp/qa2-dry`. Checked after all runs: no `mosaic-queue*` file in
|
||||
`.git/`, `.git/hooks/pre-commit` absent, `core.hooksPath` unset in every
|
||||
scope, nothing staged.
|
||||
2. **No QUEUE.md, AGENTS.md or TOOLS.md edits.** The two proposals are
|
||||
patch files. `git status` shows none of the three modified.
|
||||
3. **`scripts/test-queue.sh` is green at a HEAD with no `queue.json`:**
|
||||
24/24, the live checks skipped. After the dry-run genesis it ran 26/26, with `verify` and
|
||||
`render --check` on the live queue.
|
||||
4. **H is recorded before the canary.** `queue-commit.sh` is unchanged
|
||||
from A1, and its test "F1: H is recorded before the canary, so HEAD
|
||||
moving during the canary makes update-ref fail" passes.
|
||||
5. **The fault layer is reachable only from tests.** N5 narrows this
|
||||
further: tmpfs was the one fault-free path open to the CLI, and now
|
||||
only a layer with `allowTmpfs: true` gets it. The test asserts
|
||||
`realIo` has no such key and that `"yes"` doesn't count.
|
||||
|
||||
## Carried-forward items
|
||||
|
||||
| Item | Change | Test |
|
||||
|---|---|---|
|
||||
| N8 | `piece`, `gate`, `note`, move `reason`, brief anchor and `blockedReason` refuse `\` and `<`, at the CLI and in replay | "text the table shows refuses \ and <, everywhere it enters"; "every accepted text renders to nine cells on every row" (GFM's cell rule, no `marked` import) |
|
||||
| N11 | Replay holds every log entry, round op and claim op to `CALLER_OP_RE` and refuses `.outcome`, genesis and `accept-history` included. `LOG_OP_RE` stays for Piece D's derived ids; no verb derives one yet | "replay holds every op id to the caller's rule" |
|
||||
| N10 | `set issues` moves `closes` only if it equalled the old issues; a narrowed `closes` keeps its intersection, and the receipt says `(kept narrowed)` | "set issues keeps a logged narrowing of closes" |
|
||||
| P2 | Each round records `issue`; `review` is `{rounds}` only. A later round keeps the last round's issue | "the row schema refuses a round with a null issue, and the A1 review shape", and A1's R2 tests updated |
|
||||
| P1 | Both gate paths in `acquire` release through `releaseOrWarn`; a failed release is a message naming the lock, not a stack trace | "a release that fails on a gate path is reported, never a stack trace" |
|
||||
| P3 | `lock.unlock` returns `{result, warning}`; nothing splits on newlines | "unlock keeps a multi-line lock record on stdout" |
|
||||
| N5 | tmpfs left `FS_TYPES`; only `allowTmpfs === true` admits it | "tmpfs passes only a test layer that allows it" |
|
||||
| N12 | `--by` still wins; a different non-empty `MOSAIC_AGENT_NAME` adds a stderr warning on success and on refusal. Nothing is logged | "--by that differs from MOSAIC_AGENT_NAME warns on stderr and logs nothing more" |
|
||||
|
||||
The rest, as `carry-forward.md` records and decision 26 confirmed: N7 is a
|
||||
README line (the leftover `.git/mosaic-queue.lock.<pid>.<hex>.tmp` is
|
||||
removed by hand). N15, N16, N13-a and ext2/ext3 are won't-do.
|
||||
|
||||
## Migration
|
||||
|
||||
The map (`queue-migration-map.md`) is built from QUEUE.md blob c8e3d34e,
|
||||
which HEAD has. `node agents/darkwing/work/queue-a2/map-check.mjs` prints
|
||||
`ok: QUEUE.md matches the map (blob c8e3d34e)`. Run it again just before
|
||||
genesis. It exits 1 and lists the rows if QUEUE.md moved.
|
||||
|
||||
map-check did its job once already. HEAD moved from 8dba3ff7 to c9539baa
|
||||
while I worked, and it reported row 5 (the CHAT-03 brief pinned, lead
|
||||
decision 33). I rebuilt row 5's note, `queue-md.patch`, the marked fixture
|
||||
and the golden render against the new blob. No other row changed.
|
||||
|
||||
The map's "Choices Sage should check" has eight items. None blocks review.
|
||||
The ones that change what genesis writes: row 7 stays a live row (decision
|
||||
27), row 10's owner stays `coordinator` with `queue assign 10 sage`
|
||||
recommended as the first op, rows 12 and 13 have Sage as gate owner, row 16
|
||||
stays `waiting-on-jason` unless #1510 is closed, and rows 9 to 12 close
|
||||
nothing so row 13 closes #1508. Rows 9 to 13 have no `after`: decision 29
|
||||
took the wait on row 6 off them, and `after: 9 done` on row 10 would wait
|
||||
on a gate that needs row 10.
|
||||
|
||||
Dry run in a shared clone (`/tmp/qa2-dry`), the candidate plus
|
||||
`queue-md.patch` and `tools-md.patch` committed on top of c9539baa. I
|
||||
reran it from scratch after the rebase:
|
||||
|
||||
0. `map-check.mjs`: `ok: QUEUE.md matches the map (blob c8e3d34e)`.
|
||||
1. `scripts/test-queue.sh`: 24/24, live checks skipped.
|
||||
2. `scripts/queue-commit.sh --install-hook --by sage`: installed; canary
|
||||
passed.
|
||||
3. `scripts/mosaic queue genesis --root /tmp/qa2-dry --branch a2-dry --map
|
||||
… --op genesis-2026-09-27 --by sage`: `ok genesis-2026-09-27 rev 0
|
||||
genesis 30 rows`.
|
||||
4. `scripts/queue-commit.sh --genesis`: committed.
|
||||
5. `verify --current`: briefs match HEAD. `render --check`: current.
|
||||
`legacyView` equals the marked body byte for byte, and `mapBlob` is the
|
||||
map's blob.
|
||||
6. `next`: darkwing resumes 9, sage resumes 7, dewey resumes 5, filbert
|
||||
and rocko have nothing.
|
||||
7. `scripts/test-queue.sh`: 26/26 with the live checks.
|
||||
8. `queue assign 10 sage --op assign-10-dry --by sage`: `ok … rev 1 row 10 owner:
|
||||
coordinator→sage`.
|
||||
|
||||
The dry clone's hook is `.git/hooks/pre-commit`, mode 0755, blob
|
||||
abdf14e7. `core.hooksPath` is unset there too.
|
||||
|
||||
## Choices I made
|
||||
|
||||
- **N8 refuses instead of escaping.** An escape has to be right for every
|
||||
Markdown renderer that reads QUEUE.md; a refusal doesn't. No line in
|
||||
today's table has either character.
|
||||
- **P2 drops `review.issue`.** The last round's issue is the kept one, so
|
||||
a second copy could only disagree. Piece D reads `rounds[].issue`.
|
||||
- **N10's edge.** A `closes` narrowed to nothing stays empty whatever the
|
||||
new issues are. `set closes` with a reason is the way to widen it again.
|
||||
- **N12 warns on a refusal too,** so a seat that mistyped `--by` sees it on
|
||||
the error it was reading anyway.
|
||||
- **`scripts/mosaic queue` uses `exec`,** so exit codes and signals are the
|
||||
queue CLI's own. Any first argument other than exactly `queue` goes to
|
||||
the seat CLI, as before; the dispatch test compares argv, cwd,
|
||||
environment and stdin against the pre-A2 script.
|
||||
|
||||
## Mutations
|
||||
|
||||
Each mutation ran alone in a shared clone of the candidate, with
|
||||
`node --test packages/queue/tests/`. A killed mutation fails at least one
|
||||
test.
|
||||
|
||||
46 mutations. Each one below failed at least one test; the number is
|
||||
how many.
|
||||
|
||||
| Item | Mutations |
|
||||
|---|---|
|
||||
| N8 | backslash 2, lt 2, noteVerb 1, rowNote 2, anchor 1, reason 1 |
|
||||
| N11 | entryShape 1, outcome 2, genesisOnly 1, roundOp 1, claimOp 1 |
|
||||
| N10 | always 2, noIntersect 1, receipt 1 |
|
||||
| P2 | keptFirst 1, noCheck 1, reviewIssue 10 |
|
||||
| P1 | gateErr 1, gatePresent 1, gatePresentMsg 1, withLock 1 |
|
||||
| P3 | joined 2, storeSplit 1 |
|
||||
| N5 | inTypes 1, truthy 1, anyType 1 |
|
||||
| N12 | noWarnOk 1, noWarnErr 1, envWins 1, emptyEnv 1 |
|
||||
| Dispatch | noShift 1, unsetArg 1, prefix 1, noExec 1 |
|
||||
| Golden render | mapGate 1, mapState 1 |
|
||||
| map-check | fixed26 1, owner 1, issues 1, lineDiff 1, parkedPiece 1 |
|
||||
|
||||
Five of them survived the first pass. Each now has a test that kills it:
|
||||
|
||||
- N11 roundOp and claimOp (replay stops checking a review round's op or a
|
||||
claim's op). The N11 test now builds a claimed row and a reviewed row and
|
||||
refuses an op of 73 characters and one ending `.outcome` in each.
|
||||
- P1 withLock (the post-op release goes back to the throwing `release`).
|
||||
New write test: the lock's unlink fails with EACCES after the op, and the
|
||||
receipt and a refusal both carry "cannot release the queue lock (EACCES)".
|
||||
- N12 emptyEnv (an empty `MOSAIC_AGENT_NAME` warns). The N12 test now runs
|
||||
with it empty and expects no stderr.
|
||||
- Dispatch noExec (`node` without `exec`, so a successful queue call falls
|
||||
through to the seat CLI). The dispatch test now runs a queue call that
|
||||
exits 0 and checks that only the queue CLI ran.
|
||||
|
||||
The map and golden-render mutations ran again after the rebase and were
|
||||
still killed.
|
||||
|
||||
## Suites
|
||||
|
||||
All nine green at the final candidate: config 24, task 90, foundation
|
||||
44, conductor 17, release 14, auth 15, discord 64, extension-package 18,
|
||||
queue 24 (no `queue.json` at HEAD). `node --test packages/queue/tests/`:
|
||||
122/122.
|
||||
|
||||
## After approval (Sage)
|
||||
|
||||
1. Check the canonical tree against `build-manifest.sha256`, then commit
|
||||
the 21 files by path.
|
||||
2. Apply `queue-md.patch` (and `tools-md.patch` if wanted) and commit.
|
||||
3. `node agents/darkwing/work/queue-a2/map-check.mjs` must print `ok`.
|
||||
4. `scripts/queue-commit.sh --install-hook --by sage`.
|
||||
5. `scripts/mosaic queue genesis --root /mnt/storage/src/mosaic-stack
|
||||
--branch refactor --map agents/darkwing/work/queue-migration-map.md
|
||||
--op <id> --by sage`.
|
||||
6. `scripts/queue-commit.sh --genesis -m MSG`.
|
||||
7. `scripts/test-queue.sh` runs the live checks from here on.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,67 +0,0 @@
|
||||
# Queue A2 (#1508): items carried from A1 review
|
||||
|
||||
Darkwing, 2026-09-26. This file records Sage's ruling on A1 r1 so it isn't
|
||||
lost before A2 starts. A2's brief lists the two required items. A2's build
|
||||
note repeats each disposition. Note numbers are Filbert's, from
|
||||
`agents/filbert/work/queue-a1-review-2026-09-26.md` (sha256 6933b885).
|
||||
|
||||
## Required in A2, with tests
|
||||
|
||||
Both change what replay accepts or what the table shows, so they land
|
||||
before genesis. Genesis follows A2.
|
||||
|
||||
- **N8. `cell()` escapes `|` but not `\`.** Piece text `a\| done | x`
|
||||
renders so that `marked` splits it into an extra cell. Raw HTML passes
|
||||
through too.
|
||||
- The change: refuse `\` and `<` in rendered text fields at the CLI and
|
||||
in replay. A refusal holds up better than an escape we would have to
|
||||
get right for every Markdown renderer. None of the 34 table lines in
|
||||
today's QUEUE.md contains either character, so the migration loses
|
||||
nothing.
|
||||
- The test: every text field with `\`, `<` and `a\| done | x` is
|
||||
refused. A render of the allowed characters splits, by GFM's cell
|
||||
rule, into the same number of cells on every row. `marked` is only a
|
||||
transitive dependency in this repo, so the test doesn't import it.
|
||||
- The mutation: allow `\`, and the cell-count test must fail.
|
||||
- **N11. Replay is looser than the CLI on op ids.** `LOG_OP_RE` allows 80
|
||||
characters for any entry, `accept-history` may end in `.outcome`, and
|
||||
the genesis op isn't checked.
|
||||
- The change: replay applies `CALLER_OP_RE` to every op a caller chose.
|
||||
It allows the longer form only for the op ids the CLI derives. It
|
||||
refuses `.outcome` on `accept-history` and pattern-checks the genesis
|
||||
op.
|
||||
- The test: a hand-built file with each of the three refused shapes
|
||||
fails replay, and every op id the CLI writes still replays.
|
||||
- The mutations: restore each looser check in turn.
|
||||
|
||||
## Dispositions of the other notes
|
||||
|
||||
| Note | Disposition | Reason |
|
||||
|---|---|---|
|
||||
| N5 tmpfs accepted outside tests; `0xef53` also matches ext2 and ext3 | A2: tmpfs becomes a test-only option, like the other fault options. ext2 and ext3: won't do | `statfs` can't tell ext2, ext3 and ext4 apart. The README names ext4, and the canonical checkout is ext4. |
|
||||
| N7 temp files from killed acquires stay in `.git/` | A2: README line only | The files are small, carry the dead pid in their name, and never block a lock. Removing them safely needs the same liveness check `unlock` has, which isn't worth it for the space involved. |
|
||||
| N10 `set issues` resets `closes`, undoing a logged narrowing | A2: fix with a test | It changes replayed state, so it lands before genesis. New rule: `closes` becomes the new issues only if it equalled the old issues. Otherwise it keeps its intersection with the new issues, and the log entry says so. |
|
||||
| N12 `--by` silently overrides `MOSAIC_AGENT_NAME` | A2: stderr warning, no log field | Both values are self-asserted (J2), so a logged mismatch proves nothing a seat can't avoid. A warning catches the honest mistake, a typo or the wrong seat's shell. |
|
||||
| N15 `--install-hook --by` is self-asserted | Won't do | J2. It's protocol: Sage runs the install at bootstrap. A check on a claimed name adds nothing. |
|
||||
| N16 the hook refuses the first commit on an unborn HEAD | Won't do | The hook is installed only in the canonical checkout, which has history. It fails closed. |
|
||||
|
||||
Sage confirmed this file as written (773dbd75) on 2026-09-26. N5, N10 and
|
||||
N12 stay in A2. N10 has to land before genesis because it changes replayed
|
||||
state.
|
||||
|
||||
## From Filbert's r1 approval
|
||||
|
||||
Filbert approved A1 r1 and N13 on 2026-09-26 (review
|
||||
`agents/filbert/work/queue-a1-review-r1-2026-09-26.md`, sha256 e464be6c).
|
||||
He listed these as non-blocking and suitable for A2. The dispositions
|
||||
below are my proposal, and Sage rules on them.
|
||||
|
||||
| Note | Proposed disposition | Reason |
|
||||
|---|---|---|
|
||||
| P1 `acquire` calls `release()` unguarded on both gate paths | A2: fix with a test | If release throws (EACCES on `.git`), the CLI prints a stack trace and doesn't say the lock stayed. The fix guards it the way `withLock` does and names the lock left behind. |
|
||||
| P2 `--issue` on a later round overwrites `review.issue`; rounds don't record their own issue | A2: each round records its issue, with a test | Piece D posts per round, and the round's own issue is the evidence of where it posted. It changes the round schema and replayed state, so it lands before genesis, like N10. |
|
||||
| P3 `unlock` splits its result on newlines, so a hand-formatted record spills onto stderr | A2: fix with a test | `lock.mjs`'s `unlock` returns the result and the warning separately, so nothing is split on text. |
|
||||
| N13-a `n13-check.sh` copies the suites from the canonical tree, not from the pinned patch | Won't do | N13 is approved on `n13.patch` and the two suite hashes, and Filbert applied the patch himself. The check script is evidence, not shipped code. |
|
||||
|
||||
N13-b is Sage's: check the canonical working tree against both pins
|
||||
before committing from it.
|
||||
@@ -1,80 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
// Usage: node agents/darkwing/work/queue-a2/map-check.mjs [QUEUE.md] [MAP]
|
||||
//
|
||||
// Run before genesis. The map names the QUEUE.md blob it was built from.
|
||||
// This lists every table line that differs from that blob, then every row
|
||||
// whose piece, owner or issues no longer match the map. Reads only; the
|
||||
// one git call is `cat-file`. Exit 0 no drift, 1 drift, 4 usage.
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const top = join(dirname(fileURLToPath(import.meta.url)), "../../../..");
|
||||
const { parseMigrationMap } = await import(join(top, "packages/queue/src/queue.mjs"));
|
||||
|
||||
// Table lines by id: `| N | ...` rows, then the parked table's items
|
||||
// numbered on from the highest row, as the map numbers them.
|
||||
export function tableLines(text) {
|
||||
const out = new Map();
|
||||
const items = [];
|
||||
let parked = false;
|
||||
for (const line of text.split("\n")) {
|
||||
if (line.startsWith("## ")) parked = line.startsWith("## Parked");
|
||||
const m = /^\| (\d+) \|/.exec(line);
|
||||
if (m && !parked) out.set(Number(m[1]), line);
|
||||
else if (parked && line.startsWith("| ") && !line.startsWith("| Item ") && !line.startsWith("|---")) items.push(line);
|
||||
}
|
||||
let next = Math.max(0, ...out.keys()) + 1;
|
||||
for (const line of items) out.set(next++, line);
|
||||
return out;
|
||||
}
|
||||
|
||||
const cells = (line) => line.slice(2, -2).split(" | ");
|
||||
|
||||
export function check(queueText, mapText, oldText) {
|
||||
const drift = [];
|
||||
const now = tableLines(queueText);
|
||||
const then = tableLines(oldText);
|
||||
for (const id of new Set([...now.keys(), ...then.keys()])) {
|
||||
if (now.get(id) !== then.get(id)) drift.push(`row ${id}: ${!then.has(id) ? "added" : !now.has(id) ? "removed" : "changed"} since the map's QUEUE.md blob`);
|
||||
}
|
||||
const map = parseMigrationMap(mapText);
|
||||
const byId = new Map(map.rows.map((r) => [r.id, r]));
|
||||
for (const [id, line] of now) {
|
||||
const r = byId.get(id);
|
||||
if (!r) { drift.push(`row ${id}: in QUEUE.md, not in the map`); continue; }
|
||||
const c = cells(line);
|
||||
if (!/^\d+$/.test(c[0])) {
|
||||
if (c[0] !== r.piece) drift.push(`row ${id}: parked item ${JSON.stringify(c[0])} is not the map's piece`);
|
||||
continue;
|
||||
}
|
||||
if (c[1] !== r.piece) drift.push(`row ${id}: piece differs from the map`);
|
||||
const owner = /^[a-z][a-z0-9-]*/.exec(c[2])?.[0];
|
||||
if (owner !== r.owner) drift.push(`row ${id}: owner ${owner} in QUEUE.md, ${r.owner} in the map`);
|
||||
const issues = [...c[3].matchAll(/#(\d+)/g)].map((x) => Number(x[1]));
|
||||
if (issues.join() !== r.issues.join()) drift.push(`row ${id}: issues ${issues.join(",") || "none"} in QUEUE.md, ${r.issues.join(",") || "none"} in the map`);
|
||||
}
|
||||
for (const id of byId.keys()) if (!now.has(id)) drift.push(`row ${id}: in the map, not in QUEUE.md`);
|
||||
return drift;
|
||||
}
|
||||
|
||||
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length > 2) {
|
||||
console.error("usage: map-check.mjs [QUEUE.md] [MAP]");
|
||||
process.exit(4);
|
||||
}
|
||||
const queueText = readFileSync(args[0] ?? join(top, "docs/plans/QUEUE.md"), "utf8");
|
||||
const mapText = readFileSync(args[1] ?? join(top, "agents/darkwing/work/queue-migration-map.md"), "utf8");
|
||||
const blob = /QUEUE\.md` blob `([0-9a-f]{40})`/.exec(mapText)?.[1];
|
||||
if (!blob) {
|
||||
console.error("the map names no QUEUE.md blob");
|
||||
process.exit(1);
|
||||
}
|
||||
const oldText = execFileSync("git", ["-C", top, "cat-file", "blob", blob]).toString("utf8");
|
||||
const drift = check(queueText, mapText, oldText);
|
||||
for (const d of drift) console.log(d);
|
||||
console.log(drift.length ? `${drift.length} differences; update the map before genesis` : `ok: QUEUE.md matches the map (blob ${blob.slice(0, 8)})`);
|
||||
process.exit(drift.length ? 1 : 0);
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
diff --git a/docs/plans/QUEUE.md b/docs/plans/QUEUE.md
|
||||
index c8e3d34..dd155c6 100644
|
||||
--- a/docs/plans/QUEUE.md
|
||||
+++ b/docs/plans/QUEUE.md
|
||||
@@ -1,29 +1,32 @@
|
||||
# QUEUE — the one task list
|
||||
|
||||
-Read this first. One row per piece. Nobody needs to read the prose plans to
|
||||
-know what is next; the Brief column says which section to open only when you
|
||||
-are the owner of that row.
|
||||
+Read this first. One row per piece. The table between the markers is
|
||||
+rendered from `docs/plans/queue.json`, and `scripts/mosaic queue` is its only
|
||||
+writer. Don't edit the table by hand: `queue verify` and the commit hook
|
||||
+refuse a table that isn't the render. `packages/queue/README.md` has the verbs.
|
||||
|
||||
How to find your next thing:
|
||||
|
||||
-- **Jason**: the first row whose State starts with `Jason:`.
|
||||
-- **A seat**: the first row where Owner is you and State is `in progress`,
|
||||
- `in review` or `briefed`. If there is none, you have nothing; say so on the
|
||||
- board and stop.
|
||||
-- **Sage, project lead** (Jason's ruling 2026-09-26; Darkwing before that): update this table at every gate, before
|
||||
- anything else is written. CURRENT.md is the narrative log; this table wins
|
||||
+- **A seat**: run `scripts/mosaic queue next`. It names the row to resume,
|
||||
+ review or start, or says there is nothing; if nothing, say so on the board
|
||||
+ and stop.
|
||||
+- **Jason**: rows in state `waiting-on-jason`, and parked rows to reopen.
|
||||
+- **Sage, project lead** (Jason's ruling 2026-09-26): changes the queue with
|
||||
+ `scripts/mosaic queue` and commits `queue.json` with QUEUE.md through
|
||||
+ `scripts/queue-commit.sh`. CURRENT.md is the narrative log; the queue wins
|
||||
if they disagree.
|
||||
|
||||
-States: `queued` (no brief yet) → `briefed` (brief written, not started) →
|
||||
-`in progress` → `in review` → `Jason: <what he must do>` → `done`. `parked`
|
||||
-means not before the rows above it and not without Jason's say. `required`
|
||||
-means it cannot be parked or reordered below `queued` rows; only Jason moves it.
|
||||
+States: `queued` (brief exists, not accepted) → `briefed` → `in-progress` →
|
||||
+`in-review` → `waiting-on-jason` → `done`. `blocked` returns to the state it
|
||||
+left. `parked` rows wait for Jason to reopen them. A `required` row can't be
|
||||
+parked, and only Jason clears the flag. `after` names the rows a piece waits
|
||||
+for.
|
||||
|
||||
-Brief locations: "plan page" is `docs/plans/2026-09-12_control-board-mvp.md`.
|
||||
Gaps found while working go to `docs/plans/DEFERRED.md`, not here.
|
||||
|
||||
## Pieces (in order)
|
||||
|
||||
+<!-- mosaic-queue:begin -->
|
||||
Lead: Sage from 2026-09-26 (Jason's ruling); Darkwing is a collaborating seat.
|
||||
Jason is preparing the target for the next phase; until it arrives, the
|
||||
priority below stands.
|
||||
@@ -79,8 +82,13 @@ Gate F or when blocked."
|
||||
| Console features outside the refined session-chat brief, including Fresh creation and model switching | Deferred by WEBUI Q1; required history/control/stop now belong to row 5, not this parked item | `2026-09-13_webui-session-chat.md` |
|
||||
| Open gaps from the MVP work | Fixed only when a gate needs them | DEFERRED.md, "Open" |
|
||||
|
||||
+<!-- mosaic-queue:end -->
|
||||
+
|
||||
## Log of table changes
|
||||
|
||||
+Frozen at genesis. Since then the log in `docs/plans/queue.json` records every
|
||||
+change; the entries below are history.
|
||||
+
|
||||
- 2026-09-13 — created; rows 1 to 8 taken from CURRENT.md, DEFERRED.md and the plan page.
|
||||
- 2026-09-13 — rows 9 to 13 added (#1508): the process itself becomes data with one writer and ledger checks. Jason: "I want this iron-clad." Required, not parked; rule: these rows cannot be moved to parked, only to done.
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
diff --git a/docs/TOOLS.md b/docs/TOOLS.md
|
||||
index 5f993af0..02e0de9e 100644
|
||||
--- a/docs/TOOLS.md
|
||||
+++ b/docs/TOOLS.md
|
||||
@@ -147,6 +147,24 @@ npm-global `mosaic` CLI; run by path. Exit codes: the launch script's own
|
||||
once it runs; before that 1 could not start, 2 invalid config or seat, 4
|
||||
usage. Details and the record's fields: `packages/seat/README.md`.
|
||||
|
||||
+## Work queue (`scripts/mosaic queue`)
|
||||
+
|
||||
+```bash
|
||||
+scripts/mosaic queue list | show ID | next [SEAT]
|
||||
+scripts/mosaic queue add|move|release|assign|note|set ... --op ID [--by NAME]
|
||||
+scripts/mosaic queue verify [--current] | render [--check] | sync | unlock [--check-gate]
|
||||
+scripts/queue-commit.sh -m MSG
|
||||
+```
|
||||
+
|
||||
+`docs/plans/queue.json` holds the rows and an append-only log; the table in
|
||||
+`docs/plans/QUEUE.md` between the `mosaic-queue` markers is rendered from it.
|
||||
+Canonical checkout only. Every change needs an `--op ID` chosen before the
|
||||
+first attempt and reused on retry; only an op whose `ok <op> rev N` receipt
|
||||
+printed is done. The lead commits queue changes with `scripts/queue-commit.sh`;
|
||||
+the pre-commit guard refuses any other commit that stages the two queue files.
|
||||
+Exit codes: `0` ok · `1` failed · `2` invalid or refused · `3` uncertain,
|
||||
+retry the same op · `4` usage. Details: `packages/queue/README.md`.
|
||||
+
|
||||
## Discord connector (`scripts/discord.sh`)
|
||||
|
||||
```bash
|
||||
@@ -1,247 +0,0 @@
|
||||
# Queue D (#1508, row 12), candidate for review, round 2
|
||||
|
||||
Darkwing, 2026-09-27. Piece D is review requests, per plan section 8.9
|
||||
(`agents/filbert/work/queue-as-data-plan-2026-09-26.md`). Moving a row
|
||||
with reviewers to in-review posts one request comment on its issue as the
|
||||
acting seat, and the log records what happened. The row closes on the
|
||||
reviewers' recorded verdicts. The candidate also carries the DEFERRED
|
||||
item Sage attached to rows 12 and 13: `test-queue.sh` skips its live checks
|
||||
outside the canonical root.
|
||||
|
||||
Base is 8ffbd73b. The patch applies unchanged to HEAD cdcedb27, which
|
||||
since then has changed only `lead-decisions.md`, `QUEUE.md` and
|
||||
`queue.json`. Filbert reviews D. The helper patch is separate
|
||||
(`helper.md`); Sage approved it in lead decision 39, and it ships in the
|
||||
D commit. Sage commits. Nothing is committed, staged or pushed.
|
||||
|
||||
Round 2 fixes Filbert's C1
|
||||
(`agents/filbert/work/queue-d-review-r1-2026-09-27.md`): on a request
|
||||
round, in-review→waiting-on-jason now makes the same checks as
|
||||
in-review→done. See "Waiting-on-jason on a request round" below.
|
||||
|
||||
## Files
|
||||
|
||||
`build.patch` (sha256
|
||||
`28d0790e797e1f24f295f70b2e0162bb1b4b0a154b220c0527f7d6624f0aa1c8`) changes 7 files and adds 3.
|
||||
`build-manifest.sha256` (sha256
|
||||
`af57ead2697a88d74a9214783dfa35ba3bd171aa32a594584f018181e261e956`) pins all 10 after the
|
||||
patch. In a fresh clone at cdcedb27 the patch applies and the result
|
||||
matches the manifest 10/10.
|
||||
|
||||
- `packages/queue/src/review.mjs` (new): the credential check, the request
|
||||
body, the helper call under the deadline, and the reading of each
|
||||
answer.
|
||||
- `packages/queue/src/queue.mjs`: `SEMANTICS` 2, the five review verbs,
|
||||
the round, attempt and receipt shapes, and the rules for moves and done.
|
||||
- `packages/queue/src/store.mjs`: the request step after the move is
|
||||
logged, the resolve check, `review verify-commit`, and the retry and
|
||||
settle hints.
|
||||
- `packages/queue/src/cli.mjs`: the `review` subcommands.
|
||||
- `packages/queue/README.md`: a "Review requests" section, verbs, exit 3,
|
||||
known limits, tests.
|
||||
- `scripts/test-queue.sh`: the live checks skip outside the canonical root.
|
||||
- Tests: `review.test.mjs` (new, 20 tests), `fixtures/fake-gitea.mjs`
|
||||
(new), `fixtures/kill-at.mjs` (a step can be `NAME#N`, the Nth time it is
|
||||
reached), `store.test.mjs` (two A2 tests set row 9's reviewers to none
|
||||
first, so their moves post nothing; a note there moves from filbert to
|
||||
sage, since filbert is no longer a reviewer). 122 tests at A2, 142 now.
|
||||
|
||||
Outside the patch:
|
||||
|
||||
- `helper.patch` (sha256 `48edd46b93c54908e9d59737aab78a33e007ddfafa1e6ad4d34333015bb75430`): raw per-seat token files in
|
||||
`scripts/gitea-api.sh`, lead decision 37. Rocko reviewed two rounds,
|
||||
and Sage approved the result (decisions 38 and 39); notes in
|
||||
`helper.md`. D's tests don't need it, but the live round does.
|
||||
- `tools-md.patch` (sha256 `d30d65b842bd56077ccd9164eccd946f2aa9f53290d02eba61f284960d230e14`): `docs/TOOLS.md`, the review
|
||||
commands and the raw token file. It is Sage's file, so it's a proposal.
|
||||
|
||||
## How a request goes
|
||||
|
||||
1. **Intent.** `move ID in-review --candidate C --op OP` on a row with
|
||||
reviewers logs the round and its first attempt, `requesting`, under the
|
||||
lock, like any other op. If that write doesn't finish, nothing is sent.
|
||||
2. **Pre-send.** Without the lock: the credential file check (`lstat`
|
||||
only), then `GET user`, which must return the acting seat's login, or
|
||||
`jarvis` for sage. A failure here is `failed`, and the POST never runs.
|
||||
3. **POST** one comment on the round's issue through
|
||||
`scripts/gitea-api.sh`, under `timeout -s KILL 30`. The body carries
|
||||
two markers: `<!-- mosaic-queue-op: OP -->` and
|
||||
`<!-- mosaic-queue-round: row=N round=R candidate=DIGEST -->`.
|
||||
4. **Outcome.** A second entry, `OP.outcome`, under the lock. It holds the
|
||||
HTTP status, the comment id and a fixed detail string. Nothing from the
|
||||
response body is logged or printed.
|
||||
|
||||
Exit 0 is posted, 1 is failed (400, 401, 403, 404, 422, or pre-send), and
|
||||
3 is uncertain (anything else). A same-op retry prints the logged state and
|
||||
sends nothing. An uncertain or `requesting` attempt prints where to look
|
||||
and the two commands that settle it.
|
||||
|
||||
## Choices I made
|
||||
|
||||
- **The CLI shape of `resolve`.** The plan has
|
||||
`review resolve ID --attempt REQOP --posted <id>`. I made it
|
||||
`review resolve ID REQOP --comment N`, and abandon takes the attempt the
|
||||
same way. The attempt is always required, and `--comment` is the flag
|
||||
name `record` uses for a comment id too.
|
||||
- **Resolve checks the author.** The plan lists issue, marker, round and
|
||||
candidate. I added the comment's author: it must be the requester's
|
||||
login, fetched with the resolver's own token. Otherwise anyone could
|
||||
copy the markers into a comment of their own.
|
||||
- **Done on a request round** needs an approval recorded by every listed
|
||||
reviewer in the current round, and refuses `--evidence`. If the reviewers
|
||||
were removed after the round opened, done refuses until a privileged
|
||||
actor sets them. A round from before D, or on a row with no reviewers,
|
||||
closes with `--evidence` as in A2.
|
||||
- **Waiting-on-jason on a request round** (round 2, C1). The plan's table
|
||||
let the owner move in-review→waiting-on-jason with no other check, so a
|
||||
Jason-gated row could reach Jason, and then close, with no reviewer's
|
||||
verdict. The move now refuses while a request is unresolved, and on a
|
||||
request round it needs an approval recorded by every listed reviewer.
|
||||
`requireApprovals` in `queue.mjs` holds the approval checks that done
|
||||
and this move share. A round with no request is unchanged, so v1 replay
|
||||
is too. Filbert approved round 2 and asked for one more check (n1): an
|
||||
approval from an earlier round doesn't count in the current one. The
|
||||
Jason-gated test now has rocko approve round 2 first, and the move
|
||||
refuses until filbert approves round 2 as well. Jason gets no exemption, because the owner makes this move.
|
||||
In-review→in-progress, the changes path, is unchanged.
|
||||
- **The owner records no verdict,** even when listed as a reviewer.
|
||||
- **A late outcome on a done row.** `review-outcome` is the one entry a
|
||||
done row accepts, so a POST that answers after the row closed is still
|
||||
recorded. It can only set `conflict`; nothing reopens the row, and
|
||||
`review resolve` on it refuses without fetching the comment.
|
||||
- **A late `uncertain` after a resolve** keeps `posted`: the resolve saw
|
||||
the comment. A late `failed` after a resolve is `conflict`, because the
|
||||
server said it refused a comment someone found.
|
||||
- **Semantics per entry.** Every log entry already records `semantics`.
|
||||
Entries at 1 replay under A2's rules, so the live log (rev 13, all
|
||||
semantics 1) loads unchanged. Review verbs need 2, checked in the entry
|
||||
shape and again in apply.
|
||||
- **Body limit.** A body over 60,000 bytes is a pre-send `failed`, and
|
||||
nothing is sent. A 700-line manifest is enough to reach it.
|
||||
- **`verify-commit` on a commit candidate** compares every path the
|
||||
candidate changed against its first parent, by blob and mode, and
|
||||
requires the paths it deleted to be absent in REF. A `diff-tree` line it
|
||||
can't parse refuses with exit 1 instead of being skipped.
|
||||
- **The claim refusal message.** `ownerOrPriv` read "darkwing cannot
|
||||
resolve a request on". It now names the owner and the privileged actors
|
||||
in every case, with the claim first when there is one. No A2 test
|
||||
matched the old text; the resolve test pins the new one.
|
||||
- **`test-queue.sh`.** After genesis it reads `canonicalRoot` from
|
||||
`HEAD:docs/plans/queue.json` and compares it with the real path of the
|
||||
toplevel. If they differ, it prints
|
||||
`skip queue verify and render --check: this checkout (TOP) is not the queue's canonical root (CANON)`
|
||||
and passes. An empty `canonicalRoot` is a failure.
|
||||
|
||||
## Tests
|
||||
|
||||
`review.test.mjs` runs each case in a scratch repository with genesis
|
||||
committed. `scripts/gitea-api.sh` there is a stub that runs
|
||||
`fixtures/fake-gitea.mjs`: same argv and output as the helper, calls
|
||||
logged to a file, rules from a scenario file, and posted comments served
|
||||
back by id. The token files are dummies under the scratch directory. No
|
||||
test reads a real token or opens `~/.mosaic` or `~/.t3`.
|
||||
|
||||
Against the test list in 8.9:
|
||||
|
||||
| 8.9 asks for | Test |
|
||||
|---|---|
|
||||
| a kill before the POST, after it, at the outcome write, while retaking the lock | "a same-op retry after a kill sends nothing, even with a stale view" (steps `pre-send`, `posted`, `outcome`, `locked#2`), and "a held lock at the outcome exits 3" |
|
||||
| posted; each listed 4xx; 5xx; request failed; timeout; 201 without an id | "each transport answer maps to posted, failed or uncertain" (deadline 300 ms) |
|
||||
| a new op while `requesting` or `uncertain` refuses | "an unresolved request blocks a new request, a new round, waiting-on-jason and done", which also reaches waiting-on-jason with a late POST still out, and Jason's close refuses |
|
||||
| (round 2, C1) a Jason-gated row skips its reviewers | "a Jason-gated row reaches waiting-on-jason only on every reviewer's approval": Filbert's steps refuse, then pass after both reviewers approve a later round; reviewers removed refuse too |
|
||||
| a same-op retry after a stale view sends nothing | the kill test |
|
||||
| a late POST after abandon; a late outcome after resolve, same id and another | "late outcomes": rounds 1 to 3, plus a late 500 (stays posted) and a late 422 (conflict); "a late POST on a closed row" (abandoned, approved and closed while the POST was out) |
|
||||
| resolve with the wrong issue, marker, round or candidate | "resolve checks the comment", which adds author, id, 404, 500, a transport failure and the wrong credential |
|
||||
| `GET user` mismatch and timeout | "the pre-send checks" |
|
||||
| the lead posts as jarvis; `sage` as a login refuses | "the pre-send checks" and "the lead's request refuses a token for login sage" |
|
||||
| the credential checks: mode, another seat's path, the default file | "the credential file" (also unset, relative, missing, symlink, a linked directory, and jarvis's path for sage) |
|
||||
| request, changes, new candidate, approval, pinned | "request, changes, a new candidate, approval", which also checks no file lands under `docs/plans/reviews/` or `agents/*/work/` |
|
||||
|
||||
Every test that retries, kills or answers late counts the POSTs the fake
|
||||
saw. None sees a second POST for one attempt.
|
||||
|
||||
## Mutations
|
||||
|
||||
61 hand-written mutants: 29 in `queue.mjs`, 16 in `store.mjs`, 16 in
|
||||
`review.mjs`. Each runs alone against the whole queue test directory.
|
||||
Against the round 2 tests, 59 are killed, the same result as round 1.
|
||||
|
||||
Round 2 adds six in `queue.mjs` for C1, run against the round 2 tests.
|
||||
All six are killed:
|
||||
|
||||
- the round 1 branch restored (waiting-on-jason checks only the actor);
|
||||
- the approval check dropped from waiting-on-jason;
|
||||
- the unresolved check dropped from waiting-on-jason;
|
||||
- the unresolved check dropped from waiting-on-jason→done;
|
||||
- the no-reviewers check dropped from `requireApprovals`;
|
||||
- the approval check applied to every round, not only request rounds.
|
||||
|
||||
Seven survived earlier runs. Five of them now have a test that kills them:
|
||||
a second outcome for one attempt, the owner recording a verdict, done on a
|
||||
row whose reviewers were removed, a same-op resolve retry that fetched the
|
||||
comment again, and a resolve on a done row that fetched the comment before
|
||||
the log refused it. The last one is the new test "a late POST on a closed
|
||||
row".
|
||||
|
||||
Two survive, and each is equivalent while the other stays:
|
||||
|
||||
- the semantics check in `applyEntry` dropped;
|
||||
- the semantics check in the entry shape dropped.
|
||||
|
||||
Every entry passes the shape check before it is applied, so either check
|
||||
alone refuses a review verb at semantics 1. With both dropped, the
|
||||
semantics test fails.
|
||||
|
||||
## Suites
|
||||
|
||||
In the scratch clone with the patch and the helper patch applied: config
|
||||
24/0, task 90/0, foundation 44/0, conductor 17/0, release 14/0, auth 15/0,
|
||||
discord 64/0, extension-package 18/0, queue 27/0. `node --test
|
||||
packages/queue/tests/` passes 142, and `node --test packages/ledger/tests/`
|
||||
passes 58.
|
||||
|
||||
The scratch clone isn't the canonical root, so the queue suite printed its
|
||||
skip line for the two live checks. In the canonical tree they run.
|
||||
|
||||
In a fresh clone at cdcedb27 with `build.patch` applied: the manifest
|
||||
matches 10/10, the queue tests pass 142, the queue suite 27/0. With
|
||||
`helper.patch` applied on top, the ledger tests pass 58.
|
||||
|
||||
## Known limits
|
||||
|
||||
- **Scope of darkwing's and dewey's tokens.** They hold
|
||||
`write:repository`. If that doesn't cover an issue comment, the live
|
||||
POST gets 403, which is `failed` with nothing posted (8.9 says so). The
|
||||
scope question then goes to Sage.
|
||||
- **Post to outcome.** A kill between the POST and the outcome entry
|
||||
leaves the attempt `requesting` with a comment on the issue. The queue
|
||||
never posts again on its own; a person resolves it.
|
||||
- **Verdict comments aren't fetched.** `review record` takes the
|
||||
reviewer's comment id on trust, as the queue takes `--by`.
|
||||
- **Commit candidates** stay checkable only while a ref keeps the commit.
|
||||
- The helper's own limits are in `helper.md`.
|
||||
|
||||
## After approval (Sage)
|
||||
|
||||
1. Apply `build.patch` to the canonical tree and check it against
|
||||
`build-manifest.sha256`. Run `scripts/test-queue.sh` and
|
||||
`node --test packages/queue/tests/`. Commit the 10 files by path.
|
||||
2. Apply `helper.patch` for the same commit (decision 39). The live
|
||||
round needs it: without it the helper refuses a raw token file.
|
||||
3. `tools-md.patch` if you want it.
|
||||
4. Row 12 lists no reviewers, so moving it to in-review now would open a
|
||||
round that posts nothing. Set them first:
|
||||
`scripts/mosaic queue set 12 reviewers filbert --op OP --by sage`, and
|
||||
add rocko if the helper counts as part of this round.
|
||||
|
||||
The live round, which is mine:
|
||||
|
||||
5. `MOSAIC_GITEA_CREDENTIAL_FILE=~/.mosaic/fleet/agents/darkwing/secrets/gitea-mosaicstack-darkwing.token
|
||||
scripts/mosaic queue move 12 in-review --candidate <D's commit> --op OP --by darkwing`.
|
||||
The queue `lstat`s the file, the helper reads it, and `GET user` must
|
||||
answer `darkwing`. The request goes on #1508.
|
||||
6. Exit 0 means posted. Exit 1 with HTTP 403 is the scope question above.
|
||||
Exit 3 means I look on #1508 for the marker and resolve, or ask you to
|
||||
abandon.
|
||||
7. Filbert posts a verdict on #1508 and runs `review record 12`. Then
|
||||
`move 12 done`, and `queue-commit.sh` for the queue ops.
|
||||
@@ -1,184 +0,0 @@
|
||||
# Raw per-seat token files in `scripts/gitea-api.sh` (row 12, #1508)
|
||||
|
||||
Darkwing, 2026-09-27, after round 2 (lead decision 38). Lead decision 37 approved this patch. It ships next
|
||||
to the Piece D candidate but is a separate patch, and Rocko reviews it
|
||||
because it handles a credential. Filbert reviews D. Sage commits both.
|
||||
Nothing is committed, staged or pushed.
|
||||
|
||||
`helper.patch` (sha256 `48edd46b93c54908e9d59737aab78a33e007ddfafa1e6ad4d34333015bb75430`)
|
||||
changes `scripts/gitea-api.sh` and adds
|
||||
`packages/ledger/tests/gitea-helper-raw.test.mjs`. It applies to HEAD
|
||||
8efc0ff3 by itself. With only this patch applied, `node --test
|
||||
packages/ledger/tests/` passes 58/58. D does not depend on it, and it does
|
||||
not depend on D.
|
||||
|
||||
## Why
|
||||
|
||||
Each seat's Gitea token is at
|
||||
`~/.mosaic/fleet/agents/<seat>/secrets/gitea-mosaicstack-<seat>.token`.
|
||||
These files hold the bare token, not the JSON `mosaic.gitea.json` shape
|
||||
that the helper reads today. Without this change the helper refuses them,
|
||||
so D's live round can't post as the seat.
|
||||
|
||||
## What changed
|
||||
|
||||
Both old node snippets, the one that printed the base URL and the one that
|
||||
wrote the curl config, are now one script, `CRED_JS`, run in two modes:
|
||||
`base` and `cfg`. Each mode runs every check before it prints anything.
|
||||
|
||||
1. `lstat`: a regular file, not a symlink, no group or other bits.
|
||||
Unchanged.
|
||||
2. Read the file. A read error exits 3. It used to be caught by the JSON
|
||||
parse's catch, with the same result.
|
||||
3. `JSON.parse`. If the text parses, the JSON path runs unchanged: `url`
|
||||
must be `https://git.mosaicstack.dev` (trailing slashes stripped), and
|
||||
`api_token` must be a nonempty string. A non-`SyntaxError`, such as
|
||||
`null.mosaicstack`, exits 3.
|
||||
4. The raw path runs only on a `SyntaxError`. It accepts the file only if
|
||||
all of these hold:
|
||||
- `stat` size is 40 or 41;
|
||||
- the byte length of the text equals the `stat` size;
|
||||
- the text matches `^[0-9a-f]{40}\n?$`.
|
||||
|
||||
Then the base URL is the fixed string `https://git.mosaicstack.dev`,
|
||||
and the token is the first 40 characters. Anything else exits 3
|
||||
before any request.
|
||||
5. The second read runs, and must succeed, before curl starts. Round 1
|
||||
ran it inside `curl -K <(…)`, where bash drops its exit status, so a
|
||||
refusal gave curl an empty config and the request still went out.
|
||||
Now `CFG="$(node -e "$CRED_JS" cfg)" || exit 3` and `[ -n "$CFG" ] ||
|
||||
exit 3` run first, and curl reads `-K <(printf '%s\n' "$CFG")`.
|
||||
`printf` is a builtin, so the token is in no argv. Both curl branches,
|
||||
with and without a body, use it.
|
||||
6. `export -n CFG` runs right after the checked assignment, before any
|
||||
child starts. A `CFG` inherited from the caller's environment keeps its
|
||||
export attribute when assigned, and `SHELLOPTS=allexport` in the
|
||||
environment exports every assignment. Either way the config would reach
|
||||
curl's environment, and the body file's `mktemp` and `chmod` too.
|
||||
|
||||
## Decision 37, condition by condition
|
||||
|
||||
1. **The JSON path is unchanged.** The last test runs a good JSON file,
|
||||
four refused ones, and content that parses as JSON but would pass the
|
||||
raw pattern: 40 decimal digits, with and without a newline. Those take
|
||||
the JSON path and refuse.
|
||||
2. **One line of token characters.** The format is 40 lowercase hex
|
||||
characters. I checked by `stat` that the real files are 40 bytes (jarvis)
|
||||
or 41 (the others), all mode 600. I then ran the patched `CRED_JS` in
|
||||
`base` mode against my own file only. It printed
|
||||
`https://git.mosaicstack.dev` and exited 0, so darkwing's file matches
|
||||
the pattern. Nothing printed the token, and I read no other seat's file.
|
||||
The other seats' files are unconfirmed beyond size and mode. If one
|
||||
doesn't match, the helper exits 3 before any request. The test refuses
|
||||
16 bad contents before curl runs: empty, 39 characters, 41 characters,
|
||||
upper case, CRLF, a trailing CR, a trailing space, a trailing tab, two
|
||||
newlines, a leading space, a trailing space at 40, a second line, a
|
||||
quote, non-hex, non-ASCII, and 80 characters.
|
||||
3. **Fixed base URL.** The raw path assigns the literal. The test sets
|
||||
`MOSAIC_GITEA_URL`, `GITEA_URL` and `MOSAIC_GITEA_BASE_URL` to another
|
||||
host, and curl still gets `https://git.mosaicstack.dev/api/v1/user`.
|
||||
4. **File checks and the config stream.** Modes 640, 604, 660, 644, 000
|
||||
and 200, a symlink, a missing file and a directory all refuse before
|
||||
curl runs. 000 and 200 pass the group and other check, and the read
|
||||
refuses them. The stub curl records its argv and its `-K` stream. The
|
||||
token is in the stream only, never in argv, stdout or stderr. Every
|
||||
token in the test is a dummy that the test writes.
|
||||
5. **Only the acting seat's file.** That is D's job, not the helper's. D
|
||||
refuses unless `MOSAIC_GITEA_CREDENTIAL_FILE` resolves to
|
||||
`…/agents/<login>/secrets/gitea-mosaicstack-<login>.token` for the
|
||||
acting seat, or jarvis when Sage acts, and before posting it checks that
|
||||
`GET user` returns that login. See `review.mjs` `credCheck` and
|
||||
`checkUser` in the D candidate.
|
||||
6. This review.
|
||||
|
||||
## Round 1 and what changed for round 2
|
||||
|
||||
Rocko's round 1 (`agents/rocko/work/gitea-helper-raw-r1-review-2026-09-27.md`)
|
||||
found one blocker: a second read that refused still let curl run with an
|
||||
empty config, and a 2xx then exited 0. My round 1 notes called that a 401,
|
||||
which the server doesn't guarantee. Sage agreed it belongs in this patch.
|
||||
Step 5 above is the fix.
|
||||
|
||||
The new test, "a file that changes between the two reads refuses before
|
||||
curl runs", uses the git stub, which runs between the two reads, to
|
||||
change a valid dummy file to invalid text, to `{}`, to mode 644, to a
|
||||
symlink, and to missing. It runs each change for `GET user` and for a POST
|
||||
with a dummy body. Every case exits 3 with no curl call and nothing on
|
||||
stdout. It also changes a valid JSON file to invalid text. As a control,
|
||||
both calls reach curl with the right config when nothing changes. The
|
||||
first test now also checks that the token is not in the environment curl
|
||||
gets.
|
||||
|
||||
Rocko's round 2 (`agents/rocko/work/gitea-helper-raw-r2-review-2026-09-27.md`)
|
||||
closed that blocker and found another: the inherited export attribute in
|
||||
step 6. Lead decision 38 took the fix, `export -n CFG`, with no round 3;
|
||||
Sage checks it with Rocko's reproducer. The test "the token reaches no
|
||||
child environment, even with an inherited CFG or SHELLOPTS=allexport" runs
|
||||
raw and JSON dummy files, GET and POST, with an exported harmless `CFG`,
|
||||
with `SHELLOPTS=allexport`, and with both. Each call must exit 0 with the
|
||||
token in curl's config stream and not in its environment, argv, stdout or
|
||||
stderr. I found the `SHELLOPTS` route while checking the fix; `export -n`
|
||||
covers it too, so I added no second line for it.
|
||||
|
||||
Not in this patch: the predictable response path
|
||||
`/tmp/gitea-api-response.$$`. Sage has it on DEFERRED as "Gitea helper
|
||||
response-file hardening".
|
||||
|
||||
## What I'd like you to look at
|
||||
|
||||
- **The second read now re-validates.** Before, `gen_curl_cfg` parsed the
|
||||
file again without the `lstat` checks. Now both reads run every check.
|
||||
So a file swapped for a symlink or a wider mode between the two reads
|
||||
refuses. On the JSON path this is a tightening.
|
||||
- **The token now sits in a shell variable** for the rest of the script.
|
||||
`export -n` keeps it out of every child's environment, and nothing
|
||||
prints it. Before, it existed only in the pipe.
|
||||
- **`lstat` then `readFile` is still not atomic.** A swap between those two
|
||||
calls inside one read is the pre-existing race Rocko noted. This patch
|
||||
doesn't claim to close it.
|
||||
- **`text.slice(0, 40)`** relies on the pattern having matched. The only
|
||||
accepted texts are the token alone or the token plus `\n`.
|
||||
|
||||
## Mutations
|
||||
|
||||
25 mutants, each run alone against `gitea-helper-raw.test.mjs`: 19 of
|
||||
`CRED_JS` and 6 of the gate and the export. 15 are killed:
|
||||
|
||||
- uppercase allowed, any trailing whitespace allowed, the `^` anchor
|
||||
dropped;
|
||||
- a raw file refused outright (the old behaviour);
|
||||
- an env override of the raw base URL;
|
||||
- the token taken with its newline;
|
||||
- the mode check dropped;
|
||||
- the JSON path's host check or token check dropped, or its `|| {}`;
|
||||
- `cfg` output in `base` mode;
|
||||
- the read's own `try` dropped (killed by the 000 and 200 modes; I added
|
||||
those after this mutant first survived);
|
||||
- both gate checks dropped, and the round 1 code (no gate, curl reading
|
||||
`<(node … cfg)`), killed by the between-reads test;
|
||||
- `export -n CFG` dropped, killed by the inherited-environment test.
|
||||
|
||||
Ten survive, and each is equivalent while the other checks stay:
|
||||
|
||||
- `\n*$` for `\n?$`: the size check caps the file at 41 bytes;
|
||||
- the size check dropped: the pattern caps the length;
|
||||
- the byte-length check dropped: the pattern is ASCII, so a match means
|
||||
bytes equal characters; the check differs only if the file changes
|
||||
between `lstat` and the read;
|
||||
- the `SyntaxError` test dropped: the only non-`SyntaxError` is JSON
|
||||
`null`, which then fails the raw pattern;
|
||||
- `text.trim()` for `text.slice(0, 40)`: same result on every accepted
|
||||
text;
|
||||
- the `isSymbolicLink` test dropped: `lstat` of a symlink is never
|
||||
`isFile`;
|
||||
- `!== "base"` for `=== "cfg"`: the script calls only these two modes;
|
||||
- `|| true` for `|| exit 3` on the `CFG` line: node prints nothing when it
|
||||
refuses, so `[ -n "$CFG" ]` refuses;
|
||||
- `[ -n "$CFG" ]` dropped: `|| exit 3` refuses first. Dropping
|
||||
`|| exit 3` outright is the same, since `set -e` exits on the failed
|
||||
assignment;
|
||||
- `export CFG=…` for `CFG=…`: `export -n` on the next line clears it. In
|
||||
round 2 this one was killed; the fix makes it equivalent.
|
||||
|
||||
I kept the size, byte-length and symlink checks anyway. Decision 37 asks
|
||||
for size plus pattern, and the other two cost nothing.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user