Filbert's survey of Pi, Claude Code and Codex hooks as a record. No hook
is a hard block alone; credential scope, verbs, tool ceiling and a
container are. Two DEFERRED items.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
version 2, the action vocabulary and layered variables). I use their terms:
`withinRole`, `crossRole`, gated, and the action vocabulary from their
section 1.4.
## Versions and method
| Harness | Version | Source used |
|---|---|---|
| Pi | `@earendil-works/pi-coding-agent` 0.85.1, the `package.json` pin, matching `node_modules` | Docs and compiled source in `node_modules/@earendil-works/` |
| Claude Code | 2.1.289 (`claude --version`) | https://code.claude.com/docs/en/ and local `claude --help` |
| Codex CLI | 0.160.0 (`codex --version`) | https://developers.openai.com/codex/ and source at tag `rust-v0.160.0` (commit `a956835d0207`), plus local `--help` |
Three research subagents gathered the material. I re-checked the claims
that decide the key question against the source or docs myself (section 9).
No credential file was opened. For the local config files
| `pi:<path>:<line>` | relative to `node_modules/@earendil-works/` at 0.85.1 |
| `[cc:<page> § <section>]` | `https://code.claude.com/docs/en/<page>`, at that heading. `sdk-hooks` is `agent-sdk/hooks`, `sdk-ts` is `agent-sdk/typescript`, `sdk-perm` is `agent-sdk/permissions` |
"Tool execution was blocked" and `isAllowedTool`. The `dist/core` line
numbers are given because they are readable.
**Doc drift.** The Claude and Codex docs are living sites. Codex's describe
a newer release than 0.160.0. Where the docs and the binary or source
disagree, this survey follows the binary or source and says so.
---
## 1. Answer to the key question
**A hook refusal is a hard block only when three conditions hold:**
1.**Fail closed.** The harness runs the hook on every path to the action,
and a crash, timeout or missing hook stops the action.
2.**Recognisable action.** The hook can tell the action apart from
everything else: a typed tool call such as `mosaic_decision_raise` or
`mcp__mosaic__push`, not free shell text.
3.**No other route.** The agent has no other way to the action: no
credential in its environment or in a file it can read, and no network
path around the tool.
**How each harness meets them:**
| | Pi 0.85.1 | Claude Code 2.1.289 | Codex 0.160.0 |
|---|---|---|---|
| 1. Fail closed | **Yes** for `tool_call`. A thrown handler blocks the tool. A failed `-e` load exits 1. Every other event fails open. | **Partly.** Command and HTTP hooks fail **open** on crash, missing path, bad JSON or timeout. Deny rules hold in every mode. SDK callback hooks fail closed on PreToolUse timeout. | **No.** Hooks fail open on spawn error, timeout, bad JSON, or exit codes other than 0 and 2. Async hooks can't block. The docs call hooks "not a complete enforcement boundary". |
| 2. Recognisable action | Only for typed tools. A shell command is a string to pattern-match. | Same. The docs call `Bash(rm *)` "not a security boundary". | Same. Execpolicy rules match an argv prefix, and opaque scripts get through. |
| 3. No other route | **No**, unless there's a container. Pi has no sandbox and no path confinement, and `bash` inherits the full process environment. | Bash sandbox only (bubblewrap). File tools, MCP servers and hooks run outside it. | bwrap and seccomp sandbox for spawned commands. By default `*KEY*` and `*TOKEN*` variables pass through, and `/` stays readable. |
**Conclusion.** None of the three harnesses can make a shell-text hook a
boundary. That matches Darkwing's section 1.4: a hook is an early stop and a
log line. A decision class becomes a hard block only in these layers:
1.**Credential scope.** The role holds no token that can do a gated action.
Darkwing calls this "the hard line", and I agree.
2.**The `mosaic` verb.** It refuses a crossRole or gated action unless it is
given a resolved decision id. It must run in a process that holds the
credential the agent lacks, which is Darkwing's broker in 6.3.
3.**The harness's tool ceiling.**
- Pi: `--tools` filters the registry, and extensions can't re-add a name.
- Claude: deny rules and `--tools`.
- Codex: sandbox mode and requirements pins.
The ceiling removes direct routes. It can't stop `bash` from doing
whatever the credential and network allow.
4.**An OS boundary.** A container or bwrap with the network policy actually
enforced. Without one, condition 3 fails on every harness.
When those four are in place, a hook on the typed tool adds something
real: it refuses early, in the turn, with a pointer to the decision verb,
and it logs `action.refused` with layer `hook`. Pi's `tool_call` is the only
hook of the three that is itself fail-closed. Claude's command hooks and all
Codex hooks fail open. So on those two harnesses, any block that matters must
also be a deny rule, an execpolicy `forbidden` rule, or a missing credential.
**Per decision class:**
| Class | Prompt text (all three) | Pi | Claude Code | Codex |
|---|---|---|---|---|
| Routine | The only control. No gate is needed: these actions have no outside effect. | none | none | none |
| withinRole | States what the role may do alone. | `tool_result` / `tool_execution_end` can log, but fail open. The verb's event row is the record. | PostToolUse logs, fail-open. The verb is the record. | PostToolUse logs, fail-open. The verb is the record. |
| crossRole | Tells the agent to raise a decision. | **Hard** for typed tools via `tool_call` block. Advisory for `bash` text. The verb refuses. | **Hard** via a deny rule on the typed MCP tool. The hook is advisory (fail-open). The verb refuses. | Advisory hook. Execpolicy `forbidden` holds for an argv prefix only. The verb refuses. |
| Gated | Tells the agent to raise a decision routed to the human. | As crossRole, plus no credential in the container. | As crossRole, plus no credential. The sandbox `credentials` deny covers Bash only. | As crossRole, plus no credential. Needs a `shell_environment_policy` exclude and `deny_read`. |
---
## 2. Pi 0.85.1
### 2.1 System prompt and context files
- **Default prompt.** In order:
1. Fixed preamble
2. Tool list
3. Guidelines
4. Append section
5.`<project_context>` with each context file
6. Skills XML (only when `read` or `bash` is active)
| System prompt | `--system-prompt "<content>"` [repo:adapters/pi/adapter.sh:92], which also turns off `SYSTEM.md` discovery | n/a (advisory content) |
| Ambient context | `--offline --no-extensions --no-skills --no-prompt-templates --no-themes --no-context-files` | Yes, for what loads |
| Tools | `--tools $MOSAIC_TOOLS` [repo:adapters/pi/adapter.sh:48]. The role ceiling is intersected with the seat in `validateRole` / `resolve-role` [repo:scripts/mosaic-task.mjs:249-272, 674] | Yes (registry filter) |
| Skills | `--skill` per `MOSAIC_SKILLS` dir; a missing dir exits 2 [:57-58] | n/a |
| Hooks | none | — |
| MCP | none | — |
| Network | the role's `network` is declared and "enforced when network policy lands" [repo:scripts/mosaic-task.mjs:249-250]; `compose.yaml` sets no `network_mode` | **No** |
| Credentials | `ZAI_API_KEY`, `ANTHROPIC_API_KEY` env [repo:compose.yaml:36-37]; pi `auth.json` mounted read-only [:43] | **No.** A role with `bash` (researcher has it) can print the env keys, and `read` can open the mounted `auth.json` (Pi has no path confinement) |
| Headless | `-p "$MOSAIC_REQUEST"`, stdin empty, SIGKILL on timeout; stdout is the response, and exit 0 means success [repo:adapters/README.md:37-39] | text mode, so an LLM error exits 1 |
| Sandbox | the container: user 1000:1000, data root mounted | Yes for the filesystem, **no** for the network |
The adapter contract [repo:adapters/README.md:24-27, 40-48] passes four
inputs: the prompt file, the request, the provider and the model. It also
requires that "adapters never read configuration files". It has no slot for
a tool policy, a gate, skills chosen per role, MCP config or role
credentials. The pi adapter takes tools and skills through compose
environment variables that the contract table doesn't list. Adding a harness
requires an allowlist entry; today the allowlist is
| Gate and typed tools | `-e <pinned gate extension>`. It reads `policy.json` and registers a `tool_call` handler plus the typed tools. A load failure exits 1, so a missing gate refuses to start. **Blocked by decision 31 for Console-bound seats** (section 8) |
| Tool ceiling | `--tools <role ∩ project ∩ agent>`, including the typed tool names |
| Isolation | `PI_CODING_AGENT_DIR=<launch>/pi-agent`, holding a generated `settings.json`, no `auth.json` unless one is required, and no `trust.json` |
| Credentials | Provider key through env or `--api-key`. Role credentials stay in the broker, not in Pi's env. Anything in Pi's env is visible to `bash` |
| Headless | `-p` (text mode, so an LLM error exits 1) for workers; `--mode rpc` for Console-bound seats |
| Network | Only a container with `network_mode` set gives a hard line. Pi has no setting for it |
### 7.4 Claude Code
| Output | Form |
|---|---|
| Isolation | `CLAUDE_CONFIG_DIR=<launch>/claude`, `--setting-sources ""`, `CLAUDE_CODE_DISABLE_CLAUDE_MDS=1`, `CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`, `CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1`. Managed settings in `/etc/claude-code`, if any ever exist, still apply |
| Prompt | `--system-prompt-file` or `--append-system-prompt-file <prompt.md>`. Both are hidden in 2.1.289 and need a probe. `--append-system-prompt "$(cat …)"` works today (Rocko) |
| Settings | `--settings <launch>/settings.json`, which must contain: (1) `permissions.deny` for every built-in tool outside the ceiling, plus `mcp__mosaic__<tool>` for crossRole and gated actions; (2) `hooks.PreToolUse` with matcher `*`, pointing at the gate wrapper; (3) `permissions.disableBypassPermissionsMode: "disable"`; (4) `sandbox` with `enabled`, `failIfUnavailable`, `allowUnsandboxedCommands:false` and `network.allowedDomains` from the role's network class; (5) `sandbox.filesystem.denyRead` for credential paths |
| Tool ceiling | `--tools <mapped list>`. Pi names map to Claude names: `read`→`Read`, `write`→`Write`, `edit`→`Edit`, `bash`→`Bash`, `grep`→`Grep`, `find`→`Glob`. `ls` has no one-to-one tool, so the mapping needs checking against [cc:tools-reference] |
| Skills | `--plugin-dir <launch>/plugin` with `skills/`. Whether `--add-dir` skills still load under `--setting-sources ""` is unverified |
| MCP | `--mcp-config <launch>/mcp.json --strict-mcp-config`, with the `mosaic` broker server only |
| Gate failure mode | Command hooks fail open, so a blocked action must also be a deny rule. The wrapper must turn every internal error into exit 2 (`trap`). The launcher must run a canary call that the gate has to refuse before the session counts as launched; that catches the "mistyped path" case. In headless SDK runs, an in-process callback hook fails closed on timeout |
| Credentials | Subscription OAuth lives in `.credentials.json` under the config dir, so a private `CLAUDE_CONFIG_DIR` needs its own login, a `CLAUDE_CODE_OAUTH_TOKEN` (`claude setup-token`), or `ANTHROPIC_API_KEY` (section 8, item 5). `--bare` would be the stronger hardening, but it accepts only an API key or `apiKeyHelper` |
| Headless | `-p --output-format stream-json --permission-mode dontAsk --max-turns N`, or the Agent SDK with `settingSources: []` and callback hooks |
### 7.5 Codex CLI
| Output | Form |
|---|---|
| Isolation | `CODEX_HOME=<launch>/codex` with a generated `config.toml`. Give the project path an explicit trust entry: `untrusted` keeps project config, hooks, rules and the AGENTS.md chain out; `trusted` lets them in |
| Prompt | `developer_instructions` from `prompt.md`, or `$CODEX_HOME/AGENTS.md`. `model_instructions_file` replaces the built-in prompt but is "strongly discouraged" |
| Skills | `$CODEX_HOME/skills` (deprecated) or a plugin. `~/.agents/skills` is shared user state, not per session. Repo `.agents/skills` loads without a trust check (inferred). This repo has no `.agents/` today |
| Gate | `[hooks]` PreToolUse with a command pointing at the gate wrapper. Fail-open, and it needs hook trust: either pre-recorded hash trust or `--dangerously-bypass-hook-trust`. Back it with execpolicy rules in `$CODEX_HOME/rules/` that `forbidden` the classified commands (`git push`, `git merge`, `tea`, `docker push`, `npm publish`); those are enforced before exec, by argv prefix |
| Sandbox | `sandbox_mode = "workspace-write"` set explicitly; `sandbox_workspace_write.network_access` from the role's network class; a named profile with `[permissions.<name>.filesystem]."<credential path>" = "deny"` [cx:D:permissions]. These docs may describe a newer release than 0.160.0, so a probe must confirm it; the admin form is requirements `permissions.filesystem.deny_read`. Hooks and MCP servers run outside it |
| MCP | `[mcp_servers.mosaic]` with `required = true` and `enabled_tools` |
| Credentials | `CODEX_API_KEY` (exec and app-server only) or `cli_auth_credentials_store = "ephemeral"`. Role credentials stay in the broker |
| Headless | `codex exec --json` (exit 0/1, approval forced to `never`), or app-server JSON-RPC, where the host answers escalation approvals in-band |
| Lock | Only `/etc/codex/requirements.toml` can stop a flag such as `--dangerously-bypass-approvals-and-sandbox`. Inferred: since the launcher owns the argv, the agent can't add flags, so slice 1 doesn't need the lock |
### 7.6 Changes to `adapters/`
- **The adapter contract needs a version bump.**
- New input: `MOSAIC_LAUNCH_DIR`, which points at the generated, read-only
@@ -469,3 +469,5 @@ are never rewritten or removed; corrections are new entries.
2026-10-04T19:12:27Z | Filbert (T3 Claude Code, thread 9cb9731e) | meta-harness survey started (slice 1 step 8, Sage ASSIGNMENT) | research only in agents/filbert/work/; no commits; row 5 round 2 takes priority
2026-10-04T19:12:38Z | Sage (T3 Claude Code, thread 1ef1e4f8) | PRDY round 1, seat launches (decision 45) | PRD draft 0.1 at docs/prd/mosaic-stack.md; launched Researcher (T3 Sonnet 5.5, thread 9bac0794) for the Vikunja and Pocket ID report; assigned Darkwing the slice 1 data model note and Filbert the meta-harness survey (both T3 Opus 5.5, existing threads); Dewey on the row 5 rework; running under the new limits: Opus 3 of 4 (dewey, darkwing, filbert), Sonnet 1 of 4 (researcher)
2026-10-04T19:19:14Z | Sage (T3 Claude Code, thread 1ef1e4f8) | slice 1 data model received | Darkwing's note (948b94ce) and prototype committed as records; lead decision 46 accepts 6.1-6.5, 6.7, 6.8 and sends 6.6 (PM launching sessions) to Jason in PRDY round 3
2026-10-04T19:26:12Z | Filbert (T3 Claude Code, thread 9cb9731e) | meta-harness survey done (slice 1 step 8) | agents/filbert/work/meta-harness-survey-2026-10-04.md sha256 05f83807…0989; Pi tool_call is the only fail-closed hook of the three, so hard lines are credential scope, verbs, tool ceiling and container; no commits
2026-10-04T19:26:50Z | Sage (T3 Claude Code, thread 1ef1e4f8) | meta-harness survey received | Filbert's survey (05f83807) committed as a record; lead decision 47 rules on its section 8; two DEFERRED items (host-seat --approve, worker provider-key exposure)
- **Workers can read provider keys.** The worker container passes
`ZAI_API_KEY` and `ANTHROPIC_API_KEY` in the environment, which Pi's
`bash` inherits, and Pi's `read` can open the mounted `auth.json`.
Slice 1 role tokens stay in the broker (lead decision 47). The workers'
own exposure needs a separate piece. Filbert, survey section 8.6.
(2026-10-04)
## Queue
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Claude Opus 5.5