docs(plans): meta-harness survey; lead decision 47
Filbert's survey of Pi, Claude Code and Codex hooks as a record. No hook is a hard block alone; credential scope, verbs, tool ceiling and a container are. Two DEFERRED items. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
@@ -0,0 +1,929 @@
|
||||
# Meta-harness survey: Pi, Claude Code and Codex CLI (2026-10-04)
|
||||
|
||||
Filbert, for Sage. Slice 1, step 8 (`docs/plans/2026-10-04_foundation-direction.md`,
|
||||
lead decisions 43 and 44). This is research only: nothing here is built or committed.
|
||||
|
||||
The survey covers how each harness loads seven things: the system prompt and
|
||||
context files, skills, hooks, MCP, permissions and sandbox, per-session
|
||||
environment and credentials, and headless runs. It then maps what
|
||||
`adapters/` and the seat launchers already do, and describes what a
|
||||
generator would have to emit for each harness. Section 1 answers the key
|
||||
question: where can a role's decision classes be a hard block, and where are
|
||||
they only prompt text?
|
||||
|
||||
It reads alongside Darkwing's draft,
|
||||
`agents/darkwing/work/slice1-data-model-2026-10-04.md` (role schema
|
||||
version 2, the action vocabulary and layered variables). I use their terms:
|
||||
`withinRole`, `crossRole`, gated, and the action vocabulary from their
|
||||
section 1.4.
|
||||
|
||||
## Versions and method
|
||||
|
||||
| Harness | Version | Source used |
|
||||
|---|---|---|
|
||||
| Pi | `@earendil-works/pi-coding-agent` 0.85.1, the `package.json` pin, matching `node_modules` | Docs and compiled source in `node_modules/@earendil-works/` |
|
||||
| Claude Code | 2.1.289 (`claude --version`) | https://code.claude.com/docs/en/ and local `claude --help` |
|
||||
| Codex CLI | 0.160.0 (`codex --version`) | https://developers.openai.com/codex/ and source at tag `rust-v0.160.0` (commit `a956835d0207`), plus local `--help` |
|
||||
|
||||
Three research subagents gathered the material. I re-checked the claims
|
||||
that decide the key question against the source or docs myself (section 9).
|
||||
No credential file was opened. For the local config files
|
||||
(`~/.claude/settings.json`, `~/.codex/config.toml`, `~/.pi/agent/settings.json`),
|
||||
only key names were read.
|
||||
|
||||
**Citation keys**
|
||||
|
||||
| Key | Meaning |
|
||||
|---|---|
|
||||
| `pi:<path>:<line>` | relative to `node_modules/@earendil-works/` at 0.85.1 |
|
||||
| `[cc:<page> § <section>]` | `https://code.claude.com/docs/en/<page>`, at that heading. `sdk-hooks` is `agent-sdk/hooks`, `sdk-ts` is `agent-sdk/typescript`, `sdk-perm` is `agent-sdk/permissions` |
|
||||
| `[cc:help]` | local `claude --help`, 2.1.289 |
|
||||
| `[cx:D:<page>#<section>]` | `https://developers.openai.com/codex/<page>` |
|
||||
| `[cx:S:<path>]` | `github.com/openai/codex/blob/rust-v0.160.0/<path>` |
|
||||
| `[cx:H:<cmd>]` | local `codex <cmd> --help` |
|
||||
| `[repo:<path>:<line>]` | this repository at `bb7e37dd` |
|
||||
|
||||
**Inferred** marks a conclusion drawn from code or docs that the source
|
||||
doesn't state outright.
|
||||
|
||||
**Pi bundle note.** `pi` runs `dist/bundle/cli.js` (`pi:pi-coding-agent/package.json`
|
||||
`bin`), not the `dist/core` files cited below (lead decision 32 n1). I checked
|
||||
that the bundle contains the enforcement paths I rely on:
|
||||
`chunks/chunk-JVUZSMYM.js` holds "Extension failed, blocking execution",
|
||||
"Tool execution was blocked" and `isAllowedTool`. The `dist/core` line
|
||||
numbers are given because they are readable.
|
||||
|
||||
**Doc drift.** The Claude and Codex docs are living sites. Codex's describe
|
||||
a newer release than 0.160.0. Where the docs and the binary or source
|
||||
disagree, this survey follows the binary or source and says so.
|
||||
|
||||
---
|
||||
|
||||
## 1. Answer to the key question
|
||||
|
||||
**A hook refusal is a hard block only when three conditions hold:**
|
||||
|
||||
1. **Fail closed.** The harness runs the hook on every path to the action,
|
||||
and a crash, timeout or missing hook stops the action.
|
||||
2. **Recognisable action.** The hook can tell the action apart from
|
||||
everything else: a typed tool call such as `mosaic_decision_raise` or
|
||||
`mcp__mosaic__push`, not free shell text.
|
||||
3. **No other route.** The agent has no other way to the action: no
|
||||
credential in its environment or in a file it can read, and no network
|
||||
path around the tool.
|
||||
|
||||
**How each harness meets them:**
|
||||
|
||||
| | Pi 0.85.1 | Claude Code 2.1.289 | Codex 0.160.0 |
|
||||
|---|---|---|---|
|
||||
| 1. Fail closed | **Yes** for `tool_call`. A thrown handler blocks the tool. A failed `-e` load exits 1. Every other event fails open. | **Partly.** Command and HTTP hooks fail **open** on crash, missing path, bad JSON or timeout. Deny rules hold in every mode. SDK callback hooks fail closed on PreToolUse timeout. | **No.** Hooks fail open on spawn error, timeout, bad JSON, or exit codes other than 0 and 2. Async hooks can't block. The docs call hooks "not a complete enforcement boundary". |
|
||||
| 2. Recognisable action | Only for typed tools. A shell command is a string to pattern-match. | Same. The docs call `Bash(rm *)` "not a security boundary". | Same. Execpolicy rules match an argv prefix, and opaque scripts get through. |
|
||||
| 3. No other route | **No**, unless there's a container. Pi has no sandbox and no path confinement, and `bash` inherits the full process environment. | Bash sandbox only (bubblewrap). File tools, MCP servers and hooks run outside it. | bwrap and seccomp sandbox for spawned commands. By default `*KEY*` and `*TOKEN*` variables pass through, and `/` stays readable. |
|
||||
|
||||
**Conclusion.** None of the three harnesses can make a shell-text hook a
|
||||
boundary. That matches Darkwing's section 1.4: a hook is an early stop and a
|
||||
log line. A decision class becomes a hard block only in these layers:
|
||||
|
||||
1. **Credential scope.** The role holds no token that can do a gated action.
|
||||
Darkwing calls this "the hard line", and I agree.
|
||||
2. **The `mosaic` verb.** It refuses a crossRole or gated action unless it is
|
||||
given a resolved decision id. It must run in a process that holds the
|
||||
credential the agent lacks, which is Darkwing's broker in 6.3.
|
||||
3. **The harness's tool ceiling.**
|
||||
- Pi: `--tools` filters the registry, and extensions can't re-add a name.
|
||||
- Claude: deny rules and `--tools`.
|
||||
- Codex: sandbox mode and requirements pins.
|
||||
|
||||
The ceiling removes direct routes. It can't stop `bash` from doing
|
||||
whatever the credential and network allow.
|
||||
4. **An OS boundary.** A container or bwrap with the network policy actually
|
||||
enforced. Without one, condition 3 fails on every harness.
|
||||
|
||||
When those four are in place, a hook on the typed tool adds something
|
||||
real: it refuses early, in the turn, with a pointer to the decision verb,
|
||||
and it logs `action.refused` with layer `hook`. Pi's `tool_call` is the only
|
||||
hook of the three that is itself fail-closed. Claude's command hooks and all
|
||||
Codex hooks fail open. So on those two harnesses, any block that matters must
|
||||
also be a deny rule, an execpolicy `forbidden` rule, or a missing credential.
|
||||
|
||||
**Per decision class:**
|
||||
|
||||
| Class | Prompt text (all three) | Pi | Claude Code | Codex |
|
||||
|---|---|---|---|---|
|
||||
| Routine | The only control. No gate is needed: these actions have no outside effect. | none | none | none |
|
||||
| withinRole | States what the role may do alone. | `tool_result` / `tool_execution_end` can log, but fail open. The verb's event row is the record. | PostToolUse logs, fail-open. The verb is the record. | PostToolUse logs, fail-open. The verb is the record. |
|
||||
| crossRole | Tells the agent to raise a decision. | **Hard** for typed tools via `tool_call` block. Advisory for `bash` text. The verb refuses. | **Hard** via a deny rule on the typed MCP tool. The hook is advisory (fail-open). The verb refuses. | Advisory hook. Execpolicy `forbidden` holds for an argv prefix only. The verb refuses. |
|
||||
| Gated | Tells the agent to raise a decision routed to the human. | As crossRole, plus no credential in the container. | As crossRole, plus no credential. The sandbox `credentials` deny covers Bash only. | As crossRole, plus no credential. Needs a `shell_environment_policy` exclude and `deny_read`. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Pi 0.85.1
|
||||
|
||||
### 2.1 System prompt and context files
|
||||
|
||||
- **Default prompt.** In order:
|
||||
1. Fixed preamble
|
||||
2. Tool list
|
||||
3. Guidelines
|
||||
4. Append section
|
||||
5. `<project_context>` with each context file
|
||||
6. Skills XML (only when `read` or `bash` is active)
|
||||
7. cwd
|
||||
|
||||
Source: `pi:pi-coding-agent/dist/core/system-prompt.js:81-116`.
|
||||
- **`--system-prompt`.** Replaces items 1–3 only. The append section, context
|
||||
files, skills and cwd are still added
|
||||
(`pi:pi-coding-agent/dist/core/system-prompt.js:15-34`; `pi:pi-coding-agent/README.md:610`).
|
||||
It also turns off `SYSTEM.md` discovery
|
||||
(`pi:pi-coding-agent/dist/core/resource-loader.js:381`).
|
||||
- **`--append-system-prompt`.** Repeatable. It turns off `APPEND_SYSTEM.md`
|
||||
discovery (`pi:pi-coding-agent/dist/core/resource-loader.js:386-390`).
|
||||
Each flag value is read as a file if that path exists, and used as literal
|
||||
text otherwise (`pi:pi-coding-agent/dist/core/resource-loader.js:17-31`).
|
||||
- **`SYSTEM.md` / `APPEND_SYSTEM.md`.** Pi checks project `.pi/` first, but
|
||||
only if the project is trusted, then the global agent dir
|
||||
(`pi:pi-coding-agent/dist/core/resource-loader.js:809-830`).
|
||||
- **Context files.** Pi takes the first of `AGENTS.override.md`, `AGENTS.md`,
|
||||
`AGENTS.MD`, `CLAUDE.md`, `CLAUDE.MD` in each directory.
|
||||
- Order: the global agent-dir file first, then every ancestor from `/` down
|
||||
to the cwd (`pi:pi-coding-agent/dist/core/resource-loader.js:32-52, 82-109`).
|
||||
- Context files load **regardless of project trust**
|
||||
(`pi:pi-coding-agent/docs/security.md:27`).
|
||||
- `--no-context-files` turns them off
|
||||
(`pi:pi-coding-agent/dist/core/resource-loader.js:371-378`). It does not turn
|
||||
off `SYSTEM.md`, `APPEND_SYSTEM.md` or skills, which have their own flags
|
||||
(inferred from separate code paths, `:371-399`).
|
||||
- **Per-turn rewrite by extensions.** `before_agent_start` can replace the
|
||||
system prompt for a turn (`pi:pi-coding-agent/docs/extensions.md:530-565`).
|
||||
`before_provider_request` can rewrite the provider payload (`:705-720`).
|
||||
Enforcement implication: a loaded extension can change what the model is
|
||||
told.
|
||||
|
||||
### 2.2 Skills and commands
|
||||
|
||||
- **Skill locations.**
|
||||
- `~/.pi/agent/skills/` and `~/.agents/skills/`.
|
||||
- Project `.pi/skills/` and `.agents/skills/` (trusted projects only).
|
||||
- Packages and the `skills` setting.
|
||||
- `--skill <path>`, which still applies under `--no-skills`.
|
||||
|
||||
Source: `pi:pi-coding-agent/docs/skills.md:24-42`.
|
||||
- **What the model sees.** Only each skill's name and description go into
|
||||
the prompt (`:65-72`). `disable-model-invocation: true` hides a skill
|
||||
(`:150`).
|
||||
- **Skill `allowed-tools` is not enforced.** It is documented as experimental
|
||||
(`pi:pi-coding-agent/docs/skills.md:149`), and nothing in `dist` reads it.
|
||||
- **`/skill:name args`.** Expands in every mode, including `-p` and RPC
|
||||
(`pi:pi-coding-agent/dist/core/agent-session.js:983-996`).
|
||||
- **Prompt templates.** `--no-prompt-templates` turns them off
|
||||
(`pi:pi-coding-agent/dist/cli/args.js:167`).
|
||||
- **Extension commands.** `pi.registerCommand` runs before the `input` event
|
||||
and skips it (`pi:pi-coding-agent/dist/core/agent-session.js:828-835`).
|
||||
|
||||
### 2.3 Hooks (extensions)
|
||||
|
||||
- **Loading.** Extensions are TypeScript loaded through jiti.
|
||||
- `-e/--extension` is repeatable.
|
||||
- `--no-extensions` drops discovered and settings extensions but keeps the
|
||||
`-e` ones (`pi:pi-coding-agent/dist/core/resource-loader.js:409-411`, checked).
|
||||
- A load failure, a missing `-e` path, or a duplicate tool or flag name exits
|
||||
1 in every mode (`pi:pi-coding-agent/dist/main.js:722-730`).
|
||||
- **Events that can block.**
|
||||
- `tool_call` returns `{block: true, reason?, terminate?}`
|
||||
(`pi:pi-coding-agent/docs/extensions.md:778-818`). The first blocking
|
||||
handler wins (`pi:pi-coding-agent/dist/core/extensions/runner.js:745-763`, checked).
|
||||
- `input` can swallow a prompt.
|
||||
- The `session_before_*` events can cancel.
|
||||
- **Notification-only events.** `session_start`, `session_shutdown`,
|
||||
`agent_end`, `agent_settled`, `tool_execution_*`
|
||||
(`pi:pi-coding-agent/docs/extensions.md:277-349`).
|
||||
- **Every model tool call passes `tool_call`.** That includes extension and
|
||||
SDK tools, through `beforeToolCall`
|
||||
(`pi:pi-coding-agent/dist/core/agent-session.js:223-243`;
|
||||
`pi:pi-agent-core/dist/agent-loop.js:400-437`, checked). A blocked call
|
||||
becomes an error result, and `execute` is never called.
|
||||
- **Failure behaviour (checked).**
|
||||
- A throwing `tool_call` handler blocks the tool: the error is rethrown and
|
||||
caught in `prepareToolCall`, which returns an error result
|
||||
(`pi:pi-coding-agent/dist/core/agent-session.js:237-242`;
|
||||
`pi:pi-agent-core/dist/agent-loop.js:452-458`). The docs say "`tool_call`
|
||||
errors block the tool (fail-safe)" (`pi:pi-coding-agent/docs/extensions.md:2925`).
|
||||
- Every other handler that throws is logged, and the agent continues
|
||||
(`:2924`).
|
||||
- With no UI (`-p`, json), `ctx.ui.confirm` returns `false`
|
||||
(`pi:pi-coding-agent/dist/core/extensions/runner.js:88-91`).
|
||||
- **What the hook doesn't see.**
|
||||
- RPC `bash` and interactive `!` commands go through `user_bash`, not
|
||||
`tool_call` and not `--tools`
|
||||
(`pi:pi-coding-agent/dist/modes/rpc/rpc-mode.js:441-459`). That route is
|
||||
host-driven, not model-driven.
|
||||
- An extension can rewrite `event.input` with no re-validation
|
||||
(`pi:pi-coding-agent/docs/extensions.md:786-791`).
|
||||
- **Lead decision 31.** A Console-bound seat loads no explicit extensions and
|
||||
launches with `--no-extensions` and no `--extension`
|
||||
([repo:docs/plans/2026-09-26_lead-decisions.md:392-405]). A Pi gate
|
||||
extension conflicts with that until CHAT-06 brings reviewed, pinned
|
||||
extensions back. See section 8.
|
||||
|
||||
### 2.4 MCP
|
||||
|
||||
None, by design: "**No MCP.** Build CLI tools with READMEs (see Skills),
|
||||
or build an extension that adds MCP support"
|
||||
(`pi:pi-coding-agent/README.md:499`). The route is an extension that runs an
|
||||
MCP client and calls `pi.registerTool()` for each server tool
|
||||
(`pi:pi-coding-agent/docs/extensions.md:2365-2400`). Tools registered that way
|
||||
are subject to `--tools` and `tool_call` like any other (inferred,
|
||||
`pi:pi-coding-agent/dist/core/agent-session.js:2105-2180`).
|
||||
|
||||
### 2.5 Permissions and sandbox
|
||||
|
||||
- **No permission prompts and no sandbox.** "No permission popups. Run in a
|
||||
container, or build your own confirmation flow with extensions"
|
||||
(`pi:pi-coding-agent/README.md:503`; `pi:pi-coding-agent/docs/security.md:31-35`).
|
||||
Project trust "is not a sandbox" (`:7`).
|
||||
- **Tool allowlist.** `--tools a,b`, `--exclude-tools`, `--no-tools`,
|
||||
`--no-builtin-tools` (`pi:pi-coding-agent/dist/cli/args.js:94-111`).
|
||||
- The allowlist filters the registry, and `setActiveToolsByName` ignores
|
||||
names outside it (`pi:pi-coding-agent/dist/core/agent-session.js:2105-2159,
|
||||
659-673`, checked). An extension therefore can't switch an excluded tool
|
||||
back on.
|
||||
- Gap: an extension can register its own tool under an allowed name such as
|
||||
`read` (`pi:pi-coding-agent/docs/extensions.md:2080-2093`).
|
||||
- **No path confinement.** Absolute and `~` paths resolve as given
|
||||
(`pi:pi-coding-agent/dist/core/tools/path-utils.js:38-44`, checked). A
|
||||
read-only tool set can still read `auth.json`.
|
||||
- **Project trust.** `-a/--approve` trusts project-local files for one run,
|
||||
and `-na/--no-approve` ignores them (`pi:pi-coding-agent/README.md:615-616`;
|
||||
`pi:pi-coding-agent/dist/cli/args.js:205-208`, checked). Headless runs with no
|
||||
saved decision silently skip project `.pi/` resources
|
||||
(`pi:pi-coding-agent/docs/security.md:29`).
|
||||
|
||||
### 2.6 Per-session environment and credentials
|
||||
|
||||
- **Agent directory.** `PI_CODING_AGENT_DIR` relocates the whole agent dir
|
||||
(`pi:pi-coding-agent/dist/config.js:406, 421-462`): `auth.json`,
|
||||
`models.json`, `settings.json`, `sessions/`, `trust.json`, and the global
|
||||
`AGENTS.md`, `SYSTEM.md` and skills.
|
||||
- **Credential resolution.** `--api-key`, then `auth.json`, then the
|
||||
environment variable, then `models.json`
|
||||
(`pi:pi-coding-agent/docs/providers.md:310-317`).
|
||||
- An `auth.json` `key` of `"!command"` runs a shell command
|
||||
(`:159-179`).
|
||||
- SDK: `InMemoryCredentialStore` (`pi:pi-coding-agent/docs/sdk.md:466-474`).
|
||||
- **The `bash` tool inherits Pi's whole environment.** The spawn env is
|
||||
`...process.env` (`pi:pi-coding-agent/dist/utils/shell.js:117-123`,
|
||||
checked). The model's bash also gets `PI_SESSION_ID`, `PI_PROVIDER` and
|
||||
others (`pi:pi-coding-agent/docs/environment-variables.md:11-49`). Any key in
|
||||
Pi's environment is printable by the model.
|
||||
- **Sessions.** `--session-dir`, `--session-id`, `--fork`, `--no-session`,
|
||||
`-c` (`pi:pi-coding-agent/dist/cli/args.js:46-90`).
|
||||
|
||||
### 2.7 Headless
|
||||
|
||||
- **Modes.** `-p` (text), `--mode json` (JSONL events) and `--mode rpc`
|
||||
(JSONL commands over stdio) (`pi:pi-coding-agent/docs/rpc.md:20-37`). An
|
||||
unknown `--mode` value is ignored silently
|
||||
(`pi:pi-coding-agent/dist/cli/args.js:40-45`).
|
||||
- **Exit codes.**
|
||||
- 0 on success.
|
||||
- 1 when `stopReason` is `error` or `aborted`, in text mode only
|
||||
(`pi:pi-coding-agent/dist/modes/print-mode.js:110-118`).
|
||||
- 1 on a thrown error or a startup error.
|
||||
- 143 on SIGTERM, 129 on SIGHUP.
|
||||
- Inferred: in `--mode json`, a non-throwing LLM error exits 0.
|
||||
- **SDK.** `createAgentSession({ tools, customTools, resourceLoader, ... })`
|
||||
(`pi:pi-coding-agent/docs/sdk.md:16-66`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Claude Code 2.1.289
|
||||
|
||||
### 3.1 System prompt and context files
|
||||
|
||||
- **CLAUDE.md scopes.**
|
||||
- Managed: `/etc/claude-code/CLAUDE.md`, which can't be excluded.
|
||||
- User: `~/.claude/CLAUDE.md` and `rules/`.
|
||||
- Project: `./CLAUDE.md`, `.claude/rules/`.
|
||||
- Local: `CLAUDE.local.md`.
|
||||
|
||||
Files in the cwd and its ancestors load at launch, root first.
|
||||
Subdirectory files load lazily. `AGENTS.md` is read only when no CLAUDE.md
|
||||
exists (v2.1.277+). `@imports` go up to four hops.
|
||||
Source: [cc:memory § How CLAUDE.md files load; § AGENTS.md].
|
||||
- **CLAUDE.md arrives as a user message after the system prompt**
|
||||
[cc:memory § Claude isn't following my CLAUDE.md]. It is "not a hard
|
||||
enforcement layer" [cc:memory § Exclude specific CLAUDE.md files].
|
||||
- **System prompt flags.**
|
||||
- `--system-prompt` / `--system-prompt-file` replace the prompt;
|
||||
`--append-system-prompt` / `--append-system-prompt-file` append
|
||||
[cc:cli-reference § System prompt flags].
|
||||
- The `-file` variants and `--max-turns` are documented but absent from
|
||||
`[cc:help]`. Inferred: they are hidden flags. Not exercised.
|
||||
- Replacing the prompt does not remove CLAUDE.md
|
||||
[cc:sdk-sysprompt § Turn off the context your agent replaces].
|
||||
- **Turning CLAUDE.md off.** `CLAUDE_CODE_DISABLE_CLAUDE_MDS=1`,
|
||||
`--setting-sources` without `project`/`user`/`local`, `--bare`, or
|
||||
`--safe-mode` [cc:env-vars; cc:help]. Auto memory has its own switch,
|
||||
`CLAUDE_CODE_DISABLE_AUTO_MEMORY=1` [cc:memory § Enable or disable auto memory].
|
||||
|
||||
### 3.2 Skills and commands
|
||||
|
||||
- **Where skills are discovered.**
|
||||
- Managed, `~/.claude/skills`, and project `.claude/skills` (walking up to
|
||||
the repo root).
|
||||
- Nested directories, lazily.
|
||||
- `--add-dir` directories and plugins.
|
||||
- claude.ai sync.
|
||||
|
||||
Source: [cc:skills § Where skills live]. `.claude/commands/x.md` and
|
||||
`.claude/skills/x/SKILL.md` both create `/x`.
|
||||
- **Restricting skills.**
|
||||
- Permission rules: `Skill`, `Skill(name)`.
|
||||
- `skillOverrides`, `disable-model-invocation`, `--disable-slash-commands`
|
||||
[cc:skills § Restrict Claude's skill access].
|
||||
- A skill's `allowed-tools` pre-approves tools for one turn. It doesn't
|
||||
restrict anything, and workspace trust doesn't gate it
|
||||
[cc:skills § Pre-approve tools for a skill].
|
||||
- **Plugins, loaded per session.** `--plugin-dir` / `--plugin-url` bundle
|
||||
skills, agents, hooks, MCP and output styles [cc:help].
|
||||
|
||||
### 3.3 Hooks
|
||||
|
||||
- **Events.** These include SessionStart, UserPromptSubmit, PreToolUse,
|
||||
PermissionRequest, PostToolUse, Stop, SubagentStart/Stop and SessionEnd
|
||||
[cc:hooks § Hook lifecycle].
|
||||
- **Where hooks come from.** User, project, local and managed settings,
|
||||
plugins, skill and subagent frontmatter, and SDK callbacks. All of them
|
||||
merge [cc:hooks § Hook locations].
|
||||
- **How a hook blocks.** Exit 2 blocks PreToolUse, and JSON can't undo it.
|
||||
`hookSpecificOutput.permissionDecision: "deny"` also blocks. Deny rules are
|
||||
evaluated whatever a hook returns, so a hook's `allow` can't override them
|
||||
[cc:hooks § Exit code 2 behavior per event; cc:permissions § Extend permissions with hooks].
|
||||
- **Fail-open (checked in the fetched docs).**
|
||||
- Exit 1 or any other code is non-blocking.
|
||||
- A hook that can't start is non-blocking: "a mistyped path in
|
||||
`settings.json` leaves the gate silently disabled" (hooks.md line 837 of the
|
||||
fetched page).
|
||||
- Invalid JSON, a non-2xx HTTP response, or a command or HTTP timeout is
|
||||
also non-blocking [cc:hooks § Exit code output; § Timeouts].
|
||||
- **The exception is SDK callback hooks (checked).** On a PreToolUse timeout,
|
||||
"Claude Code doesn't run the tool call" [cc:sdk-hooks § Hook timeout]. They
|
||||
also survive `allowManagedHooksOnly` and `disableAllHooks`. What happens
|
||||
when a callback throws was not established.
|
||||
- **Coverage.**
|
||||
- Hooks fire inside subagents and for MCP tools (`mcp__server__tool`).
|
||||
- They do not fire for `@file` prompt references, and `--bare` skips them.
|
||||
- Hooks run outside the sandbox [cc:hooks; cc:sandboxing].
|
||||
- **Trust.** `-p` and SDK sessions treat the folder as trusted, so a repo's
|
||||
`.claude/settings.json` hooks run with no dialog [cc:hooks § Security considerations].
|
||||
|
||||
### 3.4 MCP
|
||||
|
||||
- **Scopes.** Local and user in `~/.claude.json`, project `.mcp.json`, plugin,
|
||||
and claude.ai connectors. Managed `managed-mcp.json` takes exclusive
|
||||
control [cc:mcp § MCP installation scopes; cc:managed-mcp].
|
||||
- **Per-session servers.** `--mcp-config <file|json>` with
|
||||
`--strict-mcp-config` uses only those servers [cc:cli-reference].
|
||||
- **`-p` connects `.mcp.json` servers without asking**
|
||||
[cc:headless § Start faster with bare mode].
|
||||
- **Tool names** are `mcp__<server>__<tool>`, and permission rules accept
|
||||
them [cc:permissions]. MCP servers run outside the Bash sandbox
|
||||
[cc:sandboxing].
|
||||
|
||||
### 3.5 Permissions and sandbox
|
||||
|
||||
- **Evaluation order.**
|
||||
1. PreToolUse hooks
|
||||
2. Deny rules
|
||||
3. Ask rules
|
||||
4. Permission mode
|
||||
5. Allow rules
|
||||
6. `canUseTool` or `--permission-prompt-tool`
|
||||
|
||||
Deny rules block in every mode, including `bypassPermissions`. "Permission
|
||||
rules are enforced by Claude Code, not by the model"
|
||||
[cc:permissions; cc:sdk-perm § How permissions are evaluated].
|
||||
- **Modes.** `default`, `acceptEdits`, `plan`, `auto`, `dontAsk` (denies
|
||||
anything that would prompt) and `bypassPermissions`
|
||||
[cc:permission-modes]. `--permission-prompts none` denies prompts unless a
|
||||
PermissionRequest hook allows them (v2.1.259+)
|
||||
[cc:headless § Turn off permission prompts in unattended runs].
|
||||
- **Bash rules are text matches.** `Bash(rm *)` does not catch `/bin/rm` or
|
||||
`bash -c 'rm …'` [cc:permissions § What a Bash rule doesn't match].
|
||||
- **Sandbox.** bubblewrap on Linux, covering Bash and its children only.
|
||||
- Off by default.
|
||||
- Keys: `sandbox.enabled`, `failIfUnavailable`, `allowUnsandboxedCommands:false`,
|
||||
`network.allowedDomains`, `filesystem.denyRead`, `credentials`
|
||||
[cc:sandboxing].
|
||||
- **Precedence.** Managed, then CLI/`--settings`, then local, then project,
|
||||
then user. A deny at any level can't be overridden
|
||||
[cc:settings § Settings precedence].
|
||||
- **`--restricted`.** Removes code-running tools and WebFetch, ignores user,
|
||||
project and local settings, and refuses bypass [cc:help].
|
||||
|
||||
### 3.6 Per-session environment and credentials
|
||||
|
||||
- **`CLAUDE_CONFIG_DIR`** relocates settings, sessions, plugins and
|
||||
`.credentials.json`. It is the documented way to run multiple accounts
|
||||
[cc:env-vars; cc:authentication § Credential management].
|
||||
- `~/.claude.json` (MCP and trust state) is not governed by
|
||||
`settingSources`. Relocate it with `CLAUDE_CONFIG_DIR`
|
||||
[cc:sdk-features § What settingSources does not control].
|
||||
- **Configuration per run.** `--settings <file|json>` fills the CLI layer.
|
||||
`--setting-sources ""` drops user, project and local; managed policy loads
|
||||
regardless [cc:cli-reference; cc:sdk-ts § Options].
|
||||
- **Credential precedence.**
|
||||
1. Cloud provider
|
||||
2. `ANTHROPIC_AUTH_TOKEN`
|
||||
3. `ANTHROPIC_API_KEY`
|
||||
4. `apiKeyHelper`
|
||||
5. `CLAUDE_CODE_OAUTH_TOKEN`
|
||||
6. `/login` OAuth
|
||||
|
||||
Source: [cc:authentication § Authentication precedence].
|
||||
- **`CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1`** strips credentials from Bash, hook
|
||||
and stdio-MCP subprocess environments [cc:env-vars].
|
||||
|
||||
### 3.7 Headless
|
||||
|
||||
- **`-p` output.** `--output-format text|json|stream-json`, `--json-schema`,
|
||||
`--max-turns`, `--max-budget-usd`. `-p` skips the trust dialog, and
|
||||
settings files that fail validation are "silently ignored" [cc:help].
|
||||
- **`--bare`.**
|
||||
- Skips hooks, plugins, CLAUDE.md, auto memory and keychain/OAuth.
|
||||
- Connects only command-line MCP servers.
|
||||
- Authenticates only from `ANTHROPIC_API_KEY` or an `apiKeyHelper`.
|
||||
- The docs say it "will become the default for `-p` in a future release"
|
||||
[cc:headless § Start faster with bare mode].
|
||||
- **Agent SDK.** `settingSources: []`, `hooks` (in-process callbacks),
|
||||
`canUseTool` (reached only when the flow falls through to a prompt),
|
||||
`disallowedTools`, `tools`, `mcpServers`, `strictMcpConfig`, and `env`
|
||||
(which replaces the environment) [cc:sdk-ts § Options].
|
||||
|
||||
---
|
||||
|
||||
## 4. Codex CLI 0.160.0
|
||||
|
||||
### 4.1 System prompt and context files
|
||||
|
||||
- **Base prompt.** `model_instructions_file`, then inline `instructions`,
|
||||
then the built-in prompt [cx:S:codex-rs/core/src/config/mod.rs ~L3990].
|
||||
- The docs call `instructions` "reserved". The source uses it.
|
||||
- `developer_instructions` becomes a developer-role message
|
||||
[cx:S:codex-rs/config/src/config_toml.rs].
|
||||
- **Global AGENTS.md.** `$CODEX_HOME/AGENTS.override.md`, else
|
||||
`$CODEX_HOME/AGENTS.md` [cx:S:codex-rs/codex-home/src/instructions/mod.rs].
|
||||
- The legacy `~/.codex/instructions.md` is not read. Inferred: the copy on
|
||||
this machine is inert.
|
||||
- **Project chain.** From the project root (`.git` marker) down to the cwd.
|
||||
- Files are concatenated root first, with a **total** budget of 32 KiB
|
||||
[cx:S:codex-rs/core/src/agents_md.rs]. The docs say the limit is per file.
|
||||
- The chain is skipped only when the project is explicitly
|
||||
`trust_level = "untrusted"` [cx:S:agents_md.rs L64].
|
||||
- **Config layers, low to high:**
|
||||
1. Defaults
|
||||
2. `/etc/codex/config.toml`
|
||||
3. Cloud bundle
|
||||
4. `$CODEX_HOME/config.toml`
|
||||
5. `$CODEX_HOME/<name>.config.toml` (`-p`)
|
||||
6. Project `.codex/config.toml` (trusted projects only)
|
||||
7. `-c` flags
|
||||
|
||||
Source: [cx:S:codex-rs/config/src/loader/mod.rs L107-134].
|
||||
|
||||
### 4.2 Skills and commands
|
||||
|
||||
- **Skill roots.** Project `<.codex>/skills`, repo `.agents/skills` (cwd up to
|
||||
the project root), user `~/.agents/skills` and `$CODEX_HOME/skills`
|
||||
(deprecated), admin `/etc/codex/skills`, bundled, and plugins
|
||||
[cx:S:codex-rs/ext/skills/src/host_roots.rs].
|
||||
- **Trust.** Inferred: repo `.agents/skills` loads with no trust check
|
||||
[cx:S:codex-rs/config/src/state.rs].
|
||||
- **Controls.** `[[skills.config]] enabled=false`;
|
||||
`policy.allow_implicit_invocation` [cx:D:skills].
|
||||
- **Custom prompts.** `~/.codex/prompts` has no loader in 0.160.0. Inferred:
|
||||
removed.
|
||||
|
||||
### 4.3 Hooks
|
||||
|
||||
- **Status.** Stable and on by default. Events include SessionStart,
|
||||
UserPromptSubmit, PreToolUse, PermissionRequest, PostToolUse, Stop and
|
||||
SessionEnd [cx:D:hooks].
|
||||
- **Sources.** `hooks.json` or a `[hooks]` table next to each config layer,
|
||||
plugins, and managed `requirements.toml` [cx:D:hooks#Where Codex looks for hooks].
|
||||
- **Trust.** Non-managed hooks are trusted by hash after review in `/hooks`.
|
||||
- `--dangerously-bypass-hook-trust` skips that check [cx:H:exec].
|
||||
- Project hooks need a trusted project.
|
||||
- `allow_managed_hooks_only` keeps only managed hooks.
|
||||
- **How a hook blocks.** PreToolUse blocks on `permissionDecision: "deny"`,
|
||||
legacy `{"decision":"block"}`, or exit 2 with non-empty stderr.
|
||||
`"ask"` and `continue:false` are unsupported, and the tool proceeds
|
||||
[cx:D:hooks#PreToolUse].
|
||||
- **Fail-open (checked in source).** In
|
||||
[cx:S:codex-rs/hooks/src/events/pre_tool_use.rs], a spawn error or timeout
|
||||
sets `Failed` with `should_block` false (L205-211). Exit 2 with empty
|
||||
stderr, invalid JSON, and any other exit code also set `Failed` without
|
||||
blocking (L254-286).
|
||||
- Async hooks can't block [cx:S:codex-rs/hooks/src/engine/mod.rs ~L154].
|
||||
- Hosted tools such as web search are not covered.
|
||||
- The docs: "Treat tool hooks as a useful guardrail, not a complete
|
||||
enforcement boundary" [cx:D:hooks#Tool coverage].
|
||||
- **Hooks run unsandboxed.** They run through the user's shell (inferred: no
|
||||
sandbox reference in the hooks crate).
|
||||
- **`notify`.** Fires after each turn, fire-and-forget.
|
||||
|
||||
### 4.4 MCP
|
||||
|
||||
- **Configuration.** Codex is a client over stdio and streamable HTTP.
|
||||
- Per-server keys include `enabled_tools`, `disabled_tools`, `required`
|
||||
and per-tool `approval_mode` [cx:D:config-reference].
|
||||
- Project servers load only in trusted projects.
|
||||
- In `exec`, a failing `required` server exits with an error
|
||||
[cx:D:noninteractive].
|
||||
- **Managed allowlist.** Requirements can pin a server's identity.
|
||||
- **Not sandboxed.** MCP servers run outside the sandbox
|
||||
[cx:D:permissions#What the network proxy does not control].
|
||||
- **`codex mcp-server` was removed** in 0.160.0 [cx:L].
|
||||
|
||||
### 4.5 Permissions and sandbox
|
||||
|
||||
- **Approval.** `-a on-request|never`; `untrusted` and `on-failure` are
|
||||
rejected by the CLI [cx:H:root]. Config also takes a `granular` table.
|
||||
- **Sandbox modes.** `read-only`, `workspace-write`, `danger-full-access`,
|
||||
and named `[permissions.<name>]` profiles [cx:D:permissions].
|
||||
- With no `sandbox_mode` set, any trust decision means `workspace-write`;
|
||||
otherwise the default is read-only
|
||||
[cx:S:codex-rs/config/src/config_toml.rs ~L826-834].
|
||||
- **Linux sandbox.** bubblewrap with `--ro-bind / /`, writable roots,
|
||||
`.git` and `.codex` re-bound read-only, `no_new_privs`, and a seccomp
|
||||
network filter [cx:S:codex-rs/linux-sandbox/src/bwrap.rs].
|
||||
- Network is off in `workspace-write` unless `network_access=true`.
|
||||
- Domain rules need the experimental `network_proxy`.
|
||||
- **Execpolicy.** `.rules` files with Starlark
|
||||
`prefix_rule(..., decision=allow|prompt|forbidden)`, next to each config
|
||||
layer. The most restrictive rule wins, and matching is on argv prefix only
|
||||
[cx:D:rules; cx:S:codex-rs/core/src/exec_policy.rs ~L395].
|
||||
- **Managed `requirements.toml`.** In `/etc/codex/` or the cloud bundle. It
|
||||
pins allowed approval policies, sandbox modes, MCP identities, hooks,
|
||||
rules, `deny_read` and features [cx:D:enterprise/managed-configuration].
|
||||
- **Bypass flags.** `--dangerously-bypass-approvals-and-sandbox` and
|
||||
`-s danger-full-access`. Inferred: only requirements can forbid them.
|
||||
|
||||
### 4.6 Per-session environment and credentials
|
||||
|
||||
- **`CODEX_HOME`.** Isolates config, auth, sessions, skills and hooks per
|
||||
process [cx:H:root].
|
||||
- **Auth.**
|
||||
- `CODEX_API_KEY` from the environment (exec and app-server).
|
||||
- Then ephemeral tokens.
|
||||
- Then the persisted store (`file`, `keyring`, `auto`, `ephemeral`)
|
||||
[cx:S:codex-rs/login/src/auth/manager.rs ~L1488].
|
||||
- `OPENAI_API_KEY` is not used for login.
|
||||
- **Model-run commands inherit secrets by default (checked).**
|
||||
`shell_environment_policy` defaults to `inherit = "all"`, and
|
||||
`ignore_default_excludes` defaults to true
|
||||
([cx:S:codex-rs/config/src/shell_environment_policy.rs:136]
|
||||
`unwrap_or(true)`). The `*KEY*`/`*SECRET*`/`*TOKEN*` excludes therefore
|
||||
don't apply unless set to false.
|
||||
- **`auth.json` is readable inside the sandbox** (inferred), unless
|
||||
`deny_read` or a keyring/ephemeral store is used.
|
||||
|
||||
### 4.7 Headless
|
||||
|
||||
- **`codex exec`.**
|
||||
- `--json` gives JSONL events. Other options: `-o FILE`, `--output-schema`,
|
||||
`--ephemeral`, `--skip-git-repo-check`, `exec resume`.
|
||||
- `exec` forces `approval_policy = never`
|
||||
[cx:S:codex-rs/exec/src/lib.rs L576].
|
||||
- Exits 0 on success and 1 on any failure.
|
||||
- `--full-auto` is rejected in 0.160.0.
|
||||
- **`codex app-server`.** JSON-RPC over stdio, unix or ws.
|
||||
- The host answers `item/commandExecution/requestApproval` and
|
||||
`item/fileChange/requestApproval` in-band [cx:D:app-server].
|
||||
- Inferred: no answer means no run, so it fails closed. It covers
|
||||
escalations only, not every tool call.
|
||||
- **TS SDK.** Wraps `codex exec --experimental-json`. Its `env` option
|
||||
replaces `process.env` [cx:S:sdk/typescript/src/exec.ts].
|
||||
|
||||
---
|
||||
|
||||
## 5. Side by side
|
||||
|
||||
| Area | Pi 0.85.1 | Claude Code 2.1.289 | Codex 0.160.0 |
|
||||
|---|---|---|---|
|
||||
| Replace the system prompt | `--system-prompt` (drops `SYSTEM.md` discovery) | `--system-prompt[-file]` | `model_instructions_file` ("strongly discouraged") |
|
||||
| Add to it | `--append-system-prompt` | `--append-system-prompt[-file]` | `developer_instructions`, `$CODEX_HOME/AGENTS.md` |
|
||||
| Turn off ambient context | `--no-context-files` | `CLAUDE_CODE_DISABLE_CLAUDE_MDS=1`, `--setting-sources ""`, `--bare` | Only an explicit `untrusted` project entry |
|
||||
| Per-session skills | `--skill <dir>` with `--no-skills` | `--plugin-dir`, `--add-dir` | `$CODEX_HOME/skills` (deprecated), plugins |
|
||||
| Pre-tool block | `tool_call`, **fail-closed** | PreToolUse, **fail-open**; SDK callback fail-closed on timeout | PreToolUse, **fail-open** |
|
||||
| Hard tool ceiling | `--tools` (registry filter) | deny rules, `--tools`, `--disallowedTools` | sandbox mode; no per-tool built-in allowlist was found |
|
||||
| MCP | none (extension only) | `--mcp-config` + `--strict-mcp-config` | `mcp_servers` in config, `enabled_tools` |
|
||||
| OS sandbox | none | Bash only (bwrap) | spawned commands (bwrap + seccomp) |
|
||||
| Config isolation | `PI_CODING_AGENT_DIR` | `CLAUDE_CONFIG_DIR` + `--settings` + `--setting-sources ""` | `CODEX_HOME` |
|
||||
| Secrets in the shell env | inherited | inherited unless `SUBPROCESS_ENV_SCRUB=1` | inherited by default |
|
||||
| Machine-wide lock | none | `/etc/claude-code` managed settings | `/etc/codex/requirements.toml` |
|
||||
| Headless | `-p`, `--mode json\|rpc`, SDK | `-p`, stream-json, Agent SDK | `exec --json`, app-server, SDK |
|
||||
|
||||
---
|
||||
|
||||
## 6. What exists today
|
||||
|
||||
### 6.1 Container worker path (`adapters/pi`)
|
||||
|
||||
`scripts/mosaic-task.mjs` runs `docker compose run --rm -T mosaic-agent`.
|
||||
Inside the container:
|
||||
|
||||
1. `src/run-agent.sh` checks the adapter name.
|
||||
2. `src/load-contracts.sh` writes `/var/lib/mosaic/system-prompt.md` from:
|
||||
1. CONSTITUTION and STANDARDS
|
||||
2. SOUL, or `MOSAIC_AGENT_SOUL_FILE`
|
||||
3. agent identity
|
||||
4. `/var/lib/mosaic/user/*.md`
|
||||
5. the mission
|
||||
3. `adapters/pi/adapter.sh` execs pi.
|
||||
|
||||
| Area | What the adapter does | Hard? |
|
||||
|---|---|---|
|
||||
| System prompt | `--system-prompt "<content>"` [repo:adapters/pi/adapter.sh:92], which also turns off `SYSTEM.md` discovery | n/a (advisory content) |
|
||||
| Ambient context | `--offline --no-extensions --no-skills --no-prompt-templates --no-themes --no-context-files` | Yes, for what loads |
|
||||
| Tools | `--tools $MOSAIC_TOOLS` [repo:adapters/pi/adapter.sh:48]. The role ceiling is intersected with the seat in `validateRole` / `resolve-role` [repo:scripts/mosaic-task.mjs:249-272, 674] | Yes (registry filter) |
|
||||
| Skills | `--skill` per `MOSAIC_SKILLS` dir; a missing dir exits 2 [:57-58] | n/a |
|
||||
| Hooks | none | — |
|
||||
| MCP | none | — |
|
||||
| Network | the role's `network` is declared and "enforced when network policy lands" [repo:scripts/mosaic-task.mjs:249-250]; `compose.yaml` sets no `network_mode` | **No** |
|
||||
| Credentials | `ZAI_API_KEY`, `ANTHROPIC_API_KEY` env [repo:compose.yaml:36-37]; pi `auth.json` mounted read-only [:43] | **No.** A role with `bash` (researcher has it) can print the env keys, and `read` can open the mounted `auth.json` (Pi has no path confinement) |
|
||||
| Headless | `-p "$MOSAIC_REQUEST"`, stdin empty, SIGKILL on timeout; stdout is the response, and exit 0 means success [repo:adapters/README.md:37-39] | text mode, so an LLM error exits 1 |
|
||||
| Sandbox | the container: user 1000:1000, data root mounted | Yes for the filesystem, **no** for the network |
|
||||
|
||||
The adapter contract [repo:adapters/README.md:24-27, 40-48] passes four
|
||||
inputs: the prompt file, the request, the provider and the model. It also
|
||||
requires that "adapters never read configuration files". It has no slot for
|
||||
a tool policy, a gate, skills chosen per role, MCP config or role
|
||||
credentials. The pi adapter takes tools and skills through compose
|
||||
environment variables that the contract table doesn't list. Adding a harness
|
||||
requires an allowlist entry; today the allowlist is
|
||||
`SUPPORTED_ADAPTERS = ["pi", "mock"]` [repo:scripts/mosaic-config.mjs:33].
|
||||
|
||||
### 6.2 Host Pi seats (`scripts/agent-host-dev.sh`)
|
||||
|
||||
The command is [repo:scripts/agent-host-dev.sh:135-141]:
|
||||
|
||||
`pi --approve --offline --no-context-files --no-extensions --no-skills
|
||||
--no-prompt-templates --no-themes --extension .pi/extensions/goal/index.ts
|
||||
--skill … --tools read,bash,edit,write,grep,find,ls,goal_report
|
||||
--append-system-prompt <context.md> …`
|
||||
|
||||
- **What it already does.** The prompt is a hashed snapshot. It assembles
|
||||
CONSTITUTION, STANDARDS, the seat's SOUL, USER.md, AGENTS.md and the seat's
|
||||
CONTEXT.md, records the sha256, and passes the result through
|
||||
`--append-system-prompt`. Context-file and skill discovery are off, and
|
||||
the `--tools` list is explicit.
|
||||
- **Gaps:**
|
||||
- It uses `--append-system-prompt`, not `--system-prompt`, so `SYSTEM.md`
|
||||
discovery is still on. `--approve` trusts project-local `.pi/`. Today
|
||||
the repo has no `.pi/SYSTEM.md` or `.pi/settings.json`, and
|
||||
`--no-extensions` keeps `.pi/extensions/` out. But a commit that adds
|
||||
`.pi/SYSTEM.md` would replace Pi's preamble in every host seat, and a
|
||||
`.pi/settings.json` would merge over the global settings.
|
||||
- There's no `PI_CODING_AGENT_DIR`. Every seat shares
|
||||
`~/.pi/agent/settings.json`, `auth.json` and `trust.json`. The model can
|
||||
read and edit any of them, because Pi has no path confinement.
|
||||
- There's no role binding. Seats get a fixed tool list, not a role ceiling.
|
||||
- There's no gate. The goal extension is the only extension, and it
|
||||
handles `before_agent_start`, `tool_result`, `agent_settled`,
|
||||
`session_start` and `session_shutdown`. None of these blocks a tool.
|
||||
|
||||
### 6.3 Rocko (`agents/rocko/launch.sh`, Claude Code)
|
||||
|
||||
The command is `claude --model sonnet --name Rocko --append-system-prompt
|
||||
"$(cat context.md)" --resume|--session-id` [repo:agents/rocko/launch.sh:86-87],
|
||||
registered through `scripts/mosaic launch --harness claude-code` [:8].
|
||||
|
||||
Nothing limits setting sources, CLAUDE.md, MCP or permissions, so:
|
||||
- Jason's `~/.claude/settings.json` applies. Its keys include `hooks`,
|
||||
`mcpServers`, `enabledPlugins` and `skipDangerousModePermissionPrompt`
|
||||
(values not read).
|
||||
- Project `CLAUDE.md` (`@AGENTS.md`) and `~/.claude/CLAUDE.md` load as a
|
||||
user message on top of the appended context. AGENTS.md therefore reaches
|
||||
the model twice.
|
||||
- Rocko's tools aren't limited by any role.
|
||||
|
||||
### 6.4 Codex
|
||||
|
||||
No adapter, launcher or seat uses Codex. Filbert's seat runs Pi with an
|
||||
`openai-codex` provider model, which is a different thing. Local
|
||||
`~/.codex/config.toml` has project `trust_level` entries and a
|
||||
`[shell_environment_policy.set]` table (values not read).
|
||||
|
||||
---
|
||||
|
||||
## 7. What a generator must emit
|
||||
|
||||
### 7.1 Inputs
|
||||
|
||||
- The role definition, `roles/<name>.json` version 2, with its contract file
|
||||
(Darkwing 1.2).
|
||||
- The resolved variables and their provenance: system, business, project,
|
||||
agent (Darkwing 2). The agent layer carries `harness`, `model` and
|
||||
thinking level.
|
||||
- `contracts/`, and the frozen action vocabulary (Darkwing 1.4).
|
||||
- Credential references, by service name. The generator never reads the
|
||||
values.
|
||||
|
||||
### 7.2 Outputs common to all harnesses
|
||||
|
||||
Each launch gets a directory, for example `<dataRoot>/launch/<run>/`. It is
|
||||
written once, made read-only, and every file's sha256 goes into the
|
||||
`session.launched` event.
|
||||
|
||||
1. **`prompt.md`** holds `contracts/`, the role contract, the resolved
|
||||
non-secret variables, and the role's authority rendered as prose:
|
||||
- what the role may do alone;
|
||||
- what needs a decision, and from whom;
|
||||
- how to raise one: name the verb or tool.
|
||||
|
||||
This is advisory. Its job is to keep the agent from reaching a block, not
|
||||
to be the block.
|
||||
2. **`policy.json`** holds the action-to-class map for this role instance,
|
||||
the tool ceiling, the network class, and the command classifier patterns
|
||||
from Darkwing 1.4 item 3. The gate reads it.
|
||||
3. **The gate.** One harness-neutral program maps `{tool, input}` to
|
||||
`{allow | block, reason, action, class}`, with a thin wrapper per
|
||||
harness. It applies one rule to every call: a typed tool call for a
|
||||
crossRole or gated action is blocked with a pointer to
|
||||
`mosaic decision raise`. A shell command that matches a classifier
|
||||
pattern is blocked the same way and logged as `hook` layer (advisory).
|
||||
4. **Skills.** The `ms-*` set chosen for the role, plus a generated
|
||||
decision-raising skill that documents the verbs.
|
||||
5. **Typed tools for vocabulary actions.** `decision.raise`, `message.send`
|
||||
and the role's withinRole actions are exposed as tools that call the
|
||||
`mosaic` verbs through the broker. On Pi they are extension tools; on
|
||||
Claude and Codex, a `mosaic` MCP server. This makes condition 2 in
|
||||
section 1 hold, and it is the only way a hook block can be exact.
|
||||
6. **A launch manifest.** argv, the names of environment variables (never
|
||||
values), and file digests.
|
||||
|
||||
### 7.3 Pi
|
||||
|
||||
| Output | Form |
|
||||
|---|---|
|
||||
| Prompt | `--system-prompt <launch>/prompt.md` (replaces the preamble and turns off `SYSTEM.md` discovery) |
|
||||
| Ambient off | `--no-context-files --no-skills --no-prompt-templates --no-themes --no-extensions --no-approve --offline` |
|
||||
| Skills | `--skill <dir>`, repeated |
|
||||
| Gate and typed tools | `-e <pinned gate extension>`. It reads `policy.json` and registers a `tool_call` handler plus the typed tools. A load failure exits 1, so a missing gate refuses to start. **Blocked by decision 31 for Console-bound seats** (section 8) |
|
||||
| Tool ceiling | `--tools <role ∩ project ∩ agent>`, including the typed tool names |
|
||||
| Isolation | `PI_CODING_AGENT_DIR=<launch>/pi-agent`, holding a generated `settings.json`, no `auth.json` unless one is required, and no `trust.json` |
|
||||
| Credentials | Provider key through env or `--api-key`. Role credentials stay in the broker, not in Pi's env. Anything in Pi's env is visible to `bash` |
|
||||
| Headless | `-p` (text mode, so an LLM error exits 1) for workers; `--mode rpc` for Console-bound seats |
|
||||
| Network | Only a container with `network_mode` set gives a hard line. Pi has no setting for it |
|
||||
|
||||
### 7.4 Claude Code
|
||||
|
||||
| Output | Form |
|
||||
|---|---|
|
||||
| Isolation | `CLAUDE_CONFIG_DIR=<launch>/claude`, `--setting-sources ""`, `CLAUDE_CODE_DISABLE_CLAUDE_MDS=1`, `CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`, `CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1`. Managed settings in `/etc/claude-code`, if any ever exist, still apply |
|
||||
| Prompt | `--system-prompt-file` or `--append-system-prompt-file <prompt.md>`. Both are hidden in 2.1.289 and need a probe. `--append-system-prompt "$(cat …)"` works today (Rocko) |
|
||||
| Settings | `--settings <launch>/settings.json`, which must contain: (1) `permissions.deny` for every built-in tool outside the ceiling, plus `mcp__mosaic__<tool>` for crossRole and gated actions; (2) `hooks.PreToolUse` with matcher `*`, pointing at the gate wrapper; (3) `permissions.disableBypassPermissionsMode: "disable"`; (4) `sandbox` with `enabled`, `failIfUnavailable`, `allowUnsandboxedCommands:false` and `network.allowedDomains` from the role's network class; (5) `sandbox.filesystem.denyRead` for credential paths |
|
||||
| Tool ceiling | `--tools <mapped list>`. Pi names map to Claude names: `read`→`Read`, `write`→`Write`, `edit`→`Edit`, `bash`→`Bash`, `grep`→`Grep`, `find`→`Glob`. `ls` has no one-to-one tool, so the mapping needs checking against [cc:tools-reference] |
|
||||
| Skills | `--plugin-dir <launch>/plugin` with `skills/`. Whether `--add-dir` skills still load under `--setting-sources ""` is unverified |
|
||||
| MCP | `--mcp-config <launch>/mcp.json --strict-mcp-config`, with the `mosaic` broker server only |
|
||||
| Gate failure mode | Command hooks fail open, so a blocked action must also be a deny rule. The wrapper must turn every internal error into exit 2 (`trap`). The launcher must run a canary call that the gate has to refuse before the session counts as launched; that catches the "mistyped path" case. In headless SDK runs, an in-process callback hook fails closed on timeout |
|
||||
| Credentials | Subscription OAuth lives in `.credentials.json` under the config dir, so a private `CLAUDE_CONFIG_DIR` needs its own login, a `CLAUDE_CODE_OAUTH_TOKEN` (`claude setup-token`), or `ANTHROPIC_API_KEY` (section 8, item 5). `--bare` would be the stronger hardening, but it accepts only an API key or `apiKeyHelper` |
|
||||
| Headless | `-p --output-format stream-json --permission-mode dontAsk --max-turns N`, or the Agent SDK with `settingSources: []` and callback hooks |
|
||||
|
||||
### 7.5 Codex CLI
|
||||
|
||||
| Output | Form |
|
||||
|---|---|
|
||||
| Isolation | `CODEX_HOME=<launch>/codex` with a generated `config.toml`. Give the project path an explicit trust entry: `untrusted` keeps project config, hooks, rules and the AGENTS.md chain out; `trusted` lets them in |
|
||||
| Prompt | `developer_instructions` from `prompt.md`, or `$CODEX_HOME/AGENTS.md`. `model_instructions_file` replaces the built-in prompt but is "strongly discouraged" |
|
||||
| Skills | `$CODEX_HOME/skills` (deprecated) or a plugin. `~/.agents/skills` is shared user state, not per session. Repo `.agents/skills` loads without a trust check (inferred). This repo has no `.agents/` today |
|
||||
| Gate | `[hooks]` PreToolUse with a command pointing at the gate wrapper. Fail-open, and it needs hook trust: either pre-recorded hash trust or `--dangerously-bypass-hook-trust`. Back it with execpolicy rules in `$CODEX_HOME/rules/` that `forbidden` the classified commands (`git push`, `git merge`, `tea`, `docker push`, `npm publish`); those are enforced before exec, by argv prefix |
|
||||
| Sandbox | `sandbox_mode = "workspace-write"` set explicitly; `sandbox_workspace_write.network_access` from the role's network class; a named profile with `[permissions.<name>.filesystem]."<credential path>" = "deny"` [cx:D:permissions]. These docs may describe a newer release than 0.160.0, so a probe must confirm it; the admin form is requirements `permissions.filesystem.deny_read`. Hooks and MCP servers run outside it |
|
||||
| Env | `shell_environment_policy`: `inherit = "core"` (or `include_only`) and `ignore_default_excludes = false` |
|
||||
| MCP | `[mcp_servers.mosaic]` with `required = true` and `enabled_tools` |
|
||||
| Credentials | `CODEX_API_KEY` (exec and app-server only) or `cli_auth_credentials_store = "ephemeral"`. Role credentials stay in the broker |
|
||||
| Headless | `codex exec --json` (exit 0/1, approval forced to `never`), or app-server JSON-RPC, where the host answers escalation approvals in-band |
|
||||
| Lock | Only `/etc/codex/requirements.toml` can stop a flag such as `--dangerously-bypass-approvals-and-sandbox`. Inferred: since the launcher owns the argv, the agent can't add flags, so slice 1 doesn't need the lock |
|
||||
|
||||
### 7.6 Changes to `adapters/`
|
||||
|
||||
- **The adapter contract needs a version bump.**
|
||||
- New input: `MOSAIC_LAUNCH_DIR`, which points at the generated, read-only
|
||||
bundle (prompt, policy, settings, skills, MCP config, manifest).
|
||||
- The rule "adapters never read configuration files" needs restating: the
|
||||
adapter reads only the generated bundle, never the system or business
|
||||
config.
|
||||
- The undocumented `MOSAIC_TOOLS` and `MOSAIC_SKILLS` inputs move into
|
||||
the bundle.
|
||||
- **`SUPPORTED_ADAPTERS`** gains `claude` and `codex` only when their
|
||||
adapters exist and pass the suites.
|
||||
- **Two kinds of agent.** Darkwing 6.5 separates role agents (interactive,
|
||||
with a role binding and git) from workers (no git, no credentials). The
|
||||
generator serves both, but the worker path must keep its invariant: no
|
||||
`git.*` and no credential-bearing typed tools.
|
||||
|
||||
---
|
||||
|
||||
## 8. Points for Sage, each with a recommendation
|
||||
|
||||
1. **Decision 31 against a Pi gate extension.** Pi's only fail-closed hook
|
||||
is an extension, and decision 31 bars explicit extensions in
|
||||
Console-bound seats until CHAT-06.
|
||||
- Recommendation: the gate extension enters through the CHAT-06 path. It
|
||||
should be a single file with no imports outside its pinned tree,
|
||||
digest-checked at launch.
|
||||
- Until then, Pi role seats get no hook, and the hard lines are the
|
||||
credential scope, the verbs and `--tools`. Nothing that matters is lost,
|
||||
because the hook was never the boundary.
|
||||
2. **Brief wording.** The slice 1 brief should name the hard layers
|
||||
(credential scope, verbs, tool ceiling, container) and call hooks early
|
||||
refusal plus logging. This agrees with Darkwing 1.4, item 3.
|
||||
3. **Machine-wide locks are out of scope.** `/etc/claude-code` and
|
||||
`/etc/codex/requirements.toml` would also govern Jason's own sessions.
|
||||
That makes them a policy change, which is gated. Recommendation: don't
|
||||
emit them in slice 1. Flags set by the launcher are enough, because the
|
||||
agent can't change its own argv.
|
||||
- One caveat: a same-user agent can edit its generated config files
|
||||
before a resume or reload. Claude notices settings changes during a
|
||||
session (it has a `ConfigChange` event for user, project, local and
|
||||
policy settings, [cc:hooks]). How a `--settings` file is treated when it
|
||||
changes is unverified. Pi reads the agent dir again on `/reload`. A read-only mode on the
|
||||
launch dir doesn't stop the same user from changing it back. Only a
|
||||
container or a separate OS user closes that.
|
||||
4. **Host-seat trust (`--approve`).** It's latent, not live.
|
||||
Recommendation:
|
||||
- the generator emits `--no-approve` and `--system-prompt`;
|
||||
- for `scripts/agent-host-dev.sh`, record a bounded follow-up (it isn't
|
||||
my file to change).
|
||||
5. **Claude credentials for isolated seats.** A private `CLAUDE_CONFIG_DIR`
|
||||
loses the shared subscription login. The options are a per-role
|
||||
`claude setup-token` OAuth token (credential minting, gated) or an API
|
||||
key (spend, gated). Either way it's Jason's decision, so it belongs with
|
||||
the per-role credentials step (step 7) and shouldn't be decided here.
|
||||
6. **Provider keys visible in workers.** This exists today and isn't new to
|
||||
slice 1, but slice 1 role tokens must not travel the same way.
|
||||
- The container passes `ZAI_API_KEY` and `ANTHROPIC_API_KEY` in the
|
||||
environment, and Pi's `bash` inherits it.
|
||||
- Pi's `read` can open the mounted `auth.json`.
|
||||
|
||||
Recommendation: role credentials stay in the broker process (Darkwing
|
||||
6.3) and never enter the agent's environment or filesystem.
|
||||
7. **Order.** As the direction doc says: Pi first, with the generator, the
|
||||
bundle and a gate wrapper that also works as a plain verb-side check;
|
||||
then Claude; then Codex. Codex is entirely new work with the weakest
|
||||
hooks, and app-server approvals cover escalations only.
|
||||
|
||||
---
|
||||
|
||||
## 9. Verification
|
||||
|
||||
**Checked myself, against source or fetched docs:**
|
||||
- **Pi `tool_call` block and fail-closed paths:**
|
||||
- `runner.js:745-763`;
|
||||
- `agent-session.js:230-243`, which wraps a non-Error throw as "Extension
|
||||
failed, blocking execution";
|
||||
- `agent-loop.js:400-458`;
|
||||
- `docs/extensions.md:2924-2925`;
|
||||
- the same strings and `isAllowedTool` are present in the shipped bundle
|
||||
chunk.
|
||||
- **Pi `--no-extensions`:** loads only the CLI `-e` set
|
||||
(`resource-loader.js:409-411`).
|
||||
- **Pi `--approve`:** means "Trust project-local files for this run"
|
||||
(`README.md:615`, `args.js:205, 307`).
|
||||
- **Pi `bash` environment:** spreads `process.env` (`utils/shell.js:117-123`).
|
||||
- **Pi allowlist:** `isAllowedTool` filters (`agent-session.js:2105-2160`).
|
||||
- **Codex PreToolUse:** spawn error or timeout sets `Failed` and doesn't
|
||||
block; any other exit code sets `Failed`
|
||||
(`hooks/src/events/pre_tool_use.rs:205-211, 278-286`).
|
||||
- **Codex `ignore_default_excludes`:** defaults to true
|
||||
(`config/src/shell_environment_policy.rs:136`).
|
||||
- **Claude command hooks:** "a mistyped path … leaves the gate silently
|
||||
disabled" (fetched hooks page).
|
||||
- **Claude SDK callbacks:** a PreToolUse timeout means the tool isn't run
|
||||
(fetched `agent-sdk/hooks` page, "Hook timeout").
|
||||
- **Repository line references** in section 6.
|
||||
|
||||
**Not checked (subagent findings I didn't re-read):**
|
||||
- **Not exercised at all.** Nothing was run against a live harness: no hook
|
||||
probes and no canary runs. The hidden Claude flags
|
||||
(`--system-prompt-file`, `--max-turns`) were not exercised.
|
||||
- **Still inferred.** These remain marked as inferred in the text:
|
||||
- the Codex project-skills trust gap;
|
||||
- Codex's readable `auth.json`;
|
||||
- the Pi json-mode exit code.
|
||||
- **Not established.** What happens when a Claude SDK callback hook throws.
|
||||
|
||||
Before the build brief relies on a block, a probe matrix should run each
|
||||
harness at its pinned version, with these cases:
|
||||
- a gate that blocks;
|
||||
- a gate that crashes;
|
||||
- a gate at a missing path;
|
||||
- a gate that times out;
|
||||
- a `bash` route to the same action.
|
||||
|
||||
**A note on the research workspace.** The Codex and Claude subagents briefly
|
||||
shared one docs scratch directory, and some Claude doc files may have been
|
||||
overwritten with Codex pages. The Claude agent refetched into its own
|
||||
directory before writing. I spot-checked the two Claude claims that matter
|
||||
most in its own copy, and both are Claude pages.
|
||||
Reference in New Issue
Block a user