diff --git a/agents/researcher/work/i1-proxy-test/REPORT.md b/agents/researcher/work/i1-proxy-test/REPORT.md new file mode 100644 index 000000000..5afa80f56 --- /dev/null +++ b/agents/researcher/work/i1-proxy-test/REPORT.md @@ -0,0 +1,274 @@ +# Row 96 (#1590): stand-in-key test of CLIProxyAPI for role sessions + +Author: Researcher seat. Brief: `docs/plans/2026-10-12_local-model-proxy.md` § "Stand-in-key +test of CLIProxyAPI for role sessions". Reviewer: Dewey. +Evidence for every claim is in `out/`. The numbers below come from the final run +(`out/run.log`, started 2026-10-13T20:57:40Z). + +## Verdict in short + +- A second CLIProxyAPI process carries both pinned clients (Claude Code 2.1.296, pi 0.85.1) + through a streamed turn and a tool round trip, using only stand-in keys. +- Claude Code works with `ANTHROPIC_BASE_URL` + `ANTHROPIC_AUTH_TOKEN` as the session + allowlist stands. **Pi does not: it ignores `ANTHROPIC_BASE_URL`.** It needs a + `models.json` in the session's `HOME` (details in "Pi finding"). +- Per-agent keys work from the command line: issue = 1 PUT, revoke = 1 DELETE, no restart, + effective on the next request. Keys survive a proxy restart. +- The upstream sees only the proxy's own key. The client key never leaves the proxy. +- What does not hold or is weak: attribution by client key (queue-only, 60 s window, plaintext + keys), the revoke call's key in the access log, silent multi-minute retry in Claude Code + on 401, and every management write rewriting the whole config file. See "What doesn't hold". + +## Bounds kept + +- No real credential, no spend, no request off the machine. Everything runs inside + `unshare -rn`: a private network namespace with only `lo`. The test processes cannot reach + the live proxy on 127.0.0.1:8317 or anything else. `out/run.log` line 3 lists the listeners + visible inside the namespace (18317, 18400 only). +- The running proxy was not called, restarted or configured. Its management key and config + content were never read. `run-all.sh` records before/after: pid 113963, start time, argv, + the 8317 listener, and `stat` (mtime, size) of its config. All identical + (`out/host-guard.before.txt`, `.after.txt`, `.diff` empty, `host-guard.txt`). +- Files under the live proxy's data directory (`core-start-output.log`, `usage.db`) did change + during the run. Those belong to the live proxy serving other clients (2 to 6 established + connections to 8317 from other host processes at the time). The test cannot touch them from + inside the namespace. They are listed as informational in `out/host-activity.txt`, not in the + verdict, because they would change whatever the test does. +- Nothing left behind: `cleanup.sh` ran, `out/host-guard.txt` says "left behind outside out/: 0". + +## Proxy version, config fields, endpoints + +`cli-proxy-api -h` and the startup line: **CLIProxyAPI 8.0.27, commit bf934162, built +2026-10-12T16:37:54Z** (`out/run.log` line 4; flags in `docs/proxy-help.txt`). +Started as `cli-proxy-api -config state/config.yaml -local-model` under `env -i HOME=… PATH=…`. + +Config fields used, from `docs/config.example.v8.0.27.yaml` (the example config shipped +with 8.0.27; the line numbers are in that file): + +| Field | Line | Used for | +|---|---|---| +| `config-version` | 10 | `8` | +| `server.host` / `port` | 24 | 127.0.0.1:18317 | +| `management.secret-key` | 89 | made-up key; an empty value disables management | +| `management.allow-remote` | 84 | `false` | +| `management.disable-control-panel` | 92 | `true` (no WebUI) | +| `access.api-keys` | 106, 108 | client keys the proxy accepts | +| `routing.retry.request-retry` | 149, 194 | `0` (no hidden proxy retries) | +| `oauth.auth-dir` | 884 | inside the packet, so request logs land there | +| `observability.logs.debug` / `logging-to-file` / `request-log` / `logs-max-total-size-mb` | 1114 to 1131 | per-request logs; size cap 200 MB | +| `observability.usage.usage-statistics-enabled` | 1133, 1135 | `true` | +| `observability.usage.redis-usage-queue-retention-seconds` | 1140 | default 60 (not set) | +| `api-keys.claude[]` (`name`, `base-url`, `keys[].api-key`) | ~650 to 720 | the proxy's own upstream: the stand-in server and its made-up key | + +Management endpoints used, from `docs/management-api-v8.0.27.md`. Auth is +`Authorization: Bearer ` (or `X-Management-Key`): + +| Endpoint | Used for | +|---|---| +| `GET/PUT /v8/management/config/access/api-keys` | list, replace the client-key list | +| `DELETE /v0/management/api-keys?value=` | remove one client key | +| `GET /v8/management/observability/usage/api-keys` | usage counters | +| `GET /v8/management/observability/usage/queue` | per-request usage events | +| `GET /v8/management/observability/logs` | log lines | + +Also noted in the doc and not used: `PATCH` on the `access/api-keys` path, and +`/observability/logs/requests/`. I did not test whether a v8 DELETE exists; the v0 one worked. + +## Stand-in upstream + +`stand-in-upstream.mjs`: HTTP on 127.0.0.1:18400, accepts only +`standin-upstream-key-0001` (`Authorization: Bearer` or `x-api-key`), records every request +as JSONL (`out/upstream.all.jsonl`), answers like the Messages API: SSE stream in three chunks, +a `Read` tool call when the prompt contains `TOOL-TEST`, and `ROUNDTRIP-OK: ` +when the history ends in the matching `tool_result`. It also answers `count_tokens` and +`/v1/models`. + +## Client runs + +Both clients ran through the repo's real adapters (`adapters/claude/adapter.sh`, +`adapters/pi/adapter.sh`) under `env -i` with only the session allowlist +(`packages/seat/src/session.mjs` `ENV_ALLOW`: `HOME`, `ANTHROPIC_API_KEY`, +`ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_BASE_URL`, `MOSAIC_CLAUDE_BIN`, …) plus the adapter's +`MOSAIC_*` inputs. Claude ran with `"}}}`, +no `/v1` suffix. `HOME` is on the allowlist, so a per-session `HOME` containing that file +works with `ANTHROPIC_API_KEY` carrying the client key. `inner.sh` kind `pi` does exactly this +(`state/home-