jason.woltje
38cfa72335
The usage queue read drains the proxy. A batch that is then not written (the client or the bus refuses its shape, an ambiguous key owner, a failed write) is lost, and the Console showed proxy totals with no sign of it. The reader now counts these per enabled business since the bus started, the proxy view returns the count as droppedBatches, and the Console proxy section says so above 0. A read that failed before the proxy answered or that it refused drained nothing and is not counted. Telemetry only; no quota reads it. The log line stays proxy-usage-read-failed.
5.2 KiB
Optional CLIProxyAPI queue telemetry (P5)
Enable through the human plugins.set transport with plugins.usage set to
"cliproxy-queue". Omission is off. The same patch may set
usage.cliproxyQueue.pollSeconds (integer 1 through 59, default 15); usage: null
removes this setting. Both switches and polling changes are human-only, including
on the tester path. The existing instance configuration and proxy key paths do
not change.
Interface 1 exports read({ readUsageQueue }). The bus hands the module one
frozen object holding one function, its management client's queue read, and
nothing else (row 333 R3), so no key write, credential call or OAuth call is
reachable from it. The module never reads credentials, configuration paths or
the environment. Each read performs exactly one bounded GET of
/v8/management/observability/usage/queue?count=100 per poll. Existing fixed
loopback, timeout, response byte cap and no-redirect/no-retry controls apply.
The proxy's array response is validated completely and mapped immediately:
api_key and source become their first 12 hexadecimal SHA-256 characters.
Only whitelisted token counts, a failure flag/status, and IDs leave the client.
No request IDs, failure bodies, client values or upstream values are retained.
Malformed batches fail atomically and yield a fixed, value-free failure code.
One bus/store writer owns the reader for its instance. It serializes polls,
drains the shared queue once, and assigns IDs to current and retiring model-key
metadata owners. Unknown msk- IDs go to the reconcile receipt. Foreign keys
without that prefix are ignored. Disabled businesses receive no telemetry.
Telemetry uses the existing append-only usage table with source
@proxy/cliproxy-queue, separate from native session source records. turnRecords
excludes it before every limit check and native usage total. Proxy totals may be
incomplete (60-second upstream retention and a 100-event drain); the Console
labels them proxy-reported and reports the reader's availability. They never
independently enforce quotas. A batch the proxy has already given up and the
bus then does not save (the client refuses its answer, the bus refuses its
shape or an ambiguous key owner, or the write fails) is lost. The reader counts
these per business since the bus started and the usage view returns the count
as droppedBatches (0 when none); the Console shows it above 0. A read that
failed before the proxy answered, or that the proxy refused, drained nothing
and is not counted. The count is in memory, restarts at 0 and is not a quota
input.
Native transcript counts carry the immutable run's safe modelKeyId in the
usage cursor, preserved at turn begin and end. The Console exposes byKey.
A business can set a daily per-key quota at
launch.limits.tokens.keys.<12-hex-key-id> with {limit, since, reset, timeZone};
the human writer stamps since. Enable limit-check: ["daily-quota"] to apply
that quota. Plain {limit,since} keys leaves remain cumulative legacy account limits, even
when the name is twelve hex characters. They use token-limit and never resolve
model metadata. Omitting a limit means no quota. Existing named account keys and
tokens.daily.<account> retain their existing behavior. Model-key limits and
resets are human-only through every organization writer. Missing, ambiguous or
foreign run-key metadata fails closed with proxy-key-refused; a run bound to a
retiring key keeps that key id in its usage records.
A key quota follows the agent its key id belongs to, across rotations and
re-issues, for as long as the agent stays hired: it applies to any key in that
agent's lineage (current, retiring and up to 64 earlier ids) and counts the
whole lineage, while usage records and the Console's per-key counts stay per key
id. A fire removes the agent's model record, so a re-hired agent starts a new
lineage and a quota named by an old id matches no agent: it limits nothing, the
Console usage view lists it, and the bus logs one value-free
limit-key-orphaned line when it becomes orphaned. An agent with no model key
under a key quota refuses key-quota-no-model-key; an id in two agents'
lineages, or a run bound to an id the agent no longer holds, refuses
proxy-key-refused. The 64 is a limit: once an id falls out of the newest 64,
its quota is orphaned while the agent stays hired.
After a rotation no move is needed. A quota moved to the agent's new key id
keeps its count (row 333 R4): the human writer gives a new key quota the
earliest since among the old file's key quotas whose ids are in the same
agent's lineage, asking the bus for that lineage (ids only, from its own
credential records), and stamps now only when none of them holds one. Only an
explicit reset restarts the count. A quota removed in one write and added back
in a later one is a new quota and gets now. If the lineage can't be read, a
write that adds a key quota refuses proxy-key-refused.
The orphan line is logged at the first limits refresh that finds the quota orphaned, which can be at boot or before a launch refusal. A quota removed and added back logs again. The bus keeps what it has logged in memory only, so a restart is a fresh observation and a quota still orphaned logs once more.