--- title: Lease-broker operations type: runbook audience: developer status: current source_of_truth: false --- # Lease-broker operations > **Status:** Current for repository inspection and test verification. Live broker > startup, recovery, lease mutation, service management, and cleanup remain > **held/non-operative**. > > **Operational authority:** This page authorizes only the non-mutating > inspection and test commands in [Safe static inspection](#safe-static-inspection). > It does not authorize starting a daemon or systemd unit, connecting to a live > socket, invoking recovery, promoting or revoking a lease, executing a > consequential runtime tool, or changing a database. The lease broker is a Linux-only internal process boundary. Its executable behavior is authoritative in the [shipped broker daemon](../../../packages/mosaic/framework/tools/lease-broker/daemon.py) and tests, not in this page. The current architecture references are: - [Broker protocol](../architecture/lease-broker-protocol.md) — framing, kernel identity, ancestry, generations, persistence, and state transitions. - [Lease-broker security](../architecture/lease-broker-security.md) — trust boundaries, filesystem hardening, observer behavior, and residuals. - [Whole mutator-class gate](../architecture/mutator-class-gate.md) — default deny, launch choke points, and broker-owned promotion order. - [Compaction revocation](../architecture/compaction-revocation.md) — Claude and Pi lifecycle observers, generation fencing, and the bounded residual. ## Safe static inspection These are the only operative commands documented here. They inspect checked-in files or run isolated tests; they do not start a user service, activate a runtime, connect to PostgreSQL, or mutate repository/product state. ### Source and launch inventory From the repository root, inspect the current implementation and its permanent runtime-launch inventory: ```bash find packages/mosaic/framework/tools/lease-broker packages/mosaic/src/lease-broker packages/mosaic/src/mutator-gate -maxdepth 1 -type f -print | sort python3 packages/mosaic/framework/tools/lease-broker/check-runtime-launches.py --root . --json ``` The launch inventory is a static completeness guard. A clean result means the checked-in production launch sites are classified by the guard; it does not prove that a broker, runtime, or service is running. ### Safe unit and static contract tests The focused standard-library tests can be run directly: ```bash python3 packages/mosaic/src/lease-broker/daemon_deadline_unittest.py python3 packages/mosaic/src/lease-broker/normative_fragments_unittest.py python3 packages/mosaic/src/lease-broker/receipt_challenge_unittest.py python3 packages/mosaic/src/lease-broker/context_recovery_unittest.py python3 packages/mosaic/src/lease-broker/state_store_unittest.py python3 packages/mosaic/src/lease-broker/framework_skill_portability_unittest.py python3 packages/mosaic/src/mutator-gate/runtime_tools_unittest.py python3 packages/mosaic/src/mutator-gate/runtime_launch_guard_unittest.py python3 packages/mosaic/src/mutator-gate/version_coupling_unittest.py ``` `recovery_runtime_unittest.py` and `recovery_b1_adversarial_unittest.py` are also safe when run as tests: they use private temporary daemons, sockets, and fixtures, never the installed user service or a model stream. They are not operator recovery instructions: ```bash python3 packages/mosaic/src/lease-broker/recovery_runtime_unittest.py python3 packages/mosaic/src/lease-broker/recovery_b1_adversarial_unittest.py ``` The [lease-broker Vitest acceptance suite](../../../packages/mosaic/src/lease-broker/lease-broker.acceptance.spec.ts) and [mutator-gate Vitest acceptance suite](../../../packages/mosaic/src/mutator-gate/mutator-gate.acceptance.spec.ts) provide additional private-fixture coverage. Test-created child processes and Unix sockets are disposable test fixtures, not live-service authority. ## Current implementation facts ### Protected paths and persistence The broker and supervisor source establish these invariants: | Object | Current contract | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Parent directory | The parent containing the socket and state must already exist and have exactly mode `0700`. | | Broker socket | The daemon refuses an existing path or symlink, binds a new Unix socket, sets it to `0600`, and removes only the inode it created during normal shutdown. | | State file | A present state file must be a regular, non-symlink file protected as mode `0600`, no larger than 4 MiB, and valid state-version-1 JSON. Corrupt, incompatible, oversized, or unsafe state refuses startup. | | State writes | Changes are serialized through a mode-`0600` temporary file, complete-write loop, `fsync`, atomic replace, and parent-directory `fsync`. Post-replace durability uncertainty poisons the store and terminates service processing. | | Volatile authority | `VERIFIED` leases are not restored as live authority after broker restart. Persisted session identity and valid pending-token state are separate from volatile lease state. | | Runtime generation | `launch-runtime.py` creates `generation-.state` beside the socket. It is an owner-only, locked, monotonic generation file; unsafe, non-regular, oversized, or non-private state fails closed. | The resolved socket path is, in order: explicit `MOSAIC_LEASE_BROKER_SOCKET`, `$XDG_RUNTIME_DIR/mosaic-lease/broker.sock`, or `/run/user//mosaic-lease/broker.sock`. The state file is `state.json` next to that resolved socket. The production observer transport is a separate `receipt-observer.sock` by default; `--test-observer-file` is a private test fixture option, not a production deployment path. The supervisor implementation can materialize a user unit, wrapper, and co-located daemon sources under caller-supplied paths, but [`applyBrokerSupervisor`](../../../packages/mosaic/src/lease-broker/broker-supervisor.ts) never runs `systemctl`, starts `daemon.py`, or enables the unit. The checked-in unit and [`start-lease-broker.sh`](../../../packages/mosaic/framework/tools/lease-broker/start-lease-broker.sh) are therefore implementation inputs, not live activation authority for this page. ### Protocol and identity boundary The protocol accepts one UTF-8 JSON object followed by one newline, capped at 64 KiB. The client must half-close its write side after the newline and before waiting for the response (`shutdown(SHUT_WR)` for POSIX clients or `socket.end()` for Node). Unterminated, multiple, delayed-second, malformed, oversized, or deadline-exceeded requests fail closed. This is an internal Unix socket protocol, not an HTTP/OpenAPI endpoint. The broker obtains `(pid, uid, gid)` from kernel `SO_PEERCRED`, binds the session to the anchor's `/proc//stat` starttime, and revalidates every walked ancestor's starttime. A caller cannot choose `session_id`; a sibling or unrelated process cannot authenticate with another process's session. A higher runtime generation replaces the prior incarnation and revokes its tokens and lease authority; a lower generation is stale. ### Lease and tool authorization The broker is the sole lease writer. The effective default-deny policy is: - Claude read-only classes: `Read`, `Grep`, `Glob`, `Ls`, and `Find`. - Pi read-only classes: `read`, `grep`, `find`, and `ls`. - Both runtimes expose the fixed `mosaic_context_recover` identity as the constrained recovery exception. - Every other built-in, unknown, custom, MCP, shell, edit, write, deployment, provider, or filesystem mutator is consequential and is denied while the session is not `VERIFIED`. The normal broker transition is revoke-first and promote-last: 1. `begin_verification` authenticates the broker-minted session and current generation, revokes existing authority and pending tokens, validates the exact source construction and binding, and enters a pending state. 2. The receipt challenge and binding are broker-generated. A trusted observer must provide the exact current-cycle assistant entry; caller-supplied `latest_assistant_message` is rejected on the public broker socket. 3. Promotion consumes the evidence-backed one-time token before volatile `VERIFIED` becomes visible. A token or receipt cannot be replayed against a later cycle. 4. `revoke_lease`, generation replacement, broker restart, or monotonic TTL expiry removes consequential-tool authority. TTL is positive and capped at 300 seconds. `launch-runtime.py` is the register-before-`exec` choke point for Claude and Pi. It performs the activation-capability version check, registers the anchor, creates the private generation file, exports the broker identity to descendants, and only then executes the requested runtime. Registration, capability, generation-file, broker-reply, or `exec` failure denies launch. Claude's raw `--dangerously-skip-permissions` flag is owned by this wrapper; callers request only its semantic dangerous mode. Claude's all-tools `PreToolUse` hook and Pi's `tool_call` handler submit the runtime-reported tool name to the gate. The gate does not inspect a shell string to decide that one command is safe. Missing identity, malformed input or reply, timeout, broker unavailability, unsafe generation state, and denial all fail closed. ### Lifecycle and recovery boundaries Claude and Pi lifecycle observers use the same authenticated session and broker state machine. Compaction revocation and same-PID replacement behavior are summarized below: | Runtime event | Current source-backed behavior | | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | Claude `PreCompact` | Revoke before compaction; a failed hook blocks the lifecycle transition. | | Claude `SessionStart` matcher `compact` | Revoke again after compacted context starts. | | Claude `SessionStart` matcher `resume\|clear` | Bump the private generation, then revoke the replacement incarnation. | | Pi `session_before_compact` | Revoke before compaction; failure cancels the transition. | | Pi `session_compact` then first `context` | Run one independent post-compaction revoke; a failed post-observer locally blocks later tools until retry. | | Pi `session_start` reason `reload`, `new`, `resume`, or `fork` | Bump the private generation before revoking and reusing the replacement session. | | Fired observer with unavailable broker | Advance the private generation as a local fence and return failure; do not continue consequential work. | The [source-resident context-refresh skill](../../../packages/mosaic/framework/skills/mosaic-context-refresh/SKILL.md) is a thin adapter over `recover-context.py`. Recovery begins with a validated normative-fragment construction, broker-side revoke-first transition, and a fresh `PENDING_DELIVERY` challenge. The trusted observer records only finalized Claude Stop-hook or Pi `message_end` content. Completion supplies no receipt or challenge argument; the broker obtains its current challenge, checks the exact observed entry, consumes evidence, and promotes last. Recovery does not accept normal-path receipt text as replayable authority. A receipt is only a delivery/liveness prerequisite. It is not a safety, obedience, comprehension, or residency proof. Absent, malformed, prefix-truncated, and observably mutated terminal receipts do not promote. A tail-preserving middle drop is explicitly not receipt-detectable and remains a server-side/T-C residual. ### Named residual If both compaction observers are missed while a lease remains unexpired, consequential tools are **ALLOWED** inside the bounded residual stale window. The mutator gate makes no within-window action-count or timing claim. After the monotonic TTL expires, the next consequential tool is **DENIED**. This is separate from a fired observer that cannot reach the broker, which fails closed through lifecycle cancellation, a generation fence, and/or the Pi local latch. Protected-branch controls and required review/CI remain the irreducible server-side backstop for the T-C total-hook-miss boundary. ## Held future procedure — non-operative There is no current command authority for the following live procedures. The sequence below records source-backed intent for a separately approved activation/recovery work package; it must not be copied into an operator shell. ### Startup and restart outline (held) 1. An activation owner would first materialize the exact framework unit, wrapper, and co-located source copies, then verify the resolved parent, socket, state, observer, and generation paths and their no-symlink/private posture. 2. The approved supervisor would start the daemon only after confirming that the exact socket path is not owned by another process. `READY` from the daemon and a live Unix socket would be health evidence; a unit file alone would not be healthy. 3. A broker crash would require preserving the state file, identifying the socket owner, and making an explicit restart decision. Restart intentionally clears volatile `VERIFIED` leases; it is not a way to restore authority. 4. A leftover socket would be handled only after the owning service is confirmed stopped and the exact path is deliberately reviewed. This page supplies no deletion, `systemctl`, enablement, or start command. The source's `daemon.py` argument parser and the user unit show how a future activation is wired, but neither source file grants this page authority to invoke that wiring. `applyBrokerSupervisor` is materialization only; the supervisor source explicitly leaves enable/start as a separate held step. ### Recovery outline (held) The future adapter flow is: runtime supplies validated construction and current non-negative epochs; broker performs `begin_recovery` revoke-first and returns one fresh receipt; the adapter delivers that exact receipt; the authenticated observer records the finalized assistant entry; and the adapter requests completion without presenting receipt text or a challenge. Any absent, malformed, stale, duplicated, or mismatched observation leaves the session `UNVERIFIED`; retry starts a new recovery cycle. Claude's adapter is restricted to the exact literal recovery argv shape checked by the gate. Pi uses only the registered `mosaic_context_recover` tool; Pi `bash` and all other tools remain gated. Do not manually invoke the revoker to restore authority, send assistant text through the public broker request, or reuse a normal-path receipt. The [context-refresh skill](../../../packages/mosaic/framework/skills/mosaic-context-refresh/SKILL.md) and its [runtime boundary tests](../../../packages/mosaic/src/lease-broker/recovery_runtime_unittest.py) are references for future adapter qualification, not an active operator route. ### Lease, mutator, and incident handling outline (held) - There is no supported operator CLI or HTTP endpoint for sending raw `begin_verification`, `promote_lease`, `revoke_lease`, or `authorize_tool` requests. Do not hand-craft JSON frames, mint tokens, replay receipts, or treat a successful read-only authorization as a promotion. - After a runtime exits, its generation file may be removed only after an approved check establishes that no process for that broker-minted session remains. Retain stale files during incident analysis; they carry no lease authority by themselves. - Corrupt, oversized, symlinked, or non-regular state must be preserved for review and not overwritten in place. Establishing new state is an explicit operational decision that invalidates prior sessions and tokens; no recovery command is supplied here. - Directory `0700` and socket/state `0600` are same-principal hardening only. They do not prevent the same UID from unlinking and replacing a socket. A stronger deployment needs an external protected proxy, ACL, or service boundary that preserves the peer identity required by `SO_PEERCRED` and ancestry checks. No such deployment procedure is current here. ## Related source and tests - [Broker daemon](../../../packages/mosaic/framework/tools/lease-broker/daemon.py) - [Register-and-exec launcher](../../../packages/mosaic/framework/tools/lease-broker/launch-runtime.py) - [Mutator gate](../../../packages/mosaic/framework/tools/lease-broker/mutator-gate.py) - [Recovery command](../../../packages/mosaic/framework/tools/lease-broker/recover-context.py) - [Lease broker acceptance tests](../../../packages/mosaic/src/lease-broker/lease-broker.acceptance.spec.ts) - [Mutator gate acceptance tests](../../../packages/mosaic/src/mutator-gate/mutator-gate.acceptance.spec.ts) - [Pi lifecycle tests](../../../packages/mosaic/src/mutator-gate/pi-compaction-lifecycle.spec.ts) This migration changes documentation placement and verified wording only. It does not start or change the broker, a runtime, systemd, PostgreSQL, or any other service.