feat(#755): add logical agent connector fencing
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful

Add normalized runtime-neutral identity, durable PostgreSQL CAS leases, monotonic epochs, short-lived server grants, fail-closed adapter validation, credential-safe audit, and concurrency/restart/abuse coverage.\n\nRefs #755
This commit is contained in:
Jarvis
2026-07-14 13:04:52 -05:00
parent 9d10bdcf87
commit dff8ce4f79
20 changed files with 6847 additions and 0 deletions

View File

@@ -0,0 +1,43 @@
# Mos Connector Lease Operations — M1
## Operational status
M1 installs the durable schema and gateway policy/adapter boundary. It does **not** activate a connector, expose a lease administration endpoint, or cut over a channel. The default gateway connector-lease policy is deny-all until a later work package supplies an authorized server-side policy and concrete adapter.
## Events to monitor
Use correlation IDs to follow `connector_lease_audit_log` events:
| Event | Meaning |
| ---------- | --------------------------------------------------------------------- |
| `acquire` | First holder inserted for an unused binding |
| `renew` | Current holder heartbeat extended the TTL |
| `takeover` | Authorized CAS replaced the holder and incremented epoch |
| `release` | Current holder explicitly relinquished authority |
| `expiry` | An expired current lease was observed |
| `reject` | Policy, CAS, expiry, scope, or fencing validation denied an operation |
Audit data is metadata-only. Raw grant objects, connector payloads, scopes, tokens, approval references, and credentials must never be added to audit output.
## Incident checks
For suspected duplicate/stale connector effects:
1. Correlate the attempted operation with its `reject`, `takeover`, or `expiry` event.
2. Compare the current row's connector ID, lease UUID, epoch, expiry, and release time with the adapter's normalized execution context.
3. Treat an old epoch, old lease UUID, expired lease, or released lease as non-authoritative. Do not retry it as the old holder.
4. Recovery uses the authorized takeover path with the observed expected epoch. Ordinary acquire is intentionally rejected for expired/released rows.
5. If an external effect may already have happened, preserve evidence and do not assume lease fencing provides exactly-once replay safety.
## Migration and rollback safety
Migration `0016_salty_morlocks.sql` is additive: it creates two new tables and indexes without modifying existing authorization/session tables. Before rollout, normal database backup and migration verification still apply. Rolling application code back leaves unused additive tables in place; dropping tables is not part of automated rollback because it would destroy lease/audit evidence.
## Security constraints
- Tenant comes from authenticated gateway context, never a connector request field.
- Logical agent, binding, connector, and scope identifiers use normalized constrained forms.
- Takeover requires explicit gateway policy authorization and an expected epoch.
- Default defense-in-depth TTL caps are 5 minutes for leases and 30 seconds for grants; policy may enforce stricter limits.
- Validation and rejection audit complete before adapter side effects.
- Existing authz and exact-action approval controls remain additional required gates; a valid connector lease does not bypass them.