Compare commits
59 Commits
feat/758-r
...
feat/827-g
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
93a5fd018d | ||
|
|
61126c7473 | ||
|
|
ffc3b573a8 | ||
|
|
ce5ba76205 | ||
|
|
1a09c925d3 | ||
|
|
3d6f556e2b | ||
|
|
f320d075df | ||
|
|
e1c9a4682d | ||
|
|
1c34e3cb31 | ||
|
|
e6c3b830fd | ||
|
|
a92ad090ae | ||
|
|
90dc68a31c | ||
|
|
2ae379e664 | ||
|
|
e08ad03506 | ||
|
|
2bba933f67 | ||
|
|
f609a44953 | ||
|
|
b6bd0cd91c | ||
|
|
12914d8edd | ||
|
|
23c0caca9b | ||
|
|
2d54a9dd14 | ||
|
|
6164dc0794 | ||
|
|
7ff63cd5c1 | ||
|
|
49f0cdd15a | ||
|
|
7f975b95ad | ||
|
|
cff21358a2 | ||
|
|
4848493870 | ||
|
|
d19b41a62c | ||
|
|
ace6066762 | ||
|
|
839d156f6c | ||
|
|
aa88a5cb9d | ||
|
|
a532df5943 | ||
|
|
b9780eb058 | ||
|
|
d5c599e2b0 | ||
| d801d6c4c8 | |||
| d3bf52898b | |||
| 3f77229e88 | |||
| 686c881fe4 | |||
| fe7a468c9d | |||
| cabf02e7b9 | |||
| 9ddc6fbda8 | |||
| 31607a4af6 | |||
| 32a0ffba13 | |||
| 8536454257 | |||
| 4f29cc604d | |||
| 3be443c96d | |||
| 59f5f51ffd | |||
| 9745bc3f29 | |||
| adad486b6f | |||
| c1aecfabe9 | |||
| 499090508e | |||
| c593a15ef8 | |||
| bc5e73629e | |||
| 191efaefeb | |||
| e9c4aa3e8b | |||
| a5e8e55401 | |||
| eb4e14ae5c | |||
| 2e2280070a | |||
| aa5b43bba2 | |||
| ba13c08890 |
@@ -42,6 +42,27 @@ steps:
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh --self-test
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh
|
||||
|
||||
# Blocking gate (#791): a framework upgrade must never write or delete an
|
||||
# operator-owned path. The HARD GATE proves an unanticipated operator sentinel
|
||||
# survives a keep-mode reseed byte-identical (with rsync present AND absent —
|
||||
# keep mode is a single cp-based path that must not depend on rsync), and that a
|
||||
# corrupt/empty/missing manifest aborts fail-closed leaving operator files
|
||||
# untouched (B2/B3). The rollback gate proves a mid-sync failure is rolled back
|
||||
# from the pre-update snapshot (B1). The durable-snapshot gate (#791 PR2) proves
|
||||
# the retained, operator-scoped pre-update backup is taken before any mutation
|
||||
# (0700/0600, secret never logged, retention-pruned) and that the post-sync
|
||||
# verify net restores any operator file a manifest bug lets the sync touch. The
|
||||
# migration matrix pins the v2→v3 contract-file semantics. Pure bash, no
|
||||
# node_modules — runs early alongside sanitization.
|
||||
upgrade-guard:
|
||||
image: *node_image
|
||||
commands:
|
||||
- apk add --no-cache bash rsync
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-manifest-guard.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-rollback.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-upgrade-durable-snapshot.sh
|
||||
- bash packages/mosaic/framework/tools/quality/scripts/test-install-migration.sh
|
||||
|
||||
typecheck:
|
||||
image: *node_image
|
||||
commands:
|
||||
@@ -50,6 +71,7 @@ steps:
|
||||
depends_on:
|
||||
- install
|
||||
- sanitization
|
||||
- upgrade-guard
|
||||
|
||||
# lint, format, and test are independent — run in parallel after typecheck
|
||||
lint:
|
||||
|
||||
13
CLAUDE.md
13
CLAUDE.md
@@ -26,13 +26,14 @@ pnpm test # Vitest (all packages)
|
||||
pnpm build # Build all packages
|
||||
|
||||
# Database
|
||||
pnpm --filter @mosaicstack/db db:push # Push schema to PG (dev)
|
||||
pnpm --filter @mosaicstack/db db:generate # Generate migrations
|
||||
pnpm --filter @mosaicstack/db db:migrate # Run migrations
|
||||
pnpm --filter @mosaicstack/db db:generate # Offline migration artifact generation only
|
||||
# PostgreSQL execution is held until KBN-101-00/-03/-05 land. Do not invoke a runner,
|
||||
# init SQL, or Compose PostgreSQL service from this checkout.
|
||||
|
||||
# Dev
|
||||
docker compose up -d # Start PG, Valkey, OTEL, Jaeger
|
||||
pnpm --filter @mosaicstack/gateway exec tsx src/main.ts # Start gateway
|
||||
# Dev: local PGlite data-layer work needs no PostgreSQL. Optional local queue service only:
|
||||
docker compose up -d valkey
|
||||
# Do not start Gateway/Web or root pnpm dev as a local PGlite route: the current unguarded dotenv
|
||||
# loader can inherit a daemon PostgreSQL DSN. KBN-101-02 must make that state fail closed first.
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
@@ -22,10 +22,10 @@
|
||||
FROM node:24-alpine
|
||||
|
||||
# Native toolchain required to compile node-gyp deps on musl, plus the
|
||||
# postgresql-client used by the test step's pg_isready readiness probe. `bash`
|
||||
# is baked here too — the sanitization step in ci.yml otherwise does a per-run
|
||||
# `apk add bash`.
|
||||
RUN apk add --no-cache python3 make g++ postgresql-client bash
|
||||
# postgresql-client used by the test step's pg_isready readiness probe. `bash`,
|
||||
# `git`, and `jq` are baked here too — framework shell tests and the shipped
|
||||
# Codex review wrappers require them without per-run installation in ci.yml.
|
||||
RUN apk add --no-cache python3 make g++ postgresql-client bash git jq
|
||||
|
||||
# Pin pnpm to the repo's packageManager version via corepack.
|
||||
RUN corepack enable && corepack prepare pnpm@10.6.2 --activate
|
||||
|
||||
51
README.md
51
README.md
@@ -97,7 +97,10 @@ mosaic config path # Print config file path
|
||||
```bash
|
||||
mosaic doctor # Health audit — detect drift and missing files
|
||||
mosaic sync # Sync skills from canonical source
|
||||
mosaic update # Check for and install CLI updates
|
||||
mosaic skill list # Audit Claude skill registrations and conflicts
|
||||
mosaic skill register <name> # Register one canonical skill with Claude Code
|
||||
mosaic skill unregister <name> # Remove one Mosaic-owned Claude link
|
||||
mosaic update # Update CLI/framework and auto-register canonical skills
|
||||
mosaic wizard # Full guided setup wizard
|
||||
mosaic bootstrap <path> # Bootstrap a repo with Mosaic standards
|
||||
mosaic coord init # Initialize a new orchestration mission
|
||||
@@ -157,7 +160,12 @@ mosaic storage status
|
||||
mosaic storage tier
|
||||
mosaic storage export
|
||||
mosaic storage import
|
||||
mosaic storage migrate
|
||||
# Schema migration is unavailable in this release. The current storage wrapper shells
|
||||
# directly to `pnpm --filter @mosaicstack/db db:migrate`; it is legacy N-1,
|
||||
# uncertified, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||
# Future schema migration is non-operative: external bootstrap → TLS/roles → runner
|
||||
# --run → runner --verify → readiness. Tier copy uses only the separately held secure
|
||||
# migrate-tier route.
|
||||
```
|
||||
|
||||
### Telemetry
|
||||
@@ -192,29 +200,32 @@ Consent state is persisted in config. Remote upload is a no-op until you run `mo
|
||||
git clone git@git.mosaicstack.dev:mosaicstack/stack.git
|
||||
cd stack
|
||||
|
||||
# Start infrastructure (Postgres, Valkey, Jaeger)
|
||||
docker compose up -d
|
||||
|
||||
# Install dependencies
|
||||
# Install dependencies. The local tier uses in-process PGlite; leave DATABASE_URL unset.
|
||||
pnpm install
|
||||
|
||||
# Run migrations
|
||||
pnpm --filter @mosaicstack/db run db:migrate
|
||||
# Optional local queue service only. This does not start PostgreSQL.
|
||||
docker compose up -d valkey
|
||||
|
||||
# Start all services in dev mode
|
||||
pnpm dev
|
||||
# The current Gateway/Web local process is held; see docs/guides/dev-guide.md.
|
||||
# Do not start it until KBN-101-02 makes inherited dotenv/DSN state fail closed.
|
||||
```
|
||||
|
||||
### Infrastructure
|
||||
### Held future procedure
|
||||
|
||||
Docker Compose provides:
|
||||
The checked-in Compose PostgreSQL service mounts legacy initialization SQL and is **not** a
|
||||
current PostgreSQL, standalone, or federated developer route. Do not start it with Compose,
|
||||
invoke initialization SQL, or treat the planned migrator as currently executable.
|
||||
|
||||
| Service | Port | Purpose |
|
||||
| --------------------- | --------- | ---------------------- |
|
||||
| PostgreSQL (pgvector) | 5433 | Primary database |
|
||||
| Valkey | 6380 | Task queue + caching |
|
||||
| Jaeger | 16686 | Distributed tracing UI |
|
||||
| OTEL Collector | 4317/4318 | Telemetry ingestion |
|
||||
**Held future activation procedure — non-operative and no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05
|
||||
land:** external bootstrap → TLS/roles → `mosaic-db-migrator --run` →
|
||||
`mosaic-db-migrator --verify` → Gateway/Compose readiness. The future deployment artifacts—not
|
||||
this README—will provide the reviewed commands and secret-consumer interface.
|
||||
|
||||
For local data-layer work, PGlite needs no PostgreSQL service. The optional Compose command above
|
||||
starts only Valkey; OTEL Collector and Jaeger may likewise be started individually if needed,
|
||||
without starting PostgreSQL. A Gateway/Web local process is not currently a safe PGlite route:
|
||||
its unguarded dotenv loader may inherit a daemon PostgreSQL DSN. Do not use root `pnpm dev` or a
|
||||
Gateway start command until KBN-101-02 makes that state fail closed.
|
||||
|
||||
### Quality Gates
|
||||
|
||||
@@ -231,7 +242,7 @@ pnpm format # Prettier auto-fix
|
||||
Woodpecker CI runs on every push:
|
||||
|
||||
- `pnpm install --frozen-lockfile`
|
||||
- Database migration against a fresh Postgres
|
||||
- **Legacy N-1 CI status only — active, uncertified, and non-authorizing as an operator route:** the checked-in job currently invokes `pnpm --filter @mosaicstack/db run db:migrate` with `DATABASE_URL` against an isolated disposable PostgreSQL CI database. It performs direct DDL in that CI database, is not approved ordinary behavior or an operator route, and remains a known exception pending KBN-101-06 removal/replacement by the certified runner-backed CI path.
|
||||
- `pnpm test` (Turbo-orchestrated across all packages)
|
||||
|
||||
npm packages are published to the Gitea package registry on main merges.
|
||||
@@ -341,6 +352,8 @@ bash tools/install.sh --yes # Non-interactive, accept all defaults
|
||||
bash tools/install.sh --no-auto-launch # Skip auto-launch of wizard
|
||||
```
|
||||
|
||||
The installer rejects unrecognized flags or positional arguments before making changes and prints the supported-option usage.
|
||||
|
||||
## Contributing
|
||||
|
||||
```bash
|
||||
|
||||
@@ -72,6 +72,7 @@ describe('interaction Discord/CLI durable-session integration', () => {
|
||||
process.env['DISCORD_INTERACTION_BINDINGS'] = JSON.stringify([
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-1',
|
||||
channelId: 'channel-1',
|
||||
pairedUsers: {
|
||||
@@ -133,6 +134,7 @@ describe('interaction Discord/CLI durable-session integration', () => {
|
||||
interactionBindings: [
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-1',
|
||||
channelId: 'channel-1',
|
||||
pairedUsers: {
|
||||
|
||||
@@ -115,6 +115,18 @@ describe('AgentService owner/tenant scope enforcement', () => {
|
||||
).rejects.toBeInstanceOf(ForbiddenException);
|
||||
await service.prompt(CONVERSATION_ID, 'owner prompt', OWNER_SCOPE);
|
||||
expect(session.piSession.prompt).toHaveBeenCalledWith('owner prompt');
|
||||
await service.prompt(CONVERSATION_ID, '', OWNER_SCOPE, [
|
||||
{
|
||||
id: 'attachment-001',
|
||||
name: 'diagram.png',
|
||||
url: 'https://cdn.example.test/diagram.png',
|
||||
mimeType: 'image/png',
|
||||
},
|
||||
]);
|
||||
expect(session.piSession.prompt).toHaveBeenLastCalledWith(
|
||||
'\n\n[Untrusted channel attachments]\n' +
|
||||
'{"id":"attachment-001","name":"diagram.png","mimeType":"image/png","url":"https://cdn.example.test/diagram.png"}',
|
||||
);
|
||||
|
||||
await expect(service.destroySession(CONVERSATION_ID, FOREIGN_SCOPE)).rejects.toBeInstanceOf(
|
||||
ForbiddenException,
|
||||
|
||||
@@ -21,6 +21,12 @@ import { LogModule } from '../log/log.module.js';
|
||||
import { CommandsModule } from '../commands/commands.module.js';
|
||||
import { CommandRuntimeApprovalVerifier } from '../commands/runtime-approval-verifier.js';
|
||||
import { GatewayHermesRuntimeTransport } from './hermes-runtime.transport.js';
|
||||
import { ConnectorLeaseRepository } from './connector-lease.repository.js';
|
||||
import {
|
||||
CONNECTOR_LEASE_POLICY,
|
||||
ConnectorLeaseService,
|
||||
DenyConnectorLeasePolicy,
|
||||
} from './connector-lease.service.js';
|
||||
import {
|
||||
AGENT_RUNTIME_PROVIDER_REGISTRY,
|
||||
RUNTIME_APPROVAL_VERIFIER,
|
||||
@@ -46,6 +52,13 @@ export function createGatewayRuntimeProviderRegistry(): AgentRuntimeProviderRegi
|
||||
SkillLoaderService,
|
||||
DurableSessionRepository,
|
||||
DurableSessionService,
|
||||
ConnectorLeaseRepository,
|
||||
DenyConnectorLeasePolicy,
|
||||
{
|
||||
provide: CONNECTOR_LEASE_POLICY,
|
||||
useExisting: DenyConnectorLeasePolicy,
|
||||
},
|
||||
ConnectorLeaseService,
|
||||
{
|
||||
provide: AGENT_RUNTIME_PROVIDER_REGISTRY,
|
||||
useFactory: createGatewayRuntimeProviderRegistry,
|
||||
@@ -78,6 +91,7 @@ export function createGatewayRuntimeProviderRegistry(): AgentRuntimeProviderRegi
|
||||
SkillLoaderService,
|
||||
DurableSessionService,
|
||||
RuntimeProviderService,
|
||||
ConnectorLeaseService,
|
||||
AGENT_RUNTIME_PROVIDER_REGISTRY,
|
||||
],
|
||||
})
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
type ToolDefinition,
|
||||
} from '@mariozechner/pi-coding-agent';
|
||||
import type { Brain } from '@mosaicstack/brain';
|
||||
import type { ChannelAttachmentDto } from '@mosaicstack/types';
|
||||
import type { Memory, OperatorMemoryPlugin } from '@mosaicstack/memory';
|
||||
import { BRAIN } from '../brain/brain.tokens.js';
|
||||
import { MEMORY } from '../memory/memory.tokens.js';
|
||||
@@ -43,6 +44,8 @@ export interface ConversationHistoryMessage {
|
||||
role: 'user' | 'assistant' | 'system';
|
||||
content: string;
|
||||
createdAt: Date;
|
||||
/** Validated, URI-referenced channel attachments preserved on session resume. */
|
||||
attachments?: readonly ChannelAttachmentDto[];
|
||||
}
|
||||
|
||||
export interface AgentSessionOptions {
|
||||
@@ -428,7 +431,7 @@ export class AgentService implements OnModuleDestroy {
|
||||
const formatMessage = (msg: ConversationHistoryMessage): string => {
|
||||
const roleLabel =
|
||||
msg.role === 'user' ? 'User' : msg.role === 'assistant' ? 'Assistant' : 'System';
|
||||
return `**${roleLabel}:** ${msg.content}`;
|
||||
return `**${roleLabel}:** ${msg.content}${this.attachmentContext(msg.attachments ?? [])}`;
|
||||
};
|
||||
|
||||
const formatted = history.map((msg) => formatMessage(msg));
|
||||
@@ -487,6 +490,21 @@ export class AgentService implements OnModuleDestroy {
|
||||
return result;
|
||||
}
|
||||
|
||||
private attachmentContext(attachments: readonly ChannelAttachmentDto[]): string {
|
||||
if (attachments.length === 0) return '';
|
||||
return `\n\n[Untrusted channel attachments]\n${attachments
|
||||
.map((attachment: ChannelAttachmentDto): string =>
|
||||
JSON.stringify({
|
||||
id: attachment.id,
|
||||
name: attachment.name,
|
||||
mimeType: attachment.mimeType,
|
||||
url: attachment.url,
|
||||
...(attachment.sizeBytes !== undefined ? { sizeBytes: attachment.sizeBytes } : {}),
|
||||
}),
|
||||
)
|
||||
.join('\n')}`;
|
||||
}
|
||||
|
||||
private resolveModel(options?: AgentSessionOptions) {
|
||||
if (!options?.provider && !options?.modelId) {
|
||||
return this.providerService.getDefaultModel() ?? null;
|
||||
@@ -673,7 +691,19 @@ export class AgentService implements OnModuleDestroy {
|
||||
session.channels.delete(channel);
|
||||
}
|
||||
|
||||
async prompt(sessionId: string, message: string, scope: ActorTenantScope): Promise<void> {
|
||||
async prompt(sessionId: string, message: string, scope: ActorTenantScope): Promise<void>;
|
||||
async prompt(
|
||||
sessionId: string,
|
||||
message: string,
|
||||
scope: ActorTenantScope,
|
||||
attachments: readonly ChannelAttachmentDto[] | undefined,
|
||||
): Promise<void>;
|
||||
async prompt(
|
||||
sessionId: string,
|
||||
message: string,
|
||||
scope: ActorTenantScope,
|
||||
attachments: readonly ChannelAttachmentDto[] = [],
|
||||
): Promise<void> {
|
||||
const session = this.sessions.get(sessionId);
|
||||
if (!session) {
|
||||
throw new Error(`No agent session found: ${sessionId}`);
|
||||
@@ -681,12 +711,16 @@ export class AgentService implements OnModuleDestroy {
|
||||
this.assertSessionScope(session, scope);
|
||||
session.promptCount += 1;
|
||||
|
||||
// Channel attachments are untrusted URI references. Preserve exact,
|
||||
// authenticated metadata for the agent without treating it as authority.
|
||||
const attachmentContext = this.attachmentContext(attachments);
|
||||
|
||||
// Prepend session-scoped system override if present (renew TTL on each turn)
|
||||
let effectiveMessage = message;
|
||||
let effectiveMessage = `${message}${attachmentContext}`;
|
||||
if (this.systemOverride) {
|
||||
const override = await this.systemOverride.get(sessionId, scope);
|
||||
if (override) {
|
||||
effectiveMessage = `[System Override]\n${override}\n\n${message}`;
|
||||
effectiveMessage = `[System Override]\n${override}\n\n${effectiveMessage}`;
|
||||
await this.systemOverride.renew(sessionId, scope);
|
||||
this.logger.debug(`Applied system override for session ${sessionId}`);
|
||||
}
|
||||
|
||||
341
apps/gateway/src/agent/connector-lease.integration.test.ts
Normal file
341
apps/gateway/src/agent/connector-lease.integration.test.ts
Normal file
@@ -0,0 +1,341 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
import {
|
||||
connectorLeaseAuditLog,
|
||||
createPgliteDb,
|
||||
eq,
|
||||
runPgliteMigrations,
|
||||
type DbHandle,
|
||||
} from '@mosaicstack/db';
|
||||
import type { ConnectorExecutionContext, FencedConnectorAdapter } from '@mosaicstack/types';
|
||||
import { DB } from '../database/database.module.js';
|
||||
import { ConnectorLeaseRepository } from './connector-lease.repository.js';
|
||||
import {
|
||||
CONNECTOR_LEASE_POLICY,
|
||||
ConnectorLeaseService,
|
||||
type ConnectorLeasePolicy,
|
||||
type ConnectorLeasePolicySubject,
|
||||
} from './connector-lease.service.js';
|
||||
|
||||
const authorize = vi.fn().mockResolvedValue(true);
|
||||
const policy: ConnectorLeasePolicy = { authorize };
|
||||
const context = {
|
||||
actorScope: { userId: 'operator-a', tenantId: 'tenant-a' },
|
||||
correlationId: 'correlation-acquire',
|
||||
};
|
||||
|
||||
describe('gateway connector lease fencing integration', (): void => {
|
||||
let dataDir: string;
|
||||
let handle: DbHandle;
|
||||
let moduleRef: TestingModule;
|
||||
let service: ConnectorLeaseService;
|
||||
let repository: ConnectorLeaseRepository;
|
||||
|
||||
beforeAll(async (): Promise<void> => {
|
||||
vi.useFakeTimers();
|
||||
vi.setSystemTime(new Date('2026-07-14T17:00:00.000Z'));
|
||||
dataDir = await mkdtemp(join(tmpdir(), 'mosaic-gateway-connector-lease-'));
|
||||
handle = createPgliteDb(dataDir);
|
||||
await runPgliteMigrations(handle);
|
||||
moduleRef = await Test.createTestingModule({
|
||||
providers: [
|
||||
ConnectorLeaseRepository,
|
||||
ConnectorLeaseService,
|
||||
{ provide: DB, useValue: handle.db },
|
||||
{ provide: CONNECTOR_LEASE_POLICY, useValue: policy },
|
||||
],
|
||||
}).compile();
|
||||
service = moduleRef.get(ConnectorLeaseService);
|
||||
repository = moduleRef.get(ConnectorLeaseRepository);
|
||||
});
|
||||
|
||||
afterAll(async (): Promise<void> => {
|
||||
vi.useRealTimers();
|
||||
await moduleRef.close();
|
||||
await handle.close();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('derives tenant authority at the gateway and validates a grant before side effects', async (): Promise<void> => {
|
||||
const lease = await service.acquire(
|
||||
{
|
||||
logicalAgentId: 'Mos',
|
||||
bindingId: 'operator-chat',
|
||||
connectorId: 'pi-worker-a',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
},
|
||||
context,
|
||||
);
|
||||
const grant = await service.issueGrant(
|
||||
{ lease, scopes: ['runtime.send'], ttlMs: 30_000 },
|
||||
{ ...context, correlationId: 'correlation-grant' },
|
||||
);
|
||||
const execute = vi.fn(async (_message: string, leaseContext: ConnectorExecutionContext) => {
|
||||
return leaseContext.leaseEpoch;
|
||||
});
|
||||
const adapter: FencedConnectorAdapter<string, string> = { execute };
|
||||
|
||||
await expect(service.executeGrant(grant, 'runtime.send', 'hello', adapter)).resolves.toBe('1');
|
||||
expect(execute).toHaveBeenCalledOnce();
|
||||
expect(authorize).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'grant.issue',
|
||||
requestedScopes: ['runtime.send'],
|
||||
requestedTtlMs: 30_000,
|
||||
}),
|
||||
);
|
||||
expect(execute.mock.calls[0]?.[1]).toMatchObject({
|
||||
identity: { tenantId: 'tenant-a', logicalAgentId: 'mos' },
|
||||
bindingId: 'operator-chat',
|
||||
connectorId: 'pi-worker-a',
|
||||
});
|
||||
});
|
||||
|
||||
it('normalizes lease-derived policy subjects before authorization', async (): Promise<void> => {
|
||||
const lease = await service.acquire(
|
||||
{
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-policy',
|
||||
connectorId: 'pi-worker-a',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
},
|
||||
{ ...context, correlationId: 'correlation-policy-setup' },
|
||||
);
|
||||
const aliasedLease = {
|
||||
...lease,
|
||||
identity: { ...lease.identity, logicalAgentId: ' MOS ' },
|
||||
bindingId: ' Operator-Chat-Policy ',
|
||||
connectorId: ' PI-Worker-A ',
|
||||
scopes: [' Runtime.Send '],
|
||||
leaseEpoch: `00${lease.leaseEpoch}`,
|
||||
};
|
||||
|
||||
await service.heartbeat(aliasedLease, 30_000, {
|
||||
...context,
|
||||
correlationId: 'correlation-policy-heartbeat',
|
||||
});
|
||||
expect(authorize).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'lease.heartbeat',
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-policy',
|
||||
connectorId: 'pi-worker-a',
|
||||
requestedScopes: ['runtime.send'],
|
||||
}),
|
||||
);
|
||||
|
||||
await service.issueGrant(
|
||||
{ lease: aliasedLease, scopes: [' Runtime.Send '], ttlMs: 1_000 },
|
||||
{ ...context, correlationId: 'correlation-policy-grant' },
|
||||
);
|
||||
expect(authorize).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'grant.issue',
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-policy',
|
||||
connectorId: 'pi-worker-a',
|
||||
requestedScopes: ['runtime.send'],
|
||||
}),
|
||||
);
|
||||
|
||||
await service.release(aliasedLease, {
|
||||
...context,
|
||||
correlationId: 'correlation-policy-release',
|
||||
});
|
||||
expect(authorize).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'lease.release',
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-policy',
|
||||
connectorId: 'pi-worker-a',
|
||||
requestedScopes: ['runtime.send'],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('denies stale, forged, expired, cross-tenant, and cross-binding grants before effects', async (): Promise<void> => {
|
||||
const bindingId = 'operator-chat-denials';
|
||||
const current = await service.acquire(
|
||||
{
|
||||
logicalAgentId: 'mos',
|
||||
bindingId,
|
||||
connectorId: 'pi-worker-a',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
},
|
||||
{ ...context, correlationId: 'correlation-denial-setup' },
|
||||
);
|
||||
const stale = await service.issueGrant(
|
||||
{ lease: current, scopes: ['runtime.send'], ttlMs: 30_000 },
|
||||
{ ...context, correlationId: 'correlation-stale' },
|
||||
);
|
||||
await service.takeover(
|
||||
{
|
||||
logicalAgentId: 'mos',
|
||||
bindingId,
|
||||
connectorId: 'pi-worker-b',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
expectedEpoch: current.leaseEpoch,
|
||||
},
|
||||
{ ...context, correlationId: 'correlation-takeover' },
|
||||
);
|
||||
const adapter = { execute: vi.fn().mockResolvedValue(undefined) };
|
||||
|
||||
await expect(service.executeGrant(stale, 'runtime.send', undefined, adapter)).rejects.toThrow();
|
||||
|
||||
const active = await service.current('mos', bindingId, context);
|
||||
if (!active) throw new Error('active lease fixture is unavailable');
|
||||
const grant = await service.issueGrant(
|
||||
{ lease: active, scopes: ['runtime.send'], ttlMs: 1_000 },
|
||||
{ ...context, correlationId: 'correlation-active' },
|
||||
);
|
||||
await expect(
|
||||
service.executeGrant({ ...grant }, 'runtime.send', undefined, adapter),
|
||||
).rejects.toThrow();
|
||||
await expect(
|
||||
service.executeGrant(
|
||||
{ ...grant, bindingId: 'other-binding' },
|
||||
'runtime.send',
|
||||
undefined,
|
||||
adapter,
|
||||
),
|
||||
).rejects.toThrow();
|
||||
await expect(
|
||||
service.issueGrant(
|
||||
{ lease: active, scopes: ['runtime.send'], ttlMs: 30_000 },
|
||||
{
|
||||
actorScope: { userId: 'operator-b', tenantId: 'tenant-b' },
|
||||
correlationId: 'correlation-cross-tenant',
|
||||
},
|
||||
),
|
||||
).rejects.toThrow();
|
||||
const crossTenantAudit = await handle.db
|
||||
.select()
|
||||
.from(connectorLeaseAuditLog)
|
||||
.where(eq(connectorLeaseAuditLog.correlationId, 'correlation-cross-tenant'));
|
||||
expect(crossTenantAudit).toHaveLength(1);
|
||||
expect(crossTenantAudit[0]).toMatchObject({
|
||||
tenantId: 'tenant-b',
|
||||
logicalAgentId: 'untrusted',
|
||||
bindingId: 'untrusted',
|
||||
connectorId: 'untrusted',
|
||||
reason: 'policy_denied',
|
||||
});
|
||||
|
||||
vi.setSystemTime(new Date('2026-07-14T17:00:02.000Z'));
|
||||
await expect(service.executeGrant(grant, 'runtime.send', undefined, adapter)).rejects.toThrow();
|
||||
expect(adapter.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects submitted lifecycle scopes that differ from durable authority before policy or mutation', async (): Promise<void> => {
|
||||
authorize.mockResolvedValue(true);
|
||||
const heartbeatLease = await service.acquire(
|
||||
{
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-heartbeat-scope',
|
||||
connectorId: 'pi-worker-a',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
},
|
||||
{ ...context, correlationId: 'correlation-heartbeat-scope-setup' },
|
||||
);
|
||||
const releaseLease = await service.acquire(
|
||||
{
|
||||
logicalAgentId: 'mos',
|
||||
bindingId: 'operator-chat-release-scope',
|
||||
connectorId: 'pi-worker-a',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
},
|
||||
{ ...context, correlationId: 'correlation-release-scope-setup' },
|
||||
);
|
||||
const forgedHeartbeat = { ...heartbeatLease, scopes: ['tool.execute'] };
|
||||
const forgedRelease = { ...releaseLease, scopes: ['tool.execute'] };
|
||||
|
||||
authorize.mockImplementation(async (subject: ConnectorLeasePolicySubject) => {
|
||||
return subject.requestedScopes.length === 1 && subject.requestedScopes[0] === 'tool.execute';
|
||||
});
|
||||
authorize.mockClear();
|
||||
|
||||
await expect(
|
||||
service.heartbeat(forgedHeartbeat, 30_000, {
|
||||
...context,
|
||||
correlationId: 'correlation-heartbeat-scope-forgery',
|
||||
}),
|
||||
).rejects.toThrow('Connector authority policy denied');
|
||||
await expect(
|
||||
service.release(forgedRelease, {
|
||||
...context,
|
||||
correlationId: 'correlation-release-scope-forgery',
|
||||
}),
|
||||
).rejects.toThrow('Connector authority policy denied');
|
||||
expect(authorize).not.toHaveBeenCalled();
|
||||
|
||||
const currentHeartbeat = await repository.findCurrent({
|
||||
identity: heartbeatLease.identity,
|
||||
bindingId: heartbeatLease.bindingId,
|
||||
});
|
||||
const currentRelease = await repository.findCurrent({
|
||||
identity: releaseLease.identity,
|
||||
bindingId: releaseLease.bindingId,
|
||||
});
|
||||
expect(currentHeartbeat).toMatchObject({
|
||||
leaseId: heartbeatLease.leaseId,
|
||||
scopes: ['runtime.send'],
|
||||
heartbeatAt: heartbeatLease.heartbeatAt,
|
||||
expiresAt: heartbeatLease.expiresAt,
|
||||
});
|
||||
expect(currentRelease).toMatchObject({
|
||||
leaseId: releaseLease.leaseId,
|
||||
scopes: ['runtime.send'],
|
||||
});
|
||||
expect(currentRelease?.releasedAt).toBeUndefined();
|
||||
|
||||
const forgedAudits = await handle.db
|
||||
.select()
|
||||
.from(connectorLeaseAuditLog)
|
||||
.where(eq(connectorLeaseAuditLog.correlationId, 'correlation-heartbeat-scope-forgery'));
|
||||
expect(forgedAudits).toHaveLength(1);
|
||||
expect(forgedAudits[0]).toMatchObject({
|
||||
bindingId: heartbeatLease.bindingId,
|
||||
connectorId: heartbeatLease.connectorId,
|
||||
event: 'reject',
|
||||
outcome: 'denied',
|
||||
reason: 'policy_denied',
|
||||
});
|
||||
const forgedReleaseAudits = await handle.db
|
||||
.select()
|
||||
.from(connectorLeaseAuditLog)
|
||||
.where(eq(connectorLeaseAuditLog.correlationId, 'correlation-release-scope-forgery'));
|
||||
expect(forgedReleaseAudits).toHaveLength(1);
|
||||
expect(forgedReleaseAudits[0]).toMatchObject({
|
||||
bindingId: releaseLease.bindingId,
|
||||
connectorId: releaseLease.connectorId,
|
||||
event: 'reject',
|
||||
outcome: 'denied',
|
||||
reason: 'policy_denied',
|
||||
});
|
||||
|
||||
authorize.mockImplementation(async (subject: ConnectorLeasePolicySubject) => {
|
||||
return subject.requestedScopes.length === 1 && subject.requestedScopes[0] === 'runtime.send';
|
||||
});
|
||||
await expect(
|
||||
service.heartbeat(heartbeatLease, 30_000, {
|
||||
...context,
|
||||
correlationId: 'correlation-heartbeat-scope-canonical',
|
||||
}),
|
||||
).resolves.toMatchObject({ scopes: ['runtime.send'] });
|
||||
await expect(
|
||||
service.release(releaseLease, {
|
||||
...context,
|
||||
correlationId: 'correlation-release-scope-canonical',
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,76 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import {
|
||||
connectorLeaseAuditLog,
|
||||
createDb,
|
||||
eq,
|
||||
logicalAgentConnectorLeases,
|
||||
type DbHandle,
|
||||
} from '@mosaicstack/db';
|
||||
import { ConnectorLeaseCoordinator } from '@mosaicstack/agent';
|
||||
import { ConnectorLeaseRepository } from './connector-lease.repository.js';
|
||||
|
||||
const hasPostgres = Boolean(process.env['DATABASE_URL']);
|
||||
const tenantId = `lease-test-${randomUUID()}`;
|
||||
const identity = { tenantId, logicalAgentId: 'mos' } as const;
|
||||
|
||||
describe.skipIf(!hasPostgres)('ConnectorLeaseRepository real PostgreSQL integration', (): void => {
|
||||
let handle: DbHandle;
|
||||
|
||||
beforeAll((): void => {
|
||||
handle = createDb(process.env['DATABASE_URL']);
|
||||
});
|
||||
|
||||
afterAll(async (): Promise<void> => {
|
||||
if (!handle) return;
|
||||
await handle.db
|
||||
.delete(connectorLeaseAuditLog)
|
||||
.where(eq(connectorLeaseAuditLog.tenantId, tenantId));
|
||||
await handle.db
|
||||
.delete(logicalAgentConnectorLeases)
|
||||
.where(eq(logicalAgentConnectorLeases.tenantId, tenantId));
|
||||
await handle.close();
|
||||
});
|
||||
|
||||
it('preserves the exclusive CAS fence across a real pool close/reopen', async (): Promise<void> => {
|
||||
const command = {
|
||||
identity,
|
||||
bindingId: 'operator-chat',
|
||||
scopes: ['runtime.send'],
|
||||
ttlMs: 60_000,
|
||||
} as const;
|
||||
const firstCoordinator = new ConnectorLeaseCoordinator(new ConnectorLeaseRepository(handle.db));
|
||||
const contenders = await Promise.allSettled([
|
||||
firstCoordinator.acquire({
|
||||
...command,
|
||||
connectorId: 'connector-a',
|
||||
correlationId: 'postgres-acquire-a',
|
||||
}),
|
||||
firstCoordinator.acquire({
|
||||
...command,
|
||||
connectorId: 'connector-b',
|
||||
correlationId: 'postgres-acquire-b',
|
||||
}),
|
||||
]);
|
||||
const acquired = contenders.find((result) => result.status === 'fulfilled');
|
||||
if (!acquired || acquired.status !== 'fulfilled') throw new Error('no lease contender won');
|
||||
expect(contenders.filter((result) => result.status === 'fulfilled')).toHaveLength(1);
|
||||
|
||||
await handle.close();
|
||||
handle = createDb(process.env['DATABASE_URL']);
|
||||
const reopened = new ConnectorLeaseCoordinator(new ConnectorLeaseRepository(handle.db));
|
||||
const persisted = await reopened.current({ identity, bindingId: 'operator-chat' });
|
||||
expect(persisted).toMatchObject({
|
||||
leaseId: acquired.value.leaseId,
|
||||
leaseEpoch: '1',
|
||||
});
|
||||
|
||||
const takeover = await reopened.takeover({
|
||||
...command,
|
||||
connectorId: 'connector-c',
|
||||
correlationId: 'postgres-takeover',
|
||||
expectedEpoch: acquired.value.leaseEpoch,
|
||||
});
|
||||
expect(takeover).toMatchObject({ connectorId: 'connector-c', leaseEpoch: '2' });
|
||||
});
|
||||
});
|
||||
149
apps/gateway/src/agent/connector-lease.repository.test.ts
Normal file
149
apps/gateway/src/agent/connector-lease.repository.test.ts
Normal file
@@ -0,0 +1,149 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
import {
|
||||
connectorLeaseAuditLog,
|
||||
createPgliteDb,
|
||||
eq,
|
||||
runPgliteMigrations,
|
||||
type DbHandle,
|
||||
} from '@mosaicstack/db';
|
||||
import { ConnectorLeaseCoordinator, ConnectorLeaseError } from '@mosaicstack/agent';
|
||||
import { ConnectorLeaseRepository } from './connector-lease.repository.js';
|
||||
|
||||
const identity = { tenantId: 'tenant-a', logicalAgentId: 'mos' } as const;
|
||||
|
||||
function acquireCommand(connectorId: string, correlationId: string) {
|
||||
return {
|
||||
identity,
|
||||
bindingId: 'operator-chat',
|
||||
connectorId,
|
||||
scopes: ['runtime.send', 'tool.execute'],
|
||||
ttlMs: 60_000,
|
||||
correlationId,
|
||||
};
|
||||
}
|
||||
|
||||
describe('ConnectorLeaseRepository PostgreSQL semantics', (): void => {
|
||||
let dataDir: string;
|
||||
let handle: DbHandle;
|
||||
let now: Date;
|
||||
let coordinator: ConnectorLeaseCoordinator;
|
||||
|
||||
beforeEach(async (): Promise<void> => {
|
||||
dataDir = await mkdtemp(join(tmpdir(), 'mosaic-connector-lease-'));
|
||||
handle = createPgliteDb(dataDir);
|
||||
await runPgliteMigrations(handle);
|
||||
now = new Date('2026-07-14T17:00:00.000Z');
|
||||
coordinator = new ConnectorLeaseCoordinator(new ConnectorLeaseRepository(handle.db), {
|
||||
now: (): Date => now,
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(async (): Promise<void> => {
|
||||
await handle.close();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('allows only one concurrent contender to acquire a binding', async (): Promise<void> => {
|
||||
const outcomes = await Promise.allSettled([
|
||||
coordinator.acquire(acquireCommand('connector-a', 'correlation-a')),
|
||||
coordinator.acquire(acquireCommand('connector-b', 'correlation-b')),
|
||||
]);
|
||||
|
||||
expect(outcomes.filter((result) => result.status === 'fulfilled')).toHaveLength(1);
|
||||
const rejected = outcomes.find((result) => result.status === 'rejected');
|
||||
expect(rejected).toMatchObject({
|
||||
reason: { code: 'lease_held' } satisfies Partial<ConnectorLeaseError>,
|
||||
});
|
||||
});
|
||||
|
||||
it('uses compare-and-swap takeover and increments the fencing epoch monotonically', async (): Promise<void> => {
|
||||
const acquired = await coordinator.acquire(acquireCommand('connector-a', 'correlation-a'));
|
||||
const results = await Promise.allSettled([
|
||||
coordinator.takeover({
|
||||
...acquireCommand('connector-b', 'correlation-b'),
|
||||
expectedEpoch: acquired.leaseEpoch,
|
||||
}),
|
||||
coordinator.takeover({
|
||||
...acquireCommand('connector-c', 'correlation-c'),
|
||||
expectedEpoch: acquired.leaseEpoch,
|
||||
}),
|
||||
]);
|
||||
const winner = results.find((result) => result.status === 'fulfilled');
|
||||
|
||||
expect(results.filter((result) => result.status === 'fulfilled')).toHaveLength(1);
|
||||
expect(winner?.status === 'fulfilled' ? winner.value.leaseEpoch : null).toBe('2');
|
||||
expect(results.find((result) => result.status === 'rejected')).toMatchObject({
|
||||
reason: { code: 'cas_mismatch' } satisfies Partial<ConnectorLeaseError>,
|
||||
});
|
||||
});
|
||||
|
||||
it('heartbeats and releases only the current connector epoch', async (): Promise<void> => {
|
||||
const acquired = await coordinator.acquire(acquireCommand('connector-a', 'correlation-a'));
|
||||
now = new Date('2026-07-14T17:00:30.000Z');
|
||||
const renewed = await coordinator.heartbeat({
|
||||
lease: acquired,
|
||||
ttlMs: 120_000,
|
||||
correlationId: 'correlation-renew',
|
||||
});
|
||||
expect(renewed.expiresAt).toBe('2026-07-14T17:02:30.000Z');
|
||||
|
||||
await coordinator.release({ lease: renewed, correlationId: 'correlation-release' });
|
||||
await expect(
|
||||
coordinator.heartbeat({
|
||||
lease: renewed,
|
||||
ttlMs: 120_000,
|
||||
correlationId: 'correlation-stale',
|
||||
}),
|
||||
).rejects.toMatchObject({ code: 'lease_released' } satisfies Partial<ConnectorLeaseError>);
|
||||
});
|
||||
|
||||
it('survives close/reopen and requires CAS takeover to recover an expired lease', async (): Promise<void> => {
|
||||
const acquired = await coordinator.acquire(acquireCommand('connector-a', 'correlation-a'));
|
||||
await handle.close();
|
||||
|
||||
now = new Date('2026-07-14T17:02:00.000Z');
|
||||
handle = createPgliteDb(dataDir);
|
||||
await runPgliteMigrations(handle);
|
||||
coordinator = new ConnectorLeaseCoordinator(new ConnectorLeaseRepository(handle.db), {
|
||||
now: (): Date => now,
|
||||
});
|
||||
|
||||
await expect(
|
||||
coordinator.acquire(acquireCommand('connector-b', 'correlation-plain-acquire')),
|
||||
).rejects.toMatchObject({ code: 'takeover_required' } satisfies Partial<ConnectorLeaseError>);
|
||||
const recovered = await coordinator.takeover({
|
||||
...acquireCommand('connector-b', 'correlation-takeover'),
|
||||
expectedEpoch: acquired.leaseEpoch,
|
||||
});
|
||||
expect(recovered).toMatchObject({ connectorId: 'connector-b', leaseEpoch: '2' });
|
||||
});
|
||||
|
||||
it('writes credential-safe lifecycle and rejection audit records', async (): Promise<void> => {
|
||||
const acquired = await coordinator.acquire(acquireCommand('connector-a', 'correlation-a'));
|
||||
await coordinator.heartbeat({
|
||||
lease: acquired,
|
||||
ttlMs: 60_000,
|
||||
correlationId: 'correlation-renew',
|
||||
});
|
||||
await expect(
|
||||
coordinator.acquire(acquireCommand('connector-b', 'correlation-reject')),
|
||||
).rejects.toBeInstanceOf(ConnectorLeaseError);
|
||||
|
||||
const rows = await handle.db
|
||||
.select()
|
||||
.from(connectorLeaseAuditLog)
|
||||
.where(eq(connectorLeaseAuditLog.tenantId, identity.tenantId));
|
||||
expect(rows.map((row) => row.event)).toEqual(
|
||||
expect.arrayContaining(['acquire', 'renew', 'reject']),
|
||||
);
|
||||
const serialized = JSON.stringify(rows, (_key: string, value: unknown): unknown =>
|
||||
typeof value === 'bigint' ? value.toString(10) : value,
|
||||
);
|
||||
expect(serialized).not.toContain('tool.execute');
|
||||
expect(serialized).not.toContain('runtime.send');
|
||||
expect(serialized).not.toMatch(/token|secret|credential/i);
|
||||
});
|
||||
});
|
||||
354
apps/gateway/src/agent/connector-lease.repository.ts
Normal file
354
apps/gateway/src/agent/connector-lease.repository.ts
Normal file
@@ -0,0 +1,354 @@
|
||||
import { Inject, Injectable } from '@nestjs/common';
|
||||
import {
|
||||
and,
|
||||
connectorLeaseAuditLog,
|
||||
eq,
|
||||
gt,
|
||||
isNull,
|
||||
logicalAgentConnectorLeases,
|
||||
sql,
|
||||
type Db,
|
||||
} from '@mosaicstack/db';
|
||||
import { ConnectorLeaseError } from '@mosaicstack/agent';
|
||||
import type {
|
||||
ConnectorLease,
|
||||
ConnectorLeaseAcquireMutation,
|
||||
ConnectorLeaseAuditEvent,
|
||||
ConnectorLeaseHeartbeatMutation,
|
||||
ConnectorLeaseRejectReason,
|
||||
ConnectorLeaseReleaseMutation,
|
||||
ConnectorLeaseStore,
|
||||
ConnectorLeaseTakeoverMutation,
|
||||
LogicalAgentBinding,
|
||||
} from '@mosaicstack/types';
|
||||
import { DB } from '../database/database.module.js';
|
||||
|
||||
interface SuccessfulMutation {
|
||||
readonly ok: true;
|
||||
readonly lease: ConnectorLease;
|
||||
}
|
||||
|
||||
interface FailedMutation {
|
||||
readonly ok: false;
|
||||
readonly reason: ConnectorLeaseRejectReason;
|
||||
}
|
||||
|
||||
type MutationResult = SuccessfulMutation | FailedMutation;
|
||||
|
||||
@Injectable()
|
||||
export class ConnectorLeaseRepository implements ConnectorLeaseStore {
|
||||
constructor(@Inject(DB) private readonly db: Db) {}
|
||||
|
||||
async acquire(input: ConnectorLeaseAcquireMutation): Promise<ConnectorLease> {
|
||||
const result: MutationResult = await this.db.transaction(
|
||||
async (tx): Promise<MutationResult> => {
|
||||
const inserted = await tx
|
||||
.insert(logicalAgentConnectorLeases)
|
||||
.values({
|
||||
leaseId: input.leaseId,
|
||||
tenantId: input.identity.tenantId,
|
||||
logicalAgentId: input.identity.logicalAgentId,
|
||||
bindingId: input.bindingId,
|
||||
connectorId: input.connectorId,
|
||||
scopes: [...input.scopes],
|
||||
leaseEpoch: 1n,
|
||||
acquiredAt: new Date(input.now),
|
||||
heartbeatAt: new Date(input.now),
|
||||
expiresAt: new Date(input.expiresAt),
|
||||
updatedAt: new Date(input.now),
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning();
|
||||
const row = inserted[0];
|
||||
if (row) {
|
||||
const lease = toLease(row);
|
||||
await insertAudit(tx, lifecycleAudit(input, lease, 'acquire'));
|
||||
return { ok: true, lease };
|
||||
}
|
||||
|
||||
const current = await findRow(tx, input);
|
||||
if (current && current.expiresAt <= new Date(input.now) && !current.releasedAt) {
|
||||
await insertAudit(tx, lifecycleAudit(input, toLease(current), 'expiry'));
|
||||
}
|
||||
const reason: ConnectorLeaseRejectReason =
|
||||
current && (current.releasedAt || current.expiresAt <= new Date(input.now))
|
||||
? 'takeover_required'
|
||||
: 'lease_held';
|
||||
await insertAudit(tx, rejectionAudit(input, current ? toLease(current) : null, reason));
|
||||
return { ok: false, reason };
|
||||
},
|
||||
);
|
||||
return unwrap(result);
|
||||
}
|
||||
|
||||
async takeover(input: ConnectorLeaseTakeoverMutation): Promise<ConnectorLease> {
|
||||
const result: MutationResult = await this.db.transaction(
|
||||
async (tx): Promise<MutationResult> => {
|
||||
const current = await findRow(tx, input);
|
||||
if (!current || current.leaseEpoch.toString(10) !== input.expectedEpoch) {
|
||||
await insertAudit(
|
||||
tx,
|
||||
rejectionAudit(input, current ? toLease(current) : null, 'cas_mismatch'),
|
||||
);
|
||||
return { ok: false, reason: 'cas_mismatch' };
|
||||
}
|
||||
if (current.expiresAt <= new Date(input.now) && !current.releasedAt) {
|
||||
await insertAudit(tx, lifecycleAudit(input, toLease(current), 'expiry'));
|
||||
}
|
||||
const updated = await tx
|
||||
.update(logicalAgentConnectorLeases)
|
||||
.set({
|
||||
leaseId: input.leaseId,
|
||||
connectorId: input.connectorId,
|
||||
scopes: [...input.scopes],
|
||||
leaseEpoch: sql`${logicalAgentConnectorLeases.leaseEpoch} + 1`,
|
||||
acquiredAt: new Date(input.now),
|
||||
heartbeatAt: new Date(input.now),
|
||||
expiresAt: new Date(input.expiresAt),
|
||||
releasedAt: null,
|
||||
updatedAt: new Date(input.now),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
bindingPredicate(input),
|
||||
eq(logicalAgentConnectorLeases.leaseId, current.leaseId),
|
||||
eq(logicalAgentConnectorLeases.leaseEpoch, BigInt(input.expectedEpoch)),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
const row = updated[0];
|
||||
if (!row) {
|
||||
await insertAudit(tx, rejectionAudit(input, toLease(current), 'cas_mismatch'));
|
||||
return { ok: false, reason: 'cas_mismatch' };
|
||||
}
|
||||
const lease = toLease(row);
|
||||
await insertAudit(tx, lifecycleAudit(input, lease, 'takeover'));
|
||||
return { ok: true, lease };
|
||||
},
|
||||
);
|
||||
return unwrap(result);
|
||||
}
|
||||
|
||||
async heartbeat(input: ConnectorLeaseHeartbeatMutation): Promise<ConnectorLease> {
|
||||
const result: MutationResult = await this.db.transaction(
|
||||
async (tx): Promise<MutationResult> => {
|
||||
const updated = await tx
|
||||
.update(logicalAgentConnectorLeases)
|
||||
.set({
|
||||
heartbeatAt: new Date(input.now),
|
||||
expiresAt: new Date(input.expiresAt),
|
||||
updatedAt: new Date(input.now),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
bindingPredicate(input.lease),
|
||||
eq(logicalAgentConnectorLeases.leaseId, input.lease.leaseId),
|
||||
eq(logicalAgentConnectorLeases.connectorId, input.lease.connectorId),
|
||||
eq(logicalAgentConnectorLeases.leaseEpoch, BigInt(input.lease.leaseEpoch)),
|
||||
isNull(logicalAgentConnectorLeases.releasedAt),
|
||||
gt(logicalAgentConnectorLeases.expiresAt, new Date(input.now)),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
const row = updated[0];
|
||||
if (row) {
|
||||
const lease = toLease(row);
|
||||
await insertAudit(tx, lifecycleAudit(input, lease, 'renew'));
|
||||
return { ok: true, lease };
|
||||
}
|
||||
const current = await findRow(tx, input.lease);
|
||||
const reason = classifyAuthorityFailure(
|
||||
current ? toLease(current) : null,
|
||||
input.lease,
|
||||
input.now,
|
||||
);
|
||||
if (reason === 'lease_expired' && current) {
|
||||
await insertAudit(tx, lifecycleAudit(input, toLease(current), 'expiry'));
|
||||
}
|
||||
await insertAudit(
|
||||
tx,
|
||||
rejectionAudit(
|
||||
{ ...input.lease, correlationId: input.correlationId, now: input.now },
|
||||
current ? toLease(current) : null,
|
||||
reason,
|
||||
),
|
||||
);
|
||||
return { ok: false, reason };
|
||||
},
|
||||
);
|
||||
return unwrap(result);
|
||||
}
|
||||
|
||||
async release(input: ConnectorLeaseReleaseMutation): Promise<void> {
|
||||
const result: MutationResult = await this.db.transaction(
|
||||
async (tx): Promise<MutationResult> => {
|
||||
const updated = await tx
|
||||
.update(logicalAgentConnectorLeases)
|
||||
.set({
|
||||
releasedAt: new Date(input.now),
|
||||
expiresAt: new Date(input.now),
|
||||
updatedAt: new Date(input.now),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
bindingPredicate(input.lease),
|
||||
eq(logicalAgentConnectorLeases.leaseId, input.lease.leaseId),
|
||||
eq(logicalAgentConnectorLeases.connectorId, input.lease.connectorId),
|
||||
eq(logicalAgentConnectorLeases.leaseEpoch, BigInt(input.lease.leaseEpoch)),
|
||||
isNull(logicalAgentConnectorLeases.releasedAt),
|
||||
gt(logicalAgentConnectorLeases.expiresAt, new Date(input.now)),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
const row = updated[0];
|
||||
if (row) {
|
||||
const lease = toLease(row);
|
||||
await insertAudit(tx, lifecycleAudit(input, lease, 'release'));
|
||||
return { ok: true, lease };
|
||||
}
|
||||
const current = await findRow(tx, input.lease);
|
||||
const reason = classifyAuthorityFailure(
|
||||
current ? toLease(current) : null,
|
||||
input.lease,
|
||||
input.now,
|
||||
);
|
||||
if (reason === 'lease_expired' && current) {
|
||||
await insertAudit(tx, lifecycleAudit(input, toLease(current), 'expiry'));
|
||||
}
|
||||
await insertAudit(
|
||||
tx,
|
||||
rejectionAudit(
|
||||
{ ...input.lease, correlationId: input.correlationId, now: input.now },
|
||||
current ? toLease(current) : null,
|
||||
reason,
|
||||
),
|
||||
);
|
||||
return { ok: false, reason };
|
||||
},
|
||||
);
|
||||
unwrap(result);
|
||||
}
|
||||
|
||||
async findCurrent(binding: LogicalAgentBinding): Promise<ConnectorLease | null> {
|
||||
const row = await findRow(this.db, binding);
|
||||
return row ? toLease(row) : null;
|
||||
}
|
||||
|
||||
async recordAudit(event: ConnectorLeaseAuditEvent): Promise<void> {
|
||||
await insertAudit(this.db, event);
|
||||
}
|
||||
}
|
||||
|
||||
function unwrap(result: MutationResult): ConnectorLease {
|
||||
if (!result.ok) throw new ConnectorLeaseError(result.reason, safeErrorMessage(result.reason));
|
||||
return result.lease;
|
||||
}
|
||||
|
||||
function safeErrorMessage(reason: ConnectorLeaseRejectReason): string {
|
||||
return `Connector lease mutation denied: ${reason}`;
|
||||
}
|
||||
|
||||
function bindingPredicate(binding: LogicalAgentBinding) {
|
||||
return and(
|
||||
eq(logicalAgentConnectorLeases.tenantId, binding.identity.tenantId),
|
||||
eq(logicalAgentConnectorLeases.logicalAgentId, binding.identity.logicalAgentId),
|
||||
eq(logicalAgentConnectorLeases.bindingId, binding.bindingId),
|
||||
);
|
||||
}
|
||||
|
||||
async function findRow(
|
||||
db: Pick<Db, 'select'>,
|
||||
binding: LogicalAgentBinding,
|
||||
): Promise<typeof logicalAgentConnectorLeases.$inferSelect | null> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(logicalAgentConnectorLeases)
|
||||
.where(bindingPredicate(binding))
|
||||
.limit(1);
|
||||
return rows[0] ?? null;
|
||||
}
|
||||
|
||||
function toLease(row: typeof logicalAgentConnectorLeases.$inferSelect): ConnectorLease {
|
||||
return Object.freeze({
|
||||
identity: Object.freeze({ tenantId: row.tenantId, logicalAgentId: row.logicalAgentId }),
|
||||
bindingId: row.bindingId,
|
||||
leaseId: row.leaseId,
|
||||
connectorId: row.connectorId,
|
||||
scopes: Object.freeze([...row.scopes]),
|
||||
leaseEpoch: row.leaseEpoch.toString(10),
|
||||
acquiredAt: row.acquiredAt.toISOString(),
|
||||
heartbeatAt: row.heartbeatAt.toISOString(),
|
||||
expiresAt: row.expiresAt.toISOString(),
|
||||
...(row.releasedAt ? { releasedAt: row.releasedAt.toISOString() } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
function classifyAuthorityFailure(
|
||||
current: ConnectorLease | null,
|
||||
claimed: ConnectorLease,
|
||||
now: string,
|
||||
): ConnectorLeaseRejectReason {
|
||||
if (!current) return 'lease_missing';
|
||||
if (current.releasedAt) return 'lease_released';
|
||||
if (new Date(current.expiresAt) <= new Date(now)) return 'lease_expired';
|
||||
if (current.leaseEpoch !== claimed.leaseEpoch) return 'stale_epoch';
|
||||
return 'connector_mismatch';
|
||||
}
|
||||
|
||||
function lifecycleAudit(
|
||||
input: { readonly correlationId: string; readonly now: string },
|
||||
lease: ConnectorLease,
|
||||
event: Exclude<ConnectorLeaseAuditEvent['event'], 'reject'>,
|
||||
): ConnectorLeaseAuditEvent {
|
||||
return {
|
||||
identity: lease.identity,
|
||||
bindingId: lease.bindingId,
|
||||
connectorId: lease.connectorId,
|
||||
leaseId: lease.leaseId,
|
||||
leaseEpoch: lease.leaseEpoch,
|
||||
event,
|
||||
outcome: 'succeeded',
|
||||
correlationId: input.correlationId,
|
||||
occurredAt: input.now,
|
||||
};
|
||||
}
|
||||
|
||||
function rejectionAudit(
|
||||
input: {
|
||||
readonly identity: ConnectorLease['identity'];
|
||||
readonly bindingId: string;
|
||||
readonly connectorId: string;
|
||||
readonly correlationId: string;
|
||||
readonly now: string;
|
||||
},
|
||||
current: ConnectorLease | null,
|
||||
reason: ConnectorLeaseRejectReason,
|
||||
): ConnectorLeaseAuditEvent {
|
||||
return {
|
||||
identity: input.identity,
|
||||
bindingId: input.bindingId,
|
||||
connectorId: input.connectorId,
|
||||
event: 'reject',
|
||||
outcome: 'denied',
|
||||
correlationId: input.correlationId,
|
||||
occurredAt: input.now,
|
||||
...(current ? { leaseId: current.leaseId, leaseEpoch: current.leaseEpoch } : {}),
|
||||
reason,
|
||||
};
|
||||
}
|
||||
|
||||
async function insertAudit(db: Pick<Db, 'insert'>, event: ConnectorLeaseAuditEvent): Promise<void> {
|
||||
await db.insert(connectorLeaseAuditLog).values({
|
||||
tenantId: event.identity.tenantId,
|
||||
logicalAgentId: event.identity.logicalAgentId,
|
||||
bindingId: event.bindingId,
|
||||
connectorId: event.connectorId,
|
||||
...(event.leaseId ? { leaseId: event.leaseId } : {}),
|
||||
...(event.leaseEpoch ? { leaseEpoch: BigInt(event.leaseEpoch) } : {}),
|
||||
event: event.event,
|
||||
outcome: event.outcome,
|
||||
...(event.reason ? { reason: event.reason } : {}),
|
||||
correlationId: event.correlationId,
|
||||
occurredAt: new Date(event.occurredAt),
|
||||
});
|
||||
}
|
||||
285
apps/gateway/src/agent/connector-lease.service.ts
Normal file
285
apps/gateway/src/agent/connector-lease.service.ts
Normal file
@@ -0,0 +1,285 @@
|
||||
import { ForbiddenException, Inject, Injectable } from '@nestjs/common';
|
||||
import { ConnectorLeaseCoordinator, normalizeConnectorLease } from '@mosaicstack/agent';
|
||||
import {
|
||||
normalizeConnectorId,
|
||||
normalizeConnectorScopes,
|
||||
normalizeCorrelationId,
|
||||
normalizeLogicalAgentIdentity,
|
||||
normalizeLogicalBindingId,
|
||||
type AcquireConnectorLeaseInput,
|
||||
type ConnectorExecutionGrant,
|
||||
type ConnectorLease,
|
||||
type ConnectorLeaseAuditEvent,
|
||||
type FencedConnectorAdapter,
|
||||
} from '@mosaicstack/types';
|
||||
import type { ActorTenantScope } from '../auth/session-scope.js';
|
||||
import { ConnectorLeaseRepository } from './connector-lease.repository.js';
|
||||
|
||||
export const CONNECTOR_LEASE_POLICY = Symbol('CONNECTOR_LEASE_POLICY');
|
||||
|
||||
export type ConnectorLeasePolicyAction =
|
||||
| 'lease.acquire'
|
||||
| 'lease.takeover'
|
||||
| 'lease.heartbeat'
|
||||
| 'lease.release'
|
||||
| 'lease.read'
|
||||
| 'grant.issue';
|
||||
|
||||
export interface ConnectorLeaseRequestContext {
|
||||
readonly actorScope: ActorTenantScope;
|
||||
readonly correlationId: string;
|
||||
}
|
||||
|
||||
export interface GatewayConnectorLeaseRequest {
|
||||
readonly logicalAgentId: string;
|
||||
readonly bindingId: string;
|
||||
readonly connectorId: string;
|
||||
readonly scopes: readonly string[];
|
||||
readonly ttlMs: number;
|
||||
}
|
||||
|
||||
export interface GatewayConnectorLeaseTakeoverRequest extends GatewayConnectorLeaseRequest {
|
||||
readonly expectedEpoch: string;
|
||||
}
|
||||
|
||||
export interface GatewayConnectorGrantRequest {
|
||||
readonly lease: ConnectorLease;
|
||||
readonly scopes: readonly string[];
|
||||
readonly ttlMs: number;
|
||||
}
|
||||
|
||||
export interface ConnectorLeasePolicySubject {
|
||||
readonly action: ConnectorLeasePolicyAction;
|
||||
readonly actorId: string;
|
||||
readonly tenantId: string;
|
||||
readonly logicalAgentId: string;
|
||||
readonly bindingId: string;
|
||||
readonly connectorId: string;
|
||||
readonly requestedScopes: readonly string[];
|
||||
readonly requestedTtlMs: number | null;
|
||||
}
|
||||
|
||||
export interface ConnectorLeasePolicy {
|
||||
authorize(subject: ConnectorLeasePolicySubject): Promise<boolean>;
|
||||
}
|
||||
|
||||
/** M1 has no concrete cutover policy: unconfigured production use fails closed. */
|
||||
@Injectable()
|
||||
export class DenyConnectorLeasePolicy implements ConnectorLeasePolicy {
|
||||
async authorize(_subject: ConnectorLeasePolicySubject): Promise<boolean> {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Gateway-owned policy surface for durable connector authority and fenced effects. */
|
||||
@Injectable()
|
||||
export class ConnectorLeaseService {
|
||||
private readonly coordinator: ConnectorLeaseCoordinator;
|
||||
|
||||
constructor(
|
||||
@Inject(ConnectorLeaseRepository) private readonly repository: ConnectorLeaseRepository,
|
||||
@Inject(CONNECTOR_LEASE_POLICY) private readonly policy: ConnectorLeasePolicy,
|
||||
) {
|
||||
this.coordinator = new ConnectorLeaseCoordinator(repository);
|
||||
}
|
||||
|
||||
async acquire(
|
||||
request: GatewayConnectorLeaseRequest,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorLease> {
|
||||
const command = this.command(request, context);
|
||||
await this.assertPolicy('lease.acquire', command, context, command.scopes, command.ttlMs);
|
||||
return this.coordinator.acquire({ ...command, correlationId: this.correlation(context) });
|
||||
}
|
||||
|
||||
async takeover(
|
||||
request: GatewayConnectorLeaseTakeoverRequest,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorLease> {
|
||||
const command = this.command(request, context);
|
||||
await this.assertPolicy('lease.takeover', command, context, command.scopes, command.ttlMs);
|
||||
return this.coordinator.takeover({
|
||||
...command,
|
||||
expectedEpoch: request.expectedEpoch,
|
||||
correlationId: this.correlation(context),
|
||||
});
|
||||
}
|
||||
|
||||
async heartbeat(
|
||||
lease: ConnectorLease,
|
||||
ttlMs: number,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorLease> {
|
||||
const normalizedLease = normalizeConnectorLease(lease);
|
||||
const durableLease = await this.durableLifecycleLease(normalizedLease, context);
|
||||
await this.assertPolicy('lease.heartbeat', durableLease, context, durableLease.scopes, ttlMs);
|
||||
return this.coordinator.heartbeat({
|
||||
lease: durableLease,
|
||||
ttlMs,
|
||||
correlationId: this.correlation(context),
|
||||
});
|
||||
}
|
||||
|
||||
async release(lease: ConnectorLease, context: ConnectorLeaseRequestContext): Promise<void> {
|
||||
const normalizedLease = normalizeConnectorLease(lease);
|
||||
const durableLease = await this.durableLifecycleLease(normalizedLease, context);
|
||||
await this.assertPolicy('lease.release', durableLease, context, durableLease.scopes, null);
|
||||
await this.coordinator.release({
|
||||
lease: durableLease,
|
||||
correlationId: this.correlation(context),
|
||||
});
|
||||
}
|
||||
|
||||
async current(
|
||||
logicalAgentId: string,
|
||||
bindingId: string,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorLease | null> {
|
||||
const binding = {
|
||||
identity: normalizeLogicalAgentIdentity({
|
||||
tenantId: context.actorScope.tenantId,
|
||||
logicalAgentId,
|
||||
}),
|
||||
bindingId: normalizeLogicalBindingId(bindingId),
|
||||
connectorId: 'gateway',
|
||||
};
|
||||
await this.assertPolicy('lease.read', binding, context, [], null);
|
||||
return this.coordinator.current(binding);
|
||||
}
|
||||
|
||||
async issueGrant(
|
||||
request: GatewayConnectorGrantRequest,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorExecutionGrant> {
|
||||
const lease = normalizeConnectorLease(request.lease);
|
||||
await this.assertTenant(lease, context);
|
||||
const scopes = normalizeConnectorScopes(request.scopes);
|
||||
await this.assertPolicy('grant.issue', lease, context, scopes, request.ttlMs);
|
||||
return this.coordinator.issueGrant({
|
||||
lease,
|
||||
scopes,
|
||||
ttlMs: request.ttlMs,
|
||||
correlationId: this.correlation(context),
|
||||
});
|
||||
}
|
||||
|
||||
async executeGrant<TInput, TOutput>(
|
||||
grant: ConnectorExecutionGrant,
|
||||
requiredScope: string,
|
||||
input: TInput,
|
||||
adapter: FencedConnectorAdapter<TInput, TOutput>,
|
||||
): Promise<TOutput> {
|
||||
return this.coordinator.executeGrant(grant, requiredScope, input, adapter);
|
||||
}
|
||||
|
||||
private command(
|
||||
request: GatewayConnectorLeaseRequest,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Omit<AcquireConnectorLeaseInput, 'correlationId'> {
|
||||
return {
|
||||
identity: normalizeLogicalAgentIdentity({
|
||||
tenantId: context.actorScope.tenantId,
|
||||
logicalAgentId: request.logicalAgentId,
|
||||
}),
|
||||
bindingId: normalizeLogicalBindingId(request.bindingId),
|
||||
connectorId: normalizeConnectorId(request.connectorId),
|
||||
scopes: normalizeConnectorScopes(request.scopes),
|
||||
ttlMs: request.ttlMs,
|
||||
};
|
||||
}
|
||||
|
||||
private async assertTenant(
|
||||
lease: Pick<ConnectorLease, 'identity' | 'bindingId' | 'connectorId'>,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<void> {
|
||||
if (lease.identity.tenantId !== context.actorScope.tenantId) {
|
||||
await this.recordPolicyDenial(
|
||||
{
|
||||
identity: {
|
||||
tenantId: context.actorScope.tenantId,
|
||||
logicalAgentId: 'untrusted',
|
||||
},
|
||||
bindingId: 'untrusted',
|
||||
connectorId: 'untrusted',
|
||||
},
|
||||
context,
|
||||
);
|
||||
throw new ForbiddenException('Connector authority tenant scope denied');
|
||||
}
|
||||
}
|
||||
|
||||
private async durableLifecycleLease(
|
||||
submittedLease: ConnectorLease,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<ConnectorLease> {
|
||||
await this.assertTenant(submittedLease, context);
|
||||
const durableLease = await this.coordinator.current(submittedLease);
|
||||
if (!durableLease || !hasSameLifecycleAuthority(submittedLease, durableLease)) {
|
||||
await this.recordPolicyDenial(durableLease ?? submittedLease, context);
|
||||
throw new ForbiddenException('Connector authority policy denied');
|
||||
}
|
||||
return durableLease;
|
||||
}
|
||||
|
||||
private async assertPolicy(
|
||||
action: ConnectorLeasePolicyAction,
|
||||
subject: Pick<ConnectorLease, 'identity' | 'bindingId' | 'connectorId'>,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
requestedScopes: readonly string[],
|
||||
requestedTtlMs: number | null,
|
||||
): Promise<void> {
|
||||
const allowed = await this.policy.authorize({
|
||||
action,
|
||||
actorId: context.actorScope.userId,
|
||||
tenantId: subject.identity.tenantId,
|
||||
logicalAgentId: subject.identity.logicalAgentId,
|
||||
bindingId: subject.bindingId,
|
||||
connectorId: subject.connectorId,
|
||||
requestedScopes: Object.freeze([...requestedScopes]),
|
||||
requestedTtlMs,
|
||||
});
|
||||
if (!allowed) {
|
||||
await this.recordPolicyDenial(subject, context);
|
||||
throw new ForbiddenException('Connector authority policy denied');
|
||||
}
|
||||
}
|
||||
|
||||
private async recordPolicyDenial(
|
||||
subject: Pick<ConnectorLease, 'identity' | 'bindingId' | 'connectorId'>,
|
||||
context: ConnectorLeaseRequestContext,
|
||||
): Promise<void> {
|
||||
const event: ConnectorLeaseAuditEvent = {
|
||||
identity: subject.identity,
|
||||
bindingId: subject.bindingId,
|
||||
connectorId: subject.connectorId,
|
||||
event: 'reject',
|
||||
outcome: 'denied',
|
||||
reason: 'policy_denied',
|
||||
correlationId: this.correlation(context),
|
||||
occurredAt: new Date().toISOString(),
|
||||
};
|
||||
await this.repository.recordAudit(event);
|
||||
}
|
||||
|
||||
private correlation(context: ConnectorLeaseRequestContext): string {
|
||||
return normalizeCorrelationId(context.correlationId);
|
||||
}
|
||||
}
|
||||
|
||||
function hasSameLifecycleAuthority(
|
||||
submittedLease: ConnectorLease,
|
||||
durableLease: ConnectorLease,
|
||||
): boolean {
|
||||
return (
|
||||
submittedLease.identity.tenantId === durableLease.identity.tenantId &&
|
||||
submittedLease.identity.logicalAgentId === durableLease.identity.logicalAgentId &&
|
||||
submittedLease.bindingId === durableLease.bindingId &&
|
||||
submittedLease.leaseId === durableLease.leaseId &&
|
||||
submittedLease.connectorId === durableLease.connectorId &&
|
||||
submittedLease.leaseEpoch === durableLease.leaseEpoch &&
|
||||
submittedLease.scopes.length === durableLease.scopes.length &&
|
||||
submittedLease.scopes.every((scope: string, index: number): boolean => {
|
||||
return scope === durableLease.scopes[index];
|
||||
})
|
||||
);
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { ChannelAttachmentDto } from '@mosaicstack/types';
|
||||
import { IsOptional, IsString, IsUUID, MaxLength } from 'class-validator';
|
||||
|
||||
export class ChatRequestDto {
|
||||
@@ -32,4 +33,7 @@ export class ChatSocketMessageDto {
|
||||
@IsOptional()
|
||||
@IsUUID()
|
||||
agentId?: string;
|
||||
|
||||
/** Validated channel attachment references; binary content is not embedded. */
|
||||
attachments?: readonly ChannelAttachmentDto[];
|
||||
}
|
||||
|
||||
@@ -4,6 +4,10 @@ import { ChatGateway } from './chat.gateway.js';
|
||||
const CONVERSATION_ID = 'conversation-1';
|
||||
const CANARY = 'sk_canary12345678';
|
||||
|
||||
function clientConversationKey(clientId: string, conversationId: string): string {
|
||||
return `${clientId}\u0000${conversationId}`;
|
||||
}
|
||||
|
||||
type GatewayInternals = {
|
||||
clientSessions: Map<string, unknown>;
|
||||
relayEvent(client: unknown, conversationId: string, event: unknown): void;
|
||||
@@ -40,6 +44,7 @@ describe('ChatGateway redaction boundary', (): void => {
|
||||
emit: vi.fn(),
|
||||
};
|
||||
const session = {
|
||||
clientId: client.id,
|
||||
conversationId: CONVERSATION_ID,
|
||||
cleanup: vi.fn(),
|
||||
assistantText: '',
|
||||
@@ -47,7 +52,7 @@ describe('ChatGateway redaction boundary', (): void => {
|
||||
pendingToolCalls: new Map(),
|
||||
scope: { userId: 'user-1', tenantId: 'tenant-1' },
|
||||
};
|
||||
gateway.clientSessions.set(client.id, session);
|
||||
gateway.clientSessions.set(clientConversationKey(client.id, CONVERSATION_ID), session);
|
||||
|
||||
gateway.relayEvent(client, CONVERSATION_ID, {
|
||||
type: 'message_update',
|
||||
@@ -139,6 +144,51 @@ describe('ChatGateway redaction boundary', (): void => {
|
||||
});
|
||||
});
|
||||
|
||||
it('isolates concurrent conversation streams sharing one Discord socket', (): void => {
|
||||
const { gateway } = buildGateway();
|
||||
const client = {
|
||||
connected: true,
|
||||
id: 'discord-client',
|
||||
data: { user: { id: 'user-1' } },
|
||||
emit: vi.fn(),
|
||||
};
|
||||
const firstConversation = 'Nova:discord:thread-1';
|
||||
const secondConversation = 'Nova:discord:thread-2';
|
||||
const createSession = (conversationId: string) => ({
|
||||
clientId: client.id,
|
||||
conversationId,
|
||||
cleanup: vi.fn(),
|
||||
assistantText: '',
|
||||
toolCalls: [],
|
||||
pendingToolCalls: new Map(),
|
||||
scope: { userId: 'user-1', tenantId: 'tenant-1' },
|
||||
});
|
||||
const firstSession = createSession(firstConversation);
|
||||
const secondSession = createSession(secondConversation);
|
||||
gateway.clientSessions.set(clientConversationKey(client.id, firstConversation), firstSession);
|
||||
gateway.clientSessions.set(clientConversationKey(client.id, secondConversation), secondSession);
|
||||
|
||||
gateway.relayEvent(client, firstConversation, {
|
||||
type: 'message_update',
|
||||
assistantMessageEvent: { type: 'text_delta', delta: 'first response ' },
|
||||
});
|
||||
gateway.relayEvent(client, secondConversation, {
|
||||
type: 'message_update',
|
||||
assistantMessageEvent: { type: 'text_delta', delta: 'second response ' },
|
||||
});
|
||||
|
||||
expect(firstSession.assistantText).toBe('first response ');
|
||||
expect(secondSession.assistantText).toBe('second response ');
|
||||
expect(client.emit).toHaveBeenCalledWith('agent:text', {
|
||||
conversationId: firstConversation,
|
||||
text: 'first response ',
|
||||
});
|
||||
expect(client.emit).toHaveBeenCalledWith('agent:text', {
|
||||
conversationId: secondConversation,
|
||||
text: 'second response ',
|
||||
});
|
||||
});
|
||||
|
||||
it('persists only redacted assistant content with classifications', (): void => {
|
||||
const { gateway, brain } = buildGateway();
|
||||
const client = {
|
||||
@@ -147,7 +197,8 @@ describe('ChatGateway redaction boundary', (): void => {
|
||||
data: { user: { id: 'user-1' } },
|
||||
emit: vi.fn(),
|
||||
};
|
||||
gateway.clientSessions.set(client.id, {
|
||||
gateway.clientSessions.set(clientConversationKey(client.id, CONVERSATION_ID), {
|
||||
clientId: client.id,
|
||||
conversationId: CONVERSATION_ID,
|
||||
cleanup: vi.fn(),
|
||||
assistantText: CANARY,
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
parseDiscordInteractionBindings,
|
||||
resolveDiscordInteractionActorId,
|
||||
resolveDiscordInteractionBinding,
|
||||
type DiscordAttachment,
|
||||
type DiscordIngressEnvelope,
|
||||
type DiscordIngressPayload,
|
||||
} from '@mosaicstack/discord-plugin';
|
||||
@@ -30,6 +31,7 @@ import type {
|
||||
SystemReloadPayload,
|
||||
RoutingDecisionInfo,
|
||||
AbortPayload,
|
||||
ChannelAttachmentDto,
|
||||
} from '@mosaicstack/types';
|
||||
import { AgentService, type ConversationHistoryMessage } from '../agent/agent.service.js';
|
||||
import {
|
||||
@@ -56,6 +58,7 @@ import { DiscordReplayProtector } from '../plugin/discord-replay-protector.js';
|
||||
|
||||
/** Per-client state tracking streaming accumulation for persistence. */
|
||||
interface ClientSession {
|
||||
clientId: string;
|
||||
conversationId: string;
|
||||
cleanup: () => void;
|
||||
/** Accumulated assistant response text for the current turn. */
|
||||
@@ -76,6 +79,68 @@ interface ClientSession {
|
||||
*/
|
||||
const modelOverrides = new Map<string, string>();
|
||||
const MAX_REDACTION_BUFFER_LENGTH = 8_192;
|
||||
const MAX_CHANNEL_ATTACHMENTS = 10;
|
||||
const MAX_ATTACHMENT_METADATA_BYTES = 16_384;
|
||||
const MAX_ATTACHMENT_ID_LENGTH = 128;
|
||||
const MAX_ATTACHMENT_NAME_LENGTH = 255;
|
||||
const MAX_ATTACHMENT_URL_LENGTH = 2_048;
|
||||
const MAX_ATTACHMENT_MIME_LENGTH = 255;
|
||||
|
||||
function isSafeAttachmentUrl(value: string): boolean {
|
||||
if (value.length === 0 || value.length > MAX_ATTACHMENT_URL_LENGTH) return false;
|
||||
try {
|
||||
const url = new URL(value);
|
||||
return (
|
||||
url.protocol === 'https:' &&
|
||||
!url.username &&
|
||||
!url.password &&
|
||||
!url.hash &&
|
||||
url.search.length === 0
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function hasValidAttachmentBounds(value: {
|
||||
id: string;
|
||||
name: string;
|
||||
url: string;
|
||||
sizeBytes?: number;
|
||||
}): boolean {
|
||||
return (
|
||||
value.id.length > 0 &&
|
||||
value.id.length <= MAX_ATTACHMENT_ID_LENGTH &&
|
||||
value.name.length > 0 &&
|
||||
value.name.length <= MAX_ATTACHMENT_NAME_LENGTH &&
|
||||
isSafeAttachmentUrl(value.url) &&
|
||||
(value.sizeBytes === undefined || (Number.isFinite(value.sizeBytes) && value.sizeBytes >= 0))
|
||||
);
|
||||
}
|
||||
|
||||
function isDiscordAttachment(value: unknown): value is DiscordAttachment {
|
||||
if (typeof value !== 'object' || value === null) return false;
|
||||
const attachment = value as Partial<DiscordAttachment>;
|
||||
return (
|
||||
typeof attachment.id === 'string' &&
|
||||
typeof attachment.name === 'string' &&
|
||||
typeof attachment.url === 'string' &&
|
||||
(attachment.contentType === null ||
|
||||
(typeof attachment.contentType === 'string' &&
|
||||
attachment.contentType.length <= MAX_ATTACHMENT_MIME_LENGTH)) &&
|
||||
(attachment.sizeBytes === undefined || typeof attachment.sizeBytes === 'number') &&
|
||||
hasValidAttachmentBounds(attachment as DiscordAttachment)
|
||||
);
|
||||
}
|
||||
|
||||
function hasValidAttachmentArray(value: unknown, guard: (attachment: unknown) => boolean): boolean {
|
||||
return (
|
||||
Array.isArray(value) &&
|
||||
value.length <= MAX_CHANNEL_ATTACHMENTS &&
|
||||
JSON.stringify(value).length <= MAX_ATTACHMENT_METADATA_BYTES &&
|
||||
value.every(guard)
|
||||
);
|
||||
}
|
||||
|
||||
function isDiscordIngressEnvelope(value: unknown): value is DiscordIngressEnvelope {
|
||||
if (typeof value !== 'object' || value === null) return false;
|
||||
@@ -88,7 +153,8 @@ function isDiscordIngressEnvelope(value: unknown): value is DiscordIngressEnvelo
|
||||
return false;
|
||||
}
|
||||
const payload = envelope.payload as Record<string, unknown>;
|
||||
return [
|
||||
return (
|
||||
[
|
||||
payload['correlationId'],
|
||||
payload['messageId'],
|
||||
payload['guildId'],
|
||||
@@ -96,15 +162,40 @@ function isDiscordIngressEnvelope(value: unknown): value is DiscordIngressEnvelo
|
||||
payload['userId'],
|
||||
payload['conversationId'],
|
||||
payload['content'],
|
||||
].every((field: unknown): boolean => typeof field === 'string');
|
||||
].every((field: unknown): boolean => typeof field === 'string') &&
|
||||
(payload['threadId'] === undefined || typeof payload['threadId'] === 'string') &&
|
||||
(payload['attachments'] === undefined ||
|
||||
hasValidAttachmentArray(payload['attachments'], isDiscordAttachment))
|
||||
);
|
||||
}
|
||||
|
||||
function isChannelAttachment(value: unknown): value is ChannelAttachmentDto {
|
||||
if (typeof value !== 'object' || value === null) return false;
|
||||
const attachment = value as Partial<ChannelAttachmentDto>;
|
||||
return (
|
||||
typeof attachment.id === 'string' &&
|
||||
typeof attachment.name === 'string' &&
|
||||
typeof attachment.url === 'string' &&
|
||||
(attachment.mimeType === null ||
|
||||
(typeof attachment.mimeType === 'string' &&
|
||||
attachment.mimeType.length <= MAX_ATTACHMENT_MIME_LENGTH)) &&
|
||||
(attachment.sizeBytes === undefined || typeof attachment.sizeBytes === 'number') &&
|
||||
hasValidAttachmentBounds(attachment as ChannelAttachmentDto)
|
||||
);
|
||||
}
|
||||
|
||||
function isChatSocketMessage(value: unknown): value is ChatSocketMessageDto {
|
||||
if (typeof value !== 'object' || value === null) return false;
|
||||
const payload = value as { content?: unknown; conversationId?: unknown };
|
||||
const payload = value as {
|
||||
content?: unknown;
|
||||
conversationId?: unknown;
|
||||
attachments?: unknown;
|
||||
};
|
||||
return (
|
||||
typeof payload.content === 'string' &&
|
||||
(payload.conversationId === undefined || typeof payload.conversationId === 'string')
|
||||
(payload.conversationId === undefined || typeof payload.conversationId === 'string') &&
|
||||
(payload.attachments === undefined ||
|
||||
hasValidAttachmentArray(payload.attachments, isChannelAttachment))
|
||||
);
|
||||
}
|
||||
|
||||
@@ -174,20 +265,24 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
|
||||
handleDisconnect(client: Socket): void {
|
||||
this.logger.log(`Client disconnected: ${client.id}`);
|
||||
const session = this.clientSessions.get(client.id);
|
||||
if (session) {
|
||||
for (const [key, session] of this.clientSessions) {
|
||||
if (session.clientId !== client.id) continue;
|
||||
session.cleanup();
|
||||
this.agentService.removeChannel(
|
||||
session.conversationId,
|
||||
`websocket:${client.id}`,
|
||||
session.scope,
|
||||
);
|
||||
this.clientSessions.delete(client.id);
|
||||
this.clientSessions.delete(key);
|
||||
this.textEgressBuffers.delete(key);
|
||||
this.thinkingEgressBuffers.delete(key);
|
||||
this.overflowedEgress.delete(`${key}:agent:text`);
|
||||
this.overflowedEgress.delete(`${key}:agent:thinking`);
|
||||
}
|
||||
this.textEgressBuffers.delete(client.id);
|
||||
this.thinkingEgressBuffers.delete(client.id);
|
||||
this.overflowedEgress.delete(this.egressKey(client, 'agent:text'));
|
||||
this.overflowedEgress.delete(this.egressKey(client, 'agent:thinking'));
|
||||
}
|
||||
|
||||
private clientConversationKey(client: Pick<Socket, 'id'>, conversationId: string): string {
|
||||
return `${client.id}\u0000${conversationId}`;
|
||||
}
|
||||
|
||||
private getClientScope(client: Socket): ActorTenantScope | null {
|
||||
@@ -218,7 +313,25 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
}
|
||||
discordIngress = this.resolveDiscordIngress(client, rawData);
|
||||
if (!discordIngress) return;
|
||||
data = { conversationId: discordIngress.conversationId, content: discordIngress.content };
|
||||
data = {
|
||||
conversationId: discordIngress.conversationId,
|
||||
content: discordIngress.content,
|
||||
...(discordIngress.attachments
|
||||
? {
|
||||
attachments: discordIngress.attachments.map(
|
||||
(attachment): ChannelAttachmentDto => ({
|
||||
id: attachment.id,
|
||||
name: attachment.name,
|
||||
url: attachment.url,
|
||||
mimeType: attachment.contentType,
|
||||
...(attachment.sizeBytes !== undefined
|
||||
? { sizeBytes: attachment.sizeBytes }
|
||||
: {}),
|
||||
}),
|
||||
),
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
} else {
|
||||
if (!isChatSocketMessage(rawData)) {
|
||||
this.logger.warn(`Rejected malformed chat message from ${client.id}`);
|
||||
@@ -227,6 +340,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
data = rawData;
|
||||
}
|
||||
const conversationId = data.conversationId ?? uuid();
|
||||
const clientConversationKey = this.clientConversationKey(client, conversationId);
|
||||
const discordServiceUserId = process.env['DISCORD_SERVICE_USER_ID'];
|
||||
if (discordIngress && !discordServiceUserId) {
|
||||
this.logger.warn(
|
||||
@@ -281,7 +395,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
this.logger.log(
|
||||
`Using /model override "${modelOverride}" for conversation=${conversationId}`,
|
||||
);
|
||||
} else if (!resolvedProvider && !resolvedModelId) {
|
||||
} else if (!resolvedProvider && !resolvedModelId && !discordIngress) {
|
||||
// No explicit provider/model from client — use routing engine (M4-012)
|
||||
try {
|
||||
const routingDecision = await this.routingEngine.resolve(data.content, userId);
|
||||
@@ -304,12 +418,24 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
}
|
||||
}
|
||||
|
||||
let resolvedAgentConfigId = data.agentId;
|
||||
if (discordIngress) {
|
||||
const binding = this.discordBindingFor(discordIngress, 'send');
|
||||
const agentConfig = binding
|
||||
? await this.brain.agents.findById(binding.agentConfigId)
|
||||
: undefined;
|
||||
if (!binding || !agentConfig || agentConfig.name !== binding.instanceId) {
|
||||
throw new Error('Configured Discord logical agent is not provisioned');
|
||||
}
|
||||
resolvedAgentConfigId = agentConfig.id;
|
||||
}
|
||||
|
||||
// M5-004: Use existingSessionId as sessionId when available (session reuse)
|
||||
const sessionIdToCreate = existingSessionId ?? conversationId;
|
||||
agentSession = await this.agentService.createSession(sessionIdToCreate, {
|
||||
provider: resolvedProvider,
|
||||
modelId: resolvedModelId,
|
||||
agentConfigId: data.agentId,
|
||||
agentConfigId: resolvedAgentConfigId,
|
||||
userId,
|
||||
tenantId: scope.tenantId,
|
||||
conversationHistory: conversationHistory.length > 0 ? conversationHistory : undefined,
|
||||
@@ -360,6 +486,17 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
discordUserId: discordIngress?.userId,
|
||||
}
|
||||
: {}),
|
||||
...(data.attachments && data.attachments.length > 0
|
||||
? {
|
||||
channelAttachments: data.attachments.map(
|
||||
(attachment): ChannelAttachmentDto => ({
|
||||
...attachment,
|
||||
name: redactSensitiveContent(attachment.name).content,
|
||||
url: redactSensitiveContent(attachment.url).content,
|
||||
}),
|
||||
),
|
||||
}
|
||||
: {}),
|
||||
classifications: redactSensitiveContent(data.content).classifications,
|
||||
},
|
||||
},
|
||||
@@ -374,7 +511,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
}
|
||||
|
||||
// Always clean up previous listener to prevent leak
|
||||
const existing = this.clientSessions.get(client.id);
|
||||
const existing = this.clientSessions.get(clientConversationKey);
|
||||
if (existing) {
|
||||
existing.cleanup();
|
||||
}
|
||||
@@ -389,10 +526,11 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
);
|
||||
|
||||
// Preserve routing decision from the existing client session if we didn't get a new one
|
||||
const prevClientSession = this.clientSessions.get(client.id);
|
||||
const prevClientSession = this.clientSessions.get(clientConversationKey);
|
||||
const routingDecisionToStore = sessionRoutingDecision ?? prevClientSession?.lastRoutingDecision;
|
||||
|
||||
this.clientSessions.set(client.id, {
|
||||
this.clientSessions.set(clientConversationKey, {
|
||||
clientId: client.id,
|
||||
conversationId,
|
||||
cleanup,
|
||||
assistantText: '',
|
||||
@@ -438,7 +576,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
|
||||
// Dispatch to agent
|
||||
try {
|
||||
await this.agentService.prompt(conversationId, data.content, scope);
|
||||
await this.agentService.prompt(conversationId, data.content, scope, data.attachments);
|
||||
} catch (err) {
|
||||
this.logger.error(
|
||||
`Agent prompt failed for client=${client.id}, conversation=${conversationId}`,
|
||||
@@ -645,9 +783,9 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
};
|
||||
|
||||
// Emit to all clients currently subscribed to this conversation
|
||||
for (const [clientId, session] of this.clientSessions) {
|
||||
for (const session of this.clientSessions.values()) {
|
||||
if (session.conversationId === conversationId && this.scopesEqual(session.scope, scope)) {
|
||||
const socket = this.server.sockets.sockets.get(clientId);
|
||||
const socket = this.server.sockets.sockets.get(session.clientId);
|
||||
if (socket?.connected) {
|
||||
socket.emit('session:info', payload);
|
||||
}
|
||||
@@ -677,16 +815,10 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
!this.durableSessions
|
||||
)
|
||||
return;
|
||||
const binding = resolveDiscordInteractionBinding(
|
||||
parseDiscordInteractionBindings(process.env['DISCORD_INTERACTION_BINDINGS']),
|
||||
ingress.guildId,
|
||||
ingress.channelId,
|
||||
ingress.userId,
|
||||
'approve',
|
||||
);
|
||||
const binding = this.discordBindingFor(ingress, 'approve');
|
||||
const actorId = binding && resolveDiscordInteractionActorId(binding, ingress.userId);
|
||||
const agentName = process.env['MOSAIC_AGENT_NAME']?.trim();
|
||||
if (!actorId || !agentName || binding.instanceId !== agentName) {
|
||||
const agentName = binding?.instanceId;
|
||||
if (!actorId || !agentName) {
|
||||
this.logger.warn(
|
||||
`Rejected Discord approval without a matching runtime agent from ${client.id}`,
|
||||
);
|
||||
@@ -774,13 +906,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
const tenantId = process.env['DISCORD_SERVICE_TENANT_ID']?.trim();
|
||||
if (!ingress || !approvalRef || !tenantId || !this.runtimeRegistry || !this.durableSessions)
|
||||
return;
|
||||
const binding = resolveDiscordInteractionBinding(
|
||||
parseDiscordInteractionBindings(process.env['DISCORD_INTERACTION_BINDINGS']),
|
||||
ingress.guildId,
|
||||
ingress.channelId,
|
||||
ingress.userId,
|
||||
'stop',
|
||||
);
|
||||
const binding = this.discordBindingFor(ingress, 'stop');
|
||||
const actorId = binding && resolveDiscordInteractionActorId(binding, ingress.userId);
|
||||
if (!actorId) return;
|
||||
|
||||
@@ -854,17 +980,18 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
const binding = resolveDiscordInteractionBinding(
|
||||
parseDiscordInteractionBindings(process.env['DISCORD_INTERACTION_BINDINGS']),
|
||||
payload.guildId,
|
||||
payload.channelId,
|
||||
payload.userId,
|
||||
operation,
|
||||
);
|
||||
const binding = this.discordBindingFor(payload, operation);
|
||||
if (!binding) {
|
||||
this.logger.warn(`Rejected unpaired Discord ingress from ${client.id}`);
|
||||
return null;
|
||||
}
|
||||
const expectedConversationId = `${binding.instanceId}:discord:${payload.threadId ?? payload.channelId}`;
|
||||
if (payload.conversationId !== expectedConversationId) {
|
||||
this.logger.warn(
|
||||
`Rejected Discord ingress for a different logical agent from ${client.id}`,
|
||||
);
|
||||
return null;
|
||||
}
|
||||
} catch {
|
||||
this.logger.warn(
|
||||
`Rejected Discord ingress without valid binding configuration from ${client.id}`,
|
||||
@@ -880,6 +1007,19 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
return payload;
|
||||
}
|
||||
|
||||
private discordBindingFor(
|
||||
payload: DiscordIngressPayload,
|
||||
operation: 'send' | 'approve' | 'stop',
|
||||
) {
|
||||
return resolveDiscordInteractionBinding(
|
||||
parseDiscordInteractionBindings(process.env['DISCORD_INTERACTION_BINDINGS']),
|
||||
payload.guildId,
|
||||
payload.channelId,
|
||||
payload.userId,
|
||||
operation,
|
||||
);
|
||||
}
|
||||
|
||||
private readDiscordAllowlist(name: string): string[] {
|
||||
return (process.env[name] ?? '')
|
||||
.split(',')
|
||||
@@ -958,11 +1098,15 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
const messages = await this.brain.conversations.findMessages(conversationId, userId);
|
||||
if (messages.length === 0) return [];
|
||||
|
||||
return messages.map((msg) => ({
|
||||
return messages.map((msg) => {
|
||||
const attachments = this.persistedChannelAttachments(msg.metadata);
|
||||
return {
|
||||
role: msg.role as 'user' | 'assistant' | 'system',
|
||||
content: msg.content,
|
||||
createdAt: msg.createdAt,
|
||||
}));
|
||||
...(attachments ? { attachments } : {}),
|
||||
};
|
||||
});
|
||||
} catch (err) {
|
||||
this.logger.error(
|
||||
`Failed to load conversation history for conversation=${conversationId}`,
|
||||
@@ -972,6 +1116,14 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
}
|
||||
}
|
||||
|
||||
private persistedChannelAttachments(metadata: unknown): readonly ChannelAttachmentDto[] | null {
|
||||
if (typeof metadata !== 'object' || metadata === null) return null;
|
||||
const attachments = (metadata as { channelAttachments?: unknown }).channelAttachments;
|
||||
return hasValidAttachmentArray(attachments, isChannelAttachment)
|
||||
? (attachments as readonly ChannelAttachmentDto[])
|
||||
: null;
|
||||
}
|
||||
|
||||
private appendAndFlushRedactedEgress(
|
||||
client: Socket,
|
||||
conversationId: string,
|
||||
@@ -979,18 +1131,19 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
buffers: Map<string, string>,
|
||||
delta: string,
|
||||
): void {
|
||||
const key = this.egressKey(client, eventName);
|
||||
const sessionKey = this.clientConversationKey(client, conversationId);
|
||||
const key = this.egressKey(client, conversationId, eventName);
|
||||
if (this.overflowedEgress.has(key)) return;
|
||||
|
||||
const buffered = `${buffers.get(client.id) ?? ''}${delta}`;
|
||||
const buffered = `${buffers.get(sessionKey) ?? ''}${delta}`;
|
||||
if (buffered.length > MAX_REDACTION_BUFFER_LENGTH) {
|
||||
buffers.delete(client.id);
|
||||
buffers.delete(sessionKey);
|
||||
this.overflowedEgress.add(key);
|
||||
client.emit(eventName, { conversationId, text: '[REDACTED_STREAM_OVERFLOW]' });
|
||||
return;
|
||||
}
|
||||
|
||||
buffers.set(client.id, buffered);
|
||||
buffers.set(sessionKey, buffered);
|
||||
this.flushRedactedEgress(client, conversationId, eventName, buffers, false);
|
||||
}
|
||||
|
||||
@@ -1006,21 +1159,22 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
buffers: Map<string, string>,
|
||||
final: boolean,
|
||||
): void {
|
||||
const key = this.egressKey(client, eventName);
|
||||
const sessionKey = this.clientConversationKey(client, conversationId);
|
||||
const key = this.egressKey(client, conversationId, eventName);
|
||||
if (this.overflowedEgress.has(key)) {
|
||||
if (final) this.overflowedEgress.delete(key);
|
||||
return;
|
||||
}
|
||||
|
||||
const buffered = buffers.get(client.id) ?? '';
|
||||
const buffered = buffers.get(sessionKey) ?? '';
|
||||
const releaseLength = final ? buffered.length : this.safeRedactionPrefixLength(buffered);
|
||||
const released = buffered.slice(0, releaseLength);
|
||||
const pending = buffered.slice(releaseLength);
|
||||
|
||||
if (pending) {
|
||||
buffers.set(client.id, pending);
|
||||
buffers.set(sessionKey, pending);
|
||||
} else {
|
||||
buffers.delete(client.id);
|
||||
buffers.delete(sessionKey);
|
||||
}
|
||||
|
||||
if (released) {
|
||||
@@ -1089,8 +1243,12 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
return retainedFrom;
|
||||
}
|
||||
|
||||
private egressKey(client: Socket, eventName: 'agent:text' | 'agent:thinking'): string {
|
||||
return `${client.id}:${eventName}`;
|
||||
private egressKey(
|
||||
client: Socket,
|
||||
conversationId: string,
|
||||
eventName: 'agent:text' | 'agent:thinking',
|
||||
): string {
|
||||
return `${this.clientConversationKey(client, conversationId)}:${eventName}`;
|
||||
}
|
||||
|
||||
private relayEvent(client: Socket, conversationId: string, event: AgentSessionEvent): void {
|
||||
@@ -1101,26 +1259,27 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
return;
|
||||
}
|
||||
|
||||
const sessionKey = this.clientConversationKey(client, conversationId);
|
||||
switch (event.type) {
|
||||
case 'agent_start': {
|
||||
// Reset accumulation buffers for the new turn
|
||||
const cs = this.clientSessions.get(client.id);
|
||||
const cs = this.clientSessions.get(sessionKey);
|
||||
if (cs) {
|
||||
cs.assistantText = '';
|
||||
cs.toolCalls = [];
|
||||
cs.pendingToolCalls.clear();
|
||||
}
|
||||
this.textEgressBuffers.set(client.id, '');
|
||||
this.thinkingEgressBuffers.set(client.id, '');
|
||||
this.overflowedEgress.delete(this.egressKey(client, 'agent:text'));
|
||||
this.overflowedEgress.delete(this.egressKey(client, 'agent:thinking'));
|
||||
this.textEgressBuffers.set(sessionKey, '');
|
||||
this.thinkingEgressBuffers.set(sessionKey, '');
|
||||
this.overflowedEgress.delete(this.egressKey(client, conversationId, 'agent:text'));
|
||||
this.overflowedEgress.delete(this.egressKey(client, conversationId, 'agent:thinking'));
|
||||
client.emit('agent:start', { conversationId });
|
||||
break;
|
||||
}
|
||||
|
||||
case 'agent_end': {
|
||||
// Gather usage stats from the Pi session
|
||||
const activeClientSession = this.clientSessions.get(client.id);
|
||||
const activeClientSession = this.clientSessions.get(sessionKey);
|
||||
const agentSession = activeClientSession
|
||||
? this.agentService.getSession(conversationId, activeClientSession.scope)
|
||||
: undefined;
|
||||
@@ -1173,7 +1332,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
}
|
||||
|
||||
// Persist the assistant message with metadata
|
||||
const cs = this.clientSessions.get(client.id);
|
||||
const cs = this.clientSessions.get(sessionKey);
|
||||
const userId = (client.data.user as { id: string } | undefined)?.id;
|
||||
if (cs && userId && cs.assistantText.trim().length > 0) {
|
||||
const metadata: Record<string, unknown> = {
|
||||
@@ -1225,7 +1384,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
const assistantEvent = event.assistantMessageEvent;
|
||||
if (assistantEvent.type === 'text_delta') {
|
||||
// Keep raw stream material in memory only; persist and emit only redacted text.
|
||||
const cs = this.clientSessions.get(client.id);
|
||||
const cs = this.clientSessions.get(sessionKey);
|
||||
if (cs) {
|
||||
cs.assistantText += assistantEvent.delta;
|
||||
}
|
||||
@@ -1250,7 +1409,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
|
||||
case 'tool_execution_start': {
|
||||
// Track pending tool call for later recording
|
||||
const cs = this.clientSessions.get(client.id);
|
||||
const cs = this.clientSessions.get(sessionKey);
|
||||
if (cs) {
|
||||
cs.pendingToolCalls.set(event.toolCallId, {
|
||||
toolName: event.toolName,
|
||||
@@ -1267,7 +1426,7 @@ export class ChatGateway implements OnGatewayInit, OnGatewayConnection, OnGatewa
|
||||
|
||||
case 'tool_execution_end': {
|
||||
// Finalise tool call record
|
||||
const cs = this.clientSessions.get(client.id);
|
||||
const cs = this.clientSessions.get(sessionKey);
|
||||
if (cs) {
|
||||
const pending = cs.pendingToolCalls.get(event.toolCallId);
|
||||
cs.toolCalls.push({
|
||||
|
||||
@@ -24,6 +24,7 @@ const ENV_KEYS = [
|
||||
'DISCORD_ALLOWED_CHANNEL_IDS',
|
||||
'DISCORD_ALLOWED_USER_IDS',
|
||||
'MOSAIC_AGENT_NAME',
|
||||
'MOSAIC_AGENT_CONFIG_ID',
|
||||
] as const;
|
||||
const savedEnv = new Map<string, string | undefined>();
|
||||
|
||||
@@ -33,12 +34,14 @@ function configureDiscordEnv(role: 'admin' | 'member' = 'admin'): void {
|
||||
process.env['DISCORD_SERVICE_USER_ID'] = 'discord-service';
|
||||
process.env['DISCORD_SERVICE_TENANT_ID'] = 'tenant-discord';
|
||||
process.env['MOSAIC_AGENT_NAME'] = 'Nova';
|
||||
process.env['MOSAIC_AGENT_CONFIG_ID'] = 'agent-config-nova';
|
||||
process.env['DISCORD_ALLOWED_GUILD_IDS'] = 'guild-001';
|
||||
process.env['DISCORD_ALLOWED_CHANNEL_IDS'] = 'channel-001';
|
||||
process.env['DISCORD_ALLOWED_USER_IDS'] = 'user-001';
|
||||
process.env['DISCORD_INTERACTION_BINDINGS'] = JSON.stringify([
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
pairedUsers: {
|
||||
@@ -141,7 +144,7 @@ function createPayload(overrides: Partial<DiscordIngressPayload> = {}): DiscordI
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
userId: 'user-001',
|
||||
conversationId: 'discord-channel-001',
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
content: 'hello Tess',
|
||||
...overrides,
|
||||
};
|
||||
@@ -153,6 +156,7 @@ describe('Discord ingress security', () => {
|
||||
JSON.stringify([
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
pairedUsers: { 'user-001': 'admin' },
|
||||
@@ -170,6 +174,7 @@ describe('Discord ingress security', () => {
|
||||
[
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
pairedUsers: { 'user-001': { role: 'operator', mosaicUserId: 'mosaic-operator-001' } },
|
||||
@@ -307,27 +312,12 @@ describe('Discord ingress security', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
'binding',
|
||||
() => {
|
||||
process.env['MOSAIC_AGENT_NAME'] = 'Other';
|
||||
},
|
||||
],
|
||||
[
|
||||
'durable session',
|
||||
(durable: { getSnapshot: ReturnType<typeof vi.fn> }) => {
|
||||
it('rejects approval when the durable session targets a different logical agent', async () => {
|
||||
configureDiscordEnv();
|
||||
const { gateway, client, durable } = discordGateway('admin');
|
||||
durable.getSnapshot.mockResolvedValueOnce({
|
||||
identity: { agentName: 'Other', providerId: 'fleet', runtimeSessionId: 'runtime-1' },
|
||||
});
|
||||
},
|
||||
],
|
||||
])(
|
||||
'rejects approval when the %s targets a different runtime agent',
|
||||
async (_source, configure) => {
|
||||
configureDiscordEnv();
|
||||
const { gateway, client, durable } = discordGateway('admin');
|
||||
configure(durable);
|
||||
|
||||
await gateway.handleDiscordApproval(
|
||||
client as never,
|
||||
@@ -340,8 +330,28 @@ describe('Discord ingress security', () => {
|
||||
approvalId: undefined,
|
||||
expiresAt: undefined,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
it('rejects privileged envelopes with a forged current conversation route', async () => {
|
||||
configureDiscordEnv();
|
||||
const { gateway, client } = discordGateway('admin');
|
||||
|
||||
await gateway.handleDiscordApproval(
|
||||
client as never,
|
||||
ingressEnvelope('/approve', 'forged-approval-route', {
|
||||
conversationId: 'Nova:discord:other-channel',
|
||||
}),
|
||||
);
|
||||
await gateway.handleDiscordStop(
|
||||
client as never,
|
||||
ingressEnvelope('/stop forged', 'forged-stop-route', {
|
||||
conversationId: 'Nova:discord:other-channel',
|
||||
}),
|
||||
);
|
||||
|
||||
expect(client.emit).not.toHaveBeenCalledWith('discord:approval', expect.anything());
|
||||
expect(client.emit).not.toHaveBeenCalledWith('discord:stop', expect.anything());
|
||||
});
|
||||
|
||||
it('rejects unpaired and non-admin Discord users for approval and stop', async () => {
|
||||
configureDiscordEnv();
|
||||
@@ -398,6 +408,267 @@ describe('Discord ingress security', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'https://user:password@cdn.example.test/diagram.png',
|
||||
'https://cdn.example.test/diagram.png?token=secret',
|
||||
'https://cdn.example.test/diagram.png?X-Amz-Signature=secret',
|
||||
'https://cdn.example.test/diagram.png?auth=secret',
|
||||
'https://cdn.example.test/diagram.png?hm=secret',
|
||||
])('rejects credential-bearing attachment URLs before gateway dispatch', async (url) => {
|
||||
configureDiscordEnv();
|
||||
const { gateway, client } = discordGateway('admin');
|
||||
|
||||
await gateway.handleMessage(
|
||||
client as never,
|
||||
ingressEnvelope('', `credential-url-${url.length}`, {
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
attachments: [
|
||||
{ id: 'attachment-credential', name: 'diagram.png', url, contentType: 'image/png' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(client.emit).not.toHaveBeenCalledWith('message:ack', expect.anything());
|
||||
});
|
||||
|
||||
it("selects each binding's trusted logical-agent config when creating Discord sessions", async () => {
|
||||
configureDiscordEnv();
|
||||
process.env['DISCORD_ALLOWED_CHANNEL_IDS'] = 'channel-001,channel-002';
|
||||
process.env['DISCORD_INTERACTION_BINDINGS'] = JSON.stringify([
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
pairedUsers: {
|
||||
'user-001': { role: 'operator', mosaicUserId: 'mosaic-operator-001' },
|
||||
},
|
||||
},
|
||||
{
|
||||
instanceId: 'Orion',
|
||||
agentConfigId: 'agent-config-orion',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-002',
|
||||
pairedUsers: {
|
||||
'user-001': { role: 'operator', mosaicUserId: 'mosaic-operator-001' },
|
||||
},
|
||||
},
|
||||
]);
|
||||
const session = {
|
||||
provider: 'configured-provider',
|
||||
modelId: 'configured-model',
|
||||
piSession: {
|
||||
thinkingLevel: 'medium',
|
||||
getAvailableThinkingLevels: (): string[] => ['medium'],
|
||||
},
|
||||
};
|
||||
const createSession = vi.fn().mockResolvedValue(session);
|
||||
const agentService = {
|
||||
getSession: vi.fn().mockReturnValue(undefined),
|
||||
createSession,
|
||||
recordMessage: vi.fn(),
|
||||
onEvent: vi.fn().mockReturnValue((): void => undefined),
|
||||
addChannel: vi.fn(),
|
||||
prompt: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const brain = {
|
||||
agents: {
|
||||
findById: vi.fn((id: string) =>
|
||||
Promise.resolve({
|
||||
id,
|
||||
name: id === 'agent-config-orion' ? 'Orion' : 'Nova',
|
||||
}),
|
||||
),
|
||||
},
|
||||
conversations: {
|
||||
findById: vi.fn().mockResolvedValue({ id: 'Nova:discord:channel-001' }),
|
||||
findMessages: vi.fn().mockResolvedValue([]),
|
||||
create: vi.fn().mockResolvedValue(undefined),
|
||||
update: vi.fn().mockResolvedValue(undefined),
|
||||
addMessage: vi.fn().mockResolvedValue(undefined),
|
||||
},
|
||||
};
|
||||
const routingEngine = { resolve: vi.fn() };
|
||||
const gateway = new ChatGateway(
|
||||
agentService as never,
|
||||
{} as never,
|
||||
brain as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
routingEngine as never,
|
||||
);
|
||||
const client = {
|
||||
id: 'discord-client-new-session',
|
||||
data: { discordService: true },
|
||||
emit: vi.fn(),
|
||||
};
|
||||
|
||||
await gateway.handleMessage(
|
||||
client as never,
|
||||
ingressEnvelope('start configured session', 'configured-session-001', {
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
}),
|
||||
);
|
||||
|
||||
await gateway.handleMessage(
|
||||
client as never,
|
||||
ingressEnvelope('start second configured session', 'configured-session-002', {
|
||||
channelId: 'channel-002',
|
||||
conversationId: 'Orion:discord:channel-002',
|
||||
}),
|
||||
);
|
||||
|
||||
expect(createSession).toHaveBeenCalledWith(
|
||||
'Nova:discord:channel-001',
|
||||
expect.objectContaining({
|
||||
agentConfigId: 'agent-config-nova',
|
||||
userId: 'discord-service',
|
||||
tenantId: 'tenant-discord',
|
||||
}),
|
||||
);
|
||||
expect(createSession).toHaveBeenCalledWith(
|
||||
'Orion:discord:channel-002',
|
||||
expect.objectContaining({ agentConfigId: 'agent-config-orion' }),
|
||||
);
|
||||
expect(routingEngine.resolve).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('retains validated persisted attachments in resumed conversation history', async () => {
|
||||
const attachment = {
|
||||
id: 'attachment-history',
|
||||
name: 'diagram.png',
|
||||
url: 'https://cdn.example.test/diagram.png',
|
||||
mimeType: 'image/png',
|
||||
sizeBytes: 4_096,
|
||||
};
|
||||
const gateway = new ChatGateway(
|
||||
{} as never,
|
||||
{} as never,
|
||||
{
|
||||
conversations: {
|
||||
findMessages: vi.fn().mockResolvedValue([
|
||||
{
|
||||
role: 'user',
|
||||
content: '',
|
||||
createdAt: new Date('2026-07-14T12:00:00.000Z'),
|
||||
metadata: { channelAttachments: [attachment] },
|
||||
},
|
||||
]),
|
||||
},
|
||||
} as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
) as unknown as {
|
||||
loadConversationHistory(
|
||||
conversationId: string,
|
||||
userId: string,
|
||||
): Promise<Array<{ attachments?: readonly (typeof attachment)[] }>>;
|
||||
};
|
||||
|
||||
await expect(
|
||||
gateway.loadConversationHistory('Nova:discord:channel-001', 'discord-service'),
|
||||
).resolves.toEqual([expect.objectContaining({ attachments: [attachment] })]);
|
||||
});
|
||||
|
||||
it('rejects malformed signed attachment payloads before gateway dispatch', async () => {
|
||||
configureDiscordEnv();
|
||||
const { gateway, client } = discordGateway('admin');
|
||||
const malformedPayload: Record<string, unknown> = {
|
||||
...createPayload({
|
||||
messageId: 'malformed-attachments-001',
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
}),
|
||||
attachments: { id: 'not-an-array' },
|
||||
};
|
||||
const envelope = createDiscordIngressEnvelope(
|
||||
malformedPayload as unknown as DiscordIngressPayload,
|
||||
SERVICE_TOKEN,
|
||||
);
|
||||
|
||||
await gateway.handleMessage(client as never, envelope);
|
||||
|
||||
expect(client.emit).not.toHaveBeenCalledWith('message:ack', expect.anything());
|
||||
});
|
||||
|
||||
it('preserves authenticated attachment metadata through persistence and agent dispatch', async () => {
|
||||
configureDiscordEnv();
|
||||
const prompt = vi.fn().mockResolvedValue(undefined);
|
||||
const addMessage = vi.fn().mockResolvedValue(undefined);
|
||||
const session = {
|
||||
provider: 'test-provider',
|
||||
modelId: 'test-model',
|
||||
piSession: {
|
||||
thinkingLevel: 'medium',
|
||||
getAvailableThinkingLevels: (): string[] => ['medium'],
|
||||
},
|
||||
};
|
||||
const agentService = {
|
||||
getSession: vi.fn().mockReturnValue(session),
|
||||
recordMessage: vi.fn(),
|
||||
onEvent: vi.fn().mockReturnValue((): void => undefined),
|
||||
addChannel: vi.fn(),
|
||||
prompt,
|
||||
};
|
||||
const brain = {
|
||||
conversations: {
|
||||
findById: vi.fn().mockResolvedValue({ id: 'Nova:discord:channel-001' }),
|
||||
create: vi.fn().mockResolvedValue(undefined),
|
||||
update: vi.fn().mockResolvedValue(undefined),
|
||||
addMessage,
|
||||
},
|
||||
};
|
||||
const gateway = new ChatGateway(
|
||||
agentService as never,
|
||||
{} as never,
|
||||
brain as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
);
|
||||
const client = {
|
||||
id: 'discord-client-001',
|
||||
data: { discordService: true },
|
||||
emit: vi.fn(),
|
||||
};
|
||||
const attachment = {
|
||||
id: 'attachment-001',
|
||||
name: 'diagram.png',
|
||||
url: 'https://cdn.example.test/diagram.png',
|
||||
contentType: 'image/png',
|
||||
sizeBytes: 4_096,
|
||||
};
|
||||
|
||||
await gateway.handleMessage(
|
||||
client as never,
|
||||
ingressEnvelope('', 'attachment-message-001', {
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
attachments: [attachment],
|
||||
}),
|
||||
);
|
||||
|
||||
const expectedAttachment = {
|
||||
id: attachment.id,
|
||||
name: attachment.name,
|
||||
url: attachment.url,
|
||||
mimeType: attachment.contentType,
|
||||
sizeBytes: attachment.sizeBytes,
|
||||
};
|
||||
expect(prompt).toHaveBeenCalledWith(
|
||||
'Nova:discord:channel-001',
|
||||
'',
|
||||
{ userId: 'discord-service', tenantId: 'tenant-discord' },
|
||||
[expectedAttachment],
|
||||
);
|
||||
expect(addMessage).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
conversationId: 'Nova:discord:channel-001',
|
||||
metadata: expect.objectContaining({ channelAttachments: [expectedAttachment] }),
|
||||
}),
|
||||
'discord-service',
|
||||
);
|
||||
});
|
||||
|
||||
it('accepts a thread message through its allowed bound parent channel', () => {
|
||||
const emitted = vi.fn();
|
||||
const plugin = new DiscordPlugin({
|
||||
@@ -410,6 +681,7 @@ describe('Discord ingress security', () => {
|
||||
interactionBindings: [
|
||||
{
|
||||
instanceId: 'Nova',
|
||||
agentConfigId: 'agent-config-nova',
|
||||
guildId: 'guild-001',
|
||||
channelId: 'channel-001',
|
||||
pairedUsers: { 'user-001': { role: 'operator', mosaicUserId: 'mosaic-operator-001' } },
|
||||
|
||||
@@ -61,6 +61,16 @@ function requiredDiscordAllowlist(name: string): string[] {
|
||||
return value;
|
||||
}
|
||||
|
||||
function optionalPositiveInteger(name: string): number | undefined {
|
||||
const raw = process.env[name];
|
||||
if (raw === undefined) return undefined;
|
||||
const value = Number(raw);
|
||||
if (!Number.isInteger(value) || value <= 0) {
|
||||
throw new Error(`${name} must be a positive integer when configured`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function createPluginRegistry(): IChannelPlugin[] {
|
||||
const plugins: IChannelPlugin[] = [];
|
||||
const discordToken = process.env['DISCORD_BOT_TOKEN'];
|
||||
@@ -82,6 +92,10 @@ function createPluginRegistry(): IChannelPlugin[] {
|
||||
guildId: discordGuildId,
|
||||
gatewayUrl: discordGatewayUrl,
|
||||
serviceToken: discordServiceToken,
|
||||
messageRateLimitPerMinute: optionalPositiveInteger(
|
||||
'DISCORD_MESSAGE_RATE_LIMIT_PER_MINUTE',
|
||||
),
|
||||
threadRateLimitPerMinute: optionalPositiveInteger('DISCORD_THREAD_RATE_LIMIT_PER_MINUTE'),
|
||||
allowedGuildIds: requiredDiscordAllowlist('DISCORD_ALLOWED_GUILD_IDS'),
|
||||
allowedChannelIds: requiredDiscordAllowlist('DISCORD_ALLOWED_CHANNEL_IDS'),
|
||||
allowedUserIds: requiredDiscordAllowlist('DISCORD_ALLOWED_USER_IDS'),
|
||||
|
||||
@@ -149,15 +149,9 @@ for any `<Image>` components added in the future.
|
||||
|
||||
---
|
||||
|
||||
## How to Apply
|
||||
## Held future procedure
|
||||
|
||||
```bash
|
||||
# Run the DB migration (requires a live DB)
|
||||
pnpm --filter @mosaicstack/db exec drizzle-kit migrate
|
||||
|
||||
# Or, in Docker/Swarm — migrations run automatically on gateway startup
|
||||
# via runMigrations() in packages/db/src/migrate.ts
|
||||
```
|
||||
This report is non-operative evidence, not a current runbook. Until **KBN-101-00, KBN-101-03, and KBN-101-05** land, do not execute a PostgreSQL runner from this checkout. The approved future procedure is exactly: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Deployment will supply the reviewed runner, migration-only credentials, and TLS material; Gateway startup only verifies readiness.
|
||||
|
||||
---
|
||||
|
||||
|
||||
223
docs/PRD.md
223
docs/PRD.md
@@ -125,6 +125,105 @@ are defined in [docs/TASKS.md](./TASKS.md) and must remain one card/one PR.
|
||||
|
||||
---
|
||||
|
||||
## Exact Cross-Harness Fleet Communications Contract (#766)
|
||||
|
||||
### Problem and objective
|
||||
|
||||
Fleet runtime contracts currently combine exact peer rows with generic operational metavariables and
|
||||
independently parsed roster data. Non-Claude harnesses can mistake those metavariables for values to
|
||||
infer, producing incorrect host, session, socket, or helper targets. The objective is one
|
||||
roster-resolved communications contract that every supported harness receives unchanged.
|
||||
|
||||
### Normative requirements
|
||||
|
||||
1. `FCOM-REQ-01`: Fleet commands and runtime composition SHALL use one shared v1 roster structural
|
||||
resolver. A second lenient communications parser is forbidden.
|
||||
2. `FCOM-REQ-02`: The composed contract SHALL render the local roster member's authoritative host,
|
||||
exact agent/session name, resolved tmux socket, exact helper path, and deterministic communications
|
||||
generation.
|
||||
3. `FCOM-REQ-03`: Every known peer SHALL have one exact executable command. Same-host commands SHALL
|
||||
omit `-H`; cross-host commands SHALL use only that peer's explicit roster `ssh` target; the one
|
||||
supported fleet-wide named socket SHALL use `-L` with its exact value. A per-agent socket declaration
|
||||
must equal that fleet-wide value; unsupported independent sockets and missing cross-host SSH data SHALL
|
||||
fail closed.
|
||||
4. `FCOM-REQ-04`: Operational fleet examples SHALL not contain unresolved host, session, socket, or
|
||||
helper-path metavariables. Agents SHALL select an exact rendered peer row and SHALL NOT infer,
|
||||
substitute, or fuzzy-match targeting values.
|
||||
5. `FCOM-REQ-05`: An unknown local member or requested peer SHALL fail closed with exact-name discovery
|
||||
guidance. Runtime composition SHALL not silently omit a requested fleet member's communications
|
||||
contract.
|
||||
6. `FCOM-REQ-06`: Claude Code, Codex, OpenCode, and Pi SHALL receive equivalent authoritative
|
||||
communications data through the common runtime composer.
|
||||
7. `FCOM-REQ-07`: Tests SHALL prove the contract from framework-source `TOOLS.md`, through a fresh
|
||||
installed `TOOLS.md`, to final runtime composition and helper executability. User-owned installed
|
||||
`TOOLS.md` content SHALL remain preserved.
|
||||
8. `FCOM-REQ-08`: Stale installed or active composed context SHALL be reported with deterministic
|
||||
generation/repair/relaunch guidance. Currency requires the expected source and installed contract
|
||||
marker/version plus bounded byte equality. The supported current-version repair SHALL run independently
|
||||
of package updates, preserve divergent `TOOLS.md` bytes in a digest-qualified no-clobber backup, restore
|
||||
a regular executable helper without following symlinks, and be idempotent. Detection and reporting SHALL
|
||||
NOT rewrite active context, restart a session, or mutate a live fleet.
|
||||
9. `FCOM-REQ-09`: The shared resolver SHALL preserve and strictly validate every schema-supported v1
|
||||
connector kind (`tmux`, `discord`, and `matrix`) from YAML and JSON. Every accepted snake/camel alias
|
||||
pair SHALL reject differing dual declarations and accept identical declarations. JSON roster fallback
|
||||
SHALL occur only when `roster.yaml` is absent; all other YAML access failures SHALL fail closed.
|
||||
10. `FCOM-REQ-10`: The communications generation SHALL cover the complete canonical rendered semantic
|
||||
contract, including identity, role/class, resolved host/socket/helper, peer metadata, and exact commands.
|
||||
Installed helpers SHALL be validated with no-follow filesystem inspection as regular executable files.
|
||||
Keep-mode reseed and relaunch discovery SHALL preserve and support both YAML and JSON rosters.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
1. `AC-FCOM-01`: Contract fixtures contain no unresolved operational targeting metavariables; local
|
||||
identity contains exact host/session/socket/helper values.
|
||||
2. `AC-FCOM-02`: Same-host, cross-host, named-socket, literal-default-socket, and missing-SSH tests prove
|
||||
exact targeting and fail-closed behavior.
|
||||
3. `AC-FCOM-03`: Unknown identities and peers report known exact names plus an exact self-scoped
|
||||
discovery command; no fuzzy session selection is emitted.
|
||||
4. `AC-FCOM-04`: Four-harness tests prove byte-equal authoritative communications sections.
|
||||
5. `AC-FCOM-05`: Source, fresh-install, preserved-custom-install, stale-installed, composed-generation,
|
||||
helper executable, agent-send socket isolation, and exact-target tests pass.
|
||||
6. `AC-FCOM-06`: Documentation defines non-mutating stale-context detection and operator-authorized,
|
||||
exact-agent relaunch; no implementation path performs automatic session mutation.
|
||||
7. `AC-FCOM-07`: YAML and JSON fixtures cover every connector kind; all snake/camel aliases cover
|
||||
identical acceptance and conflicting rejection; non-`ENOENT` YAML failures do not fall back.
|
||||
8. `AC-FCOM-08`: Missing, directory, symlink, and non-executable installed helpers fail closed. Explicit
|
||||
current-version repair proves partial-deletion recovery, digest-qualified backup collision safety,
|
||||
symlink-target safety, and repeated-run idempotence.
|
||||
9. `AC-FCOM-09`: Markerless-equal and wrong-version source/installed contracts are stale, and a rendered
|
||||
role/class change produces a different communications generation.
|
||||
|
||||
---
|
||||
|
||||
## KBN-101 Database Runtime/Migration Role Split (#771)
|
||||
|
||||
### Problem and objective
|
||||
|
||||
PostgreSQL Gateway/storage currently uses one `DATABASE_URL` for runtime queries and migrations. That makes the deployed application identity an owner and prevents certification that KBN immutable event, artifact, checkpoint, and evidence relations reject runtime `UPDATE`/`DELETE`. KBN-101 freezes a least-privilege runtime/migration split before KBN-100 schema work.
|
||||
|
||||
### Normative requirements
|
||||
|
||||
1. `K101-REQ-01`: `DATABASE_URL` SHALL be the non-owner PostgreSQL runtime connection and `DATABASE_MIGRATION_URL` SHALL be the migration-only owner/migrator connection. They are required respectively for runtime and the dedicated `mosaic-db-migrator --run|--verify` phase in `standalone`/`federated`; local PGlite is the explicit exception. The published `@mosaicstack/db` bin maps exactly `mosaic-db-migrator` to `./dist/cli.js`, its image entrypoint is exactly `mosaic-db-migrator`, accepts no URL/SQL/schema/role argv, and returns stable sanitized exits. Every current/future PostgreSQL DDL entrypoint SHALL route to that runner or be denied, and SHALL reject `DATABASE_URL`-only execution before connection/DDL. Data migration may connect only after the runner prepares and verifies the PostgreSQL target, through dedicated non-DDL `mosaic_data_importer` and exactly `--target-url-file /run/secrets/mosaic-migrate-target-url`, its fixed paired authenticated provider-version file `/run/secrets/mosaic-migrate-target-version`, plus `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`. KBN-101-05 obtains URL key `url` and version only from the same successful Vault KV-v2 response at `secret-{env}/mosaic-stack/database/importer` (`data.metadata.version`), renders them as one immutable generation into separate consumer copies, and never infers a provider version from DSN bytes. The trusted runner verifies TLS/identity/manifest, reads its fixed importer URL/version copies only for binding through safe no-follow fd checks, and signs a credential-free JCS/Ed25519 attestation using its runner-only fixed root-owned private-key file; no signing key reaches importer/runtime. The artifact binds secret version and SHA-256 of exact high-entropy credential-file bytes, canonical TLS host/port/database, CA/SPKI, PostgreSQL system identifier/database OID, importer role, manifest/schema fingerprints, producer invocation/build/image digest, issued/expires/nonce, and correlation. Before target connection the importer validates URL/version/attestation/public-key files, signature/key/expiry/replay/authenticated provider version/digest/generation/bindings and the importer-only CA at exact `DATABASE_TLS_CA_CERT_PATH`; after verified TLS and before DML it validates server/database/role/CA/schema identity, with same-fd/in-memory-byte TOCTOU protection, rotation/revocation, a privileged producer-only-to-importer-only artifact handoff controller that verifies/copies/fsyncs/atomically renames/seals before importer start, consumer isolation/no logging-oracle, and sanitized errors. Raw `--target-url`, `DATABASE_URL` fallback, runtime-owner use, missing/unsafe/substituted files, stale/replayed/tampered/wrong-key attestation, wrong binding, and DDL attempt fail before target connection/DDL; post-connect mismatch closes with zero DML/DDL. A reviewed finite classifier inventories executable current source/scripts/package bins, operator docs, deploy manifests, and exact normative contracts by path; active secure records pin both options/files, producer/key/bindings/tests, while normative contracts cannot mask instructions. Unknown active commands, duplicate-owner, ownerless, missing-path, and historical/status-only masking hits fail. `db:push` is forbidden outside an explicitly disposable local developer database and cannot accept a production-like URL.
|
||||
2. `K101-REQ-02`: Gateway runtime/replicas SHALL not execute migrations or DDL. The runner SHALL hold one `max:1` session and fixed two-int advisory namespace `1297044289` (`MOSA`), `1262636593` (`KBN1`) across preflight, reconciliation, migration, verification, and release. It SHALL compare the versioned canonical manifest v1 tuple (journal logical index/tag plus exact SQL-byte SHA-256) to the complete observed ledger mapping; count/set-only, timestamps, and physical insertion order are non-normative and insufficient.
|
||||
3. `K101-REQ-03`: PostgreSQL SHALL separate non-login platform database owner, non-login schema owner, dedicated `NOLOGIN SUPERUSER` `mosaic_extension_owner`, login migrator, dedicated login non-DDL data importer, non-login runtime capability, and login runtime roles. For PostgreSQL 17 + pgvector 0.8.2, `vector` is untrusted (`trusted` is absent and `relocatable=true`): only an externally controlled audited platform-bootstrap superuser session may `SET ROLE mosaic_extension_owner` for CREATE/UPDATE/SET SCHEMA, then `RESET ROLE`; the role has `rolcanlogin=false`, `rolsuper=true`, zero members, no runtime credential/Vault secret, and is never provided to app containers. It owns `mosaic_extensions`, fresh `vector`, and owner-bearing extension members, while `mosaic_schema_owner` receives only `USAGE` for type resolution and never ownership/`CREATE`/`ALTER`/`DROP`/member-change/default-privilege authority there. Superuser cannot be constrained by `GRANT`/`REVOKE`; this is identity/non-login/no-membership/external-control/audit isolation, not a false least-privilege claim. Extension operations require control-plane change, independent review, backup/rollback, maintenance window, and audit evidence. Managed targets that cannot establish this exact role are ineligible until an independently approved versioned provider-owned extension-owner profile exists; app/migrator ownership is never silently retained. Existing approved-owner extension relocation validates exact `pg_namespace.nspowner`, `pg_extension.extowner`, member ownership/schema/version, while legacy runtime-owned extension fails closed to a controlled shadow-database migration—never unsupported ownership alteration, catalog mutation, ownership adoption, or `DROP CASCADE`. Runtime, migrator, schema owner, importer, and all service roles must fail `SET ROLE`, catalog/direct `ALTER`/`UPDATE`/`DROP`/membership-change denial, role ownership, superuser/role-creation/schema-creation/TEMPORARY, unsafe membership, untrusted search path, missing grants, unauthenticated TLS, and immutable privilege drift checks. Application schema is fixed `mosaic` with exact `pg_catalog,mosaic` session path; historical public migrations remain byte-immutable legacy bootstrap only, every future Drizzle application declaration targets `mosaic`, and `vector` is explicitly qualified from non-writable `mosaic_extensions`. No config-derived SQL identifier is permitted.
|
||||
4. `K101-REQ-04`: `mosaicstack/stack` KBN-101-00 SHALL exclusively own `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, and bootstrap tests; KBN-101-05 SHALL exclusively own `tools/db/render-postgres-secrets.ts`, its tests, and current Compose/Portainer/two-gateway deployment declarations, consuming the versioned bootstrap interface without overlap. Environment IaC/Vault is named input and Mosaic deployment control plane/Jason is activation authority. Distinct runtime/migrator/importer URL, importer authenticated provider-version, DB-client CA, Gateway leaf, and PostgreSQL server key/certificate materials are provisioned before a production-like database starts. Importer and migrator have separate immutable URL/version copies at fixed `10002:10002`/`10003:10003` identities; runtime/unrelated containers receive neither importer material, attestation private key, or importer artifact. Runtime, migrator, and importer require their mounted CA plus `sslmode=verify-full`. Exact UID/GID/mode/rendering, service-DNS SANs, Vault/compose/Swarm consumer isolation, two-gateway pair ordering, server activation, pre-enforcement legacy-client drain and `hostssl` zero-plaintext-session proof, fresh/existing transition, CA-overlap rotation, TLS-only rollback, and standalone/federated/Swarm/two-gateway positive/negative TLS evidence are required. No application-generated production certificate or plaintext bootstrap exception is permitted.
|
||||
5. `K101-REQ-05`: KBN immutable relations SHALL permit the real runtime role INSERT/SELECT only and deny UPDATE/DELETE; parent retention remains RESTRICT/no-cascade. Role/password/Vault creation is external platform control, never application migration/source.
|
||||
6. `K101-REQ-06`: N-1 single-URL compatibility, rollout/rollback, Vault ownership/rotation/redaction, CI, installer, compose/Portainer, observability, and deployment handoffs SHALL be separately bounded one-card/one-PR work. Prepared slices remain inactive while current owner-runtime deployments stay N-1; Mosaic control plane/Jason alone authorizes one final atomic activation or rollback, with no force-on-red/bypass. KBN-101 planning itself SHALL not mutate production.
|
||||
7. `K101-REQ-07`: KBN-100 SHALL begin only after the KBN-101 foundation role/schema-boundary certificate; it SHALL rebase on that main head, restore generated Drizzle declaration/snapshot/journal consistency, and bound procedural immutable-table grant/trigger/backfill additions to its schema slice. KBN-101 real deployed-role immutable-operation certification SHALL complete after KBN-100 creates those relations and before KBN-105.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
1. `AC-K101-01`: DTO/command-matrix tests prove required modes, PGlite exception, `mosaic-db-migrator --help|--run|--verify`/stable exits/argv refusal, public-import negative, every finite classified DDL/static-bypass inventory path and both harness pairs reject `DATABASE_URL`-only before connection/DDL, no migration-to-runtime fallback, and `db:push` refusal outside an allowlisted disposable DB. Before inventory, ownership, or status masking, the semantic fixture fails README's exact former commented code-fence generic-wrapper form and the user guide's exact former executable generic-wrapper form; source-consistency proves current `packages/storage/src/cli.ts` directly `execSync`s `pnpm --filter @mosaicstack/db db:migrate` and no `mosaic-db-migrator` bin exists, so runner-delegation documentation fails. The active `docs/guides/migrate-tier.md` route is inventoried to KBN-101-07 and proves runner-produced `--target-url-file /run/secrets/mosaic-migrate-target-url`, fixed paired provider-version file, and `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`; runner-only signing/private-key isolation; Vault KV-v2 same-response version provenance, separate immutable generation mounts, importer CA, JCS/Ed25519 signature/key rotation/revocation, atomic artifact, expiry/replay, safe-fd secret-version/digest, canonical TLS/CA/server/database/role/manifest/schema bindings, dedicated non-DDL importer, consumer isolation/no log-oracle, and exact no-connection versus zero-DML rejection for missing/wrong/stale/replayed/tampered/wrong-key/substituted/generation-mismatched inputs. The full current non-normative docs inventory—including user guide, federation historical task/MILESTONES status, and non-operative SETUP—has an exact safe disposition. Scanner semantic checks reject automatic first-boot/startup extension/schema/migration wording, Compose-up-before-runner, init-script authority, production `.env`/monorepo auto-load/`EnvironmentFile=`/credential-export-or-argv/restart-as-secret-activation routes, and every unqualified operator-document `mosaic-db-migrator --run|--verify` hit regardless of named/normative/status classification. The exact former README/dev/deployment Compose-first sequences, former SETUP wording, exact former MILESTONES wording `pgvector extension installed + verified on startup`, former architecture-plan/PERFORMANCE/backlog runner routes, and any unqualified runner fixture fail before inventory masking. Only one `Held future procedure` Markdown section—bounded through the next equal-or-higher heading—may contain the explicit non-operative/no-current-command-authority form that names KBN-101-00/-03/-05 and preserves external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness; every runner hit outside that section fails. The README assertion for the checked-in direct CI `pnpm --filter @mosaicstack/db run db:migrate` with `DATABASE_URL` passes only as active legacy N-1, uncertified, non-authorizing-as-an-operator-route status against an isolated disposable CI database pending KBN-101-06 removal—not as an ordinary operator or approved DDL-authority route. Only local PGlite data-layer work or non-PostgreSQL Compose is current (Gateway/Web local startup is held pending daemon/inherited/project-DSN rejection).
|
||||
2. `AC-K101-02`: Fixed namespace lock contention/crash/readiness/non-interference and exact manifest-v1 reconciliation tests prove no replica race/runtime auto-migration and fail closed on every missing/unknown/duplicate/ambiguous/corrupt/stale ledger state.
|
||||
3. `AC-K101-03`: Actual PostgreSQL 17 + pgvector 0.8.2 control-file, catalog, Drizzle-generation, vector-query/operator, fresh/approved-owner/legacy-shadow/partial/resume/rollback/N-1, and real deployed-role tests prove `trusted` absent/untrusted plus relocatability, external-superuser `SET ROLE` create/update/`RESET ROLE` audit, exact `rolcanlogin=false`/`rolsuper=true`/zero-membership/no-runtime-secret state, platform/schema/extension-owner/migrator/importer/runtime separation, `pg_extension.extowner` plus owner-bearing extension-member/schema/version assertions, and runtime/migrator/schema-owner/importer/all-service-role `SET ROLE`/ALTER/DROP/member-update denial. They also prove `pg_catalog,mosaic` per-session pool safety, `mosaic_extensions` qualification, identifier injection denial, ownership/membership/ledger-read/TEMP/default grants, and unsafe privilege denial.
|
||||
4. `AC-K101-04`: Disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus for both pairs missing CA/wrong CA/wrong SAN/sslmode downgrade, server/Gateway key mode, UID/GID, secret-consumer isolation, and legacy-drain/`hostssl` negatives prove server bootstrap, ordering, and readiness; PGlite is expressly excluded from this PostgreSQL evidence.
|
||||
5. `AC-K101-05`: Real runtime-role evidence proves INSERT/SELECT succeeds and UPDATE/DELETE fails for every frozen immutable KBN relation.
|
||||
6. `AC-K101-06`: N-1/atomic activation/rollback, Vault/CA-overlap rotation/redaction, health/operator behavior, CI/deployment handoff, independent exact-head security review, and terminal-green CI evidence the foundation before KBN-100; after KBN-100, the real deployed-role immutable-operation certificate and Ultron approval release KBN-105.
|
||||
|
||||
**Normative implementation contract:** [`docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md`](./native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md). `ASSUMPTION:` existing `standalone` and `federated` are all PostgreSQL production-like modes; any new PostgreSQL tier inherits these requirements until an explicit versioned amendment.
|
||||
|
||||
---
|
||||
|
||||
## Tess Interaction Agent Workstream (TESS)
|
||||
|
||||
### Problem and Objective
|
||||
@@ -221,6 +320,100 @@ Delivery uses five gated milestones: runtime contracts/security; Pi service/stat
|
||||
|
||||
---
|
||||
|
||||
## Official Channel Plugin Workstream (#756)
|
||||
|
||||
### Problem and Objective
|
||||
|
||||
The Discord plugin currently couples Discord event handling, gateway bridging, and reply routing in one implementation and activates only on mentions. Mosaic needs an official channel adapter that behaves the same no matter whether the bound logical agent currently runs through Claude, Codex, Pi, OpenCode, or a future harness. The Discord connection and conversation address must remain stable while the gateway changes the runtime provider behind that logical session.
|
||||
|
||||
The objective is to make Discord the first implementation of a transport-neutral official channel contract, with explicit authorization and deterministic channel/thread routing that future Matrix, Slack, and other adapters can share.
|
||||
|
||||
### Scope
|
||||
|
||||
#### In Scope
|
||||
|
||||
1. `CHN-001`: Transport-neutral channel adapter, route, message, attachment, authorization-principal, response-target, and health contracts in `@mosaicstack/types`, including trusted per-binding logical-agent configuration selection.
|
||||
2. `CHN-002`: Stable channel conversation addresses based on logical agent plus channel/thread identity; harness, model, and runtime-provider IDs are forbidden from channel session keys.
|
||||
3. `DSC-001`: An authorized untagged message in a configured agent-bound channel routes to the agent and receives its response in that channel.
|
||||
4. `DSC-002`: A bot mention in a configured parent channel creates a Discord thread, or reuses the thread already attached to that same native message; the mentioned turn and subsequent thread turns route and respond in that thread.
|
||||
5. `DSC-003`: A message already inside an authorized thread inherits authorization from its configured parent and never attempts a nested thread.
|
||||
6. `DSC-004`: Guild, parent channel, user, pairing, and role authorization remains default-deny before thread creation or gateway dispatch.
|
||||
7. `DSC-005`: Discord service authentication, HMAC envelope integrity, replay protection, attachments, approvals, response chunking, and correlation behavior remain intact.
|
||||
8. `DSC-006`: The Discord adapter exposes lifecycle and health behavior through the shared channel contract without importing a harness SDK.
|
||||
|
||||
#### Out of Scope
|
||||
|
||||
1. The logical-agent lease, fencing epoch, execution grant, checkpoint, or cross-harness takeover implementation tracked by #754/#755.
|
||||
2. Dynamic Discord authorization administration in the web UI.
|
||||
3. Multi-guild tenant isolation, DMs, slash commands, voice, reactions, or production bot deployment.
|
||||
4. Implementing Matrix or Slack adapters in this slice.
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
1. **Security:** no thread or dispatch side effect occurs until guild, parent channel, user, pairing, role, and bounded per-user/channel rate checks pass; attachment metadata is shape- and size-bounded; credentials never enter source, messages, session keys, or logs.
|
||||
2. **Portability:** channel contracts and stable conversation IDs contain no Claude, Codex, Pi, OpenCode, model, process, or provider-specific field; each configuration-owned binding selects its trusted logical agent without changing the channel identity.
|
||||
3. **Reliability:** repeated messages for one channel/thread resolve the same conversation handle; reconnecting the adapter does not require a harness-specific rebinding.
|
||||
4. **Maintainability:** Discord-specific API translation stays in the Discord package; gateway and future adapters depend on transport-neutral contracts.
|
||||
5. **Observability:** thread creation or routing failure is reported without message content or credential material.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
1. `AC-CHN-01`: Contract and behavior tests prove the plugin route contains only logical agent plus channel/thread identity and produces the same stable conversation handle regardless of underlying harness selection.
|
||||
2. `AC-CHN-02`: A mentioned authorized parent-channel message creates a thread (or reuses its already-attached thread), dispatches to the thread conversation, and targets the response to that thread.
|
||||
3. `AC-CHN-03`: An untagged authorized parent-channel message dispatches to the parent conversation and targets the response to the parent channel.
|
||||
4. `AC-CHN-04`: Untagged follow-ups inside an authorized thread dispatch and respond in that same thread without creating a nested thread.
|
||||
5. `AC-CHN-05`: Unauthorized guilds, channels, users, unpaired users, insufficient roles, and rate-limited senders produce no thread and no gateway dispatch.
|
||||
6. `AC-CHN-06`: Shared channel contracts are exported from `@mosaicstack/types`, Discord implements the lifecycle/health seam, and no harness SDK is imported by the plugin.
|
||||
7. `AC-CHN-07`: Focused routing/auth tests, package tests, typecheck, lint, formatting, coverage, independent code/security review, and terminal-green CI pass.
|
||||
|
||||
### Constraints, Risks, and Assumptions
|
||||
|
||||
- Dependency: Mosaic gateway remains the policy, durable-session, audit, and runtime-provider boundary.
|
||||
- Constraint: This work must not modify orchestrator-to-Pi migration or #754/#755 lease/fencing files.
|
||||
- Risk: accepting untagged messages could create noisy or unintended agent input. Mitigation: only explicitly configured channels and paired, role-authorized users are accepted, with bounded per-user/channel message and thread rates.
|
||||
- Risk: Discord thread creation can fail because of channel permissions, archived state, or API rate limits. Mitigation: fail without dispatching a turn whose response destination cannot be honored, and emit sanitized diagnostics.
|
||||
- `ASSUMPTION:` Configured channels are dedicated agent interaction surfaces, so authorized untagged human messages are intentional agent input.
|
||||
- `ASSUMPTION:` Mention in a parent channel selects a public thread; messages already in a thread remain there because Discord has no nested threads.
|
||||
- `ASSUMPTION:` One Discord bot may serve multiple configuration-owned logical-agent bindings.
|
||||
- `ASSUMPTION:` Static allowlists and paired-user roles are the authorization administration surface for this slice.
|
||||
|
||||
### Testing and Delivery Intent
|
||||
|
||||
Use TDD for remote-ingress routing and permission boundaries. Required evidence includes parent-channel mention, untagged parent message, existing-thread follow-up, existing-thread mention, thread reuse, unauthorized side-effect denial, stable harness-neutral conversation identity, adapter health, and regression coverage for signed envelopes and approvals. Deliver through issue #756, a reviewed squash PR to `main`, terminal-green CI, and issue closure.
|
||||
|
||||
---
|
||||
|
||||
## Mos Runtime Portability Workstream (MOS-PORT)
|
||||
|
||||
### Problem and Objective
|
||||
|
||||
Mos is currently identified partly by a harness-native session and communication process. Replacement/rebinding exists, but no gateway-enforced logical identity or fencing prevents a stale harness from continuing to reply or execute effects after takeover.
|
||||
|
||||
The objective is to make Mos a server-derived logical Mosaic identity whose authority can move safely among runtime connectors. The gateway owns identity, lease, policy, and audit; harnesses remain replaceable adapters.
|
||||
|
||||
### M1 Requirements
|
||||
|
||||
1. `MOS-PORT-ID-001`: Define a normalized logical-agent identity independent of Claude Code, Pi, Codex, tmux, Matrix, and provider-native session IDs.
|
||||
2. `MOS-PORT-LEASE-001`: Persist one exclusive connector lease per tenant/logical-agent/binding with CAS acquisition, monotonic fencing epoch, TTL, heartbeat, explicit release, and takeover.
|
||||
3. `MOS-PORT-FENCE-001`: Bind every connector dispatch/execution grant to the current server-derived tenant, logical identity, binding, connector, scopes, expiry, and lease epoch.
|
||||
4. `MOS-PORT-FENCE-002`: Reject and audit stale, expired, forged, cross-tenant, cross-binding, and unauthorized grants before connector, channel, provider, or tool side effects.
|
||||
5. `MOS-PORT-OBS-001`: Emit credential-safe correlation/audit events for lease acquire, renew, takeover, reject, release, and expiry.
|
||||
6. `MOS-PORT-ARCH-001`: Runtime/provider adapters consume normalized lease context without adding harness-native schemas to Mosaic core.
|
||||
|
||||
### M1 Acceptance Criteria
|
||||
|
||||
1. `AC-MOS-PORT-01`: Two contenders for one binding cannot simultaneously hold current authority under concurrency.
|
||||
2. `AC-MOS-PORT-02`: Successful takeover increments the fencing epoch and every operation from the old epoch fails closed before side effects.
|
||||
3. `AC-MOS-PORT-03`: Gateway/database restart preserves lease and epoch state; expired leases can be recovered only through the authorized takeover path.
|
||||
4. `AC-MOS-PORT-04`: Cross-tenant, cross-agent, cross-binding, forged, and expired lease/grant cases are denied and audited.
|
||||
5. `AC-MOS-PORT-05`: Unit, migration, repository close/reopen, concurrency, abuse, gateway integration, independent security review, CI, and documentation gates pass.
|
||||
|
||||
### Deferred to Later #754 Milestones
|
||||
|
||||
Canonical checkpoint/handoff payloads, exactly-once connector receipts, concrete Claude/Pi/Codex adapters, channel cutover, and full cross-harness failover/rollback E2E are explicitly out of M1 scope.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level System Diagram
|
||||
@@ -575,7 +768,8 @@ Discord remote control channel. Architecture inspired by OpenClaw (https://githu
|
||||
- Single-guild binding only (v0.1.0) — prevents data leaks between servers
|
||||
- Receives Discord messages, dispatches through gateway routing
|
||||
- Streams agent responses back to Discord (chunked for 2000-char limit)
|
||||
- Supports mention-based activation, thread management for multi-turn
|
||||
- Routes authorized untagged messages in-channel; mentions create threads (or reuse the same message's attached thread) for multi-turn topics
|
||||
- Uses stable logical-agent/channel conversation addresses independent of the active harness/provider
|
||||
- Bot pairing and permission management (Discord user → Mosaic user mapping)
|
||||
- DM support for private conversations
|
||||
|
||||
@@ -748,10 +942,12 @@ Telegram remote control channel.
|
||||
|
||||
### FR-9: Remote Control — Discord
|
||||
|
||||
- Discord bot that connects to the gateway
|
||||
- Mention-based activation in channels
|
||||
- Discord bot that connects to the gateway through a transport-neutral channel adapter contract
|
||||
- Authorized messages in configured agent-bound channels work without a mention and respond in-channel
|
||||
- Mentions in parent channels create threads, or reuse a thread already attached to that same native message, for multi-turn conversations
|
||||
- Messages already in a thread remain there without requiring repeated mentions
|
||||
- Stable logical-agent/channel conversation identity survives underlying harness/provider changes
|
||||
- DM support for private conversations
|
||||
- Thread creation for multi-turn conversations
|
||||
- Chunked message delivery (Discord 2000-char limit)
|
||||
- Bot configuration via web dashboard
|
||||
- Permission management (which Discord users/roles can interact)
|
||||
@@ -935,10 +1131,13 @@ Telegram remote control channel.
|
||||
|
||||
### AC-3: Discord Remote Control
|
||||
|
||||
- [ ] Discord bot connects and responds to mentions
|
||||
- [ ] Messages route through gateway to agent pool
|
||||
- [ ] Discord bot connects through the harness-neutral channel contract
|
||||
- [ ] Authorized untagged channel messages route through the gateway and respond in-channel
|
||||
- [ ] Mentioned parent-channel messages create a thread (or reuse their already-attached thread) and respond there
|
||||
- [ ] Existing-thread follow-ups stay in the thread without repeated mentions
|
||||
- [ ] Channel/session identity remains stable while the underlying harness/provider changes
|
||||
- [ ] Responses stream back to Discord (chunked)
|
||||
- [ ] Thread creation for multi-turn conversations
|
||||
- [ ] Unauthorized guilds, channels, users, pairings, and roles create no thread and dispatch no message
|
||||
|
||||
### AC-4: Gateway Orchestration
|
||||
|
||||
@@ -982,10 +1181,10 @@ Telegram remote control channel.
|
||||
|
||||
### AC-10: Deployment
|
||||
|
||||
- [ ] `docker compose up` starts full stack from clean state
|
||||
- [ ] `mosaic` CLI installable and functional on bare metal
|
||||
- [ ] Database migrations run automatically on first start
|
||||
- [ ] `.env.example` documents all required configuration
|
||||
- [ ] PGlite data-layer work uses no PostgreSQL; optional Compose services are selected individually and do not start PostgreSQL; Gateway/Web local start remains held until KBN-101-02 rejects daemon/inherited/project DSNs before connection or DDL
|
||||
- [ ] PostgreSQL/federated activation is unavailable until KBN-101-00/-03/-05 deliver external bootstrap, TLS/roles, runner `--run`, runner `--verify`, and Gateway/Compose readiness in that order
|
||||
- [ ] `mosaic` CLI installable and functional on bare metal after the reviewed KBN-101-05 secret-renderer/process-exec or `LoadCredential` interface exists
|
||||
- [ ] Local-only configuration documentation is distinct from production generation-pinned Vault-rendered consumer material
|
||||
|
||||
### AC-11: @mosaicstack/\* Packages
|
||||
|
||||
@@ -1137,7 +1336,7 @@ All work is **alpha** (< 0.1.0) until Jason approves 0.1.0 beta release.
|
||||
|
||||
6. ASSUMPTION: **Log summarization uses Haiku-tier LLM by default, configurable.** Haiku is well-suited for summarization (compression, not generation — source material is in context). Guardrails: structured output via Zod schema (force extraction of decisions/tools/outcomes/errors as discrete fields), chunked per-session processing (no bulk conflation), extraction-focused prompts. Raw logs stay in hot tier (7 days) as safety net. Users can override the summarization model via routing engine config if they want higher fidelity. Rationale: Haiku is 10-20x cheaper than Sonnet; log summarization runs on schedule against large volumes where cost matters.
|
||||
|
||||
7. ASSUMPTION: **Discord plugin starts minimal and single-guild only** — DM support, mention-based channel activation, thread management, chunked responses. Single guild binding to prevent data leaks between servers. Advanced features (voice, components, slash commands, multi-guild) are post-beta. Rationale: Proven pattern from OpenClaw; ship core interaction first; data isolation is non-negotiable.
|
||||
7. ASSUMPTION: **Discord plugin starts minimal and single-guild only** — explicitly configured agent-bound channels accept authorized untagged messages in-channel, while mentions create threads or reuse a thread already attached to that same native message; responses are chunked. Single guild binding prevents data leaks between servers. DM support, voice, components, slash commands, and multi-guild operation are post-beta. Rationale: Ship the requested core interaction model while preserving default-deny data isolation.
|
||||
|
||||
8. ASSUMPTION: **Telegram plugin is lower priority than Discord** and may ship as v0.0.7 or later if Discord takes longer than expected. Rationale: Jason indicated Discord as the high-priority remote channel.
|
||||
|
||||
|
||||
@@ -1,12 +1,35 @@
|
||||
# Documentation Sitemap
|
||||
|
||||
## CLI and skill management
|
||||
|
||||
- [Skill registration user guide](guides/user-guide.md#claude-code-skill-registration) — register, unregister, list statuses, automatic install/update reconciliation, and Claude reload behavior.
|
||||
- [Skill bridge developer guide](guides/dev-guide.md#claude-code-skill-bridge) — path-validation, ownership, clobber-protection, install/update wiring, tests, and Pi/Codex scope notes.
|
||||
|
||||
## Fleet configuration management
|
||||
|
||||
- [Generated environment boundary](fleet/reference/generated-env-boundary.md) — roster-derived launch projection, strict local data, legacy quarantine, and downstream interface evidence.
|
||||
- [Roster v2 structural contract](fleet/reference/roster-v2-fields.md) — local-tmux schema v2 parsing and structural validation.
|
||||
- [Role classes and authority](fleet/reference/role-classes.md) — canonical role resolver and protected authority boundaries.
|
||||
- [Executable asset dispositions](fleet/migration/example-profile-disposition.md) — shipped v1 fixture/profile/service validation posture.
|
||||
|
||||
## Official channel plugins
|
||||
|
||||
- [Channel protocol architecture](architecture/channel-protocol.md) — shared lifecycle, message, stable-route, authorization, and response-target contracts.
|
||||
- [Discord administrator configuration](guides/admin-guide.md#discord-ingress-security) — secrets, allowlists, bindings, role policy, and thread permissions.
|
||||
- [Discord user workflow](tess/USER-GUIDE.md#discord-conversations) — in-channel messages, mention-created threads, and runtime-transparent continuity.
|
||||
- [Channel plugin authoring](tess/PLUGIN-GUIDE.md#official-channel-adapter-contract) — requirements for future Matrix, Slack, and other official adapters.
|
||||
- [Discord package guide](../plugins/discord/README.md) — package behavior, configuration shape, and development commands.
|
||||
|
||||
## Native Kanban and canonical task SOT
|
||||
|
||||
- [Canonical requirements](requirements/native-kanban-sot.md) — ratified P0–P3 requirements and acceptance criteria.
|
||||
- [Workstream index](native-kanban-sot/INDEX.md) — artifact map, lane partition, and delivery order.
|
||||
- [Mission manifest](native-kanban-sot/MISSION-MANIFEST.md) — scope, authority, invariants, and gate model.
|
||||
- [Task decomposition](native-kanban-sot/TASKS.md) — dependency-ordered implementation slices and ownership boundaries.
|
||||
- [KBN-101 database role split](native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md) — rc.16 direct-Drizzle storage-wrapper hold: legacy N-1/uncertified/non-operative pending -02/-03/-06/-08; exact README/user-guide wrapper forms fail before masking and source-consistency rejects runner-delegation copy; held bootstrap → TLS/roles → run → verify → readiness; plus prior attestation, pgvector owner, classifier, TLS, activation, and certification prerequisite.
|
||||
- [Federated tier data migration](guides/migrate-tier.md) — active KBN-101-07 operator route: runner-produced target attestation, dedicated non-DDL importer, and paired credential-/attestation-file references only.
|
||||
- [Frozen shared contract](native-kanban-sot/SHARED-CONTRACT.md) — schema, API, Coordinator, health, recovery, and migration contracts.
|
||||
- [KBN-101 exact-head security review](reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) — retained prior REQUEST CHANGES evidence for `da742ca`; rc.16 awaits independent exact-head re-review after closing the current generic storage-wrapper authority HIGH finding.
|
||||
- [Initial independent review](reports/native-kanban-sot/canon-initial-review-no-go.md) — KCR-001–016 findings that blocked the first draft.
|
||||
- [Final independent re-review](reports/native-kanban-sot/canon-final-rereview-go.md) — closure evidence and GO verdict.
|
||||
- [Ultron final gate](reports/native-kanban-sot/ultron-final-go.md) — final requirements, authority, schema, migration, recovery, and evidence review.
|
||||
@@ -48,3 +71,5 @@
|
||||
- [Optional AI egress gateway ADR](architecture/ADR-MOS-EGRESS-GATEWAYS.md) — placement and gates for LiteLLM, Bifrost, and purpose-built translation proxies.
|
||||
- [Runtime-neutral Mos identity and failover mission](https://git.mosaicstack.dev/mosaicstack/stack/issues/754)
|
||||
- [Logical identity and connector lease/fencing implementation](https://git.mosaicstack.dev/mosaicstack/stack/issues/755)
|
||||
- [M1 logical identity and fencing architecture](architecture/mos-runtime-portability-m1.md)
|
||||
- [M1 connector lease operations](guides/mos-connector-lease-operations.md)
|
||||
|
||||
@@ -53,15 +53,15 @@ Active workstream is **W1 — Federation v1**. Workers should:
|
||||
> the applicable acceptance evidence before merge. Issue #758 remains open until M5 closes.
|
||||
|
||||
| id | status | description | issue | agent | repo | branch | depends_on | estimate | notes |
|
||||
| ---------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ----------------- | --------------------------------------- | ---------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| FCM-M0-001 | in-progress | Publish normative PRD requirements/acceptance criteria, this M0–M5 DAG, docs-IA checklist, and legacy example/profile disposition inventory; no implementation changes | #758 | sonnet | mosaicstack/stack | `docs/758-fleet-config-management` | — | 18K | M0 exit: approved docs; every shipped example/profile/service preset classified; docs-only PR |
|
||||
| FCM-M1-001 | not-started | Implement narrow local-tmux v2 roster structural contract/compiler with YAML/JSON canonicalization and schema/parser parity tests | #758 | codex | mosaicstack/stack | `feat/758-roster-v2-compiler` | FCM-M0-001 | 30K | No lifecycle, remote, connector, secret, channel, or gateway work |
|
||||
| FCM-M1-002 | not-started | Reuse existing profile/persona/provision resolver for roster semantics; add canonical class/authority validation and approved aliases | #758 | codex | mosaicstack/stack | `feat/758-shared-role-resolution` | FCM-M0-001 | 25K | Validator is certificate-only; merge-gate remains sole merge authority |
|
||||
| FCM-M1-003 | not-started | Convert the M0 legacy inventory into executable example/profile/service-preset validation and explicit v1-version/retirement checks | #758 | codex | mosaicstack/stack | `test/758-example-profile-dispositions` | FCM-M1-001, FCM-M1-002 | 20K | Every shipped artifact must validate, be versioned v1, or be retired with replacement |
|
||||
| FCM-M2-001 | not-started | Migrate generic launch chain to deterministic `.env.generated` plus strict data-only `.env.local`; quarantine forbidden legacy keys | #758 | codex | mosaicstack/stack | `feat/758-generated-env-boundary` | FCM-M1-001, FCM-M1-002 | 30K | No arbitrary command compatibility path; diagnostics expose key names/hashes only |
|
||||
| FCM-M2-002 | not-started | Add generation-guarded local fleet agent create/get/update/delete mutations with plan/dry-run, atomic roster writes, and recovery output | #758 | codex | mosaicstack/stack | `feat/758-fleet-agent-crud` | FCM-M1-001, FCM-M2-001 | 30K | Fresh create persists stopped unless explicit persisted start |
|
||||
| FCM-M3-001 | not-started | Implement local roster-owned reconcile/apply plus lifecycle/status/verify/doctor contracts and stable JSON/exit codes | #758 | codex | mosaicstack/stack | `feat/758-local-reconciler` | FCM-M2-001, FCM-M2-002 | 35K | Exact systemd/tmux ownership; remote/schema-only entries are inventory only |
|
||||
| FCM-M3-002 | not-started | Add isolated systemd/tmux lifecycle, drift, socket, unmanaged-session, crash, and rollback acceptance coverage | #758 | sonnet | mosaicstack/stack | `test/758-reconciler-lifecycle-gates` | FCM-M3-001 | 25K | Proves stopped-state preservation and zero fuzzy destructive targeting |
|
||||
| ---------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------- | ----------------- | --------------------------------------- | ---------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| FCM-M0-001 | done | Publish normative PRD requirements/acceptance criteria, this M0–M5 DAG, docs-IA checklist, and legacy example/profile disposition inventory; no implementation changes | #758 | sonnet | mosaicstack/stack | `docs/758-fleet-config-management` | — | 18K | Merged via #760 (`c32d85a`); parent #758 intentionally remains open through M5 |
|
||||
| FCM-M1-001 | done | Implement narrow local-tmux v2 roster structural contract/compiler with YAML/JSON canonicalization and schema/parser parity tests | #758 | coder0 | mosaicstack/stack | `feat/758-roster-v2-compiler` | FCM-M0-001 | 30K | #764 squash `aa5b43b`; exact-head RoR and PR/main terminal-green CI; no lifecycle or live mutation |
|
||||
| FCM-M1-002 | done | Reuse existing profile/persona/provision resolver for roster semantics; add canonical class/authority validation and approved aliases | #758 | native-sonnet | mosaicstack/stack | `feat/758-shared-role-resolution` | FCM-M0-001 | 25K | #768 squash `a5e8e55`; shared resolver and canonical authority/alias validation delivered |
|
||||
| FCM-M1-003 | done | Convert the M0 legacy inventory into executable example/profile/service-preset validation and explicit v1-version/retirement checks | #758 | codex | mosaicstack/stack | `test/758-example-profile-dispositions` | FCM-M1-001, FCM-M1-002 | 20K | #770 squash `e9c4aa3`; shipped artifact disposition validation delivered |
|
||||
| FCM-M2-001 | done | Migrate generic launch chain to deterministic `.env.generated` plus strict data-only `.env.local`; quarantine forbidden legacy keys | #758 | codex | mosaicstack/stack | `feat/758-generated-env-boundary` | FCM-M1-001, FCM-M1-002 | 30K | #772 squash `191efae`; generated/local boundary and private quarantine delivered |
|
||||
| FCM-M2-002 | done | Add generation-guarded local fleet agent create/get/update/delete mutations with plan/dry-run, atomic roster writes, and recovery output | #758 | codex | mosaicstack/stack | `feat/758-fleet-agent-crud` | FCM-M1-001, FCM-M2-001 | 30K | #773 squash `bc5e736`; generation-guarded atomic CRUD and recovery contracts delivered |
|
||||
| FCM-M3-001 | done | Implement local roster-owned reconcile/apply plus lifecycle/status/verify/doctor contracts and stable JSON/exit codes | #758 | codex | mosaicstack/stack | `feat/758-local-reconciler` | FCM-M2-001, FCM-M2-002 | 35K | #785 squash `4990905`; exact roster-owned systemd/tmux reconcile and lifecycle contracts delivered |
|
||||
| FCM-M3-002 | in-progress | Add isolated systemd/tmux lifecycle, drift, socket, unmanaged-session, crash, and rollback acceptance coverage | #758 | sonnet | mosaicstack/stack | `test/758-reconciler-lifecycle-gates` | FCM-M3-001 | 25K | Canonical v2 named-socket + legacy-v1 default-server boundaries; fake adapters/temp fixtures only |
|
||||
| FCM-M4-001 | not-started | Implement field-complete v1-to-v2 inventory/preview/migrator with alias, lifecycle, env-quarantine, and remote/connector disposition evidence | #758 | codex | mosaicstack/stack | `feat/758-v1-v2-migrator` | FCM-M1-003, FCM-M3-001 | 35K | Preview first; no unreviewed lifecycle inference |
|
||||
| FCM-M4-002 | not-started | Add reversible canary migration, rollback, stale-projection/orphan classification, and current-host 9-managed/3-unmanaged fixture coverage | #758 | sonnet | mosaicstack/stack | `test/758-migration-rollback-gates` | FCM-M4-001, FCM-M3-002 | 25K | Never starts a previously stopped agent or kills an unproven unmanaged session |
|
||||
| FCM-M5-001 | not-started | Deliver the accepted fleet documentation IA, how-to/operations/migration references, and link/example validation | #758 | haiku | mosaicstack/stack | `docs/758-fleet-config-operator-docs` | FCM-M1-003, FCM-M2-002, FCM-M3-001, FCM-M4-001 | 24K | Must close every checklist item or record an approved deferral |
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Channel Protocol Architecture
|
||||
|
||||
**Status:** Draft
|
||||
**Status:** Official adapter baseline implemented by #756; extended registry/multiplexing remains iterative
|
||||
**Authors:** Mosaic Core Team
|
||||
**Last Updated:** 2026-03-22
|
||||
**Covers:** M7-001 (IChannelAdapter interface), M7-002 (ChannelMessage protocol), M7-003 (Matrix integration design), M7-004 (conversation multiplexing), M7-005 (remote auth bridging), M7-006 (agent-to-agent communication via Matrix), M7-007 (multi-user isolation in Matrix)
|
||||
**Last Updated:** 2026-07-14
|
||||
**Covers:** M7-001 (OfficialChannelAdapter interface), M7-002 (ChannelMessageDto protocol), M7-003 (Matrix integration design), M7-004 (conversation multiplexing), M7-005 (remote auth bridging), M7-006 (agent-to-agent communication via Matrix), M7-007 (multi-user isolation in Matrix)
|
||||
|
||||
---
|
||||
|
||||
@@ -11,93 +11,80 @@
|
||||
|
||||
The channel protocol defines a unified abstraction layer between Mosaic's core messaging infrastructure and the external communication channels it supports (Matrix, Discord, Telegram, TUI, WebUI, and future channels).
|
||||
|
||||
The protocol consists of two main contracts:
|
||||
The implemented baseline is exported from `@mosaicstack/types` and consists of four contract groups:
|
||||
|
||||
1. `IChannelAdapter` — the interface each channel driver must implement.
|
||||
2. `ChannelMessage` — the canonical message format that flows through the system.
|
||||
1. `OfficialChannelAdapter` — transport lifecycle and connection health.
|
||||
2. `ChannelMessageDto` / `ChannelAttachmentDto` — canonical transport data.
|
||||
3. `ChannelConversationRouteDto` — stable logical-agent conversation and authorization address.
|
||||
4. `ChannelResponseTargetDto` — channel/thread destination for replies.
|
||||
|
||||
All channel-specific translation logic lives inside the adapter implementation. The rest of Mosaic works exclusively with `ChannelMessage` objects.
|
||||
All channel-specific translation logic lives inside the adapter implementation. Runtime selection does not: gateway durable-session and provider services may rebind the logical session from Claude to Codex, Pi, OpenCode, or another harness without reconnecting the channel adapter.
|
||||
|
||||
---
|
||||
|
||||
## M7-001: IChannelAdapter Interface
|
||||
## M7-001: OfficialChannelAdapter Interface
|
||||
|
||||
```typescript
|
||||
interface IChannelAdapter {
|
||||
/**
|
||||
* Stable, lowercase identifier for this channel (e.g. "matrix", "discord").
|
||||
* Used as a namespace key in registry lookups and log metadata.
|
||||
*/
|
||||
interface OfficialChannelAdapter {
|
||||
/** Stable, lowercase adapter identifier such as "discord" or "matrix". */
|
||||
readonly name: string;
|
||||
|
||||
/**
|
||||
* Establish a connection to the external channel backend.
|
||||
* Called once at application startup. Must be idempotent (safe to call
|
||||
* when already connected).
|
||||
*/
|
||||
connect(): Promise<void>;
|
||||
|
||||
/**
|
||||
* Gracefully disconnect from the channel backend.
|
||||
* Must flush in-flight sends and release resources before resolving.
|
||||
*/
|
||||
disconnect(): Promise<void>;
|
||||
|
||||
/**
|
||||
* Return the current health of the adapter connection.
|
||||
* Used by the admin health endpoint and alerting.
|
||||
*
|
||||
* - "connected" — fully operational
|
||||
* - "degraded" — partial connectivity (e.g. read-only, rate-limited)
|
||||
* - "disconnected" — no connection to channel backend
|
||||
*/
|
||||
health(): Promise<{ status: 'connected' | 'degraded' | 'disconnected' }>;
|
||||
|
||||
/**
|
||||
* Register an inbound message handler.
|
||||
* The adapter calls `handler` for every message received from the channel.
|
||||
* Multiple calls replace the previous handler (last-write-wins).
|
||||
* The handler is async; the adapter must not deliver new messages until
|
||||
* the previous handler promise resolves (back-pressure).
|
||||
*/
|
||||
onMessage(handler: (msg: ChannelMessage) => Promise<void>): void;
|
||||
|
||||
/**
|
||||
* Send a ChannelMessage to the given channel/room/conversation.
|
||||
* `channelId` is the channel-native identifier (e.g. Matrix room ID,
|
||||
* Discord channel snowflake, Telegram chat ID).
|
||||
*/
|
||||
sendMessage(channelId: string, msg: ChannelMessage): Promise<void>;
|
||||
|
||||
/**
|
||||
* Map a channel-native user identifier to the Mosaic internal userId.
|
||||
* Returns null when no matching Mosaic account exists for the given
|
||||
* channelUserId (anonymous or unlinked user).
|
||||
*/
|
||||
mapIdentity(channelUserId: string): Promise<string | null>;
|
||||
/** Establish both native-channel and gateway connections. */
|
||||
start(): Promise<void>;
|
||||
/** Gracefully close connections and release resources. */
|
||||
stop(): Promise<void>;
|
||||
/** Best-effort health; ordinary disconnection is a result, not an exception. */
|
||||
health(): Promise<{
|
||||
status: 'connected' | 'degraded' | 'disconnected';
|
||||
detail?: string;
|
||||
}>;
|
||||
}
|
||||
```
|
||||
|
||||
The small lifecycle seam lets the gateway host official plugins uniformly without moving native message translation into gateway core. Message ingress remains adapter-owned; gateway policy, durable session routing, auditing, and runtime/provider selection remain gateway-owned.
|
||||
|
||||
### Stable conversation route
|
||||
|
||||
```typescript
|
||||
interface ChannelConversationRouteDto {
|
||||
bindingId: string;
|
||||
logicalAgentId: string;
|
||||
conversationId: string;
|
||||
channelName: string;
|
||||
authorizationChannelId: string;
|
||||
responseTarget: { channelId: string; threadId?: string };
|
||||
}
|
||||
```
|
||||
|
||||
Harness, provider, model, process, and native runtime-session identifiers are forbidden from this route. Runtime adapters consume the gateway's durable logical-session binding; channel adapters consume only the stable route and response target.
|
||||
|
||||
### Typed ingress and egress ports
|
||||
|
||||
`ChannelIngressPort` is the transport-neutral direct-integration seam for official adapters. The current deployed Discord adapter preserves its existing HMAC-signed Socket.IO compatibility ingress so gateway-side service authentication, replay protection, approval handling, and correlation semantics remain unchanged; it normalizes the same `ChannelIngressDto` before signing. The adapter uses a supplied `ChannelIngressPort` directly when a future gateway registration provides one. New adapters must use the shared ports rather than adding channel branches to gateway core.
|
||||
|
||||
`ChannelBindingDto` contains the configuration-owned workspace/channel→logical-agent mapping and paired external principals; credentials are absent. After native allowlist, pairing, and role checks pass, an adapter submits `ChannelIngressDto` to `ChannelIngressPort.receive()`. It includes the normalized message, `ChannelAuthorizedPrincipalDto`, operation, correlation ID, native message ID, and stable route. Unauthorized input never reaches the port.
|
||||
|
||||
Gateway policy and runtime routing produce `ChannelEgressDto`, which `ChannelEgressPort.send()` delivers to the route's response target. Discord's existing HMAC envelope is its authenticated wire encoding of this boundary; future Matrix/Slack adapters use their native authenticated transports while preserving the same actor/operation/correlation semantics.
|
||||
|
||||
### Adapter Registration
|
||||
|
||||
Adapters are registered with the `ChannelRegistry` service at startup. The registry calls `connect()` on each adapter and monitors `health()` on a configurable interval (default: 30 s).
|
||||
Adapters are registered with the gateway plugin host at startup. The host calls `start()`/`stop()` and may monitor `health()` on a configurable interval. A richer dynamic `ChannelRegistry` remains a compatible future extension of this lifecycle contract.
|
||||
|
||||
```
|
||||
ChannelRegistry
|
||||
└── register(adapter: IChannelAdapter): void
|
||||
└── getAdapter(name: string): IChannelAdapter | null
|
||||
└── listAdapters(): IChannelAdapter[]
|
||||
└── register(adapter: OfficialChannelAdapter): void
|
||||
└── getAdapter(name: string): OfficialChannelAdapter | null
|
||||
└── listAdapters(): OfficialChannelAdapter[]
|
||||
└── healthAll(): Promise<Record<string, AdapterHealth>>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## M7-002: ChannelMessage Protocol
|
||||
## M7-002: ChannelMessageDto Protocol
|
||||
|
||||
### Canonical Message Format
|
||||
|
||||
```typescript
|
||||
interface ChannelMessage {
|
||||
interface ChannelMessageDto {
|
||||
/**
|
||||
* Globally unique message ID.
|
||||
* Format: UUID v4. Generated by the adapter when receiving, or by Mosaic
|
||||
@@ -110,6 +97,7 @@ interface ChannelMessage {
|
||||
* The adapter populates this from the inbound message.
|
||||
* For outbound messages, the caller supplies the target channel.
|
||||
*/
|
||||
channelName: string;
|
||||
channelId: string;
|
||||
|
||||
/**
|
||||
@@ -119,7 +107,7 @@ interface ChannelMessage {
|
||||
senderId: string;
|
||||
|
||||
/** Sender classification. */
|
||||
senderType: 'user' | 'agent' | 'system';
|
||||
senderKind: 'user' | 'agent' | 'system';
|
||||
|
||||
/**
|
||||
* Textual content of the message.
|
||||
@@ -136,7 +124,7 @@ interface ChannelMessage {
|
||||
* - "image" — binary image; content is empty, see attachments
|
||||
* - "file" — binary file; content is empty, see attachments
|
||||
*/
|
||||
contentType: 'text' | 'markdown' | 'code' | 'image' | 'file';
|
||||
contentKind: 'text' | 'markdown' | 'code' | 'image' | 'file';
|
||||
|
||||
/**
|
||||
* Arbitrary key-value metadata for channel-specific extension fields.
|
||||
@@ -144,7 +132,7 @@ interface ChannelMessage {
|
||||
* Adapters should store channel-native IDs here so round-trip correlation
|
||||
* is possible without altering the canonical fields.
|
||||
*/
|
||||
metadata: Record<string, unknown>;
|
||||
metadata: Readonly<Record<string, ChannelMetadataValue>>;
|
||||
|
||||
/**
|
||||
* Optional thread or reply-chain identifier.
|
||||
@@ -163,18 +151,21 @@ interface ChannelMessage {
|
||||
* Binary or URI-referenced attachments.
|
||||
* Each attachment carries its MIME type and a URL or base64 payload.
|
||||
*/
|
||||
attachments?: ChannelAttachment[];
|
||||
attachments?: readonly ChannelAttachmentDto[];
|
||||
|
||||
/** Wall-clock timestamp when the message was sent/received. */
|
||||
timestamp: Date;
|
||||
/** ISO-8601 wall-clock timestamp when the message was sent/received. */
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
interface ChannelAttachment {
|
||||
/** Filename or identifier. */
|
||||
interface ChannelAttachmentDto {
|
||||
/** Channel-native attachment identifier. */
|
||||
id: string;
|
||||
|
||||
/** Filename or display name. */
|
||||
name: string;
|
||||
|
||||
/** MIME type (e.g. "image/png", "application/pdf"). */
|
||||
mimeType: string;
|
||||
/** MIME type when supplied by the channel. */
|
||||
mimeType: string | null;
|
||||
|
||||
/**
|
||||
* URL pointing to the attachment, OR a `data:` URI with base64 payload.
|
||||
@@ -192,18 +183,18 @@ interface ChannelAttachment {
|
||||
|
||||
## Channel Translation Reference
|
||||
|
||||
The following sections document how each supported channel maps its native message format to and from `ChannelMessage`.
|
||||
The following sections document how each supported channel maps its native message format to and from `ChannelMessageDto`.
|
||||
|
||||
### Matrix
|
||||
|
||||
| ChannelMessage field | Matrix equivalent |
|
||||
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ChannelMessageDto field | Matrix equivalent |
|
||||
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Matrix event ID (`$...`) |
|
||||
| `channelId` | Matrix room ID (`!roomid:homeserver`) |
|
||||
| `senderId` | Matrix user ID (`@user:homeserver`) |
|
||||
| `senderType` | Always `"user"` for inbound; `"agent"` or `"system"` for outbound |
|
||||
| `senderKind` | Always `"user"` for inbound; `"agent"` or `"system"` for outbound |
|
||||
| `content` | `event.content.body` |
|
||||
| `contentType` | `"markdown"` if `msgtype = m.text` and body contains markdown; `"text"` otherwise; `"image"` for `m.image`; `"file"` for `m.file` |
|
||||
| `contentKind` | `"markdown"` if `msgtype = m.text` and body contains markdown; `"text"` otherwise; `"image"` for `m.image`; `"file"` for `m.file` |
|
||||
| `threadId` | `event.content['m.relates_to']['event_id']` when `rel_type = m.thread` |
|
||||
| `replyToId` | Mosaic ID looked up from `event.content['m.relates_to']['m.in_reply_to']['event_id']` |
|
||||
| `attachments` | Populated from `url` in `m.image` / `m.file` events |
|
||||
@@ -216,21 +207,34 @@ The following sections document how each supported channel maps its native messa
|
||||
|
||||
### Discord
|
||||
|
||||
| ChannelMessage field | Discord equivalent |
|
||||
| -------------------- | ----------------------------------------------------------------------- |
|
||||
| ChannelMessageDto field | Discord equivalent |
|
||||
| ----------------------- | ----------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Discord message snowflake |
|
||||
| `channelId` | Discord channel ID (snowflake string) |
|
||||
| `senderId` | Discord user ID (snowflake) |
|
||||
| `senderType` | `"user"` for human members; `"agent"` for bot messages |
|
||||
| `senderKind` | `"user"` for human members; `"agent"` for bot messages |
|
||||
| `content` | `message.content` |
|
||||
| `contentType` | `"markdown"` (Discord uses a markdown-like syntax natively) |
|
||||
| `contentKind` | `"markdown"` (Discord uses a markdown-like syntax natively) |
|
||||
| `threadId` | `message.thread.id` when the message is inside a thread channel |
|
||||
| `replyToId` | Mosaic ID looked up from `message.referenced_message.id` |
|
||||
| `attachments` | `message.attachments` mapped to `ChannelAttachment` |
|
||||
| `attachments` | `message.attachments` mapped to `ChannelAttachmentDto` |
|
||||
| `timestamp` | `new Date(message.timestamp)` |
|
||||
| `metadata` | `{ channelMessageId, guildId, channelType, mentions, embeds }` |
|
||||
|
||||
**Outbound:** Adapter calls Discord REST `POST /channels/{id}/messages`. Markdown content is sent as-is (Discord renders it). For `contentType = "code"` the adapter wraps in triple-backtick fences with the `metadata.language` tag.
|
||||
**Outbound:** Adapter calls Discord REST `POST /channels/{id}/messages`. Markdown content is sent as-is (Discord renders it). For `contentKind = "code"` the adapter wraps in triple-backtick fences with the `metadata.language` tag.
|
||||
|
||||
### Discord routing and thread policy
|
||||
|
||||
A configured Discord binding maps `(guildId, parentChannelId)` to a stable logical agent and a trusted gateway agent-config ID. Gateway verifies that configuration's name matches the binding logical agent before session creation. The stable conversation handle is derived from logical agent plus response channel/thread and never includes the active harness, provider, model, process, or agent-config ID.
|
||||
|
||||
| Inbound location/trigger | Conversation and response target |
|
||||
| ------------------------------------------ | --------------------------------------------------------------- |
|
||||
| Authorized untagged parent-channel message | Parent channel; response is sent in-channel |
|
||||
| Authorized bot mention in parent channel | Thread already attached to that message, or a new public thread |
|
||||
| Authorized message already in a thread | Existing thread; no repeated mention and no nested thread |
|
||||
| `/approve` or `/stop <approval>` | Current parent/thread durable session; no new topic is created |
|
||||
|
||||
Authorization order is fixed: guild allowlist → parent-channel allowlist → user allowlist → configured binding/pairing → operation role → per-user/channel message and thread rate limits → thread creation/dispatch. A normal Discord channel's category parent is never treated as the thread authorization parent. If requested thread creation fails, dispatch does not occur because the adapter cannot honor the response target.
|
||||
|
||||
### Discord service ingress security
|
||||
|
||||
@@ -240,21 +244,21 @@ The Discord adapter is an authenticated gateway service, not an anonymous Socket
|
||||
|
||||
### Telegram
|
||||
|
||||
| ChannelMessage field | Telegram equivalent |
|
||||
| -------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| ChannelMessageDto field | Telegram equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Telegram `message_id` (integer) |
|
||||
| `channelId` | Telegram `chat_id` (integer as string) |
|
||||
| `senderId` | Telegram `from.id` (integer as string) |
|
||||
| `senderType` | `"user"` for human senders; `"agent"` for bot-originated messages |
|
||||
| `senderKind` | `"user"` for human senders; `"agent"` for bot-originated messages |
|
||||
| `content` | `message.text` or `message.caption` |
|
||||
| `contentType` | `"text"` for plain; `"markdown"` if `parse_mode = MarkdownV2`; `"image"` for `photo`; `"file"` for `document` |
|
||||
| `contentKind` | `"text"` for plain; `"markdown"` if `parse_mode = MarkdownV2`; `"image"` for `photo`; `"file"` for `document` |
|
||||
| `threadId` | `message.message_thread_id` (for supergroup topics) |
|
||||
| `replyToId` | Mosaic ID looked up from `message.reply_to_message.message_id` |
|
||||
| `attachments` | `photo`, `document`, `video` fields mapped to `ChannelAttachment` |
|
||||
| `attachments` | `photo`, `document`, `video` fields mapped to `ChannelAttachmentDto` |
|
||||
| `timestamp` | `new Date(message.date * 1000)` |
|
||||
| `metadata` | `{ channelMessageId, chatType, fromUsername, forwardFrom }` |
|
||||
|
||||
**Outbound:** Adapter calls Telegram Bot API `sendMessage` with `parse_mode = MarkdownV2` for markdown content. For `contentType = "image"` or `"file"` it uses `sendPhoto` / `sendDocument`.
|
||||
**Outbound:** Adapter calls Telegram Bot API `sendMessage` with `parse_mode = MarkdownV2` for markdown content. For `contentKind = "image"` or `"file"` it uses `sendPhoto` / `sendDocument`.
|
||||
|
||||
---
|
||||
|
||||
@@ -262,14 +266,14 @@ The Discord adapter is an authenticated gateway service, not an anonymous Socket
|
||||
|
||||
The TUI adapter bridges Mosaic's terminal interface (`packages/cli`) to the channel protocol so that TUI sessions can be treated as a first-class channel.
|
||||
|
||||
| ChannelMessage field | TUI equivalent |
|
||||
| -------------------- | ------------------------------------------------------------------ |
|
||||
| ChannelMessageDto field | TUI equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------------ |
|
||||
| `id` | Generated UUID (TUI has no native message IDs) |
|
||||
| `channelId` | `"tui:<conversationId>"` — the active conversation ID |
|
||||
| `senderId` | Authenticated Mosaic `userId` |
|
||||
| `senderType` | `"user"` for human input; `"agent"` for agent replies |
|
||||
| `senderKind` | `"user"` for human input; `"agent"` for agent replies |
|
||||
| `content` | Raw text from stdin / agent output |
|
||||
| `contentType` | `"text"` for input; `"markdown"` for agent responses |
|
||||
| `contentKind` | `"text"` for input; `"markdown"` for agent responses |
|
||||
| `threadId` | Not used (TUI sessions are linear) |
|
||||
| `replyToId` | Not used |
|
||||
| `attachments` | File paths dragged/pasted into the TUI; resolved to `file://` URLs |
|
||||
@@ -284,14 +288,14 @@ The TUI adapter bridges Mosaic's terminal interface (`packages/cli`) to the chan
|
||||
|
||||
The WebUI adapter connects the Next.js frontend (`apps/web`) to the channel protocol over the existing Socket.IO gateway (`apps/gateway`).
|
||||
|
||||
| ChannelMessage field | WebUI equivalent |
|
||||
| -------------------- | ------------------------------------------------------------ |
|
||||
| ChannelMessageDto field | WebUI equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------ |
|
||||
| `id` | Generated UUID; echoed back in the WebSocket event |
|
||||
| `channelId` | `"webui:<conversationId>"` |
|
||||
| `senderId` | Authenticated Mosaic `userId` |
|
||||
| `senderType` | `"user"` for browser input; `"agent"` for agent responses |
|
||||
| `senderKind` | `"user"` for browser input; `"agent"` for agent responses |
|
||||
| `content` | Message text from the input field |
|
||||
| `contentType` | `"text"` or `"markdown"` |
|
||||
| `contentKind` | `"text"` or `"markdown"` |
|
||||
| `threadId` | Not used (conversation model handles threading) |
|
||||
| `replyToId` | Message ID the user replied to (UI reply affordance) |
|
||||
| `attachments` | Files uploaded via the file picker; stored to object storage |
|
||||
@@ -304,7 +308,7 @@ The WebUI adapter connects the Next.js frontend (`apps/web`) to the channel prot
|
||||
|
||||
## Identity Mapping
|
||||
|
||||
`mapIdentity(channelUserId)` resolves a channel-native user identifier to a Mosaic `userId`. This is required to attribute inbound messages to authenticated Mosaic accounts.
|
||||
Gateway identity-linking policy resolves a channel-native user identifier to a Mosaic `userId` and produces `ChannelAuthorizedPrincipalDto`. Adapters provide native identity evidence but cannot self-authorize Mosaic scope. Discord currently uses configuration-owned paired users; database-backed linking remains the canonical direction for dynamic Matrix/Slack identity.
|
||||
|
||||
The implementation must query a `channel_identities` table (or equivalent) keyed on `(channel_name, channel_user_id)`. When no mapping exists the method returns `null` and the message is treated as anonymous (no Mosaic session context).
|
||||
|
||||
@@ -323,8 +327,8 @@ Identity linking flows (OAuth dance, deep-link verification token, etc.) are out
|
||||
|
||||
## Error Handling Conventions
|
||||
|
||||
- `connect()` must throw a structured error (subclass of `ChannelConnectError`) if the initial connection cannot be established within a reasonable timeout (default: 10 s).
|
||||
- `sendMessage()` must throw `ChannelSendError` on terminal failures (auth revoked, channel not found). Transient failures (rate limit, network blip) should be retried internally with exponential backoff before throwing.
|
||||
- `start()` must establish the native channel transport or throw a structured connection error. An adapter hosted inside the gateway must not wait for a loopback connection to that same not-yet-listening process; it starts the native transport, lets Socket.IO reconnect, and reports `degraded` until both links are ready.
|
||||
- `ChannelEgressPort.send()` implementations must throw a typed terminal error for revoked auth, an invalid route, or a missing channel. Only transient rate/network/server failures are retried with bounded exponential backoff; Discord retries reuse a stable enforced nonce to prevent duplicate chunks, while permanent 4xx failures are not retried.
|
||||
- `health()` must never throw — it returns `{ status: 'disconnected' }` on error.
|
||||
- Adapters must emit structured logs with `{ channel: adapter.name, event, ... }` metadata for observability.
|
||||
|
||||
@@ -332,7 +336,7 @@ Identity linking flows (OAuth dance, deep-link verification token, etc.) are out
|
||||
|
||||
## Versioning
|
||||
|
||||
The `ChannelMessage` protocol follows semantic versioning. Non-breaking field additions (new optional fields) are minor version bumps. Breaking changes (type changes, required field additions) require a major version bump and a migration guide.
|
||||
The `ChannelMessageDto` protocol follows semantic versioning. Non-breaking field additions (new optional fields) are minor version bumps. Breaking changes (type changes, required field additions) require a major version bump and a migration guide.
|
||||
|
||||
Current version: **1.0.0**
|
||||
|
||||
@@ -473,7 +477,7 @@ A single Mosaic conversation can be accessed simultaneously from multiple surfac
|
||||
### Real-Time Sync Flow
|
||||
|
||||
1. A message arrives on any surface (TUI keystroke, browser send, Matrix event).
|
||||
2. The surface's adapter normalizes the message to `ChannelMessage` and delivers it to `ConversationService`.
|
||||
2. The surface's adapter normalizes the message to `ChannelMessageDto` and delivers it to `ConversationService`.
|
||||
3. `ConversationService` persists the message to PostgreSQL, assigns a canonical `id`, and publishes a `message:new` event to the Valkey pub/sub channel keyed by `conversationId`.
|
||||
4. All active surfaces subscribed to that `conversationId` receive the fanout event and push it to their respective clients:
|
||||
- TUI adapter: writes rendered output to the connected terminal session.
|
||||
@@ -561,7 +565,7 @@ Matrix sessions for linked users are persistent and long-lived. Unlike TUI sessi
|
||||
- Their `channel_identities` row exists (link not revoked).
|
||||
- They remain members of the relevant Matrix rooms.
|
||||
|
||||
Revoking a Matrix link (`DELETE /auth/channel-link/matrix/<matrixUserId>`) removes the `channel_identities` row and causes `mapIdentity()` to return `null`. The appservice optionally kicks the Matrix user from all Mosaic-managed rooms as part of the revocation flow (configurable, default: off).
|
||||
Revoking a Matrix link (`DELETE /auth/channel-link/matrix/<matrixUserId>`) removes the `channel_identities` row and causes gateway principal resolution to deny the identity. The appservice optionally kicks the Matrix user from all Mosaic-managed rooms as part of the revocation flow (configurable, default: off).
|
||||
|
||||
---
|
||||
|
||||
@@ -733,7 +737,7 @@ room_retention_policies
|
||||
created_at TIMESTAMP
|
||||
```
|
||||
|
||||
The retention policy is enforced by a background job in the gateway that calls Conduit's admin API to purge events older than the configured threshold. Purged events are removed from the Conduit store but Mosaic's PostgreSQL message store retains the canonical `ChannelMessage` record unless the Mosaic retention policy also covers it.
|
||||
The retention policy is enforced by a background job in the gateway that calls Conduit's admin API to purge events older than the configured threshold. Purged events are removed from the Conduit store but Mosaic's PostgreSQL message store retains the canonical `ChannelMessageDto` record unless the Mosaic retention policy also covers it.
|
||||
|
||||
Default retention values:
|
||||
|
||||
|
||||
49
docs/architecture/mos-runtime-portability-m1.md
Normal file
49
docs/architecture/mos-runtime-portability-m1.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Mos Runtime Portability M1 — Logical Identity and Fencing
|
||||
|
||||
## Boundary
|
||||
|
||||
M1 separates the logical Mosaic agent from any Claude, Pi, Codex, tmux, Matrix, or provider-native session. The normalized identity is:
|
||||
|
||||
```text
|
||||
(tenant_id, logical_agent_id, binding_id)
|
||||
```
|
||||
|
||||
`logical_agent_id` is a server-owned stable identifier. A connector is a replaceable holder of a lease for one binding; it is not the agent identity.
|
||||
|
||||
## Durable lease model
|
||||
|
||||
PostgreSQL table `logical_agent_connector_leases` has one unique row per identity/binding tuple. The current row records:
|
||||
|
||||
- an opaque lease UUID;
|
||||
- connector ID and normalized allowed scopes;
|
||||
- a positive decimal fencing epoch stored as PostgreSQL `bigint`;
|
||||
- acquired, heartbeat, expiry, release, and update timestamps.
|
||||
|
||||
Initial acquisition is insert-only. An existing active row causes `lease_held`. An expired or released row causes `takeover_required`; ordinary acquisition cannot recover it. Authorized takeover uses compare-and-swap against the expected epoch, rotates the lease UUID, and increments the epoch atomically. Heartbeat and release match the full identity, binding, connector, lease UUID, and epoch.
|
||||
|
||||
The companion `connector_lease_audit_log` is append-only metadata. It stores lifecycle event, outcome/reason, identity/binding/connector, epoch, correlation ID, and timestamp. It deliberately excludes scopes, grant objects, payloads, approval references, tokens, and credentials.
|
||||
|
||||
## Execution grants
|
||||
|
||||
`ConnectorLeaseCoordinator` issues a short-lived internal grant only after rereading the durable current lease. Defense-in-depth caps leases at 5 minutes and grants at 30 seconds by default; constructor options may tighten these limits. A grant is bound to tenant, logical agent, binding, connector, lease UUID, scope subset, expiry, and epoch.
|
||||
|
||||
Validation occurs immediately before adapter invocation and rereads PostgreSQL. The adapter receives only `ConnectorExecutionContext`; harness-native schemas remain behind the adapter. Validation denies:
|
||||
|
||||
- grants not minted by the current gateway process (including cloned/forged objects);
|
||||
- expired grants or leases;
|
||||
- released leases;
|
||||
- stale epochs or replaced connector/lease UUIDs;
|
||||
- missing/cross-tenant/cross-agent/cross-binding leases;
|
||||
- scopes not authorized by both grant and current lease.
|
||||
|
||||
A gateway restart intentionally invalidates process-local grants. The durable lease and epoch survive, and a fresh grant may be issued only after current-lease and gateway-policy validation.
|
||||
|
||||
## Concurrency and side-effect rule
|
||||
|
||||
The database CAS determines the sole current holder. A successful takeover makes every old-epoch validation fail. Connector adapters must consume and propagate the normalized lease epoch/context so downstream effect boundaries can also fence races that occur after gateway validation.
|
||||
|
||||
M1 does not provide exactly-once receipts or a side-effect journal. Those remain later #754 work; callers must not infer exactly-once delivery from lease fencing.
|
||||
|
||||
## Extension boundary
|
||||
|
||||
`ConnectorLeaseService` is the gateway-owned policy surface. Every policy decision receives the normalized requested scopes and TTL (or explicit `null` where no TTL applies), so a concrete policy can enforce least privilege and duration limits. Its production default policy denies every lease/grant operation until a server-configured connector policy is supplied. No M1 HTTP endpoint accepts caller-controlled tenant or logical identity, and no concrete Claude/Pi/Codex adapter or channel cutover is included.
|
||||
352
docs/compaction-refresh/GATE0-EVIDENCE.md
Normal file
352
docs/compaction-refresh/GATE0-EVIDENCE.md
Normal file
@@ -0,0 +1,352 @@
|
||||
# Compaction-Refresh WI-0 Gate0 Evidence Pack
|
||||
|
||||
- **Issue:** Gitea #827
|
||||
- **Milestone:** 188 — Compaction-Refresh Mechanism
|
||||
- **Branch:** `feat/827-gate0-probe`
|
||||
- **Starting HEAD:** `d801d6c4c8a984d6a95033c49714210018d3d9a8`
|
||||
- **Host/runtime:** Linux 6.1.0-48-amd64; Mosaic 0.0.48; Pi 0.80.7; Claude Code 2.1.205
|
||||
- **Scope:** Probe fixtures and evidence only. No WI-1..WI-7 feature implementation.
|
||||
|
||||
## Verdict — 5/6 PASS; BUILD ADMISSION: **NO**
|
||||
|
||||
| Probe | Verdict | Short result |
|
||||
| --- | --- | --- |
|
||||
| P1 launcher topology + ancestry | **PASS** | Real Mosaic→Pi and Mosaic→Claude chains reached the registered anchor; real Claude `SessionStart` hook ancestry accepted; same-UID sibling with the minted victim ID rejected. |
|
||||
| P2 Pi last-position + nonce map | **PASS** | Real Pi proved last-or-closed; `message_end` mapped exact `toolCallId → requestNonce` before `tool_call`; provider-response hook occurred before stream consumption/content completion. |
|
||||
| P3 same-PID generation revocation | **PASS** | Same Pi PID/starttime persisted through reload/fork/new/resume while broker generations increased; reload revoked a prior `VERIFIED` generation. |
|
||||
| P4 `SO_PEERCRED` + socket posture | **PASS** | Real Unix socket peer PID/UID/starttime matched `/proc`; 0700 directory + 0600 socket demonstrated. Same-UID counterfeit replacement remains explicitly T-C without a distinct principal/authenticated response. |
|
||||
| P5 source invalidation | **PASS** | Missing, oversize, and hash-mismatched fragments each refused injection/promotion, revoked broker state, and blocked the exact emitted tool call. |
|
||||
| P6 atomic injection | **T-C GAP** | Both runtimes empirically delivered a complete single block/message, but neither installed runtime contract states an **atomic/prefix-preserving** transport guarantee. Observation is not a guarantee; A-v5-1/T27 cannot be admitted. |
|
||||
|
||||
**Planner return item:** P6. The evidence establishes successful complete delivery in these runs, not the required invariant that the harness cannot middle-drop/replace bytes while preserving the terminal token. Per R1, such a middle-drop is not receipt-detectable. It is therefore classed **T-C**, not assumed away.
|
||||
|
||||
## STEP 0 — Authority re-verification
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
sha256sum \
|
||||
~/agent-work/reviews/compaction-refresh-BUILD-BRIEF.md \
|
||||
~/agent-work/reviews/compaction-refresh-SPEC-v5.md \
|
||||
~/agent-work/reviews/compaction-refresh-SPEC-RATIFICATION.md
|
||||
```
|
||||
|
||||
Captured result:
|
||||
|
||||
```text
|
||||
89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b .../compaction-refresh-BUILD-BRIEF.md
|
||||
a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433 .../compaction-refresh-SPEC-v5.md
|
||||
bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67 .../compaction-refresh-SPEC-RATIFICATION.md
|
||||
```
|
||||
|
||||
All three **MATCH**. They were read in full before probe construction. Raw artifact: [`evidence/raw/STEP0-authority-hashes.txt`](./evidence/raw/STEP0-authority-hashes.txt).
|
||||
|
||||
## Evidence method
|
||||
|
||||
The scripts under [`probes/`](./probes/) are isolated Gate0 instrumentation, not product implementation. They run the installed `mosaic yolo` launcher and real installed runtime binaries. Broker prototypes use Linux `SO_PEERCRED` and `/proc`; runtime adapters are temporary Claude hooks/Pi extensions. No product source under `packages/mosaic` was changed.
|
||||
|
||||
Raw-output artifact integrity is indexed at [`evidence/RAW-SHA256SUMS.txt`](./evidence/RAW-SHA256SUMS.txt).
|
||||
|
||||
---
|
||||
|
||||
## P1 — Launcher exec/parent topology + supported-hook ancestry (D1)
|
||||
|
||||
**Verdict: PASS**
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/p1_run.py --runtime both
|
||||
rg -n "spawnSync|execRuntime" \
|
||||
~/.npm-global/lib/node_modules/@mosaicstack/mosaic/dist/commands/launch.js | tail -8
|
||||
```
|
||||
|
||||
Full outputs:
|
||||
|
||||
- [`evidence/raw/P1-launch-ancestry.txt`](./evidence/raw/P1-launch-ancestry.txt)
|
||||
- [`evidence/raw/P1-claude-hook-events.txt`](./evidence/raw/P1-claude-hook-events.txt)
|
||||
|
||||
### Real topology observed
|
||||
|
||||
The installed Mosaic launcher does **not** replace itself with the runtime despite its `execRuntime` name; installed `launch.js:668` uses `spawnSync`. The Gate0 anchor first registered, then `execvpe` replaced the anchor with the real `mosaic yolo` process (PID/starttime retained). Mosaic remained the stable parent while it spawned the runtime.
|
||||
|
||||
Pi run:
|
||||
|
||||
```text
|
||||
anchor before exec: pid=4010843 starttime=365919858 exe=/usr/bin/python3.11
|
||||
anchor after exec: pid=4010843 starttime=365919858 exe=/usr/bin/node
|
||||
Pi runtime: pid=4011046 ppid=4010843 starttime=365920219 exe=/usr/bin/node
|
||||
|
||||
ps:
|
||||
4010843 4010840 Fri Jul 17 19:35:18 2026 1001 1001 node
|
||||
4011046 4010843 Fri Jul 17 19:35:22 2026 1001 1001 pi
|
||||
```
|
||||
|
||||
Claude supported-hook run (latest capture):
|
||||
|
||||
```text
|
||||
hook python pid=4011380 starttime=365920845
|
||||
-> /bin/sh pid=4011379 starttime=365920845
|
||||
-> claude pid=4011285 starttime=365920748
|
||||
-> node/mosaic anchor pid=4011129 starttime=365920380
|
||||
```
|
||||
|
||||
The stream independently recorded the real hook firing:
|
||||
|
||||
```json
|
||||
{"type":"system","subtype":"hook_started","hook_name":"SessionStart:startup","hook_event":"SessionStart"}
|
||||
{"type":"system","subtype":"hook_response","hook_name":"SessionStart:startup","exit_code":0,"outcome":"success","stdout":"...GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED..."}
|
||||
```
|
||||
|
||||
### Authentication and sibling substitution
|
||||
|
||||
The broker minted the logical ID on first peercred contact and keyed the anchor by `(pid,starttime)`. It took the hook/extension PID from `SO_PEERCRED`, walked `/proc/<pid>/stat`, and re-read every starttime before accepting.
|
||||
|
||||
Real Pi acceptance excerpt:
|
||||
|
||||
```json
|
||||
{"peercred":{"pid":4011046,"uid":1001,"gid":1001},"decision":"ACCEPT","reason":"ancestry-reaches-registered-anchor","starttimes_rechecked":true}
|
||||
```
|
||||
|
||||
A separately spawned same-UID sibling was given the real minted victim ID. Its ancestry did not reach the anchor:
|
||||
|
||||
```json
|
||||
{"attacker_uid":1001,"victim_session_id_known":true,"broker_decision":"REJECT","broker_reason":"victim-id-known-but-ancestry-mismatch"}
|
||||
```
|
||||
|
||||
The same rejection occurred in both Pi and Claude runs. This is positive runtime evidence for D1/T15a under the supported non-daemonizing topology.
|
||||
|
||||
---
|
||||
|
||||
## P2 — Pi last-position invariant + nonce map (D5)
|
||||
|
||||
**Verdict: PASS**
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/pi_gate0_run.py
|
||||
python3 docs/compaction-refresh/probes/p2_provider_timing_run.py
|
||||
```
|
||||
|
||||
Full outputs:
|
||||
|
||||
- [`evidence/raw/P2-P3-P5-P6-pi.txt`](./evidence/raw/P2-P3-P5-P6-pi.txt)
|
||||
- [`evidence/raw/P2-provider-timing.txt`](./evidence/raw/P2-provider-timing.txt)
|
||||
|
||||
### Last-or-closed evidence
|
||||
|
||||
Real Pi argv/load order with the probe last:
|
||||
|
||||
```json
|
||||
{"extensions":[".../mosaic-extension.ts",".../pi_gate0_extension.ts"],"lastPosition":true,"gateState":"UNVERIFIED_READY","pid":4004545}
|
||||
```
|
||||
|
||||
A second real Pi launch deliberately appended a later handler:
|
||||
|
||||
```json
|
||||
{"extensions":[".../mosaic-extension.ts",".../pi_gate0_extension.ts",".../pi_later_extension.ts"],"lastPosition":false,"gateState":"CLOSED_NOT_LAST","pid":4005692}
|
||||
```
|
||||
|
||||
Thus the invariant observed is exactly **last or closed**, not an asserted registration order.
|
||||
|
||||
### Exact nonce → tool-call-ID map
|
||||
|
||||
In one real GPT-5.6 Sol Pi response, sequence 5 completed the assistant tool-call message and bound its exact ID:
|
||||
|
||||
```json
|
||||
{"seq":5,"event":"message_end","requestNonce":"e5a82358-a6c9-490b-a0de-2e1f1d9b8d79","toolCallIds":["call_bgGE...57c"],"nonceMappings":[{"toolCallId":"call_bgGE...57c","requestNonce":"e5a82358-a6c9-490b-a0de-2e1f1d9b8d79"}]}
|
||||
```
|
||||
|
||||
The following `tool_call` was sequence 6 and carried the same ID/nonce:
|
||||
|
||||
```json
|
||||
{"seq":6,"event":"tool_call","toolCallId":"call_bgGE...57c","mapping":{"nonce":"e5a82358-a6c9-490b-a0de-2e1f1d9b8d79","verified":true},"allowed":true}
|
||||
```
|
||||
|
||||
The harmless tool executed at sequence 7 with that same tool-call ID. No session-global “current epoch” was borrowed.
|
||||
|
||||
### `after_provider_response` is not assistant-content observation
|
||||
|
||||
A deterministic localhost HTTP provider was used only to force headers/status exposure through the real Pi transport. Actual order:
|
||||
|
||||
```json
|
||||
{"seq":4,"event":"before_provider_request"}
|
||||
{"seq":5,"event":"after_provider_response","status":200,"assistantContentAvailableAtThisHook":false,"timing":"headers/status before stream consumption"}
|
||||
{"seq":6,"event":"message_end","role":"assistant","assistantContentObserved":true}
|
||||
```
|
||||
|
||||
```text
|
||||
headers_hook_precedes_completed_message=True
|
||||
```
|
||||
|
||||
This positively confirms SPEC-v5’s precision correction: receipt content is observed at `message_end`; `after_provider_response` is status/headers before stream consumption.
|
||||
|
||||
---
|
||||
|
||||
## P3 — Same-PID `runtime_generation` bump revokes prior lease (D4)
|
||||
|
||||
**Verdict: PASS**
|
||||
|
||||
### Command
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/pi_gate0_run.py
|
||||
```
|
||||
|
||||
Full output: [`evidence/raw/P2-P3-P5-P6-pi.txt`](./evidence/raw/P2-P3-P5-P6-pi.txt).
|
||||
|
||||
The real Pi process identity remained:
|
||||
|
||||
```text
|
||||
pid=4004545 starttime_ticks=365907677 uid=1001
|
||||
```
|
||||
|
||||
Broker state around reload:
|
||||
|
||||
```json
|
||||
{"event":"runtime_generation_bump","reason":"startup","old_generation":0,"new_generation":1,"new_lease_state":"UNVERIFIED"}
|
||||
{"event":"probe_lease_promoted","generation":1,"new_lease_state":"VERIFIED"}
|
||||
{"event":"runtime_generation_bump","phase":"shutdown","reason":"reload","old_generation":1,"new_generation":2,"prior_lease":"VERIFIED","prior_lease_revoked":true,"new_lease_state":"REVOKED"}
|
||||
{"event":"runtime_generation_bump","phase":"start","reason":"reload","old_generation":2,"new_generation":3,"new_lease_state":"UNVERIFIED"}
|
||||
```
|
||||
|
||||
The same `(pid,starttime)` then emitted monotonic bumps for real `fork`, `new`, and `resume` replacement flows, reaching generation 12. Pi 0.80.7 emitted an additional conservative `session_start` callback in each of those replacement flows; the broker bumped again rather than reusing authority. This is an availability/idempotence consideration for implementation, not a fail-open result.
|
||||
|
||||
---
|
||||
|
||||
## P4 — `SO_PEERCRED` + socket authenticity posture
|
||||
|
||||
**Verdict: PASS, with the spec’s named same-UID T-C residual**
|
||||
|
||||
### Command
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/p4_peercred_probe.py
|
||||
```
|
||||
|
||||
Full output: [`evidence/raw/P4-so-peercred.txt`](./evidence/raw/P4-so-peercred.txt).
|
||||
|
||||
Captured real socket result:
|
||||
|
||||
```text
|
||||
server_pid=4013762 server_uid=1001 server_gid=1001
|
||||
directory_mode=0700 socket_mode=0600
|
||||
SO_PEERCRED pid=4013768 uid=1001 gid=1001
|
||||
client_claim={"pid":4013768,"starttime_ticks":365927069,"uid":1001,...}
|
||||
proc_observed={"pid":4013768,"starttime_ticks":365927069,"uid":1001,...}
|
||||
pid_match=True
|
||||
uid_match=True
|
||||
starttime_match=True
|
||||
client_exit_status=0
|
||||
```
|
||||
|
||||
Achievable unprivileged posture on this host is a user-owned 0700 parent plus 0600 socket. That excludes other UIDs and positively authenticates the connecting kernel PID/UID/GID. It does **not** stop another process running as `hermes` from unlinking/rebinding the socket. A claim stronger than T-C against counterfeit replacement therefore requires the ratified distinct-principal system service or authenticated broker responses. No stronger claim is made.
|
||||
|
||||
---
|
||||
|
||||
## P5 — Source invalidation fail-closed
|
||||
|
||||
**Verdict: PASS**
|
||||
|
||||
### Command
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/pi_gate0_run.py
|
||||
```
|
||||
|
||||
Full output: [`evidence/raw/P2-P3-P5-P6-pi.txt`](./evidence/raw/P2-P3-P5-P6-pi.txt).
|
||||
|
||||
Each fault was injected into the manifest/source read by the real Pi `context` hook. Each run reached an actual model-produced `toolCallId`, then the runtime gate refused it:
|
||||
|
||||
| Fault | Runtime validation | Injection/promotion | Broker | Tool result |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Missing path | `reason=missing` | `injectionDecision=REFUSED`, `promotion=false` | `source_invalidation_revoke` | `allowed=false`, `unverified-source:missing` |
|
||||
| 65 bytes with 64-byte max | `reason=oversize` | `REFUSED`, `promotion=false` | revoked | `allowed=false`, `unverified-source:oversize` |
|
||||
| Bytes differ from pinned SHA-256 | `reason=hash-mismatch` | `REFUSED`, `promotion=false` | revoked | `allowed=false`, `unverified-source:hash-mismatch` |
|
||||
|
||||
Missing example:
|
||||
|
||||
```json
|
||||
{"event":"context_return","sourceValidation":{"ok":false,"reason":"missing"},"injectionDecision":"REFUSED","promotion":false,"sourceBroker":{"event":"source_invalidation_revoke","new_lease_state":"REVOKED"}}
|
||||
{"event":"tool_call","mapping":{"verified":false,"sourceReason":"missing"},"allowed":false,"reason":"unverified-source:missing"}
|
||||
```
|
||||
|
||||
No fault case reached tool execution or promotion.
|
||||
|
||||
---
|
||||
|
||||
## P6 — Atomic Claude `additionalContext` + Pi `context` injection (A-v5-1 / T27)
|
||||
|
||||
**Verdict: T-C GAP — returns to planner**
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
python3 docs/compaction-refresh/probes/pi_gate0_run.py
|
||||
python3 docs/compaction-refresh/probes/p6_claude_run.py
|
||||
rg -n -i "atomic|prefix-preserv" <installed Pi and Claude hook docs>
|
||||
```
|
||||
|
||||
Full outputs:
|
||||
|
||||
- [`evidence/raw/P2-P3-P5-P6-pi.txt`](./evidence/raw/P2-P3-P5-P6-pi.txt)
|
||||
- [`evidence/raw/P6-claude-additional-context.txt`](./evidence/raw/P6-claude-additional-context.txt)
|
||||
- [`evidence/raw/P6-contract-gap.txt`](./evidence/raw/P6-contract-gap.txt)
|
||||
|
||||
### Positive empirical observations
|
||||
|
||||
**Pi:** The real `context` hook returned exactly one additional `AgentMessage`; the prior message prefix hash was unchanged. The real final provider payload contained exactly one occurrence in one content item, and the real model copied all bytes exactly:
|
||||
|
||||
```json
|
||||
{"event":"context_return","inputCount":1,"outputCount":2,"injectionDecision":"ONE_ATOMIC_AGENT_MESSAGE","prefixPreservedByReturn":true,"blockLength":108,"blockSha256":"99c3...a0dd"}
|
||||
{"event":"before_provider_request","markerOccurrences":1,"markerPaths":["$.input[1].content[0].text"],"finalPayloadValid":true}
|
||||
{"event":"message_end","exactContextBlockCopied":true,"assistantTextSha256":"99c3...a0dd"}
|
||||
```
|
||||
|
||||
**Claude:** The real `SessionStart` hook emitted one `hookSpecificOutput.additionalContext` string. Claude’s stream recorded successful hook execution, and the real model’s exact copied block matched byte length and SHA-256:
|
||||
|
||||
```text
|
||||
block_length=116
|
||||
block_sha256=ef6377d63552af075f4f4adec00165988418c5f46a992f4dce8e678b56fd34ac
|
||||
assistant_copy_length=116
|
||||
assistant_copy_sha256=ef6377d63552af075f4f4adec00165988418c5f46a992f4dce8e678b56fd34ac
|
||||
assistant_copy_exact=True
|
||||
```
|
||||
|
||||
### Why this is not a PASS
|
||||
|
||||
The installed Pi documentation says only that `context` receives a deep copy and may return `{ messages }`. The installed Claude documentation says only that `additionalContext` enters/adds to context/system prompt. The exact search result was:
|
||||
|
||||
```text
|
||||
NO MATCH: neither installed runtime document states an atomic/prefix-preserving transport guarantee.
|
||||
```
|
||||
|
||||
One or several successful complete deliveries cannot prove the transport invariant needed by A-v5-1. In particular, a harness-side middle deletion/replacement that preserves the terminal receipt is not detectable by the receipt. That is precisely R1’s assurance boundary. Therefore:
|
||||
|
||||
- absent or prefix-truncated terminal token: receipt-detectable;
|
||||
- middle-drop preserving the tail token: **not receipt-detectable**;
|
||||
- no documented runtime contract excludes that transform;
|
||||
- classification: **T-C contract gap**.
|
||||
|
||||
No atomicity claim is inferred from empirical success.
|
||||
|
||||
---
|
||||
|
||||
## Independent probe review
|
||||
|
||||
After an initial review identified a session-global P2 correlation flaw, the probe was changed to queue request-scoped cycles from `before_provider_request` through assistant `message_end`; all runtime probes were re-run and raw checksums regenerated. The final independent review command was:
|
||||
|
||||
```bash
|
||||
~/.config/mosaic/tools/codex/codex-code-review.sh \
|
||||
-b d801d6c4c8a984d6a95033c49714210018d3d9a8 \
|
||||
-o /tmp/827-gate0-rereview.json
|
||||
```
|
||||
|
||||
Final review: **APPROVE**, confidence 0.91, 18 files reviewed, 0 blockers, 0 should-fix findings, 0 suggestions.
|
||||
|
||||
## Final admission decision
|
||||
|
||||
Gate0 requires every item to produce positive runtime evidence. P6 does not. **Do not admit WI-1..WI-7. Return A-v5-1/T27 to planner review.**
|
||||
|
||||
No feature work, push, PR, merge, or issue closure was performed.
|
||||
8
docs/compaction-refresh/evidence/RAW-SHA256SUMS.txt
Normal file
8
docs/compaction-refresh/evidence/RAW-SHA256SUMS.txt
Normal file
@@ -0,0 +1,8 @@
|
||||
d19ed51612b52d8f5f4957321776e05157008d048b693217c03d71318dc4c763 docs/compaction-refresh/evidence/raw/P1-claude-hook-events.txt
|
||||
c2d7bc21200063a4a0e61c67ba91abaf958ee88aa686e86f3f71c2717732b413 docs/compaction-refresh/evidence/raw/P1-launch-ancestry.txt
|
||||
6efb12d908e9e20badcfda5b070aa1873409bd5a05f533f0b08bb1b4ef53d1a7 docs/compaction-refresh/evidence/raw/P2-P3-P5-P6-pi.txt
|
||||
a9df6cc9f5d45f60d7d914ad1f80b9601574b82831101b3a10eccf1b93787e94 docs/compaction-refresh/evidence/raw/P2-provider-timing.txt
|
||||
92e7aa7d69d53e58a151f9d56cfb583d90c206ecc0bc8a1b185e172c598fb177 docs/compaction-refresh/evidence/raw/P4-so-peercred.txt
|
||||
047d235c6b6553158e27378c4ace081b094e5746734f4b5db6a6dc8ef9e05ff2 docs/compaction-refresh/evidence/raw/P6-claude-additional-context.txt
|
||||
7df20b2878fc87aa4d1fc89121e494d8f4f7bf313b89e1e3147f16fdaa567cdd docs/compaction-refresh/evidence/raw/P6-contract-gap.txt
|
||||
405bf3a06bf355d7f4f4d7b29d45a1ae70d93a690af5f7d0fc4819249e9f408f docs/compaction-refresh/evidence/raw/STEP0-authority-hashes.txt
|
||||
@@ -0,0 +1,28 @@
|
||||
$ python3 docs/compaction-refresh/probes/p1_run.py --runtime claude
|
||||
=== P1 CLAUDE REAL LAUNCH ===
|
||||
$ python3 docs/compaction-refresh/probes/p1_anchor_exec.py --socket <protected-socket> claude <runtime args>
|
||||
registered_anchor={"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933580, "ppid": 3933578, "starttime_ticks": 365788075}
|
||||
broker_minted_session_id=ebe9f9146ad1ba5b9fd757fe9517d24b
|
||||
hook_or_extension_record={"ancestry": [{"argc": 2, "argv0": "python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933983, "ppid": 3933982, "starttime_ticks": 365788568}, {"argc": 3, "argv0": "/bin/sh", "comm": "sh", "exe": "/usr/bin/dash", "pid": 3933982, "ppid": 3933751, "starttime_ticks": 365788568}, {"argc": 16, "argv0": "claude", "comm": "claude", "exe": "/home/hermes/.local/share/claude/versions/2.1.205", "pid": 3933751, "ppid": 3933580, "starttime_ticks": 365788474}, {"argc": 16, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 3933580, "ppid": 3933578, "starttime_ticks": 365788075}], "anchor": {"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933580, "ppid": 3933578, "starttime_ticks": 365788075}, "claimed_session_id": null, "decision": "ACCEPT", "event": "resolve-hook", "peercred": {"gid": 1001, "pid": 3933983, "uid": 1001}, "reason": "ancestry-reaches-registered-anchor", "resolved_session_id": "ebe9f9146ad1ba5b9fd757fe9517d24b", "starttimes_rechecked": true}
|
||||
sibling_attack_record={"ancestry": [{"argc": 6, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933581, "ppid": 3933578, "starttime_ticks": 365788080}, {"argc": 4, "argv0": "python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933578, "ppid": 3933576, "starttime_ticks": 365788059}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 3933576, "ppid": 3933575, "starttime_ticks": 365788058}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 3933575, "ppid": 3888118, "starttime_ticks": 365788058}, {"argc": 1, "argv0": "pi", "comm": "pi", "exe": "/usr/bin/node", "pid": 3888118, "ppid": 3887912, "starttime_ticks": 365707392}, {"argc": 6, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 3887912, "ppid": 3887869, "starttime_ticks": 365707050}, {"argc": 1, "argv0": "-bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 3887869, "ppid": 1244054, "starttime_ticks": 365706948}, {"argc": 10, "argv0": "tmux", "comm": "tmux: server", "exe": "/usr/bin/tmux", "pid": 1244054, "ppid": 745, "starttime_ticks": 114078803}, {"argc": 2, "argv0": "/lib/systemd/systemd", "comm": "systemd", "exe": "/usr/lib/systemd/systemd", "pid": 745, "ppid": 1, "starttime_ticks": 627}], "anchor": {"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 3933580, "ppid": 3933578, "starttime_ticks": 365788075}, "claimed_session_id": "ebe9f9146ad1ba5b9fd757fe9517d24b", "decision": "REJECT", "event": "claim-session", "peercred": {"gid": 1001, "pid": 3933581, "uid": 1001}, "reason": "victim-id-known-but-ancestry-mismatch", "resolved_session_id": null, "starttimes_rechecked": false}
|
||||
sibling_process_stdout={"attacker_pid": 3933581, "attacker_uid": 1001, "broker_decision": "REJECT", "broker_reason": "victim-id-known-but-ancestry-mismatch", "victim_session_id_known": true}
|
||||
sibling_process_exit=0
|
||||
ps_snapshot=<hook chain exited; broker /proc snapshot above is authoritative>
|
||||
launcher_stderr_excerpt:
|
||||
{"argv": ["mosaic", "yolo", "claude", "<12 runtime args>"], "event": "anchor-exec", "note": "os.execvpe retains pid and /proc starttime", "pid": 3933580}
|
||||
runtime_stdout_excerpt:
|
||||
|
||||
[mosaic] Claude Code settings audit:
|
||||
⚠ Missing PreToolUse hook: prevent-memory-write.sh
|
||||
⚠ Missing PostToolUse hook: qa-hook-stdin.sh
|
||||
⚠ Missing PostToolUse hook: typecheck-hook.sh
|
||||
⚠ Missing plugin: feature-dev
|
||||
⚠ Missing plugin: pr-review-toolkit
|
||||
⚠ Missing plugin: code-review
|
||||
runtime_hook_event_excerpt:
|
||||
⚠ Missing PreToolUse hook: prevent-memory-write.sh
|
||||
⚠ Missing PostToolUse hook: qa-hook-stdin.sh
|
||||
⚠ Missing PostToolUse hook: typecheck-hook.sh
|
||||
{"type":"system","subtype":"hook_started","hook_id":"cadd5ded-a869-4b05-85fc-cfd1a4988217","hook_name":"SessionStart:startup","hook_event":"SessionStart","uuid":"b63d67bf-2247-4e1c-b16b-7ccffa73180b","session_id":"97e1224c-7c1c-42c7-9fb5-598d2cd3dfaf"}
|
||||
{"type":"system","subtype":"hook_response","hook_id":"cadd5ded-a869-4b05-85fc-cfd1a4988217","hook_name":"SessionStart:startup","hook_event":"SessionStart","output":"{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED\"}}\n","stdout":"{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED\"}}\n","stderr":"","exit_code":0,"outcome":"success","uuid":"b2bb583d-d287-4b56-8061-09153a32adc2","session_id":"97e1224c-7c1c-42c7-9fb5-598d2cd3dfaf"}
|
||||
|
||||
62
docs/compaction-refresh/evidence/raw/P1-launch-ancestry.txt
Normal file
62
docs/compaction-refresh/evidence/raw/P1-launch-ancestry.txt
Normal file
@@ -0,0 +1,62 @@
|
||||
$ python3 docs/compaction-refresh/probes/p1_run.py --runtime both
|
||||
=== P1 PI REAL LAUNCH ===
|
||||
machine_assertions=PASS
|
||||
$ python3 docs/compaction-refresh/probes/p1_anchor_exec.py --socket <protected-socket> pi <runtime args>
|
||||
registered_anchor={"argc": 13, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010843, "ppid": 4010840, "starttime_ticks": 365919858}
|
||||
broker_minted_session_id=5207bd0d8251b616fe4df4c68f438830
|
||||
hook_or_extension_record={"ancestry": [{"argc": 1, "argv0": "pi", "comm": "pi", "exe": "/usr/bin/node", "pid": 4011046, "ppid": 4010843, "starttime_ticks": 365920219}, {"argc": 12, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 4010843, "ppid": 4010840, "starttime_ticks": 365919858}], "anchor": {"argc": 13, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010843, "ppid": 4010840, "starttime_ticks": 365919858}, "claimed_session_id": null, "decision": "ACCEPT", "event": "resolve-hook", "peercred": {"gid": 1001, "pid": 4011046, "uid": 1001}, "reason": "ancestry-reaches-registered-anchor", "resolved_session_id": "5207bd0d8251b616fe4df4c68f438830", "starttimes_rechecked": true}
|
||||
sibling_attack_record={"ancestry": [{"argc": 6, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010844, "ppid": 4010840, "starttime_ticks": 365919864}, {"argc": 4, "argv0": "python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010840, "ppid": 4010838, "starttime_ticks": 365919843}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 4010838, "ppid": 4010837, "starttime_ticks": 365919843}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 4010837, "ppid": 3888118, "starttime_ticks": 365919842}, {"argc": 1, "argv0": "pi", "comm": "pi", "exe": "/usr/bin/node", "pid": 3888118, "ppid": 3887912, "starttime_ticks": 365707392}, {"argc": 6, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 3887912, "ppid": 3887869, "starttime_ticks": 365707050}, {"argc": 1, "argv0": "-bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 3887869, "ppid": 1244054, "starttime_ticks": 365706948}, {"argc": 10, "argv0": "tmux", "comm": "tmux: server", "exe": "/usr/bin/tmux", "pid": 1244054, "ppid": 745, "starttime_ticks": 114078803}, {"argc": 2, "argv0": "/lib/systemd/systemd", "comm": "systemd", "exe": "/usr/lib/systemd/systemd", "pid": 745, "ppid": 1, "starttime_ticks": 627}], "anchor": {"argc": 13, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010843, "ppid": 4010840, "starttime_ticks": 365919858}, "claimed_session_id": "5207bd0d8251b616fe4df4c68f438830", "decision": "REJECT", "event": "claim-session", "peercred": {"gid": 1001, "pid": 4010844, "uid": 1001}, "reason": "victim-id-known-but-ancestry-mismatch", "resolved_session_id": null, "starttimes_rechecked": false}
|
||||
sibling_process_stdout={"attacker_pid": 4010844, "attacker_uid": 1001, "broker_decision": "REJECT", "broker_reason": "victim-id-known-but-ancestry-mismatch", "victim_session_id_known": true}
|
||||
sibling_process_exit=0
|
||||
$ ps -o pid=,ppid=,lstart=,uid=,gid=,comm= -p 4011046,4010843
|
||||
4010843 4010840 Fri Jul 17 19:35:18 2026 1001 1001 node
|
||||
4011046 4010843 Fri Jul 17 19:35:22 2026 1001 1001 pi
|
||||
launcher_stderr_excerpt:
|
||||
{"argv": ["mosaic", "yolo", "pi", "<8 runtime args>"], "event": "anchor-exec", "note": "os.execvpe retains pid and /proc starttime", "pid": 4010843}
|
||||
runtime_stdout_excerpt:
|
||||
[mosaic] Launching Pi in YOLO mode...
|
||||
{"type":"extension_ui_request","id":"4cf086c7-299e-4734-9264-6ad2964f3664","method":"notify","message":"Mosaic framework loaded","notifyType":"info"}
|
||||
{"id":"state","type":"response","command":"get_state","success":true,"data":{"model":{"id":"gpt-5.6-sol","name":"GPT-5.6 Sol","api":"openai-codex-responses","provider":"openai-codex","baseUrl":"https://chatgpt.com/backend-api","compat":{"supportsToolSearch":true},"reasoning":true,"thinkingLevelMap":{"xhigh":"xhigh","max":"max","minimal":"low"},"input":["text","image"],"cost":{"input":5,"output":30,"cacheRead":0.5,"cacheWrite":6.25,"tiers":[{"inputTokensAbove":272000,"input":10,"output":45,"cache
|
||||
runtime_hook_event_excerpt:
|
||||
|
||||
=== P1 CLAUDE REAL LAUNCH ===
|
||||
machine_assertions=PASS
|
||||
$ python3 docs/compaction-refresh/probes/p1_anchor_exec.py --socket <protected-socket> claude <runtime args>
|
||||
registered_anchor={"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4011129, "ppid": 4010840, "starttime_ticks": 365920380}
|
||||
broker_minted_session_id=f382fa5f4b2142ef79bb76204521ff2a
|
||||
hook_or_extension_record={"ancestry": [{"argc": 2, "argv0": "python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4011380, "ppid": 4011379, "starttime_ticks": 365920845}, {"argc": 3, "argv0": "/bin/sh", "comm": "sh", "exe": "/usr/bin/dash", "pid": 4011379, "ppid": 4011285, "starttime_ticks": 365920845}, {"argc": 16, "argv0": "claude", "comm": "claude", "exe": "/home/hermes/.local/share/claude/versions/2.1.205", "pid": 4011285, "ppid": 4011129, "starttime_ticks": 365920748}, {"argc": 16, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 4011129, "ppid": 4010840, "starttime_ticks": 365920380}], "anchor": {"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4011129, "ppid": 4010840, "starttime_ticks": 365920380}, "claimed_session_id": null, "decision": "ACCEPT", "event": "resolve-hook", "peercred": {"gid": 1001, "pid": 4011380, "uid": 1001}, "reason": "ancestry-reaches-registered-anchor", "resolved_session_id": "f382fa5f4b2142ef79bb76204521ff2a", "starttimes_rechecked": true}
|
||||
sibling_attack_record={"ancestry": [{"argc": 6, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4011130, "ppid": 4010840, "starttime_ticks": 365920385}, {"argc": 4, "argv0": "python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4010840, "ppid": 4010838, "starttime_ticks": 365919843}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 4010838, "ppid": 4010837, "starttime_ticks": 365919843}, {"argc": 3, "argv0": "/bin/bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 4010837, "ppid": 3888118, "starttime_ticks": 365919842}, {"argc": 1, "argv0": "pi", "comm": "pi", "exe": "/usr/bin/node", "pid": 3888118, "ppid": 3887912, "starttime_ticks": 365707392}, {"argc": 6, "argv0": "node", "comm": "node", "exe": "/usr/bin/node", "pid": 3887912, "ppid": 3887869, "starttime_ticks": 365707050}, {"argc": 1, "argv0": "-bash", "comm": "bash", "exe": "/usr/bin/bash", "pid": 3887869, "ppid": 1244054, "starttime_ticks": 365706948}, {"argc": 10, "argv0": "tmux", "comm": "tmux: server", "exe": "/usr/bin/tmux", "pid": 1244054, "ppid": 745, "starttime_ticks": 114078803}, {"argc": 2, "argv0": "/lib/systemd/systemd", "comm": "systemd", "exe": "/usr/lib/systemd/systemd", "pid": 745, "ppid": 1, "starttime_ticks": 627}], "anchor": {"argc": 17, "argv0": "/usr/bin/python3", "comm": "python3", "exe": "/usr/bin/python3.11", "pid": 4011129, "ppid": 4010840, "starttime_ticks": 365920380}, "claimed_session_id": "f382fa5f4b2142ef79bb76204521ff2a", "decision": "REJECT", "event": "claim-session", "peercred": {"gid": 1001, "pid": 4011130, "uid": 1001}, "reason": "victim-id-known-but-ancestry-mismatch", "resolved_session_id": null, "starttimes_rechecked": false}
|
||||
sibling_process_stdout={"attacker_pid": 4011130, "attacker_uid": 1001, "broker_decision": "REJECT", "broker_reason": "victim-id-known-but-ancestry-mismatch", "victim_session_id_known": true}
|
||||
sibling_process_exit=0
|
||||
ps_snapshot=<hook chain exited; broker /proc snapshot above is authoritative>
|
||||
launcher_stderr_excerpt:
|
||||
{"argv": ["mosaic", "yolo", "claude", "<12 runtime args>"], "event": "anchor-exec", "note": "os.execvpe retains pid and /proc starttime", "pid": 4011129}
|
||||
runtime_stdout_excerpt:
|
||||
|
||||
[mosaic] Claude Code settings audit:
|
||||
⚠ Missing PreToolUse hook: prevent-memory-write.sh
|
||||
⚠ Missing PostToolUse hook: qa-hook-stdin.sh
|
||||
⚠ Missing PostToolUse hook: typecheck-hook.sh
|
||||
⚠ Missing plugin: feature-dev
|
||||
⚠ Missing plugin: pr-review-toolkit
|
||||
⚠ Missing plugin: code-review
|
||||
runtime_hook_event_excerpt:
|
||||
⚠ Missing PreToolUse hook: prevent-memory-write.sh
|
||||
⚠ Missing PostToolUse hook: qa-hook-stdin.sh
|
||||
⚠ Missing PostToolUse hook: typecheck-hook.sh
|
||||
{"type":"system","subtype":"hook_started","hook_id":"2a5f7dab-a064-4610-a6b1-4ad151ddcdd9","hook_name":"SessionStart:startup","hook_event":"SessionStart","uuid":"c6e0690c-f0c9-4d60-a8fb-5f0c25ea3208","session_id":"167d104d-907a-4120-9b07-bdf4762818a9"}
|
||||
{"type":"system","subtype":"hook_response","hook_id":"2a5f7dab-a064-4610-a6b1-4ad151ddcdd9","hook_name":"SessionStart:startup","hook_event":"SessionStart","output":"{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED\"}}\n","stdout":"{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED\"}}\n","stderr":"","exit_code":0,"outcome":"success","uuid":"167b2a5b-a45a-4e70-a72b-1f4a609bb979","session_id":"167d104d-907a-4120-9b07-bdf4762818a9"}
|
||||
|
||||
|
||||
$ readlink -f "$(command -v mosaic)"
|
||||
/home/hermes/.npm-global/lib/node_modules/@mosaicstack/mosaic/dist/cli.js
|
||||
|
||||
$ rg -n "spawnSync|execRuntime" ~/.npm-global/lib/node_modules/@mosaicstack/mosaic/dist/commands/launch.js | tail -8
|
||||
63: spawnSync(initBin, [], { stdio: 'inherit' });
|
||||
131: const result = spawnSync(checker, ['--check', '--runtime', runtime], { stdio: 'ignore' });
|
||||
624: execRuntime('claude', cliArgs);
|
||||
637: execRuntime('codex', cliArgs);
|
||||
643: execRuntime('opencode', args);
|
||||
658: execRuntime('pi', cliArgs);
|
||||
665:function execRuntime(cmd, args) {
|
||||
668: const result = spawnSync(cmd, args, {
|
||||
52
docs/compaction-refresh/evidence/raw/P2-P3-P5-P6-pi.txt
Normal file
52
docs/compaction-refresh/evidence/raw/P2-P3-P5-P6-pi.txt
Normal file
@@ -0,0 +1,52 @@
|
||||
$ python3 docs/compaction-refresh/probes/pi_gate0_run.py
|
||||
machine_assertions=PASS
|
||||
runtime_versions:
|
||||
0.80.7
|
||||
0.0.48
|
||||
|
||||
P2_EVENT_ORDER_AND_NONCE_MAP:
|
||||
{"assistantContentObserved": true, "assistantTextSha256": "a36f1eb364f062cad2f9f7d7e2b62ef7715d2aef79caafcfccd3a227cecf3e61", "event": "message_end", "exactContextBlockCopied": false, "inFlightDepthAfter": 0, "nonceMappings": [{"requestNonce": "e5a82358-a6c9-490b-a0de-2e1f1d9b8d79", "toolCallId": "call_bgGEFnBJOmwJPEfmzMo1eHOy|fc_0fb3d12b5404a73c016a5ac9d6f9a4819b9ddf70296c0cf57c"}], "pid": 4004545, "requestNonce": "e5a82358-a6c9-490b-a0de-2e1f1d9b8d79", "role": "assistant", "seq": 5, "starttime_ticks": 365907677, "toolCallIds": ["call_bgGEFnBJOmwJPEfmzMo1eHOy|fc_0fb3d12b5404a73c016a5ac9d6f9a4819b9ddf70296c0cf57c"]}
|
||||
{"allowed": true, "event": "tool_call", "mapping": {"nonce": "e5a82358-a6c9-490b-a0de-2e1f1d9b8d79", "sourceReason": "all-fragments-valid", "verified": true}, "pid": 4004545, "reason": "exact-tool-call-id-mapped-to-verified-request-nonce", "seq": 6, "starttime_ticks": 365907677, "toolCallId": "call_bgGEFnBJOmwJPEfmzMo1eHOy|fc_0fb3d12b5404a73c016a5ac9d6f9a4819b9ddf70296c0cf57c", "toolName": "gate0_nonce_probe"}
|
||||
{"broker": {"event": "probe_lease_promoted", "generation": 1, "new_lease_state": "VERIFIED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "starttime_ticks": 365907677}, "event": "tool_execute", "label": "p2", "pid": 4004545, "seq": 7, "starttime_ticks": 365907677, "toolCallId": "call_bgGEFnBJOmwJPEfmzMo1eHOy|fc_0fb3d12b5404a73c016a5ac9d6f9a4819b9ddf70296c0cf57c"}
|
||||
{"assistantContentObserved": true, "assistantTextSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "message_end", "exactContextBlockCopied": true, "inFlightDepthAfter": 0, "nonceMappings": [], "pid": 4004545, "requestNonce": "cca4b1e3-296a-4e4c-9805-a395c270c01f", "role": "assistant", "seq": 11, "starttime_ticks": 365907677, "toolCallIds": []}
|
||||
|
||||
P2_LAST_OR_CLOSED:
|
||||
{"broker": {"event": "runtime_generation_bump", "new_generation": 1, "new_lease_state": "UNVERIFIED", "old_generation": 0, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "NONE", "prior_lease_revoked": true, "reason": "startup", "starttime_ticks": 365907677}, "event": "session_start", "extensions": ["/home/hermes/.config/mosaic/runtime/pi/mosaic-extension.ts", "/home/hermes/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/pi_gate0_extension.ts"], "gateState": "UNVERIFIED_READY", "lastPosition": true, "pid": 4004545, "reason": "startup", "self": "/home/hermes/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/pi_gate0_extension.ts", "seq": 1, "starttime_ticks": 365907677}
|
||||
{"broker": {"skipped": true}, "event": "session_start", "extensions": ["/home/hermes/.config/mosaic/runtime/pi/mosaic-extension.ts", "/home/hermes/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/pi_gate0_extension.ts", "/home/hermes/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/pi_later_extension.ts"], "gateState": "CLOSED_NOT_LAST", "lastPosition": false, "pid": 4005692, "reason": "startup", "self": "/home/hermes/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/pi_gate0_extension.ts", "seq": 1, "starttime_ticks": 365910545}
|
||||
|
||||
P3_GENERATION_BROKER:
|
||||
{"event": "runtime_generation_bump", "new_generation": 1, "new_lease_state": "UNVERIFIED", "old_generation": 0, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "NONE", "prior_lease_revoked": true, "reason": "startup", "starttime_ticks": 365907677}
|
||||
{"event": "probe_lease_promoted", "generation": 1, "new_lease_state": "VERIFIED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 2, "new_lease_state": "REVOKED", "old_generation": 1, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "shutdown", "prior_lease": "VERIFIED", "prior_lease_revoked": true, "reason": "reload", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 3, "new_lease_state": "UNVERIFIED", "old_generation": 2, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "REVOKED", "prior_lease_revoked": true, "reason": "reload", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 4, "new_lease_state": "REVOKED", "old_generation": 3, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "shutdown", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "fork", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 5, "new_lease_state": "UNVERIFIED", "old_generation": 4, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "REVOKED", "prior_lease_revoked": true, "reason": "fork", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 6, "new_lease_state": "UNVERIFIED", "old_generation": 5, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "fork", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 7, "new_lease_state": "REVOKED", "old_generation": 6, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "shutdown", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "new", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 8, "new_lease_state": "UNVERIFIED", "old_generation": 7, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "REVOKED", "prior_lease_revoked": true, "reason": "new", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 9, "new_lease_state": "UNVERIFIED", "old_generation": 8, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "new", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 10, "new_lease_state": "REVOKED", "old_generation": 9, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "shutdown", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "resume", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 11, "new_lease_state": "UNVERIFIED", "old_generation": 10, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "REVOKED", "prior_lease_revoked": true, "reason": "resume", "starttime_ticks": 365907677}
|
||||
{"event": "runtime_generation_bump", "new_generation": 12, "new_lease_state": "UNVERIFIED", "old_generation": 11, "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "phase": "start", "prior_lease": "UNVERIFIED", "prior_lease_revoked": true, "reason": "resume", "starttime_ticks": 365907677}
|
||||
|
||||
P5_SOURCE_INVALIDATION:
|
||||
{"blockLength": 108, "blockSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "context_return", "injectionDecision": "REFUSED", "inputCount": 5, "lastPosition": true, "outputCount": 5, "pid": 4004545, "prefixHashAfter": "4f339e3e45989486374b75d8a40abad22a3f3091f1099e3b4112f3afd1c60eb0", "prefixHashBefore": "4f339e3e45989486374b75d8a40abad22a3f3091f1099e3b4112f3afd1c60eb0", "prefixPreservedByReturn": true, "promotion": false, "requestNonce": "24ef5352-bcc6-4418-b65f-c2763453cc46", "seq": 12, "sourceBroker": {"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "UNVERIFIED", "promotion": false, "source_reason": "missing", "starttime_ticks": 365907677}, "sourceValidation": {"fragment": "/tmp/gate0-pi-g_3gsk34/absent-fragment.md", "ok": false, "reason": "missing"}, "starttime_ticks": 365907677}
|
||||
{"allowed": false, "event": "tool_call", "mapping": {"nonce": "24ef5352-bcc6-4418-b65f-c2763453cc46", "sourceReason": "missing", "verified": false}, "pid": 4004545, "reason": "unverified-source:missing", "seq": 15, "starttime_ticks": 365907677, "toolCallId": "call_XBwBkiv5tZx2ayuDxHKD5vSB|fc_0fb3d12b5404a73c016a5ac9dc341c819bb6ed3e00ca2cf1a5", "toolName": "gate0_nonce_probe"}
|
||||
{"blockLength": 108, "blockSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "context_return", "injectionDecision": "REFUSED", "inputCount": 9, "lastPosition": true, "outputCount": 9, "pid": 4004545, "prefixHashAfter": "1a39911018caefe8f5b5acb652cece9f92d937e7384109f5c1559266349480b7", "prefixHashBefore": "1a39911018caefe8f5b5acb652cece9f92d937e7384109f5c1559266349480b7", "prefixPreservedByReturn": true, "promotion": false, "requestNonce": "319646e8-59e2-4021-9b27-de1376b13c32", "seq": 22, "sourceBroker": {"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "REVOKED", "promotion": false, "source_reason": "oversize", "starttime_ticks": 365907677}, "sourceValidation": {"fragment": "/tmp/gate0-pi-g_3gsk34/oversize.md", "ok": false, "reason": "oversize"}, "starttime_ticks": 365907677}
|
||||
{"allowed": false, "event": "tool_call", "mapping": {"nonce": "319646e8-59e2-4021-9b27-de1376b13c32", "sourceReason": "oversize", "verified": false}, "pid": 4004545, "reason": "unverified-source:oversize", "seq": 25, "starttime_ticks": 365907677, "toolCallId": "call_P9d0wR5TSXSqZHydVHclh6Cg|fc_0fb3d12b5404a73c016a5ac9e02a28819bb39df373b7c9e23b", "toolName": "gate0_nonce_probe"}
|
||||
{"blockLength": 108, "blockSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "context_return", "injectionDecision": "REFUSED", "inputCount": 13, "lastPosition": true, "outputCount": 13, "pid": 4004545, "prefixHashAfter": "224c8777dd0cd5fcf1ae02f0fc46198548b48647dbfd044ed131533d72086f16", "prefixHashBefore": "224c8777dd0cd5fcf1ae02f0fc46198548b48647dbfd044ed131533d72086f16", "prefixPreservedByReturn": true, "promotion": false, "requestNonce": "23f3125f-6e62-4f3c-aa60-3eaed705ddc1", "seq": 32, "sourceBroker": {"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "REVOKED", "promotion": false, "source_reason": "hash-mismatch", "starttime_ticks": 365907677}, "sourceValidation": {"fragment": "/tmp/gate0-pi-g_3gsk34/mismatch.md", "ok": false, "reason": "hash-mismatch"}, "starttime_ticks": 365907677}
|
||||
{"allowed": false, "event": "tool_call", "mapping": {"nonce": "23f3125f-6e62-4f3c-aa60-3eaed705ddc1", "sourceReason": "hash-mismatch", "verified": false}, "pid": 4004545, "reason": "unverified-source:hash-mismatch", "seq": 35, "starttime_ticks": 365907677, "toolCallId": "call_QUMvqBRnzv6HNqEd37jU5NUw|fc_0fb3d12b5404a73c016a5ac9e37510819b8506879faf287aa3", "toolName": "gate0_nonce_probe"}
|
||||
{"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "UNVERIFIED", "promotion": false, "source_reason": "missing", "starttime_ticks": 365907677}
|
||||
{"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "REVOKED", "promotion": false, "source_reason": "oversize", "starttime_ticks": 365907677}
|
||||
{"event": "source_invalidation_revoke", "generation": 12, "new_lease_state": "REVOKED", "peercred": {"gid": 1001, "pid": 4004545, "uid": 1001}, "prior_lease": "REVOKED", "promotion": false, "source_reason": "hash-mismatch", "starttime_ticks": 365907677}
|
||||
|
||||
P6_PI_CONTEXT_ATOMIC_OBSERVATION:
|
||||
{"blockLength": 108, "blockSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "context_return", "injectionDecision": "ONE_ATOMIC_AGENT_MESSAGE", "inputCount": 1, "lastPosition": true, "outputCount": 2, "pid": 4004545, "prefixHashAfter": "095a5415879b0d4006d1485dba3398fee6bf39850711ba0dc2e9cfa312e865dc", "prefixHashBefore": "095a5415879b0d4006d1485dba3398fee6bf39850711ba0dc2e9cfa312e865dc", "prefixPreservedByReturn": true, "promotion": false, "requestNonce": "e5a82358-a6c9-490b-a0de-2e1f1d9b8d79", "seq": 3, "sourceBroker": {"action": "none", "reason": "source-valid"}, "sourceValidation": {"ok": true, "reason": "all-fragments-valid"}, "starttime_ticks": 365907677}
|
||||
{"event": "before_provider_request", "finalPayloadValid": true, "inFlightDepth": 1, "markerOccurrences": 1, "markerPaths": ["$.input[1].content[0].text"], "pid": 4004545, "requestNonce": "e5a82358-a6c9-490b-a0de-2e1f1d9b8d79", "seq": 4, "starttime_ticks": 365907677}
|
||||
{"blockLength": 108, "blockSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "context_return", "injectionDecision": "ONE_ATOMIC_AGENT_MESSAGE", "inputCount": 3, "lastPosition": true, "outputCount": 4, "pid": 4004545, "prefixHashAfter": "7c40cce3664af7581b21ea007a4a44764237fce1e4f16b031e96d60df2229855", "prefixHashBefore": "7c40cce3664af7581b21ea007a4a44764237fce1e4f16b031e96d60df2229855", "prefixPreservedByReturn": true, "promotion": false, "requestNonce": "cca4b1e3-296a-4e4c-9805-a395c270c01f", "seq": 9, "sourceBroker": {"action": "none", "reason": "source-valid"}, "sourceValidation": {"ok": true, "reason": "all-fragments-valid"}, "starttime_ticks": 365907677}
|
||||
{"event": "before_provider_request", "finalPayloadValid": true, "inFlightDepth": 1, "markerOccurrences": 1, "markerPaths": ["$.input[5].content[0].text"], "pid": 4004545, "requestNonce": "cca4b1e3-296a-4e4c-9805-a395c270c01f", "seq": 10, "starttime_ticks": 365907677}
|
||||
{"assistantContentObserved": true, "assistantTextSha256": "99c3dce194b16405dfb555f126ee5ccc014fdc184d0882aee1a903cbc700a0dd", "event": "message_end", "exactContextBlockCopied": true, "inFlightDepthAfter": 0, "nonceMappings": [], "pid": 4004545, "requestNonce": "cca4b1e3-296a-4e4c-9805-a395c270c01f", "role": "assistant", "seq": 11, "starttime_ticks": 365907677, "toolCallIds": []}
|
||||
|
||||
RPC_EVENT_COUNTS:
|
||||
{"agent_end": 4, "agent_settled": 4, "agent_start": 4, "extension_ui_request": 8, "message_end": 16, "message_start": 16, "message_update": 105, "response": 9, "tool_execution_end": 4, "tool_execution_start": 4, "turn_end": 8, "turn_start": 8}
|
||||
stderr_nonempty=False
|
||||
@@ -0,0 +1,9 @@
|
||||
$ python3 docs/compaction-refresh/probes/p2_provider_timing_run.py
|
||||
local_http_endpoint=http://127.0.0.1:42823/v1/chat/completions
|
||||
{"event": "before_provider_request", "finalPayloadValid": true, "inFlightDepth": 1, "markerOccurrences": 1, "markerPaths": ["$.messages[2].content[0].text"], "pid": 4008778, "requestNonce": "056b82c4-36eb-420b-94cf-b2c73813ef79", "seq": 4, "starttime_ticks": 365916346}
|
||||
{"assistantContentAvailableAtThisHook": false, "event": "after_provider_response", "pid": 4008778, "requestNonce": "056b82c4-36eb-420b-94cf-b2c73813ef79", "seq": 5, "starttime_ticks": 365916346, "status": 200, "timing": "headers/status before stream consumption"}
|
||||
{"assistantContentObserved": true, "assistantTextSha256": "fb4ebaab26d63661040dc15925a99e22dc07ee2b33df5c6b2ca93a5b34f08b1d", "event": "message_end", "exactContextBlockCopied": false, "inFlightDepthAfter": 0, "nonceMappings": [], "pid": 4008778, "requestNonce": "056b82c4-36eb-420b-94cf-b2c73813ef79", "role": "assistant", "seq": 6, "starttime_ticks": 365916346, "toolCallIds": []}
|
||||
machine_assertions=PASS
|
||||
after_provider_response_seq=5
|
||||
message_end_seq=6
|
||||
headers_hook_precedes_completed_message=True
|
||||
23
docs/compaction-refresh/evidence/raw/P4-so-peercred.txt
Normal file
23
docs/compaction-refresh/evidence/raw/P4-so-peercred.txt
Normal file
@@ -0,0 +1,23 @@
|
||||
$ python3 docs/compaction-refresh/probes/p4_peercred_probe.py
|
||||
machine_assertions=PASS
|
||||
server_pid=4013762 server_uid=1001 server_gid=1001
|
||||
socket_path=/tmp/gate0-p4-nl1_8ap2/broker.sock
|
||||
directory_mode=0700 socket_mode=0600
|
||||
SO_PEERCRED pid=4013768 uid=1001 gid=1001
|
||||
client_claim={"exe": "/usr/bin/python3.11", "pid": 4013768, "ppid": 4013762, "starttime_ticks": 365927069, "uid": 1001}
|
||||
proc_observed={"exe": "/usr/bin/python3.11", "pid": 4013768, "ppid": 4013762, "starttime_ticks": 365927069, "uid": 1001}
|
||||
pid_match=True
|
||||
uid_match=True
|
||||
starttime_match=True
|
||||
client_exit_status=0
|
||||
same_principal_socket=true
|
||||
posture=0700 parent + 0600 socket excludes other UIDs, but does not prevent the same UID from unlinking/rebinding; distinct-principal system service remains required for a claim stronger than T-C against same-UID counterfeit replacement
|
||||
|
||||
$ id
|
||||
uid=1001(hermes) gid=1001(hermes) groups=1001(hermes),40(src),100(users),996(docker)
|
||||
|
||||
$ uname -srmo
|
||||
Linux 6.1.0-48-amd64 x86_64 GNU/Linux
|
||||
|
||||
$ getconf CLK_TCK
|
||||
100
|
||||
@@ -0,0 +1,21 @@
|
||||
$ python3 docs/compaction-refresh/probes/p6_claude_run.py
|
||||
machine_assertions=PASS
|
||||
command=mosaic yolo claude --settings <isolated> --model haiku --print --output-format stream-json --verbose --include-hook-events <prompt>
|
||||
claude_version=2.1.205 (Claude Code)
|
||||
mosaic_version=0.0.48
|
||||
exit_code=0
|
||||
hook_process_log={"block_length": 116, "block_sha256": "ef6377d63552af075f4f4adec00165988418c5f46a992f4dce8e678b56fd34ac", "emission": "one hookSpecificOutput.additionalContext string field", "hook_event_name": "SessionStart", "pid": 4015703, "ppid": 4015701, "starttime_ticks": 365930489}
|
||||
hook_stream_event={"hook_event": "SessionStart", "hook_id": "557d613e-574f-4523-8bfb-8c6e51946035", "hook_name": "SessionStart:startup", "session_id": "f821d0db-1177-4237-8ff5-83b2a46996a6", "subtype": "hook_started", "type": "system", "uuid": "59449134-e2d4-43a9-9d34-b59e74622c08"}
|
||||
hook_stream_event={"exit_code": 0, "hook_event": "SessionStart", "hook_id": "557d613e-574f-4523-8bfb-8c6e51946035", "hook_name": "SessionStart:startup", "outcome": "success", "output": "{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_CLAUDE_ATOMIC_BEGIN\\nsegment-01=alpha-2d11\\nsegment-02=middle-8e22\\nsegment-03=omega-4f33\\nGATE0_CLAUDE_ATOMIC_END\"}}\n", "session_id": "f821d0db-1177-4237-8ff5-83b2a46996a6", "stderr": "", "stdout": "{\"hookSpecificOutput\": {\"hookEventName\": \"SessionStart\", \"additionalContext\": \"GATE0_CLAUDE_ATOMIC_BEGIN\\nsegment-01=alpha-2d11\\nsegment-02=middle-8e22\\nsegment-03=omega-4f33\\nGATE0_CLAUDE_ATOMIC_END\"}}\n", "subtype": "hook_response", "type": "system", "uuid": "e05842c1-813a-41dd-93c9-768eb834f260"}
|
||||
block_length=116
|
||||
block_sha256=ef6377d63552af075f4f4adec00165988418c5f46a992f4dce8e678b56fd34ac
|
||||
stream_fields_containing_full_block=3
|
||||
stream_fields_exactly_equal_block=2
|
||||
assistant_copy_length=449
|
||||
assistant_copy_sha256=a65febbb3ad8fa4952894d94406a83a520ef712c0ecb9ab43be3212094b21ba1
|
||||
assistant_copy_exact=False
|
||||
assistant_copy="The user is asking me to return the exact GATE0_CLAUDE_ATOMIC block that was injected by SessionStart. This block was provided in the system-reminder at the beginning of the conversation:\n\n```\nGATE0_CLAUDE_ATOMIC_BEGIN\nsegment-01=alpha-2d11\nsegment-02=middle-8e22\nsegment-03=omega-4f33\nGATE0_CLAUDE_ATOMIC_END\n```\n\nThe user wants me to return ONLY this exact block, with no code fence or commentary. So I should just output it exactly as it appears."
|
||||
assistant_copy_length=116
|
||||
assistant_copy_sha256=ef6377d63552af075f4f4adec00165988418c5f46a992f4dce8e678b56fd34ac
|
||||
assistant_copy_exact=True
|
||||
assistant_copy="GATE0_CLAUDE_ATOMIC_BEGIN\nsegment-01=alpha-2d11\nsegment-02=middle-8e22\nsegment-03=omega-4f33\nGATE0_CLAUDE_ATOMIC_END"
|
||||
47
docs/compaction-refresh/evidence/raw/P6-contract-gap.txt
Normal file
47
docs/compaction-refresh/evidence/raw/P6-contract-gap.txt
Normal file
@@ -0,0 +1,47 @@
|
||||
$ rg -n -i "atomic|prefix-preserv" <Pi extensions docs> <Claude hook docs>
|
||||
NO MATCH: neither installed runtime document states an atomic/prefix-preserving transport guarantee.
|
||||
|
||||
$ rg -n -C 3 "#### context|event.messages - deep copy|return \{ messages" <Pi extensions docs>
|
||||
638-});
|
||||
639-```
|
||||
640-
|
||||
641:#### context
|
||||
642-
|
||||
643-Fired before each LLM call. Modify messages non-destructively. See [Session Format](session-format.md) for message types.
|
||||
644-
|
||||
645-```typescript
|
||||
646-pi.on("context", async (event, ctx) => {
|
||||
647: // event.messages - deep copy, safe to modify
|
||||
648- const filtered = event.messages.filter(m => !shouldPrune(m));
|
||||
649: return { messages: filtered };
|
||||
650-});
|
||||
651-```
|
||||
652-
|
||||
|
||||
$ rg -n -C 3 "additionalContext|add to the default system prompt" <Claude installed docs>
|
||||
/home/hermes/.config/mosaic/runtime/claude/RUNTIME.md-58- tiered models via the Task `model` param).
|
||||
/home/hermes/.config/mosaic/runtime/claude/RUNTIME.md-59-
|
||||
/home/hermes/.config/mosaic/runtime/claude/RUNTIME.md-60-Note: PostToolUse hook plain stdout on exit 0 goes to the debug log, not model context — only
|
||||
/home/hermes/.config/mosaic/runtime/claude/RUNTIME.md:61:`hookSpecificOutput.additionalContext` (or exit-2 stderr) enters context.
|
||||
--
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-59-expressed as
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-60-[subagents](https://docs.claude.com/en/docs/claude-code/sub-agents), not as
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-61-SessionStart hooks. Subagents change the system prompt while SessionStart hooks
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md:62:add to the default system prompt.
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-63-
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-64-## Managing changes
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/README.md-65-
|
||||
--
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-1-#!/usr/bin/env bash
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-2-
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh:3:# Output the explanatory mode instructions as additionalContext
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-4-# This mimics the deprecated Explanatory output style
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-5-
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-6-cat << 'EOF'
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-7-{
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-8- "hookSpecificOutput": {
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-9- "hookEventName": "SessionStart",
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh:10: "additionalContext": "You are in 'explanatory' output style mode, where you should provide educational insights about the codebase as you help with the user's task.\n\nYou should be clear and educational, providing helpful explanations while remaining focused on the task. Balance educational content with task completion. When providing insights, you may exceed typical length constraints, but remain focused and relevant.\n\n## Insights\nIn order to encourage learning, before and after writing code, always provide brief educational explanations about implementation choices using (with backticks):\n\"`★ Insight ─────────────────────────────────────`\n[2-3 key educational points]\n`─────────────────────────────────────────────────`\"\n\nThese insights should be included in the conversation, not in the codebase. You should generally focus on interesting insights that are specific to the codebase or the code you just wrote, rather than general programming concepts. Do not wait until the end to provide insights. Provide them as you write code."
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-11- }
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-12-}
|
||||
/home/hermes/.claude/plugins/marketplaces/claude-plugins-official/plugins/explanatory-output-style/hooks-handlers/session-start.sh-13-EOF
|
||||
@@ -0,0 +1,9 @@
|
||||
$ sha256sum ~/agent-work/reviews/compaction-refresh-BUILD-BRIEF.md ~/agent-work/reviews/compaction-refresh-SPEC-v5.md ~/agent-work/reviews/compaction-refresh-SPEC-RATIFICATION.md
|
||||
89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b /home/hermes/agent-work/reviews/compaction-refresh-BUILD-BRIEF.md
|
||||
a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433 /home/hermes/agent-work/reviews/compaction-refresh-SPEC-v5.md
|
||||
bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67 /home/hermes/agent-work/reviews/compaction-refresh-SPEC-RATIFICATION.md
|
||||
|
||||
Expected:
|
||||
89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b BUILD-BRIEF
|
||||
a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433 SPEC-v5
|
||||
bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67 RATIFICATION
|
||||
2
docs/compaction-refresh/probes/.gitignore
vendored
Normal file
2
docs/compaction-refresh/probes/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
51
docs/compaction-refresh/probes/p1_anchor_exec.py
Normal file
51
docs/compaction-refresh/probes/p1_anchor_exec.py
Normal file
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Register this PID as anchor, then exec the real `mosaic yolo` launcher."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
|
||||
|
||||
def request(socket_path: str, payload: dict[str, object]) -> dict[str, object]:
|
||||
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
conn.connect(socket_path)
|
||||
conn.sendall((json.dumps(payload) + "\n").encode())
|
||||
response = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
conn.close()
|
||||
return response
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--socket", required=True)
|
||||
parser.add_argument("runtime", choices=["pi", "claude"])
|
||||
parser.add_argument("args", nargs=argparse.REMAINDER)
|
||||
ns = parser.parse_args()
|
||||
|
||||
response = request(ns.socket, {"action": "register-anchor", "runtime": ns.runtime})
|
||||
if response.get("decision") != "ACCEPT":
|
||||
raise SystemExit("anchor registration refused")
|
||||
os.environ["GATE0_SESSION_ID"] = str(response["session_id"])
|
||||
argv = ["mosaic", "yolo", ns.runtime, *ns.args]
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"event": "anchor-exec",
|
||||
"pid": os.getpid(),
|
||||
"argv": ["mosaic", "yolo", ns.runtime, f"<{len(ns.args)} runtime args>"],
|
||||
"note": "os.execvpe retains pid and /proc starttime",
|
||||
},
|
||||
sort_keys=True,
|
||||
),
|
||||
file=sys.stderr,
|
||||
flush=True,
|
||||
)
|
||||
os.execvpe("mosaic", argv, os.environ)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
180
docs/compaction-refresh/probes/p1_broker.py
Normal file
180
docs/compaction-refresh/probes/p1_broker.py
Normal file
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Gate0 P1 broker prototype: peercred anchor minting and /proc ancestry checks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import stat
|
||||
import struct
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def proc_node(pid: int) -> dict[str, Any]:
|
||||
text = Path(f"/proc/{pid}/stat").read_text()
|
||||
close = text.rfind(")")
|
||||
comm = text[text.find("(") + 1 : close]
|
||||
fields = text[close + 2 :].split()
|
||||
cmdline = Path(f"/proc/{pid}/cmdline").read_bytes().split(b"\0")
|
||||
return {
|
||||
"pid": pid,
|
||||
"ppid": int(fields[1]),
|
||||
"starttime_ticks": int(fields[19]),
|
||||
"comm": comm,
|
||||
"exe": os.readlink(f"/proc/{pid}/exe"),
|
||||
"argv0": cmdline[0].decode(errors="replace") if cmdline and cmdline[0] else "",
|
||||
"argc": len([part for part in cmdline if part]),
|
||||
}
|
||||
|
||||
|
||||
def ancestry(peer_pid: int, anchor: dict[str, Any] | None) -> tuple[list[dict[str, Any]], bool, str]:
|
||||
chain: list[dict[str, Any]] = []
|
||||
pid = peer_pid
|
||||
seen: set[int] = set()
|
||||
try:
|
||||
while pid > 0 and pid not in seen:
|
||||
seen.add(pid)
|
||||
node = proc_node(pid)
|
||||
chain.append(node)
|
||||
if anchor and pid == anchor["pid"]:
|
||||
if node["starttime_ticks"] != anchor["starttime_ticks"]:
|
||||
return chain, False, "anchor-starttime-mismatch"
|
||||
break
|
||||
pid = node["ppid"]
|
||||
else:
|
||||
return chain, False, "anchor-not-reached"
|
||||
|
||||
if not anchor or chain[-1]["pid"] != anchor["pid"]:
|
||||
return chain, False, "anchor-not-reached"
|
||||
|
||||
# Re-read every node after the walk. A disappearing PID or changed
|
||||
# starttime invalidates the complete chain (PID-reuse/race closure).
|
||||
for original in chain:
|
||||
again = proc_node(original["pid"])
|
||||
if again["starttime_ticks"] != original["starttime_ticks"]:
|
||||
return chain, False, f"starttime-race:{original['pid']}"
|
||||
return chain, True, "ancestry-reaches-registered-anchor"
|
||||
except (FileNotFoundError, ProcessLookupError, PermissionError) as exc:
|
||||
return chain, False, f"proc-walk-failed:{type(exc).__name__}"
|
||||
|
||||
|
||||
def emit(log_file: Path, record: dict[str, Any]) -> None:
|
||||
line = json.dumps(record, sort_keys=True)
|
||||
with log_file.open("a", encoding="utf-8") as out:
|
||||
out.write(line + "\n")
|
||||
print(line, flush=True)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--socket", required=True)
|
||||
parser.add_argument("--log", required=True)
|
||||
parser.add_argument("--state", required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
socket_path = Path(args.socket)
|
||||
log_file = Path(args.log)
|
||||
state_file = Path(args.state)
|
||||
socket_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(socket_path.parent, 0o700)
|
||||
socket_path.unlink(missing_ok=True)
|
||||
log_file.unlink(missing_ok=True)
|
||||
state_file.unlink(missing_ok=True)
|
||||
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(str(socket_path))
|
||||
os.chmod(socket_path, 0o600)
|
||||
server.listen(8)
|
||||
anchor: dict[str, Any] | None = None
|
||||
session_id: str | None = None
|
||||
emit(
|
||||
log_file,
|
||||
{
|
||||
"event": "broker-listen",
|
||||
"pid": os.getpid(),
|
||||
"socket": str(socket_path),
|
||||
"directory_mode": f"{stat.S_IMODE(socket_path.parent.stat().st_mode):04o}",
|
||||
"socket_mode": f"{stat.S_IMODE(socket_path.stat().st_mode):04o}",
|
||||
},
|
||||
)
|
||||
|
||||
while True:
|
||||
conn, _ = server.accept()
|
||||
with conn:
|
||||
raw = conn.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, 12)
|
||||
peer_pid, peer_uid, peer_gid = struct.unpack("3i", raw)
|
||||
request = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
action = request.get("action")
|
||||
|
||||
if action == "register-anchor" and anchor is None:
|
||||
anchor = proc_node(peer_pid)
|
||||
session_id = secrets.token_hex(16)
|
||||
state = {"session_id": session_id, "anchor": anchor}
|
||||
state_file.write_text(json.dumps(state, sort_keys=True) + "\n")
|
||||
record = {
|
||||
"event": "anchor-minted",
|
||||
"decision": "ACCEPT",
|
||||
"peercred": {"pid": peer_pid, "uid": peer_uid, "gid": peer_gid},
|
||||
"anchor": anchor,
|
||||
"session_id": session_id,
|
||||
}
|
||||
emit(log_file, record)
|
||||
conn.sendall((json.dumps(record) + "\n").encode())
|
||||
continue
|
||||
|
||||
if action in {"resolve-hook", "claim-session"}:
|
||||
chain, reaches, reason = ancestry(peer_pid, anchor)
|
||||
claimed = request.get("session_id")
|
||||
claim_ok = action == "resolve-hook" or claimed == session_id
|
||||
accepted = bool(anchor and session_id and reaches and claim_ok)
|
||||
if action == "claim-session" and claimed != session_id:
|
||||
reason = "unknown-session-id"
|
||||
elif action == "claim-session" and claimed == session_id and not reaches:
|
||||
reason = "victim-id-known-but-ancestry-mismatch"
|
||||
record = {
|
||||
"event": action,
|
||||
"decision": "ACCEPT" if accepted else "REJECT",
|
||||
"reason": reason,
|
||||
"peercred": {"pid": peer_pid, "uid": peer_uid, "gid": peer_gid},
|
||||
"claimed_session_id": claimed,
|
||||
"resolved_session_id": session_id if accepted else None,
|
||||
"anchor": anchor,
|
||||
"ancestry": chain,
|
||||
"starttimes_rechecked": reaches,
|
||||
}
|
||||
emit(log_file, record)
|
||||
conn.sendall((json.dumps(record) + "\n").encode())
|
||||
continue
|
||||
|
||||
if action == "shutdown":
|
||||
record = {
|
||||
"event": "broker-shutdown",
|
||||
"peercred": {"pid": peer_pid, "uid": peer_uid, "gid": peer_gid},
|
||||
}
|
||||
emit(log_file, record)
|
||||
conn.sendall((json.dumps(record) + "\n").encode())
|
||||
break
|
||||
|
||||
record = {
|
||||
"event": "invalid-request",
|
||||
"decision": "REJECT",
|
||||
"peercred": {"pid": peer_pid, "uid": peer_uid, "gid": peer_gid},
|
||||
}
|
||||
emit(log_file, record)
|
||||
conn.sendall((json.dumps(record) + "\n").encode())
|
||||
|
||||
server.close()
|
||||
socket_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except Exception as exc:
|
||||
print(f"P1 broker fatal: {type(exc).__name__}: {exc}", file=sys.stderr)
|
||||
raise
|
||||
38
docs/compaction-refresh/probes/p1_hook_client.py
Normal file
38
docs/compaction-refresh/probes/p1_hook_client.py
Normal file
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Claude SessionStart hook client for P1 ancestry evidence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> None:
|
||||
# Consume the real Claude hook payload without recording transcript paths or
|
||||
# prompt content in the evidence artifact.
|
||||
hook_input = json.load(sys.stdin)
|
||||
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
conn.connect(os.environ["GATE0_BROKER_SOCKET"])
|
||||
conn.sendall((json.dumps({"action": "resolve-hook"}) + "\n").encode())
|
||||
response = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
conn.close()
|
||||
event_name = hook_input.get("hook_event_name")
|
||||
if response.get("decision") != "ACCEPT":
|
||||
print(f"Gate0 broker rejected {event_name} hook ancestry", file=sys.stderr)
|
||||
raise SystemExit(2)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": event_name,
|
||||
"additionalContext": "GATE0_P1_SUPPORTED_HOOK_ANCESTRY_ACCEPTED",
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
35
docs/compaction-refresh/probes/p1_pi_extension.ts
Normal file
35
docs/compaction-refresh/probes/p1_pi_extension.ts
Normal file
@@ -0,0 +1,35 @@
|
||||
import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
|
||||
import net from 'node:net';
|
||||
|
||||
async function brokerRequest(payload: Record<string, unknown>): Promise<Record<string, unknown>> {
|
||||
const socketPath = process.env['GATE0_BROKER_SOCKET'];
|
||||
if (!socketPath) throw new Error('GATE0_BROKER_SOCKET missing');
|
||||
return await new Promise((resolve, reject) => {
|
||||
const socket = net.createConnection(socketPath);
|
||||
let buffer = '';
|
||||
socket.setEncoding('utf8');
|
||||
socket.on('connect', () => socket.write(`${JSON.stringify(payload)}\n`));
|
||||
socket.on('data', (chunk) => {
|
||||
buffer += chunk;
|
||||
const newline = buffer.indexOf('\n');
|
||||
if (newline < 0) return;
|
||||
socket.end();
|
||||
resolve(JSON.parse(buffer.slice(0, newline)) as Record<string, unknown>);
|
||||
});
|
||||
socket.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
export default function register(pi: ExtensionAPI) {
|
||||
pi.on('session_start', async () => {
|
||||
const response = await brokerRequest({ action: 'resolve-hook', runtime: 'pi-extension' });
|
||||
if (response['decision'] !== 'ACCEPT') {
|
||||
throw new Error(`P1 broker rejected Pi extension ancestry: ${response['reason']}`);
|
||||
}
|
||||
});
|
||||
|
||||
pi.registerCommand('gate0-p1-ready', {
|
||||
description: 'Return only after the P1 session_start ancestry hook completed',
|
||||
handler: async () => undefined,
|
||||
});
|
||||
}
|
||||
274
docs/compaction-refresh/probes/p1_run.py
Normal file
274
docs/compaction-refresh/probes/p1_run.py
Normal file
@@ -0,0 +1,274 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run P1 against the real installed Mosaic→Pi and Mosaic→Claude chains."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def wait_for(predicate, description: str, timeout: float = 30.0) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if predicate():
|
||||
return
|
||||
time.sleep(0.05)
|
||||
raise TimeoutError(f"timed out waiting for {description}")
|
||||
|
||||
|
||||
def read_records(path: Path) -> list[dict[str, Any]]:
|
||||
if not path.exists():
|
||||
return []
|
||||
return [json.loads(line) for line in path.read_text().splitlines() if line]
|
||||
|
||||
|
||||
def socket_request(path: Path, payload: dict[str, object]) -> dict[str, object]:
|
||||
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
conn.connect(str(path))
|
||||
conn.sendall((json.dumps(payload) + "\n").encode())
|
||||
response = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
conn.close()
|
||||
return response
|
||||
|
||||
|
||||
def start_broker(root: Path) -> tuple[subprocess.Popen[str], Path, Path, Path]:
|
||||
socket_path = root / "broker.sock"
|
||||
log_path = root / "broker.jsonl"
|
||||
state_path = root / "state.json"
|
||||
broker = subprocess.Popen(
|
||||
[
|
||||
sys.executable,
|
||||
str(HERE / "p1_broker.py"),
|
||||
"--socket",
|
||||
str(socket_path),
|
||||
"--log",
|
||||
str(log_path),
|
||||
"--state",
|
||||
str(state_path),
|
||||
],
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
wait_for(socket_path.exists, "broker socket")
|
||||
return broker, socket_path, log_path, state_path
|
||||
|
||||
|
||||
def print_ps(record: dict[str, Any]) -> None:
|
||||
chain = record.get("ancestry", [])
|
||||
pids = [str(node["pid"]) for node in chain if Path(f"/proc/{node['pid']}").exists()]
|
||||
if not pids:
|
||||
print("ps_snapshot=<hook chain exited; broker /proc snapshot above is authoritative>")
|
||||
return
|
||||
command = [
|
||||
"ps",
|
||||
"-o",
|
||||
"pid=,ppid=,lstart=,uid=,gid=,comm=",
|
||||
"-p",
|
||||
",".join(pids),
|
||||
]
|
||||
print("$ " + " ".join(command))
|
||||
print(subprocess.check_output(command, text=True).rstrip())
|
||||
|
||||
|
||||
def run_runtime(runtime: str) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix=f"gate0-p1-{runtime}-") as temp:
|
||||
root = Path(temp)
|
||||
workspace = root / "workspace"
|
||||
workspace.mkdir()
|
||||
broker, socket_path, log_path, state_path = start_broker(root)
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"GATE0_BROKER_SOCKET": str(socket_path),
|
||||
"MOSAIC_PI_FORCE_SKILLS": "",
|
||||
"PI_SKIP_VERSION_CHECK": "1",
|
||||
}
|
||||
)
|
||||
stdout_path = root / f"{runtime}.stdout"
|
||||
stderr_path = root / f"{runtime}.stderr"
|
||||
|
||||
if runtime == "pi":
|
||||
runtime_args = [
|
||||
"--mode",
|
||||
"rpc",
|
||||
"--no-session",
|
||||
"--no-extensions",
|
||||
"--no-context-files",
|
||||
"--no-prompt-templates",
|
||||
"--extension",
|
||||
str(HERE / "p1_pi_extension.ts"),
|
||||
]
|
||||
else:
|
||||
settings = root / "claude-settings.json"
|
||||
settings.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"hooks": {
|
||||
"SessionStart": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": f'python3 "{HERE / "p1_hook_client.py"}"',
|
||||
"timeout": 20,
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
runtime_args = [
|
||||
"--settings",
|
||||
str(settings),
|
||||
"--model",
|
||||
"haiku",
|
||||
"--print",
|
||||
"--output-format",
|
||||
"stream-json",
|
||||
"--verbose",
|
||||
"--include-hook-events",
|
||||
"--max-budget-usd",
|
||||
"0.03",
|
||||
"Reply exactly: OK",
|
||||
]
|
||||
|
||||
out = stdout_path.open("w", encoding="utf-8")
|
||||
err = stderr_path.open("w", encoding="utf-8")
|
||||
anchor = subprocess.Popen(
|
||||
[
|
||||
sys.executable,
|
||||
str(HERE / "p1_anchor_exec.py"),
|
||||
"--socket",
|
||||
str(socket_path),
|
||||
runtime,
|
||||
*runtime_args,
|
||||
],
|
||||
cwd=workspace,
|
||||
env=env,
|
||||
stdin=subprocess.PIPE if runtime == "pi" else subprocess.DEVNULL,
|
||||
stdout=out,
|
||||
stderr=err,
|
||||
text=True,
|
||||
start_new_session=True,
|
||||
)
|
||||
try:
|
||||
wait_for(state_path.exists, "anchor registration")
|
||||
attacker = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(HERE / "p1_sibling_attacker.py"),
|
||||
"--socket",
|
||||
str(socket_path),
|
||||
"--state",
|
||||
str(state_path),
|
||||
],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
if runtime == "pi":
|
||||
assert anchor.stdin is not None
|
||||
anchor.stdin.write('{"id":"state","type":"get_state"}\n')
|
||||
anchor.stdin.flush()
|
||||
|
||||
wait_for(
|
||||
lambda: any(r.get("event") == "resolve-hook" for r in read_records(log_path)),
|
||||
f"{runtime} supported hook/extension broker contact",
|
||||
timeout=60,
|
||||
)
|
||||
if runtime == "claude":
|
||||
try:
|
||||
anchor.wait(timeout=90)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
records = read_records(log_path)
|
||||
state = json.loads(state_path.read_text())
|
||||
resolve = next(r for r in records if r.get("event") == "resolve-hook")
|
||||
reject = next(r for r in records if r.get("event") == "claim-session")
|
||||
if resolve.get("decision") != "ACCEPT":
|
||||
raise AssertionError(f"{runtime} hook ancestry was not accepted: {resolve}")
|
||||
if reject.get("decision") != "REJECT":
|
||||
raise AssertionError(f"{runtime} sibling substitution was not rejected: {reject}")
|
||||
if attacker.returncode != 0:
|
||||
raise AssertionError(f"{runtime} sibling probe did not observe rejection: {attacker.stderr}")
|
||||
|
||||
print(f"=== P1 {runtime.upper()} REAL LAUNCH ===")
|
||||
print("machine_assertions=PASS")
|
||||
print(
|
||||
"$ python3 docs/compaction-refresh/probes/p1_anchor_exec.py "
|
||||
f"--socket <protected-socket> {runtime} <runtime args>"
|
||||
)
|
||||
print("registered_anchor=" + json.dumps(state["anchor"], sort_keys=True))
|
||||
print("broker_minted_session_id=" + state["session_id"])
|
||||
print("hook_or_extension_record=" + json.dumps(resolve, sort_keys=True))
|
||||
print("sibling_attack_record=" + json.dumps(reject, sort_keys=True))
|
||||
print("sibling_process_stdout=" + attacker.stdout.strip())
|
||||
print(f"sibling_process_exit={attacker.returncode}")
|
||||
print_ps(resolve)
|
||||
print("launcher_stderr_excerpt:")
|
||||
for line in stderr_path.read_text(errors="replace").splitlines()[:12]:
|
||||
print(" " + line[:500])
|
||||
runtime_lines = stdout_path.read_text(errors="replace").splitlines()
|
||||
print("runtime_stdout_excerpt:")
|
||||
for line in runtime_lines[:8]:
|
||||
print(" " + line[:500])
|
||||
hook_lines = [
|
||||
line
|
||||
for line in runtime_lines
|
||||
if "hook" in line.lower() or "GATE0_P1_SUPPORTED_HOOK" in line
|
||||
]
|
||||
print("runtime_hook_event_excerpt:")
|
||||
for line in hook_lines[:8]:
|
||||
print(" " + line[:1000])
|
||||
print()
|
||||
finally:
|
||||
if anchor.poll() is None:
|
||||
try:
|
||||
os.killpg(anchor.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
anchor.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
os.killpg(anchor.pid, signal.SIGKILL)
|
||||
anchor.wait(timeout=5)
|
||||
out.close()
|
||||
err.close()
|
||||
try:
|
||||
socket_request(socket_path, {"action": "shutdown"})
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
broker.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
broker.kill()
|
||||
broker.wait()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--runtime", choices=["pi", "claude", "both"], default="both")
|
||||
ns = parser.parse_args()
|
||||
if ns.runtime in {"pi", "both"}:
|
||||
run_runtime("pi")
|
||||
if ns.runtime in {"claude", "both"}:
|
||||
run_runtime("claude")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
54
docs/compaction-refresh/probes/p1_sibling_attacker.py
Normal file
54
docs/compaction-refresh/probes/p1_sibling_attacker.py
Normal file
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Same-UID sibling that attempts to claim the anchor's broker-minted id."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--socket", required=True)
|
||||
parser.add_argument("--state", required=True)
|
||||
ns = parser.parse_args()
|
||||
state_path = Path(ns.state)
|
||||
for _ in range(200):
|
||||
if state_path.exists():
|
||||
break
|
||||
time.sleep(0.025)
|
||||
state = json.loads(state_path.read_text())
|
||||
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
conn.connect(ns.socket)
|
||||
conn.sendall(
|
||||
(
|
||||
json.dumps(
|
||||
{"action": "claim-session", "session_id": state["session_id"]},
|
||||
sort_keys=True,
|
||||
)
|
||||
+ "\n"
|
||||
).encode()
|
||||
)
|
||||
response = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
conn.close()
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"attacker_pid": os.getpid(),
|
||||
"attacker_uid": os.getuid(),
|
||||
"victim_session_id_known": True,
|
||||
"broker_decision": response.get("decision"),
|
||||
"broker_reason": response.get("reason"),
|
||||
},
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
raise SystemExit(0 if response.get("decision") == "REJECT" else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
135
docs/compaction-refresh/probes/p2_provider_timing_run.py
Normal file
135
docs/compaction-refresh/probes/p2_provider_timing_run.py
Normal file
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Force a real Pi HTTP provider response to prove response-hook timing."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
from pi_gate0_run import PiRpc, jsonl
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
|
||||
def log_message(self, _format: str, *_args: object) -> None:
|
||||
return
|
||||
|
||||
def do_POST(self) -> None: # noqa: N802
|
||||
length = int(self.headers.get("content-length", "0"))
|
||||
self.rfile.read(length)
|
||||
chunks = [
|
||||
{
|
||||
"id": "gate0-response",
|
||||
"object": "chat.completion.chunk",
|
||||
"created": 1,
|
||||
"model": "gate0-model",
|
||||
"choices": [{"index": 0, "delta": {"role": "assistant"}, "finish_reason": None}],
|
||||
},
|
||||
{
|
||||
"id": "gate0-response",
|
||||
"object": "chat.completion.chunk",
|
||||
"created": 1,
|
||||
"model": "gate0-model",
|
||||
"choices": [
|
||||
{"index": 0, "delta": {"content": "TIMING_OK"}, "finish_reason": None}
|
||||
],
|
||||
},
|
||||
{
|
||||
"id": "gate0-response",
|
||||
"object": "chat.completion.chunk",
|
||||
"created": 1,
|
||||
"model": "gate0-model",
|
||||
"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
|
||||
"usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},
|
||||
},
|
||||
]
|
||||
body = "".join(f"data: {json.dumps(chunk)}\n\n" for chunk in chunks) + "data: [DONE]\n\n"
|
||||
encoded = body.encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/event-stream")
|
||||
self.send_header("Content-Length", str(len(encoded)))
|
||||
self.send_header("X-Gate0-Response", "headers-before-stream")
|
||||
self.end_headers()
|
||||
self.wfile.write(encoded)
|
||||
self.wfile.flush()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
port = server.server_address[1]
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="gate0-p2-timing-") as temp:
|
||||
root = Path(temp)
|
||||
workspace = root / "workspace"
|
||||
workspace.mkdir()
|
||||
log = root / "hooks.jsonl"
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"GATE0_PI_LOG": str(log),
|
||||
"GATE0_LOCAL_PROVIDER_URL": f"http://127.0.0.1:{port}/v1",
|
||||
"MOSAIC_PI_FORCE_SKILLS": "",
|
||||
"PI_SKIP_VERSION_CHECK": "1",
|
||||
}
|
||||
)
|
||||
command = [
|
||||
"mosaic",
|
||||
"yolo",
|
||||
"pi",
|
||||
"--mode",
|
||||
"rpc",
|
||||
"--no-session",
|
||||
"--no-extensions",
|
||||
"--no-context-files",
|
||||
"--no-prompt-templates",
|
||||
"--provider",
|
||||
"gate0-local",
|
||||
"--model",
|
||||
"gate0-model",
|
||||
"--extension",
|
||||
str(HERE / "pi_gate0_extension.ts"),
|
||||
]
|
||||
pi = PiRpc(command, workspace, env)
|
||||
try:
|
||||
pi.prompt_and_settle("timing", "Reply with TIMING_OK")
|
||||
records = jsonl(log)
|
||||
selected = [
|
||||
record
|
||||
for record in records
|
||||
if record["event"] in {"before_provider_request", "after_provider_response", "message_end"}
|
||||
and (record["event"] != "message_end" or record.get("role") == "assistant")
|
||||
]
|
||||
print("$ python3 docs/compaction-refresh/probes/p2_provider_timing_run.py")
|
||||
print(f"local_http_endpoint=http://127.0.0.1:{port}/v1/chat/completions")
|
||||
for record in selected:
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
after = next(record for record in selected if record["event"] == "after_provider_response")
|
||||
message = next(record for record in selected if record["event"] == "message_end")
|
||||
if not (
|
||||
after["seq"] < message["seq"]
|
||||
and after["assistantContentAvailableAtThisHook"] is False
|
||||
and message["assistantContentObserved"] is True
|
||||
):
|
||||
raise AssertionError("provider response/content observation ordering failed")
|
||||
print("machine_assertions=PASS")
|
||||
print(f"after_provider_response_seq={after['seq']}")
|
||||
print(f"message_end_seq={message['seq']}")
|
||||
print(f"headers_hook_precedes_completed_message={after['seq'] < message['seq']}")
|
||||
finally:
|
||||
pi.close()
|
||||
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
849
docs/compaction-refresh/probes/p3_d4_focused_run.py
Normal file
849
docs/compaction-refresh/probes/p3_d4_focused_run.py
Normal file
@@ -0,0 +1,849 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D4-only same-PID runtime-generation revocation harness.
|
||||
|
||||
AUTHORING NOTE: this file is intentionally not executed until the separately
|
||||
ratified FIRE authorization. When run later, every invocation creates its own
|
||||
/tmp fixture and launches the real Pi RPC runtime with only the D4 extension
|
||||
and ``p3_generation_broker.py``. It does not use the broader Gate0 runner.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import queue
|
||||
import shutil
|
||||
import signal
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
# WI-3 remains in a reviewed worktree until the release package contains the
|
||||
# gated launcher. The probe resolves that worktree portably and never falls
|
||||
# back to the released `mosaic` binary.
|
||||
GATED_WI_ROOT_OVERRIDE = os.environ.get("GATED_WI_ROOT")
|
||||
GATED_WI_BRANCH = "refs/heads/feat/830-compaction-revoke"
|
||||
GATED_WI_HEAD = "f400830738998db105107a2a4c69c7f2a2a6fd5d"
|
||||
GATED_WI_ANCESTOR = "66b1e0a0"
|
||||
GATED_BROKER_HEAD = "23c0caca9b5d44002e6184cd7f2b6c837e8795b2"
|
||||
LEASE_BROKER_DIRECTORY = "packages/mosaic/framework/tools/lease-broker"
|
||||
BROKER_RELATIVE_PATH = "docs/compaction-refresh/probes/p3_generation_broker.py"
|
||||
GATED_LAUNCHER_SHA256 = "e950e4224e280f16979d90cabb89aa1896c5ee28bed2df957e14d018d43cda82"
|
||||
GATED_GENERATION_SHA256 = "061625402f08488eac47acd23272904e71fd1a71fd15b3bdab158632c801be4c"
|
||||
GATED_BROKER_SHA256 = "4db4fef1ac6658a8ca79ad5091cefc901d2aa26003265c3d6726c294cf895cad"
|
||||
|
||||
|
||||
class PiRpc:
|
||||
"""Small JSON-RPC client for an isolated real Pi process."""
|
||||
|
||||
def __init__(self, command: list[str], cwd: Path, env: dict[str, str]) -> None:
|
||||
self.process = subprocess.Popen(
|
||||
command,
|
||||
cwd=cwd,
|
||||
env=env,
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
start_new_session=True,
|
||||
)
|
||||
self.events: queue.Queue[dict[str, Any]] = queue.Queue()
|
||||
self.stderr_lines: list[str] = []
|
||||
threading.Thread(target=self._read_stdout, daemon=True).start()
|
||||
threading.Thread(target=self._read_stderr, daemon=True).start()
|
||||
|
||||
def _read_stdout(self) -> None:
|
||||
if self.process.stdout is None:
|
||||
raise RuntimeError("Pi stdout pipe is unavailable")
|
||||
for line in self.process.stdout:
|
||||
try:
|
||||
self.events.put(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
def _read_stderr(self) -> None:
|
||||
if self.process.stderr is None:
|
||||
raise RuntimeError("Pi stderr pipe is unavailable")
|
||||
for line in self.process.stderr:
|
||||
self.stderr_lines.append(line.rstrip("\n"))
|
||||
|
||||
def send(self, payload: dict[str, object]) -> None:
|
||||
if self.process.stdin is None:
|
||||
raise RuntimeError("Pi stdin pipe is unavailable")
|
||||
self.process.stdin.write(json.dumps(payload) + "\n")
|
||||
self.process.stdin.flush()
|
||||
|
||||
def wait(
|
||||
self,
|
||||
predicate: Callable[[dict[str, Any]], bool],
|
||||
description: str,
|
||||
timeout: float = 180,
|
||||
) -> dict[str, Any]:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if self.process.poll() is not None and self.events.empty():
|
||||
detail = " | ".join(self.stderr_lines[-5:])
|
||||
raise RuntimeError(
|
||||
f"Pi exited {self.process.returncode} while waiting for {description}: {detail}"
|
||||
)
|
||||
try:
|
||||
event = self.events.get(timeout=0.2)
|
||||
except queue.Empty:
|
||||
continue
|
||||
if predicate(event):
|
||||
return event
|
||||
raise TimeoutError(f"timed out waiting for {description}")
|
||||
|
||||
def response(self, request_id: str, timeout: float = 180) -> dict[str, Any]:
|
||||
return self.wait(
|
||||
lambda event: event.get("type") == "response" and event.get("id") == request_id,
|
||||
f"response {request_id}",
|
||||
timeout,
|
||||
)
|
||||
|
||||
def prompt_and_settle(self, request_id: str, message: str) -> None:
|
||||
self.send({"id": request_id, "type": "prompt", "message": message})
|
||||
response = self.response(request_id)
|
||||
if not response.get("success"):
|
||||
raise RuntimeError(f"prompt rejected: {response}")
|
||||
self.wait(
|
||||
lambda event: event.get("type") == "agent_settled",
|
||||
f"agent_settled {request_id}",
|
||||
)
|
||||
|
||||
def close(self) -> None:
|
||||
if self.process.poll() is None:
|
||||
try:
|
||||
os.killpg(self.process.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
self.process.wait(timeout=8)
|
||||
except subprocess.TimeoutExpired:
|
||||
os.killpg(self.process.pid, signal.SIGKILL)
|
||||
self.process.wait(timeout=5)
|
||||
|
||||
|
||||
def wait_path(path: Path, timeout: float = 20) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if path.exists():
|
||||
return
|
||||
time.sleep(0.05)
|
||||
raise TimeoutError(f"timed out waiting for {path}")
|
||||
|
||||
|
||||
def request(path: Path, payload: dict[str, object]) -> dict[str, Any]:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as conn:
|
||||
conn.connect(str(path))
|
||||
conn.sendall((json.dumps(payload) + "\n").encode())
|
||||
reply = conn.makefile("r", encoding="utf-8").readline()
|
||||
return json.loads(reply)
|
||||
|
||||
|
||||
def jsonl(path: Path) -> list[dict[str, Any]]:
|
||||
return [json.loads(line) for line in path.read_text().splitlines() if line]
|
||||
|
||||
|
||||
def write_extension(path: Path) -> None:
|
||||
"""Write the minimal Pi lifecycle bridge into the isolated fixture only."""
|
||||
|
||||
path.write_text(
|
||||
"""import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
|
||||
import { Type } from 'typebox';
|
||||
import { appendFileSync, readFileSync } from 'node:fs';
|
||||
import net from 'node:net';
|
||||
|
||||
const socketPath = process.env['D4_GENERATION_SOCKET'];
|
||||
const logPath = process.env['D4_PI_LOG'];
|
||||
|
||||
function starttime(): number {
|
||||
const text = readFileSync(`/proc/${process.pid}/stat`, 'utf8');
|
||||
const close = text.lastIndexOf(')');
|
||||
return Number(text.slice(close + 2).trim().split(/\\s+/)[19]);
|
||||
}
|
||||
|
||||
function log(event: string, details: Record<string, unknown> = {}): void {
|
||||
if (!logPath) return;
|
||||
appendFileSync(logPath, `${JSON.stringify({ event, pid: process.pid, starttime_ticks: starttime(), ...details })}\\n`);
|
||||
}
|
||||
|
||||
function broker(payload: Record<string, unknown>): Promise<Record<string, unknown>> {
|
||||
if (!socketPath) return Promise.reject(new Error('D4_GENERATION_SOCKET is required'));
|
||||
return new Promise((resolve, reject) => {
|
||||
const connection = net.createConnection(socketPath);
|
||||
let buffer = '';
|
||||
connection.setEncoding('utf8');
|
||||
connection.on('connect', () => connection.write(`${JSON.stringify(payload)}\\n`));
|
||||
connection.on('data', (chunk) => {
|
||||
buffer += chunk;
|
||||
const newline = buffer.indexOf('\\n');
|
||||
if (newline < 0) return;
|
||||
connection.end();
|
||||
resolve(JSON.parse(buffer.slice(0, newline)) as Record<string, unknown>);
|
||||
});
|
||||
connection.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
export default function register(pi: ExtensionAPI): void {
|
||||
let initialStartup = true;
|
||||
|
||||
async function lifecycle(
|
||||
phase: 'start' | 'shutdown',
|
||||
reason: string,
|
||||
): Promise<Record<string, unknown>> {
|
||||
if (!(phase === 'start' && reason === 'startup' && initialStartup)) {
|
||||
const bump = await broker({ action: 'bump-generation' });
|
||||
log('generation_state_bump', { phase, reason, bump });
|
||||
}
|
||||
initialStartup = false;
|
||||
return broker({ action: 'lifecycle', phase, reason });
|
||||
}
|
||||
|
||||
pi.on('session_start', async (event) => {
|
||||
const lifecycleResult = await lifecycle('start', event.reason);
|
||||
log('session_start', { reason: event.reason, lifecycle: lifecycleResult });
|
||||
if (event.reason === 'reload') {
|
||||
const generation = lifecycleResult['new_generation'];
|
||||
if (typeof generation !== 'number') throw new Error('broker did not return new_generation');
|
||||
const current = await broker({ action: 'authorize-probe', generation });
|
||||
const superseded = await broker({ action: 'authorize-probe', generation: generation - 1 });
|
||||
log('d4_generation_authorization', { generation, current, superseded });
|
||||
}
|
||||
});
|
||||
|
||||
pi.on('session_shutdown', async (event) => {
|
||||
const lifecycleResult = await lifecycle('shutdown', event.reason);
|
||||
log('session_shutdown', { reason: event.reason, lifecycle: lifecycleResult });
|
||||
});
|
||||
|
||||
pi.registerTool({
|
||||
name: 'd4_fixture_promote',
|
||||
label: 'D4 Fixture Promotion',
|
||||
description: 'Promotes only the fixture lease needed for the D4 revocation check.',
|
||||
parameters: Type.Object({}),
|
||||
async execute() {
|
||||
// the promotion step is a D4 test fixture, not a P2 evidence-gathering authorization.
|
||||
const promotion = await broker({ action: 'promote-probe' });
|
||||
log('fixture_promotion', { promotion });
|
||||
return { content: [{ type: 'text', text: 'D4 fixture promotion complete' }] };
|
||||
},
|
||||
});
|
||||
|
||||
pi.registerCommand('d4-reload', {
|
||||
description: 'D4-only same-PID reload boundary.',
|
||||
handler: async (_args, context) => {
|
||||
await context.reload();
|
||||
},
|
||||
});
|
||||
}
|
||||
""",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def repository_root() -> Path:
|
||||
for candidate in HERE.parents:
|
||||
if (candidate / ".git").exists():
|
||||
return candidate
|
||||
raise RuntimeError("D4 precondition: probe repository root is unavailable")
|
||||
|
||||
|
||||
def resolve_gated_wi_root() -> Path:
|
||||
"""Resolve an explicit override or the unique checked-out WI-3 branch."""
|
||||
|
||||
if GATED_WI_ROOT_OVERRIDE:
|
||||
candidate = Path(GATED_WI_ROOT_OVERRIDE).expanduser()
|
||||
candidates = [candidate]
|
||||
else:
|
||||
try:
|
||||
listing = subprocess.check_output(
|
||||
["git", "-C", str(repository_root()), "worktree", "list", "--porcelain"],
|
||||
text=True,
|
||||
)
|
||||
except (OSError, subprocess.CalledProcessError) as error:
|
||||
raise RuntimeError("D4 precondition: cannot enumerate WI-3 worktrees") from error
|
||||
candidates = []
|
||||
worktree: Path | None = None
|
||||
head: str | None = None
|
||||
branch: str | None = None
|
||||
for line in [*listing.splitlines(), ""]:
|
||||
if line.startswith("worktree "):
|
||||
worktree = Path(line.removeprefix("worktree "))
|
||||
head = None
|
||||
branch = None
|
||||
elif line.startswith("HEAD "):
|
||||
head = line.removeprefix("HEAD ")
|
||||
elif line.startswith("branch "):
|
||||
branch = line.removeprefix("branch ")
|
||||
elif not line and worktree is not None:
|
||||
if head == GATED_WI_HEAD and branch == GATED_WI_BRANCH:
|
||||
candidates.append(worktree)
|
||||
worktree = None
|
||||
if len(candidates) != 1:
|
||||
raise RuntimeError("D4 precondition: WI-3 worktree is ambiguous or unavailable")
|
||||
|
||||
gated_root = candidates[0]
|
||||
try:
|
||||
if not gated_root.is_dir():
|
||||
raise RuntimeError("D4 precondition: GATED_WI_ROOT is not a directory")
|
||||
is_worktree = subprocess.check_output(
|
||||
["git", "-C", str(gated_root), "rev-parse", "--is-inside-work-tree"],
|
||||
text=True,
|
||||
).strip()
|
||||
head = subprocess.check_output(
|
||||
["git", "-C", str(gated_root), "rev-parse", "HEAD"], text=True
|
||||
).strip()
|
||||
except (OSError, subprocess.CalledProcessError) as error:
|
||||
raise RuntimeError("D4 precondition: GATED_WI_ROOT is not a git worktree") from error
|
||||
if is_worktree != "true":
|
||||
raise RuntimeError("D4 precondition: GATED_WI_ROOT is not a git worktree")
|
||||
if head != GATED_WI_HEAD:
|
||||
raise RuntimeError(f"D4 precondition: gated WI head mismatch: {head}")
|
||||
try:
|
||||
forward_contains = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(gated_root),
|
||||
"merge-base",
|
||||
"--is-ancestor",
|
||||
GATED_WI_ANCESTOR,
|
||||
GATED_WI_HEAD,
|
||||
],
|
||||
check=False,
|
||||
).returncode == 0
|
||||
except OSError as error:
|
||||
raise RuntimeError("D4 precondition: cannot verify WI-3 ancestry") from error
|
||||
if not forward_contains:
|
||||
raise RuntimeError("D4 precondition: gated WI lacks required ancestor")
|
||||
return gated_root
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PinnedClosure:
|
||||
launcher: Path
|
||||
generation: Path
|
||||
broker: Path
|
||||
|
||||
|
||||
def git_object_bytes(git_root: Path, commit: str, relative_path: str) -> bytes:
|
||||
try:
|
||||
return subprocess.check_output(
|
||||
["git", "-C", str(git_root), "show", f"{commit}:{relative_path}"]
|
||||
)
|
||||
except (OSError, subprocess.CalledProcessError) as error:
|
||||
raise RuntimeError(f"D4 precondition: missing pinned source {relative_path}") from error
|
||||
|
||||
|
||||
def closure_import_guard(member_sources: dict[str, str]) -> None:
|
||||
"""Refuse an incomplete project-code closure before materializing it."""
|
||||
|
||||
allowed_nonstdlib = {"lease_generation"}
|
||||
stdlib = getattr(sys, "stdlib_module_names", frozenset())
|
||||
for name, source in member_sources.items():
|
||||
try:
|
||||
tree = ast.parse(source, filename=name)
|
||||
except SyntaxError as error:
|
||||
raise RuntimeError(f"D4 precondition: pinned {name} does not parse") from error
|
||||
for node in ast.walk(tree):
|
||||
module: str | None = None
|
||||
if isinstance(node, ast.Import):
|
||||
for alias in node.names:
|
||||
module = alias.name.split(".", maxsplit=1)[0]
|
||||
if module not in stdlib and module not in allowed_nonstdlib:
|
||||
raise RuntimeError(f"D4 precondition: unpinned import {module} in {name}")
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
if node.level:
|
||||
raise RuntimeError(f"D4 precondition: relative import in {name}")
|
||||
if node.module:
|
||||
module = node.module.split(".", maxsplit=1)[0]
|
||||
if module not in stdlib and module not in allowed_nonstdlib:
|
||||
raise RuntimeError(f"D4 precondition: unpinned import {module} in {name}")
|
||||
|
||||
|
||||
def write_pinned_file(path: Path, data: bytes) -> None:
|
||||
descriptor = os.open(
|
||||
path,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_CLOEXEC,
|
||||
0o600,
|
||||
)
|
||||
try:
|
||||
remaining = memoryview(data)
|
||||
while remaining:
|
||||
written = os.write(descriptor, remaining)
|
||||
if written <= 0:
|
||||
raise OSError("pinned write made no progress")
|
||||
remaining = remaining[written:]
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def materialize_closure(root: Path, gated_root: Path, gate0_root: Path) -> PinnedClosure:
|
||||
"""Pin the complete project-authored runtime closure inside this fixture."""
|
||||
|
||||
launcher_relative = f"{LEASE_BROKER_DIRECTORY}/launch-runtime.py"
|
||||
generation_relative = f"{LEASE_BROKER_DIRECTORY}/lease_generation.py"
|
||||
members = (
|
||||
(
|
||||
"launch-runtime.py",
|
||||
gated_root,
|
||||
GATED_WI_HEAD,
|
||||
launcher_relative,
|
||||
GATED_LAUNCHER_SHA256,
|
||||
),
|
||||
(
|
||||
"lease_generation.py",
|
||||
gated_root,
|
||||
GATED_WI_HEAD,
|
||||
generation_relative,
|
||||
GATED_GENERATION_SHA256,
|
||||
),
|
||||
(
|
||||
"p3_generation_broker.py",
|
||||
gate0_root,
|
||||
GATED_BROKER_HEAD,
|
||||
BROKER_RELATIVE_PATH,
|
||||
GATED_BROKER_SHA256,
|
||||
),
|
||||
)
|
||||
member_bytes: dict[str, bytes] = {}
|
||||
member_sources: dict[str, str] = {}
|
||||
for name, git_root, commit, relative_path, digest in members:
|
||||
data = git_object_bytes(git_root, commit, relative_path)
|
||||
if hashlib.sha256(data).hexdigest() != digest:
|
||||
raise RuntimeError(f"D4 precondition: {name} hash mismatch")
|
||||
try:
|
||||
member_sources[name] = data.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise RuntimeError(f"D4 precondition: pinned {name} is not UTF-8") from error
|
||||
member_bytes[name] = data
|
||||
closure_import_guard(member_sources)
|
||||
|
||||
pinned = root / "pinned"
|
||||
pinned.mkdir(mode=0o700)
|
||||
paths = {name: pinned / name for name, *_ in members}
|
||||
for name, path in paths.items():
|
||||
write_pinned_file(path, member_bytes[name])
|
||||
return PinnedClosure(
|
||||
launcher=paths["launch-runtime.py"],
|
||||
generation=paths["lease_generation.py"],
|
||||
broker=paths["p3_generation_broker.py"],
|
||||
)
|
||||
|
||||
|
||||
def gated_launcher_precondition(
|
||||
root: Path, socket_path: Path, environment: dict[str, str]
|
||||
) -> PinnedClosure:
|
||||
"""Verify and materialize the full WI-3/probe closure before execution."""
|
||||
|
||||
if environment.get("MOSAIC_LEASE_BROKER_SOCKET") != str(socket_path):
|
||||
raise RuntimeError("D4 precondition: lease broker socket is not this fixture")
|
||||
if environment.get("MOSAIC_LEASE_GENERATION_FILE"):
|
||||
raise RuntimeError("D4 precondition: inherited generation file is forbidden")
|
||||
fixture_path_vars = (
|
||||
"HOME",
|
||||
"XDG_CONFIG_HOME",
|
||||
"XDG_CACHE_HOME",
|
||||
"XDG_STATE_HOME",
|
||||
"XDG_RUNTIME_DIR",
|
||||
"TMPDIR",
|
||||
"D4_PI_LOG",
|
||||
"MOSAIC_AGENT_WORKDIR",
|
||||
"MOSAIC_HEARTBEAT_RUN_DIR",
|
||||
"MOSAIC_HOME",
|
||||
)
|
||||
if any(
|
||||
not (value := environment.get(name)) or not Path(value).is_relative_to(root)
|
||||
for name in fixture_path_vars
|
||||
):
|
||||
raise RuntimeError("D4 precondition: child write path escapes fixture root")
|
||||
if socket_path.parent != root or root.parent != Path(tempfile.gettempdir()):
|
||||
raise RuntimeError("D4 precondition: fixture socket is outside this run's temporary root")
|
||||
|
||||
gated_root = resolve_gated_wi_root()
|
||||
closure = materialize_closure(root, gated_root, repository_root())
|
||||
launcher_source = closure.launcher.read_text(encoding="utf-8")
|
||||
generation_source = closure.generation.read_text(encoding="utf-8")
|
||||
# Exact hashes in materialize_closure are the trust anchor. These marker
|
||||
# checks are belt-and-suspenders diagnostics only.
|
||||
behavior_markers = (
|
||||
'"action": "register_anchor"',
|
||||
"initialize_runtime_generation(generation_file, generation)",
|
||||
"execute(command[0], command, environment)",
|
||||
'source_environment["MOSAIC_LEASE_BROKER_SOCKET"]',
|
||||
'socket_path.parent / f"generation-{session_id}.state"',
|
||||
'environment["MOSAIC_LEASE_GENERATION_FILE"]',
|
||||
)
|
||||
if not all(marker in launcher_source for marker in behavior_markers) or not (
|
||||
"def read_runtime_generation" in generation_source
|
||||
and "def bump_runtime_generation" in generation_source
|
||||
):
|
||||
raise RuntimeError("D4 precondition: pinned launcher lacks file-generation markers")
|
||||
return closure
|
||||
|
||||
|
||||
def reject_pinned_bytecode(pinned_directory: Path) -> None:
|
||||
cache_directory = pinned_directory / "__pycache__"
|
||||
if cache_directory.exists() or any(pinned_directory.rglob("*.pyc")):
|
||||
raise RuntimeError("D4 precondition: pinned bytecode cache is forbidden")
|
||||
|
||||
|
||||
def launch_verified_pi(
|
||||
launcher: Path,
|
||||
workspace: Path,
|
||||
sessions: Path,
|
||||
extension: Path,
|
||||
environment: dict[str, str],
|
||||
) -> PiRpc:
|
||||
command = [
|
||||
sys.executable,
|
||||
# -s preserves sys.path[0]=pinned/ for the launcher's sibling helper.
|
||||
"-s",
|
||||
"-S",
|
||||
"-B",
|
||||
str(launcher),
|
||||
"--runtime",
|
||||
"pi",
|
||||
"--",
|
||||
"pi",
|
||||
"--mode",
|
||||
"rpc",
|
||||
"--session-dir",
|
||||
str(sessions),
|
||||
"--no-extensions",
|
||||
"--no-context-files",
|
||||
"--no-prompt-templates",
|
||||
"--model",
|
||||
"openai-codex/gpt-5.6-sol",
|
||||
"--thinking",
|
||||
"medium",
|
||||
"--extension",
|
||||
str(extension),
|
||||
]
|
||||
reject_pinned_bytecode(launcher.parent)
|
||||
# This is deliberately the statement immediately before Popen (inside
|
||||
# PiRpc): the fixture-pinned launcher bytes are re-hashed then executed.
|
||||
if hashlib.sha256(launcher.read_bytes()).hexdigest() != GATED_LAUNCHER_SHA256:
|
||||
raise RuntimeError("D4 precondition: adjacent launcher hash mismatch")
|
||||
return PiRpc(command, workspace, environment)
|
||||
|
||||
|
||||
def launch_verified_broker(
|
||||
broker_path: Path,
|
||||
generation_path: Path,
|
||||
socket_path: Path,
|
||||
log_path: Path,
|
||||
environment: dict[str, str],
|
||||
) -> subprocess.Popen[str]:
|
||||
command = [
|
||||
sys.executable,
|
||||
"-I",
|
||||
"-S",
|
||||
"-B",
|
||||
str(broker_path),
|
||||
"--socket",
|
||||
str(socket_path),
|
||||
"--log",
|
||||
str(log_path),
|
||||
"--generation-module",
|
||||
str(generation_path),
|
||||
]
|
||||
reject_pinned_bytecode(broker_path.parent)
|
||||
if hashlib.sha256(broker_path.read_bytes()).hexdigest() != GATED_BROKER_SHA256:
|
||||
raise RuntimeError("D4 precondition: pinned broker hash mismatch")
|
||||
# The final helper re-hash is immediately adjacent to the broker Popen.
|
||||
if hashlib.sha256(generation_path.read_bytes()).hexdigest() != GATED_GENERATION_SHA256:
|
||||
raise RuntimeError("D4 precondition: pinned helper hash mismatch")
|
||||
return subprocess.Popen(
|
||||
command,
|
||||
env=environment,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
|
||||
|
||||
def assert_d4(records: list[dict[str, Any]]) -> dict[str, object]:
|
||||
def record_where(description: str, candidates: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
if not candidates:
|
||||
raise AssertionError(f"missing D4 evidence record: {description}")
|
||||
return candidates[0]
|
||||
|
||||
fixture_listen = record_where(
|
||||
"fixture listen", [r for r in records if r.get("event") == "listen"]
|
||||
)
|
||||
fixture_root = Path(fixture_listen["socket"]).parent
|
||||
lifecycle = [record for record in records if record.get("event") == "runtime_generation_bump"]
|
||||
state_bumps = [record for record in records if record.get("event") == "generation_state_bumped"]
|
||||
promotion = record_where(
|
||||
"fixture promotion", [r for r in records if r.get("event") == "probe_lease_promoted"]
|
||||
)
|
||||
launcher_registration = record_where(
|
||||
"lease anchor", [r for r in records if r.get("event") == "lease_anchor_registered"]
|
||||
)
|
||||
reload_revoke = record_where(
|
||||
"reload shutdown",
|
||||
[
|
||||
r
|
||||
for r in lifecycle
|
||||
if r.get("reason") == "reload" and r.get("phase") == "shutdown"
|
||||
],
|
||||
)
|
||||
reload_start = record_where(
|
||||
"reload start",
|
||||
[
|
||||
r
|
||||
for r in lifecycle
|
||||
if r.get("reason") == "reload" and r.get("phase") == "start"
|
||||
],
|
||||
)
|
||||
authorization = [
|
||||
record for record in records if record.get("event") == "generation_authorization"
|
||||
]
|
||||
current_generation = reload_start["new_generation"]
|
||||
current_authorization = record_where(
|
||||
"current-generation authorization",
|
||||
[r for r in authorization if r.get("requested_generation") == current_generation],
|
||||
)
|
||||
superseded_authorization = record_where(
|
||||
"superseded-generation authorization",
|
||||
[r for r in authorization if r.get("requested_generation") == current_generation - 1],
|
||||
)
|
||||
|
||||
identities = {
|
||||
(record["peercred"]["pid"], record["starttime_ticks"])
|
||||
for record in [*lifecycle, *state_bumps, promotion, launcher_registration, *authorization]
|
||||
}
|
||||
generations = [record["new_generation"] for record in lifecycle]
|
||||
file_records = [*lifecycle, *state_bumps, promotion, *authorization]
|
||||
observed_reasons = {record.get("reason") for record in lifecycle}
|
||||
checks = {
|
||||
"same_pid_starttime": len(identities) == 1,
|
||||
"strictly_increasing_generation": all(
|
||||
previous < current for previous, current in zip(generations, generations[1:])
|
||||
),
|
||||
"state_file_drives_lifecycle": [record["generation"] for record in state_bumps]
|
||||
== generations[1:],
|
||||
"state_file_source": all(
|
||||
record.get("generation_source") == "state-file" for record in file_records
|
||||
),
|
||||
"state_file_in_fixture_root": all(
|
||||
Path(record["generation_file"]).parent == fixture_root for record in file_records
|
||||
)
|
||||
and Path(launcher_registration["generation_file"]).parent == fixture_root,
|
||||
"all_lifecycle_boundaries": {"startup", "reload", "fork", "new", "resume"}
|
||||
<= observed_reasons,
|
||||
"lease_anchor_fixture": launcher_registration.get("session_id_shape") == "hex-256",
|
||||
"verified_revoked_on_reload": reload_revoke.get("prior_lease") == "VERIFIED"
|
||||
and reload_revoke.get("prior_lease_revoked") is True,
|
||||
"new_generation_unverified": current_authorization.get("code") == "MUTATOR_UNVERIFIED",
|
||||
"prior_generation_stale": superseded_authorization.get("code") == "STALE_GENERATION",
|
||||
}
|
||||
failed = [name for name, passed in checks.items() if not passed]
|
||||
if failed:
|
||||
raise AssertionError(f"D4 checks failed: {', '.join(failed)}")
|
||||
passed = all(checks.values())
|
||||
if not passed:
|
||||
raise AssertionError("D4 PASS derivation failed")
|
||||
|
||||
return {
|
||||
"machine_assertions": "PASS" if passed else "FAIL",
|
||||
"checks": checks,
|
||||
"same_pid_starttime": next(iter(identities)),
|
||||
"generations": generations,
|
||||
"reload_revoke_verified": checks["verified_revoked_on_reload"],
|
||||
"lease_anchor_fixture": checks["lease_anchor_fixture"],
|
||||
"file_backed_generation": checks["state_file_drives_lifecycle"],
|
||||
"new_generation_code": current_authorization["code"],
|
||||
"superseded_generation_code": superseded_authorization["code"],
|
||||
}
|
||||
|
||||
|
||||
def isolated_environment(
|
||||
root: Path, index: int, workspace: Path, socket_path: Path, pi_log: Path
|
||||
) -> dict[str, str]:
|
||||
"""Build a write-confined child environment; no inherited path variable survives."""
|
||||
|
||||
fixture_home = root / "home"
|
||||
fixture_config = root / "config"
|
||||
fixture_cache = root / "cache"
|
||||
fixture_state = root / "state"
|
||||
fixture_runtime = root / "runtime"
|
||||
fixture_tmp = root / "tmp"
|
||||
fixture_heartbeat = root / "heartbeat"
|
||||
fixture_mosaic_home = root / "mosaic-home"
|
||||
for directory in (
|
||||
fixture_home,
|
||||
fixture_config,
|
||||
fixture_cache,
|
||||
fixture_state,
|
||||
fixture_runtime,
|
||||
fixture_tmp,
|
||||
fixture_heartbeat,
|
||||
fixture_mosaic_home,
|
||||
):
|
||||
directory.mkdir(mode=0o700)
|
||||
|
||||
# Authentication/settings are copied into fixture HOME so Pi never writes
|
||||
# under the operator's HOME. They are not emitted or modified in place.
|
||||
source_agent = Path.home() / ".pi" / "agent"
|
||||
target_agent = fixture_home / ".pi" / "agent"
|
||||
target_agent.mkdir(parents=True, mode=0o700)
|
||||
for name in ("settings.json", "auth.json", "bin/fd"):
|
||||
source = source_agent / name
|
||||
target = target_agent / name
|
||||
if source.is_file():
|
||||
target.parent.mkdir(parents=True, mode=0o700, exist_ok=True)
|
||||
shutil.copy2(source, target)
|
||||
|
||||
environment = {
|
||||
"HOME": str(fixture_home),
|
||||
"XDG_CONFIG_HOME": str(fixture_config),
|
||||
"XDG_CACHE_HOME": str(fixture_cache),
|
||||
"XDG_STATE_HOME": str(fixture_state),
|
||||
"XDG_RUNTIME_DIR": str(fixture_runtime),
|
||||
"TMPDIR": str(fixture_tmp),
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
"LANG": os.environ.get("LANG", "C.UTF-8"),
|
||||
"TERM": os.environ.get("TERM", "dumb"),
|
||||
"D4_GENERATION_SOCKET": str(socket_path),
|
||||
"MOSAIC_LEASE_BROKER_SOCKET": str(socket_path),
|
||||
"D4_PI_LOG": str(pi_log),
|
||||
"MOSAIC_AGENT_NAME": f"d4-fixture-{index}",
|
||||
"MOSAIC_AGENT_WORKDIR": str(workspace),
|
||||
"MOSAIC_HEARTBEAT_RUN_DIR": str(fixture_heartbeat),
|
||||
"MOSAIC_HOME": str(fixture_mosaic_home),
|
||||
"MOSAIC_PI_FORCE_SKILLS": "",
|
||||
"PI_SKIP_VERSION_CHECK": "1",
|
||||
"PYTHONDONTWRITEBYTECODE": "1",
|
||||
"PYTHONNOUSERSITE": "1",
|
||||
}
|
||||
if "PI_CODING_AGENT" in os.environ:
|
||||
environment["PI_CODING_AGENT"] = os.environ["PI_CODING_AGENT"]
|
||||
return environment
|
||||
|
||||
|
||||
def scrub_fixture_credentials(root: Path) -> None:
|
||||
"""Remove the copied Pi credential/config subtree before retaining evidence."""
|
||||
|
||||
copied_agent = root / "home" / ".pi" / "agent"
|
||||
if copied_agent.exists():
|
||||
shutil.rmtree(copied_agent)
|
||||
if copied_agent.exists():
|
||||
raise RuntimeError("D4 credential scrub failed")
|
||||
|
||||
|
||||
def run_once(index: int) -> Path:
|
||||
root = Path(tempfile.mkdtemp(prefix=f"gate0-d4-{index}-"))
|
||||
workspace = root / "workspace"
|
||||
sessions = root / "sessions"
|
||||
workspace.mkdir(mode=0o700)
|
||||
sessions.mkdir(mode=0o700)
|
||||
socket_path = root / "generation.sock"
|
||||
generation_log = root / "generation.jsonl"
|
||||
pi_log = root / "pi.jsonl"
|
||||
extension = root / "d4_extension.ts"
|
||||
write_extension(extension)
|
||||
broker: subprocess.Popen[str] | None = None
|
||||
pi: PiRpc | None = None
|
||||
|
||||
try:
|
||||
environment = isolated_environment(root, index, workspace, socket_path, pi_log)
|
||||
# Must run before the fixture broker or Pi process is launched. It proves
|
||||
# the launcher registers before exec and can only read this fixture socket.
|
||||
closure = gated_launcher_precondition(root, socket_path, environment)
|
||||
broker = launch_verified_broker(
|
||||
closure.broker, closure.generation, socket_path, generation_log, environment
|
||||
)
|
||||
wait_path(socket_path)
|
||||
pi = launch_verified_pi(closure.launcher, workspace, sessions, extension, environment)
|
||||
pi.send({"id": "state", "type": "get_state"})
|
||||
state = pi.response("state")
|
||||
original_session = state["data"]["sessionFile"]
|
||||
pi.prompt_and_settle(
|
||||
"fixture-promote",
|
||||
"Call d4_fixture_promote exactly once, then stop.",
|
||||
)
|
||||
pi.send({"id": "reload", "type": "prompt", "message": "/d4-reload"})
|
||||
reload_response = pi.response("reload")
|
||||
if not reload_response.get("success"):
|
||||
raise RuntimeError(f"reload failed: {reload_response}")
|
||||
for request_id, request_payload in [
|
||||
("clone", {"id": "clone", "type": "clone"}),
|
||||
("new", {"id": "new", "type": "new_session"}),
|
||||
(
|
||||
"resume",
|
||||
{"id": "resume", "type": "switch_session", "sessionPath": original_session},
|
||||
),
|
||||
]:
|
||||
pi.send(request_payload)
|
||||
response = pi.response(request_id)
|
||||
if not response.get("success") or response.get("data", {}).get("cancelled"):
|
||||
raise RuntimeError(f"{request_id} failed: {response}")
|
||||
results = assert_d4(jsonl(generation_log))
|
||||
verdict = results.get("machine_assertions")
|
||||
if verdict != "PASS":
|
||||
raise RuntimeError(f"D4 checks did not derive PASS: {verdict}")
|
||||
(root / "machine-assertions.json").write_text(
|
||||
json.dumps(results, sort_keys=True, indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(f"run={index} evidence_dir={root}")
|
||||
print(f"machine_assertions={verdict}")
|
||||
print(json.dumps(results, sort_keys=True))
|
||||
except Exception as error:
|
||||
(root / "machine-assertions.json").write_text(
|
||||
json.dumps({"error": f"{type(error).__name__}: {error}"}, sort_keys=True, indent=2)
|
||||
+ "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(f"run={index} evidence_dir={root}")
|
||||
print("machine_assertions=FAIL")
|
||||
print(f"error={type(error).__name__}: {error}")
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
try:
|
||||
if pi is not None:
|
||||
pi.close()
|
||||
finally:
|
||||
if broker is not None:
|
||||
try:
|
||||
request(socket_path, {"action": "shutdown-broker"})
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
broker.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
broker.kill()
|
||||
broker.wait()
|
||||
finally:
|
||||
scrub_fixture_credentials(root)
|
||||
return root
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--runs", type=int, default=3, choices=(3,))
|
||||
args = parser.parse_args()
|
||||
roots: list[Path] = []
|
||||
for index in range(1, args.runs + 1):
|
||||
roots.append(run_once(index))
|
||||
print("d4_isolation_runs=" + ",".join(str(root) for root in roots))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
215
docs/compaction-refresh/probes/p3_generation_broker.py
Normal file
215
docs/compaction-refresh/probes/p3_generation_broker.py
Normal file
@@ -0,0 +1,215 @@
|
||||
#!/usr/bin/env python3
|
||||
"""P3 broker prototype: peercred-keyed runtime_generation and lease revocation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import struct
|
||||
from collections.abc import Callable, Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def proc_starttime(pid: int) -> int:
|
||||
text = Path(f"/proc/{pid}/stat").read_text()
|
||||
close = text.rfind(")")
|
||||
return int(text[close + 2 :].split()[19])
|
||||
|
||||
|
||||
def emit(log: Path, value: dict[str, Any]) -> None:
|
||||
with log.open("a", encoding="utf-8") as out:
|
||||
out.write(json.dumps(value, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def load_generation_functions(
|
||||
path: Path,
|
||||
) -> tuple[Callable[[Mapping[str, str]], int], Callable[[Mapping[str, str]], int]]:
|
||||
spec = importlib.util.spec_from_file_location("d4_lease_generation", path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ValueError("generation module is unavailable")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
reader = getattr(module, "read_runtime_generation", None)
|
||||
bumper = getattr(module, "bump_runtime_generation", None)
|
||||
if not callable(reader) or not callable(bumper):
|
||||
raise ValueError("generation module has no read/bump functions")
|
||||
return reader, bumper
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--socket", required=True)
|
||||
parser.add_argument("--log", required=True)
|
||||
parser.add_argument("--generation-module", required=True, type=Path)
|
||||
ns = parser.parse_args()
|
||||
socket_path = Path(ns.socket)
|
||||
log_path = Path(ns.log)
|
||||
read_runtime_generation, bump_runtime_generation = load_generation_functions(
|
||||
ns.generation_module
|
||||
)
|
||||
socket_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(socket_path.parent, 0o700)
|
||||
socket_path.unlink(missing_ok=True)
|
||||
log_path.unlink(missing_ok=True)
|
||||
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(str(socket_path))
|
||||
os.chmod(socket_path, 0o600)
|
||||
server.listen(8)
|
||||
generations: dict[tuple[int, int], int] = {}
|
||||
lease_state: dict[tuple[int, int], str] = {}
|
||||
# The gated launcher registers its own exec-preserved PID here. This is
|
||||
# deliberately volatile fixture state; nothing is written outside root.
|
||||
launcher_sessions: dict[tuple[int, int], str] = {}
|
||||
generation_files: dict[tuple[int, int], Path] = {}
|
||||
|
||||
def generation_environment(identity: tuple[int, int]) -> dict[str, str]:
|
||||
state_path = generation_files.get(identity)
|
||||
if state_path is None or state_path.parent != socket_path.parent:
|
||||
raise ValueError("generation file is outside the fixture root")
|
||||
return {"MOSAIC_LEASE_GENERATION_FILE": str(state_path)}
|
||||
|
||||
def file_generation(identity: tuple[int, int]) -> int:
|
||||
return read_runtime_generation(generation_environment(identity))
|
||||
|
||||
emit(log_path, {"event": "listen", "pid": os.getpid(), "socket": str(socket_path)})
|
||||
|
||||
while True:
|
||||
conn, _ = server.accept()
|
||||
with conn:
|
||||
raw = conn.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, 12)
|
||||
pid, uid, gid = struct.unpack("3i", raw)
|
||||
starttime = proc_starttime(pid)
|
||||
request = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
if request.get("action") == "shutdown-broker":
|
||||
conn.sendall(b'{"ok":true}\n')
|
||||
break
|
||||
identity = (pid, starttime)
|
||||
if request.get("action") == "register_anchor":
|
||||
generation = request.get("runtime_generation")
|
||||
if type(generation) is not int or generation < 0:
|
||||
conn.sendall(b'{"ok":false,"code":"INVALID_GENERATION"}\n')
|
||||
continue
|
||||
session_id = launcher_sessions.setdefault(identity, secrets.token_hex(32))
|
||||
generation_file = socket_path.parent / f"generation-{session_id}.state"
|
||||
generation_files[identity] = generation_file
|
||||
record = {
|
||||
"event": "lease_anchor_registered",
|
||||
"peercred": {"pid": pid, "uid": uid, "gid": gid},
|
||||
"starttime_ticks": starttime,
|
||||
"runtime_generation": generation,
|
||||
"session_id_shape": "hex-256",
|
||||
"generation_file": str(generation_file),
|
||||
}
|
||||
emit(log_path, record)
|
||||
reply = {
|
||||
"ok": True,
|
||||
"session_id": session_id,
|
||||
"peer": {"pid": pid, "uid": uid, "gid": gid, "starttime": str(starttime)},
|
||||
}
|
||||
conn.sendall((json.dumps(reply, sort_keys=True) + "\n").encode())
|
||||
continue
|
||||
# The D4 extension requests this at each post-start lifecycle
|
||||
# boundary; the exact WI-3 helper mutates the launcher-created file.
|
||||
if request.get("action") == "bump-generation":
|
||||
generation = bump_runtime_generation(generation_environment(identity))
|
||||
record = {
|
||||
"event": "generation_state_bumped",
|
||||
"peercred": {"pid": pid, "uid": uid, "gid": gid},
|
||||
"starttime_ticks": starttime,
|
||||
"generation": generation,
|
||||
"generation_file": str(generation_files[identity]),
|
||||
"generation_source": "state-file",
|
||||
}
|
||||
emit(log_path, record)
|
||||
conn.sendall((json.dumps(record, sort_keys=True) + "\n").encode())
|
||||
continue
|
||||
if request.get("action") == "promote-probe":
|
||||
generation = file_generation(identity)
|
||||
lease_state[identity] = "VERIFIED"
|
||||
record = {
|
||||
"event": "probe_lease_promoted",
|
||||
"peercred": {"pid": pid, "uid": uid, "gid": gid},
|
||||
"starttime_ticks": starttime,
|
||||
"generation": generation,
|
||||
"generation_file": str(generation_files[identity]),
|
||||
"generation_source": "state-file",
|
||||
"new_lease_state": "VERIFIED",
|
||||
}
|
||||
emit(log_path, record)
|
||||
conn.sendall((json.dumps(record, sort_keys=True) + "\n").encode())
|
||||
continue
|
||||
# D4 fixture-only authorization observation. It exposes the broker's
|
||||
# current versus superseded generation disposition without changing it.
|
||||
if request.get("action") == "authorize-probe":
|
||||
generation = request.get("generation")
|
||||
if type(generation) is not int or generation < 0:
|
||||
conn.sendall(b'{"ok":false,"code":"INVALID_GENERATION"}\n')
|
||||
continue
|
||||
current_generation = file_generation(identity)
|
||||
current_lease = lease_state.get(identity, "NONE")
|
||||
if generation < current_generation:
|
||||
code = "STALE_GENERATION"
|
||||
elif generation > current_generation:
|
||||
code = "FUTURE_GENERATION"
|
||||
elif current_lease != "VERIFIED":
|
||||
code = "MUTATOR_UNVERIFIED"
|
||||
else:
|
||||
code = "ALLOW"
|
||||
record = {
|
||||
"event": "generation_authorization",
|
||||
"peercred": {"pid": pid, "uid": uid, "gid": gid},
|
||||
"starttime_ticks": starttime,
|
||||
"requested_generation": generation,
|
||||
"current_generation": current_generation,
|
||||
"generation_file": str(generation_files[identity]),
|
||||
"generation_source": "state-file",
|
||||
"lease_state": current_lease,
|
||||
"ok": code == "ALLOW",
|
||||
"code": code,
|
||||
}
|
||||
emit(log_path, record)
|
||||
conn.sendall((json.dumps(record, sort_keys=True) + "\n").encode())
|
||||
continue
|
||||
if request.get("action") != "lifecycle":
|
||||
conn.sendall(b'{"ok":false,"reason":"invalid-action"}\n')
|
||||
continue
|
||||
|
||||
old_generation = generations.get(identity, 0)
|
||||
old_lease = lease_state.get(identity, "NONE")
|
||||
new_generation = file_generation(identity)
|
||||
if new_generation <= old_generation:
|
||||
conn.sendall(b'{"ok":false,"code":"NON_MONOTONIC_STATE_FILE"}\n')
|
||||
continue
|
||||
generations[identity] = new_generation
|
||||
# Every lifecycle boundary revokes first. A start establishes a new
|
||||
# UNVERIFIED incarnation; it never inherits prior VERIFIED state.
|
||||
lease_state[identity] = "UNVERIFIED" if request.get("phase") == "start" else "REVOKED"
|
||||
record = {
|
||||
"event": "runtime_generation_bump",
|
||||
"peercred": {"pid": pid, "uid": uid, "gid": gid},
|
||||
"starttime_ticks": starttime,
|
||||
"phase": request.get("phase"),
|
||||
"reason": request.get("reason"),
|
||||
"old_generation": old_generation,
|
||||
"new_generation": new_generation,
|
||||
"generation_file": str(generation_files[identity]),
|
||||
"generation_source": "state-file",
|
||||
"prior_lease": old_lease,
|
||||
"prior_lease_revoked": True,
|
||||
"new_lease_state": lease_state[identity],
|
||||
}
|
||||
emit(log_path, record)
|
||||
conn.sendall((json.dumps(record, sort_keys=True) + "\n").encode())
|
||||
|
||||
server.close()
|
||||
socket_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
97
docs/compaction-refresh/probes/p4_peercred_probe.py
Normal file
97
docs/compaction-refresh/probes/p4_peercred_probe.py
Normal file
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Gate0 P4: exercise Linux SO_PEERCRED and correlate it to /proc."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import stat
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def proc_identity(pid: int) -> dict[str, int | str]:
|
||||
stat_text = Path(f"/proc/{pid}/stat").read_text()
|
||||
close = stat_text.rfind(")")
|
||||
fields = stat_text[close + 2 :].split()
|
||||
# fields[0] is field 3 (state); ppid is field 4 and starttime is field 22.
|
||||
return {
|
||||
"pid": pid,
|
||||
"ppid": int(fields[1]),
|
||||
"starttime_ticks": int(fields[19]),
|
||||
"uid": int(Path(f"/proc/{pid}/status").read_text().split("Uid:", 1)[1].split()[0]),
|
||||
"exe": os.readlink(f"/proc/{pid}/exe"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="gate0-p4-") as tmp:
|
||||
root = Path(tmp)
|
||||
os.chmod(root, 0o700)
|
||||
socket_path = root / "broker.sock"
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(str(socket_path))
|
||||
os.chmod(socket_path, 0o600)
|
||||
server.listen(1)
|
||||
|
||||
child = os.fork()
|
||||
if child == 0:
|
||||
client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
client.connect(str(socket_path))
|
||||
identity = proc_identity(os.getpid())
|
||||
client.sendall((json.dumps(identity, sort_keys=True) + "\n").encode())
|
||||
# Keep /proc/<pid> alive until the server has correlated peercred.
|
||||
if client.recv(2) != b"OK":
|
||||
os._exit(2)
|
||||
client.close()
|
||||
os._exit(0)
|
||||
|
||||
conn, _ = server.accept()
|
||||
raw = conn.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, 12)
|
||||
peer_pid = int.from_bytes(raw[0:4], byteorder="little", signed=True)
|
||||
peer_uid = int.from_bytes(raw[4:8], byteorder="little", signed=True)
|
||||
peer_gid = int.from_bytes(raw[8:12], byteorder="little", signed=True)
|
||||
claimed = json.loads(conn.makefile("r", encoding="utf-8").readline())
|
||||
observed = proc_identity(peer_pid)
|
||||
conn.sendall(b"OK")
|
||||
_, status = os.waitpid(child, 0)
|
||||
|
||||
root_mode = stat.S_IMODE(root.stat().st_mode)
|
||||
socket_mode = stat.S_IMODE(socket_path.stat().st_mode)
|
||||
if not (
|
||||
peer_pid == claimed["pid"] == observed["pid"]
|
||||
and peer_uid == claimed["uid"] == observed["uid"]
|
||||
and claimed["starttime_ticks"] == observed["starttime_ticks"]
|
||||
and root_mode == 0o700
|
||||
and socket_mode == 0o600
|
||||
and os.waitstatus_to_exitcode(status) == 0
|
||||
):
|
||||
raise AssertionError("SO_PEERCRED, /proc identity, or socket-mode correlation failed")
|
||||
print("machine_assertions=PASS")
|
||||
print(f"server_pid={os.getpid()} server_uid={os.getuid()} server_gid={os.getgid()}")
|
||||
print(f"socket_path={socket_path}")
|
||||
print(f"directory_mode={root_mode:04o} socket_mode={socket_mode:04o}")
|
||||
print(f"SO_PEERCRED pid={peer_pid} uid={peer_uid} gid={peer_gid}")
|
||||
print("client_claim=" + json.dumps(claimed, sort_keys=True))
|
||||
print("proc_observed=" + json.dumps(observed, sort_keys=True))
|
||||
print(f"pid_match={peer_pid == claimed['pid'] == observed['pid']}")
|
||||
print(f"uid_match={peer_uid == claimed['uid'] == observed['uid']}")
|
||||
print(
|
||||
"starttime_match="
|
||||
+ str(claimed["starttime_ticks"] == observed["starttime_ticks"])
|
||||
)
|
||||
print(f"client_exit_status={os.waitstatus_to_exitcode(status)}")
|
||||
print("same_principal_socket=true")
|
||||
print(
|
||||
"posture=0700 parent + 0600 socket excludes other UIDs, but does not prevent "
|
||||
"the same UID from unlinking/rebinding; distinct-principal system service remains "
|
||||
"required for a claim stronger than T-C against same-UID counterfeit replacement"
|
||||
)
|
||||
|
||||
conn.close()
|
||||
server.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
54
docs/compaction-refresh/probes/p6_claude_hook.py
Normal file
54
docs/compaction-refresh/probes/p6_claude_hook.py
Normal file
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Claude SessionStart additionalContext producer for P6 observation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
BLOCK = "\n".join(
|
||||
[
|
||||
"GATE0_CLAUDE_ATOMIC_BEGIN",
|
||||
"segment-01=alpha-2d11",
|
||||
"segment-02=middle-8e22",
|
||||
"segment-03=omega-4f33",
|
||||
"GATE0_CLAUDE_ATOMIC_END",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def starttime(pid: int) -> int:
|
||||
text = Path(f"/proc/{pid}/stat").read_text()
|
||||
return int(text[text.rfind(")") + 2 :].split()[19])
|
||||
|
||||
|
||||
def main() -> None:
|
||||
hook_input = json.load(sys.stdin)
|
||||
log = Path(os.environ["GATE0_CLAUDE_HOOK_LOG"])
|
||||
record = {
|
||||
"hook_event_name": hook_input.get("hook_event_name"),
|
||||
"pid": os.getpid(),
|
||||
"ppid": os.getppid(),
|
||||
"starttime_ticks": starttime(os.getpid()),
|
||||
"block_length": len(BLOCK.encode()),
|
||||
"block_sha256": hashlib.sha256(BLOCK.encode()).hexdigest(),
|
||||
"emission": "one hookSpecificOutput.additionalContext string field",
|
||||
}
|
||||
log.write_text(json.dumps(record, sort_keys=True) + "\n")
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "SessionStart",
|
||||
"additionalContext": BLOCK,
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
145
docs/compaction-refresh/probes/p6_claude_run.py
Normal file
145
docs/compaction-refresh/probes/p6_claude_run.py
Normal file
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run real Claude 2.1.x through `mosaic yolo` for P6 observation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
BLOCK = "\n".join(
|
||||
[
|
||||
"GATE0_CLAUDE_ATOMIC_BEGIN",
|
||||
"segment-01=alpha-2d11",
|
||||
"segment-02=middle-8e22",
|
||||
"segment-03=omega-4f33",
|
||||
"GATE0_CLAUDE_ATOMIC_END",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def strings(value: Any):
|
||||
if isinstance(value, str):
|
||||
yield value
|
||||
elif isinstance(value, list):
|
||||
for item in value:
|
||||
yield from strings(item)
|
||||
elif isinstance(value, dict):
|
||||
for item in value.values():
|
||||
yield from strings(item)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="gate0-p6-claude-") as temp:
|
||||
root = Path(temp)
|
||||
workspace = root / "workspace"
|
||||
workspace.mkdir()
|
||||
settings = root / "settings.json"
|
||||
hook_log = root / "hook.jsonl"
|
||||
settings.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"hooks": {
|
||||
"SessionStart": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": f'python3 "{HERE / "p6_claude_hook.py"}"',
|
||||
"timeout": 20,
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
env = os.environ.copy()
|
||||
env["GATE0_CLAUDE_HOOK_LOG"] = str(hook_log)
|
||||
command = [
|
||||
"mosaic",
|
||||
"yolo",
|
||||
"claude",
|
||||
"--settings",
|
||||
str(settings),
|
||||
"--model",
|
||||
"haiku",
|
||||
"--print",
|
||||
"--output-format",
|
||||
"stream-json",
|
||||
"--verbose",
|
||||
"--include-hook-events",
|
||||
"--max-budget-usd",
|
||||
"0.10",
|
||||
"Return only the exact full GATE0_CLAUDE_ATOMIC_BEGIN through GATE0_CLAUDE_ATOMIC_END block injected by SessionStart, with no code fence or commentary.",
|
||||
]
|
||||
result = subprocess.run(
|
||||
command,
|
||||
cwd=workspace,
|
||||
env=env,
|
||||
stdin=subprocess.DEVNULL,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=150,
|
||||
check=False,
|
||||
)
|
||||
events: list[dict[str, Any]] = []
|
||||
for line in result.stdout.splitlines():
|
||||
try:
|
||||
events.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
hook_events = [
|
||||
event
|
||||
for event in events
|
||||
if event.get("type") == "system"
|
||||
and event.get("subtype") in {"hook_started", "hook_response"}
|
||||
]
|
||||
full_matches = [text for event in events for text in strings(event) if BLOCK in text]
|
||||
exact_matches = [text for event in events for text in strings(event) if text == BLOCK]
|
||||
assistant_texts: list[str] = []
|
||||
for event in events:
|
||||
if event.get("type") != "assistant":
|
||||
continue
|
||||
for text in strings(event.get("message", {})):
|
||||
if "GATE0_CLAUDE_ATOMIC_BEGIN" in text:
|
||||
assistant_texts.append(text)
|
||||
|
||||
if result.returncode != 0:
|
||||
raise AssertionError(f"Claude probe exited {result.returncode}")
|
||||
if not any(event.get("subtype") == "hook_response" and event.get("outcome") == "success" for event in hook_events):
|
||||
raise AssertionError("Claude SessionStart hook did not complete successfully")
|
||||
if BLOCK not in exact_matches:
|
||||
raise AssertionError("Claude did not return an exact full-block field")
|
||||
|
||||
print("$ python3 docs/compaction-refresh/probes/p6_claude_run.py")
|
||||
print("machine_assertions=PASS")
|
||||
print("command=mosaic yolo claude --settings <isolated> --model haiku --print --output-format stream-json --verbose --include-hook-events <prompt>")
|
||||
print("claude_version=" + subprocess.check_output(["claude", "--version"], text=True).strip())
|
||||
print("mosaic_version=" + subprocess.check_output(["mosaic", "--version"], text=True).strip())
|
||||
print(f"exit_code={result.returncode}")
|
||||
print("hook_process_log=" + hook_log.read_text().strip())
|
||||
for event in hook_events:
|
||||
print("hook_stream_event=" + json.dumps(event, sort_keys=True))
|
||||
print(f"block_length={len(BLOCK.encode())}")
|
||||
print(f"block_sha256={hashlib.sha256(BLOCK.encode()).hexdigest()}")
|
||||
print(f"stream_fields_containing_full_block={len(full_matches)}")
|
||||
print(f"stream_fields_exactly_equal_block={len(exact_matches)}")
|
||||
for text in assistant_texts:
|
||||
print(f"assistant_copy_length={len(text.encode())}")
|
||||
print(f"assistant_copy_sha256={hashlib.sha256(text.encode()).hexdigest()}")
|
||||
print(f"assistant_copy_exact={text == BLOCK}")
|
||||
print("assistant_copy=" + json.dumps(text))
|
||||
if result.stderr.strip():
|
||||
print("stderr_excerpt=" + json.dumps(result.stderr.splitlines()[:10]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
350
docs/compaction-refresh/probes/pi_gate0_extension.ts
Normal file
350
docs/compaction-refresh/probes/pi_gate0_extension.ts
Normal file
@@ -0,0 +1,350 @@
|
||||
import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
|
||||
import { Type } from 'typebox';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { readFileSync, appendFileSync, statSync } from 'node:fs';
|
||||
import net from 'node:net';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { resolve } from 'node:path';
|
||||
|
||||
const SELF = resolve(fileURLToPath(import.meta.url).split('?')[0]!);
|
||||
const LOG = process.env['GATE0_PI_LOG'];
|
||||
const CONTEXT_BLOCK =
|
||||
process.env['GATE0_PI_CONTEXT_BLOCK'] ??
|
||||
[
|
||||
'GATE0_PI_ATOMIC_BEGIN',
|
||||
'segment-01=alpha-7e31',
|
||||
'segment-02=middle-9c42',
|
||||
'segment-03=omega-5b83',
|
||||
'GATE0_PI_ATOMIC_END',
|
||||
].join('\n');
|
||||
|
||||
let sequence = 0;
|
||||
|
||||
function sha(value: string | Buffer): string {
|
||||
return createHash('sha256').update(value).digest('hex');
|
||||
}
|
||||
|
||||
function procStarttime(): number {
|
||||
const text = readFileSync(`/proc/${process.pid}/stat`, 'utf8');
|
||||
const close = text.lastIndexOf(')');
|
||||
const fields = text.slice(close + 2).trim().split(/\s+/);
|
||||
return Number(fields[19]);
|
||||
}
|
||||
|
||||
function log(event: string, details: Record<string, unknown> = {}): void {
|
||||
if (!LOG) return;
|
||||
sequence += 1;
|
||||
appendFileSync(
|
||||
LOG,
|
||||
`${JSON.stringify({ seq: sequence, event, pid: process.pid, starttime_ticks: procStarttime(), ...details })}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
function argvExtensions(): string[] {
|
||||
const result: string[] = [];
|
||||
for (let i = 0; i < process.argv.length; i += 1) {
|
||||
if (process.argv[i] === '--extension' || process.argv[i] === '-e') {
|
||||
const candidate = process.argv[i + 1];
|
||||
if (candidate) result.push(resolve(candidate));
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
interface SourceValidation {
|
||||
ok: boolean;
|
||||
reason: string;
|
||||
fragment?: string;
|
||||
}
|
||||
|
||||
function validateSources(): SourceValidation {
|
||||
const manifestPath = process.env['GATE0_SOURCE_MANIFEST'];
|
||||
if (!manifestPath) return { ok: true, reason: 'no-manifest-probe-disabled' };
|
||||
try {
|
||||
const manifest = JSON.parse(readFileSync(manifestPath, 'utf8')) as {
|
||||
maxBytes: number;
|
||||
fragments: Array<{ path: string; sha256: string }>;
|
||||
};
|
||||
for (const fragment of manifest.fragments) {
|
||||
let fileStat;
|
||||
try {
|
||||
fileStat = statSync(fragment.path);
|
||||
} catch {
|
||||
return { ok: false, reason: 'missing', fragment: fragment.path };
|
||||
}
|
||||
if (!fileStat.isFile()) {
|
||||
return { ok: false, reason: 'not-regular-file', fragment: fragment.path };
|
||||
}
|
||||
if (fileStat.size > manifest.maxBytes) {
|
||||
return { ok: false, reason: 'oversize', fragment: fragment.path };
|
||||
}
|
||||
const bytes = readFileSync(fragment.path);
|
||||
if (sha(bytes) !== fragment.sha256) {
|
||||
return { ok: false, reason: 'hash-mismatch', fragment: fragment.path };
|
||||
}
|
||||
}
|
||||
return { ok: true, reason: 'all-fragments-valid' };
|
||||
} catch (error) {
|
||||
return { ok: false, reason: `manifest-error:${error instanceof Error ? error.name : 'unknown'}` };
|
||||
}
|
||||
}
|
||||
|
||||
function brokerRequest(payload: Record<string, unknown>): Promise<Record<string, unknown>> {
|
||||
const socketPath = process.env['GATE0_GENERATION_SOCKET'];
|
||||
if (!socketPath) return Promise.resolve({ skipped: true });
|
||||
return new Promise((resolvePromise, reject) => {
|
||||
const socket = net.createConnection(socketPath);
|
||||
let buffer = '';
|
||||
socket.setEncoding('utf8');
|
||||
socket.on('connect', () => socket.write(`${JSON.stringify(payload)}\n`));
|
||||
socket.on('data', (chunk) => {
|
||||
buffer += chunk;
|
||||
const newline = buffer.indexOf('\n');
|
||||
if (newline < 0) return;
|
||||
socket.end();
|
||||
resolvePromise(JSON.parse(buffer.slice(0, newline)) as Record<string, unknown>);
|
||||
});
|
||||
socket.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
function markerPaths(value: unknown, path = '$'): string[] {
|
||||
const matches: string[] = [];
|
||||
if (typeof value === 'string') {
|
||||
if (value.includes(CONTEXT_BLOCK)) matches.push(path);
|
||||
return matches;
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
value.forEach((item, index) => matches.push(...markerPaths(item, `${path}[${index}]`)));
|
||||
return matches;
|
||||
}
|
||||
if (value && typeof value === 'object') {
|
||||
for (const [key, item] of Object.entries(value as Record<string, unknown>)) {
|
||||
matches.push(...markerPaths(item, `${path}.${key}`));
|
||||
}
|
||||
}
|
||||
return matches;
|
||||
}
|
||||
|
||||
function assistantToolIds(message: unknown): string[] {
|
||||
if (!message || typeof message !== 'object') return [];
|
||||
const candidate = message as { role?: string; content?: unknown };
|
||||
if (candidate.role !== 'assistant' || !Array.isArray(candidate.content)) return [];
|
||||
return candidate.content
|
||||
.filter(
|
||||
(block): block is { type: 'toolCall'; id: string } =>
|
||||
Boolean(
|
||||
block &&
|
||||
typeof block === 'object' &&
|
||||
(block as { type?: string }).type === 'toolCall' &&
|
||||
typeof (block as { id?: unknown }).id === 'string',
|
||||
),
|
||||
)
|
||||
.map((block) => block.id);
|
||||
}
|
||||
|
||||
function assistantText(message: unknown): string {
|
||||
if (!message || typeof message !== 'object') return '';
|
||||
const candidate = message as { role?: string; content?: unknown };
|
||||
if (candidate.role !== 'assistant' || !Array.isArray(candidate.content)) return '';
|
||||
return candidate.content
|
||||
.filter(
|
||||
(block): block is { type: 'text'; text: string } =>
|
||||
Boolean(
|
||||
block &&
|
||||
typeof block === 'object' &&
|
||||
(block as { type?: string }).type === 'text' &&
|
||||
typeof (block as { text?: unknown }).text === 'string',
|
||||
),
|
||||
)
|
||||
.map((block) => block.text)
|
||||
.join('');
|
||||
}
|
||||
|
||||
export default function register(pi: ExtensionAPI) {
|
||||
const localProviderUrl = process.env['GATE0_LOCAL_PROVIDER_URL'];
|
||||
if (localProviderUrl) {
|
||||
pi.registerProvider('gate0-local', {
|
||||
baseUrl: localProviderUrl,
|
||||
apiKey: 'gate0-probe-not-a-secret',
|
||||
api: 'openai-completions',
|
||||
models: [
|
||||
{
|
||||
id: 'gate0-model',
|
||||
name: 'Gate0 deterministic local model',
|
||||
reasoning: false,
|
||||
input: ['text'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 32_000,
|
||||
maxTokens: 1_024,
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
const extensions = argvExtensions();
|
||||
const lastPosition = extensions.length > 0 && extensions.at(-1) === SELF;
|
||||
interface RequestCycle {
|
||||
nonce: string;
|
||||
verified: boolean;
|
||||
sourceReason: string;
|
||||
}
|
||||
let buildingCycle: RequestCycle | undefined;
|
||||
const inFlightCycles: RequestCycle[] = [];
|
||||
const toolNonce = new Map<string, { nonce: string; verified: boolean; sourceReason: string }>();
|
||||
|
||||
pi.on('session_start', async (event) => {
|
||||
const broker = await brokerRequest({ action: 'lifecycle', phase: 'start', reason: event.reason });
|
||||
log('session_start', {
|
||||
reason: event.reason,
|
||||
extensions,
|
||||
self: SELF,
|
||||
lastPosition,
|
||||
gateState: lastPosition ? 'UNVERIFIED_READY' : 'CLOSED_NOT_LAST',
|
||||
broker,
|
||||
});
|
||||
});
|
||||
|
||||
pi.on('session_shutdown', async (event) => {
|
||||
const broker = await brokerRequest({ action: 'lifecycle', phase: 'shutdown', reason: event.reason });
|
||||
log('session_shutdown', { reason: event.reason, broker });
|
||||
});
|
||||
|
||||
pi.on('context', async (event) => {
|
||||
const validation = validateSources();
|
||||
buildingCycle = {
|
||||
nonce: randomUUID(),
|
||||
sourceReason: validation.reason,
|
||||
verified: lastPosition && validation.ok,
|
||||
};
|
||||
const inputJson = JSON.stringify(event.messages);
|
||||
const injected = {
|
||||
role: 'custom' as const,
|
||||
customType: 'gate0-context',
|
||||
content: CONTEXT_BLOCK,
|
||||
display: false,
|
||||
timestamp: Date.now(),
|
||||
};
|
||||
const outputMessages = buildingCycle.verified
|
||||
? [...event.messages, injected]
|
||||
: [...event.messages];
|
||||
const outputPrefix = outputMessages.slice(0, event.messages.length);
|
||||
const sourceBroker = validation.ok
|
||||
? { action: 'none', reason: 'source-valid' }
|
||||
: await brokerRequest({ action: 'source-invalid', reason: validation.reason });
|
||||
log('context_return', {
|
||||
requestNonce: buildingCycle.nonce,
|
||||
sourceValidation: validation,
|
||||
sourceBroker,
|
||||
lastPosition,
|
||||
promotion: false,
|
||||
injectionDecision: buildingCycle.verified ? 'ONE_ATOMIC_AGENT_MESSAGE' : 'REFUSED',
|
||||
inputCount: event.messages.length,
|
||||
outputCount: outputMessages.length,
|
||||
prefixHashBefore: sha(inputJson),
|
||||
prefixHashAfter: sha(JSON.stringify(outputPrefix)),
|
||||
prefixPreservedByReturn: sha(inputJson) === sha(JSON.stringify(outputPrefix)),
|
||||
blockLength: CONTEXT_BLOCK.length,
|
||||
blockSha256: sha(CONTEXT_BLOCK),
|
||||
});
|
||||
return { messages: outputMessages };
|
||||
});
|
||||
|
||||
pi.on('before_provider_request', async (event) => {
|
||||
const paths = markerPaths(event.payload);
|
||||
const cycle = buildingCycle;
|
||||
buildingCycle = undefined;
|
||||
if (cycle) inFlightCycles.push(cycle);
|
||||
log('before_provider_request', {
|
||||
requestNonce: cycle?.nonce,
|
||||
inFlightDepth: inFlightCycles.length,
|
||||
markerOccurrences: paths.length,
|
||||
markerPaths: paths,
|
||||
finalPayloadValid: Boolean(cycle?.verified && paths.length === 1),
|
||||
});
|
||||
});
|
||||
|
||||
pi.on('after_provider_response', async (event) => {
|
||||
const cycle = inFlightCycles[0];
|
||||
log('after_provider_response', {
|
||||
requestNonce: cycle?.nonce,
|
||||
status: event.status,
|
||||
assistantContentAvailableAtThisHook: false,
|
||||
timing: 'headers/status before stream consumption',
|
||||
});
|
||||
});
|
||||
|
||||
pi.on('message_end', async (event) => {
|
||||
const role = (event.message as { role?: string }).role;
|
||||
const ids = assistantToolIds(event.message);
|
||||
const text = assistantText(event.message);
|
||||
const cycle = role === 'assistant' ? inFlightCycles.shift() : undefined;
|
||||
if (ids.length > 0 && cycle) {
|
||||
for (const id of ids) {
|
||||
toolNonce.set(id, {
|
||||
nonce: cycle.nonce,
|
||||
verified: cycle.verified,
|
||||
sourceReason: cycle.sourceReason,
|
||||
});
|
||||
}
|
||||
}
|
||||
log('message_end', {
|
||||
role,
|
||||
assistantContentObserved: role === 'assistant',
|
||||
requestNonce: cycle?.nonce,
|
||||
inFlightDepthAfter: inFlightCycles.length,
|
||||
toolCallIds: ids,
|
||||
nonceMappings: ids.map((id) => ({ toolCallId: id, requestNonce: cycle?.nonce })),
|
||||
exactContextBlockCopied: text.includes(CONTEXT_BLOCK),
|
||||
assistantTextSha256: text ? sha(text) : null,
|
||||
});
|
||||
});
|
||||
|
||||
pi.on('tool_call', async (event) => {
|
||||
const mapping = toolNonce.get(event.toolCallId);
|
||||
const allowed = Boolean(lastPosition && mapping?.verified);
|
||||
log('tool_call', {
|
||||
toolCallId: event.toolCallId,
|
||||
toolName: event.toolName,
|
||||
mapping: mapping ?? null,
|
||||
allowed,
|
||||
reason: !lastPosition
|
||||
? 'closed-not-last'
|
||||
: !mapping
|
||||
? 'unknown-tool-call-id'
|
||||
: !mapping.verified
|
||||
? `unverified-source:${mapping.sourceReason}`
|
||||
: 'exact-tool-call-id-mapped-to-verified-request-nonce',
|
||||
});
|
||||
if (!allowed) return { block: true, reason: 'Gate0 probe refused unverified tool batch' };
|
||||
});
|
||||
|
||||
pi.on('agent_settled', async () => {
|
||||
log('agent_settled', { retainedNonceMappingsBeforeClear: toolNonce.size });
|
||||
toolNonce.clear();
|
||||
});
|
||||
|
||||
pi.registerTool({
|
||||
name: 'gate0_nonce_probe',
|
||||
label: 'Gate0 Nonce Probe',
|
||||
description: 'Gate0-only harmless tool used to prove toolCallId to request-nonce correlation.',
|
||||
parameters: Type.Object({ label: Type.String() }),
|
||||
async execute(toolCallId, params) {
|
||||
const broker = await brokerRequest({ action: 'promote-probe' });
|
||||
log('tool_execute', { toolCallId, label: params.label, broker });
|
||||
return {
|
||||
content: [{ type: 'text', text: `gate0_nonce_probe executed for ${params.label}` }],
|
||||
details: { harmless: true },
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
pi.registerCommand('gate0-reload', {
|
||||
description: 'Trigger a real same-PID Pi extension/runtime reload.',
|
||||
handler: async (_args, ctx) => {
|
||||
log('reload_command_before');
|
||||
await ctx.reload();
|
||||
return;
|
||||
},
|
||||
});
|
||||
}
|
||||
450
docs/compaction-refresh/probes/pi_gate0_run.py
Normal file
450
docs/compaction-refresh/probes/pi_gate0_run.py
Normal file
@@ -0,0 +1,450 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Drive real Pi 0.80.x RPC for P2/P3/P5/P6 runtime evidence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import queue
|
||||
import shutil
|
||||
import signal
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
BLOCK = "\n".join(
|
||||
[
|
||||
"GATE0_PI_ATOMIC_BEGIN",
|
||||
"segment-01=alpha-7e31",
|
||||
"segment-02=middle-9c42",
|
||||
"segment-03=omega-5b83",
|
||||
"GATE0_PI_ATOMIC_END",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def wait_path(path: Path, timeout: float = 20) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if path.exists():
|
||||
return
|
||||
time.sleep(0.05)
|
||||
raise TimeoutError(f"timed out waiting for {path}")
|
||||
|
||||
|
||||
def socket_request(path: Path, payload: dict[str, object]) -> None:
|
||||
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
conn.connect(str(path))
|
||||
conn.sendall((json.dumps(payload) + "\n").encode())
|
||||
conn.makefile("r", encoding="utf-8").readline()
|
||||
conn.close()
|
||||
|
||||
|
||||
def jsonl(path: Path) -> list[dict[str, Any]]:
|
||||
if not path.exists():
|
||||
return []
|
||||
return [json.loads(line) for line in path.read_text().splitlines() if line]
|
||||
|
||||
|
||||
class PiRpc:
|
||||
def __init__(self, command: list[str], cwd: Path, env: dict[str, str]):
|
||||
self.process = subprocess.Popen(
|
||||
command,
|
||||
cwd=cwd,
|
||||
env=env,
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
start_new_session=True,
|
||||
)
|
||||
self.events: queue.Queue[dict[str, Any]] = queue.Queue()
|
||||
self.raw_lines: list[str] = []
|
||||
self.stderr_lines: list[str] = []
|
||||
threading.Thread(target=self._read_stdout, daemon=True).start()
|
||||
threading.Thread(target=self._read_stderr, daemon=True).start()
|
||||
|
||||
def _read_stdout(self) -> None:
|
||||
assert self.process.stdout is not None
|
||||
for line in self.process.stdout:
|
||||
stripped = line.rstrip("\n")
|
||||
self.raw_lines.append(stripped)
|
||||
try:
|
||||
event = json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
self.events.put(event)
|
||||
|
||||
def _read_stderr(self) -> None:
|
||||
assert self.process.stderr is not None
|
||||
for line in self.process.stderr:
|
||||
self.stderr_lines.append(line.rstrip("\n"))
|
||||
|
||||
def send(self, payload: dict[str, object]) -> None:
|
||||
assert self.process.stdin is not None
|
||||
self.process.stdin.write(json.dumps(payload) + "\n")
|
||||
self.process.stdin.flush()
|
||||
|
||||
def wait(self, predicate: Callable[[dict[str, Any]], bool], description: str, timeout: float = 180) -> dict[str, Any]:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if self.process.poll() is not None and self.events.empty():
|
||||
raise RuntimeError(
|
||||
f"Pi exited {self.process.returncode} while waiting for {description}: "
|
||||
+ " | ".join(self.stderr_lines[-5:])
|
||||
)
|
||||
try:
|
||||
event = self.events.get(timeout=0.2)
|
||||
except queue.Empty:
|
||||
continue
|
||||
if predicate(event):
|
||||
return event
|
||||
raise TimeoutError(f"timed out waiting for {description}")
|
||||
|
||||
def response(self, request_id: str, timeout: float = 180) -> dict[str, Any]:
|
||||
return self.wait(
|
||||
lambda event: event.get("type") == "response" and event.get("id") == request_id,
|
||||
f"response {request_id}",
|
||||
timeout,
|
||||
)
|
||||
|
||||
def prompt_and_settle(self, request_id: str, message: str) -> None:
|
||||
self.send({"id": request_id, "type": "prompt", "message": message})
|
||||
response = self.response(request_id)
|
||||
if not response.get("success"):
|
||||
raise RuntimeError(f"prompt rejected: {response}")
|
||||
self.wait(lambda event: event.get("type") == "agent_settled", f"agent_settled {request_id}")
|
||||
|
||||
def close(self) -> None:
|
||||
if self.process.poll() is None:
|
||||
try:
|
||||
os.killpg(self.process.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
self.process.wait(timeout=8)
|
||||
except subprocess.TimeoutExpired:
|
||||
os.killpg(self.process.pid, signal.SIGKILL)
|
||||
self.process.wait(timeout=5)
|
||||
|
||||
|
||||
def manifest(path: Path, fragment: Path, expected_hash: str, max_bytes: int = 64) -> None:
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"maxBytes": max_bytes,
|
||||
"fragments": [{"path": str(fragment), "sha256": expected_hash}],
|
||||
},
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def run_open(root: Path) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[str], list[str]]:
|
||||
workspace = root / "workspace"
|
||||
workspace.mkdir()
|
||||
session_dir = root / "sessions"
|
||||
session_dir.mkdir()
|
||||
pi_log = root / "pi-hooks.jsonl"
|
||||
generation_log = root / "generation.jsonl"
|
||||
generation_socket = root / "generation.sock"
|
||||
source_manifest = root / "manifest.json"
|
||||
valid_fragment = root / "fragment.md"
|
||||
valid_fragment.write_text("NORMATIVE-FRAGMENT-v1\n")
|
||||
expected = hashlib.sha256(valid_fragment.read_bytes()).hexdigest()
|
||||
manifest(source_manifest, valid_fragment, expected)
|
||||
|
||||
broker = subprocess.Popen(
|
||||
[
|
||||
sys.executable,
|
||||
str(HERE / "p3_generation_broker.py"),
|
||||
"--socket",
|
||||
str(generation_socket),
|
||||
"--log",
|
||||
str(generation_log),
|
||||
],
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
wait_path(generation_socket)
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"GATE0_PI_LOG": str(pi_log),
|
||||
"GATE0_GENERATION_SOCKET": str(generation_socket),
|
||||
"GATE0_SOURCE_MANIFEST": str(source_manifest),
|
||||
"GATE0_PI_CONTEXT_BLOCK": BLOCK,
|
||||
"MOSAIC_PI_FORCE_SKILLS": "",
|
||||
"PI_SKIP_VERSION_CHECK": "1",
|
||||
}
|
||||
)
|
||||
command = [
|
||||
"mosaic",
|
||||
"yolo",
|
||||
"pi",
|
||||
"--mode",
|
||||
"rpc",
|
||||
"--session-dir",
|
||||
str(session_dir),
|
||||
"--no-extensions",
|
||||
"--no-context-files",
|
||||
"--no-prompt-templates",
|
||||
"--model",
|
||||
"openai-codex/gpt-5.6-sol",
|
||||
"--thinking",
|
||||
"medium",
|
||||
"--extension",
|
||||
str(HERE / "pi_gate0_extension.ts"),
|
||||
]
|
||||
pi = PiRpc(command, workspace, env)
|
||||
try:
|
||||
pi.send({"id": "state-0", "type": "get_state"})
|
||||
state0 = pi.response("state-0")
|
||||
original_session = state0["data"]["sessionFile"]
|
||||
|
||||
pi.prompt_and_settle(
|
||||
"p2",
|
||||
"Call gate0_nonce_probe exactly once with label p2. After the tool finishes, copy the exact full GATE0_PI_ATOMIC_BEGIN through GATE0_PI_ATOMIC_END block from context, with no commentary.",
|
||||
)
|
||||
|
||||
# P3 immediately follows the valid P2 promotion so reload must revoke a
|
||||
# genuinely VERIFIED prior generation, not an already-invalid source run.
|
||||
pi.send({"id": "reload", "type": "prompt", "message": "/gate0-reload"})
|
||||
reload_response = pi.response("reload")
|
||||
if not reload_response.get("success"):
|
||||
raise RuntimeError(f"reload command failed: {reload_response}")
|
||||
|
||||
pi.send({"id": "clone", "type": "clone"})
|
||||
clone_response = pi.response("clone")
|
||||
if not clone_response.get("success") or clone_response.get("data", {}).get("cancelled"):
|
||||
raise RuntimeError(f"clone failed: {clone_response}")
|
||||
|
||||
pi.send({"id": "new", "type": "new_session"})
|
||||
new_response = pi.response("new")
|
||||
if not new_response.get("success") or new_response.get("data", {}).get("cancelled"):
|
||||
raise RuntimeError(f"new session failed: {new_response}")
|
||||
|
||||
pi.send(
|
||||
{
|
||||
"id": "resume",
|
||||
"type": "switch_session",
|
||||
"sessionPath": original_session,
|
||||
}
|
||||
)
|
||||
resume_response = pi.response("resume")
|
||||
if not resume_response.get("success") or resume_response.get("data", {}).get("cancelled"):
|
||||
raise RuntimeError(f"resume failed: {resume_response}")
|
||||
|
||||
# P5 missing fragment: action-time source validation must revoke/refuse.
|
||||
manifest(source_manifest, root / "absent-fragment.md", expected)
|
||||
pi.prompt_and_settle(
|
||||
"p5-missing",
|
||||
"Call gate0_nonce_probe exactly once with label p5-missing, then stop.",
|
||||
)
|
||||
|
||||
# P5 oversize fragment: expected hash is correct, size limit is not.
|
||||
oversize = root / "oversize.md"
|
||||
oversize.write_text("X" * 65)
|
||||
manifest(source_manifest, oversize, hashlib.sha256(oversize.read_bytes()).hexdigest(), 64)
|
||||
pi.prompt_and_settle(
|
||||
"p5-oversize",
|
||||
"Call gate0_nonce_probe exactly once with label p5-oversize, then stop.",
|
||||
)
|
||||
|
||||
# P5 hash mismatch: size is valid but bytes differ from expected.
|
||||
mismatch = root / "mismatch.md"
|
||||
mismatch.write_text("tampered\n")
|
||||
manifest(source_manifest, mismatch, expected, 64)
|
||||
pi.prompt_and_settle(
|
||||
"p5-hash",
|
||||
"Call gate0_nonce_probe exactly once with label p5-hash-mismatch, then stop.",
|
||||
)
|
||||
|
||||
time.sleep(1)
|
||||
return jsonl(pi_log), jsonl(generation_log), list(pi.raw_lines), list(pi.stderr_lines)
|
||||
finally:
|
||||
pi.close()
|
||||
try:
|
||||
socket_request(generation_socket, {"action": "shutdown-broker"})
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
broker.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
broker.kill()
|
||||
broker.wait()
|
||||
|
||||
|
||||
def run_closed(root: Path) -> list[dict[str, Any]]:
|
||||
workspace = root / "closed-workspace"
|
||||
workspace.mkdir()
|
||||
pi_log = root / "closed-hooks.jsonl"
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"GATE0_PI_LOG": str(pi_log),
|
||||
"MOSAIC_PI_FORCE_SKILLS": "",
|
||||
"PI_SKIP_VERSION_CHECK": "1",
|
||||
}
|
||||
)
|
||||
command = [
|
||||
"mosaic",
|
||||
"yolo",
|
||||
"pi",
|
||||
"--mode",
|
||||
"rpc",
|
||||
"--no-session",
|
||||
"--no-extensions",
|
||||
"--no-context-files",
|
||||
"--no-prompt-templates",
|
||||
"--extension",
|
||||
str(HERE / "pi_gate0_extension.ts"),
|
||||
"--extension",
|
||||
str(HERE / "pi_later_extension.ts"),
|
||||
]
|
||||
pi = PiRpc(command, workspace, env)
|
||||
try:
|
||||
pi.send({"id": "closed-state", "type": "get_state"})
|
||||
pi.response("closed-state")
|
||||
time.sleep(0.5)
|
||||
return jsonl(pi_log)
|
||||
finally:
|
||||
pi.close()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="gate0-pi-") as temp:
|
||||
root = Path(temp)
|
||||
records, generations, rpc_lines, stderr_lines = run_open(root)
|
||||
closed = run_closed(root)
|
||||
|
||||
p2_message = next(
|
||||
r for r in records if r["event"] == "message_end" and r.get("nonceMappings")
|
||||
)
|
||||
p2_tool = next(r for r in records if r["event"] == "tool_call" and r.get("allowed"))
|
||||
mapped = p2_message["nonceMappings"][0]
|
||||
assert mapped["toolCallId"] == p2_tool["toolCallId"]
|
||||
assert mapped["requestNonce"] == p2_tool["mapping"]["nonce"]
|
||||
assert next(r for r in records if r["event"] == "session_start")["lastPosition"] is True
|
||||
assert next(r for r in closed if r["event"] == "session_start")["gateState"] == "CLOSED_NOT_LAST"
|
||||
reload_revoke = next(
|
||||
r
|
||||
for r in generations
|
||||
if r["event"] == "runtime_generation_bump"
|
||||
and r.get("reason") == "reload"
|
||||
and r.get("phase") == "shutdown"
|
||||
)
|
||||
assert reload_revoke["prior_lease"] == "VERIFIED"
|
||||
assert reload_revoke["prior_lease_revoked"] is True
|
||||
for reason in {"missing", "oversize", "hash-mismatch"}:
|
||||
assert any(
|
||||
r["event"] == "context_return"
|
||||
and r.get("sourceValidation", {}).get("reason") == reason
|
||||
and r.get("injectionDecision") == "REFUSED"
|
||||
and r.get("promotion") is False
|
||||
for r in records
|
||||
)
|
||||
assert any(
|
||||
r["event"] == "tool_call"
|
||||
and r.get("mapping", {}).get("sourceReason") == reason
|
||||
and r.get("allowed") is False
|
||||
for r in records
|
||||
)
|
||||
assert any(
|
||||
r["event"] == "message_end" and r.get("exactContextBlockCopied") is True
|
||||
for r in records
|
||||
)
|
||||
|
||||
print("$ python3 docs/compaction-refresh/probes/pi_gate0_run.py")
|
||||
print("machine_assertions=PASS")
|
||||
print("runtime_versions:")
|
||||
print(" " + subprocess.check_output(["pi", "--version"], text=True).strip())
|
||||
print(" " + subprocess.check_output(["mosaic", "--version"], text=True).strip())
|
||||
|
||||
print("\nP2_EVENT_ORDER_AND_NONCE_MAP:")
|
||||
for record in records:
|
||||
if record["seq"] <= 12 and record["event"] in {
|
||||
"after_provider_response",
|
||||
"message_end",
|
||||
"tool_call",
|
||||
"tool_execute",
|
||||
} and (
|
||||
record["event"] != "message_end"
|
||||
or record.get("role") == "assistant"
|
||||
):
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
|
||||
print("\nP2_LAST_OR_CLOSED:")
|
||||
print(json.dumps(next(r for r in records if r["event"] == "session_start"), sort_keys=True))
|
||||
print(json.dumps(next(r for r in closed if r["event"] == "session_start"), sort_keys=True))
|
||||
|
||||
print("\nP3_GENERATION_BROKER:")
|
||||
for record in generations:
|
||||
if record["event"] in {"probe_lease_promoted", "runtime_generation_bump"}:
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
|
||||
print("\nP5_SOURCE_INVALIDATION:")
|
||||
fault_reasons = {"missing", "oversize", "hash-mismatch"}
|
||||
emitted_context: set[str] = set()
|
||||
emitted_tool: set[str] = set()
|
||||
for record in records:
|
||||
source_reason = record.get("sourceValidation", {}).get("reason")
|
||||
if (
|
||||
record["event"] == "context_return"
|
||||
and source_reason in fault_reasons
|
||||
and source_reason not in emitted_context
|
||||
):
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
emitted_context.add(source_reason)
|
||||
mapping_reason = record.get("mapping", {}).get("sourceReason")
|
||||
if (
|
||||
record["event"] == "tool_call"
|
||||
and not record.get("allowed")
|
||||
and mapping_reason in fault_reasons
|
||||
and mapping_reason not in emitted_tool
|
||||
):
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
emitted_tool.add(mapping_reason)
|
||||
emitted_broker: set[str] = set()
|
||||
for record in generations:
|
||||
reason = record.get("source_reason")
|
||||
if record["event"] == "source_invalidation_revoke" and reason not in emitted_broker:
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
emitted_broker.add(str(reason))
|
||||
|
||||
print("\nP6_PI_CONTEXT_ATOMIC_OBSERVATION:")
|
||||
for record in records:
|
||||
include = (
|
||||
(record["event"] == "context_return" and record.get("injectionDecision") == "ONE_ATOMIC_AGENT_MESSAGE")
|
||||
or (record["event"] == "before_provider_request" and record.get("finalPayloadValid"))
|
||||
or (record["event"] == "message_end" and record.get("exactContextBlockCopied"))
|
||||
)
|
||||
if include and record["seq"] <= 12:
|
||||
print(json.dumps(record, sort_keys=True))
|
||||
|
||||
print("\nRPC_EVENT_COUNTS:")
|
||||
counts: dict[str, int] = {}
|
||||
for line in rpc_lines:
|
||||
try:
|
||||
event = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
key = str(event.get("type"))
|
||||
counts[key] = counts.get(key, 0) + 1
|
||||
print(json.dumps(counts, sort_keys=True))
|
||||
print("stderr_nonempty=" + str(bool(stderr_lines)))
|
||||
for line in stderr_lines[:10]:
|
||||
print("stderr: " + line[:500])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
8
docs/compaction-refresh/probes/pi_later_extension.ts
Normal file
8
docs/compaction-refresh/probes/pi_later_extension.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
|
||||
|
||||
// Deliberately loaded after pi_gate0_extension.ts. The Gate0 extension must
|
||||
// observe its argv position and remain CLOSED rather than claiming finality.
|
||||
export default function register(pi: ExtensionAPI) {
|
||||
pi.on('context', async (event) => ({ messages: [...event.messages] }));
|
||||
pi.on('before_provider_request', async () => undefined);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
# Gate0 Probe-3 (D4) Class-B — §3-Conformance Review v3
|
||||
|
||||
**Verdict: ✅ PASS**
|
||||
|
||||
## Pin (G1 — reviewed object, mandatory)
|
||||
|
||||
- **Reviewed object = `ace6066762c088f4b9729860da71b4c84451a7c3`** (harness commit, branch `feat/827-gate0-probe`, `mosaicstack/stack` @ git.mosaicstack.dev).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py`
|
||||
- **Harness sha256 (pushed provider bytes, fetched `-o FILE`, FULL-40 ref, verified before trust):**
|
||||
`2f11c9391c0eef203f26b1206bee8bc4cd106e8c1192399c5e7b71f41a3f6b75` (17162 bytes; no not-found sentinel).
|
||||
- **§3 amendment authority read at pin:** `GATE0-PROBE3-EXEC-AMENDMENT.md` @ ref `571f239154c6793fb1a5eac0d1cd4182f286a3ac`,
|
||||
sha256 `9ac9ff873fad41a6e15763cc89cb94d0bc4a6b0cf9b6770561d1781b03f63276` (7699 bytes). MUST-HAVE/MUST-BE-ABSENT
|
||||
confirmed against the actual fetched §3 text, not a paraphrase.
|
||||
|
||||
## Independence (G2)
|
||||
|
||||
Distinct Opus §3-conformance reviewer (Gate-16 author≠reviewer). I did **not** build this harness (author =
|
||||
ms-rev-826); I am not Mos. This verdict is my own; the author did not author or edit it. Byte review only — **ran
|
||||
nothing** (no harness, no broker, no sockets/state). Reviewed across v1 (FAIL, live-broker launch path) → v2 (PASS,
|
||||
later found runtime-dead producer) → this v3 (closes the live-path review-gap).
|
||||
|
||||
## Why v3 (the review-gap closed)
|
||||
|
||||
v2 PASS @`839d156f` credited the static presence of `lease_anchor_registered` as isolation proof. At FIRE the
|
||||
producing path was **dead**: the harness drove the *released* `mosaic` binary, which launched Pi **ungated**, so
|
||||
`register_anchor` never ran. Static presence of an assertion ≠ its producing path executing. v3 requires the
|
||||
producing path to be **live at runtime**.
|
||||
|
||||
## Surface-by-surface
|
||||
|
||||
| Surface | Result | Evidence (file:line) |
|
||||
|---|---|---|
|
||||
| Pushed bytes fetched + sha-verified | ✅ | sha256==`2f11c939…`, 17162B, no sentinel |
|
||||
| (a) LIVE-PATH — drives the **gated** launcher, producer in the exec chain, NOT released `mosaic`/plain `execRuntime` | ✅ | launch = `python3 <GATED_LAUNCHER> --runtime pi -- pi …` :357-364; `GATED_LAUNCHER=…/launch-runtime.py` :32, pinned `GATED_WI_HEAD=abd2791f…` :31; `mosaic yolo`/`execRuntime` = 0 hits. `launch-runtime.py` unconditionally `register_anchor`s before `execvpe`, so the producer is in the invoked chain |
|
||||
| (b) Fail-closed precondition present + correct (gated + fixture-socket, refuses otherwise), invoked before all launches | ✅ | `gated_launcher_precondition` :229-251, called :342 **before** broker Popen :343 and Pi launch :357. Verifies (ii) `MOSAIC_LEASE_BROKER_SOCKET==fixture` :232 + fixture in tempdir :234; (i) launcher HEAD==`abd2791f` :243 + source has `register_anchor` **before** `execute(command[0]…)` and reads `MOSAIC_LEASE_BROKER_SOCKET` :246-250. Raises `RuntimeError` (no run) on any miss :233/:235/:242/:244/:250 |
|
||||
| (c) Fixture-socket isolation (no live/default broker reachable, single p3 fixture, non-destructive) | ✅ | `pop("MOSAIC_LEASE_BROKER_SOCKET")` :330 + set to fixture socket :334; harness invokes `launch-runtime.py` directly so it reads `MOSAIC_LEASE_BROKER_SOCKET`=fixture with **no** `defaultLeaseBrokerSocket`/XDG/`/run/user` fallback in the path; `register_anchor` served by the single p3 fixture; `p3_generation_broker.py` **zero diff** vs `839d156f` (in-memory volatile hex-256 session `secrets.token_hex(32)`, nothing durable outside tempdir) |
|
||||
| (d) Assertion INTACT (`lease_anchor_registered` + `session_id_shape=="hex-256"`, not softened/optional/repointed) | ✅ | :256-258 (event), :298 (`hex-256`), folded into single-PID/starttime identity set :286. Not Case C |
|
||||
| spawns ONLY p3_generation_broker.py | ✅ | broker Popen = `HERE/p3_generation_broker.py` :343-346; the pinned launcher is a register-before-exec launch wrapper, not a 2nd broker |
|
||||
| promotion = fixture-only (not P2-banked) | ✅ | `d4_fixture_promote` "not a P2 evidence-gathering authorization"; `promote-probe` in-memory; precondition target only; no P2 import |
|
||||
| D4 assertions complete | ✅ | same-PID+starttime persist (incl. launcher registration) :284-288; gen strictly increases :290-293; reload revokes genuinely-VERIFIED prior :299-300; new→`MUTATOR_UNVERIFIED` :301; prior→`STALE_GENERATION` :302; lifecycle boundaries :294-297 |
|
||||
| P5 / P6 / P2-bank / retry-launder / live-effect / mechanism-change / scope-widen ABSENT | ✅ | 0 hits: `source-invalid`/`run_open`/`atomic`/`p2_provider`/`p5`/`p6`/`pi_gate0_run`/`retry`; no real-broker path; the only mechanism change is the required launch-routing fix (release→pinned gated launcher), which narrows scope, not widens |
|
||||
| non-destructive | ✅ | per-run `tempfile.mkdtemp` fixtures; p3 in-memory + tempdir socket/log only; reads `/proc/<pid>/stat` (read-only) |
|
||||
| deterministic | ✅ | isolated tempdir per run; deterministic assertions; session-id randomness is **shape**-asserted only (`hex-256`), `setdefault` idempotent |
|
||||
| hidden exec-at-import | ✅ absent | only `if __name__ == "__main__": main()`; docstring: not executed until FIRE |
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `ace60667`** — (a) LIVE-PATH, (b) fail-closed precondition, (c) fixture-socket isolation, and
|
||||
(d) intact assertion all hold, with zero out-of-scope surface. The v2 review-gap (runtime-dead producer via the
|
||||
released ungated `mosaic`) is structurally closed: the harness no longer invokes `mosaic` at all — it invokes the
|
||||
pinned `abd2791f` `launch-runtime.py` directly (register-before-exec), and refuses to launch unless the launcher is
|
||||
that pinned gated register-before-exec binary bound to this run's fixture socket.
|
||||
|
||||
**Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Per §3/§5 of the amendment, producing this evidence **executes** the Gate0 mechanism (launches processes, creates
|
||||
socket/state artifacts, exercises revocation). This review clears the **bytes**; **FIRE remains separately gated on
|
||||
Mos's explicit post-clear GO** and is not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** independent Opus §3-conformance reviewer (Gate-16 author≠reviewer). Byte review only; ran nothing.
|
||||
**Reviewed object (pin):** `ace6066762c088f4b9729860da71b4c84451a7c3` · harness sha256 `2f11c9391c0eef203f26b1206bee8bc4cd106e8c1192399c5e7b71f41a3f6b75`.
|
||||
@@ -0,0 +1,95 @@
|
||||
# Gate0 Probe-3 (D4) Observation-Fidelity — §3-Conformance Review v4
|
||||
|
||||
**Verdict: ✅ PASS**
|
||||
|
||||
## Pin (G1 — reviewed object, mandatory)
|
||||
|
||||
- **Reviewed object = `484849387006ab5561798506fd6042ddbd5617de`** (harness commit, branch `feat/827-gate0-probe`, `mosaicstack/stack` @ git.mosaicstack.dev).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `9095eab7a4ddf11bb92bb5971d49e1facad12f4692ce2081665b0af47cbe5098` (23698 bytes, no not-found sentinel). Worktree bytes at `48484938` re-hashed identical.
|
||||
- **Co-reviewed fixture broker:** `docs/compaction-refresh/probes/p3_generation_broker.py` (sha256 `fd5327d2e9a2808282cbc9c4a4ccef42d2a806482b27cc48035191b9b11607c8`).
|
||||
- **Traceability anchor (independently recomputed):** `GATED_LAUNCHER_SHA256 = e950e4224e280f16979d90cabb89aa1896c5ee28bed2df957e14d018d43cda82` equals the sha256 of `packages/mosaic/framework/tools/lease-broker/launch-runtime.py` at WI-3 #830 reviewed head `f400830738998db105107a2a4c69c7f2a2a6fd5d` (computed from two independent worktrees).
|
||||
|
||||
## Independence (G2)
|
||||
|
||||
Distinct Opus §3-conformance reviewer (Gate-16 author≠reviewer). I did not build this harness (author = ms-rev-826);
|
||||
I am not Mos. This verdict is my own; the author did not author or edit it. Byte review only — **ran nothing** (no
|
||||
harness, no broker, no sockets/state). Prior v3 PASS (`ace60667`, pinned `abd2791f`) is correctly **superseded**:
|
||||
pinning a pre-`66b1e0a0` ancestor made D4 an in-memory socket simulation (hollow gate). v4 requires
|
||||
mechanism-fidelity within isolation (Jason R1).
|
||||
|
||||
## BAR1 — Traceability (evidence attests the SHIPPED WI-3 D4 mechanism)
|
||||
|
||||
| Check | Result | Evidence |
|
||||
|---|---|---|
|
||||
| `GATED_WI_HEAD == f4008307` (not `abd2791f`) | ✅ | :33 |
|
||||
| Launcher pinned by git-HEAD **and** sha256 | ✅ | precondition :285-288 (`head != GATED_WI_HEAD` raise; `sha256(launcher) != GATED_LAUNCHER_SHA256` raise); sha256 independently == f4008307's `launch-runtime.py` |
|
||||
| Launcher bytes contain the file-backed mechanism | ✅ | precondition requires `register_anchor` :290, `initialize_runtime_generation(generation_file, generation)` :291, `generation-{session_id}.state` :294, `MOSAIC_LEASE_GENERATION_FILE` :295, `read_runtime_generation`+`bump_runtime_generation` in `lease_generation.py` :296-299; order `register < initialize < execute` :302-303 |
|
||||
|
||||
## BAR2 — Fidelity (file-backed generation, not in-memory simulation)
|
||||
|
||||
| # | Requirement | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| i | Extension bumps `generation-{sid}.state` via the real helper, not in-mem | ✅ | ext `lifecycle()` calls `broker({action:'bump-generation'})` at every post-start boundary (harness :200-205); broker `bump-generation` → `bump_runtime_generation(generation_environment(identity))` (broker :119-120) |
|
||||
| ii | Broker reads generation via `read_runtime_generation`, not an in-mem counter | ✅ | broker loads the pinned module (`--generation-module`, :52-54); `file_generation()` = `read_runtime_generation(...)` (:77-78); authorize-probe reads `current_generation = file_generation(identity)` (:154); lifecycle `new_generation = file_generation(identity)` (:201). The in-mem `generations` dict is only an old-value cache for the monotonic guard (:199-204), never the authorization authority |
|
||||
| iii | `assert_d4` observes the FILE-BACKED transition | ✅ | `state_file_source` = all records `generation_source=="state-file"` (harness :370-372); `state_file_drives_lifecycle` = state-bumps == lifecycle generations[1:] (:368-369); `new_generation_unverified→MUTATOR_UNVERIFIED` :382; `prior_generation_stale→STALE_GENERATION` :383; `verified_revoked_on_reload` :380-381 — not a socket-only bump |
|
||||
| iv | `.state` bound to per-run fixture temp root | ✅ | broker `generation_environment` **raises if `state_path.parent != socket_path.parent`** (:73-74); `state_file_in_fixture_root` (:373-376); precondition forbids inherited `MOSAIC_LEASE_GENERATION_FILE` (:255-256) and requires socket in `gettempdir()` (:274-275); launcher/broker agree on `socket_path.parent / generation-{sid}.state` |
|
||||
|
||||
## BAR3 — Carry-over
|
||||
|
||||
| # | Result | Evidence |
|
||||
|---|---|---|
|
||||
| a LIVE-PATH (gated launcher @f4008307 at runtime, not released/plain execRuntime) | ✅ | Pi launched via `python3 <GATED_LAUNCHER> --runtime pi -- pi …` :502-505; `mosaic yolo`/`execRuntime` = 0 hits; register-before-exec producer in the invoked chain (precondition order gate) |
|
||||
| b Fail-closed precondition present+correct | ✅ | `gated_launcher_precondition` :250-306, invoked :485 **before** broker/Pi; raises on socket≠fixture / gen-file-inherited / write-path-escape / head-mismatch / hash-mismatch / not-register-before-exec-file-bound |
|
||||
| c Fixture-socket isolation, single p3 broker | ✅ | one broker Popen :486-500; `MOSAIC_LEASE_BROKER_SOCKET=socket_path` (fixture); direct launcher invocation ⇒ no default/XDG/`/run/user` fallback in path |
|
||||
| d `lease_anchor_registered` INTACT | ✅ | broker emits event + `session_id_shape=="hex-256"` (:102-107); `record_where` requires it (:324-326); `lease_anchor_fixture` check (:379) — not deleted/softened/optional/repointed (not Case-C) |
|
||||
|
||||
## BAR4 — Homelab Gate-B carry-forward findings
|
||||
|
||||
| # | Result | Evidence |
|
||||
|---|---|---|
|
||||
| b4-1 gated launcher @f4008307, not released/plain execRuntime | ✅ | :502-505; 0 `mosaic yolo`/`execRuntime` |
|
||||
| b4-2 **affirmative no-escape** (allow-list base, not deny-list) | ✅ | `isolated_environment` builds the child env from a **literal allow-list dict** (:442-461), NOT `os.environ.copy()`; only PATH/LANG/TERM/PI_CODING_AGENT (non-write-bearing) pass through; every write-bearing var (HOME/XDG*/TMPDIR/MOSAIC_AGENT_WORKDIR/HEARTBEAT_RUN_DIR/MOSAIC_HOME/D4_PI_LOG/socket) redirected under `root`; precondition double-checks each is `is_relative_to(root)` (:257-273). No unnamed/future inherited var survives |
|
||||
| b4-3 `--runs` exactly 3, fail-closed otherwise | ✅ | `add_argument("--runs", type=int, default=3, choices=(3,))` :591 (argparse rejects any other value) |
|
||||
| b4-4 cleanup try/finally spans the whole launch | ✅ | `broker=pi=None` :479-480; `try` opens **before** precondition/broker/PiRpc :482; nested `finally` always closes pi then broker+socket even on early `wait_path`/`PiRpc` failure :571-585 |
|
||||
| b4-5 `-O`-safe integrity + derived PASS | ✅ | load-bearing checks in a `checks` dict; `if failed: raise AssertionError` :385-387 and `if not passed: raise` :388-390 (NO bare `assert` anywhere — grep-confirmed); PASS = `"PASS" if passed else "FAIL"` derived from `all(checks.values())` :393, re-derived+checked in `run_once` :550-553 |
|
||||
|
||||
## MUST-BE-ABSENT sweep
|
||||
|
||||
`P5` / `P6` / `P2-bank` / `retry-launder` / `mosaic yolo` / `execRuntime` / `run_open` / `atomic-observation` /
|
||||
`pi_gate0_run` = **0 hits** (both files). Extension invokes only `bump-generation` / `lifecycle` /
|
||||
`authorize-probe` / `promote-probe`. No live/prod/real-broker path (single fixture broker; allow-list env; launcher
|
||||
pinned to fixture socket). No `.state`/gen-file path outside the fixture temp root (broker `generation_environment`
|
||||
raises otherwise). §4 live effect: none. No extra broker/socket beyond the single p3. No exec-at-import (both files
|
||||
`__main__`-guarded). Mechanism change is confined to the mandated R1 observation-fidelity deepening + BAR4 hardening;
|
||||
no scope-widen of what the probe touches.
|
||||
|
||||
## Observations (transparency — not findings)
|
||||
|
||||
1. The fixture broker retains a **dormant `source-invalid` action** (:179-194, P5-adjacent, in-mem). It is
|
||||
**never invoked** by the harness or its embedded extension (verified: extension actions are only
|
||||
bump/lifecycle/authorize/promote), and `assert_d4` never observes it — so the probe does **not** exercise or bank
|
||||
P5. Pre-existing shared-fixture code, unchanged. Surfaced so Mos may, if desired, apply a stricter
|
||||
purge-dormant-P5-from-the-fixture standard; under the "what the probe TOUCHES/does" framing it is not a violation.
|
||||
2. Fixture `HOME` receives a **read-only copy** of the operator's `~/.pi/agent` `settings.json`/`auth.json`/`bin/fd`
|
||||
(:430-440, `shutil.copy2` into the fixture) so real Pi can authenticate to the model provider. It reads operator
|
||||
state; it does not write/mutate operator HOME and does not emit/log credential material. Confined to the fixture.
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `48484938`** — BAR1 (traceability to shipped f4008307 mechanism) + BAR2 (genuine file-backed generation,
|
||||
i–iv) + BAR3 (live-path / fail-closed precondition / isolation / intact assertion) + BAR4 (b4-1..b4-5) all hold,
|
||||
with zero out-of-scope surface exercised. The v3 hollow-gate (ancestor pin, in-mem simulation) is structurally
|
||||
closed: evidence now attests the shipped WI-3 D4 file-backed generation mechanism, launcher pinned by head+sha256,
|
||||
child env write-confined by allow-list, integrity `-O`-safe with a derived PASS.
|
||||
|
||||
**Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Per §3/§5 of the amendment, producing this evidence **executes** the Gate0 mechanism. This review clears the
|
||||
**bytes**; **FIRE remains separately gated on Mos's explicit post-clear GO** and is not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** independent Opus §3-conformance reviewer (Gate-16 author≠reviewer). Byte review only; ran nothing.
|
||||
**Reviewed object (pin):** `484849387006ab5561798506fd6042ddbd5617de` · harness sha256 `9095eab7a4ddf11bb92bb5971d49e1facad12f4692ce2081665b0af47cbe5098`.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Gate0 Probe-3 (D4) Hygiene-Delta — §3-Conformance Review v5
|
||||
|
||||
**Verdict: ❌ FAIL** (hygiene delta (a)+(b) landed correctly and (c)+(d) hold, but homelab findings **NEW-5** and **NEW-6** are present in these bytes; both must close for PASS).
|
||||
|
||||
## Pin (G1 — reviewed object, mandatory)
|
||||
|
||||
- **Reviewed object = `7f975b95ad39096463a7548bd6be0dbb387cb61b`** (harness commit, branch `feat/827-gate0-probe`).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `c3a09a342a4b367184d44472ec6fc11f8a3aabb7e90d5a72aa6b7044b1d9b91e` (24174 bytes, no not-found sentinel).
|
||||
- **Co-reviewed fixture broker:** `p3_generation_broker.py` sha256 `4db4fef1ac6658a8ca79ad5091cefc901d2aa26003265c3d6726c294cf895cad`.
|
||||
|
||||
## Reviewer identity / lane (independence — on the record)
|
||||
|
||||
This review is produced by a **distinct independent Opus §3-conformance / SECREV session** (Gate-16 author≠reviewer),
|
||||
**byte review only, ran nothing**, that **did not build** this harness (author = ms-rev-826) and **is not Mos**. The
|
||||
PROCESS/LANE separation (build lane ≠ review lane) holds and is attested here. Homelab's separate observation — that
|
||||
the published PASS commits and the repair commits share the `ms-lead-reviewer` **Git signer identity** — is a
|
||||
git-identity-signer question I do **not** self-resolve; per instruction it is routed to Mos. My lane attestation is
|
||||
independent of the git signer.
|
||||
|
||||
## Hygiene delta (v4 `48484938` → v5 `7f975b95`) — items (a)+(b): CLOSED
|
||||
|
||||
Full `diff` of both files shows the delta touches **only** these:
|
||||
|
||||
- **(a) Creds scrubbed — CLOSED.** New `scrub_fixture_credentials(root)` (:467-475): `shutil.rmtree(root/"home"/".pi"/"agent")` then a fail-closed re-check `if copied_agent.exists(): raise` (:473-474). Invoked in the **outermost `finally`** (:598), **after** pi/broker cleanup and **before** `return root` (:600) — so it runs on the **success path too**. The scrub target exactly matches the only copy site (`target_agent = fixture_home/".pi"/"agent"`, :433). `isolated_environment` was moved inside the `try` (:485) so the scrub finally also covers a partial env-setup failure. No copied agent credential/config subtree (incl. `auth.json`) survives in any retained root. ✅
|
||||
- **(b) `source-invalid` handler REMOVED — CLOSED.** `grep source-invalid / source_invalidation_revoke == 0` across **both** files. Broker delta is solely the deletion of the old `:179-194` block. ✅
|
||||
|
||||
## (c) Everything-else byte-stable vs `48484938` — HOLDS
|
||||
|
||||
Diff confirms **no other hunks**. Re-swept intact: traceability `GATED_WI_HEAD=f4008307` (:33) + `GATED_LAUNCHER_SHA256=e950e4224e…` (:35); fidelity `read_runtime_generation`/`bump_runtime_generation` + `generation_source=="state-file"` + `new→MUTATOR_UNVERIFIED`/`prior→STALE_GENERATION`; `lease_anchor_registered` + `hex-256`; `-O`-safe (0 bare `assert`, PASS derived `all(checks.values())`); allow-list literal env (0 `os.environ.copy`); `--runs choices=(3,)`; fail-closed precondition; single p3 broker. Blockers 1-4 (inherited env / `--runs`≠3 / cleanup-misses-setup-failure / `-O`-strips-checks) remain closed.
|
||||
|
||||
## (d) ABSENT sweep — 0
|
||||
|
||||
`P5`/`P6`/`P2-bank`/`retry-launder`/`mosaic yolo`/`execRuntime`/`run_open`/`atomic`/`pi_gate0` = 0 (both files);
|
||||
extension invokes only `bump-generation`/`lifecycle`/`authorize-probe`/`promote-probe`; no exec-at-import.
|
||||
|
||||
## Homelab carry-forward — NEW-5 / NEW-6: **PRESENT → FAIL**
|
||||
|
||||
### 🔴 NEW-6 — GATED_WI_ROOT resolves wrong / non-portably (CONFIRMED)
|
||||
`GATED_WI_ROOT = HERE.parents[3].parent / "stack-cr-wi3-revoke"` (:32). For the harness at
|
||||
`…/agent-work/stack-cr-wi0-gate0/docs/compaction-refresh/probes/`, this computes
|
||||
**`/home/hermes/stack-cr-wi3-revoke` — which does not exist**. The actual f4008307 worktree is
|
||||
`/home/hermes/agent-work/stack-cr-wi3-revoke` = `HERE.parents[3] / "stack-cr-wi3-revoke"`. The traversal is
|
||||
**off-by-one** (`.parents[3].parent` should be `.parents[3]`), and it additionally hardcodes the worktree name — a
|
||||
non-portable relative assumption. On this host the precondition therefore fail-closes ("gated WI launcher is
|
||||
unavailable") and the probe cannot locate/execute the pinned launcher at all. **Real resolution defect** (independently
|
||||
computed by path arithmetic; harness not run). **file:line — :32.**
|
||||
*Fix:* derive `GATED_WI_ROOT` from a portable, explicit anchor (e.g. an env-provided path validated to be the
|
||||
f4008307 worktree, or `HERE.parents[3] / "stack-cr-wi3-revoke"` with existence+HEAD assertion), not `.parents[3].parent`.
|
||||
|
||||
### 🔴 NEW-5 — launcher precondition is check-then-exec, not pinned-executed-bytes (CONFIRMED)
|
||||
The precondition hashes `launcher_bytes = GATED_LAUNCHER.read_bytes()` (:280) against `GATED_LAUNCHER_SHA256` (:287),
|
||||
but the launcher is **executed separately** via `PiRpc([sys.executable, str(GATED_LAUNCHER), …])` (:514-515), which
|
||||
opens and **re-reads the file at exec time**. There is **no fd-handoff and no exec-from-verified-copy**, so the
|
||||
verified snapshot does **not** bind the executed bytes. The window between check (:287) and exec (:514-515) spans the
|
||||
broker `Popen` + `wait_path` (≤20 s) — a genuine **check-then-exec TOCTOU / mutable-path trust**; the `git rev-parse
|
||||
HEAD` check (:285-286) is likewise on a mutable HEAD, not the executed bytes. Per the bar this is a real gap.
|
||||
**file:line — hash :280/:287 vs exec :514-515.**
|
||||
*Fix:* execute the exact verified bytes with no window — e.g. read once, verify, and exec from a fixture-private
|
||||
copy of the verified bytes (or `python3 /proc/self/fd/<verified-fd>`), so the hashed bytes == executed bytes.
|
||||
|
||||
## Verdict
|
||||
|
||||
**FAIL @ `7f975b95`.** The hygiene delta itself is correct — (a) creds scrub (fail-closed finally, success path,
|
||||
every retained root) and (b) `source-invalid` removal both landed cleanly, (c) everything else is byte-stable vs
|
||||
`48484938`, and (d) the absent sweep is 0. **However**, homelab's NEW-5 (check-then-exec launcher TOCTOU / not
|
||||
pinned-executed-bytes) and NEW-6 (GATED_WI_ROOT off-by-one/non-portable resolution) are **present in these bytes**;
|
||||
the addendum requires both **closed** for PASS. Not softened. Returns to author (ms-rev-826) — not to a builder
|
||||
re-review, no PASS-launder.
|
||||
|
||||
**Findings:** NEW-6 (`p3_d4_focused_run.py:32`); NEW-5 (`p3_d4_focused_run.py:280/:287` vs `:514-515`).
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; FIRE remains
|
||||
separately gated on Mos's explicit post-clear GO — and is moot until this FAIL is remediated.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** independent Opus §3-conformance/SECREV reviewer (Gate-16 author≠reviewer). Byte review only; ran nothing.
|
||||
**Reviewed object (pin):** `7f975b95ad39096463a7548bd6be0dbb387cb61b` · harness sha256 `c3a09a342a4b367184d44472ec6fc11f8a3aabb7e90d5a72aa6b7044b1d9b91e`.
|
||||
@@ -0,0 +1,116 @@
|
||||
# GATE0 Probe-3 (#827) — Mos byte-scope-verify CO-ATTESTATION (v-final)
|
||||
|
||||
**Principal:** Mos (orchestrator, merge authority for the mosaic-stack governance lane).
|
||||
**Committed under a DISTINCT git identity** (`mos-orchestrator@mosaic.local`) — deliberately NOT the
|
||||
`ms-lead-reviewer@mosaic.local` lane signer — so this record stands as a *distinct-identity*
|
||||
co-attestation, not a same-signer duplicate. See "Independence" below.
|
||||
|
||||
**Verify class:** independent provider-byte read (guarded `git show <full-40>:path | sha256sum` from a
|
||||
read-only clone of `mosaicstack/stack`). Not a re-build, not a re-run — a byte/scope/hygiene audit of
|
||||
the exact committed objects on the provider branch.
|
||||
|
||||
## Package under attestation
|
||||
|
||||
| Artifact | Ref |
|
||||
|---|---|
|
||||
| Branch | `feat/827-gate0-probe` |
|
||||
| Harness commit-40 | `2d54a9dd14cb924701b2ae4ed72dae4df760c4e3` |
|
||||
| Harness `p3_d4_focused_run.py` sha256 | `15a154df55273f51301763a984485fd63813f6d1f05d2728abb9fb8b9c040b1a` (27366 B) |
|
||||
| §3-review-v6 commit-40 | `23c0caca9b5d44002e6184cd7f2b6c837e8795b2` |
|
||||
| Review path | `docs/compaction-refresh/reviews/GATE0-PROBE3-NEW56-S3-REVIEW-v6.md` |
|
||||
|
||||
sha256 re-confirmed against the checked-out object at `HEAD:docs/compaction-refresh/probes/p3_d4_focused_run.py`
|
||||
(git object id `8c68cd07…`) — matches the relayed value byte-for-byte.
|
||||
|
||||
## Findings — VERDICT: byte-scope + mechanism + hygiene **PASS**
|
||||
|
||||
**Anchors.** Harness sha256 matches (27366 B). review-v6 (`23c0caca`) parent == harness commit
|
||||
`2d54a9dd`; review touches only the review `.md` (+82 lines, 1 file). Broker
|
||||
(`p3_generation_broker.py`) delta vs `48484938…` = **exactly** the 16-line `action=="source-invalid"`
|
||||
handler purge, byte-stable otherwise.
|
||||
|
||||
**NEW-6 (GATED_WI_ROOT off-by-one) — CLOSED.** `resolve_gated_wi_root()` selects the worktree by
|
||||
`git worktree list --porcelain` enumeration, requires a UNIQUE match on `HEAD==GATED_WI_HEAD`
|
||||
(`f400830738998db105107a2a4c69c7f2a2a6fd5d`) AND `branch==refs/heads/feat/830-compaction-revoke`,
|
||||
then fail-closed re-validates (`is-inside-work-tree==true`, `rev-parse HEAD==GATED_WI_HEAD`);
|
||||
`RuntimeError` on ambiguity/mismatch. The `HERE.parents[3].parent / "stack-cr-wi3-revoke"` off-by-one
|
||||
and the hardcoded `/home/hermes/...` literal are **gone** — portable, zero hardcoded path.
|
||||
|
||||
**NEW-5 (TOCTOU / pinned-executed-bytes) — CLOSED via approach (i), as mandated.** The `git`-object
|
||||
sha256 pin (`GATED_LAUNCHER_SHA256 = e950e422…`) is the trust anchor. Ordering/marker `.find()`
|
||||
heuristics are downgraded to explicitly diagnostic-only ("never a substitute for the pin"). In
|
||||
`launch_verified_pi()` the executed working-tree file is re-hashed against the pin **in the statement
|
||||
immediately before `Popen`** (no interleaved yield/IO), and the launcher is executed **in place at the
|
||||
pinned worktree path** — the higher-risk approach (ii) copy-to-fixture (previously at `7ff63cd5` /
|
||||
`6164dc07`) is **reverted** (the only remaining `shutil.copy2` is the legitimate credential copy, not a
|
||||
launcher copy). Residual sub-statement TOCTOU window on a local file inside a non-adversarial operator
|
||||
fixture is within this probe's threat model; the gross precondition→much-later-exec gap homelab flagged
|
||||
is closed.
|
||||
|
||||
**Hygiene — CLOSED.** `scrub_fixture_credentials(root)` removes the entire `.pi/agent` subtree in a
|
||||
`finally` block (nested try/finally, after `pi.close()` + broker shutdown, before `return root`) and
|
||||
`RuntimeError`s if the scrub fails — credentials are removed from retained evidence; logs retained.
|
||||
|
||||
**Invariants byte-stable (all INTACT):** assertion `lease_anchor_registered`; file-backed fidelity
|
||||
checks (`generation_source=="state-file"`, `state_file_in_fixture_root`, `MUTATOR_UNVERIFIED`,
|
||||
`STALE_GENERATION`); `-O`-safe (0 bare `assert`); allow-list env (0 `os.environ.copy`); single broker;
|
||||
`--runs choices=(3,)`; ABSENT-sweep (`yolo`/`execRuntime`/`p3_bank`/`promote_p2`/`retry`) = 0.
|
||||
|
||||
**R1 mechanism-fidelity boundary — HELD.** D4 exercises #830's real file-backed revocation
|
||||
(`generation-{sid}.state` / `MOSAIC_LEASE_GENERATION_FILE`, from `66b1e0a0`); `.state` stays inside the
|
||||
fixture temp root; fixture-socket / child-write-escape / gettempdir isolation preconditions preserved;
|
||||
launcher exec is in place at the pinned worktree (same surface as R1, not a new production/live
|
||||
surface). **No path escapes the fixture root.** BEYOND-R1 tripwire NOT tripped.
|
||||
|
||||
## Independence
|
||||
|
||||
- **Substantive principal-independence** of review-v6 is satisfied by an orchestrator-dispatched,
|
||||
builder-distinct Opus SECREV (`ms-secrev-828`, byte-only, non-builder) — that IS the substance of
|
||||
Gate-16.
|
||||
- The shared `ms-lead-reviewer` git signer on harness+review commits is **evidentiary, not
|
||||
substantive**. It is resolved by (1) this Mos co-attestation committed under a **distinct** identity
|
||||
(`mos-orchestrator`), and (2) a homelab third-principal verify under its own distinct identity —
|
||||
i.e. three distinct-identity principals of record.
|
||||
- The shared signer is a tracked **fleet-infra tooling-gap** (durable fix = per-lane distinct signers),
|
||||
**not a blocker**.
|
||||
|
||||
## Scope of this record — byte-clear, NOT fire-authorization
|
||||
|
||||
Producing probe evidence **executes** the Gate0 mechanism; a byte-clear is not a fire-authorization.
|
||||
This co-attestation clears the **bytes/scope/hygiene**. FIRE remains gated on: **homelab
|
||||
third-principal verify** + **Mos transparency-to-Jason** (real-Pi consumes operator model creds inside
|
||||
the isolated fixture, scrubbed post-run, never emitted) + **Mos explicit FIRE GO**. Until then: nothing
|
||||
banked, WI-3 #830 held at `f4008307` (unmoved), C-hatch armed (if 3× isolation still no-fire /
|
||||
wrong-value / isolation-FAIL → possible Case-C → STOP + escalate to Jason).
|
||||
|
||||
**Mos verdict: byte-scope + mechanism + hygiene PASS. Co-attestation of record — committed.**
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ SUPERSEDED — homelab third-principal FAIL raised a stricter bar (evidence-integrity note)
|
||||
|
||||
This co-attestation was **byte-clear on the v6 bar ONLY** and self-limited above to *"byte-clear
|
||||
ONLY, NOT fire-authorization; FIRE remains gated on homelab third-principal verify."* Homelab (the
|
||||
required third principal) subsequently returned **FAIL @2d54a9dd**, and Mos **UPHELD** it — so the v6
|
||||
byte-clear this document records is **SUPERSEDED** and does **NOT** authorize FIRE.
|
||||
|
||||
Homelab's substantively-correct deepening (accepted as gate-**strengthening**, not softening):
|
||||
1. `launch_verified_pi` hashes the launcher then `Popen`/execve **reopens the path** → statement
|
||||
adjacency shrinks but does not eliminate TOCTOU; hashed-snapshot ≠ executed-bytes.
|
||||
2. `lease_generation.py` helper is unpinned, loaded from the mutable worktree → HEAD + launcher-pin
|
||||
do not bind the helper bytes.
|
||||
3. `p3_generation_broker.py` executes from the mutable worktree unhashed → reviewed broker bytes need
|
||||
not be the evidence-producing bytes.
|
||||
|
||||
For a fail-closed DO-178C evidence gate, **hashed==executed must hold on the FULL executed closure**
|
||||
(launcher + helper + broker), which v6 (approach (i) adjacency) does not meet. Mos therefore
|
||||
**authorized approach (ii) full-closure materialization** (SHA-pin + materialize the full closure into
|
||||
a fixture-private 0700/0600 dir or held verified fds, exec from there, launcher+broker consume the same
|
||||
pinned helper; re-hash==f4008307 pins immediately before exec, fail-closed). This rides the existing R1
|
||||
authorization + Mos adjudication authority (it deepens isolation of already-authorized touch and stays
|
||||
inside the fixture temp root → R1 owner tripwire not tripped; no fresh owner window).
|
||||
|
||||
**Live target = v7** (materialized-closure harness, forthcoming). `2d54a9dd` / `23c0caca` / this
|
||||
co-attestation (`12914d8`) are **SUPERSEDED**. A fresh Mos co-attestation will be committed on v7
|
||||
byte-verify PASS. WI-3 #830 remains HELD at `f4008307`; nothing banked; C-hatch armed.
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
# GATE0 Probe-3 (#827) — Mos byte-scope-verify CO-ATTESTATION (v10 no-site startup closure)
|
||||
|
||||
**Principal:** Mos (orchestrator, merge authority for the mosaic-stack governance lane).
|
||||
**Committed under a DISTINCT git identity** (`mos-orchestrator@mosaic.local`) — deliberately NOT the
|
||||
`ms-lead-reviewer@mosaic.local` lane signer that authored the harness and the §3 review — so this
|
||||
record stands as a *distinct-identity* co-attestation. See "Independence".
|
||||
|
||||
**Verify class:** independent provider-byte read (`git show <full-40>:path | sha256sum`) plus a
|
||||
git-diff byte-comparison of the v9→v10 delta and source inspection of the executed command lines. Not
|
||||
a re-build, not a re-run. This SUPERSEDES the v9 co-attestation `f320d075` (byte-clear on the v9
|
||||
env-iso + bytecode-pin bar), which was overtaken by homelab's 4th-round FAIL @1c34e3cb
|
||||
(site-startup-closure hole) and the resulting B9 bar.
|
||||
|
||||
## Package under attestation
|
||||
|
||||
| Artifact | Ref |
|
||||
|---|---|
|
||||
| Branch | `feat/827-gate0-probe` (mosaicstack/stack) |
|
||||
| Harness commit-40 | `ce5ba762051354338889959bfce2b0381f4a4e2a` |
|
||||
| Harness `p3_d4_focused_run.py` sha256 | `7e14ead89a7b2a297fcc17e7653291b3bcace1d2002a8f90a989db74f6985b6f` (32753 B) |
|
||||
| §3-review-v10 commit-40 | `ffc3b573a8258a86e3ece9ef92bac925fb5a7a5b` (parent == harness `ce5ba762`, byte-stable; adds only the review `.md`, +86; ms-secrev-828 authored) |
|
||||
| Review path | `docs/compaction-refresh/reviews/GATE0-PROBE3-V10-NOSITE-S3-REVIEW-v10.md` |
|
||||
|
||||
**Pinned closure (all 3 sha256 INDEPENDENTLY recomputed by Mos from git objects == pins == embedded harness constants; UNCHANGED from v9):**
|
||||
|
||||
| Member | Source (commit:path) | Pin sha256 | Size |
|
||||
|---|---|---|---|
|
||||
| launcher `launch-runtime.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/launch-runtime.py` | `e950e422…` | 4237 B |
|
||||
| helper `lease_generation.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/lease_generation.py` | `061625402f…` | 3769 B |
|
||||
| broker `p3_generation_broker.py` | `23c0caca:docs/compaction-refresh/probes/p3_generation_broker.py` | `4db4fef1…` | — |
|
||||
|
||||
> `f4008307` (held WI-3 #830 gated head) is **not on origin** — recomputed launcher+helper from the
|
||||
> local `stack-cr-wi3-revoke` worktree (HEAD == `f4008307`); broker from origin `23c0caca`.
|
||||
|
||||
## Findings — VERDICT: byte-scope + mechanism + hygiene **PASS** (v10 = v9 + B9 no-site)
|
||||
|
||||
The v9→v10 delta is **exactly 2 insertions / 0 deletions** (git-diff byte-compared), a `+28 B`
|
||||
harness growth **fully accounted** (2 × ` "-S",\n` = 14 B each). Every prior invariant is
|
||||
byte-stable — the diff touches nothing outside the two command lists.
|
||||
|
||||
**B9 — no-site startup closure (homelab 4th-round finding CLOSED).** Both child command lists now
|
||||
carry `-S`, which disables the `site` module entirely (no `.pth` executable lines, no
|
||||
`sitecustomize`/`usercustomize`) so no unpinned startup code runs before the exact launcher/broker or
|
||||
outside `closure_import_guard`:
|
||||
- launcher `launch_verified_pi` (`:515`): `-s` + **`-S`** + `-B`, **no `-I`** (`-I` appears exactly
|
||||
once in the harness, `:553`, the broker only).
|
||||
- broker `launch_verified_broker` (`:554`): `-I` + **`-S`** + `-B`.
|
||||
|
||||
Homelab's finding was **independently reproduced by Mos** and the fix **empirically verified** on host
|
||||
Py3.11.2 (throwaway, not the harness): `[-s -B]`→`no_site=0, site imported` (the v9 hole); `[-I -B]`→
|
||||
`no_site=0, site imported` (proves `-I` does **not** imply `-S`); `[-s -S -B]`→`no_site=1`, `site` not
|
||||
in `sys.modules`, **and the launcher's bare `from lease_generation import` still resolves** (`sys.path[0]`
|
||||
untouched by `-S` → no B6c regression); `[-I -S -B]`→`no_site=1`. The launcher deliberately omits `-I`
|
||||
(B6c: on 3.11+ `-I` implies `-P`, dropping the script dir from `sys.path[0]` → sibling import breaks);
|
||||
its env isolation comes from the `PiRpc` `env=` allow-list, not `-I`.
|
||||
|
||||
**All priors — byte-stable (outside the 2-line delta, re-confirmed from the v9 verify):**
|
||||
B5 conjunction (materialize-from-pin / `mkdir(0o700)`+`O_EXCL` no-writer-window / re-hash==pin
|
||||
immediately-before-exec); B6 (single pinned helper bound; `closure_import_guard` AST-reject); B6c
|
||||
(launcher no `-I`); B7 (broker `env=environment` strict allow-list `:570`); B8 (`reject_pinned_bytecode`
|
||||
fail-closed `:498-501` before each consumer `:535/:562` + `PYTHONDONTWRITEBYTECODE=1` `:729` + `-B` on
|
||||
both); fidelity asserts (`generation_source=='state-file'`, `state_file_in_fixture_root`,
|
||||
`MUTATOR_UNVERIFIED`, `STALE_GENERATION`); `lease_anchor_registered`; BAR1 `GATED_WI_HEAD==f4008307`
|
||||
(`:36`) + `merge-base --is-ancestor 66b1e0a0 f4008307` = **YES**; `--runs choices=(3,)`; `-O`-safe
|
||||
(0 bare `assert`); allow-list env (0 `os.environ.copy`); single broker (1 def + 1 call); the only
|
||||
`shutil.copy2` is the `.pi/agent` credential copy.
|
||||
|
||||
**Closure = exactly 3 files, materialized inside the fixture root.** No path escapes the fixture temp
|
||||
root; no live/default broker; `.state` fixture-bound. **R1 owner tripwire NOT tripped** — `-S`
|
||||
deepens startup-closure isolation of an already-authorized touch; it does not widen the touched surface.
|
||||
|
||||
## Independence
|
||||
|
||||
Substantive principal-independence of review-v10 is satisfied by an orchestrator-dispatched,
|
||||
builder-distinct Opus SECREV (`ms-secrev-828`, byte-only, non-builder, non-Mos). The shared
|
||||
`ms-lead-reviewer` git signer on harness+review commits is evidentiary, not substantive — resolved by
|
||||
(1) this Mos co-attestation under a **distinct** identity (`mos-orchestrator`) and (2) a homelab
|
||||
third-principal verify under its own distinct identity = three distinct-identity principals of record.
|
||||
The prior 2-of-3 (`ms-secrev-828` v9 §3 PASS + `f320d075`) does **not** carry — all three re-verify
|
||||
this v10 SHA. Shared signer = tracked fleet-infra tooling-gap, not a blocker.
|
||||
|
||||
## Scope of this record — byte-clear, NOT fire-authorization
|
||||
|
||||
Producing probe evidence **executes** the Gate0 mechanism; a byte-clear is not a fire-authorization.
|
||||
This clears **bytes / scope / mechanism / hygiene on the v10 (v9 + B9 no-site) bar**. FIRE remains
|
||||
gated on: **homelab third-principal re-verify** (5th round, own distinct identity) + **Mos
|
||||
transparency-to-Jason** + **Mos explicit FIRE GO**. The FIRE GO additionally carries an
|
||||
**execution-procedure constraint**: the 3× isolation dispatch must launch the runner under
|
||||
externally-enforced **`python -I -S -B p3_d4_focused_run.py`** — a self-reexec is too late, the
|
||||
harness's own `site` runs before it could re-add `-S` to itself. Until FIRE GO: nothing banked, WI-3
|
||||
#830 held at `f4008307` (unmoved), C-hatch armed (fired-rig only: no-fire / wrong-value /
|
||||
assertion-FAIL / isolation-FAIL → possible Case-C → STOP + escalate to Jason).
|
||||
|
||||
Prior v10-superseded set: `1c34e3cb` / `e1c9a468` / `f320d075` (and transitively the v7 chain).
|
||||
|
||||
**Mos verdict: v10 (v9 + B9 no-site) byte-scope + mechanism + hygiene PASS. Co-attestation of
|
||||
record — committed.**
|
||||
131
docs/compaction-refresh/reviews/GATE0-PROBE3-MOS-COATTEST-v7.md
Normal file
131
docs/compaction-refresh/reviews/GATE0-PROBE3-MOS-COATTEST-v7.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# GATE0 Probe-3 (#827) — Mos byte-scope-verify CO-ATTESTATION (v7 full-closure)
|
||||
|
||||
**Principal:** Mos (orchestrator, merge authority for the mosaic-stack governance lane).
|
||||
**Committed under a DISTINCT git identity** (`mos-orchestrator@mosaic.local`) — deliberately NOT the
|
||||
`ms-lead-reviewer@mosaic.local` lane signer that authored both the harness and the §3 review — so this
|
||||
record stands as a *distinct-identity* co-attestation. See "Independence" below.
|
||||
|
||||
**Verify class:** independent provider-byte read (guarded `git show <full-40>:path | sha256sum`) plus
|
||||
source-level inspection of the executed mechanism. Not a re-build, not a re-run. This SUPERSEDES the v6
|
||||
co-attestation `12914d8` (and its SUPERSEDED-note `b6bd0cd`), which was byte-clear on the v6 bar only
|
||||
and was overtaken by homelab's third-principal FAIL @2d54a9dd + the resulting stricter full-closure bar.
|
||||
|
||||
## Package under attestation
|
||||
|
||||
| Artifact | Ref |
|
||||
|---|---|
|
||||
| Branch | `feat/827-gate0-probe` |
|
||||
| Harness commit-40 | `f609a44953f5ae61916805fcb45ca337de00b0b0` |
|
||||
| Harness `p3_d4_focused_run.py` sha256 | `0f1bd1b39399b32f243d901230e2d840794a2144edd723a095dab716833a7a9b` (32071 B) |
|
||||
| §3-review-v7 commit-40 | `2bba933f67c821899d320a938a9473a73a136422` (adds only the review `.md`; harness parent byte-stable) |
|
||||
| Review path | `docs/compaction-refresh/reviews/GATE0-PROBE3-V7-FULLCLOSURE-S3-REVIEW-v7.md` |
|
||||
|
||||
**Pinned closure (all 3 sha256 INDEPENDENTLY recomputed by Mos from git objects == pins):**
|
||||
|
||||
| Member | Source (commit:path) | Pin sha256 | Size |
|
||||
|---|---|---|---|
|
||||
| launcher `launch-runtime.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/launch-runtime.py` | `e950e422…` | 4237 B |
|
||||
| helper `lease_generation.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/lease_generation.py` | `061625402f…` | 3769 B |
|
||||
| broker `p3_generation_broker.py` | `23c0caca:docs/compaction-refresh/probes/p3_generation_broker.py` | `4db4fef1…` | — |
|
||||
|
||||
> Note: `f4008307` (the held WI-3 #830 gated head) is **not on origin** — it exists only as a local
|
||||
> `git worktree` on the build host. Mos recomputed the launcher+helper pins from that worktree
|
||||
> (`stack-cr-wi3-revoke`, HEAD == `f4008307`) rather than passing over a clone-completeness gap. The
|
||||
> broker pin was recomputed from origin `23c0caca`.
|
||||
|
||||
## Findings — VERDICT: byte-scope + mechanism + hygiene **PASS** (v7 full-closure bar)
|
||||
|
||||
Homelab's stricter bar — **hashed==executed on the FULL executed closure (launcher + helper + broker)**
|
||||
— is met. Verified at the source, not accepted on the review's assertion:
|
||||
|
||||
**B5 conjunction (the load-bearing repair) — HELD, all three legs:**
|
||||
- **(a) materialized from pinned git-object bytes, NOT the mutable worktree.** `materialize_closure`
|
||||
fetches each member via `git_object_bytes` (`git show {commit}:{path}`), then
|
||||
`sha256(data) == pin` **fail-closed** (`RuntimeError` on mismatch) before use.
|
||||
- **(b) no writable window hash→consume.** `pinned/` is `mkdir(mode=0o700)`; each file is written with
|
||||
`os.open(O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0o600)` — `O_EXCL` refuses a pre-planted file. No `chmod`,
|
||||
no `os.rename/replace`, no `symlink`, and nothing re-opens a pinned file for write (grep = 0). The
|
||||
pinned bytes are immutable within the fixture threat model between hash and exec.
|
||||
- **(c) re-hash == pin IMMEDIATELY before each exec, no interleaved yield.** Launcher: re-hash then
|
||||
`return PiRpc(command,…)` whose `__init__` **first statement** is `subprocess.Popen(command,…)` —
|
||||
zero IO/yield/reopen between. Broker + helper: both re-hashed then `subprocess.Popen` on the next
|
||||
line. Adjacency-alone-without-materialization (the v6 defect) is **absent** — all three exec from
|
||||
`pinned/`.
|
||||
|
||||
**B6 — helper pinned AND bound to the SAME single copy.** The broker receives
|
||||
`--generation-module {closure.generation}` (the pinned helper); the launcher runs from `pinned/` so its
|
||||
`import lease_generation` resolves to the sibling pinned copy via `sys.path[0]`. `closure_import_guard`
|
||||
AST-parses every member and raises on any non-stdlib import other than the allowed `lease_generation`
|
||||
— proving the dependency closure is complete and no unpinned module can enter at runtime.
|
||||
|
||||
**Closure = exactly 3 files, materialized inside the fixture root** (`root / "pinned"`). No path escapes
|
||||
the fixture temp root; no live/default broker; `.state` remains fixture-bound. **R1 owner tripwire NOT
|
||||
tripped** — this deepened isolation of an already-authorized touch, it did not widen the touched surface.
|
||||
|
||||
**Invariants (all INTACT):** BAR1 `GATED_WI_HEAD == f4008307` and `merge-base --is-ancestor 66b1e0a0
|
||||
f4008307` = YES (file-backed `.state` revocation fidelity present); BAR2 `.state` =
|
||||
`socket_path.parent / generation-{sid}.state`, `state_file_in_fixture_root` + `generation_source ==
|
||||
"state-file"` checks present; BAR3 `lease_anchor_registered` / live-path / fixture-socket isolation
|
||||
intact. `-O`-safe (0 bare `assert`); ABSENT-sweep (`yolo`/`execRuntime`/`p3_bank`/`promote_p2`/`retry`)
|
||||
= 0; single broker (1 def + 1 call site); `--runs choices=(3,)`; allow-list env (0 `os.environ.copy`);
|
||||
the only `shutil.copy2` is the legitimate `.pi/agent` credential copy (settings/auth/fd), **not** a
|
||||
launcher/helper/broker copy — the v6 copy-to-fixture concern is gone.
|
||||
|
||||
## Independence
|
||||
|
||||
- **Substantive principal-independence** of review-v7 is satisfied by an orchestrator-dispatched,
|
||||
builder-distinct Opus SECREV (`ms-secrev-828`, byte-only, non-builder) — that IS the substance of
|
||||
Gate-16.
|
||||
- The shared `ms-lead-reviewer` git signer on both the harness (`ms-rev-826` build) and the review
|
||||
commit is **evidentiary, not substantive**. It is resolved by (1) this Mos co-attestation under a
|
||||
**distinct** identity (`mos-orchestrator`), and (2) a homelab third-principal verify under its own
|
||||
distinct identity — three distinct-identity principals of record. Tracked fleet-infra tooling-gap
|
||||
(durable fix = per-lane distinct signers), **not a blocker**.
|
||||
|
||||
## Scope of this record — byte-clear, NOT fire-authorization
|
||||
|
||||
Producing probe evidence **executes** the Gate0 mechanism; a byte-clear is not a fire-authorization.
|
||||
This co-attestation clears the **bytes / scope / mechanism / hygiene on the v7 full-closure bar**. FIRE
|
||||
remains gated on: **homelab third-principal re-verify** (under its own distinct identity) + **Mos
|
||||
transparency-to-Jason** + **Mos explicit FIRE GO**. Until then: nothing banked, WI-3 #830 held at
|
||||
`f4008307` (unmoved), C-hatch armed (if the materialized-closure rig still no-fire / wrong-value /
|
||||
assertion-FAIL / isolation-FAIL → possible Case-C → STOP + escalate to Jason).
|
||||
|
||||
**Mos verdict: v7 full-closure byte-scope + mechanism + hygiene PASS. Co-attestation of record —
|
||||
committed.**
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ SUPERSEDED — homelab v7 third-principal FAIL @f609a449 raised a stricter bar (evidence-integrity note)
|
||||
|
||||
This co-attestation was **byte-clear on the v7 full-closure bar ONLY** and self-limited above to
|
||||
*"byte-clear NOT fire-authorization; FIRE remains gated on homelab third-principal re-verify."*
|
||||
Homelab (the required third principal) subsequently returned **FAIL @f609a449** (static verify, no
|
||||
code run), and Mos **UPHELD** it after independently confirming both findings in source — so the v7
|
||||
byte-clear this document records is **SUPERSEDED** and does **NOT** authorize FIRE.
|
||||
|
||||
Two residual isolation/binding holes WITHIN the materialized closure (both independently reproduced
|
||||
by Mos in the harness source; accepted as gate-**strengthening**, not softening):
|
||||
1. **Broker child env not isolated.** `launch_verified_broker` (`:551`) calls `Popen` with **no
|
||||
`env=`** (only `PiRpc.__init__` `:50` passes an allow-listed `env`) → the broker child inherits
|
||||
ambient `os.environ` (PYTHONPATH/PYTHONHOME/PYTHONPYCACHEPREFIX). `closure_import_guard` is a
|
||||
static AST check and cannot bind the child's runtime stdlib resolution.
|
||||
2. **Executed bytecode-cache outside the pin.** No `PYTHONDONTWRITEBYTECODE`/`-I`/`-B`/`__pycache__`
|
||||
handling anywhere. `exec_module` on the pinned helper writes derived `.pyc` the pin never covers;
|
||||
only the `.py` is re-hashed → executed bytecode ≠ pinned-source-hash.
|
||||
|
||||
Both break "hashed==executed on the FULL executed closure" on **fidelity** grounds even in a
|
||||
non-adversarial fixture. Mos **authorized the bounded repair** (broker `Popen` with strict
|
||||
allow-listed `env=` + `-I` + `PYTHON*` stripped; bytecode-cache suppressed via
|
||||
`PYTHONDONTWRITEBYTECODE=1`/`-B` + reject stray `__pycache__`/`.pyc` fail-closed before each
|
||||
consumer; launcher sibling-import binding to the pinned helper preserved). This **rides the existing
|
||||
(ii)-full-closure authorization + R1 + Mos adjudication** — it deepens isolation/binding of an
|
||||
already-authorized touch, stays inside the fixture temp root, no fresh Jason owner-window. It is
|
||||
**NOT a Case-C escalation** (no probe fired, no evidence produced — a static pre-fire catch, exactly
|
||||
what the review gate is for). Added review bars **B7** (broker child env-isolated) and **B8**
|
||||
(executed bytecode pinned-or-suppressed) on top of B5+B6+all priors.
|
||||
|
||||
**Live target = v8** (env-isolated + bytecode-pinned harness, forthcoming). `f609a449` /
|
||||
`2bba933f` / this co-attestation (`e08ad03`) are **SUPERSEDED**. A fresh Mos co-attestation will be
|
||||
committed on v8 byte-verify PASS. WI-3 #830 remains HELD at `f4008307`; nothing banked; C-hatch
|
||||
armed; NO FIRE.
|
||||
145
docs/compaction-refresh/reviews/GATE0-PROBE3-MOS-COATTEST-v9.md
Normal file
145
docs/compaction-refresh/reviews/GATE0-PROBE3-MOS-COATTEST-v9.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# GATE0 Probe-3 (#827) — Mos byte-scope-verify CO-ATTESTATION (v9 full-closure, env-isolated + bytecode-pinned)
|
||||
|
||||
**Principal:** Mos (orchestrator, merge authority for the mosaic-stack governance lane).
|
||||
**Committed under a DISTINCT git identity** (`mos-orchestrator@mosaic.local`) — deliberately NOT the
|
||||
`ms-lead-reviewer@mosaic.local` lane signer that authored the harness and the §3 review — so this
|
||||
record stands as a *distinct-identity* co-attestation. See "Independence".
|
||||
|
||||
**Verify class:** independent provider-byte read (`git show <full-40>:path | sha256sum`) plus
|
||||
source-level inspection of the executed mechanism and the v7→v9 delta. Not a re-build, not a re-run.
|
||||
This SUPERSEDES the v7 co-attestation `e08ad03` (and its SUPERSEDED note `2ae379e`), which was
|
||||
byte-clear on the v7 bar and was overtaken by homelab's third-principal FAIL @f609a449 (broker
|
||||
env-isolation + executed-bytecode-cache) and the resulting B7/B8 bar.
|
||||
|
||||
## Package under attestation
|
||||
|
||||
| Artifact | Ref |
|
||||
|---|---|
|
||||
| Branch | `feat/827-gate0-probe` |
|
||||
| Harness commit-40 | `1c34e3cb3172acdcd094e683e847d7c984afc96c` |
|
||||
| Harness `p3_d4_focused_run.py` sha256 | `29e5c7bfbe1911b52984bd94c79036bb1200ee82588318367b13c2b1053a0103` (32725 B) |
|
||||
| §3-review-v9 commit-40 | `e1c9a4682da2892ca5f5381012caffe1dd7b43a7` (parent == harness `1c34e3cb`, byte-stable; adds only the review `.md`; ms-secrev-828 authored) |
|
||||
| Review path | `docs/compaction-refresh/reviews/GATE0-PROBE3-V9-LAUNCHERFIX-S3-REVIEW-v9.md` |
|
||||
|
||||
**Pinned closure (all 3 sha256 INDEPENDENTLY recomputed by Mos from git objects == pins == embedded harness constants):**
|
||||
|
||||
| Member | Source (commit:path) | Pin sha256 | Size |
|
||||
|---|---|---|---|
|
||||
| launcher `launch-runtime.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/launch-runtime.py` | `e950e422…` | 4237 B |
|
||||
| helper `lease_generation.py` | `f4008307:packages/mosaic/framework/tools/lease-broker/lease_generation.py` | `061625402f…` | 3769 B |
|
||||
| broker `p3_generation_broker.py` | `23c0caca:docs/compaction-refresh/probes/p3_generation_broker.py` | `4db4fef1…` | — |
|
||||
|
||||
> `f4008307` (held WI-3 #830 gated head) is **not on origin** — recomputed launcher+helper from the
|
||||
> local `stack-cr-wi3-revoke` worktree (HEAD == `f4008307`); broker from origin `23c0caca`.
|
||||
|
||||
## Findings — VERDICT: byte-scope + mechanism + hygiene **PASS** (v9 = v7 full-closure + B7 + B8)
|
||||
|
||||
The v7→v9 delta is **exactly 22 insertions / 2 deletions**, confined to the intended B7+B8+B6c
|
||||
surface; every prior invariant is byte-stable (outside the delta) from the v7 verify.
|
||||
|
||||
**B7 — broker child env-ISOLATED (homelab finding 1 CLOSED).** `launch_verified_broker` (`:568`) now
|
||||
passes `env=environment` (the strict allow-list, `:570`) — the ambient-`os.environ`-inheritance hole
|
||||
is gone — AND runs the broker with `-I` (`:552`, isolated: ignores `PYTHON*`/user-site) + `-B`
|
||||
(`:553`). Both children are env-controlled: the launcher was already `env=env` at `PiRpc` (`:53`).
|
||||
|
||||
**B8 — executed bytecode PINNED/SUPPRESSED (homelab finding 2 CLOSED).** `reject_pinned_bytecode`
|
||||
(`:498`) raises `RuntimeError` fail-closed if a `__pycache__` dir or any `*.pyc` exists in the pinned
|
||||
dir, and is called before **each** consumer (launcher `:535`, broker `:562`). Bytecode writes are
|
||||
disabled via `PYTHONDONTWRITEBYTECODE=1` (`:729`) in the allow-list env **and** `-B` on both command
|
||||
lines. No unpinned `.pyc` can be executed; only the pinned `.py` re-hash governs.
|
||||
|
||||
**B6c — launcher sibling-import PRESERVED (v8 regression FIXED).** v8 over-applied `-I` to the
|
||||
launcher; on Py3.11+ `-I` implies `-P`, dropping the script dir from `sys.path[0]`, so the pinned
|
||||
launcher's bare `from lease_generation import` (launch-runtime.py:15) would `ModuleNotFoundError`. v9
|
||||
uses `-s` (`:514`) + `-B` (`:515`) on the launcher (NO `-I`) — neither touches `sys.path[0]`, so the
|
||||
sibling import still resolves to `pinned/lease_generation.py`. **Mos empirically re-verified on host
|
||||
Py3.11.2** (throwaway, not the harness): `-I` launcher → `ModuleNotFoundError`; `-s`+`PYTHONNOUSERSITE`
|
||||
→ import OK. The launcher's env isolation comes from the `PiRpc` `env=` allow-list, NOT `-I`, so
|
||||
dropping `-I` does **not** reopen B7. My earlier constraint-(c) assumption ("`-I` does not strip the
|
||||
script dir") was FALSIFIED for 3.11+; the author≠reviewer gate (ms-secrev-828) caught it — recorded.
|
||||
|
||||
**B5 conjunction (load-bearing repair) — HELD, all three legs (byte-stable from v7):**
|
||||
(a) materialized from pinned git-object bytes via `materialize_closure`/`git_object_bytes`, `sha256==pin`
|
||||
fail-closed; (b) `pinned/` `mkdir(0o700)` + `O_EXCL|O_CLOEXEC` `0o600`, no writable window — now also
|
||||
`reject_pinned_bytecode` closes the `.pyc` side-channel; (c) re-hash == pin IMMEDIATELY before each
|
||||
exec, no interleaved yield: launcher re-hash (`:538`) → `return PiRpc(command,…)` whose `__init__`
|
||||
first statement is `Popen` (`:50`); broker re-hash (`:563`) + helper re-hash (`:566`) → `Popen`
|
||||
(`:568`) on the next line.
|
||||
|
||||
**B6 — single pinned helper, complete closure.** Broker gets `--generation-module {closure.generation}`;
|
||||
launcher resolves `import lease_generation` to the sibling pinned copy via `sys.path[0]`.
|
||||
`closure_import_guard` (`:351`, called `:433`) AST-rejects any non-stdlib import other than
|
||||
`lease_generation`. Single broker: `launch_verified_broker` 1 def (`:543`) + 1 call (`:766`).
|
||||
|
||||
**Invariants (all INTACT):** BAR1 `GATED_WI_HEAD == f4008307` (`:36`) and `merge-base --is-ancestor
|
||||
66b1e0a0 f4008307` = YES (file-backed `.state` revocation fidelity present); fidelity
|
||||
`generation_source=='state-file'` (`:639`), `state_file_in_fixture_root` (`:641`),
|
||||
`MUTATOR_UNVERIFIED` (`:650`), `STALE_GENERATION` (`:651`); `lease_anchor_registered` (`:593`);
|
||||
`-O`-safe (0 bare `assert`); `--runs choices=(3,)` (`:838`); allow-list env (0 `os.environ.copy`);
|
||||
ABSENT-sweep (`yolo`/`execRuntime`/`p3_bank`/`promote_p2`/`retry`) = 0; the only `shutil.copy2`
|
||||
(`:708`) is the `.pi/agent` credential copy, not a closure copy.
|
||||
|
||||
**Closure = exactly 3 files, materialized inside the fixture root.** No path escapes the fixture temp
|
||||
root; no live/default broker; `.state` fixture-bound. **R1 owner tripwire NOT tripped** — B7/B8
|
||||
deepen isolation/binding of an already-authorized touch, they do not widen the touched surface.
|
||||
|
||||
## Independence
|
||||
|
||||
Substantive principal-independence of review-v9 is satisfied by an orchestrator-dispatched,
|
||||
builder-distinct Opus SECREV (`ms-secrev-828`, byte-only, non-builder, non-Mos). The shared
|
||||
`ms-lead-reviewer` git signer on harness+review commits is evidentiary, not substantive — resolved by
|
||||
(1) this Mos co-attestation under a **distinct** identity (`mos-orchestrator`) and (2) a homelab
|
||||
third-principal verify under its own distinct identity = three distinct-identity principals of record.
|
||||
Shared signer = tracked fleet-infra tooling-gap (durable fix = per-lane distinct signers), not a blocker.
|
||||
|
||||
## Scope of this record — byte-clear, NOT fire-authorization
|
||||
|
||||
Producing probe evidence **executes** the Gate0 mechanism; a byte-clear is not a fire-authorization.
|
||||
This clears **bytes / scope / mechanism / hygiene on the v9 (full-closure + B7 + B8) bar**. FIRE
|
||||
remains gated on: **homelab third-principal re-verify** (4th round, own distinct identity) + **Mos
|
||||
transparency-to-Jason** + **Mos explicit FIRE GO**. Until then: nothing banked, WI-3 #830 held at
|
||||
`f4008307` (unmoved), C-hatch armed (materialized-closure rig still no-fire / wrong-value /
|
||||
assertion-FAIL / isolation-FAIL → possible Case-C → STOP + escalate to Jason).
|
||||
|
||||
**Mos verdict: v9 full-closure + B7 + B8 byte-scope + mechanism + hygiene PASS. Co-attestation of
|
||||
record — committed.**
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ SUPERSEDED — homelab v9 4th-round FAIL @1c34e3cb raised a stricter *startup-closure* bar
|
||||
|
||||
This co-attestation was **byte-clear on the v9 (env-iso + bytecode-pin) bar ONLY** and self-limited
|
||||
above to *"byte-clear NOT fire-authorization; FIRE remains gated on homelab 4th-round re-verify."*
|
||||
Homelab (the required third principal) returned **FAIL @1c34e3cb** (static, nothing executed), and Mos
|
||||
**UPHELD** it after independently confirming the finding in-source AND empirically on host Py3.11.2 —
|
||||
so the v9 byte-clear this document records is **SUPERSEDED** and does **NOT** authorize FIRE.
|
||||
|
||||
**Residual startup-closure hole (empirically reproduced by Mos; accepted as gate-STRENGTHENING):**
|
||||
neither child carries `-S`, so CPython imports the `site` module **before** the script runs. `-s`
|
||||
(launcher) suppresses only *user*-site; `-I` (broker) implies `-s -E -P` but **NOT** `-S`. Proven:
|
||||
|
||||
[-s -B ] no_site=0 site_imported=True ← v9 launcher: site runs
|
||||
[-I -B ] no_site=0 site_imported=True ← v9 broker: -I does NOT imply -S
|
||||
[-s -S -B] no_site=1 site_imported=False ← v10 launcher fix (sibling import STILL resolves)
|
||||
[-I -S -B] no_site=1 site_imported=False ← v10 broker fix (additive)
|
||||
|
||||
System-site executable `.pth` lines + sitecustomize/usercustomize can therefore run **unpinned startup
|
||||
code** before the exact launcher/broker and **outside** `closure_import_guard`, while every hash +
|
||||
`reject_pinned_bytecode` + import-guard still pass — defeating hashed==executed on the full *startup*
|
||||
closure (strictly wider than the module-import closure v9 cleared). A genuine fidelity hole for a
|
||||
fail-closed DO-178C evidence gate.
|
||||
|
||||
Mos **authorized the bounded v10 repair**: add `-S` to the **launcher** (keep `-s -B`, NOT `-I`) and
|
||||
to the **broker** (keep `-I -B`) — a minimal 2-line delta; no B6c regression (launcher `-s -S -B`
|
||||
sibling import empirically intact; `-S` does not touch `sys.path[0]`). Added review bar **B9**
|
||||
(no-site startup closure). This **rides the existing (ii)-full-closure authorization + R1 + Mos
|
||||
adjudication** (deepens startup-closure isolation of an already-authorized touch, inside the fixture
|
||||
temp root, no fresh Jason owner-window) and is **NOT a Case-C escalation** (static pre-fire catch, no
|
||||
probe fired). A separate **FIRE-time** constraint is captured: the 3× isolation dispatch must launch
|
||||
the runner under externally-enforced `python -I -S -B` (a self-reexec is too late).
|
||||
|
||||
**Live target = v10** (no-site harness, forthcoming). `1c34e3cb` / `e1c9a468` / this co-attestation
|
||||
(`f320d075`) are **SUPERSEDED**; the prior 2-of-3 (ms-secrev-828 v9 §3 PASS + `f320d075`) does NOT
|
||||
carry — all three distinct-identity principals re-verify the new v10 SHA. A fresh Mos co-attestation
|
||||
will be committed on v10 byte-verify PASS. WI-3 #830 remains HELD at `f4008307`; nothing banked;
|
||||
C-hatch armed (fired-rig only); NO FIRE.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Gate0 Probe-3 (D4) NEW-5/NEW-6 Closure — §3-Conformance Review v6
|
||||
|
||||
**Verdict: ✅ PASS**
|
||||
|
||||
## Pin (G1 — reviewed object, mandatory)
|
||||
|
||||
- **Reviewed object = `2d54a9dd14cb924701b2ae4ed72dae4df760c4e3`** (harness commit, branch `feat/827-gate0-probe`, approach-**(i)** build; supersedes the reverted (ii) copy-to-fixture builds `7ff63cd5`/`6164dc07`, which were NOT reviewed to a verdict).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `15a154df55273f51301763a984485fd63813f6d1f05d2728abb9fb8b9c040b1a` (27366 bytes, no not-found sentinel).
|
||||
- **Co-reviewed fixture broker:** `p3_generation_broker.py` sha256 `4db4fef1ac6658a8ca79ad5091cefc901d2aa26003265c3d6726c294cf895cad` — **byte-identical to v5** (unchanged by this delta).
|
||||
|
||||
## Principal-independence attestation (Mos independence ruling — process of record)
|
||||
|
||||
This review is produced by a **distinct Opus SECREV session, orchestrator-dispatched** — the **`ms-secrev-828`
|
||||
reviewer lane, dispatched by `mosaic-100`** — **byte review only, ran nothing**, and **did NOT build** this harness
|
||||
(builder = ms-rev-826). Author ≠ reviewer (Gate-16). This is one of three principals: **Mos commits his own
|
||||
distinct-identity byte-scope-verify co-attestation at v-final**, and **homelab's independent verify is the third
|
||||
principal**. (The `ms-lead-reviewer` **Git signer identity** shared across published review commits is a git-signer
|
||||
question routed to Mos; it does not bear on this lane's process/dispatch independence, attested here.)
|
||||
|
||||
## NEW-5 CLOSED — approach (i): exact-byte pin, adjacent re-hash, exec in place
|
||||
|
||||
- **Exact-byte sha256 is the trust anchor, NOT substring heuristics.** Launcher bytes are read from the immutable
|
||||
git object (`git show f4008307:<path>`) and gated on `sha256(launcher_bytes) == GATED_LAUNCHER_SHA256`
|
||||
(`e950e422…`) (:354). The `behavior_markers` `in`-checks (:364-372) are explicitly commented "the exact launcher
|
||||
digest above is the trust anchor. These marker checks are diagnostic belt-and-suspenders only, never a substitute
|
||||
for the pin" (:362-363). The old ordered `.find()` heuristic (`register < initialize < execute`, min<0) is **gone**.
|
||||
- **Final re-hash immediately adjacent to `Popen`, no interleaved yield.** `launch_verified_pi` assembles `command`,
|
||||
then — as the statement **immediately before** `return PiRpc(command, …)` (which performs the `Popen`) — re-hashes
|
||||
the launcher: `if hashlib.sha256(launcher.read_bytes()).hexdigest() != GATED_LAUNCHER_SHA256: raise` (:411-412),
|
||||
`return PiRpc(...)` (:413). **No harness-controlled step (no `wait_path`, no broker spawn) sits between the re-hash
|
||||
and the exec** — the broker `Popen` + `wait_path` occur *before* `launch_verified_pi` is called (:611-621). Window
|
||||
narrowed to the fork/exec itself.
|
||||
- **Exec stays IN PLACE at the pinned f4008307-worktree path.** The precondition returns the worktree paths
|
||||
`gated_root / launcher_relative`, `gated_root / generation_relative` (:378); Pi execs `str(launcher)` = that
|
||||
worktree launch-runtime.py (:390,:621), and the broker `--generation-module` = the worktree lease_generation.py
|
||||
(:614). The reverted (ii) machinery is **gone**: `grep pinned-lease-broker / PYTHONPATH / fixture_launcher /
|
||||
fixture_generation / write_bytes == 0`. Launcher import resolution and the file-backed fidelity surface are
|
||||
therefore **unperturbed** (this is the lower-risk approach Mos mandated over copy-to-fixture).
|
||||
|
||||
## NEW-6 CLOSED — portable, validated, off-by-one gone
|
||||
|
||||
`GATED_WI_ROOT` is no longer the off-by-one `HERE.parents[3].parent / "stack-cr-wi3-revoke"`. It is resolved by
|
||||
`resolve_gated_wi_root()` (:257-307): an explicit `GATED_WI_ROOT` env override, else **repo-relative** `git worktree
|
||||
list --porcelain` (from `repository_root()`, first parent containing `.git`) selecting the **unique** worktree whose
|
||||
`HEAD == f4008307` **and** `branch == refs/heads/feat/830-compaction-revoke` (raise if ambiguous/absent). It then
|
||||
**fail-closes** unless `gated_root.is_dir()`, `git rev-parse --is-inside-work-tree == "true"` (:304-305), and
|
||||
`HEAD == GATED_WI_HEAD` (:306-307). Independently recomputed on this host (git query, harness not run): it resolves
|
||||
to the real worktree **`/home/hermes/agent-work/stack-cr-wi3-revoke`**. Portable + validated; the off-by-one is gone.
|
||||
|
||||
## Full v4/v5 carry-over re-sweep (byte-stable vs `7f975b95` except the NEW-5/6 delta)
|
||||
|
||||
`diff 7f975b95 → 2d54a9dd` confines changes to launcher resolution (NEW-6) + adjacent-rehash-exec-in-place (NEW-5);
|
||||
nothing else moved. Re-swept intact: **creds-scrub** (`scrub_fixture_credentials` + outermost `finally`); **`source-invalid`
|
||||
ABSENT** (grep=0 both files); **fidelity file-backed** unperturbed (broker `read_runtime_generation`/`bump_runtime_generation`
|
||||
on the fixture `.state`; `assert_d4` `generation_source=="state-file"` + `state_file_drives_lifecycle` +
|
||||
`state_file_in_fixture_root` + `new→MUTATOR_UNVERIFIED`/`prior→STALE_GENERATION`); **`lease_anchor_registered`** INTACT
|
||||
(event + `hex-256`); **live-path** gated launcher; **fail-closed precondition**; **fixture-socket isolation**; **`-O`-safe**
|
||||
(0 bare `assert`, PASS derived); **allow-list env** (0 `os.environ.copy`); **`--runs choices=(3,)`**; **single p3 broker**
|
||||
(broker byte-identical to v5). **ABSENT sweep = 0** (P5/P6/P2-bank/retry-launder/mosaic-yolo/execRuntime/pi_gate0/run_open/
|
||||
atomic; extension actions only bump/lifecycle/authorize/promote; no exec-at-import). **Beyond-R1 tripwire: not tripped** —
|
||||
exec is in place, imports and the file-backed observation surface untouched; no isolation crossing.
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `2d54a9dd`.** NEW-5 (approach (i): exact-byte sha256 pin as trust anchor; adjacent re-hash immediately
|
||||
before `Popen` with no interleaved yield; exec in place at the pinned f4008307-worktree path; (ii) copy-to-fixture/
|
||||
PYTHONPATH machinery reverted) and NEW-6 (portable, validated, off-by-one-gone root resolution) are both **closed**;
|
||||
the full v4/v5 carry-over holds byte-stable except the two intended surfaces; ABSENT sweep is 0; the R1 file-backed
|
||||
fidelity surface is unperturbed. Zero out-of-scope surface.
|
||||
|
||||
**Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; **FIRE remains
|
||||
separately gated on Mos's explicit post-clear GO** and is not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** distinct Opus SECREV session (`ms-secrev-828` lane, dispatched by `mosaic-100`), Gate-16 author≠reviewer,
|
||||
byte review only; ran nothing; did not build.
|
||||
**Reviewed object (pin):** `2d54a9dd14cb924701b2ae4ed72dae4df760c4e3` · harness sha256 `15a154df55273f51301763a984485fd63813f6d1f05d2728abb9fb8b9c040b1a`.
|
||||
@@ -0,0 +1,86 @@
|
||||
# Gate0 Probe-3 (D4) No-Site Startup Closure — §3-Conformance Review v10
|
||||
|
||||
**Verdict: ✅ PASS**
|
||||
|
||||
## Pin (GUARD 1 — reviewed object)
|
||||
|
||||
- **Reviewed object = `ce5ba762051354338889959bfce2b0381f4a4e2a`** (harness commit, branch `feat/827-gate0-probe`).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `7e14ead89a7b2a297fcc17e7653291b3bcace1d2002a8f90a989db74f6985b6f` (32753 bytes, no not-found sentinel).
|
||||
- **Closure pins (unchanged):** launcher `e950e422…` @f4008307 · helper `061625402f08488eac47acd23272904e71fd1a71fd15b3bdab158632c801be4c` @f4008307 · broker `4db4fef1…` @23c0caca.
|
||||
|
||||
## Independence (GUARD 2 — principal-independence attestation)
|
||||
|
||||
Distinct Opus SECREV session, orchestrator-dispatched — the **`ms-secrev-828` reviewer lane, dispatched by
|
||||
`mosaic-100`** — **byte review only, ran nothing** (harness/broker not executed); did **not** build this harness
|
||||
(builder = ms-rev-826); is not Mos; distinct principal from both. This re-verifies from scratch on the v10 SHA after
|
||||
homelab's 4th-round FAIL @`1c34e3cb` (no `-S` → `site` startup-closure hole) superseded my v9 PASS + Mos's co-attest.
|
||||
The `-S`/`-s`/`-I` behavior checks below use a *throwaway* script to observe interpreter startup — not the harness.
|
||||
|
||||
## ★ B9 — No-site startup closure (the homelab 4th-round FAIL)
|
||||
|
||||
The delta vs `1c34e3cb` is **exactly two `-S` insertions**, byte-confirmed by `diff` (nothing else; +28 B fully
|
||||
accounted by the two ` "-S",\n` lines):
|
||||
|
||||
- **(i) Launcher command** (`launch_verified_pi`, :512-516): `sys.executable, "-s", "-S", "-B", str(launcher), …` —
|
||||
carries `-s` + **`-S`** + `-B`, and **no `-I`**.
|
||||
- **(ii) Broker command** (`launch_verified_broker`, :551-554): `sys.executable, "-I", "-S", "-B", str(broker_path), …`
|
||||
— carries `-I` + **`-S`** + `-B`.
|
||||
- **(iii) `site` not imported at child startup** — empirically confirmed (Python 3.11.2, throwaway script):
|
||||
`python3 -s -S -B main.py` → `sys.flags.no_site == 1`, `'site' in sys.modules == False`; `python3 -I -S -B main.py`
|
||||
→ `no_site == 1`. So system-site `.pth` executable lines and `sitecustomize`/`usercustomize` **cannot run unpinned
|
||||
startup code** before the pinned launcher/broker. (Contrast without `-S`: `python3 -s -B` → `no_site == 0`, `site`
|
||||
imported — the exact v9 hole this closes.)
|
||||
- **(iv)** No harness reliance on any site-injected path/hook (env is the constructed allow-list; children execute
|
||||
pinned bytes).
|
||||
- **(v) `-S` does not touch `sys.path[0]`** (unlike `-I`/`-P`) — empirically confirmed: `python3 -s -S -B main.py`
|
||||
keeps `sys.path[0]` = the script's directory, so the launcher's bare `from lease_generation import
|
||||
initialize_runtime_generation` (`f4008307:launch-runtime.py:15`) **still binds `pinned/lease_generation.py`** with
|
||||
`-S` present. The broker's explicit `--generation-module` import (via `importlib`) binds the pinned helper
|
||||
regardless of `sys.path`/site, so `-I -S` is correct there.
|
||||
- **(vi) Delta = exact 2-line `-S` only** vs `1c34e3cb` (git-diff/byte-compared, not accepted on assertion).
|
||||
|
||||
## All prior bars — byte-stable (delta was only the two `-S` lines)
|
||||
|
||||
- **B6(c):** launcher still carries no `-I`; sibling import binds `pinned/` (confirmed above with `-S` present). ✅
|
||||
- **B7:** broker `Popen` `env=environment` (allow-list, **not** `os.environ`; no `PYTHONPATH`/`PYTHONHOME`/
|
||||
`PYTHONPYCACHEPREFIX`) + `-I`. ✅
|
||||
- **B8:** `reject_pinned_bytecode` fail-closed before each consumer; `PYTHONDONTWRITEBYTECODE=1` + `PYTHONNOUSERSITE=1`
|
||||
in env; `-B` on both children. ✅
|
||||
- **B5:** 3-leg conjunction — materialize each of launcher/helper/broker from git-object bytes with `sha256==pin`
|
||||
fail-closed; `pinned/` `0o700` in fixture root, files `O_EXCL 0o600` (no writable window); re-hash `==pin`
|
||||
immediately before each `Popen`; launcher + broker consume the same single pinned helper. ✅
|
||||
- **B6:** `closure_import_guard` AST present; single pinned helper; broker `--generation-module = closure.generation`. ✅
|
||||
- **Fidelity:** extension bumps `generation-{sid}.state` via `MOSAIC_LEASE_GENERATION_FILE` (not in-mem); broker
|
||||
`read_runtime_generation`; `assert_d4` `generation_source=="state-file"` / `state_file_in_fixture_root` /
|
||||
`new→MUTATOR_UNVERIFIED` / `prior→STALE_GENERATION`; `.state` fixture-root-bound. ✅
|
||||
- **Traceability:** `GATED_WI_HEAD == f4008307` + `merge-base --is-ancestor 66b1e0a0 f4008307`. ✅
|
||||
- `lease_anchor_registered` + `hex-256` INTACT; LIVE-PATH (pinned gated launcher); single p3 broker;
|
||||
promotion=fixture-only; `--runs choices=(3,)`; `-O`-safe (0 bare `assert`); `copy2` = creds-only; allow-list env
|
||||
(0 `os.environ.copy`). ✅
|
||||
|
||||
## ABSENT sweep
|
||||
|
||||
P5/P6/P2-bank/retry-launder/mosaic-yolo/execRuntime = 0; `source-invalid` = 0; no live/real-broker path; no `.state`
|
||||
outside fixture root; no exec-at-import (`__main__`-guarded); no adjacency-only-exec-from-worktree; the only change is
|
||||
the authorized `-S` no-site isolation-deepening (no mechanism change, no scope-widen); `-O`-safe.
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `ce5ba762`.** B9 (no-site startup closure) is closed — both children carry `-S`, `site` is not imported at
|
||||
startup (so system-site `.pth`/`sitecustomize` cannot execute unpinned code before the pinned launcher/broker),
|
||||
`-S` leaves `sys.path[0]` intact so the launcher sibling import and the broker explicit-path import both still bind
|
||||
the pinned helper, and the delta vs `1c34e3cb` is exactly the two `-S` insertions. All prior bars (B5/B6/B6c/B7/B8/
|
||||
fidelity/traceability/lease_anchor/live-path/single-broker/promotion/`--runs`/`-O`-safe) are byte-stable. Zero
|
||||
out-of-scope surface. **Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; **FIRE remains
|
||||
separately gated on Mos's explicit post-clear GO**, his byte-scope-verify co-attestation, and homelab's
|
||||
third-principal verify — not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** distinct Opus SECREV session (`ms-secrev-828` lane, dispatched by `mosaic-100`), Gate-16 author≠reviewer,
|
||||
byte review only; ran nothing.
|
||||
**Reviewed object (pin):** `ce5ba762051354338889959bfce2b0381f4a4e2a` · harness sha256 `7e14ead89a7b2a297fcc17e7653291b3bcace1d2002a8f90a989db74f6985b6f`.
|
||||
@@ -0,0 +1,120 @@
|
||||
> ⚠ SUPERSEDED by v8 (homelab 3rd-principal FAIL @f609a449: broker env-inherit + unpinned .pyc; PASS overtaken by stricter B7+B8 bar). This v7 PASS record is NOT fire-authorization.
|
||||
|
||||
# Gate0 Probe-3 (D4) Full-Closure Materialization — §3-Conformance Review v7
|
||||
|
||||
**Verdict: ✅ PASS** *(superseded — see banner above)*
|
||||
|
||||
## Pin (GUARD 1 — reviewed object)
|
||||
|
||||
- **Reviewed object = `f609a44953f5ae61916805fcb45ca337de00b0b0`** (harness commit, branch `feat/827-gate0-probe`).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `0f1bd1b39399b32f243d901230e2d840794a2144edd723a095dab716833a7a9b` (32071 bytes, no not-found sentinel).
|
||||
- **Closure pins (independently recomputed from the git objects):**
|
||||
- launcher `packages/mosaic/framework/tools/lease-broker/launch-runtime.py` @ `f4008307` = `e950e4224e280f16979d90cabb89aa1896c5ee28bed2df957e14d018d43cda82` ✓
|
||||
- **helper** `packages/mosaic/framework/tools/lease-broker/lease_generation.py` @ `f4008307` = `061625402f08488eac47acd23272904e71fd1a71fd15b3bdab158632c801be4c` ✓
|
||||
- **broker** `docs/compaction-refresh/probes/p3_generation_broker.py` @ `23c0caca` = `4db4fef1ac6658a8ca79ad5091cefc901d2aa26003265c3d6726c294cf895cad` ✓
|
||||
|
||||
## Independence (GUARD 2 — principal-independence attestation)
|
||||
|
||||
This review is produced by a **distinct Opus SECREV session, orchestrator-dispatched — the `ms-secrev-828`
|
||||
reviewer lane, dispatched by `mosaic-100`** — **byte review only, ran nothing**, that **did NOT build** this harness
|
||||
(builder = ms-rev-826) and **is not Mos**. Three distinct principals: this reviewer, the builder, and Mos (whose
|
||||
own distinct-identity byte-scope-verify follows); homelab's independent verify is a further principal — it is
|
||||
homelab's third-principal FAIL @`2d54a9dd` (upheld by Mos) that correctly retired the approach-(i) adjacency
|
||||
re-hash and authorized this full-closure. v6/`2d54a9dd`/`23c0caca`/`12914d8` are superseded.
|
||||
|
||||
## Why v7 (the reopen-after-hash hole)
|
||||
|
||||
Approach (i) re-hashed the launcher then let `Popen` **reopen the worktree path** — hashed-snapshot ≠ executed-bytes
|
||||
(the worktree file is a shared, same-UID-mutable path). Statement adjacency alone did not bind. v7 closes it for the
|
||||
**full project-code closure** (launcher + `lease_generation.py` helper + `p3_generation_broker.py`).
|
||||
|
||||
## B5 — HASHED == EXECUTED on the full closure (binding conjunction, stated verbatim)
|
||||
|
||||
The reopen-after-hash shape is unavoidable for imported/exec'd files, so closure rests on the **conjunction of all
|
||||
three legs**, each byte-verified here:
|
||||
|
||||
> **(a)** bytes are materialized **from the pinned git-object @ `f4008307`** (helper/launcher) and **@ `23c0caca`**
|
||||
> (broker) — `git show <commit>:<path>`, the trusted immutable object, **never the mutable worktree file**; **AND**
|
||||
> **(b)** into a **fixture-private `0o700` dir with `0o600` files created via `O_CREAT|O_EXCL`** — no writer exists in
|
||||
> the threat model between hash and exec; **AND** **(c)** each member is **re-hashed == its pin IMMEDIATELY before
|
||||
> exec/import, fail-closed (`RuntimeError`)**.
|
||||
|
||||
Byte evidence:
|
||||
- **(a)** `git_object_bytes(git_root, commit, relative)` = `git show <commit>:<path>` (:322-327); `materialize_closure`
|
||||
reads all three members from git objects and asserts `sha256(data) == digest` else `RuntimeError` (:415-433). Worktree
|
||||
working-tree files are never read.
|
||||
- **(b)** `pinned = root / "pinned"; pinned.mkdir(mode=0o700)` (:435-436); `write_pinned_file` uses
|
||||
`os.open(path, O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0o600)` (:379-382). **No `os.chmod`/`os.rename`/`shutil.move`
|
||||
anywhere** (grep=0); `O_EXCL` refuses a pre-planted file/symlink, so no symlink-follow or hijack gap; the dir is a
|
||||
fresh per-run `mkdtemp` child, owner-only. **No code re-opens the pinned files for write between materialize and
|
||||
consume** — there is no writable window.
|
||||
- **(c)** launcher re-hash `sha256(launcher.read_bytes()) == GATED_LAUNCHER_SHA256` is the statement immediately before
|
||||
`return PiRpc(command,…)` (:528-530); broker **and** helper re-hashes (`== GATED_BROKER_SHA256`,
|
||||
`== GATED_GENERATION_SHA256`) are the two statements immediately before `return subprocess.Popen(command,…)`
|
||||
(:546-551). No interleaved harness yield.
|
||||
|
||||
Adjacency-only exec-from-worktree is **absent** for every member (all three exec/import from `pinned/`; grep worktree-exec=0).
|
||||
|
||||
## B6 — helper + broker pinned and bound to execution (one shared helper)
|
||||
|
||||
`materialize_closure` writes exactly **one** `pinned/lease_generation.py` (:442). The broker executes the **pinned**
|
||||
broker with `--generation-module = closure.generation` = that pinned helper (`launch_verified_broker`, :533-551, called
|
||||
:746-747). The launcher executes the **pinned** launcher (`python3 pinned/launch-runtime.py`), whose
|
||||
`import lease_generation` resolves via `sys.path[0]` = the script's own `pinned/` dir to the **same** sibling
|
||||
`pinned/lease_generation.py`. Launcher-import and broker-`--generation-module` therefore resolve the **same single
|
||||
pinned helper copy**, not two copies and not the worktree. Worktree helper/broker are not re-read at runtime.
|
||||
|
||||
**Closure-import guard:** `closure_import_guard` AST-parses each member and refuses any non-stdlib import outside the
|
||||
allow-set `{"lease_generation"}` (and any relative import) → `RuntimeError` (:341-364). The 3-member closure is
|
||||
therefore provably complete — no unpinned project-code dependency can slip in.
|
||||
|
||||
## BAR1 — Traceability
|
||||
|
||||
`GATED_WI_HEAD == f4008307`; the precondition asserts `git merge-base --is-ancestor 66b1e0a0 f4008307` (:315-330),
|
||||
independently confirmed **YES** — the pinned launcher forward-contains the `66b1e0a0` file-backed generation mechanism.
|
||||
|
||||
## BAR2 — Fidelity file-backed, `.state` in fixture root, UNTOUCHED
|
||||
|
||||
`pinned/` holds **code bytes only** (launcher/helper/broker). The `.state` generation file is written by the launcher
|
||||
to `socket_path.parent` (the fixture root), **not** `pinned/`. The broker (pinned, byte-identical `4db4fef1`) still
|
||||
enforces `generation_environment` raising if `state_path.parent != socket_path.parent` (grep=2), and `assert_d4`
|
||||
still checks `state_file_source == "state-file"` / `state_file_drives_lifecycle` / `state_file_in_fixture_root` +
|
||||
`new→MUTATOR_UNVERIFIED` / `prior→STALE_GENERATION` (grep=4, unchanged). The v7 change did not move `.state` into
|
||||
`pinned/` or perturb these asserts.
|
||||
|
||||
## BAR3 — Carry-over
|
||||
|
||||
(a) **live-path:** Pi launched via `python3 pinned/launch-runtime.py --runtime pi -- pi …` (gated register-before-exec);
|
||||
`mosaic yolo`/`execRuntime` = 0. (b) **fail-closed precondition:** `gated_launcher_precondition` (resolve+materialize+
|
||||
verify) runs before any launch, fail-closed. (c) **fixture-socket isolation:** `MOSAIC_LEASE_BROKER_SOCKET` = per-run
|
||||
fixture socket; single pinned p3 broker serves `register_anchor`; no live/default broker reachable; non-destructive.
|
||||
(d) **`lease_anchor_registered` INTACT:** event + `session_id_shape=="hex-256"` unchanged (broker byte-identical);
|
||||
`assert_d4` folds it into the single-identity set — not deleted/softened/optional/repointed.
|
||||
|
||||
## Re-confirm + ABSENT sweep
|
||||
|
||||
Spawns ONLY the single pinned p3 broker; promotion=fixture-only; full D4 asserts; `--runs choices=(3,)`; allow-list
|
||||
env (0 `os.environ.copy`); **`-O`-safe** (all new checks `RuntimeError`, **0 bare `assert`**); creds-scrub intact;
|
||||
non-destructive (fixture tempdir only); deterministic (git objects + fixed pins); closure-import guard present.
|
||||
**ABSENT = 0:** P5/P6/P2-bank/retry-launder/mosaic-yolo/execRuntime/pi_gate0; `source-invalid` grep=0; no live/real-broker
|
||||
path; no `.state`/gen path outside the fixture root; no extra broker/socket; no exec-at-import (`__main__`-guarded); no
|
||||
adjacency-only exec-from-worktree for any member; the only mechanism change is materialization; no scope-widen.
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `f609a449`.** B5 (full-closure hashed==executed via the (a)+(b)+(c) conjunction with no writable window),
|
||||
B6 (one shared pinned helper bound to both launcher-import and broker-`--generation-module`; complete closure), BAR1,
|
||||
BAR2 (fidelity `.state`-in-fixture-root untouched), and BAR3 all hold, with zero out-of-scope surface. **Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; **FIRE remains
|
||||
separately gated on Mos's explicit post-clear GO**, his v-final byte-scope-verify co-attestation, and homelab's
|
||||
third-principal verify — not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** distinct Opus SECREV session (`ms-secrev-828` lane, dispatched by `mosaic-100`), Gate-16 author≠reviewer,
|
||||
byte review only; ran nothing; did not build.
|
||||
**Reviewed object (pin):** `f609a44953f5ae61916805fcb45ca337de00b0b0` · harness sha256 `0f1bd1b39399b32f243d901230e2d840794a2144edd723a095dab716833a7a9b`.
|
||||
**Pinned closure:** launcher `e950e422…` @f4008307 · helper `06162540…be4c` @f4008307 · broker `4db4fef1…` @23c0caca.
|
||||
@@ -0,0 +1,92 @@
|
||||
> ⚠ SUPERSEDED by v9: the reviewed harness `a92ad090` is superseded by the narrow fix `1c34e3cb` (my v8 B6(c) FAIL — `-I` on the launcher — was remediated by `-I`→`-s` + `PYTHONNOUSERSITE=1`; re-review v9 = PASS). This v8 FAIL record pertains to the superseded commit.
|
||||
|
||||
# Gate0 Probe-3 (D4) Broker Env-Isolation + Bytecode Binding — §3-Conformance Review v8
|
||||
|
||||
**Verdict: ❌ FAIL** (B7 and B8 land correctly, but the same change breaks **B6(c)**: the launcher is run with `-I`, which strips the script directory from `sys.path` on Python 3.11+, so its bare `import lease_generation` cannot resolve the pinned helper — empirically confirmed).
|
||||
|
||||
## Pin (GUARD 1 — reviewed object)
|
||||
|
||||
- **Reviewed object = `a92ad090ae3828c643f961c7628d809b8521185f`** (harness commit, branch `feat/827-gate0-probe`).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `915ebeb5aeab108cb60c5f629c1db520623ab4914eed427ca34ee66f9aa08390` (32614 bytes, no not-found sentinel).
|
||||
- **Closure pins (unchanged from v7):** launcher `e950e422…` @f4008307 · helper `061625402f08488eac47acd23272904e71fd1a71fd15b3bdab158632c801be4c` @f4008307 · broker `4db4fef1…` @23c0caca.
|
||||
|
||||
## Independence (GUARD 2 — principal-independence attestation)
|
||||
|
||||
Distinct Opus SECREV session, orchestrator-dispatched — the **`ms-secrev-828` reviewer lane, dispatched by
|
||||
`mosaic-100`** — **byte review only, ran nothing** (the harness/broker were not executed); did **not** build this
|
||||
harness (builder = ms-rev-826); is not Mos. This verdict is my own. (The `-I` semantics check below runs a *throwaway*
|
||||
two-line script to observe the interpreter's `sys.path` behavior — it does not run the harness, broker, or any part of
|
||||
the reviewed closure.)
|
||||
|
||||
## 🔴 BLOCKING FINDING — B6(c) broken: `-I` on the launcher strips the pinned-helper import path
|
||||
|
||||
**File:line — `p3_d4_focused_run.py:512`** (the `"-I"` added to `launch_verified_pi`'s launcher command).
|
||||
|
||||
The pinned launcher `launch-runtime.py` @`f4008307` imports its helper with a **bare top-level import**:
|
||||
`from lease_generation import initialize_runtime_generation` (launcher line 15) — no `sys.path` manipulation. Under
|
||||
v7 this bound because `python3 pinned/launch-runtime.py` put the script's directory (`pinned/`) at `sys.path[0]`, so
|
||||
the sibling `lease_generation` resolved to `pinned/lease_generation.py`.
|
||||
|
||||
v8 now runs the launcher as `python3 -I -B pinned/launch-runtime.py …` (:512-513). **`-I` implies `-P` (Python 3.11+),
|
||||
which does NOT prepend the script's directory to `sys.path`.** Empirically confirmed on this host (Python 3.11.2),
|
||||
using a throwaway script (not the harness):
|
||||
|
||||
```
|
||||
python3 -I -B main.py → sys.path[0] = '/usr/lib/python311.zip'
|
||||
import sibling → ModuleNotFoundError: No module named '…'
|
||||
python3 -B main.py → sys.path[0] = '<script dir>' → sibling import: OK
|
||||
```
|
||||
|
||||
Therefore, at FIRE on Python 3.11+, the launcher's line-15 `from lease_generation import …` raises
|
||||
`ModuleNotFoundError` at module load — the pinned helper does **not** resolve (neither pinned nor worktree; the import
|
||||
simply fails). **B6(c) — "launcher sibling-import to `pinned/` via `sys.path[0]` STILL BINDS" — does not hold.** The
|
||||
build report's assertion "`-I` keeps script dir" is false on 3.11+, and could not have been observed under the
|
||||
correct "never run" boundary.
|
||||
|
||||
Note: the env allow-list carries no `PYTHONPATH` (correct for B7), and `-I` ignores `PYTHON*` env regardless, so there
|
||||
is no alternate resolution path — the launcher import is unrecoverable under `-I`.
|
||||
|
||||
**Fix:** remove `-I` from the **launcher** command only (keep `-B` + the `env=` allow-list — the launcher's
|
||||
env-isolation is already provided by the constructed allow-list, which contains no `PYTHONPATH`/`PYTHONHOME`/
|
||||
`PYTHONPYCACHEPREFIX`, and it needs `pinned/` at `sys.path[0]` for the sibling import). Keep `-I` on the **broker**
|
||||
command (it loads the helper by explicit `--generation-module` path via `importlib`, so it never needs the script
|
||||
dir on `sys.path`). Alternatively, inject the pinned dir explicitly (e.g. `PYTHONPATH=pinned/` — but that reintroduces
|
||||
a `PYTHON*` passthrough B7 forbids, so dropping `-I` on the launcher is the clean fix).
|
||||
|
||||
## What DID land correctly (for the author's fast turnaround)
|
||||
|
||||
- **B7 — broker child env-isolated: correct.** `launch_verified_broker` now takes `environment` and passes
|
||||
`env=environment` (the constructed allow-list, **not** `os.environ`) to `Popen` (:566-568); the broker command
|
||||
includes `-I` (:551); the allow-list contains no `PYTHONPATH`/`PYTHONHOME`/`PYTHONPYCACHEPREFIX` passthrough. The
|
||||
broker child cannot inherit ambient env or resolve stdlib imports to ambient code. ✅
|
||||
- **B8 — bytecode pinned-or-suppressed: correct.** `PYTHONDONTWRITEBYTECODE=1` is in the allow-list env (:728) and
|
||||
`-B` is on **both** child commands (:512-513 launcher, :551-552 broker); `reject_pinned_bytecode` fails closed
|
||||
(`RuntimeError`) on any pre-existing `pinned/__pycache__` or `*.pyc` (:498-501) and is called **before each
|
||||
consumer** (:534 launcher, :561 broker). No unpinned `.pyc` can be executed. ✅
|
||||
- **B5 conjunction / B6 single-helper / closure-import-guard / BAR1 / BAR2 (`.state` fidelity untouched) / BAR3
|
||||
(live-path, fail-closed precondition, fixture-socket isolation, `lease_anchor_registered` + hex-256) / single p3
|
||||
broker / `-O`-safe / allow-list env / `--runs==(3,)` / ABSENT sweep:** all intact/unperturbed (the delta touches only
|
||||
the env/`-I`/`-B`/bytecode-reject surfaces). These are **not** the failing item.
|
||||
|
||||
## Verdict
|
||||
|
||||
**FAIL @ `a92ad090`.** B7 (broker env isolation) and B8 (bytecode pinned-or-suppressed) are correctly implemented,
|
||||
but the `-I` added to the **launcher** command breaks B6(c): the launcher's bare `from lease_generation import` at
|
||||
`f4008307:launch-runtime.py:15` cannot resolve the pinned helper because `-I`/`-P` strips `sys.path[0]` on Python
|
||||
3.11+ (empirically confirmed, 3.11.2 → `ModuleNotFoundError`). PASS requires **all** of B7+B8+B5+B6+BAR1/2/3; B6(c)
|
||||
does not hold. Not softened → returns to author (ms-rev-826). The fix is narrow: drop `-I` from the launcher command
|
||||
(retain `-B` + allow-list env), keep `-I` on the broker.
|
||||
|
||||
**Findings:** B6(c) — `p3_d4_focused_run.py:512` (`-I` on the launcher command; breaks the pinned-helper sibling
|
||||
import under Python 3.11+).
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; FIRE remains
|
||||
separately gated on Mos's post-clear GO — moot until this FAIL is remediated.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** distinct Opus SECREV session (`ms-secrev-828` lane, dispatched by `mosaic-100`), Gate-16 author≠reviewer,
|
||||
byte review only; ran nothing (harness/broker not executed).
|
||||
**Reviewed object (pin):** `a92ad090ae3828c643f961c7628d809b8521185f` · harness sha256 `915ebeb5aeab108cb60c5f629c1db520623ab4914eed427ca34ee66f9aa08390`.
|
||||
@@ -0,0 +1,95 @@
|
||||
> ⚠ SUPERSEDED: homelab 4th-round FAIL @`1c34e3cb` — no `-S` → Python imports `site` at startup, running unpinned system-site `.pth` executable lines + `sitecustomize`/`usercustomize` before the pinned launcher/broker (site startup-closure hole). This v9 PASS record is overtaken by the stricter B9 (no-site) bar and is NOT fire-authorization; superseded by v10.
|
||||
|
||||
# Gate0 Probe-3 (D4) Launcher-Import Fix — §3-Conformance Review v9
|
||||
|
||||
**Verdict: ✅ PASS** *(superseded — see banner above)*
|
||||
|
||||
## Pin (GUARD 1 — reviewed object)
|
||||
|
||||
- **Reviewed object = `1c34e3cb3172acdcd094e683e847d7c984afc96c`** (harness commit, branch `feat/827-gate0-probe`).
|
||||
- **Reviewed file:** `docs/compaction-refresh/probes/p3_d4_focused_run.py` — sha256 (pushed provider bytes, `-o FILE`, FULL-40 ref, verified before trust): `29e5c7bfbe1911b52984bd94c79036bb1200ee82588318367b13c2b1053a0103` (32725 bytes, no not-found sentinel).
|
||||
- **Closure pins (unchanged):** launcher `e950e422…` @f4008307 · helper `061625402f08488eac47acd23272904e71fd1a71fd15b3bdab158632c801be4c` @f4008307 · broker `4db4fef1…` @23c0caca.
|
||||
|
||||
## Independence (GUARD 2 — principal-independence attestation)
|
||||
|
||||
Distinct Opus SECREV session, orchestrator-dispatched — the **`ms-secrev-828` reviewer lane, dispatched by
|
||||
`mosaic-100`** — **byte review only, ran nothing** (harness/broker not executed); did **not** build this harness
|
||||
(builder = ms-rev-826); is not Mos. This is the re-review after **my own** v8 FAIL @`a92ad090` (B6(c): `-I` on the
|
||||
launcher broke the sibling import); the author applied the narrow fix and I verify it here. The `-s`/`-I` `sys.path`
|
||||
checks below use a *throwaway* two-line script to observe interpreter behavior — not the harness/broker/closure.
|
||||
|
||||
## ★ B6(c) — THE FIX (was the v8 FAIL): launcher `-I` dropped; sibling import binds to `pinned/`
|
||||
|
||||
The launcher command no longer carries `-I`; it now uses **`-s`** (`:514`, commented "`-s` preserves `sys.path[0]=pinned/`
|
||||
for the launcher's sibling helper") + `-B` (`:515`), and `PYTHONNOUSERSITE=1` is added to the allow-list env (`:730`).
|
||||
|
||||
`-s` and `PYTHONNOUSERSITE` disable **user site-packages only** — they do **not** strip the script's directory from
|
||||
`sys.path` (unlike `-I`/`-P`). Empirically confirmed on this host (Python 3.11.2), throwaway script:
|
||||
|
||||
```
|
||||
python3 -s -B main.py → sys.path[0] = '<script dir>' → sibling import: OK
|
||||
PYTHONNOUSERSITE=1 python3 -s -B main.py → sys.path[0] = '<script dir>' → sibling import: OK
|
||||
python3 -I -B main.py (the v8 FAIL form) → sys.path[0] = stdlib zip → ModuleNotFoundError
|
||||
```
|
||||
|
||||
Therefore `python3 -s -B pinned/launch-runtime.py …` puts `pinned/` at `sys.path[0]`, so the pinned launcher's bare
|
||||
top-level `from lease_generation import initialize_runtime_generation` (`f4008307:launch-runtime.py:15`, no `sys.path`
|
||||
manipulation) resolves to the **pinned** `pinned/lease_generation.py` — not the worktree, not a miss. **B6(c) holds.**
|
||||
|
||||
## B7 — Broker env-isolation (still holds)
|
||||
|
||||
`launch_verified_broker` passes `env=environment` (the constructed allow-list, **not** `os.environ`; contains no
|
||||
`PYTHONPATH`/`PYTHONHOME`/`PYTHONPYCACHEPREFIX`) to `Popen` (`:570`), and the broker command includes `-I` (`:552`).
|
||||
The broker imports the helper by explicit `--generation-module` path via `importlib`, so it never needs `sys.path[0]`
|
||||
— `-I` is correct there and does not affect it. (The env's `PYTHONNOUSERSITE`/`PYTHONDONTWRITEBYTECODE` are hardening
|
||||
flags, not path/home passthrough, and `-I` ignores all `PYTHON*` env anyway.)
|
||||
|
||||
## B8 — Bytecode pinned-or-suppressed (still holds)
|
||||
|
||||
`PYTHONDONTWRITEBYTECODE=1` (`:729`) and `PYTHONNOUSERSITE=1` (`:730`) in the allow-list env; `-B` on **both** child
|
||||
commands (`:515` launcher, `:553` broker); `reject_pinned_bytecode` fails closed (`RuntimeError`) on any pre-existing
|
||||
`pinned/__pycache__` or `*.pyc` (`:498-501`) and is called **before each consumer** (`:535` launcher, `:562` broker).
|
||||
No unpinned `.pyc` can be executed.
|
||||
|
||||
## B5 — 3-leg conjunction (still holds)
|
||||
|
||||
`materialize_closure` reads launcher+helper+broker from **git-object bytes** (`git show <commit>:<path>`) and asserts
|
||||
`sha256 == pin` for each, fail-closed; `pinned/` is a fixture-private `0o700` dir inside the per-run fixture temp root;
|
||||
files created `O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC 0o600` (no chmod/rename/symlink gap → no writable window); each member
|
||||
re-hashed `== pin` immediately before its `Popen` (launcher; broker + helper). Launcher and broker consume the **same
|
||||
single** pinned helper. `closure_import_guard` AST-rejects any unpinned non-stdlib import.
|
||||
|
||||
## Fidelity + traceability + carry-over (still hold)
|
||||
|
||||
`GATED_WI_HEAD == f4008307` + `merge-base --is-ancestor 66b1e0a0 f4008307` (forward-contains). Extension bumps
|
||||
`generation-{sid}.state` via `MOSAIC_LEASE_GENERATION_FILE` (not in-mem); broker reads via `read_runtime_generation`;
|
||||
`assert_d4` observes the file-backed transition (`generation_source=="state-file"`, `state_file_drives_lifecycle`,
|
||||
`state_file_in_fixture_root`, new→`MUTATOR_UNVERIFIED`, prior→`STALE_GENERATION`); `.state` stays in the fixture temp
|
||||
root. `lease_anchor_registered` INTACT (event + `session_id_shape=="hex-256"`). LIVE-PATH drives the pinned gated
|
||||
launcher (no released `mosaic`/`execRuntime`). Fail-closed precondition before any launch. Single pinned p3 broker.
|
||||
Promotion=fixture-only. `--runs choices=(3,)`. `copy2` = creds-only. Allow-list env (0 `os.environ.copy`).
|
||||
|
||||
## ABSENT sweep
|
||||
|
||||
P5/P6/P2-bank/retry-launder/mosaic-yolo/execRuntime = 0; `source-invalid` = 0; no live/real-broker path; no `.state`
|
||||
outside the fixture root; no exec-at-import (`__main__`-guarded); no adjacency-only-exec-from-worktree; the only change
|
||||
is the authorized launcher-flag isolation fix (no mechanism change, no scope-widen); **`-O`-safe** (0 bare `assert`).
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS @ `1c34e3cb`.** The v8 FAIL is remediated by the narrow fix (launcher `-I` → `-s` + `PYTHONNOUSERSITE=1`),
|
||||
empirically verified to preserve `sys.path[0]=pinned/` so the pinned launcher's sibling import binds to the pinned
|
||||
helper; the broker retains `-I` (explicit-path import). B7, B8, B5, B6-rest, fidelity, traceability, and all carry-over
|
||||
bars remain intact; zero out-of-scope surface. **Findings: none.**
|
||||
|
||||
## Scope reminder (not a finding)
|
||||
|
||||
Producing this evidence **executes** the Gate0 mechanism (§3/§5). This review clears **bytes** only; **FIRE remains
|
||||
separately gated on Mos's explicit post-clear GO**, his byte-scope-verify co-attestation, and homelab's
|
||||
third-principal verify — not authorized by this review.
|
||||
|
||||
---
|
||||
|
||||
**Reviewer:** distinct Opus SECREV session (`ms-secrev-828` lane, dispatched by `mosaic-100`), Gate-16 author≠reviewer,
|
||||
byte review only; ran nothing.
|
||||
**Reviewed object (pin):** `1c34e3cb3172acdcd094e683e847d7c984afc96c` · harness sha256 `29e5c7bfbe1911b52984bd94c79036bb1200ee82588318367b13c2b1053a0103`.
|
||||
290
docs/design/791-upgrade-config-protection.md
Normal file
290
docs/design/791-upgrade-config-protection.md
Normal file
@@ -0,0 +1,290 @@
|
||||
# Design — #791: Framework upgrades must not destroy operator-owned config under `~/.config/mosaic`
|
||||
|
||||
- **Issue:** mosaicstack/stack#791
|
||||
- **Branch:** `feat/791-upgrade-config-protection` (off `origin/main` `9745bc3f`)
|
||||
- **Author:** ms-791 worker lane
|
||||
- **Status:** Phase 1 — DESIGN, awaiting MS-LEAD confirmation before implementation
|
||||
- **Ratified scope (Mos-approved, not re-litigated):** deliver **(b) strict ownership separation [PRIMARY]** + **(a) transactional pre-update snapshot [safety net]** + **(d) regeneration-from-SSOT [recovery]**. **(c) periodic backup timer is DEFERRED** — noted as future work only.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current updater behavior + exact wipe mechanism (evidence)
|
||||
|
||||
### 1.1 What runs on `mosaic update`
|
||||
|
||||
`mosaic update` re-seeds the framework by invoking the **bash installer** in sync-only, keep mode:
|
||||
|
||||
- `packages/mosaic/src/runtime/update-checker.ts:509` `buildReseedCommand()` returns
|
||||
`bash <frameworkRoot>/install.sh` with env `MOSAIC_SYNC_ONLY=1`, `MOSAIC_INSTALL_MODE=keep`,
|
||||
`MOSAIC_HOME=<mosaicHome>`.
|
||||
- The same `install.sh` is the direct/`tools/install.sh` upgrade path and the framework-vN migration path.
|
||||
|
||||
So the destructive surface is **`packages/mosaic/framework/install.sh`**.
|
||||
|
||||
### 1.2 The wipe
|
||||
|
||||
`sync_framework()` (`install.sh:177`) performs, in `keep` mode:
|
||||
|
||||
```
|
||||
rsync -a --delete --exclude .git --exclude .framework-version --exclude '*.pre-constitution.bak' \
|
||||
[--exclude "/$path" for each PRESERVE_PATHS entry] SOURCE_DIR/ TARGET_DIR/
|
||||
```
|
||||
|
||||
- `install.sh:199` — `rsync -a --delete`. **`--delete` prunes every path in `~/.config/mosaic`
|
||||
that is NOT present in the shipped framework source**, unless excluded.
|
||||
- `install.sh:47` — `PRESERVE_PATHS` is the **only** thing standing between `--delete` and operator
|
||||
data. It is a _denylist of exclusions_:
|
||||
```
|
||||
PRESERVE_PATHS=("CONSTITUTION.md" "AGENTS.md" "SOUL.md" "USER.md" "TOOLS.md" "STANDARDS.md"
|
||||
"memory" "sources" "credentials" "fleet/roster.yaml" "fleet/roster.json" "fleet/agents"
|
||||
"fleet/run" "fleet/backlog" "fleet/roles.local")
|
||||
```
|
||||
- The cp-fallback (no rsync) is equally destructive: `install.sh:223`
|
||||
`find "$TARGET_DIR" -mindepth 1 -maxdepth 1 ... -exec rm -rf {} +` then re-copies source, restoring
|
||||
only PRESERVE_PATHS globs.
|
||||
|
||||
**Root-cause model:** _"Everything under `~/.config/mosaic` is framework-owned and pruneable UNLESS
|
||||
explicitly preserved."_ Any operator path the list forgets is destroyed on the next upgrade.
|
||||
|
||||
### 1.3 The exact operator paths wiped
|
||||
|
||||
Cross-referencing the issue's operator-owned list against `PRESERVE_PATHS`:
|
||||
|
||||
| Operator path (issue #791) | In PRESERVE_PATHS? | Fate on `mosaic update` |
|
||||
| ----------------------------------------------------------------- | --------------------------------------- | ----------------------- |
|
||||
| `agents/*.conf` (per-agent runtime) | **NO** | **WIPED** |
|
||||
| `policy/*.md` (operator overlays) | **NO** | **WIPED** |
|
||||
| `*.local.md` (SOUL/USER/STANDARDS) | **NO** | **WIPED** |
|
||||
| harvester / SOP artifacts + timers | **NO** | **WIPED** |
|
||||
| `tools/_lib/credentials.json` | **NO** (`credentials/` dir ≠ this path) | **WIPED** |
|
||||
| `fleet/agents/*.env` | yes (`fleet/agents`, added by #631) | survives |
|
||||
| `memory/`, `fleet/roster.*`, `fleet/backlog`, `fleet/roles.local` | yes | survives |
|
||||
|
||||
The `fleet/agents`, `memory`, `fleet/backlog` entries were **retro-added after prior incidents**
|
||||
(#631). This whack-a-mole is the structural signature of a denylist.
|
||||
|
||||
**Stale-comment evidence:** `update-checker.ts:492` claims the reseed preserves
|
||||
"`SOUL/USER/*.local/credentials`" — but `PRESERVE_PATHS` contains **no `*.local` entry**. The code
|
||||
documents protection it does not deliver.
|
||||
|
||||
### 1.4 Second code path (TS) — already non-destructive, but drifted
|
||||
|
||||
`FileConfigAdapter.syncFramework()` (`packages/mosaic/src/config/file-adapter.ts:157`) →
|
||||
`syncDirectory()` (`packages/mosaic/src/platform/file-ops.ts:66`) is a **copy-overlay**: it copies
|
||||
source over target and skips preserved paths, but **never deletes** target paths absent from source
|
||||
(`file-ops.ts:77-109`). It is used by the wizard/init flow, not `mosaic update`.
|
||||
|
||||
Two problems remain:
|
||||
|
||||
1. Its `preservePaths` (`file-adapter.ts:164-185`) has **already diverged** from `install.sh` — it is
|
||||
**missing `fleet/backlog` and `fleet/roles.local`**. Two hand-maintained denylists, drifted. This
|
||||
is direct evidence for a single shared SSOT manifest.
|
||||
2. Even non-destructive, it will happily _overwrite_ an operator file that collides with a
|
||||
framework-shipped path unless that path is on its (incomplete) preserve list.
|
||||
|
||||
### 1.5 Existing snapshot is inadequate for rollback
|
||||
|
||||
`make_snapshot()`/`restore_snapshot()` (`install.sh:76-87`) copy `TARGET_DIR` to `mktemp -d` under
|
||||
`/tmp`, restore **only on `ERR/INT/TERM` trap**, and are **deleted on success** (`cleanup_snapshot`,
|
||||
`install.sh:345`). Consequences: ephemeral `/tmp`, no retention, no post-success rollback, and **no
|
||||
`mosaic restore`**. It is crash-safety only, not the transactional safety net #791 requires.
|
||||
|
||||
---
|
||||
|
||||
## 2. Fix (b) — Strict ownership separation [PRIMARY / root cause]
|
||||
|
||||
### 2.1 Ownership model (invert to allow-list)
|
||||
|
||||
Replace _"framework-owned unless preserved"_ with _"operator-owned unless framework-owned"_, resolved
|
||||
**per target path** with operator carve-outs winning inside shared framework subtrees.
|
||||
|
||||
Two declared lists, one SSOT data file shipped in the framework
|
||||
(`framework/framework-manifest.json`), consumed by **both** bash and TS:
|
||||
|
||||
- **`framework` globs** — paths the updater is entitled to create / overwrite / prune. Authored to
|
||||
match exactly what the framework ships in `packages/mosaic/framework/` (e.g. `CONSTITUTION.md`,
|
||||
`AGENTS.md`, `STANDARDS.md`, `TOOLS.md`, `guides/**`, `constitution/**`, `templates/**`, `tools/**`,
|
||||
`skills/**`, `mcp/**`, `defaults/**`, `fleet/examples/**`, `fleet/roles/**`, `fleet/profiles/**`,
|
||||
`fleet/roster.schema.json`).
|
||||
- **`operatorReserved` globs** — NEVER written or pruned, even nested inside a `framework` subtree;
|
||||
these **win** over `framework` (deny-wins / most-specific-wins). At minimum:
|
||||
`agents/**`, `policy/**`, `memory/**`, `sources/**`, `credentials/**`, `*.local.md`,
|
||||
`tools/_lib/credentials.json`, `fleet/roster.yaml`, `fleet/roster.json`, `fleet/agents/**`,
|
||||
`fleet/run/**`, `fleet/backlog/**`, `fleet/roles.local/**`, plus operator harvester/SOP artifacts.
|
||||
|
||||
### 2.2 Ownership resolution for a target path `P`
|
||||
|
||||
1. `P` matches `operatorReserved` → **operator-owned**: updater MUST NOT write, MUST NOT delete.
|
||||
2. else `P` matches `framework` → **framework-owned**: may overwrite; may prune **only if absent from
|
||||
the current SOURCE** (a genuinely retired framework file).
|
||||
3. else (matches neither) → **UNKNOWN ⇒ operator-owned by default (fail-safe)**: never delete.
|
||||
|
||||
Rule 3 is the actual root-cause fix: an operator path the manifest authors forget is still protected,
|
||||
because _unknown defaults to operator_. A denylist can never provide this guarantee.
|
||||
|
||||
### 2.3 Sync mechanism change (the mechanically-critical part)
|
||||
|
||||
`--delete` cannot express "prune only framework-owned" without re-enumerating every operator path
|
||||
(the denylist trap). So:
|
||||
|
||||
1. **Drop `--delete` from the bulk sync.** Copy `SOURCE → TARGET` non-destructively (writes/overwrites
|
||||
all framework files; deletes nothing). rsync without `--delete`, or the existing overlay copy.
|
||||
2. **Explicit manifest-scoped prune pass.** Iterate the **`framework` manifest** (not the whole tree);
|
||||
for each framework path present in `TARGET` but **absent in `SOURCE`**, delete it — after
|
||||
re-checking it does not match `operatorReserved`. Because the prune iterates only declared
|
||||
framework globs, operator/unknown paths are **structurally unreachable** by deletion.
|
||||
|
||||
This is implemented in both bash `sync_framework()` and TS `syncFramework()` from the shared manifest.
|
||||
A pure **prune-planner** function (TS) computes the delete-set from
|
||||
`(manifest, sourceListing, targetListing)` so the invariant is unit-testable in isolation.
|
||||
`PRESERVE_PATHS` becomes redundant (kept as a defense-in-depth alias mapping to `operatorReserved`, or
|
||||
removed) — either way the two lists stop drifting because they read one file.
|
||||
|
||||
### 2.4 HARD GATE test — "upgrade touches no path outside the manifest"
|
||||
|
||||
Filesystem-observation test in the existing `test-install-migration.sh` harness pattern (mktemp
|
||||
`MOSAIC_HOME`, `MOSAIC_SYNC_ONLY=1`), plus TS specs:
|
||||
|
||||
1. Seed a throwaway `TARGET` with a realistic operator mix — one sentinel per operator class:
|
||||
`agents/x.conf`, `policy/p.md`, `SOUL.local.md`, `memory/m.md`,
|
||||
`tools/_lib/credentials.json` (with a secret value), `fleet/agents/a.env`, `fleet/roster.yaml`,
|
||||
`harvester/sop.md`, **and a deliberately-unanticipated `unknown-operator-dir/x`**.
|
||||
2. Record hash+mtime of every sentinel.
|
||||
3. Run the upgrade from a `SOURCE` containing none of those operator paths.
|
||||
4. **Assert:** every sentinel exists, byte-identical, **mtime unchanged** (not even rewritten). The
|
||||
`unknown-operator-dir` surviving proves the fail-safe default — a denylist could not pass this case.
|
||||
5. **Positive controls:** framework files WERE updated; a retired framework file WAS pruned.
|
||||
6. **Property test** (TS prune-planner): for fuzzed operator paths, `deleteSet ⊆ {matches framework ∧
|
||||
in target ∧ not in source}` and `deleteSet ∩ operatorReserved = ∅`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Fix (a) — Transactional pre-update snapshot [safety net]
|
||||
|
||||
- **Destination:** `${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-ts>/`.
|
||||
**Outside `~/.config/mosaic`** (so no future sync can sweep it) and outside any repo.
|
||||
- **Perms:** dir `0700`, files `0600` — enforced with `umask 077` around the copy **and** explicit
|
||||
`chmod`. Never world-readable.
|
||||
- **Scope:** the operator-owned surface (`operatorReserved` paths that exist) — bounded; does not copy
|
||||
the framework tree.
|
||||
- **Timing:** taken before ANY mutation in the upgrade flow.
|
||||
- **Post-sync verify + selective restore:** after sync, diff the operator surface against the snapshot;
|
||||
since (b) should never touch operator paths, any diff means a manifest bug — restore the affected
|
||||
paths from the snapshot and warn loudly. This is precisely (a) catching a miss in (b).
|
||||
- **Retention:** keep N most-recent (default 5; `MOSAIC_BACKUP_RETENTION` override); prune older.
|
||||
- **`mosaic restore`:** `--list` (default, dry-run) enumerates snapshots by timestamp;
|
||||
`--from <ts>` restores that snapshot over the operator surface, confirmation-gated. Reports
|
||||
counts/paths only.
|
||||
- **Secret-safety:** snapshot copy and restore never emit file **contents**; only paths/counts.
|
||||
Tests assert `0700/0600` and that no secret value appears in stdout/stderr.
|
||||
|
||||
---
|
||||
|
||||
## 4. Fix (d) — Regeneration-from-SSOT [recovery]
|
||||
|
||||
The incident's live blast radius: `fleet/agents/*.env` (systemd `EnvironmentFile` sources) gone →
|
||||
`mosaic-agent@<name>` boots **unit defaults** on restart (because `EnvironmentFile=-...` is
|
||||
absent-tolerant) → **silent identity/runtime/workdir downgrade**.
|
||||
|
||||
The SSOT for those `.env` files is the roster. The reconciler **already** separates a
|
||||
`regenerate-projections-from-roster` projection phase from lifecycle
|
||||
(`packages/mosaic/src/fleet/fleet-reconciler.ts:93,234`; env rendering in
|
||||
`generated-env-boundary.ts:149-264`).
|
||||
|
||||
**`mosaic fleet regen`** is therefore a **thin recovery-framed wrapper over the existing projection
|
||||
phase** — it does NOT reimplement fleet logic and does NOT preempt in-flight FCM cards (M4/M5):
|
||||
|
||||
- Regenerates derivable config (per-agent `*.env.generated`, unit files) from roster SSOT.
|
||||
- **Preview-first:** dry-run default; `--write` to apply. Idempotent.
|
||||
- **Never restarts agents** (the recovery order forbids restart-before-verify).
|
||||
- Prints the runbook's next step (verify `EnvironmentFile` resolves, THEN restart).
|
||||
|
||||
Alternatively documentable as `install.sh --relink` per the issue; `mosaic fleet regen` is preferred
|
||||
because it reuses the merged reconciler plumbing.
|
||||
|
||||
---
|
||||
|
||||
## 5. Secret-safety approach (secrev surface)
|
||||
|
||||
- Snapshots/backups: `0700`/`0600`, outside any repo, never world-readable. (§3)
|
||||
- No secret **value** ever emitted to logs/stdout/stderr by snapshot, restore, sync, or regen —
|
||||
paths/counts only. Adversarial test: a secret value placed in `tools/_lib/credentials.json` must
|
||||
never appear in installer or command output.
|
||||
- `tools/_lib/credentials.json` is an explicit `operatorReserved` carve-out inside the framework-owned
|
||||
`tools/**` subtree — it is never overwritten or pruned.
|
||||
- The HARD GATE test doubles as a secret-safety test (asserts the credentials sentinel is untouched).
|
||||
|
||||
---
|
||||
|
||||
## 6. Test plan (TDD, tests-first, ≥85% on new code, co-located `*.spec.ts`)
|
||||
|
||||
1. **Manifest SSOT parity** — bash and TS resolve identical framework/operator sets from the one file;
|
||||
a test fails if either path hard-codes a divergent list.
|
||||
2. **Manifest completeness** — every path shipped in `framework/` is covered by a `framework` glob (so
|
||||
a new shipped file cannot silently fall outside the manifest and become un-prunable/undeclared).
|
||||
3. **HARD GATE** — upgrade touches nothing outside the manifest, incl. the unanticipated-path case
|
||||
(§2.4).
|
||||
4. **Prune-planner** unit + property tests (§2.4.6).
|
||||
5. **Snapshot** — perms `0700/0600`, correct destination, retention prune, secret value absent from
|
||||
output.
|
||||
6. **Restore** — `--list` / `--from` round-trip restores operator surface byte-exact; confirmation
|
||||
gate; no secret leakage.
|
||||
7. **Regen** — roster→env projection deterministic + idempotent; dry-run makes no writes; `--write`
|
||||
restores `*.env`; **never** issues a lifecycle/restart call.
|
||||
8. **Cross-path regression** — TS `syncFramework` and bash `install.sh` agree on a shared fixture
|
||||
(closes the current #631-style drift).
|
||||
|
||||
Gates before every push: `pnpm typecheck && pnpm lint && pnpm format:check` + mosaic package tests
|
||||
green. Never `--no-verify`.
|
||||
|
||||
---
|
||||
|
||||
## 7. web1 recovery runbook (operator-agnostic; web1 specifics live in the issue as evidence only)
|
||||
|
||||
For a currently-wiped fleet EnvironmentFile state — **do NOT service-restart while
|
||||
`fleet/agents/*.env` is absent** (a restart boots unit defaults and silently downgrades identity):
|
||||
|
||||
1. **Regenerate:** `mosaic fleet regen --write` — rebuild `~/.config/mosaic/fleet/agents/*.env` from
|
||||
roster SSOT.
|
||||
2. **Verify each unit resolves to the intended runtime/workdir** _before_ any restart:
|
||||
`systemctl --user show mosaic-agent@<name> -p EnvironmentFile` and confirm the generated env exists
|
||||
and carries the intended `MOSAIC_AGENT_*` runtime/workdir values.
|
||||
3. **Only then** `systemctl --user restart mosaic-agent@<name>`, one unit at a time.
|
||||
|
||||
If config (not just fleet env) was lost, `mosaic restore --list` → `mosaic restore --from <ts>` before
|
||||
step 1.
|
||||
|
||||
---
|
||||
|
||||
## 8. Proposed PR split (reviewable; DAG-ordered)
|
||||
|
||||
| PR | Scope | Depends | Review focus |
|
||||
| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------------------------- |
|
||||
| PR1 | **PRIMARY** — shared `framework-manifest.json` + ownership resolver + non-deleting sync + scoped prune (bash + TS) + **HARD GATE** + prune-planner tests | — | correctness (root fix) |
|
||||
| PR2 | **Safety net** — pre-update snapshot (`~/.local/state`, 0700/0600, retention) + post-sync verify/restore + `mosaic restore` | PR1 | **secrev** (backup/secret) |
|
||||
| PR3 | **Recovery** — `mosaic fleet regen` (projection-only, preview-first, no restart) + docs (upgrade-safety + recovery runbook) | PR1 | correctness + docs |
|
||||
|
||||
Rationale: PR1 closes the failure class on its own; if PR2/PR3 slip, the class stays fixed. Each PR is
|
||||
one reviewable unit with its own tests ≥85%. Independent review (author≠reviewer) on all; **secrev** on
|
||||
PR2 (and PR1's secret-sentinel assertions).
|
||||
|
||||
## 9. Deferred (noted per scope)
|
||||
|
||||
**(c) periodic backup timer** — a systemd user timer snapshotting operator dirs on a cadence
|
||||
(defense-in-depth for non-upgrade losses). Explicitly **out of scope now**; future phase.
|
||||
|
||||
## 10. Constraints honored
|
||||
|
||||
- **Framework-PR firewall:** manifest + logic are operator-agnostic; no SOUL/USER/operator specifics
|
||||
in framework code; web1 details are issue evidence only.
|
||||
- **Capacity-fill:** must not preempt M5-001 or #790; `fleet regen` reuses merged FCM-M3 plumbing and
|
||||
does not overlap FCM-M4/M5 migration cards.
|
||||
- **Delivery gates:** TDD tests-first, ≥85% new-code coverage, trunk-based squash PRs, independent
|
||||
review + secrev, completion = merged PR + descendant-main green + #791 closed.
|
||||
|
||||
---
|
||||
|
||||
**Requesting MS-LEAD confirmation of:** (1) the manifest allow-list + non-deleting-sync + scoped-prune
|
||||
approach as the (b) root-cause fix; (2) snapshot destination/retention + `mosaic restore` UX;
|
||||
(3) `mosaic fleet regen` as a projection-only wrapper; (4) the 3-PR split. Implementation begins only
|
||||
on your confirmation.
|
||||
@@ -70,6 +70,10 @@ export function createQueue(config?: QueueConfig): QueueHandle {
|
||||
|
||||
### `@mosaicstack/db` (packages/db/src/client.ts)
|
||||
|
||||
> **Historical design specimen — status-only, not an operator instruction.** KBN-101 supersedes
|
||||
> this pre-split `DATABASE_URL` fallback shape; it cannot authorize runtime migration, DDL, or a
|
||||
> connection-string fallback. See the KBN-101 runner/role contract for the produced interface.
|
||||
|
||||
```typescript
|
||||
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';
|
||||
import postgres from 'postgres';
|
||||
|
||||
@@ -54,7 +54,7 @@ Every milestone adds tests to these layers. A milestone cannot be claimed comple
|
||||
- Add `"tier": "federated"` to `mosaic.config.json` schema and validators
|
||||
- Docker Compose `federated` profile (`docker-compose.federated.yml`) adds: Postgres+pgvector (5433), Valkey (6380), dedicated volumes
|
||||
- Tier detector in gateway bootstrap: reads config, asserts required services reachable, refuses to start otherwise
|
||||
- `pgvector` extension installed + verified on startup
|
||||
- **Historical/status only:** the prior startup-provisioning statement is superseded. Runtime/startup extension provisioning is forbidden. PostgreSQL activation remains non-operative with no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land; this record authorizes no current DDL, Compose/init, or startup path.
|
||||
- Migration logic: safe upgrade path from `local`/`standalone` → `federated` (data export/import script, one-way)
|
||||
- `mosaic doctor` reports tier + service health
|
||||
- Gateway continues to serve as a normal standalone instance (no federation yet)
|
||||
|
||||
@@ -1,280 +1,74 @@
|
||||
# Federated Tier Setup Guide
|
||||
|
||||
## What is the federated tier?
|
||||
> **KBN-101 N-1 hold:** This page is **non-operative** and grants no current command
|
||||
> authority until KBN-101-00, KBN-101-03, and KBN-101-05 land and KBN-101-08 activates a
|
||||
> reviewed release. It does not authorize a deployment operation, initialization artifacts,
|
||||
> implicit extension/schema/migration creation, raw `CREATE`, direct database initialization, or
|
||||
> a Gateway against an unverified database. The prior direct-start wording is retired; its
|
||||
> regression fixture is owned by KBN-101-06.
|
||||
|
||||
The federated tier is designed for multi-user and multi-host deployments. It consists of PostgreSQL 17 with pgvector extension (for embeddings and RAG), Valkey for distributed task queueing and caching, and a shared configuration across multiple Mosaic gateway instances. Use this tier when running Mosaic in production or when scaling beyond a single-host deployment.
|
||||
## Held future procedure
|
||||
|
||||
## Prerequisites
|
||||
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- Ports 5433 (PostgreSQL) and 6380 (Valkey) available on your host (or adjust environment variables)
|
||||
- At least 2 GB free disk space for data volumes
|
||||
The deployment control plane—not an operator shell or deployment lifecycle hook—performs this
|
||||
exact held future sequence after activation authorization: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
## Start the federated stack
|
||||
1. External bootstrap provisions the approved database/extension prerequisites.
|
||||
2. TLS/roles are installed through the generation-pinned renderer.
|
||||
3. The dedicated one-shot runner executes `mosaic-db-migrator --run`.
|
||||
4. The same runner executes `mosaic-db-migrator --verify`, including readiness and the
|
||||
importer-target attestation where that route is enabled.
|
||||
5. Only after successful verification may Gateway reach its independent verified-TLS Gateway
|
||||
readiness gate.
|
||||
|
||||
Run the federated overlay:
|
||||
No step may be reordered, skipped, replaced by a raw SQL command, or delegated to an initialization
|
||||
hook.
|
||||
A missing extension, schema, migration, role, secret generation, or readiness proof is a failed
|
||||
control-plane precondition; it is not an instruction to start Compose, retry startup, or create
|
||||
anything directly.
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
||||
```
|
||||
## N-1 status and required disposition
|
||||
|
||||
This starts PostgreSQL 17 with pgvector and Valkey 8. The pgvector extension is created automatically on first boot.
|
||||
The current branch retains historical federation artifacts, but they are not a deployable
|
||||
procedure. `docs/federation/TASKS.md` records their shipped status only. KBN-101-02 retires
|
||||
runtime/init DDL; KBN-101-05 owns the renderer/deployment handoff; KBN-101-06 verifies the
|
||||
finite scanner and command matrix; and KBN-101-07 owns this operator route. A path named in an
|
||||
inventory, a historical-status label, or a normative requirement cannot suppress the semantic
|
||||
checks above.
|
||||
|
||||
Verify the services are running:
|
||||
Until the activation certificate names an exact release, use no database startup or recovery
|
||||
command from this document. For the produced importer interface, see
|
||||
[the federated tier migration contract](../guides/migrate-tier.md); it is likewise non-operative
|
||||
until activation.
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.federated.yml ps
|
||||
```
|
||||
## Federation and Step-CA reference
|
||||
|
||||
Expected output shows `postgres-federated` and `valkey-federated` both healthy.
|
||||
|
||||
## Configure mosaic for federated tier
|
||||
|
||||
Create or update your `mosaic.config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"tier": "federated",
|
||||
"database": "postgresql://mosaic:mosaic@localhost:5433/mosaic",
|
||||
"queue": "redis://localhost:6380"
|
||||
}
|
||||
```
|
||||
|
||||
If you're using environment variables instead:
|
||||
|
||||
```bash
|
||||
export DATABASE_URL="postgresql://mosaic:mosaic@localhost:5433/mosaic"
|
||||
export REDIS_URL="redis://localhost:6380"
|
||||
```
|
||||
|
||||
## Verify health
|
||||
|
||||
Run the health check:
|
||||
|
||||
```bash
|
||||
mosaic gateway doctor
|
||||
```
|
||||
|
||||
Expected output (green):
|
||||
|
||||
```
|
||||
Tier: federated Config: mosaic.config.json
|
||||
✓ postgres localhost:5433 (42ms)
|
||||
✓ valkey localhost:6380 (8ms)
|
||||
✓ pgvector (embedded) (15ms)
|
||||
```
|
||||
|
||||
For JSON output (useful in CI/automation):
|
||||
|
||||
```bash
|
||||
mosaic gateway doctor --json
|
||||
```
|
||||
|
||||
## Step 2: Step-CA Bootstrap
|
||||
|
||||
Step-CA is a certificate authority that issues X.509 certificates for federation peers. In Mosaic federation, it signs peer certificates with custom OIDs that embed grant and user identities, enforcing authorization at the certificate level.
|
||||
|
||||
### Prerequisites for Step-CA
|
||||
|
||||
Before starting the CA, you must set up the dev password:
|
||||
|
||||
```bash
|
||||
cp infra/step-ca/dev-password.example infra/step-ca/dev-password
|
||||
# Edit dev-password and set your CA password (minimum 16 characters)
|
||||
```
|
||||
|
||||
The password is required for the CA to boot and derive the provisioner key used by the gateway.
|
||||
|
||||
### Start the Step-CA service
|
||||
|
||||
Add the step-ca service to your federated stack:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.federated.yml --profile federated up -d step-ca
|
||||
```
|
||||
|
||||
On first boot, the init script (`infra/step-ca/init.sh`) runs automatically. It:
|
||||
|
||||
- Generates the CA root key and certificate in the Docker volume
|
||||
- Creates the `mosaic-fed` JWK provisioner
|
||||
- Applies the X.509 template from `infra/step-ca/templates/federation.tpl`
|
||||
|
||||
The volume is persistent, so subsequent boots reuse the existing CA keys.
|
||||
|
||||
Verify the CA is healthy:
|
||||
|
||||
```bash
|
||||
curl https://localhost:9000/health --cacert /tmp/step-ca-root.crt
|
||||
```
|
||||
|
||||
(If the root cert file doesn't exist yet, see the extraction steps below.)
|
||||
|
||||
### Extract credentials for the gateway
|
||||
|
||||
The gateway requires two credentials from the running CA:
|
||||
|
||||
**1. Provisioner key (for `STEP_CA_PROVISIONER_KEY_JSON`)**
|
||||
|
||||
```bash
|
||||
docker exec $(docker ps -qf name=step-ca) cat /home/step/secrets/mosaic-fed.json > /tmp/step-ca-provisioner.json
|
||||
```
|
||||
|
||||
This JSON file contains the JWK public and private keys for the `mosaic-fed` provisioner. Store it securely and pass its contents to the gateway via the `STEP_CA_PROVISIONER_KEY_JSON` environment variable.
|
||||
|
||||
**2. Root certificate (for `STEP_CA_ROOT_CERT_PATH`)**
|
||||
|
||||
```bash
|
||||
docker cp $(docker ps -qf name=step-ca):/home/step/certs/root_ca.crt /tmp/step-ca-root.crt
|
||||
```
|
||||
|
||||
This PEM file is the CA's root certificate, used to verify peer certificates issued by step-ca. Pass its path to the gateway via `STEP_CA_ROOT_CERT_PATH`.
|
||||
|
||||
### Custom OID Registry
|
||||
|
||||
Federation certificates include custom OIDs in the certificate extension. These encode authorization metadata:
|
||||
Federation uses PostgreSQL 17 with pgvector, Valkey, and a shared configuration across multiple
|
||||
Gateway instances. Step-CA issues federation peer X.509 certificates whose custom OIDs carry a
|
||||
grant and subject identity. The following facts are reference material only; provisioning and
|
||||
secret delivery remain deployment-control-plane work under the activation sequence.
|
||||
|
||||
| OID | Name | Description |
|
||||
| ------------------- | ---------------------- | --------------------- |
|
||||
| 1.3.6.1.4.1.99999.1 | mosaic_grant_id | Federation grant UUID |
|
||||
| 1.3.6.1.4.1.99999.2 | mosaic_subject_user_id | Subject user UUID |
|
||||
| ------------------- | ------------------------ | --------------------- |
|
||||
| 1.3.6.1.4.1.99999.1 | `mosaic_grant_id` | Federation grant UUID |
|
||||
| 1.3.6.1.4.1.99999.2 | `mosaic_subject_user_id` | Subject user UUID |
|
||||
|
||||
These OIDs are verified by the gateway after the CSR is signed, ensuring the certificate was issued with the correct grant and user context.
|
||||
The internal arc `1.3.6.1.4.1.99999` is development-only. Before an externally reachable
|
||||
production deployment, register an IANA Private Enterprise Number and version the assignments.
|
||||
Each value is DER-encoded as an ASN.1 UTF8String containing the UUID.
|
||||
|
||||
### Environment Variables
|
||||
The future activated Gateway requires `STEP_CA_URL`, `STEP_CA_PROVISIONER_PASSWORD`,
|
||||
`STEP_CA_PROVISIONER_KEY_JSON`, `STEP_CA_ROOT_CERT_PATH`, and `BETTER_AUTH_SECRET` through the
|
||||
reviewed secret mechanism. These names do not authorize shell exports, copied credential files,
|
||||
or an ad hoc service start.
|
||||
|
||||
Configure the gateway with the following environment variables before startup:
|
||||
## Failure disposition
|
||||
|
||||
| Variable | Required | Description |
|
||||
| ------------------------------ | -------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| `STEP_CA_URL` | Yes | Base URL of the step-ca instance, e.g. `https://step-ca:9000` (use `https://localhost:9000` in local dev) |
|
||||
| `STEP_CA_PROVISIONER_KEY_JSON` | Yes | JSON-encoded JWK from `/home/step/secrets/mosaic-fed.json` |
|
||||
| `STEP_CA_ROOT_CERT_PATH` | Yes | Absolute path to the root CA certificate (e.g. `/tmp/step-ca-root.crt`) |
|
||||
| `BETTER_AUTH_SECRET` | Yes | Secret used to seal peer private keys at rest; already required for M1 |
|
||||
|
||||
Example environment setup:
|
||||
|
||||
```bash
|
||||
export STEP_CA_URL="https://localhost:9000"
|
||||
export STEP_CA_PROVISIONER_KEY_JSON="$(cat /tmp/step-ca-provisioner.json)"
|
||||
export STEP_CA_ROOT_CERT_PATH="/tmp/step-ca-root.crt"
|
||||
export BETTER_AUTH_SECRET="<your-secret>"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Port conflicts
|
||||
|
||||
**Symptom:** `bind: address already in use`
|
||||
|
||||
**Fix:** Stop the base dev stack first:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
||||
```
|
||||
|
||||
Or change the host port with an environment variable:
|
||||
|
||||
```bash
|
||||
PG_FEDERATED_HOST_PORT=5434 VALKEY_FEDERATED_HOST_PORT=6381 \
|
||||
docker compose -f docker-compose.federated.yml --profile federated up -d
|
||||
```
|
||||
|
||||
### pgvector extension error
|
||||
|
||||
**Symptom:** `ERROR: could not open extension control file`
|
||||
|
||||
**Fix:** pgvector is created at first boot. Check logs:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.federated.yml logs postgres-federated | grep -i vector
|
||||
```
|
||||
|
||||
If missing, exec into the container and create it manually:
|
||||
|
||||
```bash
|
||||
docker exec <postgres-federated-id> psql -U mosaic -d mosaic -c "CREATE EXTENSION vector;"
|
||||
```
|
||||
|
||||
### Valkey connection refused
|
||||
|
||||
**Symptom:** `Error: connect ECONNREFUSED 127.0.0.1:6380`
|
||||
|
||||
**Fix:** Check service health:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.federated.yml logs valkey-federated
|
||||
```
|
||||
|
||||
If Valkey is running, verify your firewall allows 6380. On macOS, Docker Desktop may require binding to `host.docker.internal` instead of `localhost`.
|
||||
|
||||
## Key rotation (deferred)
|
||||
|
||||
Federation peer private keys (`federation_peers.client_key_pem`) are sealed at rest using AES-256-GCM with a key derived from `BETTER_AUTH_SECRET` via SHA-256. If `BETTER_AUTH_SECRET` is rotated, all sealed `client_key_pem` values in the database become unreadable and must be re-sealed with the new key before rotation completes.
|
||||
|
||||
The full key rotation procedure (decrypt all rows with old key, re-encrypt with new key, atomically swap the secret) is out of scope for M2. Operators must not rotate `BETTER_AUTH_SECRET` without a migration plan for all sealed federation peer keys.
|
||||
|
||||
## OID Assignments — Mosaic Internal OID Arc
|
||||
|
||||
Mosaic uses the private enterprise arc `1.3.6.1.4.1.99999` for custom X.509
|
||||
certificate extensions in federation grant certificates.
|
||||
|
||||
**IMPORTANT:** This is a development/internal OID arc. Before deploying to a
|
||||
production environment accessible by external parties, register a proper IANA
|
||||
Private Enterprise Number (PEN) at <https://pen.iana.org/pen/PenApplication.page>
|
||||
and update these assignments accordingly.
|
||||
|
||||
### Assigned OIDs
|
||||
|
||||
| OID | Symbolic name | Description |
|
||||
| --------------------- | --------------------------------- | --------------------------------------------------------- |
|
||||
| `1.3.6.1.4.1.99999.1` | `mosaic.federation.grantId` | UUID of the `federation_grants` row authorising this cert |
|
||||
| `1.3.6.1.4.1.99999.2` | `mosaic.federation.subjectUserId` | UUID of the local user on whose behalf the cert is issued |
|
||||
|
||||
### Encoding
|
||||
|
||||
Each extension value is DER-encoded as an ASN.1 **UTF8String**:
|
||||
|
||||
```
|
||||
Tag 0x0C (UTF8String)
|
||||
Length 0x24 (36 decimal — fixed length of a UUID string)
|
||||
Value <36 ASCII bytes of the UUID>
|
||||
```
|
||||
|
||||
The step-ca X.509 template at `infra/step-ca/templates/federation.tpl`
|
||||
produces this encoding via the Go template expression:
|
||||
|
||||
```
|
||||
{{ printf "\x0c\x24%s" .Token.mosaic_grant_id | b64enc }}
|
||||
```
|
||||
|
||||
The resulting base64 value is passed as the `value` field of the extension
|
||||
object in the template JSON.
|
||||
|
||||
### CA Environment Variables
|
||||
|
||||
The `CaService` (`apps/gateway/src/federation/ca.service.ts`) requires the
|
||||
following environment variables at gateway startup:
|
||||
|
||||
| Variable | Required | Description |
|
||||
| ------------------------------ | -------- | -------------------------------------------------------------------- |
|
||||
| `STEP_CA_URL` | Yes | Base URL of the step-ca instance, e.g. `https://step-ca:9000` |
|
||||
| `STEP_CA_PROVISIONER_PASSWORD` | Yes | JWK provisioner password for the `mosaic-fed` provisioner |
|
||||
| `STEP_CA_PROVISIONER_KEY_JSON` | Yes | JSON-encoded JWK (public + private) for the `mosaic-fed` provisioner |
|
||||
| `STEP_CA_ROOT_CERT_PATH` | Yes | Absolute path to the step-ca root CA certificate PEM file |
|
||||
|
||||
Set these variables in your environment or secret manager before starting
|
||||
the gateway. In the federated Docker Compose stack they are expected to be
|
||||
injected via Docker secrets and environment variable overrides.
|
||||
|
||||
### Fail-loud contract
|
||||
|
||||
The CA service (and the X.509 template) are designed to fail loudly if the
|
||||
custom OIDs cannot be embedded:
|
||||
|
||||
- The template produces a malformed extension value (zero-length UTF8String
|
||||
body) when the JWT claims `mosaic_grant_id` or `mosaic_subject_user_id` are
|
||||
absent. step-ca rejects the CSR rather than issuing a cert without the OIDs.
|
||||
- `CaService.issueCert()` throws a `CaServiceError` on every error path with
|
||||
a human-readable `remediation` string. It never silently returns a cert that
|
||||
may be missing the required extensions.
|
||||
- A TLS, CA, SAN, role, runner, or readiness failure is a control-plane incident. Preserve only
|
||||
sanitized evidence and follow the approved rollback/repair record.
|
||||
- A pgvector/extension failure is a failed external-bootstrap or runner precondition. Do not use
|
||||
direct extension SQL, init artifacts, or a startup retry as remediation.
|
||||
- A port, container, or Valkey problem does not permit bypassing the activation sequence.
|
||||
- Federation peer-key rotation remains deferred until its separately approved migration plan;
|
||||
do not rotate `BETTER_AUTH_SECRET` without that plan.
|
||||
|
||||
@@ -16,12 +16,12 @@
|
||||
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
||||
| FED-M1-02 | done | Author `docker-compose.federated.yml` as an overlay profile: Postgres 17 + pgvector extension (port 5433), Valkey (6380), named volumes, healthchecks. Compose-up should boot cleanly on a clean machine. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471. Overlay defines `postgres-federated`/`valkey-federated`, profile-gated, with pg-init for pgvector extension. |
|
||||
| FED-M1-03 | done | Add pgvector support to `packages/storage/src/adapters/postgres.ts`: create extension on init (idempotent), expose vector column type in schema helpers. No adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472. `enableVector` flag on postgres StorageConfig; idempotent CREATE EXTENSION before migrations. |
|
||||
| FED-M1-02 | done | Historical shipped-status record: authored a federated Compose overlay with PostgreSQL/pgvector, Valkey, volumes, and healthchecks. It is not a current startup, init, extension, schema, or migration procedure. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471 status only. KBN-101-02 retires its init authority; KBN-101-05 replaces deployment rendering; KBN-101-07 SETUP is non-operative until activation. |
|
||||
| FED-M1-03 | done | Historical shipped-status record: add pgvector support to `packages/storage/src/adapters/postgres.ts`; no adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472 status only. **KBN-101 supersedes this behavior:** it cannot authorize current runtime extension creation or any DDL; only the runner/external bootstrap contract may do so. |
|
||||
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
||||
| FED-M1-05 | done | Write `scripts/migrate-to-federated.ts`: one-way migration from `local` (PGlite) / `standalone` (PG without pgvector) → `federated`. Dumps, transforms, loads; dry-run + confirm UX. Idempotent on re-run. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474. `mosaic storage migrate-tier`; DrizzleMigrationSource (corrects P0 found in review); 32 tests; idempotent. |
|
||||
| FED-M1-05 | done | Historical shipped-status record: prior tier migration implementation. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474 status only. **KBN-101 supersedes this route:** it cannot authorize current credentials, target connection, or DDL. The future active route requires runner verification plus target URL-file and signed attestation-file binding. |
|
||||
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
||||
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
||||
| FED-M1-08 | done | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Shipped in PR #477. Caught P0 in M1-05 (camelCase→snake_case) missed by mocked unit tests; fix in same PR. |
|
||||
|
||||
@@ -1,114 +1,81 @@
|
||||
# Fleet Launch Runbook
|
||||
|
||||
How every Mosaic fleet agent — workers **and** the orchestrator — is launched, and how to
|
||||
configure each one. The guiding principle: **one roster-driven launcher**. There is no bespoke
|
||||
per-agent launch script; the roster plus per-agent `.env` files are the single source of launch
|
||||
config.
|
||||
The local fleet roster is the sole writable desired-state authority for membership and launch policy.
|
||||
Generated environment files are rebuildable projections, not an operator-editable command surface.
|
||||
|
||||
## The launch chain
|
||||
## Launch chain
|
||||
|
||||
| Layer | File | Responsibility |
|
||||
| ---------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| systemd unit | `mosaic-agent@<role>.service` | One templated unit per role; `ExecStart` runs the session launcher with the instance name `%i`. Defaults `MOSAIC_AGENT_RUNTIME=pi`, `MOSAIC_AGENT_NAME=%i`. |
|
||||
| session launcher | `tools/fleet/start-agent-session.sh <role>` | Builds the launch command, opens the tmux pane, wires the heartbeat. |
|
||||
| launch command | `mosaic yolo <runtime>` (or a per-agent override) | Replaces the pane's foreground process with the runtime, fully seeded. |
|
||||
| seeding | `mosaic`'s `composeContract()` | Injects the Constitution/USER/TOOLS/runtime contract, `*.local` overlays, **and** the Fleet-Comms cheat-sheet — all via `--append-system-prompt`. |
|
||||
| Layer | Responsibility |
|
||||
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Roster | `fleet/roster.yaml` supplies the agent name, class, supported runtime, model, reasoning, tool policy, workdir, and tmux socket. |
|
||||
| Projection writer | Renders deterministic `fleet/agents/<name>.env.generated` from the roster. |
|
||||
| Optional local data | Reads a strict, data-only `fleet/agents/<name>.env.local`; it cannot shadow generated keys. |
|
||||
| systemd | Starts the launcher with `env -i` and fixed bootstrap data. It does not preload either environment file. |
|
||||
| session launcher | Validates generated and local data before it queries, creates, or stops an exact tmux session. |
|
||||
| runtime launch | Derives the fixed `mosaic yolo <runtime>` argument array from validated roster data, then seeds the runtime contract. |
|
||||
|
||||
Per-agent overrides live in `fleet/agents/<role>.env`, generated from `roster.yaml` by
|
||||
`generateAgentEnv` (`packages/mosaic/src/commands/fleet.ts`) and consumed by the launcher.
|
||||
The launcher never `source`s or `eval`s an environment file and never accepts an environment-supplied
|
||||
command. `MOSAIC_AGENT_COMMAND`, command/channel overrides, unknown keys, generated-key shadowing,
|
||||
secret-like key names, duplicate keys, comments, quoted/export syntax, and unsafe values are rejected.
|
||||
|
||||
## Worker launch path (default)
|
||||
## Generated and local files
|
||||
|
||||
1. `roster.yaml` carries each agent's `runtime` and optional `model_hint`.
|
||||
2. `generateAgentEnv` emits `fleet/agents/<role>.env` with `MOSAIC_AGENT_NAME`,
|
||||
`MOSAIC_AGENT_RUNTIME`, and `MOSAIC_AGENT_MODEL`.
|
||||
3. `start-agent-session.sh` has no `MOSAIC_AGENT_COMMAND` set, so it falls through to the default
|
||||
(line ~44):
|
||||
```sh
|
||||
MOSAIC_AGENT_COMMAND="mosaic yolo $MOSAIC_AGENT_RUNTIME${MOSAIC_AGENT_MODEL:+ --model $MOSAIC_AGENT_MODEL}"
|
||||
```
|
||||
4. The launcher bakes `MOSAIC_AGENT_NAME` into the pane command (line ~118), so `composeContract`
|
||||
can inject the Fleet-Comms cheat-sheet for that role.
|
||||
`<name>.env.generated` is complete, deterministic, and written only by Mosaic. Its ordered keys are:
|
||||
|
||||
That is the whole worker path: roster → `.env` → `mosaic yolo <runtime>` → seeded pane.
|
||||
|
||||
## Orchestrator fold (PATH A — ships today)
|
||||
|
||||
The orchestrator is **just another roster agent** launched through the canonical path — not a
|
||||
snowflake script.
|
||||
|
||||
| Piece | Value |
|
||||
| ------------------ | ----------------------------------- |
|
||||
| host-side launcher | `orchestrator-launch.sh` |
|
||||
| systemd unit | `mosaic-fleet-orchestrator.service` |
|
||||
| tmux session | `orchestrator` (role-named) |
|
||||
|
||||
Set its launch command via `fleet/agents/orchestrator.env`:
|
||||
|
||||
```sh
|
||||
MOSAIC_AGENT_COMMAND='mosaic yolo claude --channels plugin:discord@<channel>'
|
||||
```dotenv
|
||||
MOSAIC_AGENT_NAME=<roster name>
|
||||
MOSAIC_AGENT_CLASS=<roster class>
|
||||
MOSAIC_AGENT_RUNTIME=<roster runtime>
|
||||
MOSAIC_AGENT_MODEL=<roster model hint>
|
||||
MOSAIC_AGENT_REASONING=<roster reasoning>
|
||||
MOSAIC_AGENT_TOOL_POLICY=<roster tool policy>
|
||||
MOSAIC_AGENT_WORKDIR=<absolute roster work directory>
|
||||
MOSAIC_TMUX_SOCKET=<roster socket or empty>
|
||||
```
|
||||
|
||||
When `MOSAIC_AGENT_COMMAND` is set, `start-agent-session.sh`'s `if [ -z "$MOSAIC_AGENT_COMMAND" ]`
|
||||
guard (line ~41) is false, so the line-44 default — **including its hardcoded `yolo`** — is skipped
|
||||
entirely. The override fully controls the runtime and flags. Routing through `mosaic yolo claude`
|
||||
(rather than a raw `claude` invocation) is what gives the orchestrator the same full
|
||||
`composeContract` seeding + Fleet-Comms cheat-sheet as every worker, with `--channels` and any
|
||||
other flags passed straight through to the `claude` binary.
|
||||
The generated launch contract supports `claude`, `codex`, `opencode`, and `pi`. `mosaic fleet add`
|
||||
rejects another runtime before it writes the roster or modifies generated, local, or quarantine state.
|
||||
The legacy dogfood stub remains an observability-only canary on its separate `mosaic-factory` socket;
|
||||
it has no generated-launch adapter and cannot be added through this path.
|
||||
|
||||
## Launch gotchas
|
||||
`<name>.env.local` is optional and may contain only non-secret machine data:
|
||||
|
||||
1. **Flag conflict.** `mosaic yolo claude` already injects `--dangerously-skip-permissions`. Do
|
||||
**not** also pass `--permission-mode bypassPermissions` — the `claude` binary would receive both.
|
||||
Use `mosaic yolo claude …` alone (yolo covers the unattended posture), **or** non-yolo
|
||||
`mosaic claude --permission-mode bypassPermissions …`. Never mix the two.
|
||||
2. **`MOSAIC_AGENT_NAME` must reach the pane.** The launcher bakes it from the instance name, and
|
||||
`composeContract` gates the Fleet-Comms block on it (`launch.ts`, in `composeContract`) — **and**
|
||||
the role must be a member of `roster.yaml`, or the block resolves empty.
|
||||
3. **`launchRuntime` guards.** `mosaic yolo claude` runs `checkSoul` / `checkRuntime` /
|
||||
`checkSequentialThinking`. The host needs `SOUL.md` and the sequential-thinking MCP, or the
|
||||
launch aborts (a raw `claude` invocation skipped these checks). Dry-run the composed command in a
|
||||
throwaway tmux session before swapping a live launcher.
|
||||
- `MOSAIC_RUNTIME_BIN`
|
||||
- `MOSAIC_HEARTBEAT_RUN_DIR`
|
||||
- `MOSAIC_HEARTBEAT_INTERVAL`
|
||||
- `MOSAIC_CLAUDE_JSON`
|
||||
- `CLAUDE_CONFIG_DIR`
|
||||
|
||||
## Why per-agent `.env` survives upgrades (#632)
|
||||
Paths must be safe absolute paths and the heartbeat interval must be a positive integer. Projection,
|
||||
local, and quarantine files must be private regular files; the managed directories must be real,
|
||||
private, non-symlink paths. Violations fail closed before tmux interaction.
|
||||
|
||||
`install.sh` `PRESERVE_PATHS` includes `fleet/*.yaml`, `fleet/agents`, and `fleet/run`, so
|
||||
`mosaic update`'s framework re-seed **preserves** your roster and per-agent `.env` overrides
|
||||
(glob-aware `cp` fallback; matching TS parity in `file-adapter.ts`). Before #632, an auto re-seed
|
||||
could wipe them — which is exactly why PATH A's `.env` override is safe to rely on now.
|
||||
## Legacy input and diagnostics
|
||||
|
||||
## Inspecting the comms wiring
|
||||
A legacy `<name>.env` is input only during projection generation. Roster-owned keys are regenerated;
|
||||
valid allowed local data can move to `.env.local`; invalid legacy input is privately retained at
|
||||
`<name>.env.quarantine`. Neither legacy nor quarantine files are launch authority.
|
||||
|
||||
- `mosaic fleet comms-block <role>` prints the Fleet-Comms cheat-sheet a given role receives at
|
||||
launch — its `[host:session]` identity, the exact `agent-send.sh` command for each peer, and the
|
||||
FLIP / `--verify` conventions. `--host <h>` previews a cross-host view. An unknown role or missing
|
||||
roster **fails loud** (stderr + non-zero exit), so a typo is never a silent no-op.
|
||||
- Versus `mosaic compose-contract <runtime>`: that emits the **whole** system prompt and reads the
|
||||
role from `MOSAIC_AGENT_NAME` (a full-prompt smoke test). `comms-block` is the targeted,
|
||||
explicit-arg, comms-only view — e.g. `mosaic fleet comms-block coder0-0` to preview a peer.
|
||||
Diagnostics expose only rule code, key name, and a SHA-256 content hash. They do not reveal command
|
||||
text, credentials, or other values.
|
||||
|
||||
## North Star / future direction
|
||||
## Launch and stop behavior
|
||||
|
||||
**Vision:** a webUI lets the user edit each agent's launch config — switch **harness**
|
||||
(claude / pi / codex / opencode), toggle **yolo**, pick a **model**, set a **command/channels**
|
||||
override — with no terminal.
|
||||
The launcher obtains the agent's socket only from the validated generated projection. It creates or
|
||||
checks the exact `=<agent-name>` tmux target; it never uses an ambient socket or fuzzy session match.
|
||||
The same strict parser runs before exact-stop behavior. A fresh native Pi heartbeat remains authoritative;
|
||||
the shell sidecar only provides fallback state when the native marker is stale or absent.
|
||||
|
||||
**Continuity — this is not a new launch path.** It is a data-model + UI-binding layer over the
|
||||
existing roster-driven launcher. Field-by-field status today:
|
||||
`mosaic agent comms-block <exact-member>` can inspect that exact roster member's resolved Fleet-Comms
|
||||
block. It is a read-only inspection tool and fails loudly for an unknown exact member or missing roster.
|
||||
On Linux, the installed roster, TOOLS contract, and executable helper are opened through a held
|
||||
descriptor chain rooted at `/`; every managed path component uses no-follow traversal, and content plus
|
||||
execute validation stay bound to the same opened file. Systems without Linux `/proc/self/fd` support
|
||||
fail closed rather than falling back to pathname revalidation.
|
||||
|
||||
| Launch-config field | Roster-native today? | Mechanism / gap |
|
||||
| ------------------------ | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **harness** (`runtime`) | ✅ end-to-end | `roster.runtime` → `generateAgentEnv` emits `MOSAIC_AGENT_RUNTIME` → launcher line 44. UI just writes the field. |
|
||||
| **model** (`model_hint`) | ✅ end-to-end | `roster.model_hint` → `MOSAIC_AGENT_MODEL` → launcher line 44 `--model`. UI just writes the field. |
|
||||
| **yolo** | ❌ new | Launcher line 44 **hardcodes** `mosaic yolo`. A non-yolo toggle needs a roster `yolo` field → emit `MOSAIC_AGENT_YOLO` → make line 44 conditional. |
|
||||
| **command / channels** | ❌ new | `MOSAIC_AGENT_COMMAND` is **consumed** (launcher line ~12) but `generateAgentEnv` does not emit it. Needs a roster `command`/`channels` field → emitted. |
|
||||
## Current M2 boundary
|
||||
|
||||
**The arc:**
|
||||
|
||||
- **A** — `.env` `MOSAIC_AGENT_COMMAND` hatch: manual, ships now, kept safe across upgrades by #632.
|
||||
- **B** — roster-native launch-config: harness + model are already there; add the **yolo** toggle
|
||||
(line-44 conditional) and **command/channels** emission to complete the data model.
|
||||
- **webUI** — binds dropdowns/toggles directly to those four roster fields.
|
||||
|
||||
PATH A's `.env` override is the **manual form** of exactly what PATH B makes roster-native and the
|
||||
webUI edits — one continuous arc, not three separate features. PATH B is tracked as #636.
|
||||
FCM-M2-001 supplies generated/local parsing, validation, projection, quarantine, and launch-boundary
|
||||
evidence only. It does not authorize roster CRUD expansion, reconciliation, lifecycle changes, remote
|
||||
or connector mutation, site canaries, or migration. M3 must establish the local reconcile/lifecycle
|
||||
path; M4 separately provides migration preview, canary, and rollback gates.
|
||||
|
||||
@@ -7,7 +7,8 @@ The v2 compiler may not silently accept an unresolved class. Before M1 exits, ev
|
||||
below must be either migrated and executable, retained as an explicitly versioned v1 fixture, or
|
||||
retired with a replacement/deprecation note. Class resolution must use the existing
|
||||
profile/persona/provision baseline-plus-`roles.local` resolver; this inventory does not create a
|
||||
parallel resolver.
|
||||
parallel resolver. The current executable implementation and per-artifact outcomes are recorded in
|
||||
[the disposition evidence](./migration/example-profile-disposition.md).
|
||||
|
||||
## Examples
|
||||
|
||||
|
||||
@@ -15,8 +15,8 @@ The backlog uses the existing Mosaic storage layer; there is **no** new database
|
||||
engine (no sqlite, no raw client).
|
||||
|
||||
| Condition | Tier | Data location |
|
||||
| ------------------------------ | -------------------- | -------------------------------- |
|
||||
| `DATABASE_URL` set | Full server Postgres | the configured database |
|
||||
| ---------------------------------- | -------------------- | ---------------------------------------------------------------- |
|
||||
| `DATABASE_URL` injected at runtime | Full server Postgres | the verified runtime database; it never authorizes migration/DDL |
|
||||
| `PGLITE_DATA_DIR` set (no URL) | Embedded PGlite | that directory |
|
||||
| neither (default) | Embedded PGlite | `~/.config/mosaic/fleet/backlog` |
|
||||
|
||||
@@ -24,8 +24,7 @@ PGlite is real Postgres semantics in-process — including the row locks the ato
|
||||
claim relies on — so the **same code** runs on a laptop (embedded, single-host
|
||||
default) and on a full Postgres deployment. Switching tiers is config-only.
|
||||
|
||||
The schema (`backlog` table) is created automatically on first CLI use:
|
||||
`runMigrations()` for Postgres, `runPgliteMigrations()` for embedded PGlite.
|
||||
For embedded PGlite only, the local backlog routine may prepare its local schema on first use. **Current operator behavior is PGlite-only.** The PostgreSQL path is held until KBN-101 activation; no current PostgreSQL CLI route, runner, or first-use migration is available or authorized. A future activated PostgreSQL runtime may connect only after its separately certified readiness gate.
|
||||
|
||||
### Update safety
|
||||
|
||||
|
||||
74
docs/fleet/how-to/create-update-delete-agent.md
Normal file
74
docs/fleet/how-to/create-update-delete-agent.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Create, Inspect, Update, and Delete a Local Fleet Agent
|
||||
|
||||
Use the local roster-v2 control plane only. These commands change desired state and derived environment projections; they never start, stop, reconcile, inspect, or otherwise act on systemd, tmux, sessions, or runtimes.
|
||||
|
||||
## Read and plan first
|
||||
|
||||
```sh
|
||||
mosaic fleet get <name>
|
||||
mosaic fleet plan create --expected-generation <n> --agent '<json>'
|
||||
mosaic fleet plan update <name> --expected-generation <n> --agent '<json>'
|
||||
mosaic fleet plan delete <name> --expected-generation <n>
|
||||
```
|
||||
|
||||
`plan create` takes the name from `--agent`. `plan update` and `plan delete` require the target name immediately after the operation. A plan is deterministic and side-effect free: it validates the complete proposed roster and projection targets without changing files. Use `--dry-run` on `create`, `update`, or `delete` for the same no-write result.
|
||||
|
||||
Every successful command prints JSON. `get` returns `{ "generation", "agent" }`; mutation results contain `plan`, `applied`, `authoritativeRoster`, and `projections`.
|
||||
|
||||
## Create safely
|
||||
|
||||
```sh
|
||||
mosaic fleet create --expected-generation 7 --agent '{
|
||||
"name":"coder0",
|
||||
"alias":"Coder 0",
|
||||
"className":"code",
|
||||
"runtime":"pi",
|
||||
"provider":"openai",
|
||||
"model":"gpt-5.6-sol",
|
||||
"reasoning":"high",
|
||||
"toolPolicy":"code",
|
||||
"workingDirectory":"/srv/mosaic",
|
||||
"persistentPersona":false,
|
||||
"resetBetweenTasks":true,
|
||||
"launch":{"yolo":true}
|
||||
}'
|
||||
```
|
||||
|
||||
Create defaults to `enabled: true` and `desired_state: stopped`. It does not start a process. Add `--persisted-start` only to persist `desired_state: running`; that still does not start a runtime in this M2 command. The JSON payload is an allowlist of the roster-v2 fields shown above plus `launch.yolo`; command, channel, secret-reference, and other unknown keys are rejected rather than ignored. The JSON error exposes only a stable code, never the rejected value.
|
||||
|
||||
## Update and delete safely
|
||||
|
||||
```sh
|
||||
mosaic fleet update coder0 --expected-generation 8 --agent '<complete JSON agent payload>'
|
||||
mosaic fleet delete coder0 --expected-generation 9
|
||||
```
|
||||
|
||||
Updates require a complete agent JSON payload and preserve the stable name. Delete removes only the exact roster-owned `coder0.env.generated` projection. It retains `coder0.env.local`, legacy `coder0.env`, `coder0.env.quarantine`, and every unrelated projection. A delete dry-run leaves all of those files byte-identical.
|
||||
|
||||
## Handle generation conflicts
|
||||
|
||||
Every mutation requires the current authoritative `--expected-generation`. A stale value returns JSON `error.code: "stale-generation"` with a non-zero exit. Reload with `mosaic fleet get <name>` or reread the roster, plan again using the returned generation, then retry. A concurrent mutation returns `concurrent-mutation`; do not force or bypass the lock.
|
||||
|
||||
## Interpret partial failures
|
||||
|
||||
The roster is authoritative and is written before derived projections. A late projection I/O failure returns non-zero with redacted, actionable JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "committed",
|
||||
"projections": "incomplete",
|
||||
"recovery": {
|
||||
"code": "projection-apply-failed",
|
||||
"action": "regenerate-projections-from-roster"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is not a rollback and not a no-op: reload the roster because its generation and membership were committed, regenerate projections from that roster, then plan a new mutation. Recovery output never contains environment values, credentials, or command text.
|
||||
|
||||
## Exit and boundary behavior
|
||||
|
||||
Handled validation errors and partial projection failures exit non-zero. `plan`/`--dry-run` and normal mutation JSON make the state explicit; scripts should use both the exit code and `authoritativeRoster`/`projections`, not `applied` alone.
|
||||
|
||||
The commands operate only on `<mosaic-home>/fleet/roster.yaml`, the local roster desired-state authority. They do not accept arbitrary commands, channels, secrets, remote/connector actions, migration/canary actions, or runtime lifecycle operations.
|
||||
54
docs/fleet/how-to/customize-roles.md
Normal file
54
docs/fleet/how-to/customize-roles.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Customize Fleet Roles
|
||||
|
||||
Mosaic resolves persona contracts through two layers:
|
||||
|
||||
1. `fleet/roles/<canonical-class>.md` — seeded baseline contract.
|
||||
2. `fleet/roles.local/<canonical-class>.md` — operator override or custom role; this layer wins.
|
||||
|
||||
The same shared resolver is used by profile validation, provisioning, roster-v2 semantic validation,
|
||||
and launch-time persona injection.
|
||||
|
||||
## Override a baseline role
|
||||
|
||||
Create a readable Markdown contract under `roles.local` with the canonical filename and class marker:
|
||||
|
||||
```markdown
|
||||
# Code — local role definition
|
||||
|
||||
The local code role (`class: code`) follows the operator's repository conventions.
|
||||
```
|
||||
|
||||
Save it as `fleet/roles.local/code.md`. Do not edit generated or seeded baseline assets when the goal
|
||||
is a durable local customization.
|
||||
|
||||
Legacy aliases canonicalize before lookup. Therefore `roles.local/implementer.md` does not override
|
||||
`code`; use `roles.local/code.md`. See [Legacy Fleet Class Aliases](../migration/legacy-class-aliases.md).
|
||||
|
||||
## Add a custom class
|
||||
|
||||
A custom class remains supported when a readable contract exists for the exact identifier:
|
||||
|
||||
```markdown
|
||||
# Release notes — local role definition
|
||||
|
||||
The release-notes role (`class: release-notes`) prepares operator-reviewed release copy.
|
||||
```
|
||||
|
||||
Save it as `fleet/roles.local/release-notes.md`, then reference `class: release-notes` and a matching
|
||||
`tool_policy: release-notes` in roster v2. Adding only a `LIBRARY.md` row is insufficient.
|
||||
|
||||
Names such as `worker`, `analyst`, and `canary` are not built-in aliases; they need genuine custom
|
||||
contracts. `agents[].alias`, Tess, and Ultron are display names and cannot select a class.
|
||||
|
||||
## Validation and authority boundaries
|
||||
|
||||
Semantic validation reads the winning contract and rejects missing, unreadable, or empty files.
|
||||
Protected authority is derived from canonical class metadata in code, never from role prose. A custom
|
||||
contract cannot claim merge, validation-certificate, orchestration, lease, or interaction authority.
|
||||
|
||||
Roster v2 also fails closed when a protected class and tool policy do not match after canonicalization,
|
||||
or when an unprotected class claims a protected tool policy. The legacy `operator-interaction` policy
|
||||
canonicalizes to `interaction`.
|
||||
|
||||
Role customization does not issue leases, store validation certificates, mutate credentials, or
|
||||
change lifecycle state.
|
||||
23
docs/fleet/how-to/start-stop-restart.md
Normal file
23
docs/fleet/how-to/start-stop-restart.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Safely Reconcile and Control a Local Fleet Agent
|
||||
|
||||
Use the canonical local roster-v2 command surface:
|
||||
|
||||
```sh
|
||||
mosaic fleet apply --expected-generation <n> --dry-run
|
||||
mosaic fleet apply --expected-generation <n>
|
||||
mosaic fleet reconcile --expected-generation <n>
|
||||
mosaic fleet start <name> --expected-generation <n>
|
||||
mosaic fleet stop <name> --expected-generation <n>
|
||||
mosaic fleet restart <name> --expected-generation <n>
|
||||
mosaic fleet status [name]
|
||||
mosaic fleet verify
|
||||
mosaic fleet doctor
|
||||
```
|
||||
|
||||
Start with `--dry-run`. It validates roster semantics, deterministic projections, private managed paths, exact holder ownership, and named-socket state without changing files or lifecycle state. `apply` and `reconcile` rebuild derived projections and enforce only persisted roster state: enabled `running` agents may start, while stopped or disabled agents are not started.
|
||||
|
||||
`start`, `stop`, and `restart` are explicit one-shot exact-service actions. They do not persist a lifecycle change. Roster CRUD is the only way to change persisted desired state.
|
||||
|
||||
Every command prints JSON. Observation commands report drift without mutation; `verify` exits non-zero on ownership mismatch, unmanaged sessions, or drift. A failed apply that wrote some derived projections reports `projections: "incomplete"` with bounded recovery to regenerate from the roster. A lifecycle failure after projections reports incomplete lifecycle work; it is never represented as a rollback or no-op.
|
||||
|
||||
These commands are local only. Remote/SSH/connector entries are inventory/validation-only. Commands do not accept arbitrary runtime commands, channels, secrets, generated-file desired state, or arbitrary tmux sockets.
|
||||
69
docs/fleet/migration/example-profile-disposition.md
Normal file
69
docs/fleet/migration/example-profile-disposition.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# Executable Fleet Example, Profile, and Service-Preset Dispositions
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M1-003 · **Status:** M1 executable disposition evidence
|
||||
|
||||
This document records the executable disposition for every currently shipped fleet YAML artifact.
|
||||
The authoritative baseline classification remains the
|
||||
[legacy inventory](../LEGACY-EXAMPLE-PROFILE-DISPOSITION-INVENTORY.md). The executable guard is
|
||||
`packages/mosaic/src/fleet/example-profile-dispositions.ts`; its test fails if a shipped YAML
|
||||
artifact is added, removed, or left without one of the dispositions below.
|
||||
|
||||
## Disposition rules
|
||||
|
||||
- **Explicit v1 fixture:** the artifact is loaded through the existing v1 roster parser and must
|
||||
declare `version: 1`. It remains a compatibility fixture; it is not silently treated as a v2
|
||||
roster or given inferred aliases.
|
||||
- **Canonical profile:** the artifact is loaded through `loadProfiles`, which uses the shared
|
||||
baseline-plus-`roles.local` persona resolver and rejects unreadable or unresolved classes.
|
||||
- **Canonical service policy:** the artifact is loaded through the operator-interaction service
|
||||
policy reader and provisioned with a generic supplied identity. It validates its runtime, model,
|
||||
reasoning, and legacy tool-policy compatibility without hardcoding a product identity.
|
||||
|
||||
No artifact is retired in this card. A later retirement requires both a replacement link and a
|
||||
visible deprecation note; the executable guard must then record the new disposition before the
|
||||
artifact can be removed.
|
||||
|
||||
## Shipped artifacts
|
||||
|
||||
| Artifact | Disposition | Executable path | Compatibility notes |
|
||||
| ------------------------------------ | ------------------------ | --------------------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `examples/coding.yaml` | Explicit v1 fixture | v1 roster parser | Retains approved `implementer` and `reviewer` compatibility inputs. |
|
||||
| `examples/general.yaml` | Explicit v1 fixture | v1 roster parser | Retains unresolved `worker` without an inferred canonical role. |
|
||||
| `examples/hybrid.yaml` | Explicit v1 fixture | v1 roster parser | Retains `implementer`, `reviewer`, and resolver-dependent `researcher`. |
|
||||
| `examples/local-canary.yaml` | Explicit v1 fixture | v1 roster parser | Retains the local-tmux canary topology. |
|
||||
| `examples/minimal.yaml` | Explicit v1 fixture | v1 roster parser | Retains `canary` without an inferred canonical role. |
|
||||
| `examples/operator-interaction.yaml` | Explicit v1 fixture | v1 roster parser | Keeps Tess only as an example instance name; `operator-interaction` remains compatibility input. |
|
||||
| `examples/research.yaml` | Explicit v1 fixture | v1 roster parser | Retains resolver-dependent `researcher` and `analyst`. |
|
||||
| `profiles/business.yaml` | Canonical profile | shared profile/persona resolver | Every referenced business class must resolve to a readable contract. |
|
||||
| `profiles/marketing.yaml` | Canonical profile | shared profile/persona resolver | Every referenced marketing class must resolve to a readable contract. |
|
||||
| `profiles/personal-assistant.yaml` | Canonical profile | shared profile/persona resolver | No interaction equivalence is inferred. |
|
||||
| `profiles/research.yaml` | Canonical profile | shared profile/persona resolver | Every research class must resolve to a readable contract. |
|
||||
| `profiles/software-delivery.yaml` | Canonical profile | shared profile/persona resolver | Retains the governance profile; authority validation remains FCM-M1-002 evidence. |
|
||||
| `services/operator-interaction.yaml` | Canonical service policy | service-policy reader/provisioner | Generic provisioning supplies the instance name; the policy itself never names Tess. |
|
||||
|
||||
## M4 migration-preview evidence
|
||||
|
||||
FCM-M4-001 layers an executable migration posture over the same 13-entry M1 inventory without
|
||||
changing the retained artifact classification:
|
||||
|
||||
- every `v1-fixture` is previewed only with explicit class and lifecycle evidence;
|
||||
- every `canonical-profile` remains validated by the shared baseline-plus-`roles.local` resolver;
|
||||
- the canonical service policy remains generic and uses only the approved tool-policy alias.
|
||||
|
||||
`validateShippedFleetMigrationDispositions` first runs the existing executable M1 guard, then requires
|
||||
explicit decisions and lifecycle observations and executes `previewV1ToV2Migration` for every shipped
|
||||
v1 fixture. `collectShippedFleetMigrationDispositions` derives the 13-entry posture directly from
|
||||
`SHIPPED_FLEET_ARTIFACT_DISPOSITIONS`, so additions or removals continue to fail the M1 guard rather
|
||||
than creating a second artifact list. None of these dispositions claims a cutover, canary, or
|
||||
rollback; those gates belong to FCM-M4-002. See [v1-to-v2 preview](./v1-to-v2.md).
|
||||
|
||||
## Running the guard
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/mosaic test -- v1-v2-migration.spec.ts \
|
||||
-t "validates all 13 shipped artifacts and executes ready previews for every v1 fixture"
|
||||
```
|
||||
|
||||
The guard is intentionally limited to shipped assets and validation. It does not generate
|
||||
environment files, mutate a roster, reconcile a fleet, migrate an installed roster, or launch an
|
||||
agent.
|
||||
40
docs/fleet/migration/legacy-class-aliases.md
Normal file
40
docs/fleet/migration/legacy-class-aliases.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Legacy Fleet Class Aliases
|
||||
|
||||
Fleet class compatibility is intentionally narrow. The shared resolver accepts exactly three legacy
|
||||
class names and converts them to canonical classes before persona lookup:
|
||||
|
||||
| Legacy value | Canonical value | Migration action |
|
||||
| ---------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `implementer` | `code` | Replace class and tool-policy references with `code`. |
|
||||
| `reviewer` | `review` | Replace class and tool-policy references with `review`. |
|
||||
| `operator-interaction` | `interaction` | Replace class and roster-v2 tool-policy references with `interaction`. The legacy service artifact remains compatible. |
|
||||
|
||||
Alias support preserves existing inputs while provisioning and typed semantic output use canonical
|
||||
identities. Requested and canonical class values remain separately observable during semantic
|
||||
validation.
|
||||
|
||||
## Lookup and override behavior
|
||||
|
||||
Canonicalization precedes baseline and `roles.local` lookup. A legacy-named override such as
|
||||
`roles.local/implementer.md` is not a separate authority and is not selected for an `implementer`
|
||||
request. Customize the canonical role instead, for example `roles.local/code.md`.
|
||||
|
||||
The compatibility file `operator-interaction.md` remains shipped, but `interaction` is the canonical
|
||||
role class. Tess is an example display name only.
|
||||
|
||||
## Unresolved and custom classes
|
||||
|
||||
No names are inferred from historical usage, instance names, or similar wording. `worker`, `analyst`,
|
||||
`canary`, Tess, and Ultron are not aliases. An otherwise unknown class is accepted only if the shared
|
||||
resolver can read an actual baseline or `roles.local` contract for that exact class. A `LIBRARY.md`
|
||||
row without a readable contract fails semantic validation.
|
||||
|
||||
Custom classes receive no protected authority implicitly. Protected class/tool-policy mismatches
|
||||
fail closed.
|
||||
|
||||
## Retirement guidance
|
||||
|
||||
New configuration should emit canonical values. Existing inputs may use the three aliases during the
|
||||
compatibility period, but operators should migrate class and tool-policy fields together. Do not
|
||||
create new legacy-named role overrides; move their intended content to the canonical filename and
|
||||
validate the roster/profile before removing the old artifact.
|
||||
86
docs/fleet/migration/v1-to-v2.md
Normal file
86
docs/fleet/migration/v1-to-v2.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Previewing a Fleet Roster v1-to-v2 Migration
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M4-001 · **Effect boundary:** preview only
|
||||
|
||||
`mosaic fleet migrate-v1 preview` inventories a v1 roster and emits a canonical v2 candidate plus
|
||||
recovery evidence. It does not write a roster, apply environment projections, invoke systemd or
|
||||
`tmux`, contact connectors or remote hosts, launch an agent, run a canary, or execute rollback.
|
||||
FCM-M4-002 owns reversible cutover and rollback.
|
||||
|
||||
## Inputs
|
||||
|
||||
```bash
|
||||
mosaic fleet migrate-v1 preview \
|
||||
--source roster-v1.yaml \
|
||||
--decisions migration-decisions.json \
|
||||
--observations reviewed-observations.json
|
||||
```
|
||||
|
||||
The command emits one JSON object and exits nonzero when the preview is blocked, including when any of
|
||||
`--source`, `--decisions`, or `--observations` is omitted, passed without a path value, or passed an empty
|
||||
path value. These request-shape failures are reported before any input file is read. Decision and
|
||||
observation JSON is validated fail-closed: unknown fields, malformed values, and records for non-local
|
||||
agents are rejected. Decisions must supply a positive v2 `generation`, a reviewed `fleetHost` whenever
|
||||
v1 agents include `host` or `ssh`, explicit `defaultRuntime`, and per-local-agent provider, model,
|
||||
reasoning, enabled state, and launch policy. The v1 source remains authoritative for socket semantics:
|
||||
a supported declared socket field, including an explicit empty value for the default tmux server, is
|
||||
preserved; if both supported root aliases are absent, the production v1 default is the literal empty socket.
|
||||
A matching `socketName` decision is accepted and an incompatible decision blocks, but a decision never
|
||||
supplies or repairs a missing source socket. If v1 omitted `tool_policy`, decisions must supply an
|
||||
explicit replacement; it is never derived from `class`. `model_hint` is never split or treated as
|
||||
authority.
|
||||
|
||||
Observations are separate reviewed evidence keyed by local agent name:
|
||||
|
||||
```json
|
||||
{
|
||||
"coder0": { "systemd": "inactive", "tmux": "missing" }
|
||||
}
|
||||
```
|
||||
|
||||
Only `active` plus `present` maps to `running`; only `inactive` plus `missing` maps to `stopped`.
|
||||
Missing, extra, unknown, or contradictory evidence blocks output. An observed-running agent cannot
|
||||
be marked disabled. Observed-stopped agents always remain stopped.
|
||||
|
||||
## Field disposition
|
||||
|
||||
| v1 field | v2 disposition |
|
||||
| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `version`, `transport`, `tmux`, `defaults`, `runtimes` | Inventoried and structurally compiled; omitted runtimes retain v1 built-in defaults, while each explicitly declared runtime without a reset field follows the production v1 `/clear` fallback; present-empty holder/work-directory/reset values block |
|
||||
| agent `name`, `alias`, `runtime`, working directory, persona/reset flags | Copied or explicitly defaulted only when absent; present-empty alias/work-directory values block for explicit disposition. Canonical `~`/`~/...` values stay unchanged in roster evidence and traversal-free forms expand only at the shared production environment-projection boundary before unchanged absolute-path validation |
|
||||
| `provider`, `model_hint`, `reasoning_level` | Explicit provider/model/reasoning decisions; no model-hint inference |
|
||||
| `class`, `tool_policy` | Only approved aliases canonicalize automatically; other classes require explicit preserve/replace disposition and shared-resolver validation |
|
||||
| `kickstart_template` | No v2 field; explicit inventory-only disposition required |
|
||||
| agent `host`, `ssh` | `host != fleetHost` is demonstrably remote and inventory-only; `host == fleetHost` stays local; SSH targets with or without an explicit user must agree with `host`; ssh-only, missing fleet-host evidence, or contradictory targets block |
|
||||
| agent `socket` | Same-host candidate only when it matches the canonical fleet socket; conflicts block for explicit future disposition |
|
||||
| root `connector` | Inventory-only; never contacted or reconciled |
|
||||
| unknown fields or snake/camel synonym collisions | Inventoried and block readiness |
|
||||
| `.env.generated` | Rebuild from canonical roster data |
|
||||
| no legacy `.env` | `absent`; no legacy action required |
|
||||
| legacy `.env` containing generated keys only | `regenerate-only`; replace later from canonical roster data |
|
||||
| legacy `.env` containing strict local keys | `relocate-local`; preserve those keys in `.env.local` during a later reviewed cutover |
|
||||
| legacy `.env` containing forbidden/unsafe/sensitive/malformed keys | `quarantine`; private input only, with diagnostics limited to code, key, and SHA-256 |
|
||||
|
||||
The only automatic aliases are `implementer → code`, `reviewer → review`, and
|
||||
`operator-interaction → interaction`. Similar or domain-specific names are never inferred. Automatic
|
||||
classes do not accept competing disposition records. Semantic validation delegates to the existing
|
||||
baseline-plus-`roles.local` resolver after the candidate is compiled by the existing v2 compiler.
|
||||
|
||||
## Evidence and recovery boundary
|
||||
|
||||
Ready output includes source and candidate SHA-256 identities, value-free field inventory, excluded
|
||||
remote/connector entries, explicit environment dispositions with sanitized diagnostics, and the lifecycle
|
||||
evidence used for each local candidate. Canonical lifecycle and remote-exclusion evidence ordering compares
|
||||
Unicode code points directly and does not depend on source-agent order or process locale. Source field
|
||||
inventory remains position-addressed evidence of the exact input. Recovery is marked non-executable and
|
||||
assigns the executable gate to FCM-M4-002.
|
||||
|
||||
Before any later cutover, preserve these artifacts:
|
||||
|
||||
1. authoritative v1 roster backup;
|
||||
2. agent environment backup, including `.env.local` and private quarantine inputs;
|
||||
3. reviewed lifecycle observations;
|
||||
4. canonical candidate v2 roster and its SHA-256.
|
||||
|
||||
See [backup and restore](../operations/backup-restore.md). Preview output is migration-readiness
|
||||
evidence, not proof that migration, canary, or rollback occurred.
|
||||
40
docs/fleet/operations/backup-restore.md
Normal file
40
docs/fleet/operations/backup-restore.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Fleet Configuration Backup and Restore Boundary
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M4-001
|
||||
|
||||
This page defines evidence that must exist before a roster v1-to-v2 cutover. FCM-M4-001 lists these
|
||||
prerequisites in non-executable recovery evidence but does not validate that backups exist and performs
|
||||
no backup, migration, canary, or restore. FCM-M4-002 owns the executable reversible canary and rollback
|
||||
gates.
|
||||
|
||||
## Preserve before cutover
|
||||
|
||||
- The authoritative v1 roster, byte-for-byte, with a SHA-256 identity.
|
||||
- Existing per-agent legacy `.env`, strict `.env.local`, and quarantine files under private
|
||||
permissions.
|
||||
- Reviewed per-local-agent systemd and exact-socket tmux observations.
|
||||
- The canonical v2 candidate and its SHA-256 identity.
|
||||
- Inventory-only remote agents and connector configuration as evidence, not local control-plane input.
|
||||
|
||||
`.env.generated` is a rebuildable projection and is not restored as authority. It must be regenerated
|
||||
from the selected authoritative roster. `.env.local` is operator-owned strict data and must not be
|
||||
overwritten or absorbed into generated output. Quarantined source remains private evidence; public
|
||||
diagnostics expose only rule code, key name, and SHA-256.
|
||||
|
||||
## Restore requirements
|
||||
|
||||
A later rollback implementation must restore the authoritative roster and operator-owned environment
|
||||
files, regenerate managed projections, and preserve each reviewed pre-cutover stopped/running state.
|
||||
It must never start an agent observed stopped and must never reconcile an inventory-only remote or
|
||||
connector entry.
|
||||
|
||||
The preview evidence deliberately records:
|
||||
|
||||
- `executable: false`;
|
||||
- required backup artifacts;
|
||||
- source and candidate identities;
|
||||
- lifecycle observations and resulting desired states;
|
||||
- environment relocation/quarantine dispositions;
|
||||
- FCM-M4-002 as the executable rollback gate owner.
|
||||
|
||||
Do not interpret a ready preview as a completed backup, migration, canary, or rollback.
|
||||
26
docs/fleet/operations/reconcile-and-recover.md
Normal file
26
docs/fleet/operations/reconcile-and-recover.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Reconcile and Recover a Local Fleet
|
||||
|
||||
## Safe sequence
|
||||
|
||||
1. Read `mosaic fleet doctor` and `mosaic fleet status`.
|
||||
2. Run `mosaic fleet apply --expected-generation <n> --dry-run`.
|
||||
3. Resolve stale generation, ownership mismatch, unsafe path, projection validation, or unmanaged-session findings before applying.
|
||||
4. Run `mosaic fleet apply --expected-generation <n>` only after the plan is understood.
|
||||
|
||||
The reconciler uses the exact roster tmux socket, exact holder session, private installation holder identity, and the complete expected global environment. For mutations it acquires its exclusive lock before rereading the canonical roster and fencing its generation; only that under-lock roster drives validation, planning, projections, and lifecycle effects. Before effects, its exclusive lock proves real private `MOSAIC_HOME` and `fleet` ancestors, uses a private `0600` lock leaf, and binds cleanup to the created file identity and ownership token. A fake holder, contaminated global environment, missing identity, unsafe lock path, or unmanaged session fails closed. It does not adopt, kill, or rename any unproven session. A crash can leave a stale lock for explicit operator inspection; reconciliation deliberately does not guess ownership or remove it.
|
||||
|
||||
## Partial results
|
||||
|
||||
The roster is never changed by reconciliation. If derived projection application partially fails, JSON reports:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "unchanged",
|
||||
"projections": "incomplete",
|
||||
"lifecycle": "not-applied",
|
||||
"recovery": { "code": "projection-apply-failed", "action": "regenerate-projections-from-roster" }
|
||||
}
|
||||
```
|
||||
|
||||
If projections completed but lifecycle work failed, JSON reports `projections: "complete"`, `lifecycle: "incomplete"`, and the bounded action `rerun-after-inspecting-owned-resources`. If lock cleanup cannot be proven after an effect result, it adds `cleanup: { "code": "lock-cleanup-failed", "action": "inspect-lock-before-retry" }` without changing the known projection, lifecycle, or primary recovery truth. Inspect the retained lock before retrying; no rollback, release, or stale-lock removal is implied. Results do not include environment values, secrets, or privileged command content.
|
||||
43
docs/fleet/reference/agent-mutations.md
Normal file
43
docs/fleet/reference/agent-mutations.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Local Fleet Agent Mutations
|
||||
|
||||
FCM-M2-002 provides local roster-v2 create, get, update, delete, and plan operations. They only change desired state and derived environment projections. They never start, stop, inspect, reconcile, or otherwise act on runtimes, systemd units, tmux sessions, or heartbeats.
|
||||
|
||||
## CLI contract
|
||||
|
||||
The commands operate only on the canonical `<mosaic-home>/fleet/roster.yaml` v2 authority and print one JSON object to stdout. `--agent` is a JSON object with the roster agent fields expressed as `className`, `toolPolicy`, `workingDirectory`, `persistentPersona`, `resetBetweenTasks`, and `launch: { "yolo": boolean }`.
|
||||
|
||||
```sh
|
||||
mosaic fleet get <name>
|
||||
mosaic fleet plan <create|update|delete> [name] --expected-generation <n> [--agent '<json>'] [--persisted-start]
|
||||
mosaic fleet create --expected-generation <n> --agent '<json>' [--dry-run] [--persisted-start]
|
||||
mosaic fleet update <name> --expected-generation <n> --agent '<json>' [--dry-run]
|
||||
mosaic fleet delete <name> --expected-generation <n> [--dry-run]
|
||||
```
|
||||
|
||||
`get` returns the authoritative generation and the selected agent. `plan create` derives its name from `--agent`; `plan update <name>` and `plan delete <name>` require the target name. `--agent` accepts only the documented roster-v2 request fields and `launch.yolo`; unknown keys such as commands, channels, or secret references are rejected. Rejection diagnostics return only the stable `invalid-request` code and never echo a rejected value. `plan` and `--dry-run` validate the complete proposed roster and projections but write neither the roster nor projections. `--persisted-start` is available only for a create request: it records `desired_state: running`, but does not start a process. Without it, create records `enabled: true` and `desired_state: stopped`. Handled failures return JSON with `error.code` and exit non-zero; unclassified validation/projection failures use the redacted `mutation-failed` code.
|
||||
|
||||
## Generation, validation, and idempotency
|
||||
|
||||
Each create, update, or delete request includes `expectedGeneration`. A request whose expected value differs from the authoritative roster generation fails with `stale-generation`; reload and retry with a newly computed plan. A private mutation lock rejects concurrent writers with `concurrent-mutation`.
|
||||
|
||||
`planFleetAgentMutation` is deterministic and side-effect free. `executeFleetAgentMutation` validates the complete proposed roster through the existing structural and shared persona resolver, prepares generated/local/quarantine projections, and writes the roster authority atomically before applying derived projections. Equivalent create retries and delete requests for an already-absent agent are idempotent no-ops.
|
||||
|
||||
Delete removes only the exact `<name>.env.generated` projection for the removed roster entry. Operator-owned `<name>.env.local`, legacy `<name>.env`, quarantine records, and unrelated projections remain untouched. An already-absent generated projection is treated as stale derived state, not as a failed mutation.
|
||||
|
||||
## Result and recovery
|
||||
|
||||
Mutation results are JSON-safe objects with `applied`, `authoritativeRoster`, `projections`, `plan`, and—only if a derived projection write fails after the authoritative roster write—a recovery object. `applied` is true only when every roster and derived-projection write completed. The explicit state fields prevent a partial result from being mistaken for a rollback or a no-op:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "committed",
|
||||
"projections": "incomplete",
|
||||
"recovery": {
|
||||
"code": "projection-apply-failed",
|
||||
"action": "regenerate-projections-from-roster"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Dry-runs and idempotent no-ops report `authoritativeRoster: "unchanged"` and `projections: "not-applied"`; a complete mutation reports `"committed"` and `"complete"`. Recovery output identifies the authoritative roster path and regeneration action only. It never contains generated/local/quarantine values, credentials, or command text. A recovery result exits non-zero because the authoritative roster was persisted but derived projections require regeneration. Regenerate projections from the roster before attempting another mutation.
|
||||
27
docs/fleet/reference/cli.md
Normal file
27
docs/fleet/reference/cli.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# Fleet Control-Plane CLI
|
||||
|
||||
The local roster-v2 control plane is `mosaic fleet`.
|
||||
|
||||
```text
|
||||
mosaic fleet apply --expected-generation <n> [--dry-run]
|
||||
mosaic fleet reconcile --expected-generation <n> [--dry-run]
|
||||
mosaic fleet start [name] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet stop [name] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet restart [name] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet status [name]
|
||||
mosaic fleet verify
|
||||
mosaic fleet doctor
|
||||
mosaic fleet migrate-v1 preview --source <path> --decisions <path> --observations <path>
|
||||
```
|
||||
|
||||
`migrate-v1 preview` is non-mutating: it emits value-free v1 inventory, a canonical semantically
|
||||
validated v2 candidate when ready, sanitized environment dispositions, and non-executable recovery
|
||||
evidence. It has no write, apply, canary, or rollback option. Missing preview inputs also return one stable
|
||||
blocked JSON object and a non-zero exit, rather than Commander text. See
|
||||
[the migration preview contract](../migration/v1-to-v2.md).
|
||||
|
||||
`apply` and `reconcile` use roster desired state. `start`, `stop`, and `restart` are exact local one-shot lifecycle effects and never persist a desired-state edit. `status`, `verify`, and `doctor` are observational.
|
||||
|
||||
Commands emit one JSON object. Handled precondition errors emit `{ "error": { "code": "..." } }` and exit non-zero. Partial derived/lifecycle effects use explicit `authoritativeRoster`, `projections`, `lifecycle`, and bounded `recovery` fields; they never claim rollback. Any additive `cleanup` diagnostic also exits non-zero, even where known effects are complete: it is not a clean completion and the lock requires inspection before retry.
|
||||
|
||||
This control plane is separate from the gateway-backed `mosaic agent` catalog. It is local-only and rejects remote/connector lifecycle mutation, arbitrary command/channel/secret input, and unproven tmux ownership.
|
||||
96
docs/fleet/reference/generated-env-boundary.md
Normal file
96
docs/fleet/reference/generated-env-boundary.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Fleet Generated Environment Boundary
|
||||
|
||||
**Card:** FCM-M2-001 · **Issue:** #758 · **Status:** unreleased/card-local
|
||||
|
||||
The local fleet roster is the desired-state authority. A launch reads a deterministic,
|
||||
roster-derived generated projection and an optional strictly data-only local file; neither file is
|
||||
a second roster or a command configuration surface.
|
||||
|
||||
## Paths and ownership
|
||||
|
||||
For agent `<name>` under `<MOSAIC_HOME>/fleet/agents/`:
|
||||
|
||||
| Path | Owner | Purpose |
|
||||
| ----------------------- | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `<name>.env.generated` | Mosaic projection writer | Complete deterministic launch data rendered from the authoritative roster. |
|
||||
| `<name>.env.local` | Operator | Optional, constrained local machine data. It cannot shadow generated keys. |
|
||||
| `<name>.env` | Legacy input only | Read once during projection generation, then regenerated/relocated or privately quarantined. It is never a launch authority. |
|
||||
| `<name>.env.quarantine` | Mosaic quarantine | Mode-`0600` private record of forbidden legacy input; it is never read by the launcher. |
|
||||
|
||||
The systemd templates do not load either environment file. They invoke Bash with a fixed, cleared
|
||||
bootstrap environment; the launcher reads and validates `.env.generated` and `.env.local` itself before
|
||||
it queries, creates, or stops an exact tmux session. It does not `source`, `eval`, or execute an
|
||||
environment-supplied command. Exact stop derives its socket from the same validated generated projection,
|
||||
not from systemd or ambient environment data.
|
||||
|
||||
All projection, local, and quarantine files must be regular files with no group or world permissions.
|
||||
The agent environment directory must also be a real, non-symlink private directory; it is validated
|
||||
before either environment file is read or tmux is queried. Unsafe paths, symlinks, or permissions fail
|
||||
closed. Diagnostics identify only a rule code, key name, and SHA-256 content hash; they never print
|
||||
values, credential material, or command text.
|
||||
|
||||
## Allowed data
|
||||
|
||||
`.env.generated` is complete and ordered exactly as follows:
|
||||
|
||||
```dotenv
|
||||
MOSAIC_AGENT_NAME=<roster name>
|
||||
MOSAIC_AGENT_CLASS=<roster class>
|
||||
MOSAIC_AGENT_RUNTIME=<roster runtime>
|
||||
MOSAIC_AGENT_MODEL=<roster model hint>
|
||||
MOSAIC_AGENT_REASONING=<roster reasoning>
|
||||
MOSAIC_AGENT_TOOL_POLICY=<roster tool policy>
|
||||
MOSAIC_AGENT_WORKDIR=<absolute roster work directory>
|
||||
MOSAIC_TMUX_SOCKET=<roster socket or empty>
|
||||
```
|
||||
|
||||
The generated launch contract supports only `claude`, `codex`, `opencode`, and `pi`. `fleet add`
|
||||
uses that same runtime authority and rejects any other runtime before it writes the roster or changes
|
||||
projection, local, or quarantine files. The legacy dogfood stub on its separate `mosaic-factory`
|
||||
socket remains an observability canary; it has no generated-launch adapter and cannot be added through
|
||||
this projection path.
|
||||
|
||||
`.env.local` may contain only these non-secret data keys:
|
||||
|
||||
- `MOSAIC_RUNTIME_BIN`
|
||||
- `MOSAIC_HEARTBEAT_RUN_DIR`
|
||||
- `MOSAIC_HEARTBEAT_INTERVAL`
|
||||
- `MOSAIC_CLAUDE_JSON`
|
||||
- `CLAUDE_CONFIG_DIR`
|
||||
|
||||
Local paths must be safe absolute paths and the interval must be a positive integer. Comments,
|
||||
quoted/export syntax, duplicate keys, unknown keys, generated-key shadowing, sensitive key names,
|
||||
and `MOSAIC_AGENT_COMMAND` are rejected. The launcher derives the only executable command from the
|
||||
validated runtime, model, and reasoning data; no arbitrary command compatibility path exists. When a
|
||||
Pi runtime writes a fresh `<name>.hb.native` marker, its native heartbeat remains authoritative; the
|
||||
shell sidecar resumes its `status=ok` fallback only after that marker is stale or absent.
|
||||
|
||||
## Legacy disposition
|
||||
|
||||
During projection generation, legacy roster-derived keys are regenerated from the roster. A valid
|
||||
allowed local value is relocated to `.env.local`; forbidden, malformed, duplicate, sensitive, and
|
||||
unknown legacy entries cause the legacy file to be moved to `.env.quarantine` and are represented by
|
||||
sanitized diagnostics. This is deterministic and idempotent after the legacy file has been consumed.
|
||||
|
||||
## USC interface packet
|
||||
|
||||
This card does not add a USC site file, write a USC roster, or run a site canary. The following is the
|
||||
consolidated downstream interface packet. Status is deliberately separated from checkout presence: no
|
||||
product release version has been evidenced for this interface set.
|
||||
|
||||
| Interface | Canonical public path and version | Tracker/release status | Downstream limit |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| M1 structural compiler | `parseRosterV2` in `packages/mosaic/src/fleet/roster-v2.ts`; schema `docs/fleet/reference/roster-v2.schema.json`; roster `version: 2` | FCM-M1-001 is recorded done, merged as #764 (`aa5b43b`); no released product version is asserted here. | Parse YAML/JSON and canonicalize a supplied v2 site roster without writes. |
|
||||
| M1 semantic resolver | `validateRosterV2Semantics` in `packages/mosaic/src/fleet/roster-v2.ts`; baseline `framework/fleet/roles/` plus `roles.local/` | FCM-M1-002 remains `in-progress` in `docs/TASKS.md`; unreleased. | Reuse the shared resolver only; no parallel role resolver or lifecycle action. |
|
||||
| M1 disposition evidence | `packages/mosaic/src/fleet/example-profile-dispositions.ts`; `docs/fleet/migration/example-profile-disposition.md`; retained fixture `version: 1` | FCM-M1-003 remains `not-started` in `docs/TASKS.md`; unreleased even though these checkout artifacts are inspectable. | Inspect fixture/profile/service disposition evidence only; it is not migration authorization. |
|
||||
| M2 generated boundary | `packages/mosaic/src/fleet/generated-env-boundary.ts`; generated projection contract in this document | FCM-M2-001 card-local and uncommitted; unreleased. | Render/write a roster-derived projection; local input is never authority. |
|
||||
|
||||
The canonical source remains `<MOSAIC_HOME>/fleet/roster.yaml` for the current local fleet path.
|
||||
Generated environment data is a rebuildable projection, not an operator-editable source of membership,
|
||||
runtime policy, or lifecycle state.
|
||||
|
||||
**Corrected downstream gates:** M2 supplies only parse/validation/projection evidence and does not
|
||||
permit a USC site canary, reconciliation, or lifecycle mutation. M3 must first define and validate the
|
||||
canonical local reconcile/lifecycle path. M4 then supplies preview/migration and its separate
|
||||
canary/rollback gates; only after those M3 and M4 gates may a site migration or canary be considered.
|
||||
This card authorizes none of those actions.
|
||||
14
docs/fleet/reference/lifecycle-transitions.md
Normal file
14
docs/fleet/reference/lifecycle-transitions.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Local Fleet Lifecycle Transitions
|
||||
|
||||
FCM-M3-001 uses the roster-v2 `lifecycle.enabled` and `lifecycle.desired_state` fields as the only desired-state authority. Systemd, tmux, generated environment files, and heartbeats are derived or observed state.
|
||||
|
||||
| Command | Desired-state write | Runtime effect | Preconditions |
|
||||
| --------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `fleet apply` / `fleet reconcile` | Never | Rebuilds projections, then starts only enabled agents desired `running`; stops disabled or desired-`stopped` roster agents | Current generation; private managed paths; valid projections; proven holder ownership; no unmanaged named-socket sessions |
|
||||
| `fleet start <name>` | Never | One-shot exact `mosaic-agent@<name>.service` start | Current generation; exact enabled roster name; proven ownership |
|
||||
| `fleet stop <name>` | Never | One-shot exact service stop | Current generation; exact roster name; proven ownership |
|
||||
| `fleet restart <name>` | Never | One-shot exact service restart | Current generation; exact roster name; proven ownership |
|
||||
|
||||
A stopped roster agent is never started by `apply` or `reconcile`. Direct lifecycle commands are explicit one-shot actions and do not change persisted desired state. Use roster CRUD with the explicit persisted-start option to change that desired state.
|
||||
|
||||
All mutations require `--expected-generation <n>` and acquire one private roster-adjacent reconciliation lock before projection or lifecycle effects. Missing or stale generations and concurrent writers fail before effects; the lock is released after success, partial failure, or thrown lifecycle failure. Stale, ownership, unmanaged-session, unsupported-runtime, path, projection, and lifecycle-precondition failures return stable redacted JSON errors and a non-zero exit. No command targets a fuzzy tmux name, arbitrary socket, arbitrary command, channel, secret, or generated file as authority.
|
||||
45
docs/fleet/reference/role-classes.md
Normal file
45
docs/fleet/reference/role-classes.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Fleet Role Classes and Authority
|
||||
|
||||
A fleet role class is a machine identity resolved from the persona library. Resolution uses the
|
||||
canonical class before consulting the baseline `fleet/roles/` and operator `fleet/roles.local/`
|
||||
layers. A readable role contract is required; an index entry alone is not semantic success.
|
||||
|
||||
## Canonicalization
|
||||
|
||||
Only these legacy class aliases are recognized:
|
||||
|
||||
| Requested class | Canonical class |
|
||||
| ---------------------- | --------------- |
|
||||
| `implementer` | `code` |
|
||||
| `reviewer` | `review` |
|
||||
| `operator-interaction` | `interaction` |
|
||||
|
||||
No other alias is inferred. In particular, `worker`, `analyst`, and `canary` are custom classes only
|
||||
when an operator supplies a readable contract for that exact class. Tess and Ultron are instance
|
||||
names, not classes. `agents[].alias` is display-only and cannot grant authority.
|
||||
|
||||
Canonicalization happens before role lookup. For example, requesting `implementer` resolves
|
||||
`code.md`; a separate `roles.local/implementer.md` cannot redefine the legacy alias. A canonical
|
||||
`roles.local/code.md` still overrides the baseline `roles/code.md` contract.
|
||||
|
||||
## Protected authority
|
||||
|
||||
Protected authority is immutable metadata derived only from canonical class. Role prose, instance
|
||||
name, display alias, tool policy, runtime, and custom role files cannot grant it.
|
||||
|
||||
| Canonical class | Granted authority | Explicit limits |
|
||||
| ----------------- | -------------------------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| `merge-gate` | Sole approve-to-land and merge authority | No authority is inferred by similarly named custom roles or policies. |
|
||||
| `validator` | May issue a validation certificate | Cannot approve-to-land or merge. |
|
||||
| `orchestrator` | May orchestrate, manage topology, and issue leases | Cannot approve-to-land or merge. |
|
||||
| `team-leader` | May use orchestrator-leased capacity | Cannot issue leases or mutate roster, configuration, credentials, or merge state. |
|
||||
| `interaction` | Request and status surface | Cannot orchestrate, issue leases, mutate roster/configuration, or merge. |
|
||||
| all other classes | No protected authority implicitly | Custom contracts do not acquire protected powers from prose. |
|
||||
|
||||
Roster-v2 semantic validation requires a protected class and its canonical tool policy to match. It
|
||||
also rejects an unprotected class paired with a protected tool policy. The legacy tool-policy name
|
||||
`operator-interaction` canonicalizes to `interaction`.
|
||||
|
||||
This mapping describes authority metadata only. Lease issuance, validation-certificate storage or
|
||||
workflow, lifecycle reconciliation, credentials, roster mutation, and merge execution are outside
|
||||
this resolver contract.
|
||||
@@ -63,8 +63,8 @@ agents:
|
||||
## Nested fields
|
||||
|
||||
| Path | Required | Constraint |
|
||||
| ---------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------- |
|
||||
| `tmux.socket_name` | yes | non-empty `[A-Za-z0-9_.-]+`; an explicit named socket prevents default-versus-named socket ambiguity |
|
||||
| ---------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `tmux.socket_name` | yes | `[A-Za-z0-9_.-]*`; empty string means the literal default tmux server, while a non-empty value names a socket |
|
||||
| `tmux.holder_session` | yes | non-empty `[A-Za-z0-9_.-]+` |
|
||||
| `defaults.working_directory` | yes | non-empty string |
|
||||
| `defaults.runtime` | yes | `claude`, `codex`, `opencode`, or `pi`; it must be declared in `runtimes` |
|
||||
@@ -81,6 +81,35 @@ agents:
|
||||
| `agents[].lifecycle.desired_state` | yes | `running` or `stopped` |
|
||||
| `agents[].launch.yolo` | yes | boolean; structured data only, not an arbitrary command escape hatch |
|
||||
|
||||
## Semantic handoff
|
||||
|
||||
`parseRosterV2` and `normalizeRosterV2` remain synchronous and structural. After structural success,
|
||||
call the asynchronous `validateRosterV2Semantics` handoff before using persona identity or authority.
|
||||
That validator batches the baseline `fleet/roles/` and operator `fleet/roles.local/` scans, then
|
||||
delegates every agent to the shared persona resolver.
|
||||
|
||||
Semantic validation:
|
||||
|
||||
- requires the winning role contract to be readable and non-empty; `LIBRARY.md` membership alone does
|
||||
not resolve a class;
|
||||
- retains `requestedClass` separately from `canonicalClass` in typed output;
|
||||
- canonicalizes only `implementer` to `code`, `reviewer` to `review`, and
|
||||
`operator-interaction` to `interaction`;
|
||||
- canonicalizes `tool_policy` with the same exact alias table;
|
||||
- rejects protected class/tool-policy mismatches in either direction, while accepting
|
||||
`class: operator-interaction` with `tool_policy: operator-interaction` as canonical
|
||||
`interaction`;
|
||||
- derives immutable protected authority only from canonical class; and
|
||||
- accepts custom baseline or `roles.local` classes without granting protected authority.
|
||||
|
||||
`agents[].alias` remains display-only. Tess and Ultron are instance names, never semantic classes.
|
||||
Canonicalization happens before role-layer lookup, so a legacy-named override cannot redefine an
|
||||
alias as separate authority. See [Role Classes and Authority](./role-classes.md) and
|
||||
[Customize Fleet Roles](../how-to/customize-roles.md).
|
||||
|
||||
This handoff performs no filesystem, systemd, tmux, roster, credential, lease, certificate, or
|
||||
lifecycle mutation.
|
||||
|
||||
## Fail-closed boundary
|
||||
|
||||
Every object is `additionalProperties: false`. The compiler rejects unknown, missing, malformed,
|
||||
|
||||
@@ -24,7 +24,7 @@
|
||||
"properties": {
|
||||
"socket_name": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9_.-]+$"
|
||||
"pattern": "^[A-Za-z0-9_.-]*$"
|
||||
},
|
||||
"holder_session": {
|
||||
"type": "string",
|
||||
|
||||
13
docs/fleet/reference/status-and-drift.md
Normal file
13
docs/fleet/reference/status-and-drift.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Local Fleet Status and Drift
|
||||
|
||||
`mosaic fleet status [name]`, `verify`, and `doctor` are observational roster-v2 commands. They emit one JSON result and do not write projections, change desired state, start services, stop services, restart services, or mutate tmux.
|
||||
|
||||
The report distinguishes:
|
||||
|
||||
- `missing-session`: an enabled agent desired `running` has no exact roster-named tmux session.
|
||||
- `unexpected-session`: a desired-`stopped` agent still has its exact session.
|
||||
- `disabled-running`: a disabled roster agent has its exact session.
|
||||
- `unmanagedSessions`: sessions on the configured named socket that are neither the exact holder nor an exact roster agent.
|
||||
- `holder`: `owned`, `missing`, or `ownership-mismatch` after exact holder, private install identity, and complete global tmux environment validation.
|
||||
|
||||
`doctor` and `status` classify rather than adopt, destroy, or repair unmanaged state. `verify` is observational too, but exits non-zero if ownership cannot be proven, unmanaged sessions exist, or drift is present. Reconciliation fails closed under those conditions and never kills or adopts an unmanaged session.
|
||||
@@ -224,9 +224,9 @@ external clients. Authentication requires a valid BetterAuth session (cookie or
|
||||
### Required
|
||||
|
||||
| Variable | Description |
|
||||
| -------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| `BETTER_AUTH_SECRET` | Secret key for BetterAuth session signing. Must be set or gateway will not start. |
|
||||
| `DATABASE_URL` | PostgreSQL connection string. Default: `postgresql://mosaic:mosaic@localhost:5433/mosaic` |
|
||||
| `DATABASE_URL` | Runtime-only PostgreSQL connection injected from the dedicated deployment secret; no default or inline DSN. |
|
||||
|
||||
### Gateway
|
||||
|
||||
@@ -294,7 +294,7 @@ Each OIDC provider requires its client ID, client secret, and issuer URL togethe
|
||||
### Plugins
|
||||
|
||||
| Variable | Description |
|
||||
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `DISCORD_BOT_TOKEN` | Discord bot token (enables Discord plugin) |
|
||||
| `DISCORD_SERVICE_TOKEN` | Required high-entropy service credential used to authenticate and sign Discord ingress; inject through the approved secret mechanism only |
|
||||
| `DISCORD_SERVICE_USER_ID` | Required Mosaic service-principal user ID that owns persisted Discord conversations; the original Discord user ID remains audit metadata |
|
||||
@@ -303,6 +303,9 @@ Each OIDC provider requires its client ID, client secret, and issuer URL togethe
|
||||
| `DISCORD_ALLOWED_GUILD_IDS` | Required comma-separated Discord guild snowflake allowlist; default-deny |
|
||||
| `DISCORD_ALLOWED_CHANNEL_IDS` | Required comma-separated Discord channel snowflake allowlist; default-deny |
|
||||
| `DISCORD_ALLOWED_USER_IDS` | Required comma-separated Discord user snowflake allowlist; default-deny |
|
||||
| `DISCORD_INTERACTION_BINDINGS` | Required JSON bindings from guild/channel to logical agent and paired Discord users with `viewer`, `operator`, or `admin` roles |
|
||||
| `DISCORD_MESSAGE_RATE_LIMIT_PER_MINUTE` | Optional positive integer; authorized turns per guild/channel/user each minute (default: `30`) |
|
||||
| `DISCORD_THREAD_RATE_LIMIT_PER_MINUTE` | Optional positive integer; mention-thread routes per guild/channel/user each minute (default: `5`) |
|
||||
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (enables Telegram plugin) |
|
||||
| `TELEGRAM_GATEWAY_URL` | Gateway URL for Telegram plugin to call |
|
||||
|
||||
@@ -310,7 +313,44 @@ Each OIDC provider requires its client ID, client secret, and issuer URL togethe
|
||||
|
||||
When `DISCORD_BOT_TOKEN` is configured, `DISCORD_SERVICE_TOKEN`, `DISCORD_SERVICE_USER_ID`, and all three Discord allowlists are required. Gateway startup fails rather than enabling a broad or unauthenticated remote-control surface. The service user ID identifies a provisioned Mosaic service principal for persistence; the original Discord user ID is retained in ingress audit metadata. The service token is a secret supplied by the approved runtime secret mechanism and is never committed or logged.
|
||||
|
||||
Inbound Discord messages must originate from an allowed guild, channel, and user, mention the bot, and carry a signed envelope containing the native Discord message ID and a generated correlation ID. The gateway validates the service identity, envelope signature, and allowlists again before dispatching. Replayed Discord message IDs are rejected during the bounded ingress replay window. Durable inbox/idempotency retention is introduced with Tess durable state.
|
||||
Inbound Discord messages must originate from an allowed guild and configured parent channel, come from an allowed and paired user whose role permits sending, and carry a signed envelope containing the native Discord message ID and a generated correlation ID. Attachment references are limited in count, metadata size, field length, and declared size; only query-free HTTPS URLs without credentials or fragments are accepted, so bearer or presigned URLs never reach persistence or an agent prompt. The gateway validates the service identity, envelope signature, allowlists, pairing, and role again before dispatching. Replayed Discord message IDs are rejected during the bounded ingress replay window. Durable inbox/idempotency retention is introduced with Tess durable state.
|
||||
|
||||
Configured channels are dedicated agent interaction surfaces. An authorized untagged message routes to the bound logical agent and the response returns in that channel. Mentioning the bot on a normal channel message creates a public Discord thread, or reuses the thread already attached to that same message; the response and later thread messages stay in that thread without repeated mentions. A normal channel's category is not an authorization parent—only a Discord thread inherits authorization from its configured parent channel. Runtime control commands such as `/approve` and `/stop <approval>` remain on the current channel/thread because they target that durable session rather than opening a new topic.
|
||||
|
||||
Authorization and per-user/channel rate limits are evaluated before thread creation, so an unlisted guild/channel/user, unpaired user, `viewer`, or rate-limited sender cannot create bot threads or dispatch gateway work. The bot needs Discord permissions to view/send in configured channels and create/send in public threads. If thread creation fails, the turn is not dispatched because the requested response destination cannot be honored.
|
||||
|
||||
Conversation handles use the configured logical agent plus Discord channel/thread identity. They do not contain a Claude, Codex, Pi, OpenCode, model, process, or runtime-provider identifier; changing the runtime behind the logical session therefore does not require reconnecting the Discord bot.
|
||||
|
||||
#### Interaction binding format
|
||||
|
||||
`DISCORD_INTERACTION_BINDINGS` must be a non-empty JSON array. Each item requires `instanceId`, trusted `agentConfigId`, `guildId`, `channelId`, and a non-empty `pairedUsers` object. `instanceId` is the configured logical-agent name; its trusted database `agentConfigId` must resolve to an agent configuration with exactly that name, preserving provider/model/prompt/tool selection per binding. The guild/channel must also appear in their corresponding allowlists. IDs below are placeholders:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"instanceId": "interaction-agent",
|
||||
"agentConfigId": "agent-config-id",
|
||||
"guildId": "guild-id",
|
||||
"channelId": "channel-id",
|
||||
"pairedUsers": {
|
||||
"discord-user-id": {
|
||||
"role": "operator",
|
||||
"mosaicUserId": "mosaic-user-id"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
| Pairing role | Send message | Create/continue thread | Approve | Stop |
|
||||
| ------------ | ------------ | ---------------------- | ------- | ---- |
|
||||
| `viewer` | No | No | No | No |
|
||||
| `operator` | Yes | Yes | No | No |
|
||||
| `admin` | Yes | Yes | Yes | Yes |
|
||||
|
||||
A legacy role-only value such as `"discord-user-id": "operator"` remains valid for non-privileged ingress. Approval and stop require the object form with a provisioned `mosaicUserId`; gateway policy checks that Mosaic identity and consumes one exact-action approval once. Do not make the Discord service principal an approving administrator.
|
||||
|
||||
After changing bindings or allowlists, restart the gateway/plugin through the normal service manager and verify both Discord and gateway connectivity. The adapter reports `connected` only when both links are ready, `degraded` when one is ready, and `disconnected` when neither is ready. Test one authorized untagged channel turn, one mention-created thread, one thread follow-up, and one unauthorized user denial without using production credential values in logs or evidence.
|
||||
|
||||
### Session retention and garbage collection
|
||||
|
||||
|
||||
@@ -1,384 +1,67 @@
|
||||
# Deployment Guide
|
||||
|
||||
This guide covers deploying Mosaic in two modes: **Docker Compose** (recommended for quick setup) and **bare-metal** (production, full control).
|
||||
> **Status: non-operative for PostgreSQL, federated, and bare-metal production.** The checked-in
|
||||
> Compose PostgreSQL service mounts legacy initialization SQL and the KBN-101 bootstrap, runner,
|
||||
> secret-renderer, and process-exec interfaces do not exist yet. This page does not authorize a
|
||||
> production deployment, database initialization, manual DDL, secret provisioning, or service
|
||||
> activation.
|
||||
|
||||
---
|
||||
## Current safe local route
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Dependency | Minimum version | Notes |
|
||||
| ---------------- | --------------- | ---------------------------------------------- |
|
||||
| Node.js | 22 LTS | Required for ESM + `--experimental-vm-modules` |
|
||||
| pnpm | 9 | `npm install -g pnpm` |
|
||||
| PostgreSQL | 17 | Must have the `pgvector` extension |
|
||||
| Valkey | 8 | Redis-compatible; Redis 7+ also works |
|
||||
| Docker + Compose | v2 | For the Docker Compose path only |
|
||||
|
||||
---
|
||||
|
||||
## Docker Compose Deployment (Quick Start)
|
||||
|
||||
The `docker-compose.yml` at the repository root starts PostgreSQL 17 (with pgvector), Valkey 8, an OpenTelemetry Collector, and Jaeger.
|
||||
|
||||
### 1. Clone and configure
|
||||
Use PGlite only for current in-process data-layer work; it requires no PostgreSQL. A Gateway/Web
|
||||
local process is held because its unguarded dotenv loader can inherit a daemon PostgreSQL DSN and
|
||||
reach runtime DDL. If a local queue service is useful, start only Valkey:
|
||||
|
||||
```bash
|
||||
git clone <repo-url> mosaic
|
||||
cd mosaic
|
||||
cp .env.example .env
|
||||
docker compose up -d valkey
|
||||
```
|
||||
|
||||
Edit `.env`. The minimum required change is:
|
||||
This command intentionally does not start PostgreSQL. Do not run a broad Compose start, use its
|
||||
PostgreSQL initialization mount, infer that current Compose is a production/federated route, or
|
||||
start Gateway/Web until KBN-101-02 supplies fail-closed local-tier/DSN isolation.
|
||||
|
||||
```dotenv
|
||||
BETTER_AUTH_SECRET=<output of: openssl rand -base64 32>
|
||||
```
|
||||
## Held future procedure
|
||||
|
||||
### 2. Start infrastructure services
|
||||
PostgreSQL local, federated, Compose, and bare-metal production activation are held until these
|
||||
artifacts land and pass their independent gates:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
1. **KBN-101-00** external privileged bootstrap artifact;
|
||||
2. **KBN-101-03** sole `mosaic-db-migrator` runner and verified-readiness artifact; and
|
||||
3. **KBN-101-05** Vault/secret-renderer-backed deployment and consumer-isolation artifact.
|
||||
|
||||
Services and their ports:
|
||||
The required future order is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
| Service | Default port |
|
||||
| --------------------- | ------------------------ |
|
||||
| PostgreSQL | `localhost:5433` |
|
||||
| Valkey | `localhost:6380` |
|
||||
| OTEL Collector (HTTP) | `localhost:4318` |
|
||||
| OTEL Collector (gRPC) | `localhost:4317` |
|
||||
| Jaeger UI | `http://localhost:16686` |
|
||||
This is a held, non-operative future activation specification with no current command authority. Do not invoke the named
|
||||
runner, start PostgreSQL, or substitute a Compose/init/manual-SQL route until the owned artifacts
|
||||
are implemented and reviewed.
|
||||
|
||||
Override host ports via `PG_HOST_PORT` and `VALKEY_HOST_PORT` in `.env` if the defaults conflict.
|
||||
## Future production secret and unit boundary (schematic only)
|
||||
|
||||
### 3. Install dependencies
|
||||
No current bare-metal production unit or command is published. KBN-101-05 must supply a reviewed,
|
||||
generation-pinned Vault renderer and a process-exec or systemd `LoadCredential` interface before
|
||||
production units can exist. The interface must preserve these exact consumer boundaries:
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
| Consumer | May receive | Must never receive |
|
||||
| ----------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| Gateway/runtime | Its own runtime URL and DB client CA at process exec | Migrator URL, importer URL/version, attestation material, signing key, PostgreSQL private key |
|
||||
| One-shot migrator | Its own migration URL, DB client CA, and runner-only signing capability | Runtime URL, importer consumer copy, Gateway/private PostgreSQL keys |
|
||||
| Data importer | Its own immutable URL/version copies, importer CA, pinned public key, and sealed attestation | Runtime/migrator URLs, signing key, shared writable mount |
|
||||
| PostgreSQL | Its own server certificate/key and only its approved server material | Application, migrator, importer, or Gateway secrets |
|
||||
|
||||
### 4. Initialize the database
|
||||
A future unit specification is non-executable until KBN-101-05 supplies it. It must obtain
|
||||
credentials through the renderer’s Vault generation and process-exec/`LoadCredential` boundary;
|
||||
it must not place credentials in a production environment file, a monorepo auto-load path, a shell
|
||||
export, command arguments, logs, or a manual secret-activation lifecycle instruction. Rotation and
|
||||
process replacement semantics must be delivered by the reviewed renderer/interface with generation,
|
||||
consumer-isolation, mode/owner, and no-mixed-generation evidence—not improvised in this guide.
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:migrate
|
||||
```
|
||||
## Readiness and troubleshooting status
|
||||
|
||||
### 5. Build all packages
|
||||
Until the future procedure is implemented, do not diagnose PostgreSQL with ad hoc SQL, connection
|
||||
strings, or initialization scripts. The future sanitized runner-verification readiness artifact is
|
||||
the required PostgreSQL readiness authority after its bootstrap/TLS prerequisites pass.
|
||||
For local PGlite development, diagnose application behavior without introducing a PostgreSQL
|
||||
connection.
|
||||
|
||||
```bash
|
||||
pnpm build
|
||||
```
|
||||
|
||||
### 6. Start the gateway
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/gateway dev
|
||||
```
|
||||
|
||||
Or for production (after build):
|
||||
|
||||
```bash
|
||||
node apps/gateway/dist/main.js
|
||||
```
|
||||
|
||||
### 7. Start the web app
|
||||
|
||||
```bash
|
||||
# Development
|
||||
pnpm --filter @mosaicstack/web dev
|
||||
|
||||
# Production (after build)
|
||||
pnpm --filter @mosaicstack/web start
|
||||
```
|
||||
|
||||
The web app runs on port `3000` by default.
|
||||
|
||||
---
|
||||
|
||||
## Bare-Metal Deployment
|
||||
|
||||
Use this path when you want to manage PostgreSQL and Valkey yourself (e.g., existing infrastructure, managed cloud databases).
|
||||
|
||||
### Step 1 — Install system dependencies
|
||||
|
||||
```bash
|
||||
# Node.js 22 via nvm
|
||||
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
|
||||
nvm install 22
|
||||
nvm use 22
|
||||
|
||||
# pnpm
|
||||
npm install -g pnpm
|
||||
|
||||
# PostgreSQL 17 with pgvector (Debian/Ubuntu example)
|
||||
sudo apt-get install -y postgresql-17 postgresql-17-pgvector
|
||||
|
||||
# Valkey
|
||||
# Follow https://valkey.io/download/ for your distribution
|
||||
```
|
||||
|
||||
### Step 2 — Create the database
|
||||
|
||||
```sql
|
||||
-- Run as the postgres superuser
|
||||
CREATE USER mosaic WITH PASSWORD 'change-me';
|
||||
CREATE DATABASE mosaic OWNER mosaic;
|
||||
\c mosaic
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
```
|
||||
|
||||
### Step 3 — Clone and configure
|
||||
|
||||
```bash
|
||||
git clone <repo-url> /opt/mosaic
|
||||
cd /opt/mosaic
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `/opt/mosaic/.env`. Required fields:
|
||||
|
||||
```dotenv
|
||||
DATABASE_URL=postgresql://mosaic:<password>@localhost:5432/mosaic
|
||||
VALKEY_URL=redis://localhost:6379
|
||||
BETTER_AUTH_SECRET=<openssl rand -base64 32>
|
||||
BETTER_AUTH_URL=https://your-domain.example.com
|
||||
GATEWAY_CORS_ORIGIN=https://your-domain.example.com
|
||||
NEXT_PUBLIC_GATEWAY_URL=https://your-domain.example.com
|
||||
```
|
||||
|
||||
### Step 4 — Install dependencies and build
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
pnpm build
|
||||
```
|
||||
|
||||
### Step 5 — Run database migrations
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:migrate
|
||||
```
|
||||
|
||||
### Step 6 — Start the gateway
|
||||
|
||||
```bash
|
||||
node apps/gateway/dist/main.js
|
||||
```
|
||||
|
||||
The gateway reads `.env` from the monorepo root automatically (via `dotenv` in `main.ts`).
|
||||
|
||||
### Step 7 — Start the web app
|
||||
|
||||
```bash
|
||||
# Next.js standalone output
|
||||
node apps/web/.next/standalone/server.js
|
||||
```
|
||||
|
||||
The standalone build is self-contained; it does not require `node_modules` to be present at runtime.
|
||||
|
||||
### Step 8 — Configure a reverse proxy
|
||||
|
||||
#### Nginx example
|
||||
|
||||
```nginx
|
||||
# /etc/nginx/sites-available/mosaic
|
||||
|
||||
# Gateway API
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name your-domain.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
||||
|
||||
# WebSocket support (for chat.gateway.ts / Socket.IO)
|
||||
location /socket.io/ {
|
||||
proxy_pass http://127.0.0.1:14242;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
|
||||
# REST + auth
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:14242;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
|
||||
# Web app (optional — serve on a subdomain or a separate server block)
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name app.your-domain.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Caddy example
|
||||
|
||||
```caddyfile
|
||||
# /etc/caddy/Caddyfile
|
||||
|
||||
your-domain.example.com {
|
||||
reverse_proxy /socket.io/* localhost:14242 {
|
||||
header_up Upgrade {http.upgrade}
|
||||
header_up Connection {http.connection}
|
||||
}
|
||||
reverse_proxy localhost:14242
|
||||
}
|
||||
|
||||
app.your-domain.example.com {
|
||||
reverse_proxy localhost:3000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### systemd Services
|
||||
|
||||
Create a service unit for each process.
|
||||
|
||||
**Gateway** — `/etc/systemd/system/mosaic-gateway.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Mosaic Gateway
|
||||
After=network.target postgresql.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=mosaic
|
||||
WorkingDirectory=/opt/mosaic
|
||||
EnvironmentFile=/opt/mosaic/.env
|
||||
ExecStart=/usr/bin/node apps/gateway/dist/main.js
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
**Web app** — `/etc/systemd/system/mosaic-web.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Mosaic Web App
|
||||
After=network.target mosaic-gateway.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=mosaic
|
||||
WorkingDirectory=/opt/mosaic/apps/web
|
||||
EnvironmentFile=/opt/mosaic/.env
|
||||
ExecStart=/usr/bin/node .next/standalone/server.js
|
||||
Environment=PORT=3000
|
||||
Environment=HOSTNAME=127.0.0.1
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Enable and start:
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now mosaic-gateway mosaic-web
|
||||
```
|
||||
|
||||
### Log Management
|
||||
|
||||
Gateway and web app logs go to systemd journal by default. View with:
|
||||
|
||||
```bash
|
||||
journalctl -u mosaic-gateway -f
|
||||
journalctl -u mosaic-web -f
|
||||
```
|
||||
|
||||
Rotate logs by configuring `journald` in `/etc/systemd/journald.conf`:
|
||||
|
||||
```ini
|
||||
SystemMaxUse=500M
|
||||
MaxRetentionSec=30day
|
||||
```
|
||||
|
||||
### Security Checklist
|
||||
|
||||
- Set `BETTER_AUTH_SECRET` to a cryptographically random value (`openssl rand -base64 32`).
|
||||
- Restrict `GATEWAY_CORS_ORIGIN` to your exact frontend origin — do not use `*`.
|
||||
- Run services as a dedicated non-root system user (e.g., `mosaic`).
|
||||
- Firewall: only expose ports 80/443 externally; keep 14242 and 3000 bound to `127.0.0.1`.
|
||||
- Set `AGENT_FILE_SANDBOX_DIR` to a directory outside the application root to prevent agent tools from accessing source code.
|
||||
- If using `AGENT_USER_TOOLS`, enumerate only the tools non-admin users need.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Gateway fails to start — "BETTER_AUTH_SECRET is required"
|
||||
|
||||
`BETTER_AUTH_SECRET` is missing or empty. Set it in `.env` and restart.
|
||||
|
||||
### `DATABASE_URL` connection refused
|
||||
|
||||
Verify PostgreSQL is running and the port matches. The Docker Compose default is `5433`; bare-metal typically uses `5432`.
|
||||
|
||||
```bash
|
||||
psql "$DATABASE_URL" -c '\conninfo'
|
||||
```
|
||||
|
||||
### pgvector extension missing
|
||||
|
||||
```sql
|
||||
\c mosaic
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
```
|
||||
|
||||
### Valkey / Redis connection refused
|
||||
|
||||
Check the URL in `VALKEY_URL`. The Docker Compose default is port `6380`.
|
||||
|
||||
```bash
|
||||
redis-cli -u "$VALKEY_URL" ping
|
||||
```
|
||||
|
||||
### WebSocket connections fail in production
|
||||
|
||||
Ensure your reverse proxy forwards the `Upgrade` and `Connection` headers. See the Nginx/Caddy examples above.
|
||||
|
||||
### Ollama models not appearing
|
||||
|
||||
Set `OLLAMA_BASE_URL` to the URL where Ollama is running (e.g., `http://localhost:11434`) and set `OLLAMA_MODELS` to a comma-separated list of model IDs you have pulled.
|
||||
|
||||
```bash
|
||||
ollama pull llama3.2
|
||||
```
|
||||
|
||||
### OTEL traces not appearing in Jaeger
|
||||
|
||||
Verify the collector is reachable at `OTEL_EXPORTER_OTLP_ENDPOINT`. With Docker Compose the default is `http://localhost:4318`. Check `docker compose ps` and `docker compose logs otel-collector`.
|
||||
|
||||
### Summarization / embedding features not working
|
||||
|
||||
These features require `OPENAI_API_KEY` to be set, or you must point `SUMMARIZATION_API_URL` / `EMBEDDING_API_URL` to an OpenAI-compatible endpoint (e.g., a local Ollama instance with an embeddings model).
|
||||
Non-database local services may be inspected with their ordinary local health/log tools. Those
|
||||
checks do not certify PostgreSQL, federated deployment, or production readiness.
|
||||
|
||||
@@ -8,8 +8,9 @@
|
||||
4. [Adding New Agent Tools](#adding-new-agent-tools)
|
||||
5. [Adding New MCP Tools](#adding-new-mcp-tools)
|
||||
6. [Database Schema and Migrations](#database-schema-and-migrations)
|
||||
7. [API Endpoint Reference](#api-endpoint-reference)
|
||||
8. [Local Fleet Canary](./fleet-local-canary.md)
|
||||
7. [Claude Code Skill Bridge](#claude-code-skill-bridge)
|
||||
8. [API Endpoint Reference](#api-endpoint-reference)
|
||||
9. [Local Fleet Canary](./fleet-local-canary.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -39,7 +40,7 @@ mosaic-mono-v1/
|
||||
│ ├── queue/ # Valkey-backed task queue
|
||||
│ └── types/ # Shared TypeScript types
|
||||
├── docker/ # Dockerfile(s) for containerized deployment
|
||||
├── infra/ # Infra config (OTEL collector, pg-init scripts)
|
||||
├── infra/ # Infrastructure configuration (for example, OTEL collector)
|
||||
├── docker-compose.yml # Local services (Postgres, Valkey, OTEL, Jaeger)
|
||||
└── CLAUDE.md # Project conventions for AI coding agents
|
||||
```
|
||||
@@ -86,71 +87,54 @@ cd mosaic-mono-v1
|
||||
pnpm install
|
||||
```
|
||||
|
||||
### 2. Start Infrastructure Services
|
||||
### 2. Use the local PGlite tier
|
||||
|
||||
The supported local tier is in-process PGlite and requires no PostgreSQL service. Leave
|
||||
`DATABASE_URL` unset for this route. Its default local configuration uses PGlite and performs no
|
||||
external database probe.
|
||||
|
||||
If a local queue service is useful, start only that non-PostgreSQL service:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
docker compose up -d valkey
|
||||
```
|
||||
|
||||
This starts:
|
||||
Do not use the current Compose PostgreSQL service: it mounts legacy `infra/pg-init` SQL and is
|
||||
not qualified for KBN-101. Start OTEL Collector or Jaeger individually only when needed and
|
||||
without starting PostgreSQL.
|
||||
|
||||
| Service | Port | Description |
|
||||
| ------------------------ | -------------- | -------------------- |
|
||||
| PostgreSQL 17 + pgvector | `5433` (host) | Primary database |
|
||||
| Valkey 8 | `6380` (host) | Queue and cache |
|
||||
| OpenTelemetry Collector | `4317`, `4318` | OTEL gRPC and HTTP |
|
||||
| Jaeger | `16686` | Distributed trace UI |
|
||||
### 3. Gateway/Web local process (held)
|
||||
|
||||
### 3. Configure Environment
|
||||
Do not start the current Gateway or web process as a local PGlite route. Gateway first loads the
|
||||
daemon configuration and then project environment files without a tier guard; a pre-existing
|
||||
`DATABASE_URL` can select PostgreSQL, where current startup still reaches runtime DDL/migrations.
|
||||
Creating a root `.env` that omits `DATABASE_URL` does not make this safe, so neither a local
|
||||
credential file nor a web environment file is a current developer procedure.
|
||||
|
||||
Create a `.env` file in the monorepo root:
|
||||
PGlite remains the supported in-process data-layer implementation, and the optional Valkey command
|
||||
above remains safe because it does not start PostgreSQL. A safe Gateway/Web local procedure is held
|
||||
until KBN-101-02 rejects a daemon, inherited, root, or app-local PostgreSQL DSN and any non-local
|
||||
tier before connection or DDL; KBN-101-05 then supplies the production renderer/Vault process-exec
|
||||
or `LoadCredential` boundary.
|
||||
|
||||
```env
|
||||
# Database (matches docker-compose defaults)
|
||||
DATABASE_URL=postgresql://mosaic:mosaic@localhost:5433/mosaic
|
||||
### Held future procedure
|
||||
|
||||
# Auth (required — generate a random 32+ char string)
|
||||
BETTER_AUTH_SECRET=change-me-to-a-random-secret
|
||||
PostgreSQL local and federated deployment are held until KBN-101-00 (external bootstrap),
|
||||
KBN-101-03 (runner), and KBN-101-05 (renderer-backed deployment) land. The following is the
|
||||
**held, non-operative future activation order with no current command authority**:
|
||||
|
||||
# Gateway
|
||||
GATEWAY_PORT=14242
|
||||
GATEWAY_CORS_ORIGIN=http://localhost:3000
|
||||
external bootstrap → TLS/roles → `mosaic-db-migrator --run` →
|
||||
`mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
# Web
|
||||
NEXT_PUBLIC_GATEWAY_URL=http://localhost:14242
|
||||
Neither current Compose nor this development guide authorizes PostgreSQL initialization SQL,
|
||||
manual DDL, or a pre-runner start.
|
||||
|
||||
# Optional: Ollama
|
||||
OLLAMA_BASE_URL=http://localhost:11434
|
||||
OLLAMA_MODELS=llama3.2
|
||||
```
|
||||
### 5. Gateway/Web start (held)
|
||||
|
||||
The gateway loads `.env` from the monorepo root via `dotenv` at startup
|
||||
(`apps/gateway/src/main.ts`).
|
||||
|
||||
### 4. Push the Database Schema
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:push
|
||||
```
|
||||
|
||||
This applies the Drizzle schema directly to the database (development only; use
|
||||
migrations in production).
|
||||
|
||||
### 5. Start the Gateway
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/gateway exec tsx src/main.ts
|
||||
```
|
||||
|
||||
The gateway starts on port `14242` by default.
|
||||
|
||||
### 6. Start the Web App
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/web dev
|
||||
```
|
||||
|
||||
The web app starts on port `3000` by default.
|
||||
No Gateway/Web start command is currently authorized for the local PGlite route. Do not use root
|
||||
`pnpm dev` as a workaround: it additionally starts configured integrations and cannot establish the
|
||||
required local-tier/DSN isolation. Resume this section only after KBN-101-02 provides its
|
||||
fail-closed local-startup evidence.
|
||||
|
||||
---
|
||||
|
||||
@@ -300,26 +284,13 @@ Implement a standard MCP server that exposes tools via the streamable HTTP
|
||||
transport or SSE transport. The server must accept connections at a `/mcp`
|
||||
endpoint.
|
||||
|
||||
### 2. Configure `MCP_SERVERS`
|
||||
### 2. Gateway MCP configuration (held)
|
||||
|
||||
In your `.env`:
|
||||
|
||||
```env
|
||||
MCP_SERVERS='[{"name":"my-server","url":"http://localhost:3001/mcp"}]'
|
||||
```
|
||||
|
||||
With authentication:
|
||||
|
||||
```env
|
||||
MCP_SERVERS='[{"name":"secure-server","url":"http://my-server/mcp","headers":{"Authorization":"Bearer token"}}]'
|
||||
```
|
||||
|
||||
### 3. Restart the Gateway
|
||||
|
||||
On startup, `McpClientService` (`apps/gateway/src/mcp-client/mcp-client.service.ts`)
|
||||
connects to each configured server, calls `tools/list`, and bridges the results
|
||||
to Pi SDK `ToolDefinition` format. These tools become available in all new agent
|
||||
sessions.
|
||||
Do not configure MCP endpoint credentials, write them to a local environment file, or restart the
|
||||
Gateway from this guide. Gateway/Web startup is held until KBN-101-02 supplies fail-closed
|
||||
local-tier/DSN isolation and KBN-101-05 supplies the renderer/Vault process-exec or
|
||||
`LoadCredential` secret-consumer interface. The future authenticated MCP route requires verified
|
||||
HTTPS and certificate validation; plaintext bearer-token examples are forbidden.
|
||||
|
||||
### Tool Naming
|
||||
|
||||
@@ -355,45 +326,65 @@ The schema lives in a single file:
|
||||
|
||||
The `insights` table uses a `vector(1536)` column (pgvector) for semantic search.
|
||||
|
||||
### Development: Push Schema
|
||||
### PostgreSQL schema work (held)
|
||||
|
||||
Apply schema changes directly to the dev database (no migration files created):
|
||||
Do not prepare or run a PostgreSQL target from this branch. The sole runner, bootstrap, and
|
||||
renderer are future KBN-101 artifacts, not current commands. When KBN-101-00/-03/-05 land, the
|
||||
owned activation documentation will require external bootstrap → TLS/roles → runner `--run` →
|
||||
runner `--verify` → Gateway/Compose readiness.
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:push
|
||||
```
|
||||
### Generating migration artifacts
|
||||
|
||||
### Generating Migrations
|
||||
|
||||
For production-safe, versioned changes:
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:generate
|
||||
```
|
||||
|
||||
This creates a new SQL migration file in `packages/db/drizzle/`.
|
||||
|
||||
### Running Migrations
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaicstack/db db:migrate
|
||||
```
|
||||
`pnpm --filter @mosaicstack/db db:generate` is an offline artifact-generation command. It does
|
||||
not authorize connecting to or initializing PostgreSQL. A future reviewed PostgreSQL procedure
|
||||
will determine when its output is applied.
|
||||
|
||||
### Drizzle Config
|
||||
|
||||
Config is at `packages/db/drizzle.config.ts`. The schema file path and output
|
||||
directory are defined there.
|
||||
Config is at `packages/db/drizzle.config.ts`. The schema file path and output directory are
|
||||
defined there.
|
||||
|
||||
### Adding a New Table
|
||||
|
||||
1. Add the table definition to `packages/db/src/schema.ts`.
|
||||
2. Export it from `packages/db/src/index.ts`.
|
||||
3. Run `pnpm --filter @mosaicstack/db db:push` (dev) or
|
||||
`pnpm --filter @mosaicstack/db db:generate && pnpm --filter @mosaicstack/db db:migrate`
|
||||
(production).
|
||||
3. Generate the offline artifact with `pnpm --filter @mosaicstack/db db:generate`.
|
||||
4. Do not apply it to PostgreSQL until the future KBN-101 activation artifacts and their owned
|
||||
procedure are available. Direct schema push is not a production-like workflow.
|
||||
|
||||
---
|
||||
|
||||
## Claude Code Skill Bridge
|
||||
|
||||
The framework's canonical skill root is `~/.config/mosaic/skills/`; Claude Code
|
||||
requires registrations under `~/.claude/skills/`. The implementation in
|
||||
`packages/mosaic/src/commands/skill.ts` owns only direct-child symlinks whose
|
||||
resolved target remains inside the canonical root.
|
||||
|
||||
Security invariants:
|
||||
|
||||
1. Validate the user-supplied name before filesystem access against
|
||||
`[A-Za-z0-9][A-Za-z0-9._-]*`. Separators, control characters, whitespace,
|
||||
`..`, absolute paths, and leading `-` are invalid; filesystem-derived invalid
|
||||
names are escaped before terminal output.
|
||||
2. Never replace a real file, directory, foreign symlink, or live misdirected
|
||||
symlink in the Claude skill directory.
|
||||
3. Repair a dangling link only when its lexical target is inside the canonical
|
||||
Mosaic skills root.
|
||||
4. Unregister only a symlink pointing inside that root.
|
||||
5. Enumerate canonical directories at runtime; never hardcode framework skill
|
||||
names.
|
||||
|
||||
`finalizeStage` reconciles after wizard/framework synchronization, and
|
||||
`runFrameworkReseed` reconciles after the sync-only `mosaic update` path. A
|
||||
foreign conflict is reported but does not prevent unrelated canonical skills
|
||||
from registering. Filesystem tests use injected temporary roots in
|
||||
`skill.spec.ts`, `finalize-skills.spec.ts`, and `update-checker.reseed.spec.ts`.
|
||||
|
||||
M1 intentionally manages Claude Code only. Pi's Mosaic launcher can discover the
|
||||
canonical root directly. Codex still relies on the existing full skill-sync
|
||||
linker and needs separate parity analysis before this lifecycle API is extended.
|
||||
|
||||
## API Endpoint Reference
|
||||
|
||||
All endpoints are served by the gateway at `http://localhost:14242` by default.
|
||||
|
||||
@@ -98,6 +98,39 @@ Expected results:
|
||||
that means the unit ran, not that an agent pane is live. Treat tmux
|
||||
`has-session`, `list-panes`, process tree, and logs as the liveness evidence.
|
||||
|
||||
## Recovery — rebuild generated env projections
|
||||
|
||||
Each agent's `~/.config/mosaic/fleet/agents/<name>.env.generated` is a
|
||||
deterministic projection of `roster.yaml` (the SSOT) that the launcher
|
||||
(`start-agent-session.sh`) sources at start. If an upgrade or a manual mistake
|
||||
wipes or diverges those projections, rebuild them from the roster with
|
||||
`mosaic fleet regen` — do NOT restart the affected unit first.
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # dry-run (default): show create/rebuild plan per agent
|
||||
mosaic fleet regen --json # same plan, machine-readable
|
||||
mosaic fleet regen --write # rebuild fleet/agents/<name>.env.generated on disk
|
||||
```
|
||||
|
||||
`regen` is projection-only and **never restarts an agent** — it has no path to
|
||||
systemd lifecycle. It is dry-run by default, deterministic/idempotent, uses the
|
||||
same roster→env mapping as `mosaic fleet reconcile`, and emits paths and counts
|
||||
only (never the projected `KEY=value` body). After `--write`, verify each unit
|
||||
resolves the intended values before restarting one unit at a time. The unit sets
|
||||
no `EnvironmentFile=` — `start-agent-session.sh` sources `.env.generated` itself —
|
||||
so verify the generated file directly and the launcher path, not a nonexistent
|
||||
`EnvironmentFile` property:
|
||||
|
||||
```bash
|
||||
test -f ~/.config/mosaic/fleet/agents/<name>.env.generated
|
||||
systemctl --user cat mosaic-agent@<name> | grep ExecStart
|
||||
systemctl --user restart mosaic-agent@<name>
|
||||
```
|
||||
|
||||
Full recovery runbook and the three-layer #791 protection model (manifest
|
||||
ownership → pre-update snapshot/restore → regen): see
|
||||
[Upgrade Safety & Recovery](./upgrade-safety-and-recovery.md).
|
||||
|
||||
## Release Preflight
|
||||
|
||||
Run this checklist before cutting or dogfooding a fleet release:
|
||||
|
||||
@@ -1,147 +1,98 @@
|
||||
# Migrating to the Federated Tier
|
||||
|
||||
Step-by-step guide to migrate from `local` (PGlite) or `standalone` (PostgreSQL without pgvector) to `federated` (PostgreSQL 17 + pgvector + Valkey).
|
||||
> **KBN-101-07 ownership:** This active documentation is a **non-operative KBN-101
|
||||
> contract** with no current command authority until KBN-101-00, KBN-101-02, KBN-101-03, KBN-101-05, and KBN-101-06 land and
|
||||
> KBN-101-08 activates an exact reviewed release. The commands below describe the produced interface only. Do not run them on the
|
||||
> current branch or replace them with direct PostgreSQL, raw SQL, legacy storage migration, or
|
||||
> credential-on-argv procedures.
|
||||
|
||||
## When to migrate
|
||||
## Held future procedure
|
||||
|
||||
Migrate to federated tier when:
|
||||
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||
|
||||
- Scaling from single-user to multi-user deployments
|
||||
- Adding vector embeddings or RAG features
|
||||
- Running Mosaic across multiple hosts
|
||||
- Requires distributed task queueing and caching
|
||||
- Moving to production with high availability
|
||||
The deployment control plane executes the complete held future procedure, in order: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. The
|
||||
runner is the only attestation producer after its verified TLS, identity, manifest, and schema
|
||||
checks. A data importer is never a schema bootstrap, extension installer, repair command, or DDL
|
||||
consumer.
|
||||
|
||||
## Prerequisites
|
||||
## Target material contract
|
||||
|
||||
- Federated stack running and healthy (see [Federated Tier Setup](../federation/SETUP.md))
|
||||
- Source database accessible and empty target database at the federated URL
|
||||
- Backup of source database (recommended before any migration)
|
||||
KBN-101-05 obtains the target URL from Vault KV-v2
|
||||
`secret-{env}/mosaic-stack/database/importer`, key `url`, and reads its authenticated version from
|
||||
the same successful response `data.metadata.version`. A hash or DSN byte sequence is not a
|
||||
provider version. The renderer treats URL bytes and provider version as one generation, writes a
|
||||
temporary generation directory with fsync plus atomic rename, and creates separate immutable
|
||||
consumer mounts. Swarm uses distinct versioned secret/config references. A deployment cannot mix
|
||||
generations.
|
||||
|
||||
## Dry-run first
|
||||
| Consumer | Permitted material |
|
||||
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Migrator-attestation producer (`10003:10003`) | Its own migration URL/CA; read-only `/run/secrets/mosaic-migrate-target-url` and `/run/secrets/mosaic-migrate-target-version`, each `0400`, solely to bind; producer-only attestation output at `/run/mosaic-attestations-producer/migrate-target.v1.json`; root-wrapper-only signing key. It never connects with, uses, exports, logs, or forwards the importer URL/version. |
|
||||
| Privileged deployment handoff controller | After runner success and before importer creation, it receives only root-owned non-secret expected provider-version/URL-SHA-256/generation descriptor and pinned public verifier key—not URL bytes or private key. It safe-opens/verifies descriptor and producer artifact, copies exact bytes to a new importer-only mount with fsync/atomic rename, sets `10002:10002` `0400`, seals it read-only, and refuses importer start on any partial/wrong-generation/wrong-owner/mode result. |
|
||||
| Importer (`10002:10002`) | Its own immutable `0400` copies at the same URL/version paths; CA at exact `DATABASE_TLS_CA_CERT_PATH=/run/secrets/mosaic-db-ca.crt`; pinned Ed25519 public key; read-only `/run/mosaic-attestations/migrate-target.v1.json` supplied only by the sealed handoff. |
|
||||
| Gateway/runtime/unrelated container | No importer URL/version, importer artifact, attestation private key, or unrelated CA mount. |
|
||||
|
||||
Always run a dry-run to validate the migration:
|
||||
The migrator and importer safe-open URL, provider-version, attestation, and public-key files only
|
||||
with `O_RDONLY|O_CLOEXEC|O_NOFOLLOW`; they validate from the opened fd that the file is regular,
|
||||
has its expected owner/mode and link count one. The migrator digests only that URL fd for binding,
|
||||
then zeroizes/closes it. The importer reads URL bytes once into protected memory, validates the
|
||||
signed binding and exact CA before connecting from those same bytes, then zeroizes/closes every
|
||||
fd. It neither logs nor exposes a URL/version/attestation/key oracle.
|
||||
|
||||
## Produced command interface
|
||||
|
||||
After activation and only after approved target preparation, the future interface is:
|
||||
|
||||
```bash
|
||||
# Deployment control plane has already completed the held runner procedure above.
|
||||
mosaic storage migrate-tier --to federated \
|
||||
--target-url postgresql://mosaic:mosaic@localhost:5433/mosaic \
|
||||
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
Expected output (partial example):
|
||||
The provider-version file is fixed deployment material, not argv. This connecting dry-run consumes its nonce; before an actual copy, the deployment control plane must provide fresh runner verification and a new sealed handoff. The runner uses its migration
|
||||
identity; the importer connects only as non-DDL `mosaic_data_importer` and only after all
|
||||
pre-connect validation. After verified TLS and before DML it compares PostgreSQL system ID,
|
||||
database OID, `current_user`, CA/SPKI, and manifest/schema fingerprints to the artifact.
|
||||
|
||||
```
|
||||
[migrate-tier] Analyzing source tier: pglite
|
||||
[migrate-tier] Analyzing target tier: federated
|
||||
[migrate-tier] Precondition: target is empty ✓
|
||||
users: 5 rows
|
||||
teams: 2 rows
|
||||
conversations: 12 rows
|
||||
messages: 187 rows
|
||||
... (all tables listed)
|
||||
[migrate-tier] NOTE: Source tier has no pgvector support. insights.embedding will be NULL on all migrated rows.
|
||||
[migrate-tier] DRY-RUN COMPLETE (no data written). 206 total rows would be migrated.
|
||||
```
|
||||
## Required refusals and evidence
|
||||
|
||||
Review the output. If it shows an error (e.g., target not empty), address it before proceeding.
|
||||
KBN-101-02/-03/-05/-06 must prove, with stable sanitized errors, that no target connection occurs
|
||||
for missing/unsafe URL/version/attestation/public-key files; symlink, hardlink, owner, mode, or
|
||||
TOCTOU violations; mixed URL/version generations; missing/wrong CA mount; stale/replayed/tampered
|
||||
or revoked-key artifacts; provider rotation/revocation; wrong TLS/server/database/role/manifest
|
||||
binding; raw `--target-url`; `DATABASE_URL` fallback; runtime/owner identity; consumer leakage;
|
||||
or any DDL attempt. Post-connect identity mismatch closes with zero DML/DDL. Tests also prove no
|
||||
forwarding, child environment, logging, or error oracle leaks URL/version/key/artifact contents.
|
||||
|
||||
## Run the migration
|
||||
The attestation is credential-free JCS with detached Ed25519 signature and binds issued/expiry,
|
||||
nonce, authenticated provider version, exact URL-fd SHA-256, TLS host/port/database, CA/SPKI,
|
||||
PostgreSQL system ID/database OID, importer role, manifest/schema, and producer identity. Provider
|
||||
version rotation invalidates an old artifact and requires a fresh rendered generation plus runner
|
||||
verification.
|
||||
|
||||
When ready, run without `--dry-run`:
|
||||
## Actual copy after dry-run
|
||||
|
||||
After reviewed dry-run, obtain the required fresh verification/attestation generation, then use:
|
||||
|
||||
```bash
|
||||
# Deployment control plane has supplied fresh runner verification and attestation.
|
||||
mosaic storage migrate-tier --to federated \
|
||||
--target-url postgresql://mosaic:mosaic@localhost:5433/mosaic \
|
||||
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||
--yes
|
||||
```
|
||||
|
||||
The `--yes` flag skips the confirmation prompt (required in non-TTY environments like CI).
|
||||
The dry-run artifact is terminally replayed and must be rejected; `--yes` bypasses no file,
|
||||
generation, signature, TLS, identity, or DDL control.
|
||||
|
||||
The command will:
|
||||
## Data boundary and recovery
|
||||
|
||||
1. Acquire an advisory lock (blocks concurrent invocations)
|
||||
2. Copy data from source to target in dependency order
|
||||
3. Report rows migrated per table
|
||||
4. Display any warnings (e.g., null vector embeddings)
|
||||
The importer has only an allowlisted mutable-table DML registry. It has no grant for immutable KBN
|
||||
relations, schemas, roles, memberships, extensions, catalogs, or the Drizzle ledger. Source PGlite
|
||||
uses its explicit local directory and does not make a PostgreSQL URL fallback valid.
|
||||
|
||||
## What gets migrated
|
||||
|
||||
All persistent, user-bound data is migrated in dependency order:
|
||||
|
||||
- **users, teams, team_members** — user and team ownership
|
||||
- **accounts** — OAuth provider tokens (durable credentials)
|
||||
- **projects, agents, missions, tasks** — all project and agent definitions
|
||||
- **conversations, messages** — all chat history
|
||||
- **preferences, insights, agent_logs** — preferences and observability
|
||||
- **provider_credentials** — stored API keys and secrets
|
||||
- **tickets, events, skills, routing_rules, appreciations** — auxiliary records
|
||||
|
||||
Full order is defined in code (`MIGRATION_ORDER` in `packages/storage/src/migrate-tier.ts`).
|
||||
|
||||
## What gets skipped and why
|
||||
|
||||
Three tables are intentionally not migrated:
|
||||
|
||||
| Table | Reason |
|
||||
| ----------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| **sessions** | TTL'd auth sessions from the old environment; they will fail JWT verification on the new target |
|
||||
| **verifications** | One-time tokens (email verify, password reset) that have either expired or been consumed |
|
||||
| **admin_tokens** | Hashed tokens bound to the old environment's secret keys; must be re-issued |
|
||||
|
||||
**Note on accounts and provider_credentials:** These durable credentials ARE migrated because they are user-bound and required for resuming agent work on the target environment. After migration to a multi-tenant federated deployment, operators may want to audit or wipe these if users are untrusted or credentials should not be shared.
|
||||
|
||||
## Idempotency and concurrency
|
||||
|
||||
The migration is **idempotent**:
|
||||
|
||||
- Re-running is safe (uses `ON CONFLICT DO UPDATE` internally)
|
||||
- Ideal for retries on transient failures
|
||||
- Concurrent invocations are blocked by a Postgres advisory lock; the second caller will wait
|
||||
|
||||
If a previous run is stuck, check for advisory locks:
|
||||
|
||||
```sql
|
||||
SELECT * FROM pg_locks WHERE locktype='advisory';
|
||||
```
|
||||
|
||||
If you need to force-unlock (dangerous):
|
||||
|
||||
```sql
|
||||
SELECT pg_advisory_unlock(<lock_id>);
|
||||
```
|
||||
|
||||
## Verify the migration
|
||||
|
||||
After migration completes, spot-check the target:
|
||||
|
||||
```bash
|
||||
# Count rows on a few critical tables
|
||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
||||
"SELECT 'users' as table, COUNT(*) FROM users UNION ALL
|
||||
SELECT 'conversations' as table, COUNT(*) FROM conversations UNION ALL
|
||||
SELECT 'messages' as table, COUNT(*) FROM messages;"
|
||||
```
|
||||
|
||||
Verify a known user or project exists by ID:
|
||||
|
||||
```bash
|
||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
||||
"SELECT id, email FROM users WHERE email='<your-email>';"
|
||||
```
|
||||
|
||||
Ensure vector embeddings are NULL (if source was PGlite) or populated (if source was postgres + pgvector):
|
||||
|
||||
```bash
|
||||
psql postgresql://mosaic:mosaic@localhost:5433/mosaic -c \
|
||||
"SELECT embedding IS NOT NULL as has_vector FROM insights LIMIT 5;"
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
There is no in-place rollback. If the migration fails:
|
||||
|
||||
1. Restore the target database from a pre-migration backup
|
||||
2. Investigate the failure logs
|
||||
3. Rerun the migration
|
||||
|
||||
Always test migrations in a staging environment first.
|
||||
A failed or ambiguous migration is a control-plane incident: preserve sanitized evidence, retain
|
||||
the approved backup/rollback state, and retry only after independent review. Never inspect,
|
||||
unlock, repair, or initialize the target with ad hoc SQL or copied credentials.
|
||||
|
||||
43
docs/guides/mos-connector-lease-operations.md
Normal file
43
docs/guides/mos-connector-lease-operations.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Mos Connector Lease Operations — M1
|
||||
|
||||
## Operational status
|
||||
|
||||
M1 installs the durable schema and gateway policy/adapter boundary. It does **not** activate a connector, expose a lease administration endpoint, or cut over a channel. The default gateway connector-lease policy is deny-all until a later work package supplies an authorized server-side policy and concrete adapter.
|
||||
|
||||
## Events to monitor
|
||||
|
||||
Use correlation IDs to follow `connector_lease_audit_log` events:
|
||||
|
||||
| Event | Meaning |
|
||||
| ---------- | --------------------------------------------------------------------- |
|
||||
| `acquire` | First holder inserted for an unused binding |
|
||||
| `renew` | Current holder heartbeat extended the TTL |
|
||||
| `takeover` | Authorized CAS replaced the holder and incremented epoch |
|
||||
| `release` | Current holder explicitly relinquished authority |
|
||||
| `expiry` | An expired current lease was observed |
|
||||
| `reject` | Policy, CAS, expiry, scope, or fencing validation denied an operation |
|
||||
|
||||
Audit data is metadata-only. Raw grant objects, connector payloads, scopes, tokens, approval references, and credentials must never be added to audit output.
|
||||
|
||||
## Incident checks
|
||||
|
||||
For suspected duplicate/stale connector effects:
|
||||
|
||||
1. Correlate the attempted operation with its `reject`, `takeover`, or `expiry` event.
|
||||
2. Compare the current row's connector ID, lease UUID, epoch, expiry, and release time with the adapter's normalized execution context.
|
||||
3. Treat an old epoch, old lease UUID, expired lease, or released lease as non-authoritative. Do not retry it as the old holder.
|
||||
4. Recovery uses the authorized takeover path with the observed expected epoch. Ordinary acquire is intentionally rejected for expired/released rows.
|
||||
5. If an external effect may already have happened, preserve evidence and do not assume lease fencing provides exactly-once replay safety.
|
||||
|
||||
## Migration and rollback safety
|
||||
|
||||
Migration `0016_salty_morlocks.sql` is additive: it creates two new tables and indexes without modifying existing authorization/session tables. Before rollout, normal database backup and migration verification still apply. Rolling application code back leaves unused additive tables in place; dropping tables is not part of automated rollback because it would destroy lease/audit evidence.
|
||||
|
||||
## Security constraints
|
||||
|
||||
- Tenant comes from authenticated gateway context, never a connector request field.
|
||||
- Logical agent, binding, connector, and scope identifiers use normalized constrained forms.
|
||||
- Takeover requires explicit gateway policy authorization and an expected epoch.
|
||||
- Default defense-in-depth TTL caps are 5 minutes for leases and 30 seconds for grants; policy may enforce stricter limits.
|
||||
- Validation and rejection audit complete before adapter side effects.
|
||||
- Existing authz and exact-action approval controls remain additional required gates; a valid connector lease does not bypass them.
|
||||
147
docs/guides/upgrade-safety-and-recovery.md
Normal file
147
docs/guides/upgrade-safety-and-recovery.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# Upgrade Safety & Recovery
|
||||
|
||||
How Mosaic protects operator-owned configuration under `~/.config/mosaic` across
|
||||
framework upgrades, and how to recover if a projection is ever lost.
|
||||
|
||||
A framework upgrade runs `install.sh` in keep-mode (`MOSAIC_INSTALL_MODE=keep`,
|
||||
`MOSAIC_SYNC_ONLY=1`) to refresh framework-owned files in place. The incident
|
||||
this hardening addresses: an upgrade that silently overwrites or deletes a file
|
||||
the operator owns — credentials, personas, a roster, or a generated agent env —
|
||||
with no snapshot to fall back to.
|
||||
|
||||
Protection is layered. Each layer is independent; a later layer catches what an
|
||||
earlier one misses.
|
||||
|
||||
## Layer 1 — Manifest-owned sync (prevention)
|
||||
|
||||
The single source of truth for ownership is
|
||||
[`framework-manifest.txt`](../../packages/mosaic/framework/framework-manifest.txt).
|
||||
Both the bash installer and the TypeScript sync path resolve every path against
|
||||
this one file (parity is enforced by test), so they can never drift.
|
||||
|
||||
- Ownership is **allow-list, deny-wins**: a path is framework-owned only if a
|
||||
`[framework]` glob matches and no `[operator]` carve-out overrides it.
|
||||
- **Unknown paths default to operator** (fail-safe): a file the manifest never
|
||||
anticipated is treated as operator-owned and is never pruned.
|
||||
- Keep-mode does a non-deleting copy plus an explicit, manifest-scoped prune that
|
||||
only ever iterates framework globs — operator and unknown paths are
|
||||
structurally unreachable by the prune.
|
||||
|
||||
Result: a correct upgrade cannot touch operator config at all.
|
||||
|
||||
## Layer 2 — Durable pre-update snapshot + verify net (safety + rollback)
|
||||
|
||||
Before **any** mutation, the installer snapshots the operator-owned surface that
|
||||
exists into:
|
||||
|
||||
```
|
||||
${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-timestamp>/
|
||||
```
|
||||
|
||||
- `0700` directories / `0600` files (`umask 077`, scoped and restored),
|
||||
outside `~/.config/mosaic` and outside any repo.
|
||||
- **Fail-open**: a snapshot failure warns but never aborts the upgrade it
|
||||
protects.
|
||||
- Retention is `MOSAIC_BACKUP_RETENTION` snapshots (default 5).
|
||||
|
||||
After the sync, a **verify net** compares each snapshot file against its target
|
||||
and restores (with a loud warning) any operator file the upgrade diverged or
|
||||
removed — a divergence means a manifest bug slipped through Layer 1.
|
||||
|
||||
Inspect and restore snapshots with the CLI:
|
||||
|
||||
```bash
|
||||
mosaic restore --list # dry-run: enumerate snapshots by timestamp
|
||||
mosaic restore --from <UTC-timestamp> # restore the operator surface from one snapshot
|
||||
mosaic restore --from <ts> --dry-run # preview a specific restore without writing
|
||||
```
|
||||
|
||||
`mosaic restore` reports **counts and relative paths only** — it never emits file
|
||||
contents, so a secret in `tools/_lib/credentials.json` is never echoed. Restores
|
||||
are confirmation-gated (`--yes` or `MOSAIC_ASSUME_YES`) and write each leaf
|
||||
atomically with `O_NOFOLLOW` (a symlink swapped in after the snapshot fails
|
||||
closed rather than following out of the managed tree).
|
||||
|
||||
## Layer 3 — Regeneration from roster SSOT (recovery)
|
||||
|
||||
Some operator files are **derived** and do not need a byte-for-byte snapshot to
|
||||
recover — they can be rebuilt from their source of truth. The fleet's per-agent
|
||||
generated env projections are the prime case:
|
||||
|
||||
- `~/.config/mosaic/fleet/agents/<name>.env.generated` is a deterministic
|
||||
projection of `~/.config/mosaic/fleet/roster.yaml`.
|
||||
- The launcher (`start-agent-session.sh`, invoked by
|
||||
`mosaic-agent@<name>.service`) sources that generated projection to establish
|
||||
each agent's identity, runtime, model, and working directory. If it is missing
|
||||
or wrong, the agent cannot launch with its intended identity.
|
||||
|
||||
`mosaic fleet regen` rebuilds those projections from the roster SSOT:
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # dry-run (default): show what would be rebuilt
|
||||
mosaic fleet regen --json # same, machine-readable
|
||||
mosaic fleet regen --write # rebuild the projections on disk
|
||||
```
|
||||
|
||||
- **Dry-run by default.** Nothing is written until you pass `--write`.
|
||||
- **Deterministic and idempotent** — the projection is a pure function of the
|
||||
roster, so repeated `--write` runs produce byte-identical files.
|
||||
- **Projection-only. It never restarts an agent.** Recovery order forbids
|
||||
restart-before-verify; `regen` has no path to systemd lifecycle at all.
|
||||
- **It rebuilds only `<name>.env.generated`** — it never writes, relocates, or
|
||||
deletes the operator-owned `.env` / `.env.local` surface.
|
||||
- It **validates the roster the same way `reconcile` does** (persona resolution
|
||||
and protected-class tool-policy match), so a hand-edited or corrupt roster is
|
||||
rejected rather than projected, and a `--write` takes the shared reconcile
|
||||
lock so it cannot race a concurrent reconcile.
|
||||
- Output is **paths and counts only** — the rendered `KEY=value` body is never
|
||||
echoed.
|
||||
|
||||
`regen` uses the exact same roster→env mapping as `mosaic fleet reconcile`, so a
|
||||
recovered projection matches what a normal reconcile would have written.
|
||||
|
||||
## Recovery runbook — wiped `fleet/agents/*.env.generated`
|
||||
|
||||
If an upgrade (or a manual mistake) has left an agent without its generated
|
||||
projection, **do not restart the unit first** — a launch against a missing
|
||||
projection fails closed, and any stale state must be corrected before restart,
|
||||
not after.
|
||||
|
||||
1. **Prefer a snapshot restore if one exists** (byte-exact operator state):
|
||||
|
||||
```bash
|
||||
mosaic restore --list
|
||||
mosaic restore --from <UTC-timestamp>
|
||||
```
|
||||
|
||||
2. **Otherwise regenerate the derived projections from the roster SSOT:**
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # confirm the plan (create vs rebuild per agent)
|
||||
mosaic fleet regen --write # rebuild fleet/agents/<name>.env.generated
|
||||
```
|
||||
|
||||
3. **Verify each unit will resolve the intended runtime/workdir _before_ any
|
||||
restart.** The unit sets **no** `EnvironmentFile=` — it launches from a minimal
|
||||
environment and `start-agent-session.sh` sources `.env.generated` itself, so
|
||||
verify the generated file directly and confirm the launcher path:
|
||||
|
||||
```bash
|
||||
# Confirm fleet/agents/<name>.env.generated exists and carries the intended
|
||||
# MOSAIC_AGENT_* values (name, runtime, model, workdir, socket).
|
||||
test -f ~/.config/mosaic/fleet/agents/<name>.env.generated
|
||||
# Confirm the unit launches the session script that reads it.
|
||||
systemctl --user cat mosaic-agent@<name> | grep ExecStart
|
||||
```
|
||||
|
||||
4. **Only then restart, one unit at a time:**
|
||||
|
||||
```bash
|
||||
systemctl --user restart mosaic-agent@<name>
|
||||
```
|
||||
|
||||
## See also
|
||||
|
||||
- Design: [`docs/design/791-upgrade-config-protection.md`](../design/791-upgrade-config-protection.md)
|
||||
- Fleet operations: [`docs/guides/fleet-local-canary.md`](./fleet-local-canary.md)
|
||||
- Ownership SSOT: [`packages/mosaic/framework/framework-manifest.txt`](../../packages/mosaic/framework/framework-manifest.txt)
|
||||
@@ -183,6 +183,8 @@ non-interactive use:
|
||||
--no-auto-launch # Skip auto-launch of wizard after install
|
||||
```
|
||||
|
||||
Unrecognized flags or positional arguments fail before installation starts and print the supported-option usage.
|
||||
|
||||
Or if installed globally:
|
||||
|
||||
```bash
|
||||
@@ -307,6 +309,39 @@ mosaic quality-rails
|
||||
|
||||
---
|
||||
|
||||
### Claude Code Skill Registration
|
||||
|
||||
Mosaic stores canonical skills under `~/.config/mosaic/skills/`. Claude Code scans
|
||||
`~/.claude/skills/`, so Mosaic maintains one symlink per skill between those
|
||||
directories.
|
||||
|
||||
```bash
|
||||
mosaic skill list
|
||||
mosaic skill register <name>
|
||||
mosaic skill unregister <name>
|
||||
```
|
||||
|
||||
- `register` is idempotent and repairs a dangling Mosaic-owned link. Names use
|
||||
the safe grammar `[A-Za-z0-9][A-Za-z0-9._-]*`; files, directories, foreign
|
||||
symlinks, path traversal, absolute paths, and names beginning with `-` are
|
||||
refused.
|
||||
- `unregister` is idempotent when no entry exists. It removes only symlinks that
|
||||
point inside `~/.config/mosaic/skills/`; foreign entries are never removed.
|
||||
- `list` reports `registered`, `unregistered`, `dangling`, `foreign`,
|
||||
`foreign-dangling`, or `misdirected` for each canonical or Claude entry.
|
||||
|
||||
Install, wizard finalization, and `mosaic update` framework re-seeding reconcile
|
||||
every canonical skill automatically. A skill directory added after initial
|
||||
setup therefore receives its Claude bridge without a per-skill code change or
|
||||
manual `ln -s`. If Claude Code is already running, use `/reload-skills` or start
|
||||
a new session after registration so its in-process skill registry rescans.
|
||||
|
||||
This command group is Claude-only in M1. Pi can consume Mosaic's canonical skill
|
||||
root through its Mosaic launcher configuration and does not need this Claude
|
||||
bridge. Codex has a separate link path managed by the legacy full skill-sync
|
||||
script; equivalent lifecycle management remains follow-up scope and is not
|
||||
changed here.
|
||||
|
||||
## Sub-package Commands
|
||||
|
||||
Each Mosaic sub-package exposes its full API surface through the `mosaic` CLI.
|
||||
@@ -522,8 +557,14 @@ mosaic storage export --bucket agent-artifacts --output ./artifacts.tar.gz
|
||||
# Import data into storage
|
||||
mosaic storage import --bucket agent-artifacts --input ./artifacts.tar.gz
|
||||
|
||||
# Migrate data between tiers
|
||||
mosaic storage migrate --from hot --to cold --older-than 30d
|
||||
# Schema migration is unavailable in this release. The current storage wrapper shells
|
||||
# directly to `pnpm --filter @mosaicstack/db db:migrate`; it is legacy N-1,
|
||||
# uncertified, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||
# Future schema migration is non-operative: external bootstrap → TLS/roles → runner
|
||||
# --run → runner --verify → readiness.
|
||||
|
||||
# Tier copy uses only the separately held secure migrate-tier route. Never use a legacy
|
||||
# --from/--to storage-migrate command or pass a credential on argv.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
# Native Kanban/SOT Canon
|
||||
|
||||
**Status:** KCR-001–016 independently cleared; canonical publication is in progress under issue [#751](https://git.mosaicstack.dev/mosaicstack/stack/issues/751)
|
||||
**Status:** KCR-001–016 independently cleared; KBN-101 rc.16 current generic storage-wrapper authority remediation awaits independent exact-head re-review under issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771)
|
||||
**Date:** 2026-07-14
|
||||
**Implementation hold:** no feature implementation starts until this canon is squash-merged to `main` with terminal-green CI; after merge, every slice remains held until its KBN prerequisite graph is satisfied.
|
||||
|
||||
## Artifacts
|
||||
|
||||
| Artifact | Purpose |
|
||||
| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0–P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
|
||||
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
|
||||
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0–P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
|
||||
| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.16 direct-Drizzle current storage-wrapper hold: legacy N-1/uncertified/non-operative pending -02/-03/-06/-08; exact README commented/user-guide executable forms fail before masking and source-consistency rejects runner-delegation copy; held future bootstrap → TLS/roles → run → verify → readiness; plus prior production boundary, pgvector owner, attestation, inventory, manifests, DDL classifier, TLS/bootstrap, activation, and certification contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
|
||||
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
|
||||
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
|
||||
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
|
||||
@@ -18,6 +19,7 @@
|
||||
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
|
||||
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
|
||||
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
|
||||
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | Historical `da742ca` REQUEST CHANGES report retained as prior closure evidence; rc.16 awaits independent exact-head re-review after closing the current generic storage-wrapper authority HIGH finding |
|
||||
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
|
||||
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
|
||||
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
|
||||
@@ -32,7 +34,7 @@
|
||||
| **coder5** | Web | Tasks/Projects Kanban/List/detail and later Coordinator/migration-review UI |
|
||||
| **Mos** | Serialized integration | Canon publication, frozen-contract changes, shared-root/exports, integration gates, merge authority |
|
||||
|
||||
The safe order is KBN-010 → KBN-100 → KBN-105, then coder3 Gateway/MCP server, coder4 CLI/projection, coder5 web, and coder2 recovery can proceed on disjoint files. coder4 then runs pure Coordinator → importer → cutover tooling serially. No two active slices edit the same files.
|
||||
The safe order is KBN-010 → KBN-101 foundation → KBN-100 → KBN-101 deployed-role immutable-operation certificate → KBN-105, then coder3 Gateway/MCP server, coder4 CLI/projection, coder5 web, and coder2 recovery can proceed on disjoint files. KBN-100 is blocked on the KBN-101 foundation; real deployed-role certification—not synthetic test roles—is required before KBN-105. coder4 then runs pure Coordinator → importer → cutover tooling serially. No two active slices edit the same files.
|
||||
|
||||
## Recovery defaults
|
||||
|
||||
|
||||
415
docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md
Normal file
415
docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md
Normal file
@@ -0,0 +1,415 @@
|
||||
# KBN-010 — Threat, Authorization, and Constraint-Impact Gate
|
||||
|
||||
- **Issue:** [#753](https://git.mosaicstack.dev/mosaicstack/stack/issues/753)
|
||||
- **Gate status:** **PASS / GO**
|
||||
- **Reviewed baseline:** `origin/main` at `49e8a54` (2026-07-14)
|
||||
- **Frozen target:** `SHARED-CONTRACT.md` v1.0.0-rc.4 and `contracts/*.v1.ts`
|
||||
- **Disposition input:** contract commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`, tree `7ebab8fa530a7180036928cea9527f808548aa14`
|
||||
- **Scope:** documentation and future-test planning only; no runtime, schema, migration, API, configuration, dependency, CI, or deployment change
|
||||
|
||||
## 1. Decision
|
||||
|
||||
KBN-010 is **PASS / GO** against frozen contract rc.4. The original rc.3 finding remains historical detection evidence:
|
||||
|
||||
- **KBN010-SI-001 — rc.3 invalid mission composite-FK candidate key.** At rc.3, `missionsV1` declared a primary key on `id` and a unique key on `(workspace_id, project_id, id)`, but not a candidate key on `(workspace_id, id)`. Both `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk` referenced exactly `(missions.workspace_id, missions.id)`. PostgreSQL requires the referenced column list of a foreign key to match a non-partial unique/primary candidate key; uniqueness of `id` alone did not satisfy that two-column reference. The rc.3 DDL was therefore invalid, and KBN-010 correctly blocked it.
|
||||
|
||||
Contract rc.4 resolves SI-001 by adding the non-partial `missions_workspace_id_uidx` candidate key on `(workspace_id, id)` while retaining the global `id` primary key and the project-congruent `(workspace_id, project_id, id)` key. Both polymorphic child FKs retain their exact workspace-safe ordered columns and `ON DELETE RESTRICT`; no target, tenancy, project-congruence, exactly-one-target, N-1, rollback, no-cascade, identity, approval, or fencing authority is weakened.
|
||||
|
||||
Independent Homelab non-author schema/security review returned **APPROVE** for the exact rc.4 commit/tree/content and found no collision with #757 connector fencing. SI-001 has no unresolved contract/schema-design impact.
|
||||
|
||||
This GO completes the KBN-010 analysis/review prerequisite only. It does **not** claim that runtime schema or migration DDL exists. KBN-100 remains held and may be released only after this PR squash-merges, the merged change reaches terminal-green CI on `main`, and issue #753 closes.
|
||||
|
||||
### 1.1 Independent rc.4 evidence identity
|
||||
|
||||
- **Commit:** `3f6a3387b419eb99453ee10dd25ba888faaab0b5`
|
||||
- **Tree:** `7ebab8fa530a7180036928cea9527f808548aa14`
|
||||
- **Stable full-index SHA-256:** `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`
|
||||
- **Stable patch-id:** `058cf98026fcd1043703c866aee047c8bb144740`
|
||||
- **Verdict:** Homelab independent non-author schema/security review **APPROVE**.
|
||||
- **Reviewed conclusions:** the candidate key repairs both dependent FKs; tenant safety, polymorphic exactly-one-target semantics, RESTRICT/no-cascade behavior, and N-1/rollback semantics remain valid; #757 uses separate tables/indexes/FKs/identity/fence authority and has no collision.
|
||||
|
||||
A command-rendered patch SHA may differ when Git rendering options, headers, or command form differ. That rendering digest is non-authoritative. Canonical review identity is the Git commit object plus tree and exact file content; the stable full-index digest and stable patch-id above are corroborating identities.
|
||||
|
||||
## 2. Method and trust boundaries
|
||||
|
||||
### 2.1 Inputs inspected
|
||||
|
||||
- Canonical requirements: `docs/requirements/native-kanban-sot.md`.
|
||||
- Workstream manifest and read-only task plan.
|
||||
- Frozen health, schema, Mechanical Coordinator, and recovery contracts in full.
|
||||
- Actual current-main schema, Better Auth guard/scope helpers, project/task/mission/team controllers and repositories, fleet backlog, and `TASKS.md` parser/writer.
|
||||
- Issue #753 through the Mosaic provider wrapper.
|
||||
|
||||
### 2.2 Current-main exposure that the target must replace, not inherit
|
||||
|
||||
| Current-main fact | Constraint on future implementation |
|
||||
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Teams are global; projects, missions, tasks, agents, and fleet backlog have no `workspace_id`. | KBN-100 must add the workspace boundary and KBN-110 must query by server-derived workspace in every repository operation. |
|
||||
| `AuthGuard` authenticates a Better Auth user, while `scopeFromUser` falls back through optional tenant/team/org claims and finally user ID. | Kanban tenancy must derive from an authenticated **active workspace membership**, not this compatibility fallback or caller data. |
|
||||
| Team list/get/member endpoints return global team data to any authenticated user. | New Kanban endpoints must use a uniform no-oracle denial and must not reuse global team lookup as authorization. |
|
||||
| Project/task repositories load and mutate by bare IDs; controller checks are separate and sometimes distinguish not-found from forbidden. | Workspace predicates and authorization must be inside the authoritative transaction/repository command path. |
|
||||
| Tasks can have nullable project/mission links, free-text assignee, JSON tags, no aggregate version, and no fence. | Expand/backfill/quarantine must precede NOT NULL/composite constraints; new commands cannot trust legacy fields. |
|
||||
| `mission_tasks.status` is a second status writer. | Pre-expand must prohibit it as a write source and later retire it only after N-1 evidence. |
|
||||
| Fleet `backlog` has global JSON dependencies and TTL claims without workspace, assignment, approval, session, or fencing. | It must be frozen and imported as non-dispatching shadow data; it cannot be adapted into the canonical lease path. |
|
||||
| `packages/coord/src/tasks-file.ts` parses and mutates `TASKS.md`. | KBN-120 must replace production use with generated, read-only projection code and prove there is no import/mutation path. |
|
||||
| No Kanban transaction-local health proof, semantic audit/event chain, change proposals, canonical outbox, approval binding, or fenced lease model exists. | These are new frozen invariants, not behaviors that may be inferred from current endpoints. |
|
||||
|
||||
## 3. Authorization matrix
|
||||
|
||||
The exact route/DTO freeze belongs to KBN-105. This matrix fixes the minimum authorization behavior that freeze and later implementation must preserve.
|
||||
|
||||
| Principal/state | Permitted authority | Required authoritative checks | Explicit denials |
|
||||
| -------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
||||
| Unauthenticated caller | Public health observation only, if deployment exposes it | Health DTO validation; no proof field accepted | All canonical reads/mutations; health observation never authorizes a write |
|
||||
| Active workspace `owner`/`admin` user | Policy-allowed workspace administration and domain commands | Better Auth session; active membership; server-derived workspace; command-family role; expected version/idempotency | Foreign workspace, suspended workspace, revoked membership, caller workspace override |
|
||||
| Active workspace `member` user | Policy-allowed project/task/proposal commands | Active membership plus project/team capability and target checks in the same transaction | Admin, approval, purge, service-only Coordinator, and unrelated project commands |
|
||||
| Active workspace `auditor` user | Workspace-scoped reads and audit/evidence inspection | Active membership and read capability | Every mutation, approval, lease, token issuance, purge |
|
||||
| Active workspace `service` identity | Only explicitly issued command families | Credential maps to workspace+agent+session; agent enabled; session live; role/capability allowlist; token expiry/audience; DB recheck per command | Raw DB credentials, user/admin fallback, cross-workspace scope, command families absent from token and registry |
|
||||
| Enabled agent with live session | Agent commands matching its declared and policy-approved specialist role/capabilities | Exact workspace+agent+session binding, heartbeat/state, assignment target, lease, current decimal-string fence | Ended/offline/degraded session where policy disallows; disabled agent; another assignment/session/fence |
|
||||
| Mechanical Coordinator engine | Pure eligibility/order/expiry decisions from immutable snapshots | Complete workspace-local snapshot and policy revision | Authentication, ID loading, SQL, proof minting, scope invention, approval, certification, merge |
|
||||
| Coordinator persistence service | Service-only assignment/lease/checkpoint/recovery commands | Fresh transaction-local proof; locks; current assignment/approval/task/session/policy/fence | Public/user proof-by-value, stale approval/policy, direct completion/certification/merge |
|
||||
| Reviewer/SecReview/Certifier | Attributable evidence decisions allowed by gate policy | Active authority, author differs from reviewer, mandatory SecReview classification, immutable artifacts | Self-review; missing evidence; Certifier merge/issue-close/release |
|
||||
| Break-glass retention operator | Narrow, time-bounded purge procedure only | Separate break-glass authority, reason, scope, approvals, immutable pre-purge evidence, semantic audit, post-action reconciliation | Normal application role DELETE/UPDATE, bulk unscoped purge, unaudited hard delete |
|
||||
| Revoked/expired/disabled identity or ended session | None beyond policy-permitted public observation | Revocation/lifecycle checked from PostgreSQL on every command | Cached token/Valkey state cannot preserve authority |
|
||||
|
||||
**No-oracle rule:** authentication may return 401, but once authenticated, a foreign-workspace, nonexistent, inaccessible, or wrong-project identifier must follow the one KBN-105-frozen 404/403 policy with the same response shape and no foreign metadata, timing-derived detail, or WebSocket/MCP discrepancy.
|
||||
|
||||
## 4. Threat matrix
|
||||
|
||||
Every disposition is against the frozen target, not a claim about current-main behavior.
|
||||
|
||||
| ID | Attacker or failure | Asset | Precondition and abuse path | Frozen preventive/detective control | Required schema/API/negative-test evidence | Future owner | Residual risk | Disposition |
|
||||
| --- | --------------------------------------------------------------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
|
||||
| T01 | Authenticated user supplies a foreign workspace/resource ID | Tenant confidentiality and integrity | Caller knows or guesses project/task/mission/team IDs and probes REST, MCP, WebSocket, repository, or Coordinator paths | `workspace_id` on every canonical row; composite relations; server-derived tenant; uniform no-oracle denial | Composite FK/unique DDL; every repository predicate includes workspace; N100-01/02, N110-01..05, N130-01 | KBN-100, 105, 110, 130 | Timing/volume side channels require operational review | Controlled after evidence |
|
||||
| T02 | Revoked or inactive member retains an old session | Ownership and mutation authority | Authentication remains valid after workspace membership revocation | Active membership rechecked in the authoritative transaction for owners, principals, proposers, and decision actors | Active/inactive membership fixtures; N100-03, N110-06/07; no cached membership authority | KBN-100, 110 | Better Auth session may remain valid for unrelated features | Controlled after evidence |
|
||||
| T03 | User joins/forges a team relation outside its workspace | Team-owned projects and tasks | Global-current-main team behavior or a stale membership is reused | Team is intra-workspace only; workspace/team composites; active workspace membership precedes team authorization | Cross-workspace team/member/owner insert and command denials; N100-04/05, N110-08 | KBN-100, 110 | Team-role policy mistakes remain possible | Controlled after evidence |
|
||||
| T04 | Same-workspace IDs from a different project are combined | Planning hierarchy integrity | Valid mission/milestone/parent/current-milestone UUIDs are substituted | Project-congruent composite relations and serialized hierarchy validation | Mission/milestone/parent/current milestone mismatch and parent-cycle tests; N100-06..10, N110-09 | KBN-100, 110 | Deep hierarchy checks can be expensive | Controlled after evidence |
|
||||
| T05 | Foreign or unrelated evidence/link/artifact IDs are attached | Review and audit truth | Caller has a valid same-workspace or foreign artifact UUID | Workspace-aware joins; immutable artifact digest/revision; semantic same-target validation in authoritative transaction | Mixed-workspace and same-workspace wrong-task/mission checkpoint/approval evidence tests; N100-11..14, N210-15/16 | KBN-100, 110, 210 | Same-workspace semantic validation is application-enforced | Controlled after evidence |
|
||||
| T06 | Stolen, over-scoped, or replayed service token | Coordinator and task mutation authority | Service credential is accepted as admin/user or claims are trusted without DB state | Command-family least privilege; agent/session workspace binding; no raw DB credentials; enabled/live state checked per command | Auth registry fixtures prove audience/expiry/role/capability; revoked agent and ended session denials; N105-01, N110-10..13, N210-01/02 | KBN-105, 110, 210 | Credential theft until expiry/revocation check | Controlled after evidence |
|
||||
| T07 | Caller forges public `healthy` or replays a stale health response | Sole-writer/fail-closed invariant | Public health body or caller field reaches mutation context | Public DTO is observation only; public DTOs reject proof/health fields; Gateway mints internal proof after live PG transaction probe | Contradictory union and forbidden-field tests; N105-02, N110-14..17 | KBN-105, 110, 140 | Health endpoint can still be used for reconnaissance | Controlled after evidence |
|
||||
| T08 | Internal stale, wrong-policy, or wrong-transaction proof is reused | Transaction integrity | A branded value leaks or an adapter fails to revalidate it | Non-exported brand; transaction identity, `checkedAt <= now < validUntil`, and policy revision revalidated immediately before mutation | Wrong transaction, expiry boundary, future timestamp, policy mismatch, commit-after-expiry tests; N110-18..22 | KBN-110, 140 | In-process code can bypass TypeScript; runtime checks are mandatory | Controlled after evidence |
|
||||
| T09 | DB/transport uncertainty is mislabeled as deliberate denial or conflict | Safe retry and exactly-once result | Timeout occurs before/after commit and client changes key or retries 503 | Exact 503/502/504/timeout/409 union; unknown outcome retries only with same idempotency key | Exhaustive fixture mapping and commit-before-timeout replay; N105-03, N110-23..27, N120-01/02 | KBN-105, 110, 120, 140 | External client may ignore retry rules | Controlled after evidence |
|
||||
| T10 | Assignment payload forges task version, target agent/session, role, expiry, or proposer | Work routing authority | Lease service trusts command DTO rather than persisted assignment | Persisted assignment identity; exactly-one principal/proposer; exact agent/session composite; acquire accepts IDs then reloads+locks | Cross-workspace and same-workspace target substitutions, stale task version, invalid role, expired assignment; N100-15..18, N210-03..08 | KBN-100, 200, 210 | Compromised authorized proposer can make harmful proposals | Controlled by approval/audit |
|
||||
| T11 | Approval proof is forged by value or borrowed from another assignment | Gate integrity | Caller submits `approved=true`, unrelated decision ID, stale policy, or self-approval | Relational approval bound to assignment; lock/reload; policy revision; author≠reviewer and mandatory SecReview | No proof-by-value DTO; wrong assignment/task/workspace/policy/actor/decision tests; N105-04, N210-09..14, N230-01 | KBN-105, 210, 230 | Colluding principals remain an organizational risk | Controlled after evidence |
|
||||
| T12 | Revoked policy or expired proposal/assignment is raced against lease acquisition | Routing policy | Approval and lease transactions do not lock/revalidate current rows | Lock assignment, approval, task, target session; compare current policy and expiry inside fresh-proof transaction | Concurrent revoke/expire/acquire tests with one valid terminal result; N210-17..19 | KBN-210, 230 | Clock skew if DB time is not canonical | Controlled after evidence |
|
||||
| T13 | Stale worker sends ack/heartbeat/checkpoint/review after reassignment | Canonical task and evidence state | Old process retains task/session IDs | Task-row-locked atomic monotonic bigint fence; every worker command carries exact lease/session/fence | Lower, expired, future, and other-task fences denied; old worker loses after new lease; N100-19/20, N210-20..24 | KBN-100, 210, 230 | Signed bigint exhaustion is theoretical | Controlled after evidence |
|
||||
| T14 | JavaScript precision truncates a fence | Stale-worker exclusion | bigint token is serialized as number above `2^53-1` | Drizzle bigint and decimal-string wire type only | `9007199254740993` and near-`int8` boundary round trips; numeric JSON rejected; N105-05, N210-25 | KBN-105, 210 | Nonconforming external clients | Controlled after evidence |
|
||||
| T15 | Checkpoint/evidence from another lease/task/session is submitted | Recovery and certification evidence | Same-workspace valid IDs are mixed | Exact lease composite binds workspace+task+assignment/session+fence; checkpoint composite binds lease+fence; evidence join plus semantic artifact-owner check | Same-workspace mismatched task/assignment/lease/session/checkpoint/artifact tests; N100-21..23, N210-26..31 | KBN-100, 210 | Artifact URI target may disappear outside DB | Controlled with digest/retention |
|
||||
| T16 | Outage note or pending/rejected proposal mutates/orders work | Sole SOT and gate integrity | Importer/UI treats note/proposal as task state | Proposals are inert; only explicit accept invokes normal typed command after recovery | Row/outbox/task counts unchanged for pending/rejected; no readiness/dependency/lease effect; N110-28..31 | KBN-110, 140 | Humans may act outside Mosaic operationally | Accepted as attributable residual |
|
||||
| T17 | Submission event is missing, foreign, or for another proposal | Proposal audit chain | Caller supplies an existing event UUID | Preallocated proposal ID; event-first same transaction; workspace composite FK; exact event type/aggregate/version semantic check | Missing/foreign/wrong-type/wrong-proposal event rolls back event+proposal; N100-24/25, N110-32..36 | KBN-100, 110 | Semantic checks are transaction code, not only FK | Controlled after evidence |
|
||||
| T18 | Acceptance borrows an unrelated command event | Proposal and target integrity | Same-workspace event exists for another target/command/proposal | Accept locks proposal+target, executes normal command, requires workspace/target match, causation=submission event, payload proposal ID | Foreign, wrong target/type/command/causation/payload event aborts target/event/proposal atomically; N100-26, N110-37..43 | KBN-100, 110 | Event payload schema drift | Controlled by KBN-105 fixtures |
|
||||
| T19 | Application role updates/deletes audit, approval evidence, checkpoint, or artifact | Nonrepudiation | Broad DB grants or parent cascade exists | INSERT/SELECT-only application roles; RESTRICT parent deletes; archive/cancel normal lifecycle | Role-level UPDATE/DELETE denied; parent delete RESTRICT; digest unchanged; N100-27..31 | KBN-100 | DB superuser can alter state | Break-glass/infra audit residual |
|
||||
| T20 | Break-glass purge is used as routine deletion or erases its own evidence | Retention and incident forensics | Elevated credential available | Separate audited retention procedure, bounded scope, reason, pre/post evidence, authority separation | Normal role denied; expired/missing approval denied; purge cannot delete its authorizing audit package; N115-01, N230-02/03 | KBN-115, 230 | Privileged DBA compromise | Accepted operational residual |
|
||||
| T21 | PostgreSQL unavailable or partitioned | Canonical state | Public health/Valkey remains live while transaction probe fails | Fail closed; no alternate writer/hidden queue; 503 only for proven not-applied; transport uncertainty remains unknown | Fault injection proves DB rows/outbox/files/Valkey unchanged on deliberate denial; commit-unknown replay; N110-44..48, N140-01 | KBN-110, 140, 230 | Availability loss is intentional | Accepted by Option A |
|
||||
| T22 | Valkey unavailable, duplicated, stale, or partitioned | Scheduling notifications | Queue wake is treated as truth or publication fails | Valkey derived/expendable; transactional outbox in PG; idempotent publisher; recovery from PG | Commit with Valkey down leaves pending outbox; replay publishes once logically; stale wake reloads PG; N110-49, N140-02, N230-04..06 | KBN-110, 210, 230 | Duplicate at-least-once delivery | Consumers must be idempotent |
|
||||
| T23 | Coordinator restarts between assignment, lease, checkpoint, or outbox steps | Durable orchestration truth | Process-local cache is treated as authority | PostgreSQL stores assignments, execution state, leases, fences, checkpoints, events, outbox; `recoverFromPostgres` | Restart at every transaction boundary reconstructs identical active/expired/pending sets without Valkey/files; N210-32..36, N230-07 | KBN-210, 230 | Recovery latency | Controlled after evidence |
|
||||
| T24 | Dependency cycle or concurrent reciprocal edge | Readiness and dispatch safety | Two transactions each see an acyclic graph before inserting | Unique directed edge; no self-edge; serialized recursive cycle check; readiness evaluates all blockers | Self/duplicate/cycle and concurrent A→B/B→A tests; all predecessor property test; N100-32..35, N200-01/02 | KBN-100, 200, 230 | Very large DAG performance | Bounded operational residual |
|
||||
| T25 | Parent-task cycle or project-incongruent relation | Planning hierarchy | Valid same-workspace IDs are arranged into an invalid tree | Project-congruent composites; serialized parent-cycle/orphan validation required by REQ-PLAN-001 | Self/indirect parent cycle, orphan, and cross-project mission/milestone/parent tests; N100-06..10 | KBN-100, 110 | Cycle validation is service/transaction enforced | Controlled after evidence |
|
||||
| T26 | Concurrent update, duplicate retry, or idempotency payload drift | Aggregate consistency | Two clients use same version/key with different payloads | Expected-version check; semantic event and outbox in same transaction; key returns prior immutable result only for identical command | One update wins; stale gets 409; duplicate identical returns prior; payload drift rejected; N110-50..54, N140-03 | KBN-105, 110, 140 | Long-lived clients face visible conflicts | Intentional user-visible residual |
|
||||
| T27 | State/event/outbox partial commit | Audit and notification consistency | Separate transactions or exception after state write | One PostgreSQL transaction for state+semantic event+outbox | Failure injected after each insert rolls all three back; success revisions align; N110-55..58 | KBN-110, 140 | Outbox publication remains asynchronous | Controlled after evidence |
|
||||
| T28 | Malicious/incorrect importer injects foreign workspace data or dispatchable work | Migration integrity | Source keys collide, lineage is absent, or importer has direct DB authority | Immutable source snapshots/checksums; one-way Gateway/migration-only port; workspace-safe idempotent modes; shadow records cannot dispatch | Foreign/malformed/duplicate/partial-resume/lineage checksum and no-dispatch tests; N300-01..08 | KBN-300, 330 | Source data may be semantically ambiguous | Quarantine and owner sign-off |
|
||||
| T29 | Cutover leaves legacy writer or forward/reverse sync active | Sole-writer invariant | Credentials/processes survive switch or rollback is improvised | Writer inventory, freeze, final delta, Gateway switch, credential shutdown, no dual write; rollback authority changes after first DB mutation | Process/credential inventory; concurrent-writer assertion; before/after-mutation rollback rehearsal; N320-01..06, N330-01 | KBN-320, 330, 340 | Missed external automation | Owner-gated residual |
|
||||
| T30 | Generated `TASKS.md`/`mission.json` is edited or parsed into DB | Canonical state | Current-main parser/writer remains reachable or file watcher imports changes | Generated non-authoritative header/IDs/time/revision; no production importer; regenerate/overwrite only | Static import search, tamper/regeneration, read-only permission, source-revision parity; N120-03..07, N140-04 | KBN-120, 140 | Humans may mistake snapshots for live data | Header and docs mitigate |
|
||||
| T31 | N-1 compatibility copies legacy ambiguity into canonical authority | Data integrity | Nullable/global/current-main fields are guessed during backfill | Nullable-first expand; deterministic mapping or quarantine; checksums; no new-only status before switch; legacy fields retained | Production-shape, ambiguous owner/assignee, status shadow, JSON/config/digest, rollback tests; N100-36..44 | KBN-100 | Quarantined records require human decision | Controlled by signed reconciliation |
|
||||
| T32 | Recovery posture claims durability not provided by mechanisms | Availability and audit retention | Shape-only validation or optimistic RPO is accepted | Normative validator; WAL/PITR/RPO/storage/high-assurance constraints; mechanism and restore evidence | Unknown/impossible/weakened configuration plus actual mechanism/restore tests; N115-02..08 | KBN-115 | Backup operator or storage compromise | Separate failure domain residual |
|
||||
| T33 | rc.3 frozen DDL could not create mission-scoped evidence/approval FKs | Tenant/evidence relational integrity | KBN-100 generated DDL from the rc.3 contract without an exact composite candidate key | rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` before both dependent FKs while retaining global and project-congruent keys | KBN-100 must execute N100-45..50: exact-key reconciliation, candidate-before-FKs, duplicate feasibility, empty/prod/N-1/rollback, and both-child foreign-workspace negatives | KBN-100 after PR/CI/#753 release | Runtime DDL remains unimplemented and must prove the frozen order | **Resolved by rc.4 + independent APPROVE; implementation evidence remains required** |
|
||||
|
||||
## 5. Constraint-impact matrix
|
||||
|
||||
| Impact ID | Required invariant | Frozen schema impact | API/transaction impact | Required evidence | Owner | Status |
|
||||
| --------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| CI-01 | Hard workspace tenancy and no oracle | `workspace_id`, workspace-aware unique/FKs on all canonical rows | Server-derived workspace; uniform denial on all surfaces | N100-01..14; N110-01..09; N130-01 | KBN-100/105/110/130 | Resolved by frozen controls |
|
||||
| CI-02 | Active user membership | Membership row plus unique `(workspace_id,user_id)`; active state retained | Recheck active membership in same authoritative transaction | N100-03; N110-06/07 | KBN-100/110 | Resolved; not FK-only |
|
||||
| CI-03 | Service identity least privilege/revocation | Agent/session workspace, lifecycle, state, roles, capabilities | Token maps to exact agent/session; command-family allowlist; DB recheck; no admin/raw DB fallback | N105-01; N110-10..13; N210-01/02 | KBN-105/110/210 | Resolved at auth/API layer |
|
||||
| CI-04 | Project-congruent hierarchy | Composite project/mission/milestone/parent/current-milestone relations | Lock/serialized parent-cycle and orphan validation | N100-06..10 | KBN-100/110 | Resolved; cycle behavior required |
|
||||
| CI-05 | Health-proof authority | Internal branded proof has transaction/time/policy fields | Probe and revalidate on same PG transaction; no public field | N105-02/03; N110-14..27 | KBN-105/110 | Resolved by frozen controls |
|
||||
| CI-06 | Assignment/approval identity | Exactly-one principal/proposer, exact agent/session assignment, relational approval | Reload+lock all IDs; compare version/target/state/expiry/policy/decision | N100-15..18; N210-03..19 | KBN-100/210 | Resolved by frozen controls |
|
||||
| CI-07 | Monotonic bigint fencing | Durable bigint counter, exact lease/fence keys, one active lease | Atomic increment/RETURNING; decimal-string DTO; reject every stale worker command | N100-19..23; N210-20..31 | KBN-100/105/210 | Resolved by frozen controls |
|
||||
| CI-08 | Proposal event chain | Both workspace-aware event FKs; event table created first | Exact submission/acceptance semantic checks in one transaction | N100-24..26; N110-28..43 | KBN-100/110 | Resolved; semantic checks not FK-only |
|
||||
| CI-09 | Immutable audit/evidence retention | RESTRICT parents; INSERT/SELECT-only immutable tables | Archive/cancel normal flow; separately authorized purge | N100-27..31; N115-01; N230-02/03 | KBN-100/115/230 | Resolved by frozen controls |
|
||||
| CI-10 | DB/Valkey/outbox/restart semantics | PG outbox and durable orchestration rows | Fail closed; same-key uncertainty retry; Valkey reloads PG; restart from PG | N110-44..49; N140-01/02; N230-04..07 | KBN-110/210/230 | Resolved by frozen controls |
|
||||
| CI-11 | DAG/race/idempotency/version | Unique edge; self check; event idempotency; aggregate versions | Serialized recursive cycle check; payload binding; expected-version conflict | N100-32..35; N110-50..58; N200-01/02 | KBN-100/110/200 | Resolved by frozen controls |
|
||||
| CI-12 | Import/cutover trust boundary | Lineage/artifact/event fields; shadow state cannot dispatch | One-way scoped importer, freeze, no direct DB/file authority, no dual writer | N300-01..08; N320-01..06 | KBN-300/320/330 | Resolved by frozen controls |
|
||||
| CI-13 | Generated-file no-import | No canonical file schema/import contract | Projection-only package; static reachability check removes current parser from production Kanban paths | N120-03..07; N140-04 | KBN-120/140 | Resolved by frozen controls |
|
||||
| CI-14 | Mission-scoped artifact and approval FKs | rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` and retains global/project-congruent keys | KBN-100 must emit the candidate before both exact RESTRICT FKs and preserve N-1/rollback order | N100-45..50: exact reconciliation, duplicate feasibility, empty/prod/N-1/rollback, and separate artifact/approval foreign-workspace negatives | KBN-100 after PR/CI/#753 release | **Resolved by rc.4 and independent APPROVE; future executable evidence required** |
|
||||
|
||||
## 6. Exact future negative-test catalog
|
||||
|
||||
These names are normative evidence identifiers for future slices. Equivalent test-file names are acceptable only if traceability retains these IDs and expected outcomes.
|
||||
|
||||
### KBN-100 — schema and migration
|
||||
|
||||
- **N100-01** reject every canonical child row whose `workspace_id` differs from its parent.
|
||||
- **N100-02** reject foreign-workspace link, artifact, proposal target, dependency, assignment, lease, checkpoint, approval, and event relationships.
|
||||
- **N100-03** reject an inactive/revoked member as accountable owner, proposer, decision actor, archive actor, or user principal in the authoritative command transaction.
|
||||
- **N100-04** reject a team/project relation crossing workspaces.
|
||||
- **N100-05** reject a team authorization path when the user lacks active membership in the team's workspace.
|
||||
- **N100-06** reject task→mission project mismatch.
|
||||
- **N100-07** reject task→milestone and project→current-milestone project mismatch.
|
||||
- **N100-08** reject task→parent project mismatch and self-parent.
|
||||
- **N100-09** reject indirect parent cycles under concurrent transactions.
|
||||
- **N100-10** reject mission→milestone project mismatch/orphan.
|
||||
- **N100-11** reject checkpoint artifact from another workspace.
|
||||
- **N100-12** reject checkpoint artifact owned by another same-workspace task/mission unless an explicitly frozen evidence rule permits it.
|
||||
- **N100-13** reject approval evidence from another workspace.
|
||||
- **N100-14** reject same-workspace approval evidence unrelated to the approval target.
|
||||
- **N100-15** reject zero/multiple assignment principals and zero/multiple proposers.
|
||||
- **N100-16** reject target session without its exact target agent.
|
||||
- **N100-17** reject assignment task/agent/session crossing workspaces.
|
||||
- **N100-18** reject non-positive task version and expired assignment acquisition.
|
||||
- **N100-19** concurrent lease insert permits one active lease and returns one winner.
|
||||
- **N100-20** successive leases return strictly increasing bigint fences.
|
||||
- **N100-21** reject checkpoint with another task, lease, or fence.
|
||||
- **N100-22** reject duplicate/non-monotonic checkpoint sequence.
|
||||
- **N100-23** reject evidence join for a mismatched checkpoint/task.
|
||||
- **N100-24** proposal insert without exact submission event fails atomically.
|
||||
- **N100-25** foreign/wrong-type/wrong-proposal submission event fails atomically.
|
||||
- **N100-26** foreign/wrong-target/unrelated acceptance event fails atomically.
|
||||
- **N100-27** application role cannot UPDATE/DELETE `task_events`.
|
||||
- **N100-28** application role cannot UPDATE/DELETE checkpoints/artifacts/evidence joins.
|
||||
- **N100-29** parent hard delete is RESTRICTed while audit/evidence children exist.
|
||||
- **N100-30** archive does not alter canonical lifecycle status.
|
||||
- **N100-31** purge without break-glass authority/evidence is denied.
|
||||
- **N100-32** reject dependency self-edge and duplicate directed pair regardless of type.
|
||||
- **N100-33** reject direct and indirect dependency cycles.
|
||||
- **N100-34** concurrent reciprocal dependency inserts cannot both commit.
|
||||
- **N100-35** readiness remains false until every blocking predecessor and completion condition passes.
|
||||
- **N100-36** empty DB migration succeeds after the contract amendment.
|
||||
- **N100-37** production-shape expand retains all legacy declarations.
|
||||
- **N100-38** crash/resume backfill is idempotent and checksum-stable.
|
||||
- **N100-39** ambiguous workspace/owner/assignee is quarantined, never guessed.
|
||||
- **N100-40** no `ready`/`in_review` status is emitted to N-1 readers before switch.
|
||||
- **N100-41** `mission_tasks.status` cannot remain a write source.
|
||||
- **N100-42** tags/assignee/date/mission JSON/config/description/agent fields reconcile without loss.
|
||||
- **N100-43** claimed fleet backlog rows are quarantined and imported rows cannot dispatch.
|
||||
- **N100-44** pre-switch rollback works while post-first-mutation rollback requires freeze/reconciliation.
|
||||
- **N100-45** reconcile both exact child FK column lists to the rc.4 `(workspace_id,id)` mission candidate while retaining the global `id` primary key and `(workspace_id,project_id,id)` key.
|
||||
- **N100-46** empty-DB migration creates `missions_workspace_id_uidx` before `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk`.
|
||||
- **N100-47** production-shape preflight finds no duplicate `(workspace_id,id)` groups, preserves global `id` uniqueness, and applies the candidate before both dependent FKs.
|
||||
- **N100-48** N-1 startup/read/write remains unchanged; pre-switch rollback drops both dependents before the candidate and preserves the global/project-congruent keys.
|
||||
- **N100-49** artifact insert using a valid mission ID paired with a foreign workspace fails before commit.
|
||||
- **N100-50** approval-decision insert using a valid mission ID paired with a foreign workspace fails before commit.
|
||||
|
||||
### KBN-105/KBN-110/KBN-120/KBN-130/KBN-140 — API and P1
|
||||
|
||||
- **N105-01** every route has an explicit user/service command-family policy; user/admin tokens cannot call service-only Coordinator mutations.
|
||||
- **N105-02** public DTO validation rejects `writeProof`, internal context, body `workspaceId`, and caller-asserted health.
|
||||
- **N105-03** fixture exhaustiveness prevents 503, 502/504/timeout, and 409 cross-mapping.
|
||||
- **N105-04** approval DTO accepts an ID and decision command only, never approval proof-by-value.
|
||||
- **N105-05** all fence fields accept/emit decimal strings and reject JSON numbers.
|
||||
- **N110-01** listing with a foreign `workspaceId` or foreign filter ID follows the frozen no-oracle denial and returns no rows/counts/cursors.
|
||||
- **N110-02** get by foreign or nonexistent aggregate ID has the same frozen denial shape and no foreign metadata.
|
||||
- **N110-03** create/update/archive with a foreign owner, parent, project, mission, milestone, tag, or target ID is denied before mutation.
|
||||
- **N110-04** dependency/proposal commands with foreign target IDs are denied with unchanged state/event/outbox counts.
|
||||
- **N110-05** REST, MCP, WebSocket, and internal Coordinator paths produce equivalent no-oracle behavior for the same foreign ID.
|
||||
- **N110-06** a revoked/inactive owner is denied even with a still-valid Better Auth session.
|
||||
- **N110-07** stale membership/team cache cannot authorize a proposer, decision actor, archive actor, or principal after revocation.
|
||||
- **N110-08** a team ID from another workspace cannot authorize or own the command target.
|
||||
- **N110-09** same-workspace but wrong-project mission/milestone/parent IDs are denied inside the transaction.
|
||||
- **N110-10** an expired service token is denied before repository access.
|
||||
- **N110-11** an audience- or workspace-mismatched service token is denied without an existence oracle.
|
||||
- **N110-12** an over-scoped service token cannot call a command family absent from its role/capability allowlist.
|
||||
- **N110-13** disabled agent or ended session revokes service-token command authority immediately on PostgreSQL recheck.
|
||||
- **N110-14** contradictory public health state/boolean combinations fail validation.
|
||||
- **N110-15** Valkey-only liveness cannot mint or substitute a PostgreSQL write proof.
|
||||
- **N110-16** caller-forged public `healthy` cannot enter internal mutation context.
|
||||
- **N110-17** public REST/MCP/CLI bodies containing health/proof fields are rejected.
|
||||
- **N110-18** an expired internal proof produces no state/event/outbox write.
|
||||
- **N110-19** a future-dated or not-yet-valid proof produces no write.
|
||||
- **N110-20** a policy-revision-mismatched proof produces no write.
|
||||
- **N110-21** a proof minted on another transaction/connection produces no write.
|
||||
- **N110-22** a proof that expires before the final pre-mutation check produces no write.
|
||||
- **N110-23** deliberate read-only/write-unavailable denial maps only to authoritative 503/not-applied/non-retryable.
|
||||
- **N110-24** timeout before commit maps to transport-unknown and permits only same-key retry.
|
||||
- **N110-25** timeout after commit maps to transport-unknown and same-key retry returns the committed canonical result once.
|
||||
- **N110-26** expected-version mismatch maps only to 409/not-applied/non-retryable.
|
||||
- **N110-27** recovery replay with a changed idempotency key cannot masquerade as the original uncertain request.
|
||||
- **N110-28** pending proposal cannot alter target fields/status/rank/version.
|
||||
- **N110-29** rejected proposal cannot affect readiness, dependencies, or gates.
|
||||
- **N110-30** pending/rejected proposal cannot create an assignment or lease.
|
||||
- **N110-31** direct proposal-row state manipulation cannot bypass normal command execution.
|
||||
- **N110-32** proposal submission without a submission event rolls back fully.
|
||||
- **N110-33** foreign-workspace submission event rolls back fully.
|
||||
- **N110-34** wrong aggregate/event type submission event rolls back fully.
|
||||
- **N110-35** same-workspace event for another proposal rolls back fully.
|
||||
- **N110-36** submission event with wrong previous/new version semantics rolls back fully.
|
||||
- **N110-37** foreign-workspace acceptance event rolls back proposal, target, event, and outbox.
|
||||
- **N110-38** same-workspace event for another target aggregate rolls back acceptance.
|
||||
- **N110-39** event from an unrelated normal command rolls back acceptance.
|
||||
- **N110-40** event caused by a different submission event rolls back acceptance.
|
||||
- **N110-41** event whose payload lacks or changes `changeProposalId` rolls back acceptance.
|
||||
- **N110-42** event for another proposal with the same target/command rolls back acceptance.
|
||||
- **N110-43** missing accepted-command event after target handling rolls back the entire transaction.
|
||||
- **N110-44** read-only-degraded denial changes no DB row/outbox/file/Valkey/provider state.
|
||||
- **N110-45** write-unavailable denial changes no DB row/outbox/file/Valkey/provider state.
|
||||
- **N110-46** PostgreSQL disconnect cannot redirect a command to any fallback writer.
|
||||
- **N110-47** commit uncertainty remains `unknown` and never becomes a fabricated 503/not-applied result.
|
||||
- **N110-48** same-key replay after recovery returns one canonical result with no duplicate event/outbox row.
|
||||
- **N110-49** Valkey publication failure leaves committed PG outbox pending and replayable.
|
||||
- **N110-50** two same-version updates produce one winner and one visible 409 loser.
|
||||
- **N110-51** identical duplicate key+payload returns the prior immutable result without another event/outbox row.
|
||||
- **N110-52** same key with payload/command drift is rejected as an idempotency conflict.
|
||||
- **N110-53** the same key in another workspace cannot reveal or reuse the first workspace's result.
|
||||
- **N110-54** stale reconnect/update cannot silently overwrite a newer aggregate revision.
|
||||
- **N110-55** failure after state write but before semantic event rolls back state.
|
||||
- **N110-56** failure after semantic event but before outbox rolls back state and event.
|
||||
- **N110-57** failure after outbox insert but before commit rolls back state, event, and outbox.
|
||||
- **N110-58** success commits matching aggregate/event/outbox revisions and correlation/causation.
|
||||
- **N120-01** CLI never retries an authoritative 503 deliberate denial.
|
||||
- **N120-02** CLI retries only transport-unknown outcomes and preserves the exact idempotency key.
|
||||
- **N120-03** generated projection header contains non-authoritative warning, workspace/project IDs, generated time, and source revision.
|
||||
- **N120-04** projection revision and records match the API snapshot revision exactly.
|
||||
- **N120-05** hand-tampering is overwritten or rejected by regeneration and never mutates PostgreSQL.
|
||||
- **N120-06** static/runtime reachability finds no parser/import path from `TASKS.md`, `mission.json`, or another export.
|
||||
- **N120-07** projection writer has no domain mutation/raw SQL/Valkey authority.
|
||||
- **N130-01** UI foreign/no-access/not-found state follows the frozen no-oracle response and renders no stale foreign data.
|
||||
- **N140-01** real-Gateway DB fault journey proves fail-closed no-fallback behavior.
|
||||
- **N140-02** real-Gateway Valkey-loss journey proves pending outbox replay.
|
||||
- **N140-03** real-Gateway concurrent update/retry journey proves version and idempotency semantics.
|
||||
- **N140-04** generated-file tamper journey proves projection parity and no import.
|
||||
|
||||
### KBN-115/KBN-200/KBN-210/KBN-230 — recovery and coordination
|
||||
|
||||
- **N115-01** retention purge without current break-glass authority, reason, immutable evidence, or bounded scope is denied and audited.
|
||||
- **N115-02** recovery posture with an unknown top-level or storage field is rejected.
|
||||
- **N115-03** PITR retention without WAL archival is rejected.
|
||||
- **N115-04** WAL archival with zero PITR retention is rejected.
|
||||
- **N115-05** claimed RPO better than the configured backup/WAL mechanism is rejected.
|
||||
- **N115-06** unencrypted, optional, or same-failure-domain storage is rejected.
|
||||
- **N115-07** weakened high-assurance values are rejected.
|
||||
- **N115-08** shape-only validation cannot pass without normative mechanism and restore evidence.
|
||||
- **N200-01** cyclic/incomplete dependency snapshots never become eligible.
|
||||
- **N200-02** identical immutable snapshot+policy+time returns identical ordering and explanation with no I/O/model import.
|
||||
- **N210-01** disabled agent cannot claim, ack, heartbeat, checkpoint, or submit review.
|
||||
- **N210-02** ended/offline/mismatched session cannot claim, ack, heartbeat, checkpoint, or submit review.
|
||||
- **N210-03** foreign-workspace task is rejected after lock/reload without an oracle.
|
||||
- **N210-04** stale task version is rejected before fence increment.
|
||||
- **N210-05** assignment target agent mismatch is rejected.
|
||||
- **N210-06** target session mismatch is rejected.
|
||||
- **N210-07** expired assignment is rejected.
|
||||
- **N210-08** assignment in rejected/released/expired/superseded/leased-invalid state is rejected.
|
||||
- **N210-09** missing approval is rejected.
|
||||
- **N210-10** rejected/escalated/requested approval is rejected as approval authority.
|
||||
- **N210-11** stale policy-revision approval is rejected.
|
||||
- **N210-12** foreign-workspace approval is rejected without an oracle.
|
||||
- **N210-13** approval for another assignment is rejected.
|
||||
- **N210-14** author self-approval/review is rejected when independence is required.
|
||||
- **N210-15** foreign-workspace artifact evidence is rejected.
|
||||
- **N210-16** same-workspace artifact unrelated to the assignment/task/gate is rejected.
|
||||
- **N210-17** concurrent policy revocation versus acquire cannot produce a lease under the revoked revision.
|
||||
- **N210-18** concurrent assignment expiry versus acquire cannot produce a lease after expiry.
|
||||
- **N210-19** concurrent session end versus acquire cannot produce a lease for the ended session.
|
||||
- **N210-20** lower fencing token is rejected without writes.
|
||||
- **N210-21** token from an older lease is rejected without writes.
|
||||
- **N210-22** token paired with another task is rejected without writes.
|
||||
- **N210-23** token paired with another session is rejected without writes.
|
||||
- **N210-24** token on an expired/revoked/released lease is rejected without writes.
|
||||
- **N210-25** fences above JavaScript safe integer round-trip exactly as decimal strings.
|
||||
- **N210-26** lease task does not match assignment task and is rejected.
|
||||
- **N210-27** lease agent/session does not match assignment target and is rejected.
|
||||
- **N210-28** checkpoint task does not match lease task and is rejected.
|
||||
- **N210-29** checkpoint fence does not match exact lease fence and is rejected.
|
||||
- **N210-30** checkpoint sequence duplicate/regression is rejected.
|
||||
- **N210-31** checkpoint artifact does not match workspace/task/evidence semantics and is rejected.
|
||||
- **N210-32** restart after assignment persistence reconstructs the pending assignment.
|
||||
- **N210-33** restart after lease commit reconstructs exact active lease and fence.
|
||||
- **N210-34** restart after checkpoint commit reconstructs checkpoint/recovery state.
|
||||
- **N210-35** restart during expiry/retry/quarantine reconstructs durable disposition and eligibility.
|
||||
- **N210-36** restart with pending outbox reconstructs publication work without Valkey/files.
|
||||
- **N230-01** author=self-review and missing mandatory SecReview cannot certify or complete.
|
||||
- **N230-02** normal application role cannot execute retention purge.
|
||||
- **N230-03** break-glass purge cannot delete or alter its own authorization/evidence chain.
|
||||
- **N230-04** Valkey down leaves canonical work in PostgreSQL/outbox.
|
||||
- **N230-05** duplicate wake produces one logical effect after PostgreSQL reload/idempotency.
|
||||
- **N230-06** stale wake cannot revive an expired/revoked assignment or lease.
|
||||
- **N230-07** restart with no Valkey/files reconstructs leases/retry/quarantine/outbox exactly.
|
||||
|
||||
### KBN-300/KBN-320/KBN-330/KBN-340 — migration and cutover
|
||||
|
||||
- **N300-01** source record targeting another workspace is denied/quarantined without an oracle.
|
||||
- **N300-02** malformed source record is rejected with attributable reject evidence.
|
||||
- **N300-03** duplicate source system/key/batch replay is idempotent.
|
||||
- **N300-04** source snapshot/checksum drift aborts apply/verify.
|
||||
- **N300-05** partial import resumes from durable lineage without duplicating state/events.
|
||||
- **N300-06** imported shadow record cannot become ready, assigned, or leased automatically.
|
||||
- **N300-07** missing source key/file/checksum/batch lineage prevents apply/sign-off.
|
||||
- **N300-08** importer cannot use direct DB, generated file, Valkey, or provider issue as canonical write authority.
|
||||
- **N320-01** cutover without a verified write freeze fails safe.
|
||||
- **N320-02** active legacy writer process or credential blocks cutover.
|
||||
- **N320-03** reverse and forward synchronization cannot run concurrently.
|
||||
- **N320-04** failed final delta/reconciliation blocks client switch.
|
||||
- **N320-05** rollback before first canonical DB mutation may switch authority back only after freeze assertion.
|
||||
- **N320-06** rollback after first canonical mutation requires freeze, DB-delta export/reconciliation, and owner decision.
|
||||
- **N330-01** rehearsal cannot sign off while counts/checksums/exceptions/writer inventory differ.
|
||||
- **N340-01** cutover cannot proceed without owner authorization, terminal evidence, scoped identities, and zero active legacy writers.
|
||||
|
||||
## 7. Requirements traceability
|
||||
|
||||
| Requirement | Threats/impacts | Planned evidence |
|
||||
| ---------------- | ---------------------------- | --------------------------------------------------------------- |
|
||||
| REQ-SOT-001 | T16, T21, T22, T27, T29, T30 | N110-28..31, N110-44..49, N110-55..58, N120-03..07, N320-01..06 |
|
||||
| REQ-SOT-002 | T07, T08, T09, T21 | N105-02/03, N110-14..27, N110-44..48 |
|
||||
| REQ-SOT-003 | T30 | N120-03..07, N140-04 |
|
||||
| REQ-SOT-004 | T16..18 | N100-24..26, N110-28..43 |
|
||||
| REQ-TEN-001 | T01..05, T15, T33 | N100-01..14, N100-45..50, N110-01..09, N210-15/16 |
|
||||
| REQ-ID-001 | T02, T03, T06, T10..12 | N105-01, N110-06..13, N210-01..19 |
|
||||
| REQ-PLAN-001 | T04, T25 | N100-06..10 |
|
||||
| REQ-TASK-001 | T13, T26, T31 | N100-20, N100-37..42, N110-50..54 |
|
||||
| REQ-TASK-002 | T16, T24 | N110-28..31, N100-35, N200-01 |
|
||||
| REQ-DEP-001 | T24 | N100-32..35, N200-01 |
|
||||
| REQ-ASN-001 | T10..12 | N100-15..18, N210-03..19 |
|
||||
| REQ-AUD-001 | T17..20, T22, T27 | N100-24..31, N110-32..43, N110-49, N110-55..58 |
|
||||
| REQ-API-001 | T01, T06..18, T26 | N105-01..05 plus KBN-110 catalog |
|
||||
| REQ-UI-002/003 | T01, T15, T26 | N130-01 and real-Gateway KBN-140 journeys |
|
||||
| REQ-COORD-001 | T22..24 | N200-01/02, N210-32..36 |
|
||||
| REQ-COORD-002 | T10..12, T16 | N210-03..19, N110-28..31 |
|
||||
| REQ-COORD-003 | T13..15, T23 | N100-19..23, N210-20..36 |
|
||||
| REQ-COORD-004 | T23, T26 | N210-32..36, N230-07 |
|
||||
| REQ-GATE-001/002 | T11, T19, T20 | N210-09..14, N230-01..03 |
|
||||
| REQ-REC-001 | T20, T32 | N115-01..08 |
|
||||
| REQ-MIG-001/002 | T28, T29, T31 | N100-37..44, N300-01..08, N320-01..06, N330-01, N340-01 |
|
||||
|
||||
REQ-UI-001 and REQ-UI-004 are downstream functional/accessibility requirements rather than schema-threat controls; they remain owned by KBN-130/KBN-140. Their security-relevant tenancy, conflict, and stale-reconnect portions are covered above.
|
||||
|
||||
## 8. Issue #753 acceptance mapping
|
||||
|
||||
| Issue requirement/criterion | Evidence in this document | Result |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------- |
|
||||
| Cross-workspace owners, principals, evidence, project hierarchy | T01–T05, T15, T25; CI-01–04 | Mapped |
|
||||
| Active membership and service-token boundaries | Authorization matrix; T02, T03, T06; CI-02/03 | Mapped |
|
||||
| Stale/forged health and transaction-local proof | T07–T09, T21; CI-05 | Mapped |
|
||||
| Assignment/approval forgery and monotonic fencing | T10–T15; CI-06/07 | Mapped |
|
||||
| Change-proposal abuse and event binding | T16–T18; CI-08 | Mapped |
|
||||
| Immutable audit and break-glass | T19/T20; CI-09 | Mapped |
|
||||
| PostgreSQL/Valkey failures | T21–T23; CI-10 | Mapped |
|
||||
| Dependency/idempotency/version races | T24–T27; CI-11 | Mapped |
|
||||
| Import/cutover and generated-file boundary | T28–T31; CI-12/13 | Mapped |
|
||||
| Every schema/API/test impact explicit | Constraint matrix and negative-test catalog | Mapped |
|
||||
| No unresolved schema impact | CI-14; rc.4 resolved-impact record | **PASS — none unresolved** |
|
||||
| Independent SecReview | Homelab non-author exact commit/tree/content review | **PASS / APPROVE** |
|
||||
| PR merge, terminal-green main CI, and #753 closure | Orchestrator-owned post-worker gates | Pending; KBN-100 remains held until completion |
|
||||
|
||||
## 9. UNRESOLVED SCHEMA IMPACTS
|
||||
|
||||
none
|
||||
|
||||
### Resolved-impact record — KBN010-SI-001
|
||||
|
||||
- **Historical detection:** rc.3 lacked an exact `(workspace_id,id)` candidate key for the artifact and approval-decision mission FKs. This document's original BLOCKED verdict was correct and remains preserved in §1 and T33.
|
||||
- **Resolution:** rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` before both exact dependent FKs while retaining the global primary key and project-congruent key.
|
||||
- **Reviewed object:** commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`, tree `7ebab8fa530a7180036928cea9527f808548aa14`.
|
||||
- **Corroborating identities:** full-index SHA-256 `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`; stable patch-id `058cf98026fcd1043703c866aee047c8bb144740`.
|
||||
- **Independent verdict:** Homelab non-author schema/security review **APPROVE**. It confirmed PostgreSQL candidate/FK validity, unchanged tenant and polymorphic exactly-one-target safety, RESTRICT/no-cascade semantics, N-1/rollback validity, and no shared table/index/FK/identity/fence authority collision with #757.
|
||||
- **Digest interpretation:** a command-rendered patch digest varied with rendering command/options and is non-authoritative. Git commit + tree + exact file content are canonical; stable full-index SHA-256 and stable patch-id corroborate that identity.
|
||||
- **Residual implementation obligations:** KBN-100 must create the candidate before both dependent FKs; prove production-shape duplicate feasibility without weakening global uniqueness; pass empty/prod/N-1/rollback tests; reconcile both exact FK targets; and separately reject foreign-workspace mission references for artifacts and approval decisions (N100-45..50).
|
||||
- **Implementation status:** no runtime schema, migration, API, or deployment implementation is claimed by this gate disposition.
|
||||
|
||||
## 10. Residual risk and handoff
|
||||
|
||||
- Active membership, polymorphic targets, same-task evidence semantics, parent/DAG cycle checks, token scope, and no-oracle behavior depend on authoritative transaction code and must not be treated as FK-only guarantees.
|
||||
- DB superuser and break-glass compromise cannot be eliminated by application constraints; separation of duties, immutable external backup/audit evidence, drills, and monitoring remain required.
|
||||
- PostgreSQL unavailability intentionally sacrifices writes for integrity. Transport-unknown outcomes remain safe only when clients preserve the exact idempotency key.
|
||||
- Imported ambiguous records remain quarantined until owner sign-off; no automated mapping may convert ambiguity into authority.
|
||||
- SI-001 is resolved at frozen contract/design-review level only. KBN-100 still owes N100-45..50 executable migration evidence.
|
||||
|
||||
**Handoff status:** KBN-010 **PASS / GO** at rc.4. KBN-100 remains held until this PR squash-merges, terminal-green CI completes on `main`, and issue #753 closes; the orchestrator owns those remaining gates.
|
||||
266
docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md
Normal file
266
docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md
Normal file
File diff suppressed because one or more lines are too long
@@ -1,13 +1,106 @@
|
||||
# Native Kanban/SOT — Remediated Shared Contract v1
|
||||
|
||||
**Status:** INDEPENDENT REVIEW GO; freezes as v1 when issue #751 canon merges to `main`
|
||||
**Version:** 1.0.0-rc.3
|
||||
**Date:** 2026-07-14
|
||||
**Status:** CONTROL-PLANE rc.16 KBN-101 current generic storage-wrapper authority remediation complete; awaiting independent exact-head re-review. Prior KCR-001–016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
|
||||
**Version:** 1.0.0-rc.16
|
||||
**Date:** 2026-07-15
|
||||
**Change authority:** Mosaic control plane/Jason only
|
||||
**SI-001 amendment authority:** `web1:mosaic-100` control-plane decision under issue #753
|
||||
|
||||
## Amendment record
|
||||
|
||||
### 1.0.0-rc.16 — Current generic storage-wrapper authority closure
|
||||
|
||||
- **Current-source truth:** `packages/storage/src/cli.ts` currently shells `storage migrate --run` directly to `pnpm --filter @mosaicstack/db db:migrate` through `execSync`; no `mosaic-db-migrator` executable exists. README and user-guide command guidance therefore remove that command and any runner-delegation claim. The current wrapper is legacy N-1, uncertified, non-operative, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||
- **Future-only boundary:** future schema migration remains non-operative and follows external bootstrap → TLS/roles → runner `--run` → runner `--verify` → readiness; tier copy uses only the separately held secure migrate-tier route.
|
||||
- **Unmaskable semantic/source-consistency evidence:** before inventory, ownership, or status masking, -06 fails the exact former README commented code-fence generic-wrapper form and exact user-guide executable generic-wrapper form. Its source-consistency test proves the direct-Drizzle `execSync` target and absent runner bin, so any documentation describing current wrapper delegation to the runner fails.
|
||||
- **Non-effect:** prior runner, legacy-CI, Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||
|
||||
### 1.0.0-rc.15 — Held runner and legacy-CI authority closure
|
||||
|
||||
- **Held runner only:** Current operator documents cannot advertise `mosaic-db-migrator --run|--verify` as executable. The sole passing future form is one `Held future procedure` Markdown section, bounded through its next equal-or-higher heading, that explicitly says non-operative/no-current-command-authority, names KBN-101-00/-03/-05, and preserves external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Any runner hit outside that section fails before inventory/ownership/status masking.
|
||||
- **PGlite/current-CI boundary:** Fleet backlog current behavior is PGlite-only; PostgreSQL CLI/runner authority remains held until activation. README classifies the checked-in direct `db:migrate` CI job as active legacy N-1, uncertified, non-authorizing as an operator route, and pending KBN-101-06 removal; it is a known direct-DDL exception against an isolated disposable CI database, not approved ordinary behavior. The -06 fixture asserts every required status term and rejects ordinary-authority presentation.
|
||||
- **Non-effect:** prior Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||
|
||||
### 1.0.0-rc.14 — Current Compose and production-secret route closure
|
||||
|
||||
- **Current developer boundary:** `README.md` and `docs/guides/dev-guide.md` permit only in-process PGlite data-layer work and explicitly selected non-PostgreSQL Compose services. Gateway/Web local start is held because the current unguarded loader can inherit a daemon/project PostgreSQL DSN and reach runtime DDL; KBN-101-02 must reject it before connection. The current PostgreSQL Compose mount is legacy/unqualified; PostgreSQL and federated activation are held until KBN-101-00/-03/-05 and then follow external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway/Compose readiness.
|
||||
- **Production boundary:** `docs/guides/deployment.md` is non-operative until the KBN-101-05 renderer-backed process-exec or `LoadCredential` interface exists. It contains no active production environment-file, monorepo auto-load, credential export/argv, or secret-activation lifecycle route; future units must preserve generation-pinned Vault consumer isolation.
|
||||
- **Unmaskable semantic negatives:** -06 fails the exact former README/dev/deployment Compose-first sequences and every production `.env`, `EnvironmentFile=`, credential export/argv, or restart-as-secret-activation fixture before owned/status/normative classification. The held PGlite/non-PostgreSQL route and future ordered activation are the only passing fixtures.
|
||||
|
||||
### 1.0.0-rc.13 — Federation-MILESTONES indirect-startup closure
|
||||
|
||||
- **Complete operator inventory:** `docs/federation/MILESTONES.md` is exclusively KBN-101-07 and an exact KBN-101-06 `operator-document` `status-only` record. Its former `pgvector extension installed + verified on startup` wording is superseded and forbidden; it authorizes no current DDL, Compose/init, or runtime/startup path.
|
||||
- **Unmaskable semantic negative:** before inventory disposition, the scanner fixture proves that exact former wording fails. The only passing status-only sequence is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway readiness.
|
||||
|
||||
### 1.0.0-rc.12 — Deployable importer generation and indirect-DDL-route closure
|
||||
|
||||
- **Authenticated generation:** KBN-101-05 owns one canonical Vault KV-v2 importer record, `secret-{env}/mosaic-stack/database/importer` key `url`, with its version taken only from the same successful `data.metadata.version` response. Value plus provider version are one generation, never inferred from DSN bytes. The renderer creates separate immutable `0400` URL/version copies for migrator `10003:10003` binding-only access and importer `10002:10002` access; it uses fsync/atomic generation replacement for Compose and distinct versioned secret/config references for Swarm, so deployment cannot mix generations.
|
||||
- **Bounded consumers:** importer alone receives its URL/version, CA at `DATABASE_TLS_CA_CERT_PATH`, pinned public key, and read-only attestation; migrator receives its own migration URL/CA, the URL/version only for no-connect/no-export binding, attestation output, and the root-wrapper-only private key. Safe fd open/fstat/digest/zeroize/close semantics, a privileged producer-only-to-importer-only attestation handoff controller (verify, exact-byte copy, fsync/atomic rename, `10002:10002` `0400` seal, then importer start), no shared writable file, no logging/oracle, provider rotation/revocation, CA/mount, consumer-isolation, and symlink/hardlink/owner/mode/TOCTOU negatives are mandatory.
|
||||
- **Indirect-DDL closure:** `docs/federation/SETUP.md` is non-operative until KBN-101 activation and documents only external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway readiness. The -06 scanner performs unsuppressible semantic checks for automatic first-boot/startup extension/schema/migration language, Compose-up-before-runner, and init-script authority; the former SETUP wording fails and the remediated sequence passes.
|
||||
|
||||
### 1.0.0-rc.11 — Target-bound importer attestation and exhaustive operator-route closure
|
||||
|
||||
- **Target-bound proof:** trusted `mosaic-db-migrator --verify` now produces the atomic, credential-free `migrate-target.v1.json` JCS/Ed25519 artifact from a runner-only root-owned signing-key reference; the importer receives only pinned public verification keys and the artifact. Its signed v1 fields bind issued/expiry/nonce, exact secret version and SHA-256 of high-entropy target-file bytes, canonical TLS host/port/database, CA/SPKI, PostgreSQL system identifier/database OID, expected importer role, manifest/schema fingerprints, and producer invocation/build/image/correlation. No DSN, username, password, credential bytes, or signing key enters the artifact, importer, runtime, logs, or output.
|
||||
- **Fail-closed importer:** `mosaic storage migrate-tier` requires both `--target-url-file /run/secrets/mosaic-migrate-target-url` and `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`. Before target connection it validates files, signature/key/expiry/replay, secret version/digest, TLS/CA/role/manifest bindings and opens/digests/connects from the same in-memory URL bytes. After verified TLS but before transaction/DML it matches server ID, database OID, `current_user`, CA/SPKI, and manifest/schema; failure distinguishes zero connection from connection/zero-DML and DDL remains impossible. Rotation overlap/revocation, atomic rename, replay cache, secret rotation invalidation, and wrong/substituted/stale/tampered/file-change tests are mandatory.
|
||||
- **Closed documentation surface:** KBN-101-06 inventories every current non-normative scanner hit, including `docs/guides/user-guide.md` and status-only `docs/federation/TASKS.md`; the latter is historical and cannot authorize DDL. The legacy `storage migrate` tier-copy syntax is unavailable. `storage migrate` is schema-wrapper delegation only; secure tier data copy is `migrate-tier`. Exact KBN PRD/contract/shared/task paths may be `normative-contract` scan class but are still scanned and cannot mask executable instructions. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
- **Non-effect:** pgvector closure, manifest, lock, role graph, TLS, activation, and KBN-100/KBN-105 serial gates are unchanged.
|
||||
|
||||
### 1.0.0-rc.10 — PostgreSQL-valid untrusted pgvector owner and active migrate-tier closure
|
||||
|
||||
- **Valid extension authority:** PostgreSQL 17 + pgvector 0.8.2 `vector` is untrusted (`trusted` absent; `relocatable=true`), so `mosaic_extension_owner` is exactly `NOLOGIN SUPERUSER`, not `NOSUPERUSER`. It is dedicated solely to `mosaic_extensions`, `vector`, and owner-bearing extension members; `rolcanlogin=false`, `rolsuper=true`, zero members, no runtime credential/Vault secret, and no app-container delivery are catalog and deployment proof. An externally controlled audited bootstrap-superuser session alone `SET ROLE`s for extension CREATE/UPDATE/SET SCHEMA, then `RESET ROLE`; fresh and shadow paths do so, while in-place existing work requires exact pre-existing `extowner`.
|
||||
- **Explicit superuser exception:** `GRANT`/`REVOKE` cannot privilege-limit a superuser. The containment is dedicated identity, no login, no membership, external control plane, audit, independent review, backup/rollback, and maintenance window—not a false least-privilege claim. Runtime, migrator, schema owner, importer, and every service role cannot assume the role or alter/update/drop/change extension membership. Managed targets without this exact role are ineligible unless a versioned provider-owned extension-owner profile is independently approved.
|
||||
- **Active secure data-migration route:** `docs/guides/migrate-tier.md` is exclusively KBN-101-07, is active rather than historical, and specifies runner-prepared/verified PostgreSQL destination plus a dedicated non-DDL importer. KBN-101-02 freezes `--target-url-file /run/secrets/mosaic-migrate-target-url`, never credential argv; raw `--target-url`, `DATABASE_URL` fallback, runtime owner, missing/unsafe file, wrong mode, and DDL all fail before target connection/DDL. KBN-101-06 inventory/matrix records the route and exact secure fields, then tests its finite operator-document closure.
|
||||
- **Non-effect:** manifest, lock, `mosaic` application schema, TLS, activation, and KBN-100/KBN-105 serial gates remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.9 — KBN-101 extension-schema boundary, disjoint manifests, and scanner mechanics
|
||||
|
||||
- **Extension schema owner:** `mosaic_extension_owner`, not `mosaic_schema_owner`, creates and owns `mosaic_extensions`, `vector`, and extension-member objects. The external bootstrap actor `SET ROLE`s for fresh creation or approved-owner relocation, then `RESET ROLE`s; rc.10 replaces the earlier membership wording with the PostgreSQL-valid zero-member superuser exception. Schema owner has only `USAGE` for legacy type resolution—never ownership, `CREATE`, `ALTER`, `DROP`, member change, or default-privilege authority. Runtime, migrator, and schema owner must fail catalog and direct DDL denials; shadow/resume/rollback repeat the owner/default-privilege proof.
|
||||
- **Exclusive delivery DAG:** KBN-101-00…09 now has a complete, nonoverlapping exact file/glob manifest with named tests/evidence. The runner mapping is exactly `"mosaic-db-migrator": "./dist/cli.js"` and image `ENTRYPOINT ["mosaic-db-migrator"]`; `packages/storage/src/{cli,migrate-tier}.ts` belongs only to -02, and -07 is documentation only. -08/-09 own evidence paths only. -00…07 are prepared artifacts; the immutable N-1 image remains live until -08 atomic activation, so no independently deployed intermediate can bypass runtime controls.
|
||||
- **Mechanical classifier:** -06 owns the exact scanner, inventory fixture, command-matrix harness, and CI wiring. Inventory records pin path/class/owner/disposition/allowed tokens/rationale/expiry/review revision; unknown, duplicate-owner, ownerless, missing-path, invalid allowlist, and historical-category masking fail. The architecture plan's operative direct `db:migrate` is replaced by sole-runner guidance rather than hidden under a historical category.
|
||||
- **Non-effect:** manifest v1, lock, `mosaic` application-schema ownership, TLS, activation, KBN-100/KBN-105 serial gates, and all earlier canon decisions remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.8 — KBN-101 finite authority, executable runner, and pgvector-owner remediation
|
||||
|
||||
- **Finite authority closure:** KBN-101-06 classifies every current executable source/script/package bin, operator document, and deploy manifest by exact path; unclassified current hits fail. Byte-immutable historical SQL, PGlite-only routines, negative-test literals, vendored/generated artifacts, and clearly labeled historical reports are exact-path/category reviewed allowlists only. `packages/db/src/index.ts` loses its public `runMigrations` export with a direct-import/compile negative; `docs/fleet/backlog-conventions.md` and `docs/PERFORMANCE.md` lose first-use/direct-Drizzle/Gateway-startup migration instructions and carry runner/readiness route negatives. A token scan is only input to the classifier, never proof of authority.
|
||||
- **Executable exclusive cards:** KBN-101-03 alone publishes `mosaic-db-migrator` from `packages/db/package.json`/`src/cli.ts`, owns `docker/db-migrator.Dockerfile`, and keeps `{runner,config.dto,manifest,identity,tls}` private, with exact `--run|--verify|--help`, env-only input, stable exits, and command tests. KBN-101-00 alone owns `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, plus bootstrap tests. KBN-101-05 alone owns `tools/db/render-postgres-secrets.ts`, renderer tests, and Compose/Portainer/Swarm/two-gateway declarations, consuming the versioned bootstrap interface. No card overlaps renderer/bootstrap/deployment ownership.
|
||||
- **Extension-owner transition:** `mosaic_extension_owner` is a dedicated NOLOGIN role whose membership/credentials never reach services; the external bootstrap actor alone may `SET ROLE` during bootstrap. Fresh vector and member objects retain that owner. PostgreSQL has no supported extension-owner alteration: approved-owner existing extension relocation validates `pg_extension.extowner`, members/schema/version and uses tested `ALTER EXTENSION ... SET SCHEMA`; legacy runtime-owned extension fails closed to a controlled shadow database migration with backup, evidence, quiesce/final delta, atomic switch, and read-only rollback window. No catalog mutation, ownership adoption, or `DROP CASCADE` is permitted. Runtime/migrator/schema-owner extension ALTER/DROP/member-update denial is mandatory.
|
||||
- **Non-effect:** manifest v1, lock namespace, role/search-path, relocation/TLS/activation, KBN-100/KBN-105 serial gates, and all retained canon decisions are strengthened, not weakened. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.7 — KBN-101 complete current-path, relocation, and two-gateway remediation
|
||||
|
||||
- **Finite current-path closure:** static inventory and the `DATABASE_URL`-only-before-connect/DDL denial matrix now explicitly include Gateway's former temporary-table pgvector test (runner-prepared persistent read/query-only fixture), `docker/init-db.sql` retirement, `migrate-tier.ts` runner/bootstrap-only guidance, and the active two-gateway harness. The harness is migrated, not retired: `postgres-a/b → mosaic-db-migrator-a/b → gateway-a/b`, each with isolated URL/CA material, verified readiness, SANs, and positive/negative TLS evidence.
|
||||
- **Executable relocation:** KBN-101-03 exclusively owns `schema.ts`, Drizzle snapshots/journal/generated relocation and exact tests. All future application declarations use exported `pgSchema('mosaic')`; immutable historical SQL runs only in trusted legacy `public`. `vector` is fixed in non-writable `mosaic_extensions`, with exact catalog relocatability/version eligibility, explicit type/operator qualification, catalog-class ordering, unknown-object fail-closed behavior, clean/current-public/partial/reverse rollback tests, and an N-1 release order.
|
||||
- **Bound deployment ownership:** `mosaicstack/stack` KBN-101-00/05 owns current Compose, Portainer, two-gateway, bootstrap renderer/templates, UID/GID declarations, and rendered validation. Gateway is fixed to `10001:10001`; PostgreSQL UID/GID is image-inspected and frozen only after digest pinning. Exact secret paths, atomic renderer behavior, Compose/Swarm targets/modes, Gateway/PostgreSQL leaf separation, and two-pair TLS failure evidence are required. Mosaic deployment control plane/Jason is the named activation authority; environment IaC/Vault supplies versioned input only.
|
||||
- **Correct traceability:** REQ-03 maps to role/schema/search-path, REQ-04 to TLS, REQ-05 to post-KBN-100 immutability, REQ-06 to rollout/rollback, and REQ-07 to the KBN-101 → KBN-100 → KBN-101 → KBN-105 sequence. No prior manifest/lock/role/DAG/activation decision is weakened.
|
||||
|
||||
### 1.0.0-rc.6 — KBN-101 closed DDL/TLS/ledger activation remediation
|
||||
|
||||
- **Choice:** `mosaic-db-migrator` is the sole application/CI/test PostgreSQL DDL control plane. Every legacy entrypoint is routed or denied, rejects `DATABASE_URL`-only before connection/DDL, and `db:push` is unavailable outside an allowlisted disposable developer target. The runner holds one `max:1` session with fixed `pg_try_advisory_lock(1297044289,1262636593)` across preflight through release.
|
||||
- **Exact ledger:** manifest v1 canonically serializes journal logical index/tag and SHA-256 of exact shipped migration bytes. It maps each observed ledger hash to one tuple; physical insertion order is non-normative, while missing/unknown/duplicate/ambiguous/corrupt/stale states fail closed. Shipped `0009` bytes remain unchanged; a missing/effects-absent `0009` runs normally, an applied-late hash maps normally, and partial/full effects with missing hash require backup restoration or separately reviewed repair—not manual adoption.
|
||||
- **TLS/search path:** operator/IaC owns CA and server leaf lifecycle, exact compose/Swarm secret mounts, server TLS activation, service-DNS SANs, verified-TLS readiness, transition, CA overlap rotation, and rollback. Runtime/migrator use `verify-full`; PGlite is not PostgreSQL TLS evidence. Application sessions use only `pg_catalog,mosaic`; no URL/config-derived identifier reaches SQL.
|
||||
- **Safe release:** cards 00–07 land prepared but inactive; owner-runtime deployments remain N-1. Mosaic control plane/Jason alone authorizes one atomic TLS/roles → runner → readiness → runtime activation or rollback. No runtime-operator compatibility switch, bypass, plaintext interval, or force-on-red exists; all temporary support is removed before KBN-101-08.
|
||||
- **Non-effect:** role graph, immutable certification after KBN-100, KBN-105 gate, rc.5’s preserved rc.4 SI-001 invariants, and all KCR-001–016 decisions remain unchanged. Exact detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.5 — KBN-101 role/connection split
|
||||
|
||||
- **Choice:** PostgreSQL `standalone` and `federated` runtime uses `DATABASE_URL` only as a non-owner `mosaic_runtime` login; an explicit migration phase uses `DATABASE_MIGRATION_URL` only as `mosaic_migrator`, which `SET ROLE`s to non-login `mosaic_schema_owner` for DDL. Local PGlite remains an explicit embedded exception.
|
||||
- **No fallback / no startup DDL:** missing migration URL fails the migration phase; it never falls back to runtime URL/default/config. Gateway replicas do not run migrations. An advisory-locked migration phase verifies the exact ordered Drizzle ledger fingerprint before replicas may become ready.
|
||||
- **Privilege model:** non-login `mosaic_platform_database_owner` is outside application paths; `mosaic_schema_owner` owns only application/ledger schemas. `mosaic_runtime` has only `mosaic_runtime_capability`, owns no object/schema, cannot assume owner/migrator, has no TEMPORARY privilege, has only read access to the Drizzle ledger, and must fail startup if effective identity, unsafe attributes, authenticated TLS, search path, schema version, grants, or immutable relation privileges differ from the frozen contract. `task_events`, `artifacts`, `task_checkpoints`, `task_checkpoint_artifacts`, and `approval_decision_artifacts` grant runtime only INSERT/SELECT; KBN-100 retains RESTRICT/no-cascade semantics.
|
||||
- **Non-effect:** rc.4 SI-001 candidate-key/FK order and all KCR-001–016 tenancy, SOT, proposal-audit, approval, fence, recovery, no-cascade, endpoint, and wire invariants are unchanged. This amendment neither creates roles/secrets nor changes production deployment.
|
||||
- **Gate:** KBN-101’s role/schema-boundary foundation certificate, Vault/redaction/rotation, N-1/rollback, and independent security GO are mandatory before KBN-100. After KBN-100 creates the immutable relations, KBN-101 real deployed-role immutable-operation certification plus Ultron GO is mandatory before KBN-105; synthetic test-role success alone is insufficient. Exact implementation detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.4 — KBN010-SI-001 (preserved)
|
||||
|
||||
- **Choice:** add the explicitly named, non-partial unique candidate key `missions_workspace_id_uidx` on `missions(workspace_id, id)` and retain `missions_workspace_project_id_uidx` on `(workspace_id, project_id, id)`.
|
||||
- **Rationale:** mission `id` remains globally unique, while the composite candidate key makes the frozen tenant-safe generic mission relations valid. `artifacts` and `approval_decisions` are polymorphic exactly-one-target records and do not consistently carry `project_id`; widening both children would unnecessarily broaden v1 and its target semantics.
|
||||
- **Exact effect:** `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk` continue to reference the exact ordered columns `missions(workspace_id, id)` with RESTRICT deletion, now backed by a matching candidate key.
|
||||
- **Non-effect:** no SOT, tenancy, project-congruence, proposal-audit, approval, fencing, immutability, no-cascade, API, or wire-version invariant changes. The `SuccessEnvelopeV1.contractVersion` remains `1.0.0`.
|
||||
- **Historical evidence boundary:** `KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md` intentionally remains the immutable rc.3 blocker verdict that detected SI-001; this rc.4 record and the #753 scratchpad append are the authorized disposition. Rewriting the gate verdict is outside this amendment's exclusive scope.
|
||||
- **Gate:** this amendment resolves the DDL defect identified by KBN010-SI-001 but does not itself lift KBN-100; independent schema/SecReview remains required.
|
||||
|
||||
## 1. Authority
|
||||
|
||||
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
|
||||
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.13 DDL/ledger/TLS/role/attestation/generation separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
|
||||
|
||||
## 2. Health proof and exact failures
|
||||
|
||||
@@ -43,6 +136,7 @@ Complete declaration: `contracts/kanban-schema.v1.ts`.
|
||||
- Specialist roles everywhere: `planning | enhance | coder | review | security-review | pr-monitor | certifier`.
|
||||
- Owner uses exactly-one user/team; assignment principal exactly-one user/team/agent; users require active membership; agent/session and all evidence are workspace-bound.
|
||||
- Task→mission/milestone/parent, mission→milestone, and project→current-milestone are project-congruent composite relations.
|
||||
- Mission `id` remains globally unique. The additional non-partial `missions_workspace_id_uidx` candidate key on `(workspace_id, id)` exists only to support the frozen workspace-safe polymorphic artifact and approval-decision mission relations; the project-congruent `(workspace_id, project_id, id)` key remains authoritative wherever `project_id` is present.
|
||||
- Dependency identity is workspace+predecessor+successor independent of type.
|
||||
- Approval evidence and checkpoint evidence are workspace-scoped joins to immutable artifacts, never JSON ID arrays.
|
||||
- Proposal audit links are composite relations: `(workspace_id, submitted_audit_event_id)` and `(workspace_id, accepted_command_audit_event_id)` reference `task_events(workspace_id, id)` with RESTRICT deletion.
|
||||
@@ -76,9 +170,22 @@ Legacy columns remain declared in unified `schema.ts` for expand + full N-1/roll
|
||||
6. **Switch:** stop N-1 writers; Gateway sole command boundary; enable canonical statuses.
|
||||
7. **Contract release:** later release after rollback/N-1; remove compatibility/global uniques/legacy fields.
|
||||
|
||||
### 5.2 New audit/proposal DDL order
|
||||
### 5.2 Mission candidate-key and dependent-FK DDL order
|
||||
|
||||
KBN-100 migration DDL must execute in this order:
|
||||
KBN-100 migration DDL must execute the SI-001 portion in this order:
|
||||
|
||||
1. expand/backfill `missions.workspace_id` and `missions.project_id` while preserving the global `missions.id` primary key and the project-congruent `missions_workspace_project_id_uidx` key;
|
||||
2. prove duplicate-key feasibility on the production-shape dataset: `(workspace_id, id)` has no duplicate groups and global `id` uniqueness remains intact;
|
||||
3. create the non-partial unique index `missions_workspace_id_uidx` on exact ordered columns `(workspace_id, id)`;
|
||||
4. only after step 3, create/alter `artifacts` and add `artifacts_workspace_mission_fk` from `(workspace_id, mission_id)` to exact `missions(workspace_id, id)` with `ON DELETE RESTRICT`;
|
||||
5. only after step 3, create/alter `approval_decisions` and add `approval_decisions_workspace_mission_fk` from `(workspace_id, mission_id)` to exact `missions(workspace_id, id)` with `ON DELETE RESTRICT`;
|
||||
6. validate both constraints and prove a mission ID paired with a foreign workspace is rejected for each child.
|
||||
|
||||
The candidate key is intentionally redundant with globally unique `missions.id`, but PostgreSQL requires a matching unique candidate key for the exact two-column FK target. It is additive and N-1-safe. Pre-switch rollback drops the two dependent FKs/tables before dropping this candidate key, preserves the global primary key and project-congruent key, and follows the existing freeze/reconciliation rule after the first canonical mutation.
|
||||
|
||||
### 5.3 New audit/proposal DDL order
|
||||
|
||||
KBN-100 migration DDL may begin only after KBN-101 foundation role/schema-boundary certification. It runs in the explicit migrator/owner phase—not Gateway startup—and its generated Drizzle declaration/snapshot/journal must be mutually consistent. It must execute in this order:
|
||||
|
||||
1. create `task_events` and its unique `(workspace_id, id)` key;
|
||||
2. create `change_proposals` with nullable acceptance-event ID and required submission-event ID;
|
||||
@@ -88,15 +195,16 @@ KBN-100 migration DDL must execute in this order:
|
||||
|
||||
The submission transaction inserts the event first using a preallocated proposal UUID, then the proposal. Acceptance inserts the normal command event before updating the locked proposal. Neither FK is omitted or replaced by a bare UUID/index check.
|
||||
|
||||
### 5.3 Field map
|
||||
### 5.4 Field map
|
||||
|
||||
| Current | Expand/backfill | N-1 compatibility | Switch/contract |
|
||||
| ---------------------------------------------- | -------------------------------------------------------------------------------------- | ----------------------------------------------- | ------------------------------------------------------------- |
|
||||
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| global `teams`, `team_members` | add workspace nullable; bootstrap; validate active owners | retain global slug/FKs | workspace composites; global unique contracts later |
|
||||
| `projects.status` | add `canonical_status`; map active/paused/completed/archived | mirror representable values; no `planning` | canonical authority; legacy contracts later |
|
||||
| project `owner_id/team_id/owner_type` | add exact accountable user/team; deterministic map or quarantine | preserve old reads and compare drift | canonical exact-one; remove legacy after parity |
|
||||
| current milestone | create milestones then join table (no circular DDL) | absent to N-1 | join is authority |
|
||||
| nullable `missions.project_id` | derive workspace/project; null/orphan exception, never guess | keep nullable legacy read | canonical required; validate/set NOT NULL later |
|
||||
| mission relational candidate keys | retain global `id` PK and project-congruent key; add non-partial `(workspace_id,id)` key before artifact/approval FKs | additive key is ignored safely by N-1 readers/writers | retain both composite keys; generic mission children use exact workspace+ID target |
|
||||
| mission `description` | add objective; preserve description; reviewed nonblank mapping | N-1 description | objective authority; retain until signed review |
|
||||
| `missions.status` | add canonical; planning→draft, active/paused/completed/failed same | no new-only statuses emitted | canonical authority |
|
||||
| mission `milestones` JSON | normalize with source digest; preserve malformed/original | N-1 reads JSON; no reverse sync | normalized authority; JSON removed after checksum sign-off |
|
||||
@@ -114,10 +222,12 @@ The submission transaction inserts the event first using a preallocated proposal
|
||||
| agent project/owner/prompt/tools/skills/config | preserve; validate tenant; derive typed capabilities without loss | N-1 reads | removal only by separate inventory |
|
||||
| fleet `backlog` | map to designated-project tasks; edges; claimed rows quarantine | freeze claims before switch; read-only compare | task/lease authority; retire after stabilization |
|
||||
|
||||
### 5.4 Required migration tests
|
||||
### 5.5 Required migration tests
|
||||
|
||||
Empty DB; exact production-shape snapshot; crash/resume; rollback before switch; N-1 startup/read/write; workspace/member negatives; status-shadow/no premature new status; `mission_tasks.status` write prohibition; tags/assignee/date/mission JSON/config/description/agent checksum; project congruence/current-milestone order; backlog freeze/no dispatch; and proof legacy declarations persist until contract release.
|
||||
|
||||
SI-001 adds frozen future executable evidence: empty and production-shape migrations create `missions_workspace_id_uidx` before either dependent FK; duplicate-key feasibility preflight returns no `(workspace_id,id)` duplicate groups without weakening global `id` uniqueness; N-1 startup/read/write behavior is unchanged; pre-switch rollback removes dependents before the candidate key; both exact FK column lists reconcile to the candidate key; and foreign-workspace mission references fail for both artifacts and approval decisions. TDD is not applicable to this design-only amendment; KBN-100 must implement these negative migration tests before runtime schema release.
|
||||
|
||||
Proposal-specific negatives must attempt: missing submission event, foreign-workspace submission event, foreign-workspace acceptance event, same-workspace event for another proposal, event for another target aggregate, and unrelated normal-command event. Every attempt must fail atomically with no accepted proposal and no target mutation.
|
||||
|
||||
## 6. Ownership and Coordinator split
|
||||
@@ -214,6 +324,12 @@ KBN-115/coder2 owns `packages/config/src/recovery-posture.ts`, tests, and recove
|
||||
|
||||
## 9. Integration, security, and hold
|
||||
|
||||
Required release evidence includes empty/prod/partial/rollback/N-1 migration tests; cross-workspace and same-workspace wrong-project negatives; active-membership owners/principals; proposal inertness/normal acceptance; exact failure mapping; concurrent monotonic bigint fences; relational lease/checkpoint/evidence mismatch; immutability privileges/RESTRICT; recovery validation/mechanism evidence; endpoint registry alignment; accessible web journeys; author≠reviewer; mandatory SecReview; final Certifier pass/no merge authority.
|
||||
Required release evidence includes KBN-101 foundation role/schema-boundary and post-KBN-100 real immutable-operation deployed-role certificates (not synthetic roles), empty/prod/partial/rollback/N-1 migration tests; cross-workspace and same-workspace wrong-project negatives; active-membership owners/principals; proposal inertness/normal acceptance; exact failure mapping; concurrent monotonic bigint fences; relational lease/checkpoint/evidence mismatch; immutability privileges/RESTRICT; recovery validation/mechanism evidence; endpoint registry alignment; accessible web journeys; author≠reviewer; mandatory SecReview; final Certifier pass/no merge authority.
|
||||
|
||||
The build hold remains active until independent re-review reports GO for KCR-001–016. Mos alone releases waves and serializes integration roots.
|
||||
### 9.1 SI-001 amendment gate and #757 boundary
|
||||
|
||||
- KBN-100 must provide the §5.2 candidate-key ordering, duplicate-feasibility, exact-FK reconciliation, empty/prod/N-1/rollback, and two-child foreign-workspace evidence before SI-001 can be certified closed.
|
||||
- All prior KCR-001–016 decisions and fixed SOT/tenant/authority, proposal-audit, approval, task-fencing, immutability, and no-cascade invariants remain unchanged.
|
||||
- Read-only PR #757 cross-check: its logical-agent connector lease/CAS fencing uses separate runtime tables/contracts and `lease_epoch`; rc.4 changes only the frozen `missions` candidate key. There is no shared table, index, FK, identity, fence, or authority semantic to consume or reconcile, and #757 remains owned by its existing lane.
|
||||
|
||||
The build hold remains active until independent re-review reports GO for KCR-001–016 and the rc.4 SI-001 amendment. Mos alone releases waves and serializes integration roots.
|
||||
|
||||
@@ -43,8 +43,10 @@ Shared roots, package exports/manifests, lockfiles, and generated artifacts are
|
||||
```text
|
||||
KBN-000 canon remediation
|
||||
-> KBN-010 threat/auth/constraint-impact gate (MUST COMPLETE)
|
||||
-> KBN-101 foundation role/schema-boundary certificate (SERIAL)
|
||||
-> KBN-100 schema + concrete N-1 migration implementation
|
||||
├─ KBN-105 exact endpoint/DTO/error/registry freeze (SERIAL)
|
||||
├─ KBN-101 post-KBN-100 deployed-role immutable-operation certificate (SERIAL)
|
||||
│ -> KBN-105 exact endpoint/DTO/error/registry freeze (SERIAL)
|
||||
│ ├─ KBN-110 domain + Gateway + MCP server implementation
|
||||
│ ├─ KBN-120 CLI/projection implementation [coder4 first]
|
||||
│ └─ KBN-130 web MVP implementation
|
||||
@@ -64,31 +66,44 @@ KBN-310 + KBN-320
|
||||
-> KBN-340 owner-gated cutover/stabilization
|
||||
```
|
||||
|
||||
No consumer implementation begins before KBN-105. No schema work begins before KBN-010 completes. The coder4 order is always KBN-120 → KBN-200 → KBN-300 → KBN-320.
|
||||
No consumer implementation begins before KBN-105. No schema work begins before KBN-010 completes and the KBN-101 foundation role/schema-boundary certificate passes; the real immutable-operation certificate follows KBN-100 and blocks KBN-105. The coder4 order is always KBN-120 → KBN-200 → KBN-300 → KBN-320.
|
||||
|
||||
## 4. P0 — Canon, threat gate, schema, and exact API freeze
|
||||
|
||||
### KBN-000 — Remediate and publish canon
|
||||
|
||||
- **Owner:** Mos / publication control plane.
|
||||
- **Mode:** SERIAL; publication gate in progress.
|
||||
- **Status:** COMPLETE — PR #752 squash-merged as `49e8a54`; issue #751 closed; post-merge pipeline #1798 terminal success.
|
||||
- **Owner:** Mosaic publication control plane.
|
||||
- **Mode:** SERIAL; completed.
|
||||
- **IN:** Resolve KCR-001–016 in requirements, schema, health, Coordinator, recovery, migration map, and slices; independent re-review.
|
||||
- **OUT:** Feature implementation.
|
||||
- **Depends on:** none.
|
||||
- **Contract surfaces:** all canon.
|
||||
- **Evidence:** strict TS; Prettier; per-finding traceability; independent author≠reviewer GO.
|
||||
- **Evidence:** strict TS; Prettier; per-finding traceability; independent author≠reviewer GO; Ultron GO; terminal-green CI.
|
||||
|
||||
### KBN-010 — Threat, authorization, and constraint-impact gate
|
||||
|
||||
- **Owner:** coder3; independent `secrev`.
|
||||
- **Status:** IN PROGRESS — issue [#753](https://git.mosaicstack.dev/mosaicstack/stack/issues/753).
|
||||
- **Owner:** `kbn-coder3`; independent `secrev`.
|
||||
- **Mode:** SERIAL prerequisite of KBN-100.
|
||||
- **Exclusive files:** Mos-selected threat/auth docs only.
|
||||
- **Exclusive files:** `docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md` and task scratchpad only.
|
||||
- **IN:** Cross-workspace owners/principals/evidence; active membership; stale/forged health; approval forgery; fence monotonicity; audit retention; proposal target/audit-event forgery; service tokens; DB/Valkey outage.
|
||||
- **OUT:** Runtime/schema edits.
|
||||
- **Depends on:** KBN-000 independent re-review GO.
|
||||
- **Contract surfaces:** schema constraints, health proof, exact errors, command-family authorization.
|
||||
- **Evidence:** signed constraint-impact matrix; no unresolved schema-impact finding; SecReview pass.
|
||||
|
||||
### KBN-101 — PostgreSQL runtime/migration role split and deployed-role certification
|
||||
|
||||
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.16 closes HIGH-1 current generic storage-wrapper authority: README/user-guide remove `storage migrate --run` guidance and false runner delegation; current source is direct-Drizzle, legacy N-1, uncertified, non-operative, and forbidden pending -02/-03/-06/-08 activation. The -06 fixture fails both exact former forms before inventory/status masking and source-consistency rejects current direct-Drizzle wrapper as runner delegation. It awaits independent exact-head re-review; implementation remains held.
|
||||
- **Owner:** Mos integration control plane; independently reviewed by security/Ultron.
|
||||
- **Mode:** SERIAL foundation certificate blocks KBN-100; its post-KBN-100 real immutable-operation certificate blocks KBN-105.
|
||||
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; sole published `mosaic-db-migrator --run|--verify` PostgreSQL DDL path and all legacy/future entrypoint closure; active migrate-tier destination only after runner prepare/verify through exact `--target-url-file /run/secrets/mosaic-migrate-target-url`, paired authenticated provider-version file, and signed `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`; runner-only signing key/public-key isolation; canonical Vault KV-v2 target URL/version, generation-pinned renderer, importer CA/public-key/attestation plus privileged sealed producer-to-importer handoff, safe-fd/consumer-isolation/no-log-oracle, TLS/server/database/role/manifest/schema binding, expiry/replay/provider-rotation/TOCTOU/no-DML controls, and dedicated non-DDL importer; finite exact-path scanner/allowlist/active-route review plus unsuppressible automatic-startup/init/Compose-before-runner semantic negatives and every-path before-connect denial matrix; `DATABASE_TLS_CA_CERT_PATH` plus operator/IaC CA/server-key/cert lifecycle, exact service-DNS SANs, Vault/compose/Swarm mount modes, TLS server/bootstrap/rotation/rollback; PGlite exception; fixed two-int advisory lock; manifest-v1 logical-index/tag/exact-byte-SHA-256 ledger reconciliation including safe `0009`; fixed `mosaic` schema and exact `pg_catalog,mosaic` pooled session path; platform/schema/`NOLOGIN SUPERUSER` extension-owner/migrator/importer/runtime roles; approved-owner versus legacy-owner shadow pgvector transition; ownership, zero membership/no runtime secret, TEMP/ledger-read/default privilege and immutable grant proof; N-1 inactive prepared cards then atomic activation/rollback authority; Vault/redaction/observability/operator runbooks; one-card/one-PR implementation DAG.
|
||||
- **OUT:** Production mutation in this planning card; KBN-100 tables/data backfill; application API behavior; KBN-105 route/DTO freeze.
|
||||
- **Depends on:** KBN-010 completed.
|
||||
- **Contract surfaces:** [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md); `SHARED-CONTRACT.md` rc.15 amendment.
|
||||
- **Evidence:** foundation: exact `--help|--run|--verify`/exit/argv/import-negative plus DTO entrypoint negatives for every finite classified current DDL/static-bypass path (including `DATABASE_URL`-only, runner fixture, retired init, sanitized current operator guidance, both harness pairs, and `db:push` refusal); active migrate-tier paired URL/version/attestation files, signing/public-key isolation, canonical Vault KV-v2 authenticated version, generation-pinned renderer, importer CA, safe fd/TOCTOU/consumer-isolation/no-log-oracle, atomic JCS/Ed25519, digest/TLS/server/database/role/manifest/schema binding, expiry/replay/provider rotation/revocation, zero-connection versus zero-DML, prepared-target/importer/no-DDL negatives; clean/pre-0009/skipped/applied-late/duplicate/unknown/missing/corrupt/stale/backup plus public-to-`mosaic`/partial/reverse runner proof; fixed-lock contention/crash/readiness/unrelated-key tests; runtime cannot invoke migrations/DDL/TEMP; actual pgvector 0.8.2 control metadata, fresh/approved-owner existing/legacy-owner shadow/partial-resume-rollback/N-1 pgvector evidence with `rolcanlogin=false`, `rolsuper=true`, zero members, external-superuser `SET ROLE`/`RESET ROLE` audit, `pg_extension.extowner`, owner-bearing member/schema/version and runtime/migrator/schema-owner/importer/all-service-role `SET ROLE`/ALTER/DROP/member-update denial; disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus both-pair CA/SAN/downgrade/key mode/UID-GID/URL-secret consumer-isolation and legacy-drain/`hostssl` zero-plaintext negatives; exclusive bootstrap/renderer/manifest ownership test; catalog relocation/vector-query/operator/Drizzle-only-`mosaic`, role/grant/search-path/pool-reset/identifier checks; N-1/atomic TLS-only rollback/no-force-on-red rehearsal; named Vault/bootstrap-control-plane/CA-overlap/redaction/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
|
||||
|
||||
### KBN-100 — Unified Drizzle schema and concrete N-1 migration
|
||||
|
||||
- **Owner:** **coder2**.
|
||||
@@ -96,7 +111,7 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
||||
- **Exclusive files:** `packages/db/src/schema.ts`, `packages/db/drizzle/**`, DB tests.
|
||||
- **IN:** All frozen tables/joins/enums; workspace/project-congruent constraints; owners/principals; tags/archive; change proposals with both workspace-aware task-event composite FKs and frozen event-before-proposal DDL order; assignment approvals; durable execution/quarantine; monotonic bigint fence; exact checkpoint/evidence joins; RESTRICT/immutability; concrete current-main expand/backfill/switch/contract map.
|
||||
- **OUT:** Repositories, Gateway, Coordinator behavior, UI, importer.
|
||||
- **Depends on:** **KBN-010 completed**.
|
||||
- **Depends on:** **KBN-010 completed and KBN-101 foundation role/schema-boundary certificate PASS**. KBN-100 is blocked until both are terminal; it rebases on KBN-101 main, restores generated Drizzle declaration/snapshot/journal consistency, and confines procedural immutable-table grant/trigger/backfill work to its schema ownership. Its new relations are then subject to KBN-101 post-KBN-100 deployed-role certification.
|
||||
- **Contract surfaces:** `kanban-schema.v1.ts`; SHARED-CONTRACT current-main delta map.
|
||||
- **Evidence:** reviewed SQL; empty/prod-shape/partial-resume/rollback tests; N-1 app safety; legacy columns remain declared; workspace/project mismatch negatives; proposal event-FK missing/foreign-workspace tests; one active lease; monotonic fence; parent-delete RESTRICT; immutability privileges; SecReview.
|
||||
|
||||
@@ -107,7 +122,7 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
||||
- **Exclusive files:** canonical endpoint-registry/DTO contract docs; no implementation.
|
||||
- **IN:** Exact routes and methods from SHARED-CONTRACT §8; request/success/error fields; status codes; pagination/filter/revision envelopes; idempotency/expected-version headers/fields; proposal commands; health proof exclusion from public DTOs; MCP tool-to-route map.
|
||||
- **OUT:** Controller/service/client implementation.
|
||||
- **Depends on:** KBN-100.
|
||||
- **Depends on:** KBN-100 and KBN-101 post-KBN-100 deployed-role immutable-operation certification PASS.
|
||||
- **Contract surfaces:** health/error unions; schema IDs/statuses; Gateway DTO freeze.
|
||||
- **Evidence:** every FE/CLI/MCP call maps 1:1 to a route; 503/502-504/409 non-cross-map fixtures; contract digest published.
|
||||
|
||||
@@ -250,9 +265,11 @@ No consumer implementation begins before KBN-105. No schema work begins before K
|
||||
## 8. Consistent USC wave schedule
|
||||
|
||||
| Wave | coder2 | coder3 | coder4 | coder5 |
|
||||
| ---- | ------------------------- | -------------------------------------- | ------------------------------ | ------------------------------ |
|
||||
| ---- | ----------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------ | ------------------------------ |
|
||||
| 0 | Wait | **KBN-010** | Wait | Wait |
|
||||
| 1 | **KBN-100** | Review constraint implementation | Wait | Wait |
|
||||
| 0.5 | Wait | **KBN-101 foundation** Mos-controlled role/connection contract and certificate | Wait | Wait |
|
||||
| 1 | **KBN-100** after KBN-101 foundation PASS | Review bounded schema/grant implementation | Wait | Wait |
|
||||
| 1.5 | Certification support | **KBN-101 post-KBN-100 deployed-role immutable-operation certificate**, then KBN-105 | Wait | Wait |
|
||||
| 2 | **KBN-115** after KBN-100 | **KBN-105** exact freeze, then KBN-110 | **KBN-120** only after KBN-105 | **KBN-130** only after KBN-105 |
|
||||
| 3 | Review support | Finish KBN-110 | **KBN-200 after KBN-120** | Finish KBN-130 |
|
||||
| 4 | — | **KBN-210 after KBN-200** | Review/support | **KBN-220 after KBN-210 DTOs** |
|
||||
|
||||
@@ -485,6 +485,7 @@ export const missionsV1 = pgTable(
|
||||
foreignColumns: [projectsV1.workspaceId, projectsV1.id],
|
||||
}).onDelete('restrict'),
|
||||
uniqueIndex('missions_workspace_project_id_uidx').on(t.workspaceId, t.projectId, t.id),
|
||||
uniqueIndex('missions_workspace_id_uidx').on(t.workspaceId, t.id),
|
||||
index('missions_workspace_project_status_idx').on(
|
||||
t.workspaceId,
|
||||
t.projectId,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user