mosaicstack/stack
1
0
Fork 1
Code Issues 271 Pull Requests 10 Actions Packages Projects Releases 15 Wiki Activity

docs: establish canonical documentation architecture (#1210)
ci/woodpecker/push/publish Pipeline failed
Details

Browse Source
This commit was merged in pull request #1210.
This commit is contained in:
mos-dt-0
2026-08-13 17:56:13 +00:00
parent f82307c4dc
commit 7a6fb024b4
241 changed files with 4722 additions and 1579 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/archive/README.md
+35
View File
@@ -0,0 +1,35 @@
# Documentation Archive
> **Status:** Current archive index. Pages linked here are historical or superseded and are not current product, operator, or developer guidance.
Use [`docs/README.md`](../README.md) for current placement and source-of-truth rules. Historical pages remain discoverable here only when retaining their context is useful; each migration should identify a replacement or explain why the record is retained.
## Archived TUI workstream
The following branch-specific records are retained because their implementation claims and worktree paths no longer match the current checkout:
- [`TUI improvements PRD`](tui/PRD-TUI_Improvements.md) — historical Phase 7 requirements; it names the deleted `packages/cli` package.
- [`TUI improvements task ledger`](tui/TASKS-TUI_Improvements.md) — historical task/status record; its relative PRD link remains valid within this archive directory.
Do not use these pages as instructions for the current TUI. Current TUI implementation is under `packages/mosaic`; any new requirements require a separately approved plan or PRD.
## Archived missions
- [`Mission archive index`](missions/README.md) — completed and superseded CLI, harness, install UX, and storage-abstraction mission records.
Archived mission manifests and task ledgers preserve their original status and context. They do not replace current orchestrator-owned [`docs/TASKS.md`](../TASKS.md) or authorize old installation procedures.
## Archived planning
- [`Planning archive index`](planning/README.md) — historical briefs, reviews, and work-package specifications.
- [`Monorepo consolidation bundle`](planning/monorepo-consolidation/README.md) — prior Forge, MACP, and framework-plugin consolidation planning. Current package existence does not validate every historical criterion.
## Archived work records
- [`Work-record archive`](work-records/README.md) — unreferenced historical task scratchpads retained as evidence, not active status or guidance.
## Retention rules
- Preserve historical wording unless a migration task explicitly requires a rewrite.
- Label replacements and current status in the owning index rather than silently reviving archived claims.
- Do not link archive pages from current workflow instructions as if they were current.
docs/archive/missions/README.md
+37
View File
@@ -0,0 +1,37 @@
# Archived Missions
> **Status:** Historical mission index. These records describe completed or superseded delivery work and are not current task state, requirements, installation guidance, or command authority.
## CLI unification — 2026-04-04
- [Mission manifest](cli-unification-20260404/MISSION-MANIFEST.md)
- [Task ledger](cli-unification-20260404/TASKS.md)
## Harness foundation — 2026-03-21
- [Mission manifest](harness-20260321/MISSION-MANIFEST.md)
- [Scoped PRD](harness-20260321/PRD.md)
## Install UX hardening — 2026-04-05
- [Mission manifest](install-ux-hardening-20260405/MISSION-MANIFEST.md)
- [Task ledger](install-ux-hardening-20260405/TASKS.md)
## Install UX v2 — 2026-04-05
- [Mission manifest](install-ux-v2-20260405/MISSION-MANIFEST.md)
- [Task ledger](install-ux-v2-20260405/TASKS.md)
- [IUV-M03 design](install-ux-v2-20260405/iuv-m03-design.md)
- [Orchestrator scratchpad](install-ux-v2-20260405/scratchpad.md)
## Storage abstraction retrofit
- [Task ledger](storage-abstraction/TASKS.md)
Historical statuses, commands, package paths, and completion claims are retained for provenance and may not match the current checkout. Use [`docs/TASKS.md`](../../TASKS.md) only for orchestrator-owned current task state.
## Related
- [[archive/README|Documentation archive]]
- [[SITEMAP|Documentation sitemap]]
- [[reports/README|Documentation reports]]
docs/archive/missions/install-ux-hardening-20260405/MISSION-MANIFEST.md
+1 -1
View File
@@ -12,7 +12,7 @@
**Progress:** 3 / 3 milestones
**Status:** complete
**Last Updated:** 2026-04-05 (mission complete)
**Parent Mission:** [cli-unification-20260404](./archive/missions/cli-unification-20260404/MISSION-MANIFEST.md) (complete)
**Parent Mission:** [cli-unification-20260404](../cli-unification-20260404/MISSION-MANIFEST.md) (complete)
## Context
docs/archive/missions/install-ux-v2-20260405/MISSION-MANIFEST.md
+1 -1
View File
@@ -13,7 +13,7 @@
**Status:** complete
**Last Updated:** 2026-04-19 (archived during MVP manifest authoring; IUV-M03 substantively shipped via PR #446 — drill-down menu + provider-first flow + quick start; releases 0.0.27 → 0.0.29)
**Archived to:** `docs/archive/missions/install-ux-v2-20260405/`
**Parent Mission:** [install-ux-hardening-20260405](./archive/missions/install-ux-hardening-20260405/MISSION-MANIFEST.md) (complete — `mosaic-v0.0.25`)
**Parent Mission:** [install-ux-hardening-20260405](../install-ux-hardening-20260405/MISSION-MANIFEST.md) (complete — `mosaic-v0.0.25`)
## Context
docs/archive/planning/README.md
+15
View File
@@ -0,0 +1,15 @@
# Archived planning records
> **Status:** Historical planning index. These records preserve prior intent and review context; they are not current requirements, task state, implementation evidence, or operational authority.
## Monorepo consolidation
- [Planning bundle](monorepo-consolidation/README.md) — historical brief, board review, and Forge/MACP/framework-plugin work-package specifications.
## Legacy plans and deferred stubs
- [Legacy planning index](legacy/README.md) — unreferenced implementation plans, a superseded SSO setup record, and explicitly deferred design stubs.
- [Archived Matrix/MACP proposals](matrix-macp/README.md) — historical draft communications and deployment RFCs for functionality not established by current source/tests.
- [Archived standalone designs](designs/README.md) — historical prerelease-pipeline and storage-abstraction designs.
Use current package source, tests, audience guides, and approved control documents for present behavior and status.
docs/archive/planning/designs/README.md
+8
View File
@@ -0,0 +1,8 @@
# Archived standalone designs
> **Status:** Historical design records. These files moved byte-identically from migration quarantine on 2026-08-10 and are not current implementation or release contracts.
- [npm prerelease `@next` lane](prerelease-next-dist-tag-pipeline.md) — prior release-pipeline design; verify current CI and package scripts before use.
- [Storage and queue abstraction](storage-abstraction-middleware.md) — prior middleware/tier design. Current storage abstractions exist, but this record does not prove its complete target architecture or operational procedures.
Use current package source, manifests, tests, and canonical safety guidance for present behavior. The coupled #791 upgrade design and normative framework constitution remain in migration quarantine pending their owning workstreams.
docs/archive/planning/designs/prerelease-next-dist-tag-pipeline.md
+63
View File
@@ -0,0 +1,63 @@
# npm `@next` prerelease lane
Status: **IMPLEMENTED**
## Current behavior
`tools/install.sh --next` provides the prerelease integration lane for the permanent `next` branch.
The lane is fast-by-default:
1. Install framework files from the `next` source archive.
2. Resolve the Gitea npm registry `next` dist-tag for the globally installed packages:
```bash
npm view @mosaicstack/gateway@next version
npm view @mosaicstack/mosaic@next version
```
3. Require both resolved versions to share the same `next.<pipeline>` suffix, then install the exact resolved versions.
4. If either `@next` package is missing, unreachable, mismatched, or fails to install, fall back to the source-build path at `next`.
`--next` never hard-fails solely because the prerelease npm dist-tag is unavailable.
## Published packages
The `next` publish pipeline publishes non-private `@mosaicstack/*` packages to the Mosaic Gitea npm registry:
```text
https://git.mosaicstack.dev/api/packages/mosaicstack/npm/
```
Observed `next` dist-tags after enabling the pipeline:
```text
@mosaicstack/mosaic@next -> 0.0.49-next.1633
@mosaicstack/gateway@next -> 0.0.7-next.1633
```
The gateway also publishes a Docker image as `gateway:sha-<short>` on `next` merges. The installer fast path uses the npm gateway package when available; the Docker image is for deployed gateway/runtime harness flows.
## Explicit source lanes
Source builds remain available and are still the authority for explicit ref validation:
- `--dev` always builds from source.
- `--ref <ref>` / `MOSAIC_REF=<ref>` wins over `--next` and uses the source path for that exact ref.
## Pipeline shape
1. Trigger on `next` merges.
2. Compute the next prerelease version from the upcoming stable version plus the Woodpecker pipeline number (`<target-stable>-next.<CI_PIPELINE_NUMBER>`).
3. Build and publish non-private packages in CI.
4. Publish to the Mosaic Gitea npm registry with dist-tag `next`.
5. Keep `latest` untouched; only main/release promotion can update `latest`.
6. Publish gateway Docker images from `next` as `gateway:sha-<short>` only.
## Guardrails
- `@next` is mutable prerelease convenience, not a deployment pin.
- Stable installs continue to use `@latest`.
- Contributor validation remains available through `--dev --ref <branch>`.
- Pipeline output traces every prerelease package back to the source commit on `next`.
- The installer falls back to source rather than hard-failing on prerelease registry issues.
docs/archive/planning/designs/storage-abstraction-middleware.md
+559
View File
@@ -0,0 +1,559 @@
# Storage & Queue Abstraction — Middleware Architecture
Design
Status: Design (retrofit required)
date: 2026-04-02
context: Agents coupled directly to infrastructure backends, bypassing intended middleware layer
---
## The Problem
Current packages are **direct adapters**, not **middleware**:
| Package | Current State | Intended Design |
|---------|---------------|-----------------|
| `@mosaicstack/queue` | `ioredis` hardcoded | Interface → BullMQ OR local-files |
| `@mosaicstack/db` | Drizzle + Postgres hardcoded | Interface → Postgres OR SQLite OR JSON/MD |
| `@mosaicstack/memory` | pgvector required | Interface → pgvector OR sqlite-vec OR keyword-search |
## The gateway and TUI import these packages directly, which means they they're coupled to specific infrastructure. Users cannot run Mosaic Stack without Postgres + Valkey.
## The Intended Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Gateway / TUI / CLI │
│ (agnostic of storage backend, talks to middleware) │
└───────────────────────────┬─────────────────────────────────────┘
│
┌───────────────────┼───────────────────┐
│ │ │
▼─────────────────┴─────────────────┴─────────────────┘
| | | |
▼─────────────────┴───────────────────┴─────────────────┘
| | | |
Queue Storage Memory
| | | |
┌─────────┬─────────┬─────────┬─────────────────────────────────┐
| BullMQ | | Local | | Postgres | SQLite | JSON/MD | pgvector | sqlite-vec | keyword |
|(Valkey)| |(files) | | | | | |
└─────────┴─────────┴─────────┴─────────────────────────────────┘
```
The gateway imports the interface, not the backend. At startup it reads config and instantiates the correct adapter.
## The Drift
```typescript
// What should have happened:
gateway/queue.service.ts → @mosaicstack/queue (interface) → queue.adapter.ts
// What actually happened:
gateway/queue.service.ts → @mosaicstack/queue → ioredis (hardcoded)
```
## The Current State Analysis
### `@mosaicstack/queue` (packages/queue/src/queue.ts)
```typescript
import Redis from 'ioredis'; // ← Direct import of backend
export function createQueue(config?: QueueConfig): QueueHandle {
const url = config?.url ?? process.env['VALKEY_URL'] ?? DEFAULT_VALKEY_URL;
const redis = new Redis(url, { maxRetriesPerRequest: 3 });
// ...queue ops directly on redis...
}
```
**Problem:** `ioredis` is imported in the package, not the adapter interface. Consumers cannot swap backends.
### `@mosaicstack/db` (packages/db/src/client.ts)
> **Historical design specimen — status-only, not an operator instruction.** KBN-101 supersedes
> this pre-split `DATABASE_URL` fallback shape; it cannot authorize runtime migration, DDL, or a
> connection-string fallback. See the KBN-101 runner/role contract for the produced interface.
```typescript
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';
import postgres from 'postgres';
export function createDb(url?: string): DbHandle {
const connectionString = url ?? process.env['DATABASE_URL'] ?? DEFAULT_DATABASE_URL;
const sql = postgres(connectionString, { max: 20, idle_timeout: 30, connect_timeout: 5 });
const db = drizzle(sql, { schema });
// ...
}
```
**Problem:** Drizzle + Postgres is hardcoded. No SQLite, JSON, or file-based options.
### `@mosaicstack/memory` (packages/memory/src/memory.ts)
```typescript
import type { Db } from '@mosaicstack/db'; // ← Depends on Drizzle/PG
export function createMemory(db: Db): Memory {
return {
preferences: createPreferencesRepo(db),
insights: createInsightsRepo(db),
};
}
```
**Problem:** Memory package is tightly coupled to `@mosaicstack/db` (which is Postgres-only). No alternative storage backends.
## The Target Interfaces
### Queue Interface
```typescript
// packages/queue/src/types.ts
export interface QueueAdapter {
readonly name: string;
enqueue(queueName: string, payload: TaskPayload): Promise<void>;
dequeue(queueName: string): Promise<TaskPayload | null>;
length(queueName: string): Promise<number>;
publish(channel: string, message: string): Promise<void>;
subscribe(channel: string, handler: (message: string) => void): () => void;
close(): Promise<void>;
}
export interface TaskPayload {
id: string;
type: string;
data: Record<string, unknown>;
createdAt: string;
}
export interface QueueConfig {
type: 'bullmq' | 'local';
url?: string; // For bullmq: Valkey/Redis URL
dataDir?: string; // For local: directory for JSON persistence
}
```
### Storage Interface
```typescript
// packages/storage/src/types.ts
export interface StorageAdapter {
readonly name: string;
// Entity CRUD
create<T>(collection: string, data: O): Promise<T>;
read<T>(collection: string, id: string): Promise<T | null>;
update<T>(collection: string, id: string, data: Partial<O>): Promise<T | null>;
delete(collection: string, id: string): Promise<boolean>;
// Queries
find<T>(collection: string, filter: Record<string, unknown>): Promise<T[]>;
findOne<T>(collection: string, filter: Record<string, unknown): Promise<T | null>;
// Bulk operations
createMany<T>(collection: string, items: O[]): Promise<T[]>;
updateMany<T>(collection: string, ids: string[], data: Partial<O>): Promise<number>;
deleteMany(collection: string, ids: string[]): Promise<number>;
// Raw queries (for complex queries)
query<T>(collection: string, query: string, params?: unknown[]): Promise<T[]>;
// Transaction support
transaction<T>(fn: (tx: StorageTransaction) => Promise<T>): Promise<T>;
close(): Promise<void>;
}
export interface StorageTransaction {
commit(): Promise<void>;
rollback(): Promise<void>;
}
export interface StorageConfig {
type: 'postgres' | 'sqlite' | 'files';
url?: string; // For postgres
path?: string; // For sqlite/files
}
```
### Memory Interface (Vector + Preferences)
```typescript
// packages/memory/src/types.ts
export interface MemoryAdapter {
readonly name: string;
// Preferences (key-value storage)
getPreference(userId: string, key: string): Promise<unknown | null>;
setPreference(userId: string, key: string, value: unknown): Promise<void>;
deletePreference(userId: string, key: string): Promise<boolean>;
listPreferences(
userId: string,
category?: string,
): Promise<Array<{ key: string; value: unknown }>>;
// Insights (with optional vector search)
storeInsight(insight: NewInsight): Promise<Insight>;
getInsight(id: string): Promise<Insight | null>;
searchInsights(query: string, limit?: number, filter?: InsightFilter): Promise<SearchResult[]>;
deleteInsight(id: string): Promise<boolean>;
// Embedding provider (optional, null = no vector search)
readonly embedder?: EmbeddingProvider | null;
close(): Promise<void>;
}
export interface NewInsight {
id: string;
userId: string;
content: string;
embedding?: number[]; // If embedder is available
source: 'agent' | 'user' | 'summarization' | 'system';
category: 'decision' | 'learning' | 'preference' | 'fact' | 'pattern' | 'general';
relevanceScore: number;
metadata?: Record<string, unknown>;
createdAt: Date;
decayedAt?: Date;
}
export interface InsightFilter {
userId?: string;
category?: string;
source?: string;
minRelevance?: number;
fromDate?: Date;
toDate?: Date;
}
export interface SearchResult {
documentId: string;
content: string;
distance: number;
metadata?: Record<string, unknown>;
}
export interface MemoryConfig {
type: 'pgvector' | 'sqlite-vec' | 'keyword';
storage: StorageAdapter;
embedder?: EmbeddingProvider;
}
export interface EmbeddingProvider {
embed(text: string): Promise<number[]>;
embedBatch(texts: string[]): Promise<number[][]>;
readonly dimensions: number;
}
```
## Three Tiers
### Tier 1: Local (Zero Dependencies)
**Target:** Single user, single machine, no external services
| Component | Backend | Storage |
| --------- | --------------------------------------------- | ------------ |
| Queue | In-process + JSON files in `~/.mosaic/queue/` |
| Storage | SQLite (better-sqlite3) `~/.mosaic/data.db` |
| Memory | Keyword search | SQLite table |
| Vector | None | N/A |
**Dependencies:**
- `better-sqlite3` (bundled)
- No Postgres, No Valkey, No pgvector
**Upgrade path:**
1. Run `mosaic gateway configure` → select "local" tier
2. Gateway starts with SQLite database
3. Optional: run `mosaic gateway upgrade --tier team` to migrate to Postgres
### Tier 2: Team (Postgres + Valkey)
**Target:** Multiple users, shared server, CI/CD environments
| Component | Backend | Storage |
| --------- | -------------- | ------------------------------ |
| Queue | BullMQ | Valkey |
| Storage | Postgres | Shared PG instance |
| Memory | pgvector | Postgres with vector extension |
| Vector | LLM embeddings | Configured provider |
**Dependencies:**
- PostgreSQL 17+ with pgvector extension
- Valkey (Redis-compatible)
- LLM provider for embeddings
**Migration from Local → Team:**
1. `mosaic gateway backup` → creates dump of SQLite database
2. `mosaic gateway upgrade --tier team` → restores to Postgres
3. Queue replays from BullMQ (may need manual reconciliation for in-flight jobs)
4. Memory embeddings regenerated if vector search was new
### Tier 3: Enterprise (Clustered)
**Target:** Large teams, multi-region, high availability
| Component | Backend | Storage |
| --------- | --------------------------- | ----------------------------- |
| Queue | BullMQ cluster | Multiple Valkey nodes |
| Storage | Postgres cluster | Primary + replicas |
| Memory | Dedicated vector DB | Qdrant, Pinecone, or pgvector |
| Vector | Dedicated embedding service | Separate microservice |
## MarkdownDB Integration
For file-based storage, we use [MarkdownDB](https://markdowndb.com) to parse MD files into queryable data.
**What it provides:**
- Parses frontmatter (YAML/JSON/TOML)
- Extracts links, tags, metadata
- Builds index in JSON or SQLite
- Queryable via SQL-like interface
**Usage in Mosaic:**
```typescript
// Local tier with MD files for documents
const storage = createStorageAdapter({
type: 'files',
path: path.join(mosaicHome, 'docs'),
markdowndb: {
parseFrontmatter: true,
extractLinks: true,
indexFile: 'index.json',
},
});
```
## Dream Mode — Memory Consolidation
Automated equivalent to Claude Code's "Dream: Memory Consolidation" cycle
**Trigger:** Every 24 hours (if 5+ sessions active)
**Phases:**
1. **Orient** — What happened, what's the current state
- Scan recent session logs
- Identify active tasks, missions, conversations
- Calculate time window (last 24h)
2. **Gather** — Pull in relevant context
- Load conversations, decisions, agent logs
- Extract key interactions and outcomes
- Identify patterns and learnings
3. **Consolidate** — Summarize and compress
- Generate summary of the last 24h
- Extract key decisions and their rationale
- Identify recurring patterns
- Compress verbose logs into concise insights
4. **Prune** — Archive and cleanup
- Archive raw session files to dated folders
- Delete redundant/temporary data
- Update MEMORY.md with consolidated content
- Update insight relevance scores
**Implementation:**
```typescript
// In @mosaicstack/dream (new package)
export async function runDreamCycle(config: DreamConfig): Promise<DreamResult> {
const memory = await loadMemoryAdapter(config.storage);
// Orient
const sessions = await memory.getRecentSessions(24 * 60 * 60 * 1000);
if (sessions.length < 5) return { skipped: true, reason: 'insufficient_sessions' };
// Gather
const context = await gatherContext(memory, sessions);
// Consolidate
const consolidated = await consolidateWithLLM(context, config.llm);
// Prune
await pruneArchivedData(memory, config.retention);
// Store consolidated insights
await memory.storeInsights(consolidated.insights);
return {
sessionsProcessed: sessions.length,
insightsCreated: consolidated.insights.length,
bytesPruned: consolidated.bytesRemoved,
};
}
```
---
## Retrofit Plan
### Phase 1: Interface Extraction (2-3 days)
**Goal:** Define interfaces without changing existing behavior
1. Create `packages/queue/src/types.ts` with `QueueAdapter` interface
2. Create `packages/storage/src/types.ts` with `StorageAdapter` interface
3. Create `packages/memory/src/types.ts` with `MemoryAdapter` interface (refactor existing)
4. Add adapter registry pattern to each package
5. No breaking changes — existing code continues to work
### Phase 2: Refactor Existing to Adapters (3-5 days)
**Goal:** Move existing implementations behind adapters
#### 2.1 Queue Refactor
1. Rename `packages/queue/src/queue.ts` → `packages/queue/src/adapters/bullmq.ts`
2. Create `packages/queue/src/index.ts` to export factory function
3. Factory function reads config, instantiates correct adapter
4. Update gateway imports to use factory
#### 2.2 Storage Refactor
1. Create `packages/storage/` (new package)
2. Move Drizzle logic to `packages/storage/src/adapters/postgres.ts`
3. Create SQLite adapter in `packages/storage/src/adapters/sqlite.ts`
4. Update gateway to use storage factory
5. Deprecate direct `@mosaicstack/db` imports
#### 2.3 Memory Refactor
1. Extract existing logic to `packages/memory/src/adapters/pgvector.ts`
2. Create keyword adapter in `packages/memory/src/adapters/keyword.ts`
3. Update vector-store.ts to be adapter-agnostic
### Phase 3: Local Tier Implementation (2-3 days)
**Goal:** Zero-dependency baseline
1. Implement `packages/queue/src/adapters/local.ts` (in-process + JSON persistence)
2. Implement `packages/storage/src/adapters/files.ts` (JSON + MD via MarkdownDB)
3. Implement `packages/memory/src/adapters/keyword.ts` (TF-IDF search)
4. Add `packages/dream/` for consolidation cycle
5. Wire up local tier in gateway startup
### Phase 4: Configuration System (1-2 days)
**Goal:** Runtime backend selection
1. Create `packages/config/src/storage.ts` for storage configuration
2. Add `mosaic.config.ts` schema with storage tier settings
3. Update gateway to read config on startup
4. Add `mosaic gateway configure` CLI command
5. Add tier migration commands (`mosaic gateway upgrade`)
### Phase 5: Testing & Documentation (2-3 days)
1. Unit tests for each adapter
2. Integration tests for factory pattern
3. Migration tests (local → team)
4. Update README and architecture docs
5. Add configuration guide
---
## File Changes Summary
### New Files
```
packages/
├── config/
│ └── src/
│ ├── storage.ts # Storage config schema
│ └── index.ts
├── dream/ # NEW: Dream mode consolidation
│ ├── src/
│ │ ├── index.ts
│ │ ├── orient.ts
│ │ ├── gather.ts
│ │ ├── consolidate.ts
│ │ └── prune.ts
│ └── package.json
├── queue/
│ └── src/
│ ├── types.ts # NEW: QueueAdapter interface
│ ├── index.ts # NEW: Factory function
│ └── adapters/
│ ├── bullmq.ts # MOVED from queue.ts
│ └── local.ts # NEW: In-process adapter
├── storage/ # NEW: Storage abstraction
│ ├── src/
│ │ ├── types.ts # StorageAdapter interface
│ │ ├── index.ts # Factory function
│ │ └── adapters/
│ │ ├── postgres.ts # MOVED from @mosaicstack/db
│ │ ├── sqlite.ts # NEW: SQLite adapter
│ │ └── files.ts # NEW: JSON/MD adapter
│ └── package.json
└── memory/
└── src/
├── types.ts # UPDATED: MemoryAdapter interface
├── index.ts # UPDATED: Factory function
└── adapters/
├── pgvector.ts # EXTRACTED from existing code
├── sqlite-vec.ts # NEW: SQLite with vectors
└── keyword.ts # NEW: TF-IDF search
```
### Modified Files
```
packages/
├── db/ # DEPRECATED: Logic moved to storage adapters
├── queue/
│ └── src/
│ └── queue.ts # → adapters/bullmq.ts
├── memory/
│ ├── src/
│ │ ├── memory.ts # → use factory
│ │ ├── insights.ts # → use factory
│ │ └── preferences.ts # → use factory
│ └── package.json # Remove pgvector from dependencies
└── gateway/
└── src/
├── database/
│ └── database.module.ts # Update to use storage factory
├── memory/
│ └── memory.module.ts # Update to use memory factory
└── queue/
└── queue.module.ts # Update to use queue factory
```
---
## Breaking Changes
1. **`@mosaicstack/db`** → **`@mosaicstack/storage`** (with migration guide)
2. Direct `ioredis` imports → Use `@mosaicstack/queue` factory
3. Direct `pgvector` queries → Use `@mosaicstack/memory` factory
4. Gateway startup now requires storage config (defaults to local)
## Non-Breaking Migration Path
1. Existing deployments with Postgres/Valkey continue to work (default config)
2. New deployments can choose local tier
3. Migration commands available when ready to upgrade
---
## Success Criteria
- [ ] Local tier runs with zero external dependencies
- [ ] All three tiers (local, team, enterprise) work correctly
- [ ] Factory pattern correctly selects backend at runtime
- [ ] Migration from local → team preserves all data
- [ ] Dream mode consolidates 24h of sessions
- [ ] Documentation covers all three tiers and migration paths
- [ ] All existing tests pass
- [ ] New adapters have >80% coverage
docs/archive/planning/legacy/2026-03-13-gateway-security-hardening.md
+98
View File
@@ -0,0 +1,98 @@
# Gateway Security Hardening Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Finish the requested gateway security hardening fixes in the existing `fix/gateway-security` worktree and produce a PR-ready branch.
**Architecture:** Tighten NestJS gateway boundaries in-place by enforcing auth guards, session validation, ownership checks, DTO validation, and Fastify security defaults. Preserve the current module structure and existing ESM import conventions.
**Tech Stack:** NestJS 11, Fastify, Socket.IO, Better Auth, class-validator, Vitest, pnpm, TypeScript ESM
---
### Task 1: Reconcile Security Tests
**Files:**
- Modify: `apps/gateway/src/chat/__tests__/chat-security.test.ts`
- Modify: `apps/gateway/src/__tests__/resource-ownership.test.ts`
**Step 1: Write the failing test**
- Encode the requested DTO constraints and socket-auth contract exactly.
**Step 2: Run test to verify it fails**
Run: `pnpm --filter @mosaicstack/gateway test -- src/chat/__tests__/chat-security.test.ts src/__tests__/resource-ownership.test.ts`
Expected: FAIL on current DTO/helper mismatch.
**Step 3: Write minimal implementation**
- Update DTO/helper/controller code only where tests prove a gap.
**Step 4: Run test to verify it passes**
Run the same command and require green.
### Task 2: Align Gateway Runtime Hardening
**Files:**
- Modify: `apps/gateway/src/conversations/conversations.dto.ts`
- Modify: `apps/gateway/src/chat/chat.dto.ts`
- Modify: `apps/gateway/src/chat/chat.gateway-auth.ts`
- Modify: `apps/gateway/src/chat/chat.gateway.ts`
- Modify: `apps/gateway/src/main.ts`
- Modify: `apps/gateway/src/app.module.ts`
**Step 1: Verify remaining requested deltas**
- Confirm code matches requested guard, rate limit, helmet, body limit, env validation, and CORS settings.
**Step 2: Apply minimal patch**
- Keep changes scoped to requested behavior only.
**Step 3: Run targeted tests**
Run: `pnpm --filter @mosaicstack/gateway test -- src/chat/__tests__/chat-security.test.ts src/__tests__/resource-ownership.test.ts`
Expected: PASS.
### Task 3: Verification, Review, and Delivery
**Files:**
- Create: `docs/reports/code-review/gateway-security-20260313.md`
- Create: `docs/reports/qa/gateway-security-20260313.md`
- Modify: `docs/scratchpads/gateway-security-20260313.md`
**Step 1: Run baseline gates**
Run:
```bash
pnpm typecheck
pnpm lint
```
**Step 2: Perform manual code review**
- Record correctness/security/testing/doc findings.
**Step 3: Commit and publish**
Run:
```bash
git add -A
git commit -m "fix(gateway): security hardening — auth guards, ownership checks, validation, rate limiting"
git push origin fix/gateway-security
```
**Step 4: Open PR and notify**
- Open PR titled `fix(gateway): security hardening — auth guards, ownership checks, validation, rate limiting`
- Run `openclaw system event --text "PR ready: mosaic-mono-v1 fix/gateway-security — 7 security fixes" --mode now`
- Remove worktree after PR is created.
docs/archive/planning/legacy/2026-03-15-agent-platform-architecture.md
+1571
View File
File diff suppressed because it is too large Load Diff
docs/archive/planning/legacy/2026-03-15-wave2-tui-layout-navigation.md
+998
View File
@@ -0,0 +1,998 @@
# Wave 2 — TUI Layout & Navigation Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Add conversation sidebar, keybindings, scrollable message history, and search to the Mosaic TUI.
**Architecture:** The TUI gains a sidebar panel for conversation management (list/create/switch) fetched via REST from the gateway. A `useConversations` hook manages REST calls. A `useScrollableViewport` hook wraps the message list with virtual viewport logic. An app-level focus/mode state machine (`useAppMode`) controls which panel receives input. All new socket events for conversation listing use the existing REST API (`GET /api/conversations`).
**Tech Stack:** Ink 5, React 18, socket.io-client, fetch (for REST), @mosaicstack/types
---
## Dependency Graph
```
TUI-010 (scrollable history) ← TUI-011 (search)
TUI-008 (sidebar) ← TUI-009 (keybindings)
```
TUI-008 and TUI-010 are independent — can be built in parallel.
TUI-009 depends on TUI-008. TUI-011 depends on TUI-010.
---
## Task 1: TUI-010 — Scrollable Message History
### 1A: Create `use-viewport` hook
**Files:**
- Create: `packages/cli/src/tui/hooks/use-viewport.ts`
**Step 1: Write the hook**
This hook tracks a scroll offset and viewport height for the message list.
Ink's `useStdout` gives us terminal rows. We calculate visible slice.
```ts
import { useState, useCallback, useMemo } from 'react';
import { useStdout } from 'ink';
export interface UseViewportOptions {
/** Total number of renderable lines (message count as proxy) */
totalItems: number;
/** Lines reserved for chrome (top bar, input bar, bottom bar) */
reservedLines?: number;
}
export interface UseViewportReturn {
/** Index of first visible item (0-based) */
scrollOffset: number;
/** Number of items that fit in viewport */
viewportSize: number;
/** Whether user has scrolled up from bottom */
isScrolledUp: boolean;
/** Scroll to bottom (auto-follow mode) */
scrollToBottom: () => void;
/** Scroll by delta (negative = up, positive = down) */
scrollBy: (delta: number) => void;
/** Scroll to a specific offset */
scrollTo: (offset: number) => void;
/** Whether we can scroll up/down */
canScrollUp: boolean;
canScrollDown: boolean;
}
export function useViewport(opts: UseViewportOptions): UseViewportReturn {
const { totalItems, reservedLines = 10 } = opts;
const { stdout } = useStdout();
const terminalRows = stdout?.rows ?? 24;
// Viewport = terminal height minus chrome
const viewportSize = Math.max(1, terminalRows - reservedLines);
const maxOffset = Math.max(0, totalItems - viewportSize);
const [scrollOffset, setScrollOffset] = useState(0);
// Track if user explicitly scrolled up
const [autoFollow, setAutoFollow] = useState(true);
// Effective offset: if auto-following, always show latest
const effectiveOffset = autoFollow ? maxOffset : Math.min(scrollOffset, maxOffset);
const scrollBy = useCallback(
(delta: number) => {
setAutoFollow(false);
setScrollOffset((prev) => {
const next = Math.max(0, Math.min(prev + delta, maxOffset));
// If scrolled to bottom, re-enable auto-follow
if (next >= maxOffset) {
setAutoFollow(true);
}
return next;
});
},
[maxOffset],
);
const scrollToBottom = useCallback(() => {
setAutoFollow(true);
setScrollOffset(maxOffset);
}, [maxOffset]);
const scrollTo = useCallback(
(offset: number) => {
const clamped = Math.max(0, Math.min(offset, maxOffset));
setAutoFollow(clamped >= maxOffset);
setScrollOffset(clamped);
},
[maxOffset],
);
return {
scrollOffset: effectiveOffset,
viewportSize,
isScrolledUp: !autoFollow,
scrollToBottom,
scrollBy,
scrollTo,
canScrollUp: effectiveOffset > 0,
canScrollDown: effectiveOffset < maxOffset,
};
}
```
**Step 2: Typecheck**
Run: `pnpm --filter @mosaicstack/cli typecheck`
Expected: PASS
**Step 3: Commit**
```bash
git add packages/cli/src/tui/hooks/use-viewport.ts
git commit -m "feat(cli): add use-viewport hook for scrollable message history"
```
### 1B: Integrate viewport into MessageList and wire PgUp/PgDn
**Files:**
- Modify: `packages/cli/src/tui/components/message-list.tsx`
- Modify: `packages/cli/src/tui/app.tsx`
**Step 1: Update MessageList to accept viewport props and slice messages**
In `message-list.tsx`, add viewport props and render only the visible slice. Add a scroll indicator when scrolled up.
```tsx
// Add to MessageListProps:
export interface MessageListProps {
messages: Message[];
isStreaming: boolean;
currentStreamText: string;
currentThinkingText: string;
activeToolCalls: ToolCall[];
// New viewport props
scrollOffset: number;
viewportSize: number;
isScrolledUp: boolean;
}
```
In the component body, slice messages:
```tsx
const visibleMessages = messages.slice(scrollOffset, scrollOffset + viewportSize);
```
Replace `messages.map(...)` with `visibleMessages.map(...)`. Add a scroll-up indicator at the top:
```tsx
{
isScrolledUp && (
<Box justifyContent="center">
<Text dimColor>↑ {scrollOffset} more messages ↑</Text>
</Box>
);
}
```
**Step 2: Wire viewport hook + keybindings in app.tsx**
In `app.tsx`:
1. Import and call `useViewport({ totalItems: socket.messages.length })`
2. Pass viewport props to `<MessageList>`
3. Add PgUp/PgDn/Home/End keybindings in the existing `useInput`:
- `key.pageUp` → `viewport.scrollBy(-viewport.viewportSize)`
- `key.pageDown` → `viewport.scrollBy(viewport.viewportSize)`
- Shift+Up → `viewport.scrollBy(-1)` (line scroll)
- Shift+Down → `viewport.scrollBy(1)` (line scroll)
Note: Ink's `useInput` key object supports `pageUp`, `pageDown`. For Home/End, check `key.meta && ch === '<'` / `key.meta && ch === '>'` as Ink doesn't have built-in home/end.
**Step 3: Typecheck and lint**
Run: `pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint`
Expected: PASS
**Step 4: Commit**
```bash
git add packages/cli/src/tui/components/message-list.tsx packages/cli/src/tui/app.tsx
git commit -m "feat(cli): scrollable message history with PgUp/PgDn viewport"
```
---
## Task 2: TUI-008 — Conversation Sidebar
### 2A: Create `use-conversations` hook (REST client)
**Files:**
- Create: `packages/cli/src/tui/hooks/use-conversations.ts`
**Step 1: Write the hook**
This hook fetches conversations from the gateway REST API and provides create/switch actions.
```ts
import { useState, useEffect, useCallback, useRef } from 'react';
export interface ConversationSummary {
id: string;
title: string | null;
archived: boolean;
createdAt: string;
updatedAt: string;
}
export interface UseConversationsOptions {
gatewayUrl: string;
sessionCookie?: string;
/** Currently active conversation ID from socket */
activeConversationId: string | undefined;
}
export interface UseConversationsReturn {
conversations: ConversationSummary[];
loading: boolean;
error: string | null;
refresh: () => Promise<void>;
createConversation: (title?: string) => Promise<ConversationSummary | null>;
deleteConversation: (id: string) => Promise<boolean>;
renameConversation: (id: string, title: string) => Promise<boolean>;
}
export function useConversations(opts: UseConversationsOptions): UseConversationsReturn {
const { gatewayUrl, sessionCookie } = opts;
const [conversations, setConversations] = useState<ConversationSummary[]>([]);
const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const mountedRef = useRef(true);
const headers: Record<string, string> = {
'Content-Type': 'application/json',
...(sessionCookie ? { Cookie: sessionCookie } : {}),
};
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
const res = await fetch(`${gatewayUrl}/api/conversations`, { headers });
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const data = (await res.json()) as ConversationSummary[];
if (mountedRef.current) {
setConversations(data);
}
} catch (err) {
if (mountedRef.current) {
setError(err instanceof Error ? err.message : String(err));
}
} finally {
if (mountedRef.current) setLoading(false);
}
}, [gatewayUrl, sessionCookie]);
const createConversation = useCallback(
async (title?: string): Promise<ConversationSummary | null> => {
try {
const res = await fetch(`${gatewayUrl}/api/conversations`, {
method: 'POST',
headers,
body: JSON.stringify({ title: title ?? 'New Conversation' }),
});
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const conv = (await res.json()) as ConversationSummary;
await refresh();
return conv;
} catch {
return null;
}
},
[gatewayUrl, sessionCookie, refresh],
);
const deleteConversation = useCallback(
async (id: string): Promise<boolean> => {
try {
const res = await fetch(`${gatewayUrl}/api/conversations/${id}`, {
method: 'DELETE',
headers,
});
if (!res.ok) return false;
await refresh();
return true;
} catch {
return false;
}
},
[gatewayUrl, sessionCookie, refresh],
);
const renameConversation = useCallback(
async (id: string, title: string): Promise<boolean> => {
try {
const res = await fetch(`${gatewayUrl}/api/conversations/${id}`, {
method: 'PATCH',
headers,
body: JSON.stringify({ title }),
});
if (!res.ok) return false;
await refresh();
return true;
} catch {
return false;
}
},
[gatewayUrl, sessionCookie, refresh],
);
useEffect(() => {
mountedRef.current = true;
void refresh();
return () => {
mountedRef.current = false;
};
}, []);
return {
conversations,
loading,
error,
refresh,
createConversation,
deleteConversation,
renameConversation,
};
}
```
**Step 2: Typecheck**
Run: `pnpm --filter @mosaicstack/cli typecheck`
Expected: PASS
**Step 3: Commit**
```bash
git add packages/cli/src/tui/hooks/use-conversations.ts
git commit -m "feat(cli): add use-conversations hook for REST conversation management"
```
### 2B: Create `use-app-mode` hook (focus/mode state machine)
**Files:**
- Create: `packages/cli/src/tui/hooks/use-app-mode.ts`
**Step 1: Write the hook**
This manages which panel has focus and the current UI mode.
```ts
import { useState, useCallback } from 'react';
export type AppMode = 'chat' | 'sidebar' | 'search';
export interface UseAppModeReturn {
mode: AppMode;
setMode: (mode: AppMode) => void;
toggleSidebar: () => void;
/** Whether sidebar panel should be visible */
sidebarOpen: boolean;
}
export function useAppMode(): UseAppModeReturn {
const [mode, setModeState] = useState<AppMode>('chat');
const [sidebarOpen, setSidebarOpen] = useState(false);
const setMode = useCallback((m: AppMode) => {
setModeState(m);
if (m === 'sidebar') setSidebarOpen(true);
}, []);
const toggleSidebar = useCallback(() => {
setSidebarOpen((prev) => {
const next = !prev;
if (!next) {
// Closing sidebar → return to chat mode
setModeState('chat');
} else {
setModeState('sidebar');
}
return next;
});
}, []);
return { mode, setMode, toggleSidebar, sidebarOpen };
}
```
**Step 2: Typecheck**
Run: `pnpm --filter @mosaicstack/cli typecheck`
Expected: PASS
**Step 3: Commit**
```bash
git add packages/cli/src/tui/hooks/use-app-mode.ts
git commit -m "feat(cli): add use-app-mode hook for panel focus state machine"
```
### 2C: Create `Sidebar` component
**Files:**
- Create: `packages/cli/src/tui/components/sidebar.tsx`
**Step 1: Write the component**
The sidebar shows a scrollable list of conversations with the active one highlighted. It handles keyboard navigation when focused.
```tsx
import React from 'react';
import { Box, Text, useInput } from 'ink';
import type { ConversationSummary } from '../hooks/use-conversations.js';
export interface SidebarProps {
conversations: ConversationSummary[];
activeConversationId: string | undefined;
selectedIndex: number;
onSelectIndex: (index: number) => void;
onSwitchConversation: (id: string) => void;
onDeleteConversation: (id: string) => void;
loading: boolean;
focused: boolean;
width: number;
}
function truncate(str: string, maxLen: number): string {
if (str.length <= maxLen) return str;
return str.slice(0, maxLen - 1) + '…';
}
function formatDate(iso: string): string {
const d = new Date(iso);
const now = new Date();
const diffMs = now.getTime() - d.getTime();
const diffDays = Math.floor(diffMs / (1000 * 60 * 60 * 24));
if (diffDays === 0) {
return d.toLocaleTimeString('en-US', { hour: '2-digit', minute: '2-digit', hour12: false });
}
if (diffDays < 7) return `${diffDays}d ago`;
return d.toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
}
export function Sidebar({
conversations,
activeConversationId,
selectedIndex,
onSelectIndex,
onSwitchConversation,
onDeleteConversation,
loading,
focused,
width,
}: SidebarProps) {
useInput(
(ch, key) => {
if (!focused) return;
if (key.upArrow) {
onSelectIndex(Math.max(0, selectedIndex - 1));
} else if (key.downArrow) {
onSelectIndex(Math.min(conversations.length - 1, selectedIndex + 1));
} else if (key.return) {
const conv = conversations[selectedIndex];
if (conv) onSwitchConversation(conv.id);
} else if (ch === 'd' || ch === 'D') {
const conv = conversations[selectedIndex];
if (conv && conv.id !== activeConversationId) {
onDeleteConversation(conv.id);
}
}
},
{ isActive: focused },
);
const titleWidth = width - 4; // padding + borders
return (
<Box
flexDirection="column"
width={width}
borderStyle="single"
borderColor={focused ? 'cyan' : 'gray'}
>
<Box paddingX={1}>
<Text bold color={focused ? 'cyan' : undefined}>
Conversations
</Text>
{loading && <Text dimColor> …</Text>}
</Box>
{conversations.length === 0 && (
<Box paddingX={1}>
<Text dimColor>No conversations</Text>
</Box>
)}
{conversations.map((conv, i) => {
const isActive = conv.id === activeConversationId;
const isSelected = i === selectedIndex && focused;
const title = conv.title ?? `Untitled (${conv.id.slice(0, 6)})`;
const displayTitle = truncate(title, titleWidth);
return (
<Box key={conv.id} paddingX={1}>
<Text
bold={isActive}
color={isSelected ? 'cyan' : isActive ? 'green' : undefined}
inverse={isSelected}
>
{isActive ? '● ' : ' '}
{displayTitle}
</Text>
</Box>
);
})}
{focused && (
<Box paddingX={1} marginTop={1}>
<Text dimColor>↑↓ navigate · ↵ switch · d delete</Text>
</Box>
)}
</Box>
);
}
```
**Step 2: Typecheck**
Run: `pnpm --filter @mosaicstack/cli typecheck`
Expected: PASS
**Step 3: Commit**
```bash
git add packages/cli/src/tui/components/sidebar.tsx
git commit -m "feat(cli): add conversation sidebar component"
```
### 2D: Wire sidebar + conversation switching into app.tsx
**Files:**
- Modify: `packages/cli/src/tui/app.tsx`
- Modify: `packages/cli/src/tui/hooks/use-socket.ts`
**Step 1: Add `switchConversation` to useSocket**
In `use-socket.ts`, add a method to switch conversations. When switching, clear local messages and set the new conversation ID. The socket will pick up the new conversation on next `message` emit.
Add to `UseSocketReturn`:
```ts
switchConversation: (id: string) => void;
clearMessages: () => void;
```
Implementation:
```ts
const switchConversation = useCallback((id: string) => {
setConversationId(id);
setMessages([]);
setIsStreaming(false);
setCurrentStreamText('');
setCurrentThinkingText('');
setActiveToolCalls([]);
}, []);
const clearMessages = useCallback(() => {
setMessages([]);
}, []);
```
**Step 2: Update app.tsx layout to include sidebar**
1. Import `useAppMode`, `useConversations`, `Sidebar`
2. Add `useAppMode()` call
3. Add `useConversations({ gatewayUrl, sessionCookie, activeConversationId: socket.conversationId })`
4. Track `sidebarSelectedIndex` state
5. Wrap the main content area in a horizontal `<Box>`:
```tsx
<Box flexDirection="row" flexGrow={1}>
{appMode.sidebarOpen && (
<Sidebar
conversations={convos.conversations}
activeConversationId={socket.conversationId}
selectedIndex={sidebarSelectedIndex}
onSelectIndex={setSidebarSelectedIndex}
onSwitchConversation={(id) => {
socket.switchConversation(id);
appMode.setMode('chat');
}}
onDeleteConversation={(id) => void convos.deleteConversation(id)}
loading={convos.loading}
focused={appMode.mode === 'sidebar'}
width={30}
/>
)}
<Box flexDirection="column" flexGrow={1}>
<MessageList ... />
</Box>
</Box>
```
6. InputBar should be disabled (or readonly placeholder) when mode is not 'chat'
**Step 3: Typecheck and lint**
Run: `pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint`
Expected: PASS
**Step 4: Commit**
```bash
git add packages/cli/src/tui/app.tsx packages/cli/src/tui/hooks/use-socket.ts
git commit -m "feat(cli): wire conversation sidebar with create/switch/delete"
```
---
## Task 3: TUI-009 — Keybinding System
**Files:**
- Modify: `packages/cli/src/tui/app.tsx`
**Step 1: Add global keybindings in the existing useInput**
Add these bindings to the `useInput` in `app.tsx`:
| Binding | Action |
| ----------- | ----------------------------------------- |
| `Ctrl+L` | Toggle sidebar visibility |
| `Ctrl+N` | Create new conversation + switch to it |
| `Ctrl+K` | Toggle search mode (TUI-011) |
| `Escape` | Return to chat mode from any panel |
| `Ctrl+T` | Cycle thinking level (already exists) |
| `PgUp/PgDn` | Scroll viewport (already wired in Task 1) |
```ts
useInput((ch, key) => {
if (key.ctrl && ch === 'c') {
exit();
return;
}
// Global keybindings (work in any mode)
if (key.ctrl && ch === 'l') {
appMode.toggleSidebar();
if (!appMode.sidebarOpen) void convos.refresh();
return;
}
if (key.ctrl && ch === 'n') {
void convos.createConversation().then((conv) => {
if (conv) {
socket.switchConversation(conv.id);
appMode.setMode('chat');
}
});
return;
}
if (key.ctrl && ch === 'k') {
appMode.setMode(appMode.mode === 'search' ? 'chat' : 'search');
return;
}
if (key.escape) {
if (appMode.mode !== 'chat') {
appMode.setMode('chat');
return;
}
// In chat mode, Escape could scroll to bottom
viewport.scrollToBottom();
return;
}
// Ctrl+T: cycle thinking (existing)
if (key.ctrl && ch === 't') {
const levels = socket.availableThinkingLevels;
if (levels.length > 0) {
const currentIdx = levels.indexOf(socket.thinkingLevel);
const nextIdx = (currentIdx + 1) % levels.length;
const next = levels[nextIdx];
if (next) socket.setThinkingLevel(next);
}
return;
}
// Viewport scrolling (only in chat mode)
if (appMode.mode === 'chat') {
if (key.pageUp) {
viewport.scrollBy(-viewport.viewportSize);
} else if (key.pageDown) {
viewport.scrollBy(viewport.viewportSize);
}
}
});
```
**Step 2: Add keybinding hints to bottom bar**
In `bottom-bar.tsx`, add a hints line above the status lines (or integrate into line 1):
```tsx
<Text dimColor>^L sidebar · ^N new · ^K search · ^T thinking · PgUp/Dn scroll</Text>
```
**Step 3: Typecheck and lint**
Run: `pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint`
Expected: PASS
**Step 4: Commit**
```bash
git add packages/cli/src/tui/app.tsx packages/cli/src/tui/components/bottom-bar.tsx
git commit -m "feat(cli): keybinding system — Ctrl+L sidebar, Ctrl+N new, Ctrl+K search, Escape"
```
---
## Task 4: TUI-011 — Message Search
**Files:**
- Create: `packages/cli/src/tui/components/search-bar.tsx`
- Create: `packages/cli/src/tui/hooks/use-search.ts`
- Modify: `packages/cli/src/tui/app.tsx`
- Modify: `packages/cli/src/tui/components/message-list.tsx`
### 4A: Create `use-search` hook
**Step 1: Write the hook**
```ts
import { useState, useCallback, useMemo } from 'react';
import type { Message } from './use-socket.js';
export interface SearchMatch {
messageIndex: number;
/** Character offset within message content */
charOffset: number;
}
export interface UseSearchReturn {
query: string;
setQuery: (q: string) => void;
matches: SearchMatch[];
currentMatchIndex: number;
nextMatch: () => void;
prevMatch: () => void;
clear: () => void;
/** Total match count */
totalMatches: number;
}
export function useSearch(messages: Message[]): UseSearchReturn {
const [query, setQuery] = useState('');
const [currentMatchIndex, setCurrentMatchIndex] = useState(0);
const matches = useMemo(() => {
if (!query || query.length < 2) return [];
const q = query.toLowerCase();
const result: SearchMatch[] = [];
for (let i = 0; i < messages.length; i++) {
const content = messages[i]!.content.toLowerCase();
let pos = 0;
while ((pos = content.indexOf(q, pos)) !== -1) {
result.push({ messageIndex: i, charOffset: pos });
pos += q.length;
}
}
return result;
}, [query, messages]);
const nextMatch = useCallback(() => {
if (matches.length === 0) return;
setCurrentMatchIndex((prev) => (prev + 1) % matches.length);
}, [matches.length]);
const prevMatch = useCallback(() => {
if (matches.length === 0) return;
setCurrentMatchIndex((prev) => (prev - 1 + matches.length) % matches.length);
}, [matches.length]);
const clear = useCallback(() => {
setQuery('');
setCurrentMatchIndex(0);
}, []);
return {
query,
setQuery,
matches,
currentMatchIndex: matches.length > 0 ? currentMatchIndex % matches.length : 0,
nextMatch,
prevMatch,
clear,
totalMatches: matches.length,
};
}
```
**Step 2: Commit**
```bash
git add packages/cli/src/tui/hooks/use-search.ts
git commit -m "feat(cli): add use-search hook for message search"
```
### 4B: Create SearchBar component
**Step 1: Write the component**
```tsx
import React from 'react';
import { Box, Text } from 'ink';
import TextInput from 'ink-text-input';
export interface SearchBarProps {
query: string;
onQueryChange: (q: string) => void;
totalMatches: number;
currentMatch: number;
onNext: () => void;
onPrev: () => void;
onClose: () => void;
focused: boolean;
}
export function SearchBar({
query,
onQueryChange,
totalMatches,
currentMatch,
onClose,
focused,
}: SearchBarProps) {
return (
<Box paddingX={1} borderStyle="single" borderColor={focused ? 'yellow' : 'gray'}>
<Text color="yellow">🔍 </Text>
<TextInput
value={query}
onChange={onQueryChange}
placeholder="search messages…"
focus={focused}
/>
<Box marginLeft={1}>
{query.length >= 2 ? (
<Text dimColor>
{totalMatches > 0 ? `${currentMatch + 1}/${totalMatches}` : 'no matches'}
{' · ↑↓ navigate · Esc close'}
</Text>
) : (
<Text dimColor>type to search…</Text>
)}
</Box>
</Box>
);
}
```
**Step 2: Commit**
```bash
git add packages/cli/src/tui/components/search-bar.tsx
git commit -m "feat(cli): add search bar component"
```
### 4C: Wire search into app.tsx and message-list
**Step 1: Integrate**
In `app.tsx`:
1. Import `useSearch` and `SearchBar`
2. Call `useSearch(socket.messages)`
3. When mode is 'search', render `<SearchBar>` above `<InputBar>`
4. In search mode, Up/Down arrows call `search.nextMatch()`/`search.prevMatch()` and scroll the viewport to the matched message
5. Pass `searchHighlights` to `MessageList` — the set of message indices that match
In `message-list.tsx`:
1. Add optional `highlightedMessageIndices?: Set<number>` and `currentHighlightIndex?: number` props
2. Highlighted messages get a yellow left border or background tint
3. The current match gets a brighter highlight
**Step 2: Typecheck and lint**
Run: `pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint`
Expected: PASS
**Step 3: Commit**
```bash
git add packages/cli/src/tui/app.tsx packages/cli/src/tui/components/message-list.tsx packages/cli/src/tui/components/search-bar.tsx
git commit -m "feat(cli): wire message search with highlight and viewport scroll"
```
---
## Task 5: Final Integration & Quality Gates
**Files:**
- Modify: `docs/TASKS-TUI_Improvements.md` (update status)
**Step 1: Full typecheck across all affected packages**
```bash
pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint
pnpm --filter @mosaicstack/types typecheck
```
Expected: All PASS
**Step 2: Manual smoke test (held)**
This historical TUI smoke test is unavailable until KBN-101-02 supplies a fail-closed Gateway local
startup route. Do not start current Compose PostgreSQL or infer a local Gateway from PGlite support.
A future reviewed test must use the correct Mosaic CLI package and an independently verified Gateway.
Verify:
- [ ] Messages scroll with PgUp/PgDn
- [ ] Ctrl+L opens/closes sidebar
- [ ] Sidebar shows conversations from REST API
- [ ] Arrow keys navigate sidebar when focused
- [ ] Enter switches conversation, clears messages
- [ ] Ctrl+N creates new conversation
- [ ] Ctrl+K opens search bar
- [ ] Typing in search highlights matches
- [ ] Up/Down in search mode cycles through matches
- [ ] Escape returns to chat from any mode
- [ ] Ctrl+T still cycles thinking levels
- [ ] Auto-scroll follows new messages at bottom
**Step 3: Update task tracker**
Mark TUI-008, TUI-009, TUI-010, TUI-011 as ✅ done in `docs/TASKS-TUI_Improvements.md`
**Step 4: Commit and push**
```bash
git add -A
git commit -m "docs: mark Wave 2 tasks complete"
git push
```
---
## File Summary
| Action | Path |
| ------ | -------------------------------------------------- |
| Create | `packages/cli/src/tui/hooks/use-viewport.ts` |
| Create | `packages/cli/src/tui/hooks/use-conversations.ts` |
| Create | `packages/cli/src/tui/hooks/use-app-mode.ts` |
| Create | `packages/cli/src/tui/hooks/use-search.ts` |
| Create | `packages/cli/src/tui/components/sidebar.tsx` |
| Create | `packages/cli/src/tui/components/search-bar.tsx` |
| Modify | `packages/cli/src/tui/app.tsx` |
| Modify | `packages/cli/src/tui/hooks/use-socket.ts` |
| Modify | `packages/cli/src/tui/components/message-list.tsx` |
| Modify | `packages/cli/src/tui/components/bottom-bar.tsx` |
| Modify | `docs/TASKS-TUI_Improvements.md` |
docs/archive/planning/legacy/2026-05-06-hermes-mosaic-alignment.md
+238
View File
@@ -0,0 +1,238 @@
# Hermes-Mosaic Alignment Plan
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
**Goal:** Package Mosaic's mechanical coordination primitives as a native Hermes toolset so any Hermes profile gets mission management, task decomposition, handoff, and session continuity without depending on the Mosaic gateway or OpenClaw runtime.
**Architecture:** Extract the coordination logic from Mosaic's `packages/coord` (TypeScript, file-first) into a Hermes Python toolset that wraps the same file conventions. The Mosaic Stack repo remains the canonical upstream for the file formats (TASKS.md schema, mission.json schema, handoff packet schema). Hermes implements native Python tools that read/write those same files, plus tool-calls for churn detection and handoff generation that have no Mosaic equivalent today.
**Tech Stack:** Python (Hermes toolset), SQLite (Hermes Kanban), JSON + Markdown (Mosaic file conventions)
---
## Alignment Map
### What Mosaic has that Hermes needs
| Mosaic Component | What it does | Natural Hermes home | Why |
| -------------------------------- | --------------------------------------------------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `packages/coord` (mission.ts) | Mission CRUD, session tracking, milestone state | **Hermes toolset: `mission`** | Mission state is session-scoped, not gateway-scoped. Hermes sessions already have identity, process tracking, and context windows. |
| `packages/coord` (tasks-file.ts) | Parse/write TASKS.md tables | **Hermes toolset: `mission`** (same) | Hermes already reads/writes files. The TASKS.md parser is ~300 lines of pure string manipulation — trivial Python port. |
| `packages/coord` (runner.ts) | Spawn claude/codex workers with continuation prompts | **Already covered by `delegate_task`** | Hermes delegate_task already does isolated subagent spawning with restricted toolsets. The runner's "find next task and build continuation prompt" logic moves into a tool-call. |
| `packages/coord` (status.ts) | Mission health, task progress, next task | **Hermes toolset: `mission`** (same) | Status readout fits naturally as a tool-call. No gateway needed. |
| `packages/prdy` | PRD generation wizard | **Hermes skill: `prdy`** | PRD generation is a prompt + template problem, not infrastructure. A Hermes skill with templates is the right fit. |
| `plugins/mosaic-framework` | before_agent_start + subagent_spawning hooks | **Hermes system prompt injection** | Hermes already injects system context via skills and config. The framework preamble and worktree rules become standard Hermes skills loaded by the orchestrator profile. |
| `plugins/macp` | OpenClaw ACP bridge (spawn codex/claude) | **Already covered by `delegate_task` + ACP** | Hermes already has ACP support and delegate_task. The MACP bridge is redundant when running natively in Hermes. |
| Churn detection (planned) | Detect compaction loops, repeated tool calls, no progress | **Hermes middleware** | This needs to live inside Hermes's turn loop where it can observe tool-call patterns. Mosaic can't see this from outside. |
| Handoff packet (planned) | Structured context summary for session rotation | **Hermes toolset: `mission`** | Handoff is a serialization of mission + session state. Hermes owns the session, so it should own the handoff. |
### What Hermes already has that replaces Mosaic infrastructure
| Mosaic concept | Hermes equivalent | Notes |
| -------------------- | ------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| Gateway (NestJS) | Hermes gateway | Hermes already has a gateway with WebSocket, Discord, Telegram, CLI. No need for a second one. |
| Pi SDK agent runtime | Hermes agent loop | Hermes IS the agent runtime. OpenClaw's Pi SDK is a different runtime that Mosaic targets. |
| MACP ACP bridge | `delegate_task` + ACP tools | Same capability, already native. |
| Session identity | Hermes session IDs + process_registry | Hermes already tracks session identity, PIDs, and background processes. |
| Task execution board | Hermes Kanban | Fully functional SQLite-backed Kanban with dispatcher, triage, events, comments. |
| Worker spawning | Hermes dispatcher + cron | Kanban dispatcher + cron already handle this. |
| Context injection | Hermes skills + system prompt | Skills are loaded at session start and injected into context. Exactly what mosaic-framework plugin does. |
| File checkpoints | Hermes checkpoint_manager | Already tracks file mutations with shadow git. |
### What Mosaic keeps as its own entity
| Component | Why it stays in Mosaic |
| --------------------- | --------------------------------------------------- |
| `apps/gateway` | NestJS API surface — Mosaic's web platform offering |
| `apps/web` | Next.js dashboard — Mosaic's UI offering |
| `packages/types` | Shared TS contracts for Mosaic gateway plugins |
| `packages/db` | Drizzle ORM + PG — Mosaic's data layer |
| `packages/auth` | BetterAuth — Mosaic's auth system |
| `packages/brain` | PG-backed data layer for Mosaic web app |
| `packages/queue` | Valkey task queue for Mosaic gateway |
| `plugins/discord` | OpenClaw Discord plugin |
| `plugins/telegram` | OpenClaw Telegram plugin |
| `packages/mosaic` CLI | The `mosaic` CLI — Mosaic's own command surface |
---
## Architecture: `mission` Toolset for Hermes
### New files under `/opt/hermes/tools/`
```
mission_tools.py — Tool-call surface (mission_create, mission_status,
mission_next_task, mission_update_task, mission_handoff,
mission_resume)
mission_state.py — State management (read/write mission.json, parse TASKS.md,
parse MISSION-MANIFEST.md)
mission_churn.py — Churn detection (tool-loop counter, compaction counter,
progress scorer)
mission_handoff.py — Handoff packet generation and loading
```
### Tool-calls exposed to the agent
| Tool | What it does | When the agent calls it |
| --------------------- | --------------------------------------------------------------------------------- | ------------------------------------------- |
| `mission_create` | Initialize mission.json + TASKS.md + MISSION-MANIFEST.md in a project dir | When starting a new mission |
| `mission_status` | Read current mission state, milestone progress, next task, active session | At session start, or when checking progress |
| `mission_next_task` | Find the next `not-started` task whose dependencies are met, return its full spec | When the agent needs work to do |
| `mission_update_task` | Update a task row status in TASKS.md | When completing or blocking a task |
| `mission_handoff` | Generate a handoff packet from current session context + mission state | Before session rotation or at session end |
| `mission_resume` | Load a handoff packet and inject it as context for the new session | At session start after rotation |
### Toolset registration
The `mission` toolset follows the same pattern as `kanban`:
1. **Gating**: Tools are available when:
- The profile has `mission` in its toolsets config, OR
- A `HERMES_MISSION_DIR` env var is set (cron/dispatcher spawned workers)
2. **File conventions**: The toolset reads/writes the same file formats as Mosaic `packages/coord`:
- `.mosaic/orchestrator/mission.json` — mission state
- `docs/TASKS.md` — task table
- `docs/MISSION-MANIFEST.md` — mission manifest
- `docs/scratchpads/<id>.md` — session scratchpad
3. **Kanban bridge**: Optional bidirectional sync between mission TASKS.md rows and Kanban task cards, so the dashboard sees mission tasks.
### Churn detection (middleware)
Churn detection lives in Hermes's turn loop, NOT as a tool-call. It observes:
- Repeated compaction events (context window pressure)
- Identical tool-call sequences (loop detection)
- No file state changes across N turns
- Repeated permission denials
When churn score exceeds threshold:
1. `mission_handoff` is called automatically
2. Session is rotated (fresh context window)
3. `mission_resume` is called in the new session
This is new infrastructure that only Hermes can provide (Mosaic runs outside the agent loop).
---
## Implementation Tasks
### Phase 1: Core state management (Python port of coord)
| Task | Files | Estimate |
| -------------------------------------------------- | ----------------------------- | -------- |
| 1.1 Port mission.json read/write to Python | `mission_state.py` | 2h |
| 1.2 Port TASKS.md parser to Python | `mission_state.py` | 2h |
| 1.3 Port MISSION-MANIFEST.md reader to Python | `mission_state.py` | 1h |
| 1.4 Implement `mission_create` tool-call | `mission_tools.py` | 1h |
| 1.5 Implement `mission_status` tool-call | `mission_tools.py` | 1h |
| 1.6 Implement `mission_next_task` tool-call | `mission_tools.py` | 1h |
| 1.7 Implement `mission_update_task` tool-call | `mission_tools.py` | 1h |
| 1.8 Register `mission` toolset in Hermes registry | `tools/registry.py` | 30m |
| 1.9 Add `mission` to orchestrator profile toolsets | `config.yaml` | 10m |
| 1.10 Write unit tests for mission_state | `tests/test_mission_state.py` | 2h |
| 1.11 Write unit tests for TASKS.md parser | `tests/test_tasks_parser.py` | 1h |
**Phase 1 estimate:** ~13h
### Phase 2: Handoff and session continuity
| Task | Files | Estimate |
| ------------------------------------------------- | ---------------------------------------- | -------- |
| 2.1 Define handoff packet schema (JSON) | `mission_handoff.py` | 1h |
| 2.2 Implement `mission_handoff` tool-call | `mission_handoff.py`, `mission_tools.py` | 2h |
| 2.3 Implement `mission_resume` tool-call | `mission_handoff.py`, `mission_tools.py` | 2h |
| 2.4 Wire handoff into session start (auto-resume) | agent loop hook | 2h |
| 2.5 Write tests for handoff round-trip | `tests/test_mission_handoff.py` | 1h |
**Phase 2 estimate:** ~8h
### Phase 3: Churn detection
| Task | Files | Estimate |
| -------------------------------------------------------------- | ----------------------------- | -------- |
| 3.1 Define churn signal weights and thresholds | `mission_churn.py` | 1h |
| 3.2 Implement tool-loop detector (consecutive identical calls) | `mission_churn.py` | 2h |
| 3.3 Implement compaction pressure detector | `mission_churn.py` | 1h |
| 3.4 Implement progress scorer (file state delta) | `mission_churn.py` | 2h |
| 3.5 Wire churn scoring into agent turn loop | agent loop middleware | 2h |
| 3.6 Implement auto-rotation trigger | agent loop + handoff | 2h |
| 3.7 Write tests for churn scoring | `tests/test_mission_churn.py` | 1h |
**Phase 3 estimate:** ~11h
### Phase 4: Kanban bridge + CLI surface
| Task | Files | Estimate |
| ---------------------------------------------------- | ------------------------ | -------- |
| 4.1 Implement TASKS.md → Kanban sync (one-way first) | `mission_kanban_sync.py` | 2h |
| 4.2 Add `hermes mission` CLI subcommand | `mission_cli.py` | 2h |
| 4.3 Add `hermes mission status` command | `mission_cli.py` | 1h |
| 4.4 Add `hermes mission init` command | `mission_cli.py` | 1h |
| 4.5 Add `hermes mission handoff` command | `mission_cli.py` | 1h |
| 4.6 Add `hermes mission resume` command | `mission_cli.py` | 1h |
**Phase 4 estimate:** ~8h
---
## File Format Compatibility
The Python implementation MUST read and write the exact same file formats as Mosaic's TypeScript `packages/coord`. This means:
1. **mission.json** schema is identical to `Mission` type in `packages/coord/src/types.ts`
2. **TASKS.md** table format is identical to what `packages/coord/src/tasks-file.ts` parses
3. **MISSION-MANIFEST.md** is free-form markdown (no parser needed — just read the file)
4. **Handoff packets** are a new JSON format defined in this toolset (Mosaic doesn't have them yet)
This way a project can use Hermes mission tools OR Mosaic `mosaic coord` commands interchangeably. The files are the contract.
---
## Relationship Diagram
```
Mosaic Stack (TypeScript) Hermes Agent (Python)
┌─────────────────────────┐ ┌─────────────────────────┐
│ packages/coord │ │ tools/mission_tools.py │
│ ├─ mission.ts │◄──────►│ ├─ mission_state.py │
│ ├─ tasks-file.ts │ same │ ├─ mission_handoff.py │
│ ├─ status.ts │ files │ ├─ mission_churn.py │
│ └─ runner.ts │ │ └─ mission_tools.py │
│ │ │ │
│ packages/prdy │ │ skills/prdy/ │
│ └─ templates, wizard │◄──────►│ └─ SKILL.md + templates │
│ │ │ │
│ plugins/mosaic-framework│ │ skills/ (existing) │
│ └─ context injection │◄──────►│ └─ kanban-orchestrator │
│ │ │ + mosaic-coding-* │
│ plugins/macp │ │ tools/delegate_task.py │
│ └─ ACP bridge │◄──────►│ └─ already covers this │
│ │ │ │
│ (stays in Mosaic) │ │ tools/kanban_tools.py │
│ apps/gateway │ │ └─ Hermes Kanban DB │
│ apps/web │ │ │
│ packages/db │ │ tools/cronjob_tools.py │
│ packages/queue │ │ └─ already covers cron │
└─────────────────────────┘ └─────────────────────────┘
```
---
## Open Questions
1. **Should the `mission` toolset ship with Hermes core, or as a plugin?**
- Recommendation: ship as a **built-in toolset** (like `kanban`) since mission coordination is a core agent capability, not an optional integration. The file formats are stable and the code is small.
2. **Should churn detection be per-profile configurable?**
- Recommendation: yes. Add `mission.churn_threshold` and `mission.churn_weights` to profile config.yaml. Default threshold = 5 consecutive no-progress turns.
3. **Should handoff packets live in the project dir or in Hermes home?**
- Recommendation: **project dir** (`.mosaic/handoffs/<session-id>.json`). This keeps them version-controlled and accessible regardless of which agent runtime picks up the project.
4. **Bidirectional Kanban sync?**
- Recommendation: **one-way first** (TASKS.md → Kanban). Bidirectional adds conflict resolution complexity. Ship one-way, add reverse sync in v2 if needed.
5. **PRD generation — skill or tool-call?**
- Recommendation: **skill** (`prdy`). PRD generation is a prompt engineering problem with templates. Skills already handle this pattern perfectly.
docs/archive/planning/legacy/2026-05-07-coordination-resilience.md
+236
View File
@@ -0,0 +1,236 @@
# Mosaic Stack ↔ Hermes Coordination Resilience
> Purpose: document the self-healing coordination patterns that emerged while implementing the Hermes mission toolset, distress-card protocol, and auto-heal watchers, so the same mechanics can be reimplemented in Mosaic Stack or any similar agent platform.
## Summary
The coordination layer should be treated as a system of mechanical recovery loops rather than a single interactive agent session.
## SIBKISS operational summary
- mission on
- heartbeat always
- resume from packet
- block with `[BLOCKED]`
- reassign
- keep tasks tiny
- auto-heal dead workers
The design has four parts:
1. Atomic task decomposition — workers operate only within a small, explicit scope.
2. Distress signaling — workers create a standardized `[BLOCKED]` card when they encounter a blocker outside their scope.
3. Mechanical fallback — if the worker cannot phone home because of rate limits or dead context, a cron-style watcher synthesizes the distress card for them.
4. Auto-heal / reassignment — stale workers are reaped, crash-loops are reset, and rate-limited work is reassigned to a different profile/provider.
## Why this exists
Observed failure modes:
- Scope creep: a worker completes the target fix, then spends the rest of its budget chasing downstream cascade work.
- Silent failure / dead worker: the worker PID is gone, but the task remains running or blocked.
- Rate-limited worker: the worker is too constrained to create a help card itself, so it spins or fails without a clean handoff.
The answer is not to raise iteration caps or ask the worker to keep trying longer. The answer is to make the coordination layer self-healing and the work items atomic.
## Core workflow
### 1) Atomic task boundaries
Every task should have:
- one concern
- explicit files/packages in scope
- explicit files/packages out of scope
- a maximum file count if possible
- a stated expected iteration budget
When a worker discovers work outside scope, it must stop fixing it and hand off.
### 2) Worker-authored distress card
If the worker can still report status, it creates a card like:
- Title: `[BLOCKED] t_<source_id> <blocker_type>`
- Assignee: `tuesday` / orchestrator role
- Status: `ready`
- Body: standardized distress template with source task, blocker type, completed work, cannot-touch scope, and needed action
The orchestrator receives the card, acts on it, and closes the loop.
## Routing rules
### Distress card routing
- Title: `[BLOCKED] t_<source_id> <blocker_type>`
- Assignee: `tuesday` / orchestrator role
- Status: `ready`
- Body: standardized distress template with source task, blocker type, completed work, cannot-touch scope, and needed action
- Source task stays linked to the distress card so the recovery trail is auditable
The orchestrator receives the card, acts on it, and closes the loop.
### 3) Mechanical fallback for rate-limited workers
If the worker is too rate-limited or unstable to create the distress card itself, a no-agent watcher must synthesize the card from the task row and failure metadata.
That watcher should:
- inspect running / blocked tasks
- detect repeated 429 / 503 / overload errors
- create the same standardized `[BLOCKED]` card on behalf of the worker
- link the distress card to the source task
- add a comment to the source task
- allow the dispatcher to pick up the new card immediately
This is the key fix for the logic issue: the worker does not need to be able to phone home if the watcher can do it mechanically.
### 4) Auto-heal for dead workers
A separate no-agent watcher should:
- reap dead PIDs stuck in `running`
- reset crash-loops whose failures are infrastructure-related
- escalate tasks that have been reset too many times
This watcher prevents stale tasks from clogging the board and keeps the dispatch queue moving.
## Distress card contract
### Canonical title
```text
[BLOCKED] t_<source_task_id> <blocker_type>
```
### Canonical blocker types
- `scope_boundary`
- `env_blocker`
- `credential_failure`
- `dependency`
- `iteration_budget`
- `rate_limited`
### Canonical body
```markdown
## Distress Signal
- Blocked task: t_xxx
- Worker: <profile_name>
- Branch: <git_branch_name>
- Workspace: <path>
- Blocker type: <type>
- Completed: <what was done>
- Cannot touch: <out-of-scope packages/files>
- Needs: <what the orchestrator should do>
- State: committed | uncommitted | stashed(<stash_name>)
## Scope Guard
DO NOT touch: anything outside diagnosing and remediating the blocker described above
Only fix: assign, split, reassign, or unblock the source task
```
## Routing rules
### Distress card routing
- `[BLOCKED]` title prefix should bypass normal triage.
- The card should go directly to the orchestration profile.
- The orchestrator should start from a clean session each time.
### Rate-limit fallback
When the source task is rate-limited:
- do not keep retrying in the worker
- let the watcher synthesize the distress card
- have the orchestrator reassign the source task to a different profile/provider combo
### Provider fallback principle
Never reassign rate-limited work back to the same provider if the failure was provider pressure. Use a different provider when possible.
### Suggested fallback order
1. Keep the current task body and scope guards intact.
2. Reassign to a different profile on a different provider.
3. If that is impossible, reassign to a different profile on the same provider only for non-rate-limit blockers.
4. If repeated failures continue, split the task into a narrower atomic card.
## Related recovery docs
- Mission packet recovery contract: `/opt/hermes/docs/mission-toolset-heartbeat.md`
- Hermes mission implementation plan: `/opt/hermes/docs/plans/mission-toolset-implementation.md`
- The same packet-first resume rule applies: inspect the latest packet before re-reading mission files.
- New-session trigger: when a profile config changes, start a fresh session or `/reset` so the updated toolset is actually loaded.
## Watchers to implement
### Auto-heal watcher
Responsibilities:
- reap stale workers
- reset dead-PID crash loops
- track reset counts
- escalate after repeated resets
### Distress synthesizer watcher
Responsibilities:
- detect rate-limited / stuck workers
- create `[BLOCKED]` cards mechanically
- link the card to the source task
- leave a comment for traceability
### Iteration-budget watcher
Responsibilities:
- detect long-running tasks and repeated failure patterns
- recommend splits when a task is clearly over-scoped
- report tasks that need human review after multiple resets
## Operational principle
If a task cannot cleanly finish within its atomic scope, the right response is to surface a smaller coordination problem, not to keep burning context.
This is what makes the system robust across compaction, rate limits, and dead workers.
## Suggested implementation order
1. Atomic task metadata in task bodies
2. Worker-authored distress card protocol
3. Mechanical distress synthesizer watcher
4. Auto-heal watcher for dead workers
5. Orchestrator routing rules for `[BLOCKED]`
6. Rate-limit fallback / model reassignment table
## Where this fits in Hermes
- Kanban = durable work graph and status engine
- Watchers = mechanical healing and distress synthesis
- Orchestrator = split / reassign / unblock decision-maker
- Workers = execution inside atomic task boundaries
## Where this fits in Mosaic Stack
- PRD / coordination infra should encode the same patterns
- Mosaic can use the same distress-card contract and watcher logic
- The coordination model should be runtime-agnostic: any agent system can use it if it can write a task card and react to a ready queue
## Cross-project takeaway
The important pattern is not the specific tool names. It is the mechanical feedback loop:
- detect failure without requiring the failing worker to succeed
- create a standardized help artifact
- route that artifact to a fresh orchestrator context
- repair the assignment graph
- continue the mission
That pattern is reusable anywhere.
docs/archive/planning/legacy/README.md
+28
View File
@@ -0,0 +1,28 @@
# Legacy plans and deferred design stubs
> **Status:** Historical planning archive. These files preserve prior proposals and implementation approaches; they are not proof of shipped behavior or authority to run commands.
The records below moved byte-identically from migration quarantine on 2026-08-10. Validate every claim against current source, tests, configuration, and safety policy before reuse.
## Implementation plans
- [Gateway security hardening](2026-03-13-gateway-security-hardening.md)
- [Agent platform architecture](2026-03-15-agent-platform-architecture.md)
- [Wave 2 TUI layout and navigation](2026-03-15-wave2-tui-layout-navigation.md)
- [Hermes–Mosaic alignment](2026-05-06-hermes-mosaic-alignment.md)
- [Coordination resilience](2026-05-07-coordination-resilience.md)
- [Gateway token recovery](gateway-token-recovery.md)
## Setup record
- [Authentik SSO setup](authentik-sso-setup.md) — superseded for current administration by the canonical [SSO provider guide](../../../ADMIN-GUIDE/security/sso-providers.md).
## Explicitly deferred stubs
- [Chroot agent sandboxing](chroot-sandboxing.md)
- [Gatekeeper service](gatekeeper-service.md)
- [Task queue unification](task-queue-unification.md)
## Exclusions
The Agent Reflection PRD remains in quarantine because a live MACP test names its intended canonical path. The WebUI/Fleet Claude bridge draft remains authority-gated and coupled to Fleet decisions. Neither was moved in this archival slice.
docs/archive/planning/legacy/authentik-sso-setup.md
+40
View File
@@ -0,0 +1,40 @@
# Authentik SSO Setup
## Create the Authentik application
1. In Authentik, create an OAuth2/OpenID Provider.
2. Create an Application and link it to that provider.
3. Copy the generated client ID and client secret.
## Required environment variables
Set these values for the gateway/auth runtime:
```bash
AUTHENTIK_CLIENT_ID=your-client-id
AUTHENTIK_CLIENT_SECRET=your-client-secret
AUTHENTIK_ISSUER=https://authentik.example.com
```
`AUTHENTIK_ISSUER` should be the Authentik base URL, for example `https://authentik.example.com`.
## Redirect URI
Configure this redirect URI in the Authentik provider/application:
```text
{BETTER_AUTH_URL}/api/auth/callback/authentik
```
Example:
```text
https://mosaic.example.com/api/auth/callback/authentik
```
## Test the flow
1. Start the gateway with `BETTER_AUTH_URL` and the Authentik environment variables set.
2. Open the Mosaic login flow and choose the Authentik provider.
3. Complete the Authentik login.
4. Confirm the browser returns to Mosaic and a session is created successfully.
docs/archive/planning/legacy/chroot-sandboxing.md
+60
View File
@@ -0,0 +1,60 @@
# Chroot Agent Sandboxing — Process Isolation for Agent Tool Execution
> **Status:** Stub — deferred. Referenced from `2026-03-15-agent-platform-architecture.md` (Phase 7 Workspaces → Chroot Agent Sandboxing).
> Implement after Workspaces (P8-015) is complete. Requires workspace directory structure and `WorkspaceService` to be operational.
**Date:** 2026-03-15
**Packages:** `apps/gateway`
---
## Problem Statement
Agent sessions can use file, git, and shell tools. Path validation in tools is defense-in-depth but insufficient alone — an agent with shell access can run `cat /opt/mosaic/.workspaces/other_user/...` and bypass gateway RBAC.
Chroot provides OS-level enforcement: tool processes literally cannot see outside their workspace directory.
---
## Design (Sweet Spot)
Chroot strikes the balance between full container isolation (too heavy per session) and path validation only (escape-prone):
- Gateway spawns tool processes inside a chroot rooted at the session's `sandboxDir`
- Requires `CAP_SYS_CHROOT` capability on the gateway process (not full root)
- Chroot environment provisioned by `WorkspaceService` on workspace creation (minimal deps: git, shell utils, language runtimes as needed)
- Alternative for Docker deployments: Linux `unshare` namespaces (lighter, no chroot env setup)
---
## Scope (To Be Designed)
- [ ] Chroot environment provisioning — `WorkspaceService.provisionChroot(workspacePath)` on project creation
- [ ] Minimal chroot deps — identify required binaries/libs per tool type (file: none; git: git binary; shell: bash, common utils)
- [ ] Gateway capability — document `CAP_SYS_CHROOT` requirement; Dockerfile and docker-compose.yml changes
- [ ] Tool process spawning — modify `createShellTools`, `createFileTools`, `createGitTools` to spawn via chroot wrapper
- [ ] Docker alternative — `unshare --mount --pid --user` namespace wrapper as fallback for environments without chroot capability
- [ ] Defense-in-depth layering — chroot + path validation both active; neither alone is sufficient
- [ ] Chroot cleanup — integrate with `SessionGCService` / workspace deletion
- [ ] AppArmor/SELinux profiles (v2) — restrict gateway process file access patterns for multi-tenant hardening
---
## Security Constraints
- What lives **inside** the chroot (agent-accessible): workspace files, git repo, language runtimes
- What lives **outside** the chroot (gateway-only, never agent-accessible): Valkey connection, PG connection, other users' workspaces, gateway config, OTEL endpoint, credentials
---
## Dependencies
- Workspaces (P8-015) — chroot is rooted at workspace directory; workspace must exist first
- Tool hardening (P8-016) — path validation stays active as defense-in-depth alongside chroot
---
## References
- Original design context: `docs/plans/2026-03-15-agent-platform-architecture.md` → "Chroot Agent Sandboxing" section
- Current tool implementations: `apps/gateway/src/agent/tools/`
docs/archive/planning/legacy/gatekeeper-service.md
+53
View File
@@ -0,0 +1,53 @@
# Gatekeeper Service — PR Review, Quality Gates & Merge Authority
> **Status:** Stub — deferred. Referenced from `2026-03-15-agent-platform-architecture.md` (Phase 7 Workspaces).
> Implement after Workspaces (P8-015) is complete and the workspace/git infrastructure is operational.
**Date:** 2026-03-15
**Packages:** `apps/gateway`, `packages/types`, `packages/agent`
---
## Problem Statement
Project agents create PRs but cannot review or merge their own work. A separate, isolated agent service with read-only code access and quality gate enforcement is needed to act as the authoritative merge authority.
The Gatekeeper existed in the old Mosaic codebase and must be ported/redesigned for mosaic-mono-v1.
---
## Key Design Constraints
- **Isolated trust boundary** — project agents cannot invoke Gatekeeper directly; it listens for PR events from the git provider
- **`isSystem: true`** — system agent, not editable by users
- **Read-only code access** — reads diffs and runs checks; cannot commit or push
- **Quality gates required before merge** — lint, typecheck, test results must pass
- **Cannot self-approve** — the agent that authored the PR cannot be the Gatekeeper for that PR
---
## Scope (To Be Designed)
- [ ] Gatekeeper agent bootstrap — system agent config, tool set, prompt engineering
- [ ] PR event listener — Gitea/GitHub webhook integration (PR opened/updated/ready)
- [ ] Quality gate runner — trigger CI checks, poll for results, enforce pass criteria
- [ ] Review generation — LLM-driven code review comment generation
- [ ] Merge execution — approve + merge when gates pass; reject with comments when they fail
- [ ] Configurable strictness — per-project required checks, review depth
- [ ] Trust boundary enforcement — gateway rejects Gatekeeper tool calls that exceed read-only scope
- [ ] Audit trail — OTEL spans for all Gatekeeper decisions (approve/reject/merge)
---
## Dependencies
- Workspaces (P8-015) — Gatekeeper needs project workspace layout to locate code
- Git provider API tools — PR creation/review/merge API (Gitea/GitHub/GitLab)
- CI/CD tool integration — Woodpecker pipeline status polling
---
## References
- Original design context: `docs/plans/2026-03-15-agent-platform-architecture.md` → "Gatekeeper Service" section
- Workspace RBAC and agent trust model: same document → "RBAC & Filesystem Security"
docs/archive/planning/legacy/gateway-token-recovery.md
+193
View File
@@ -0,0 +1,193 @@
# Gateway Admin Token Recovery — Implementation Plan
**Mission:** `cli-unification-20260404`
**Task:** `CU-03-01` (planning only — no runtime code changes)
**Status:** Design locked (Session 1) — BetterAuth cookie-based recovery
---
## 1. Problem Statement
The gateway installer strands operators when the admin user exists but the admin
API token is missing. Concrete trigger:
- `~/.config/mosaic/gateway/meta.json` was deleted / regenerated.
- The installer was re-run after a previous successful bootstrap.
Flow today (`packages/mosaic/src/commands/gateway/install.ts:375-400`):
1. `bootstrapFirstUser` hits `GET /api/bootstrap/status`.
2. Server returns `needsSetup: false` because `users` count > 0.
3. Installer logs `Admin user already exists — skipping setup. (No admin token on file — sign in via the web UI to manage tokens.)` and returns.
4. The operator now has:
- No token in `meta.json`.
- No CLI path to mint a new one (`mosaic gateway <anything>` that needs the token fails).
- `POST /api/bootstrap/setup` locked out — it only runs when `users` count is zero (`apps/gateway/src/admin/bootstrap.controller.ts:34-37`).
- `POST /api/admin/tokens` gated by `AdminGuard` — requires either a bearer token (which they don't have) or a BetterAuth session (which they don't have in the CLI).
Dead end. The web UI is the only escape hatch today, and for headless installs even that may be inaccessible.
## 2. Design Summary
The BetterAuth session cookie is the authority. The operator runs
`mosaic gateway login` to sign in with email/password, which persists a session
cookie via `saveSession` (reusing `packages/mosaic/src/auth.ts`). With a valid
session, `mosaic gateway config recover-token` (stranded-operator entry point)
and `mosaic gateway config rotate-token` call the existing authenticated admin
endpoint `POST /api/admin/tokens` using the cookie, then persist the returned
plaintext to `meta.json` via `writeMeta`. **No new server endpoints are
required** — `AdminGuard` already accepts BetterAuth session cookies via its
`validateSession` path (`apps/gateway/src/admin/admin.guard.ts:90-120`).
## 3. Surface Contract
### 3.1 Server — no changes required
| Endpoint | Status | Notes |
| ------------------------------ | --------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `POST /api/admin/tokens` | **Reuse as-is** | `admin-tokens.controller.ts:46-72`. Returns `{ id, label, scope, expiresAt, lastUsedAt, createdAt, plaintext }`. |
| `GET /api/admin/tokens` | **Reuse** | Useful for `mosaic gateway config tokens list` follow-on (out of scope for CU-03-01, but trivial once auth path exists). |
| `DELETE /api/admin/tokens/:id` | **Reuse** | Used by rotate flow for optional old-token revocation. |
| `POST /api/bootstrap/setup` | **Unchanged** | Remains first-user-only; not part of recovery. |
`AdminGuard.validateSession` takes BetterAuth cookies from `request.raw.headers`
via `fromNodeHeaders` and calls `auth.api.getSession({ headers })`. It also
enforces `role === 'admin'`. This is exactly the path the CLI will hit with
`Cookie: better-auth.session_token=...`.
**Confirmed feasible** during CU-03-01 investigation.
### 3.2 `mosaic gateway login`
Thin wrapper over the existing top-level `mosaic login`
(`packages/mosaic/src/cli.ts:42-76`) with gateway-specific defaults pulled from
`readMeta()`.
| Aspect | Behavior |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Default gateway URL | `http://${meta.host}:${meta.port}` from `readMeta()`, fallback `http://localhost:14242`. |
| Flow | Prompt email + password -> `signIn()` -> `saveSession()`. |
| Persistence | `~/.mosaic/session.json` via existing `saveSession` (7-day expiry). |
| Decision | **Thin wrapper**, not alias. Rationale: defaults differ (reads `meta.json`), and discoverability under `mosaic gateway --help`. |
| Implementation | Share the sign-in logic by extracting a small `runLogin(gatewayUrl, email?, password?)` helper; both commands call it. |
### 3.3 `mosaic gateway config rotate-token`
| Aspect | Behavior |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Precondition | Valid session (via `loadSession` + `validateSession`). On failure, print: "Not signed in — run `mosaic gateway login`" and exit non-zero. |
| Request | `POST ${gatewayUrl}/api/admin/tokens` with header `Cookie: <session>`, body `{ label: "CLI token (rotated YYYY-MM-DD)" }`. |
| On success | Read meta via `readMeta()`, set `meta.adminToken = plaintext`, `writeMeta(meta)`. Print the token banner (reuse `printAdminTokenBanner` shape). |
| Old token | **Optional `--revoke-old`** flag. When set and a previous `meta.adminToken` existed, call `DELETE /api/admin/tokens/:id` after rotation. Requires listing first to find the id; punt to CU-03-02 decision. Document as nice-to-have. |
| Exit codes | `0` success; `1` network error; `2` auth error; `3` server rejection. |
### 3.4 `mosaic gateway config recover-token`
Superset of `rotate-token` with an inline login nudge — the "stranded operator"
entry point.
| Step | Action |
| ---- | -------------------------------------------------------------------------------------------------------------------------------- |
| 1 | `readMeta()` — derive gateway URL. If meta is missing entirely, fall back to `--gateway` flag or default. |
| 2 | `loadSession(gatewayUrl)` then `validateSession`. If either fails, prompt inline: email + password -> `signIn` -> `saveSession`. |
| 3 | `POST /api/admin/tokens` with cookie, label `"Recovered via CLI YYYY-MM-DDTHH:mm"`. |
| 4 | Persist plaintext to `meta.json` via `writeMeta`. |
| 5 | Print the token banner and next-steps hints (e.g. `mosaic gateway status`). |
| 6 | Exit `0`. |
Key property: this command is **runnable with nothing but email+password in hand**.
It assumes the gateway is up but assumes no prior CLI session state.
### 3.5 File touch list (for CU-03-02..05 execution)
| File | Change |
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `packages/mosaic/src/commands/gateway.ts` | Register `login`, `config recover-token`, `config rotate-token` subcommands under `gw`. |
| `packages/mosaic/src/commands/gateway/config.ts` | Add `runRecoverToken`, `runRotateToken` handlers; export from module. |
| `packages/mosaic/src/commands/gateway/login.ts` (new) | Thin wrapper calling shared `runLogin` helper with meta-derived default URL. |
| `packages/mosaic/src/auth.ts` | No change expected. Possibly export a `requireSession(gatewayUrl)` helper (reuse pattern). |
| `packages/mosaic/src/commands/gateway/install.ts` | `bootstrapFirstUser` branch: "user exists, no token" -> offer recovery (see Section 4). |
## 4. Installer Fix (CU-03-06 preview)
Current stranding point is `install.ts:388-395`. The fix:
```
if (!status.needsSetup) {
if (meta.adminToken) {
// unchanged — happy path
} else {
// NEW: prompt "Admin exists but no token on file. Recover now? [Y/n]"
// If yes -> call runRecoverToken(gatewayUrl) inline (interactive):
// - prompt email + password
// - signIn -> saveSession
// - POST /api/admin/tokens
// - writeMeta(meta) with returned plaintext
// - print banner
// If no -> print the current stranded message but include:
// "Run `mosaic gateway config recover-token` when ready."
}
}
```
Shape notes (actual code lands in CU-03-06):
- Extract the recovery body so it can be called **both** from the standalone
command and from `bootstrapFirstUser` without duplicating prompts.
- Reuse the same `rl` readline interface already open in `bootstrapFirstUser`
for the inline prompts.
- Preserve non-interactive behavior: if `process.stdin.isTTY` is false, skip the
prompt and emit the "run recover-token" hint only.
## 5. Test Strategy (CU-03-07 scope)
### 5.1 Happy paths
| Command | Scenario | Expected |
| ------------------------------------- | ------------------------------------------------ | -------------------------------------------------------- |
| `mosaic gateway login` | Valid creds | `session.json` written, 7-day expiry, exit 0 |
| `mosaic gateway config rotate-token` | Valid session, server reachable | `meta.json` updated, banner printed, new token usable |
| `mosaic gateway config recover-token` | No session, valid creds, server reachable | Prompts for creds, writes session + meta, exit 0 |
| Installer inline recovery | Re-run after `meta.json` wipe, operator says yes | Meta restored, banner printed, no manual CLI step needed |
### 5.2 Error paths (must all produce actionable messages and non-zero exit)
| Failure | Expected handling |
| --------------------------------- | --------------------------------------------------------------------------------- |
| Invalid email/password | BetterAuth 401 surfaced as "Sign-in failed: <server message>", exit 2 |
| Expired stored session | Recover command silently re-prompts; rotate command exits 2 with "run login" hint |
| Gateway down / connection refused | "Could not reach gateway at <url>" exit 1 |
| Server rejects token creation | Print status + body excerpt, exit 3 |
| Meta file missing (recover) | Fall back to `--gateway` flag or default; warn that meta will be created |
| Non-admin user | `AdminGuard` 403 surfaced as "User is not an admin", exit 2 |
### 5.3 Integration test (recommended)
Spin up gateway in test harness, create admin user via `/api/bootstrap/setup`,
wipe `meta.json`, invoke `mosaic gateway config recover-token` programmatically,
assert new `meta.adminToken` works against `GET /api/admin/tokens`.
## 6. Risks & Open Questions
| # | Item | Severity | Mitigation |
| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------- |
| 1 | `AdminGuard.validateSession` calls `getSession` with `fromNodeHeaders(request.raw.headers)`. CLI sends `Cookie:` header only. Confirm BetterAuth reads from `Cookie`, not `Set-Cookie`. | Low | Confirmed — `mosaic login` + `mosaic tui` already use this flow successfully (`cli.ts:137-181`). |
| 2 | Session cookie local expiry (7d) vs BetterAuth server-side expiry may drift. | Low | `validateSession` hits `get-session`; handle 401 by re-prompting. |
| 3 | Label collision / unbounded token growth if operators run `recover-token` repeatedly. | Low | Include ISO timestamp in label. Optional `--revoke-old` in CU-03-02. Add `tokens list/prune` later. |
| 4 | `mosaic login` exists at top level and `mosaic gateway login` is a wrapper — risk of confusion. | Low | Document that `gateway login` is the preferred entry for gateway operators; top-level stays for compatibility. |
| 5 | `meta.json` write is not atomic. Crash between token creation and `writeMeta` leaves an orphan token server-side with no plaintext on disk. | Medium | Accept for now — re-running `recover-token` mints a fresh token. Document as known limitation. |
| 6 | Non-TTY installer runs (CI, headless provisioners) cannot prompt for creds interactively. | Medium | Installer inline recovery must skip prompt when `!process.stdin.isTTY`; emit the recover-token hint. |
| 7 | If `BETTER_AUTH_SECRET` rotates between login and recover, the session cookie is invalid — user must re-login. Acceptable but surface a clear error. | Low | Error handler maps 401 on recover -> "Session invalid; re-run `mosaic gateway login`". |
| 8 | No MFA today. When MFA lands, BetterAuth sign-in will return a challenge, not a cookie — recovery UX will need a second prompt step. | Future | Out of scope for this mission. Flag for future CLI work. |
## 7. Downstream Task Hooks
| Task | Scope |
| -------- | -------------------------------------------------------------------------- |
| CU-03-02 | Implement `mosaic gateway login` wrapper + shared `runLogin` extraction. |
| CU-03-03 | Implement `mosaic gateway config rotate-token`. |
| CU-03-04 | Implement `mosaic gateway config recover-token`. |
| CU-03-05 | Wire commands into `gateway.ts` registration, update `--help` copy. |
| CU-03-06 | Installer inline recovery hook in `bootstrapFirstUser`. |
| CU-03-07 | Tests per Section 5. |
| CU-03-08 | Docs: update gateway install README + operator runbook with recovery flow. |
docs/archive/planning/legacy/task-queue-unification.md
+60
View File
@@ -0,0 +1,60 @@
# Task Queue Unification — @mosaicstack/queue as Unified Orchestration Layer
> **Status:** Stub — deferred. Referenced from `2026-03-15-agent-platform-architecture.md` (Task Queue & Orchestration section).
> Implement after Workspaces (P8-015) is complete. Requires workspace file structure to be in place.
**Date:** 2026-03-15
**Packages:** `packages/queue`, `packages/coord`, `packages/db`, `apps/gateway`
---
## Problem Statement
Two disconnected task systems exist:
1. **`@mosaicstack/coord`** — file-based missions (`mission.json`, `TASKS.md`), file locks, subprocess spawning. Single-machine orchestrator pattern.
2. **PG tables** (`tasks`, `mission_tasks`, `missions`) — DB-backed CRUD, REST API, Brain repos.
An agent using `coord_mission_status` gets file data. The dashboard shows DB data. They are never in sync.
---
## Vision
`@mosaicstack/queue` becomes the unified task orchestration service bridging PG, workspace files, and Valkey:
- DB is source of truth for structured state (status, assignees, timestamps)
- Workspace files (`TASKS.md`, PRDs) are working copies for agent interaction
- Valkey handles real-time assignment queues and agent claim locks
- Flatfile fallback for no-DB single-machine deployments (preserves `@mosaicstack/coord` pattern)
---
## Scope (To Be Designed)
- [ ] `@mosaicstack/queue` refactor — elevate from ioredis primitive to task orchestration service
- [ ] DB ↔ file sync layer — writes to PG propagate to `TASKS.md`; file edits by agents sync back
- [ ] Task assignment queue — Valkey-backed RPUSH/BLPOP for agent task claiming
- [ ] Agent claim locks — `mosaic:queue:project:{id}:lock:{taskId}` with TTL
- [ ] `@mosaicstack/coord` consolidation — file-based ops ported into queue service; `@mosaicstack/coord` becomes thin adapter or deprecated
- [ ] Flatfile fallback — queue service writes JSON manifests when PG unavailable
- [ ] Status pub/sub — real-time task status updates via Valkey pub/sub
- [ ] Dependency resolution — block task assignment until dependencies are met
- [ ] Orchestrator monitor — gateway process watches task queue, assigns next based on dependency graph
- [ ] API surface — queue service exposes typed interface used by agents, gateway, and CLI
---
## Dependencies
- Workspaces (P8-015) — file sync targets the workspace directory structure
- Teams architecture (P8-007) — project ownership determines queue namespacing
- DB schema stable — task/mission tables must not change mid-unification
---
## References
- Original design context: `docs/plans/2026-03-15-agent-platform-architecture.md` → "Task Queue & Orchestration" section
- Current `@mosaicstack/coord` implementation: `packages/coord/src/`
- Current `@mosaicstack/queue` implementation: `packages/queue/src/`
docs/archive/planning/matrix-macp/README.md
+10
View File
@@ -0,0 +1,10 @@
# Archived Matrix/MACP proposals
> **Status:** Historical draft proposals. These records moved byte-identically from migration quarantine on 2026-08-10 and have no implementation or operational authority.
- [RFC-001: MACP Matrix-native communications](rfc-001-macp-matrix-native.md)
- [RFC-002: install, configuration, and topology](rfc-002-install-config-topology.md)
Current source and focused tests do not establish the proposed Matrix adapter, identity mapping, persistence, homeserver/appservice topology, or federation operations. See the canonical [channel protocol](../../../DEVELOPER-GUIDE/architecture/channel-protocol.md) for the implemented Discord boundary and explicit Matrix limitations.
Do not use these archived RFCs as deployment instructions or as evidence that Matrix/MACP functionality shipped.
docs/archive/planning/matrix-macp/rfc-001-macp-matrix-native.md
+449
View File
@@ -0,0 +1,449 @@
# RFC-001 — MACP: A Mosaic-Native, Matrix-Native Comms Layer
- **Status:** DRAFT — for Team Lead → Orchestrator staffing
- **Author:** MS-LEAD (reviewer identity `ms-lead-reviewer`)
- **Sponsor / veto:** Jason (human lead)
- **Date:** 2026-07-24
- **Program:** Mosaic Stack comms-evolution
- **Supersedes backbone:** the Hermes MCP chat bridge (strangler-retired, see §9)
- **Audience:** Team Leads, the Mosaic orchestrator, infra, and any harness maintainer (Claude Code / Codex / Pi / Goose)
> This is a **design document**. No code ships from this RFC. It exists to be decomposed into missions (P1→P5, §10) with per-phase acceptance criteria. Where a claim is uncertain or needs live validation, it is flagged **[VERIFY]**.
---
## 0. TL;DR
We are building a **Mosaic-native comms layer on Matrix**. We self-host a **Synapse** homeserver and register a privileged **Mosaic Application Service** (the "appservice") that the orchestrator controls. The appservice bulk-provisions one Matrix identity per agent-spin, creates and manages rooms, posts agent introductions, and tracks presence/liveness. Agents talk to it through a thin `packages/comms` client SDK. **tmux stays the P0 same-host fast path**; Matrix is the durable, presence-aware, federated layer above it, and **MACP** (the Mosaic Agent Comms Protocol) is the standard that says which path to use when. Federation is **per-site Synapse homeservers federated over TLS we control** — a direct answer to "the homelab agent went dark and took comms with it." We adopt **Buzz's patterns** (auto-detect/enroll, signed identity, unified event log, humans-and-agents on one surface) without adopting Buzz/Nostr as transport. We migrate off Hermes by the **strangler** pattern: stand native alongside, move channels as proven, retire at parity.
The **first standalone shippable slice is presence** (P1).
---
## 1. Goals / Non-Goals
### 1.1 Goals
- **G1 — Presence & liveness first.** A Team Lead must be able to answer "is my coordinator online, away, or dead?" in seconds, not by polling for 13 hours. Presence is the P1 slice and ships before anything else.
- **G2 — A native backbone we own.** Replace _Hermes-as-backbone_ with a self-hosted Synapse + a Mosaic-controlled appservice. External chat bridging becomes an optional edge, not the spine.
- **G3 — Turnkey harness enrollment.** `mosaic enroll` auto-detects the harness and self-registers the agent via the appservice on spin. No hand-rolled per-bot identity juggling.
- **G4 — A real protocol (MACP v1).** Structured, versioned event schema over Matrix custom event types; a documented routing contract for tmux vs Matrix; a documented escalation policy.
- **G5 — No central SPOF.** Per-site homeservers federated over TLS/DNS we already control, so one site going dark cannot take the fleet's comms with it.
- **G6 — Gate-action integrity.** Reviews / merges / approvals carry **signed authorship** (Buzz pattern) so a gate-critical action is cryptographically attributable, retiring the fragile "distinct bot identity" juggling.
### 1.2 Non-Goals
- **NG1 — Do NOT rip out working comms mid-MVP.** tmux fast-path and the existing `mos-comms` git-branch channel keep working until their replacement is proven at parity. This RFC is strangler, not big-bang.
- **NG2 — tmux is NOT being replaced.** tmux inter-agent comms remains **P0**. Matrix is _above_ it, not instead of it. MACP defines the boundary; it does not move it.
- **NG3 — Not adopting Buzz/Nostr as transport.** We adopt Buzz's _patterns_; the wire is Matrix.
- **NG4 — Not building a new chat client in P1–P4.** HIL uses an existing Matrix client (Element or equivalent) until/unless a custom client is justified (open question, §11).
- **NG5 — Not federating to the public Matrix network.** Federation is Mosaic-site-to-Mosaic-site over infrastructure we control. Public `matrix.org` federation is out of scope (and should likely be firewalled off).
- **NG6 — Not a Hermes feature-clone.** We reach _parity on the channels that matter_ (§9 checklist), not bug-for-bug Hermes compatibility.
---
## 2. Architecture
### 2.1 Layer diagram
```
┌───────────────────────────────────────────────┐
│ HUMAN (Jason / HIL) │
│ Element (or custom client) — §11 │
└───────────────────────┬───────────────────────┘
│ (same Matrix surface as agents)
│
┌──────────────────────────────────────────────▼──────────────────────────────────────────────┐
│ SYNAPSE HOMESERVER (self-hosted, ours) │
│ - Client-Server API (agents + humans send/receive events) │
│ - Application Service API (privileged AS hooks: transactions, user/room namespaces) │
│ - Presence EDUs, receipts, typing │
│ - Federation API (S2S) over TLS ── to peer site homeservers (§6, P4) │
└───────▲───────────────────────────────────▲──────────────────────────────────────▲───────────┘
│ AS API (hs_token / as_token) │ C-S API (per-agent access_token) │ S2S
│ │ │
┌───────┴───────────────────────┐ ┌────────┴─────────────────┐ ┌─────────┴──────────┐
│ MOSAIC APPSERVICE │ │ packages/comms (SDK) │ │ PEER SITE Synapse │
│ (apps/matrix-appservice) │ │ used by every harness │ │ (site-B, site-C…) │
│ THE "native layer" │ │ - login/whoami │ │ own appservice │
│ replacing Hermes-backbone │ │ - send MACP events │ │ own agents │
│ │ │ - subscribe/sync │ └────────────────────┘
│ - bulk-provision MXIDs │◄───┤ - presence heartbeat │
│ (@mosaic_<agent>:site) │ │ - signed-authorship │
│ - create/manage rooms │ │ envelope (gate acts) │
│ - post introductions │ └────────────┬─────────────┘
│ - track presence/liveness │ │ in-process / IPC
│ - enforce room taxonomy │ ┌─────────▼──────────────────────────────────────────┐
│ - escalation watchdog (§5) │ │ AGENT HARNESS │
│ - controlled by ORCHESTRATOR │ │ Claude Code / Codex / Pi / Goose │
└───────▲───────────────────────┘ │ `mosaic enroll` runs on spin (§4.1) │
│ orchestrator drives AS └─────────┬───────────────────────────────────────────┘
┌───────┴───────────────────────┐ │
│ MOSAIC ORCHESTRATOR │ │ P0 FAST PATH (same host, low-latency)
│ (~/.config/mosaic) │ ┌─────────▼──────────┐ tmux send-keys / pane I/O
│ spins agents, owns rooms, │◄──────►│ tmux (P0) │◄─►│ peer agent on same host │
│ sets escalation policy │ MACP └────────────────────┘ └─────────────────────────┘
└───────────────────────────────┘ routing rules decide tmux vs Matrix per message (§4.6)
```
Key idea: **the appservice is the backbone.** It is a long-lived privileged process registered with Synapse via an appservice registration file (`hs_token`/`as_token`, namespaces). It is the thing that used to be "Hermes-as-backbone," except we own it, it is inside the orchestrator's control plane, and it speaks native Matrix.
### 2.2 Message flow: agent spin-up → auto-enroll → room join → introduction → presence-online
```
Orchestrator Harness (mosaic enroll) Mosaic Appservice Synapse
│ │ │ │
1. spin agent ─────────────────► │ │ │
│ │ 2. auto-detect harness │ │
│ │ (Claude/Codex/Pi/Goose) │ │
│ │ 3. POST /enroll {agent meta} ─► │
│ │ │ 4. provision MXID │
│ │ │ @mosaic_<agent>:site │
│ │ │ via AS API register ─► (201, in namespace)
│ │ │ 5. mint access_token │
│ │ 6. ◄── {mxid, token, rooms}──┤ (or as_token masq) │
│ │ │ 7. invite+join rooms ─► (mission/team/fleet)
│ │ 8. /sync (via packages/comms)─────────────────────────► (joined state)
│ │ │ 9. post introduction ─► m.room.message +
│ │ │ (mosaic.introduction) custom event → rooms
│ │ 10. set presence ONLINE ─────────────────────────────► presence EDU
│ │ 11. start heartbeat loop │ │
│ │ (mosaic.presence ping) │ │
│ 12. appservice reports agent │ │ │
│ ◄──── live in fleet room ────┤ (watchdog now tracks liveness) │
```
Notes on the steps that matter:
- **Step 4/5** use the **Application Service API**: the appservice can register users inside its namespace (`@mosaic_*:site`) and act on their behalf. Two viable modes: (a) mint a real per-agent `access_token` via appservice login, or (b) have the appservice **masquerade** using `user_id` query param on C-S calls with the `as_token`. **Recommendation: mint per-agent tokens** for P2 so the agent process holds only its own credential (blast-radius containment, §8); reserve masquerade for bulk/bootstrap operations the appservice itself performs. **[VERIFY]** exact token-lifetime and refresh behavior against the running Synapse version.
- **Step 9** — the introduction is both a human-readable `m.room.message` _and_ a structured `mosaic.introduction` custom event (so other agents can machine-parse capabilities without scraping prose).
- **Step 10/11** — presence goes online immediately, then a **heartbeat** keeps liveness fresh. Native Matrix presence auto-decays to `unavailable`/`offline`, but we do **not** rely solely on it (Synapse presence timeouts are coarse and federation presence is lossy **[VERIFY]**); MACP adds an explicit `mosaic.presence` heartbeat event for deterministic liveness (§4.5, §5).
---
## 3. Repo-home decision (RESOLVED — recommendation)
The core tension: **product monorepo** (`mosaicstack/stack`, this checkout `/src/mosaic-stack`) vs **framework** (`~/.config/mosaic`, the agent/harness runtime that every agent shares regardless of product). The boundary rule I am ratifying:
> **Product-monorepo owns the deployed _services and libraries_. Framework owns the _agent/harness contract_ — anything an agent needs the moment it spins, before any product code is checked out.**
Applying that rule:
| Piece | Home | Rationale |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Synapse deployment** (compose/helm, config, TLS, `.well-known`, pipelines) | **Product monorepo** → `infra/matrix/` + CI in the monorepo's pipeline dir | It is deployed infrastructure with the same lifecycle/observability as Gateway/Web/DB. Lives beside other `infra/` and Woodpecker pipelines. |
| **Mosaic Appservice** | **Product monorepo** → `apps/matrix-appservice` | It is a first-class deployed service (NestJS-style app, same conventions: ESM, `@Inject()`, DTOs, OTEL-before-bootstrap). It talks to Synapse, holds `hs_token`/`as_token`, and is operated like any other app. It is **controlled by** the orchestrator but **is not** the orchestrator. |
| **Client SDK** | **Product monorepo** → `packages/comms` | A workspace library consumed by product apps _and_ by harnesses. Published/linked like `packages/queue`, `packages/db`. Versioned with the product. |
| **MACP spec** (the standard doc + JSON schemas) | **Framework** → `~/.config/mosaic/spec/macp/` (mirrored/vendored into `packages/comms` at build) | The protocol is an agent-level contract that must exist independent of any one product checkout. Framework is the source of truth; `packages/comms` vendors a pinned copy so the SDK and spec cannot drift silently. |
| **`mosaic enroll` harness glue** (auto-detect, spin hook) | **Framework** → `~/.config/mosaic/tools/enroll/` | Agents/harnesses are framework-level. Enrollment must run _before_ the agent has product context; it cannot depend on `/src/mosaic-stack` being present. This is exactly parallel to the existing `~/.config/mosaic/tools/*` wrappers. |
**Boundary summary:** _the wire and the services are product; the contract and the spin-time glue are framework._ The one deliberate coupling is **MACP**: framework is authoritative, but `packages/comms` pins a vendored copy and CI fails if they diverge, so an agent enrolling via framework and a service validating via `packages/comms` agree on the schema by construction.
**Rejected alternative:** putting the appservice in the framework. Rejected because the appservice is a stateful, deployed, secret-holding network service that needs the product's CI/observability/secret plumbing; burying it in `~/.config/mosaic` would split its operational story from every other Mosaic service.
---
## 4. MACP v1 — the standard
MACP (Mosaic Agent Comms Protocol) v1 is a **profile of Matrix**: it does not invent a transport, it constrains how Mosaic agents use Matrix so that behavior is uniform across harnesses. Versioned via a `macp_version` field on every custom event; v1 is frozen at ratification (P3).
### 4.1 Enrollment contract
`mosaic enroll` MUST, on agent spin, in order:
1. **Auto-detect harness.** Detection order + signal:
- Claude Code — presence of the Claude Code runtime/env (e.g. `CLAUDE_CODE_*` env, `~/.claude`) **[VERIFY exact signal per harness]**
- Codex — Codex runtime markers
- Pi — Pi SDK runtime (`packages/agent` / `packages/mosaic` context)
- Goose — Goose runtime markers
- Fallback: explicit `--harness` flag; if undetectable, enroll as `generic` and warn.
2. **Provision identity** — call appservice `POST /enroll` with `{agent_slug, harness, host, mission_id?, team_id?, capabilities[]}`. Appservice returns `{mxid, access_token, homeserver, rooms[]}` (§2.2 step 4–6).
3. **Join rooms** — accept invites / join the returned room set per taxonomy (§4.6).
4. **Introduce** — post `mosaic.introduction` (+ human-readable `m.room.message`) to each joined room.
5. **Go present** — set Matrix presence `online` and start the `mosaic.presence` heartbeat loop.
Enrollment is **idempotent**: re-running `mosaic enroll` for an existing agent slug rebinds to the same MXID (re-mints token if needed) rather than creating a duplicate identity. This is what retires the "distinct bot identity juggling."
### 4.2 Structured event schema (Matrix custom event types)
All MACP events carry a common envelope in `content`:
```jsonc
{
"macp_version": "1.0",
"macp_type": "<see below>",
"agent": { "mxid": "@mosaic_teamlead-3:site-a", "slug": "teamlead-3", "harness": "claude-code" },
"ts": 1753300000000,
"mission_id": "KBN-101", // optional
"signature": { ... } // present ONLY for gate actions, §4.4
// ...type-specific fields...
}
```
Event types (Matrix `type` shown; timeline events use `m.room.message` with a custom `msgtype` where a human-visible fallback is desirable, state events use a dotted custom `type`):
| MACP type | Matrix carrier | Purpose | Notable fields |
| --------------------- | --------------------------------------------------------------------------------------- | -------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| **message** | `m.room.message`, `msgtype: "mosaic.message"` (+ `body` fallback) | ordinary agent/human chat | `body`, `thread?` |
| **presence** | `m.room.message` msgtype `mosaic.presence` in fleet room **or** custom EDU-backed state | heartbeat/liveness ping | `status: online\|away\|offline`, `seq`, `interval_ms` |
| **workflow-step** | state event `mosaic.workflow.step` (state_key = step id) | durable record of a workflow/mission step | `step`, `status: started\|blocked\|done\|failed`, `detail` |
| **review / approval** | `m.room.message` msgtype `mosaic.review` | a review verdict or merge/approval | `subject` (PR/commit ref), `verdict: approve\|reject\|request-changes`, `signature` (REQUIRED) |
| **escalation** | `m.room.message` msgtype `mosaic.escalation` | raise a stuck/dark condition to HIL/fallback | `reason`, `target`, `severity`, `since_ts` |
Rationale for the carrier split: **timeline events** (`m.room.message` variants) are the durable, receipted, replayable log (this _is_ Buzz's "unified event log," §7). **State events** (`mosaic.workflow.step`, presence-as-state) give last-writer-wins current status that a newly-joined agent reads instantly from room state without replaying history.
Every custom event is validated against a JSON Schema shipped in the MACP spec (§3). Unknown/newer `macp_version` → consumers MUST degrade gracefully (render `body`, ignore unknown fields).
### 4.3 Agent identity model
- **MXID:** appservice-namespaced `@mosaic_<slug>:<site-domain>`. The `mosaic_` prefix is the AS **user namespace** declared in the registration file (`namespaces.users` regex `@mosaic_.*`), so Synapse routes those users to our appservice and no human can squat the namespace.
- **Provisioning:** exclusively via the appservice (AS API register). Never hand-created. One MXID per agent-spin; idempotent rebind on re-enroll (§4.1).
- **Signed authorship** overlays identity for gate actions only (§4.4). MXID answers "who is this account"; signature answers "did the real key-holder authorize this gate action."
### 4.4 Signed-authorship for gate-critical actions (Buzz pattern, scoped)
Gate-critical = **reviews, merges, approvals** — anything that can move code to `main` or unblock a mission gate.
- Each enrolled agent is issued (or generates) an **Ed25519 keypair**; the **public** key is registered with the appservice at enrollment and published as agent profile state (`mosaic.identity.pubkey`). Private key custody per §8.
- A gate action event carries `content.signature = { alg: "ed25519", key_id, sig }` over a canonical serialization of the envelope (canonical-JSON of `{macp_type, agent.mxid, subject, verdict, ts, mission_id}`).
- Verifiers (the appservice gate-watcher, and any agent acting on a verdict) MUST reject an unsigned or bad-signature gate event. Non-gate events are unsigned (keeps the hot path cheap).
This is deliberately **narrow**: we do not sign every chat line (Buzz signs everything; we take the pattern only where forgery has teeth). Scope may widen post-P5 if warranted.
### 4.5 Presence & liveness model
Three visible states plus an explicit heartbeat:
- **online** — agent set presence online AND last `mosaic.presence` heartbeat within `heartbeat_interval` (default **30s [VERIFY tuning]**) × miss-tolerance (default 2).
- **away** — presence `unavailable`, or heartbeats late but < dark threshold.
- **offline / dark** — no heartbeat for `dark_threshold` (default **N minutes**, policy value, §5/§11) OR presence `offline`.
Why not rely on native Matrix presence alone: Synapse presence is (a) coarse-grained, (b) can be disabled for load reasons, and (c) **degrades across federation** **[VERIFY]**. So MACP layers an explicit heartbeat carried as a lightweight timeline/state event in the **fleet presence room**, giving a deterministic, federation-safe liveness signal the escalation watchdog (§5) can reason about. Native presence EDUs are still emitted (they make Element show the right dot for humans) but the _authoritative_ liveness source is the heartbeat.
### 4.6 Room / channel taxonomy (orchestrator-owned)
The **orchestrator** (via the appservice) owns room lifecycle. Agents never create backbone rooms ad hoc.
| Room | Scope | Membership | Purpose |
| ----------------------- | ------------------------------------------ | ---------------------------------------- | ----------------------------------------------------------------------- |
| **Fleet presence room** | one per site (federated view across sites) | every enrolled agent + HIL | heartbeats, the single "who's alive" board. This is the P1 deliverable. |
| **Per-mission room** | one per mission (e.g. `#mission-KBN-101`) | agents on that mission + Team Lead + HIL | workflow-steps, mission chat, reviews for that mission |
| **Per-team room** | one per team | team members + Team Lead | intra-team coordination |
| **HIL room** | one (or one per site) | humans + escalation-privileged agents | where escalations land; Jason's pane on the fleet |
Rooms are created with orchestrator-controlled power levels: appservice = admin (PL100), Team Leads elevated, worker agents default. Room aliases (`#mission-KBN-101:site-a`) are stable handles.
### 4.7 tmux ↔ Matrix routing rules (the fast-path/durable boundary)
MACP mandates this decision per message. **Default bias: if it must survive the agent, be seen by an offline party, cross a host, or be audited — Matrix. If it is same-host, synchronous, and ephemeral — tmux.**
| Signal | Route | Why |
| ---------------------------------------------------------------- | ---------------------------------------------- | --------------------------------------------------------------------------- |
| Same-host, live pane-to-pane prompt/nudge, sub-second need | **tmux (P0)** | lowest latency, no server round-trip; this is the working fast path we keep |
| Recipient may be offline / on another host | **Matrix** | durability + store-and-forward; tmux can't reach a dark or remote pane |
| Presence / heartbeat / liveness | **Matrix** | must be observable fleet-wide, including by the watchdog and HIL |
| Workflow-step, review/approval, escalation | **Matrix** | must be durable, receipted, auditable, signed (gate acts) |
| Cross-site anything | **Matrix (federated)** | tmux is same-host only |
| Bulk log spew / high-frequency scratch between co-located agents | **tmux**, with periodic **Matrix** checkpoints | avoid flooding the durable log; keep an audit checkpoint |
Rule of thumb encoded in the SDK: `comms.send()` takes a `durability` hint; `ephemeral+same-host` short-circuits to tmux, everything else goes Matrix. A message can be **dual-routed** (tmux for immediacy + a Matrix checkpoint) when both speed and durability matter.
---
## 5. Coordinator-availability + HIL escalation (the 13h-stall / homelab-dark fix)
**The failure we are killing:** a Team Lead blocked ~13h, polling every 15 min, unable to distinguish "coordinator offline" from "coordinator busy"; and a homelab agent that goes dark taking comms with it.
**The fix — presence-driven, policy-encoded escalation:**
1. **Deterministic liveness (§4.5).** Every agent heartbeats into the fleet presence room. The appservice **escalation watchdog** subscribes and maintains `last_seen` per agent. No polling by the Team Lead — it _subscribes_ (Matrix `/sync` long-poll) and is pushed state changes.
2. **Encoded policy in MACP:** a machine-readable escalation policy attached to each agent/role, e.g.:
```jsonc
{
"role": "team-lead",
"coordinator": "@mosaic_coordinator-1:site-a",
"dark_threshold_min": 10, // OPEN QUESTION §11 — Jason/Mos to set N
"on_coordinator_dark": {
"action": "escalate",
"fallback": "@mosaic_coordinator-2:site-b", // cross-site fallback
"then": "notify-HIL",
"hil_room": "#hil:site-a",
},
}
```
3. **Detection & action by the Team Lead:** when the watchdog (or the Team Lead's own SDK) observes `coordinator.last_seen > dark_threshold`, it:
- emits a `mosaic.escalation` event into the mission room and the HIL room (`reason: "coordinator dark", since_ts, severity`),
- **re-routes** to the declared fallback coordinator (possibly on another site — this is why federation matters),
- if no fallback answers within a second threshold, pages **HIL** (Jason) in the HIL room.
The Team Lead **never sits blocked polling**; a dark coordinator is a _pushed event_, and the fallback/HIL path is automatic.
4. **Homelab-dark specifically:** because heartbeats are federated into a cross-site fleet room, a whole _site_ going dark is visible from other sites — the watchdog on site-B sees site-A's coordinator stop heartbeating and triggers the same escalation. A dark homelab can no longer silently strand its agents, because the liveness signal and the fallback live _off that host_.
**Design invariant:** liveness authority and fallback targets must never be co-located with the thing they monitor. The watchdog for site-A's coordinator should also run (or be mirrored) on site-B.
---
## 6. Federation (P4)
**Model:** each site runs its **own Synapse homeserver** with its **own Mosaic appservice** and its own agents. Sites **federate** with each other over the standard Matrix server-to-server (S2S) API, restricted to Mosaic sites.
**Why per-site, not one central server:**
- **No SPOF.** The homelab going dark is the founding trauma of this program. A single central homeserver would recreate exactly that risk at fleet scale. Per-site means a site outage is contained: its agents drop, but every other site's comms and the cross-site fleet room survive.
- **Locality.** Same-site agents get low-latency local homeserver traffic; only cross-site events pay the federation cost.
- **Blast radius.** A compromised or misbehaving site can be defederated without touching the rest.
**How federation is wired (real Matrix mechanics):**
- **Server discovery** via `https://<domain>/.well-known/matrix/server` returning `{"m.server": "matrix.<domain>:443"}`, and/or an `_matrix._tcp` **SRV** record. We control the DNS/domains, so we control the federation graph. **[VERIFY]** current `.well-known` vs SRV precedence for the deployed Synapse version.
- **TLS:** federation requires valid TLS on the federation endpoint; we terminate with certs from our own CA/ACME on domains we own.
- **Allowlist:** use Synapse `federation_domain_whitelist` to restrict federation to the set of Mosaic site domains — **no public-network federation** (NG5). This is a hard security boundary.
- **Cross-site rooms:** the fleet presence room and any cross-mission rooms are federated rooms whose membership spans site homeservers. Room state replicates via S2S; presence heartbeats propagate as events (not relying on lossy presence EDUs across federation, §4.5).
**Cross-site identity:** an agent on site-B is `@mosaic_<slug>:site-b`. The signed-authorship pubkey travels in profile state, so a site-A verifier can validate a site-B agent's gate action without trusting site-B's homeserver blindly (signature ≠ homeserver trust).
---
## 7. Buzz-pattern adoption map
We adopt Buzz's **ideas**, on Matrix rails, phased:
| Buzz idea | Adopt? | How, on Matrix | Phase |
| --------------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| **Harness auto-detect / enroll** | **Yes** | `mosaic enroll` detects Claude/Codex/Pi/Goose and self-registers via the appservice (§4.1) | P2 (enroll v1); scan-machine-and-offer-integrate is **v2** |
| **Keypair signed identity** | **Yes, scoped** | Ed25519 signed-authorship on **gate actions only** (reviews/merges/approvals), pubkey in profile state (§4.4) | P5 (hardening); keys issued at enroll from P2 |
| **Unified event log (humans + agents, one log)** | **Yes** | Matrix room timeline _is_ the unified, receipted, replayable event log; MACP custom events are first-class entries (§4.2) | P2→P3 |
| **Humans and agents on the same surface** | **Yes** | HIL uses the same Matrix rooms via Element/custom client; escalations land where Jason already is (§4.6, §5) | P1 (fleet room) → P2 |
| **Scan machine, offer to integrate existing tools** | **Deferred (v2)** | enroll v1 auto-detects the _harness_; scanning a host for other integratable tools is explicitly **enroll v2**, post-P5 | v2 |
| **Buzz/Nostr as transport** | **No** | transport is Matrix; only patterns are borrowed (NG3) | — |
---
## 8. Security
- **Homeserver hardening.** Disable open registration (`enable_registration: false`); agents come only via the appservice. `federation_domain_whitelist` to Mosaic sites only (§6). Rate-limiting on. Media repo locked down or disabled if unused. Admin API bound to localhost / behind auth. Run Synapse behind our reverse proxy with TLS termination we control. **[VERIFY]** current recommended hardening flags for the deployed Synapse version.
- **Appservice token custody (`hs_token` / `as_token`).** These are the crown jewels — `as_token` lets the holder act as any user in the namespace; `hs_token` authenticates the homeserver → appservice callbacks. They live **only** in the appservice's secret store (see Vault interaction below), never in an agent process, never in the SDK, never in git. Registration file (which contains them) is a secret artifact, mounted at deploy, not committed. Rotate on suspected exposure; rotation requires updating both Synapse's registration and the appservice.
- **Per-agent credentials (the hard part — fleet key management).**
- Agents hold **only their own** per-agent `access_token` (minted by the appservice at enroll), not the `as_token`. Compromise of one agent ≠ compromise of the namespace.
- **Issuance:** at `mosaic enroll`, the appservice mints the token and returns it over the enroll channel (which must itself be authenticated — enroll requests carry a spin-time bootstrap secret / orchestrator-signed nonce **[VERIFY design in P2]**, so a random process can't enroll a rogue agent).
- **Signed-authorship keys:** Ed25519 private keys should be generated agent-side and only the public key leaves the agent (best custody: private key never transits the network). Where agents are ephemeral, keys are minted per-spin and discarded on teardown; the pubkey-in-profile record remains for audit.
- **Rotation:** tokens are short-lived where the Synapse version supports refresh **[VERIFY]**; otherwise the appservice supports explicit re-issue on re-enroll. A rotation runbook is a P2 deliverable.
- **Federation TLS.** Valid certs on federation endpoints; whitelist-only federation; monitor for cert expiry (a cert lapse silently defederates a site — add to observability).
- **Interaction with existing Vault/secrets.** The appservice's `hs_token`/`as_token` and the enroll bootstrap secret are stored in the existing secret manager (Vault or the project's chosen store — **open question §11**) and injected at deploy, consistent with how Gateway/DB secrets are handled today. No new bespoke secret store. Per-agent tokens are _transient runtime_ secrets, not persisted to Vault (they're re-mintable). **[VERIFY]** current Mosaic secret-management choice — the CLAUDE.md notes secrets/KBN work in flight, so align with whatever KBN-101 lands.
---
## 9. Migration — strangler off Hermes
**Principle:** stand the native layer up _alongside_ Hermes, move channels over **as each is proven at parity**, retire Hermes only when the parity checklist is green. Never a flag-day cutover (NG1).
**Sequence:**
1. **Stand alongside.** Native Synapse + appservice + fleet presence room live in parallel; Hermes still carries everything it carries today. Presence (P1) is _additive_ — it gives us something Hermes never had, at zero risk to existing flows.
2. **Move channels as proven.** Per channel (fleet presence → mission coordination → reviews/approvals → external-chat edge), cut traffic to Matrix, keep Hermes as hot fallback until the channel meets parity for a soak period.
3. **Retire at parity.** When every checklist item is green and soaked, decommission the Hermes MCP bridge as backbone. (External chat platforms, if still needed, can be re-attached as a _bridge at the edge_ of Matrix rather than the spine.)
**Parity checklist (must all be green before Hermes retires):**
- [ ] Every message class Hermes carries today has a MACP equivalent (message, review/approval, escalation, workflow-step).
- [ ] Presence/liveness is strictly better than today (it is: today = none).
- [ ] Cross-site delivery works over federation with no central SPOF.
- [ ] HIL (Jason) can see and act on escalations on the Matrix surface.
- [ ] Signed-authorship enforced on gate actions (no unsigned merge/approve accepted).
- [ ] `mosaic enroll` auto-onboards all four harnesses (Claude/Codex/Pi/Goose) with zero manual identity setup.
- [ ] Delivery receipts / durability demonstrably ≥ Hermes (no lost messages over a soak window).
- [ ] Runbooks exist: appservice token rotation, site defederation, dark-site escalation, homeserver restore.
- [ ] Observability: appservice + Synapse traced into OTEL/Jaeger like the rest of the stack.
- [ ] Rollback path documented (re-enable Hermes channel) for the soak period.
---
## 10. Phased delivery plan (P1 → P5) with acceptance criteria
> This section is the decomposition surface: each phase → one or more missions for the orchestrator.
### P1 — Presence / availability on a minimal single-site Synapse _(the first standalone shippable slice)_
**Scope:** one Synapse homeserver, a **minimal** appservice (or even a scripted provisioner) whose only job is: register a handful of agent MXIDs, create the **fleet presence room**, and carry heartbeats; a minimal `packages/comms` slice that sets presence and heartbeats; native presence surfaced to a human via Element.
**Acceptance criteria:**
- A1. Single-site Synapse deployed (`infra/matrix/`), reachable over TLS, open registration OFF.
- A2. ≥3 agents enroll (even if semi-manually) and appear in a **fleet presence room** with a live online/away/offline indicator.
- A3. `mosaic.presence` heartbeat implemented; an agent killed hard flips to **offline/dark** within `dark_threshold` deterministically (not dependent on native presence timeout alone).
- A4. A human (Jason) can open Element, join the fleet room, and see fleet liveness at a glance.
- A5. Zero impact to existing tmux + `mos-comms` flows (they still work untouched).
### P2 — Native appservice + orchestrator auto-enroll / room-provisioning
**Scope:** full **`apps/matrix-appservice`** registered with Synapse (`hs_token`/`as_token`, namespaces); `mosaic enroll` harness auto-detect; orchestrator-owned room taxonomy; per-agent token minting; introductions.
**Acceptance criteria:**
- B1. Appservice registered with Synapse via registration file; owns `@mosaic_*` user namespace and room-alias namespace.
- B2. `mosaic enroll` auto-detects all four harnesses (Claude/Codex/Pi/Goose) and self-registers on spin, idempotently.
- B3. On spin, an agent is provisioned an MXID, minted its **own** access token, joined to the correct mission/team/fleet rooms, and posts a `mosaic.introduction`.
- B4. Orchestrator can create/destroy mission & team rooms with correct power levels via the appservice.
- B5. Enroll bootstrap is authenticated (a rogue local process cannot enroll a rogue agent).
- B6. Appservice + Synapse traced into OTEL/Jaeger.
### P3 — MACP v1 spec ratified
**Scope:** freeze the standard (§4): envelope, event types + JSON Schemas, identity model, presence model, room taxonomy, tmux↔Matrix routing rules. Spec lives in framework (`~/.config/mosaic/spec/macp`), vendored+pinned into `packages/comms` with CI drift-check.
**Acceptance criteria:**
- C1. MACP v1 document ratified (MS-LEAD sign-off, Jason veto window closed).
- C2. JSON Schemas for all five event types published; `packages/comms` validates outbound/inbound against them.
- C3. CI fails if framework spec and vendored `packages/comms` copy diverge.
- C4. Routing-rule conformance test: SDK provably sends ephemeral+same-host over tmux, everything else over Matrix.
- C5. Unknown-`macp_version` graceful-degrade behavior tested.
### P4 — Federation
**Scope:** a second site homeserver + appservice; S2S federation over our DNS/TLS; cross-site fleet room; whitelist-only federation; cross-site escalation.
**Acceptance criteria:**
- D1. Two sites federate via `.well-known`/SRV over TLS we control; `federation_domain_whitelist` restricts to Mosaic sites (no public federation).
- D2. A cross-site fleet presence room shows agents from both sites; heartbeats propagate as events across federation.
- D3. **Homelab-dark test:** killing site-A's coordinator is observed from site-B within `dark_threshold`, and the escalation/fallback fires cross-site (§5).
- D4. Cross-site gate action: a site-B agent's signed review is verified by a site-A verifier.
- D5. Defederation runbook proven (a site can be cut off cleanly).
### P5 — Buzz-hardening + signed-authorship + Hermes retired
**Scope:** Ed25519 signed-authorship enforced on gate actions; security hardening pass; complete the strangler and retire Hermes at parity.
**Acceptance criteria:**
- E1. Every merge/approve/review gate action is signed; unsigned or bad-sig gate events are rejected by the appservice watcher and by consuming agents.
- E2. Token/key rotation runbooks executed at least once in anger (rotate `as_token`, rotate a per-agent key).
- E3. Security review complete: homeserver hardening flags, token custody, federation TLS/whitelist all verified.
- E4. **Parity checklist (§9) fully green + soaked.**
- E5. Hermes MCP bridge retired as backbone (optionally re-attached as an edge bridge only).
---
## 11. Open questions for Jason / Mos (need a human/coordinator ruling)
1. **DNS / domains per site.** What domain(s) do we own and want to use per site for homeserver names and `.well-known` (e.g. `site-a.mosaicstack.dev`)? Federation identity is permanent-ish once agents mint MXIDs against it — this needs a ruling **before P1 hardens** because MXIDs bake in the domain.
2. **Secret-management choice.** Is it Vault, or whatever KBN-101 lands? The appservice `hs_token`/`as_token` and enroll bootstrap secret custody depend on this (§8). CLAUDE.md signals secrets work is in flight — need the authoritative target.
3. **N-minute escalation threshold.** What is `dark_threshold_min` for a coordinator, and the second threshold before HIL is paged (§5)? Default proposed: 10 min → fallback, +5 min → HIL. Jason/Mos to confirm per role.
4. **HIL client: Element vs custom.** Do humans use off-the-shelf **Element** (fast, free, P1-ready) or do we invest in a custom HIL client? Proposed: **Element for P1–P4**, revisit custom only if HIL ergonomics demand it.
5. **Ephemeral vs persistent agent keys.** For signed-authorship, do we mint Ed25519 keys per-spin (simplest custody, no long-term private key at rest) or issue durable per-agent keys (stable identity across spins, but key-at-rest custody problem)? Proposed: **per-spin**, pubkey retained for audit.
6. **Federation topology / trust.** Full mesh between all sites, or hub-and-spoke-with-redundancy? Full mesh maximizes no-SPOF but grows O(n²); needs a call once site count is known.
7. **Fallback-coordinator assignment authority.** Who assigns each Team Lead's fallback coordinator, and is it always cross-site? (Design invariant §5 wants the fallback off the monitored host — confirm this is acceptable operationally.)
8. **Retention / compliance.** How long do we retain the Matrix event log (the unified audit trail)? Affects Synapse storage sizing and any purge policy.
---
## Appendix A — Real Matrix concepts this RFC leans on (quick reference)
- **Application Service (AS) API** — a privileged service registered with the homeserver via a registration YAML declaring `id`, `url`, `as_token`, `hs_token`, and `namespaces` (users/aliases/rooms regexes). The homeserver pushes events to the AS in transactions; the AS can register/act-as users in its namespace. _(This is our appservice backbone.)_
- **`hs_token` / `as_token`** — `hs_token`: homeserver→AS authentication on pushed transactions; `as_token`: AS→homeserver authentication, grants acting-as any namespaced user. Both are high-value secrets (§8).
- **Masquerade (`user_id` query param)** — the AS may act as a namespaced user on C-S calls using `as_token` + `?user_id=`. We prefer per-agent tokens for blast-radius; masquerade for AS-internal bulk ops.
- **Custom event types** — timeline events via `m.room.message` with a custom `msgtype` (keeps a human-visible `body` fallback) and/or fully custom `type` (dotted, e.g. `mosaic.workflow.step`); **state events** for last-writer-wins current status readable from room state without history replay.
- **Presence EDUs** — native online/unavailable/offline signals; coarse and lossy over federation, so MACP adds an explicit heartbeat event as the authoritative liveness source (§4.5).
- **Federation (S2S API)** — server-to-server over TLS; discovery via `/.well-known/matrix/server` and/or `_matrix._tcp` SRV; restrictable with `federation_domain_whitelist`.
- **Synapse config knobs cited** — `enable_registration`, `federation_domain_whitelist`, appservice registration file, rate-limiting, admin API binding. **[VERIFY]** exact flags/paths against the deployed Synapse version at implementation time.
_All Matrix mechanics above are cited from architecture knowledge and MUST be re-verified against the actual deployed Synapse version during P1 — every **[VERIFY]** in this document is a checkpoint, not an assumption._
docs/archive/planning/matrix-macp/rfc-002-install-config-topology.md
+622
View File
@@ -0,0 +1,622 @@
# RFC-002 — Install, Configuration & Topology for the Mosaic Matrix/MACP Comms System
- **Status:** DRAFT — for Team Lead → Orchestrator staffing
- **Author:** MS-LEAD (reviewer identity `ms-lead-reviewer`)
- **Sponsor / veto:** Jason (human lead)
- **Date:** 2026-07-24
- **Program:** Mosaic Stack comms-evolution
- **Companion to:** RFC-001 — _MACP: A Mosaic-Native, Matrix-Native Comms Layer_. RFC-001 is the architecture (self-hosted Synapse + Mosaic appservice backbone + `packages/comms` SDK + MACP standard + per-site federation). **RFC-002 is the config substrate the whole thing installs and runs on.**
- **Audience:** Team Leads, the Mosaic orchestrator, infra, harness maintainers, and — critically — **strangers who install this open-source product on hardware we will never see.**
> This is a **design document**. No code ships from this RFC. It is written to be decomposed into missions with per-phase acceptance criteria, and it slots under RFC-001's P1→P5. Every uncertain or must-live-validate claim is flagged **[VERIFY]**.
> **The one framing that governs every decision below:** this is an **open-source product**. Someone we have never met will `git clone` it and run it on their own domains, their own DNS, their own certs, their own hardware. **NOTHING may hardcode our fleet's topology.** There is no `woltje.com` in the code, no assumption that DNS exists, no assumption that a second site exists. Every topology fact is **user-supplied config**. Where this doc uses `mosaic.woltje.com` / `mosaic.uscllc.com`, those are **illustrative operator values** (Jason's real installs), never defaults and never literals in the product.
---
## 0. TL;DR
The comms system installs against a **user-supplied topology**, never a baked-in one. At install the operator declares exactly one of **three topology modes**: **(A) split-domain** (identity `server_name` ≠ homeserver host, wired via Matrix delegation — this is Jason's `mosaic.woltje.com` identity + `matrix.woltje.com` host setup), **(B) single-domain** (`server_name` == homeserver host), or **(C) IP-only standalone** (no DNS, no federation, fully supported for local/airgapped). **The PRIMARY/home instance is ALWAYS configured; federation is OPTIONAL.** A single standalone instance MUST work with zero federation.
**Federation is a hard-gated capability: it REQUIRES DNS + valid certificates. IP-only federation is not possible and is not supported.** IP-only means standalone-only, forever, until the operator acquires DNS + certs.
Certificates are **one ACME integration** with a user-chosen **directory URL**: either **step-ca** (self-hosted private ACME CA, for total control and private/internal domains public CAs can't issue for) or **Let's Encrypt** (public ACME, ease-of-use). The operator also picks a challenge type (HTTP-01 / DNS-01 / TLS-ALPN-01); **DNS-01 is the answer for private/split-horizon domains.**
Secrets go through a **pluggable `SecretBackend` interface** — no forced paid dependency. Ships with a **Vault** implementation and a **Vaultwarden** implementation; the operator picks at install. The Vaultwarden model (org + orchestrator enrolled as authority + per-agent scoped access) is designed-for, with an honest **[VERIFY]** on how far Vaultwarden's machine-account coverage has matured.
Config is **DB-backed with sane defaults and install-time overrides**. Precedence: **install-time → DB override → default.** Config is split into **install-time-immutable** (e.g. `server_name`, which is baked into every MXID and cannot change without re-homing every identity) and **runtime-tunable** (e.g. dark-threshold).
RFC-002 is the substrate; **RFC-001's P1 (presence) needs only Mode A/B single-instance clean-domain and does NOT require federation, IP-only, or the secret-backend rotation story resolved.**
---
## 1. Goals / Non-Goals
### 1.1 Goals
- **G1 — Installable by a stranger.** A person with no relationship to our fleet can install, configure, and run the comms system from published artifacts and a guided installer, on their own hardware and domains, with no edits to product code.
- **G2 — Zero hardcoded topology.** Every topology fact — `server_name`, homeserver host/IP, delegation method, federation peers, cert mode, secret backend — is **user-supplied config**, validated at install, stored in the product DB. No fleet-specific literal ships in the product.
- **G3 — Standalone MUST work.** The PRIMARY instance is always fully functional with **zero federation**, including with **no DNS at all** (Mode C, IP-only). Presence, rooms, MACP, HIL-via-Element all work single-instance.
- **G4 — Federation is optional but honestly gated.** Federation is opt-in and, when opted into, **requires DNS + valid certificates as a hard precondition.** The installer must refuse to _claim_ federation is working when the DNS/cert preconditions aren't met.
- **G5 — One ACME integration, two CA choices.** Build a single ACME cert-provisioning path; the operator selects step-ca or Let's Encrypt by supplying an **ACME directory URL** plus a challenge type. No second, bespoke cert path.
- **G6 — No forced paid dependency for secrets.** A pluggable `SecretBackend` with at least Vault and Vaultwarden implementations, chosen at install. Open-source ethos: the free/self-hostable path must be first-class.
- **G7 — Defaults that just work, overrides where they matter.** DB-backed config with sane defaults so most operators change little; install-time overrides for the topology-critical values; a clear immutable-vs-tunable boundary so operators can't foot-gun `server_name`.
- **G8 — A clean upgrade path.** An operator who starts standalone can later turn on federation with a documented, honest procedure (including the real cost if they started IP-only and must now acquire a stable `server_name`).
### 1.2 Non-Goals
- **NG1 — Not hosting a managed service.** This RFC is about _self-install_. We are not building multi-tenant SaaS provisioning; each operator runs their own instance(s).
- **NG2 — Not a new cert stack.** We do not write our own CA, our own ACME client protocol, or a non-ACME cert path. We integrate ACME and let the operator point it at step-ca or Let's Encrypt. (We _may_ bundle/recommend step-ca as the self-hosted CA, but via its standard ACME provisioner, not a fork.)
- **NG3 — Not a new secret manager.** We define an interface and ship adapters. We do not build a secret store; we do not force one.
- **NG4 — Not public-network Matrix federation.** Consistent with RFC-001 NG5: federation is Mosaic-site-to-Mosaic-site over infrastructure the operator controls, allowlisted. No `matrix.org` federation.
- **NG5 — Not making IP-only federate.** We will not ship a hack (self-signed S2S trust bundles, `/etc/hosts` federation) that pretends IP-only can federate. IP-only is standalone. This is a deliberate, honest boundary (§2.4, §7).
- **NG6 — Not re-homing identities silently.** We will not offer a "just change your `server_name`" button that quietly orphans every MXID. Any path that changes `server_name` is a flagged, gated, documented identity re-home (§5.3, §7).
---
## 2. The topology model
### 2.1 The core split: `server_name` vs homeserver host
Matrix has exactly the split Jason described, natively:
- **`server_name`** — the Synapse config value that is the server's **identity domain**. It is the part after the colon in every MXID (`@mosaic_agent:mosaic.woltje.com`) and every room alias (`#mission-KBN-101:mosaic.woltje.com`). It is **baked into every identity the moment that identity is minted.** Changing it re-homes everything. This is `server_name` in Synapse's `homeserver.yaml`.
- **Homeserver host** — the actual network location (hostname:port or IP:port) where the Synapse process answers federation and (optionally proxied) client traffic. It **can differ** from `server_name`. Matrix reconciles the difference through **delegation**: `https://<server_name>/.well-known/matrix/server` returning `{"m.server": "matrix.woltje.com:443"}`, and/or a `_matrix._tcp.<server_name>` **SRV** record. **[VERIFY]** `.well-known` vs SRV precedence on the deployed Synapse version (RFC-001 §6 flags the same).
So Jason's "mosaic._ app-domain + matrix._ homeserver-domain" split maps precisely: **`server_name = mosaic.woltje.com` (identity, in MXIDs), homeserver runs at `matrix.woltje.com` (discovered via delegation).** That is **Mode A**.
### 2.2 The topology config schema
One canonical config object, stored in the product DB (§5), populated at install (§6). Illustrative shape (field names decomposition-ready, not frozen):
```jsonc
{
"topology": {
"mode": "split-domain | single-domain | ip-only-standalone", // A | B | C — install-time-immutable
"identity": {
"server_name": "mosaic.woltje.com", // INSTALL-TIME-IMMUTABLE. In MXIDs. Never change without re-home (§5.3).
"server_name_kind": "domain | ip", // "ip" only legal in Mode C
},
"homeserver": {
"host": "matrix.woltje.com", // where Synapse actually listens (Mode A: differs from server_name)
"port": 8448, // federation port (default 8448) or 443 if proxied — [VERIFY] per deploy
"client_bind": "https://matrix.woltje.com", // C-S API public URL (proxied)
"bind_ip": null, // Mode C: e.g. "192.168.1.50" ; Modes A/B: null (DNS-resolved)
},
"delegation": {
"method": "well-known | srv | none", // Mode A: well-known or srv ; Mode B/C: none
"well_known_server": { "m.server": "matrix.woltje.com:443" }, // if method=well-known
"srv_record": "_matrix._tcp.mosaic.woltje.com. 3600 IN SRV 10 0 443 matrix.woltje.com.", // if method=srv (documented, operator provisions)
},
"federation": {
"enabled": true, // OPTIONAL. Mode C forces false.
"domain_whitelist": [
// Synapse federation_domain_whitelist — allowlist ONLY
"mosaic.woltje.com",
"mosaic.uscllc.com",
],
"peers": [
// operator-declared peer sites (for room/fleet wiring)
{ "server_name": "mosaic.uscllc.com", "role": "secondary", "fleet_room": true },
],
},
"tls": {
"acme": {
"directory_url": "https://acme.mosaic.woltje.com/acme/acme/directory", // step-ca OR https://acme-v02.api.letsencrypt.org/directory
"ca_kind": "step-ca | letsencrypt", // informational label; the directory_url is the real switch
"challenge": "dns-01 | http-01 | tls-alpn-01",
"account_email": "[email protected]", // ACME account contact
"eab": { "kid": null, "hmac_key_ref": null }, // External Account Binding if the CA requires it (some step-ca provisioners) — secret via SecretBackend
},
"client_tls_mode": "acme | self-signed", // Mode C may use self-signed for local C-S TLS (weaker trust, §8)
},
"secrets": {
"backend": "vault | vaultwarden", // pluggable, install-time choice (§4)
"connection": {
"address": "https://vault.woltje.com:8200", // or Vaultwarden/Bitwarden server URL
"auth_ref": "…", // how the appservice authenticates to the backend (bootstrap, §4/§8)
"namespace_or_org": "mosaic-fleet", // Vault namespace / mount, OR Vaultwarden org id
},
},
},
}
```
### 2.3 The three modes, concretely
Exactly three supported modes (Jason's ruling — no others):
**Mode A — split-domain (identity ≠ host, delegated).** _Jason's PRIMARY._ Federation-capable. This is the recommended production shape because it lets identity live on a clean app-domain while the homeserver runs on a separate operational host.
```jsonc
// Mode A — mosaic.woltje.com identity, matrix.woltje.com host, federated with a second site
{
"topology": {
"mode": "split-domain",
"identity": { "server_name": "mosaic.woltje.com", "server_name_kind": "domain" },
"homeserver": {
"host": "matrix.woltje.com",
"port": 443,
"client_bind": "https://matrix.woltje.com",
"bind_ip": null,
},
"delegation": {
"method": "well-known",
"well_known_server": { "m.server": "matrix.woltje.com:443" },
},
"federation": {
"enabled": true,
"domain_whitelist": ["mosaic.woltje.com", "mosaic.uscllc.com"],
"peers": [{ "server_name": "mosaic.uscllc.com", "role": "secondary", "fleet_room": true }],
},
"tls": {
"acme": {
"directory_url": "https://acme-v02.api.letsencrypt.org/directory", // public LE, or a step-ca directory
"ca_kind": "letsencrypt",
"challenge": "dns-01",
"account_email": "[email protected]",
},
"client_tls_mode": "acme",
},
"secrets": {
"backend": "vaultwarden",
"connection": { "address": "https://vw.woltje.com", "namespace_or_org": "mosaic-fleet" },
},
},
}
```
MXIDs on this instance: `@mosaic_coordinator-1:mosaic.woltje.com`. A human/agent's homeserver is discovered by resolving `.well-known/matrix/server` on `mosaic.woltje.com` → `matrix.woltje.com:443`.
**Mode B — single-domain (identity == host).** Simpler; the `server_name` _is_ the host. No delegation needed. Federation-capable (still needs DNS + cert on that one domain).
```jsonc
// Mode B — one domain does everything
{
"topology": {
"mode": "single-domain",
"identity": { "server_name": "matrix.example.org", "server_name_kind": "domain" },
"homeserver": {
"host": "matrix.example.org",
"port": 8448,
"client_bind": "https://matrix.example.org",
"bind_ip": null,
},
"delegation": { "method": "none" },
"federation": { "enabled": false, "domain_whitelist": [], "peers": [] }, // optional — off here
"tls": {
"acme": {
"directory_url": "https://acme-v02.api.letsencrypt.org/directory",
"ca_kind": "letsencrypt",
"challenge": "http-01",
"account_email": "[email protected]",
},
"client_tls_mode": "acme",
},
"secrets": {
"backend": "vault",
"connection": { "address": "https://vault.example.org:8200", "namespace_or_org": "mosaic" },
},
},
}
```
MXIDs: `@mosaic_coordinator-1:matrix.example.org`. Here `server_name == host`, so `@:matrix.example.org` is both the identity domain and where Synapse actually answers.
**Mode C — IP-only standalone (no DNS, no federation).** Fully supported for local/airgapped/homelab-without-DNS. `server_name` is an IP:port literal. **Cannot federate — ever — in this mode** (federation requires a real domain + valid CA cert; §2.4). Client TLS may be self-signed (weaker trust, §8).
```jsonc
// Mode C — airgapped / local, no DNS, standalone only
{
"topology": {
"mode": "ip-only-standalone",
"identity": { "server_name": "192.168.1.50:8448", "server_name_kind": "ip" },
"homeserver": {
"host": "192.168.1.50",
"port": 8448,
"client_bind": "https://192.168.1.50:8448",
"bind_ip": "192.168.1.50",
},
"delegation": { "method": "none" },
"federation": { "enabled": false, "domain_whitelist": [], "peers": [] }, // FORCED false in Mode C
"tls": { "acme": null, "client_tls_mode": "self-signed" }, // may use a private step-ca or self-signed for C-S TLS
"secrets": {
"backend": "vaultwarden",
"connection": { "address": "http://192.168.1.51:8080", "namespace_or_org": "mosaic-local" },
},
},
}
```
MXIDs: `@mosaic_coordinator-1:192.168.1.50:8448`. **Warning surfaced at install:** this `server_name` is an IP literal; if the operator ever wants federation they must move to a domain, which is an **identity re-home** (§5.3, §7).
> **[VERIFY]** Synapse accepts an `ip:port` `server_name` and mints usable MXIDs against it for local/standalone use. This is believed workable for non-federated operation but must be validated against the deployed Synapse version; some Synapse versions/tools assume a DNS-resolvable `server_name`. If an IP literal is rejected, Mode C falls back to a **fabricated local domain** (e.g. `mosaic.local`) resolved via `/etc/hosts` or a local resolver — still standalone-only, same re-home caveat.
### 2.4 The hard federation gate (Jason's HARD STOP)
**Federation REQUIRES DNS + valid certificates. This is a hard stop, enforced by the installer and by the config validator, not a suggestion.**
| Precondition | Why | Enforced where |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `server_name` resolves in public/peer DNS (or delegated target does) | S2S discovery uses `.well-known`/SRV over DNS; peers must resolve you | installer reachability check (§6); config validator rejects `federation.enabled=true` with `server_name_kind=ip` |
| Valid TLS cert on the federation endpoint, chained to a CA the peer trusts | S2S is TLS; a peer validates your cert. Self-signed/untrusted ⇒ peer refuses ⇒ you are defederated | installer cert probe (§6); cert monitor (§3.4) |
| Federation `domain_whitelist` non-empty and mutually consistent with peers | allowlist-only federation (RFC-001 NG5/§6) | config validator |
**IP-only ⇒ federation is impossible.** There is no valid public/peer CA cert for a bare IP in our trust model (and we will not ship a self-signed S2S trust hack — NG5). Therefore **Mode C is standalone-only by construction**, and the config validator makes `mode=ip-only-standalone ∧ federation.enabled=true` an **illegal state that cannot be persisted.**
This is the honest, load-bearing boundary of the whole topology model:
```
DNS + valid cert?
┌─────────────┴─────────────┐
YES NO
│ │
Mode A or B Mode C (IP-only)
federation OPTIONAL STANDALONE ONLY
(opt-in, allowlisted) (federation impossible)
```
---
## 3. Certificate provisioning — one ACME integration
### 3.1 Single integration, directory-URL as the switch
We build **exactly one** cert-provisioning integration: an **ACME client integration**. Both supported CAs are ACME CAs. The operator does **not** choose between two code paths; they choose an **ACME directory URL** and a **challenge type**. That is the entire surface.
| CA choice | What it is | ACME directory URL (illustrative) | Why an operator picks it |
| ----------------------- | ------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **step-ca (Smallstep)** | self-hosted **private** ACME CA | `https://acme.<internal-domain>/acme/<provisioner>/directory` | **Total control**; issues for **private/internal/split-horizon domains** a public CA can't (e.g. `mosaic.internal`, RFC-1918 split-horizon); airgap-friendly; you own the root |
| **Let's Encrypt** | public ACME CA | `https://acme-v02.api.letsencrypt.org/directory` (staging: `.../acme-staging-v02...`) | **Ease of use**; universally trusted chain (ISRG Root X1); zero CA to operate; ideal for public domains |
Because both speak ACME, the same client (account key, order, authorization, challenge, finalize, cert-fetch, renew) drives either. The `ca_kind` label in config is informational for UX; the **`directory_url` is the real determinant**. **[VERIFY]** whether the chosen ACME library requires per-CA quirks (LE rate limits, staging switch; step-ca **External Account Binding** on some provisioners — if EAB is required the `kid`/`hmac_key` come from the SecretBackend, §4).
### 3.2 Challenge-type matrix (which challenge for which topology)
The operator picks one challenge type per the domains they're covering. This is the crux for **public vs private/split-horizon**:
| Challenge | How it proves control | Best for | Cannot / caveat |
| --------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **HTTP-01** | CA GETs `http://<domain>/.well-known/acme-challenge/<token>` on port 80 | **Public, single hostname**, port 80 reachable from CA (Mode B, or Mode A's homeserver host) | Needs inbound :80 from the CA; **cannot** do wildcards; **useless for private domains** a public CA can't reach |
| **DNS-01** | CA checks a `_acme-challenge.<domain>` **TXT** record you publish | **Private / split-horizon / internal domains**, **wildcards**, and any domain where inbound HTTP from the CA is impossible | Requires **programmatic DNS API** access to publish TXT (or manual for step-ca where you own the resolver). **This is the answer for step-ca on private domains** and for Mode A when the homeserver host isn't publicly HTTP-reachable |
| **TLS-ALPN-01** | CA connects TLS on :443 with ALPN `acme-tls/1` | Public host where **:443 is free** but :80 is blocked | Needs the ACME client to own the :443 TLS handshake briefly; awkward behind some reverse proxies — **[VERIFY]** against our proxy (RFC-001 terminates TLS at a reverse proxy) |
**Guidance baked into the installer:**
- **Let's Encrypt + public domain, port 80 open →** HTTP-01 (simplest). Wildcard or no inbound :80 → DNS-01.
- **step-ca + private/internal/split-horizon domain →** **DNS-01** (the private CA can validate against a resolver you control; public HTTP reachability is irrelevant). This is the combination that lets a private homelab domain get real certs.
- **:443-only public host →** TLS-ALPN-01.
### 3.3 Delegation setup for split-domain (Mode A)
For Mode A, the cert and the delegation must agree. Concretely, at install for `server_name=mosaic.woltje.com`, host `matrix.woltje.com`:
1. **Cert(s):** obtain a valid cert for **`matrix.woltje.com`** (the federation/host endpoint — this is where the TLS handshake actually lands). If serving `.well-known` over HTTPS on `mosaic.woltje.com`, that origin **also** needs a valid cert for `mosaic.woltje.com`. So Mode A typically provisions **two** SANs/certs: identity-domain (`mosaic.woltje.com`, serves `.well-known`) and host (`matrix.woltje.com`, serves S2S+C-S). **[VERIFY]** whether a single multi-SAN cert is preferable operationally.
2. **Delegation record**, one of:
- **`.well-known`:** serve `https://mosaic.woltje.com/.well-known/matrix/server` → `{"m.server":"matrix.woltje.com:443"}` (and `.well-known/matrix/client` for C-S discovery so agents/Element find the host).
- **SRV:** `_matrix._tcp.mosaic.woltje.com. IN SRV 10 0 443 matrix.woltje.com.` The installer **documents and validates** the record but the operator provisions it in their DNS (we don't run their DNS). **[VERIFY]** `.well-known` vs SRV precedence on the deployed Synapse.
3. **Validate:** installer fetches the operator's own `.well-known`/SRV and confirms it points at the configured host, and that the host presents a valid cert (§6). Only then does it declare Mode A "federation-ready."
### 3.4 Renewal & monitoring — a lapsed federation cert silently defederates
**This is the operational trap and it must alarm.** ACME certs are short-lived (LE = 90 days; step-ca often shorter by policy). A federation cert that lapses does **not** throw a loud error — peers simply **stop trusting the S2S handshake and the site silently drops out of federation.** From inside, everything looks fine; from peers, the site went dark. That is exactly the "homelab went dark and took comms with it" trauma (RFC-001 §5), but caused by a cert, not a host.
Requirements:
- **Auto-renew** on the standard ACME schedule (renew at ~⅓ lifetime remaining; LE guidance ~30 days before expiry). The ACME integration owns this loop.
- **Expiry monitoring as a first-class alarm.** Emit cert-days-remaining into OTEL/Jaeger metrics (consistent with RFC-001 §8's "monitor for cert expiry — a cert lapse silently defederates"). Alarm thresholds (e.g. warn <14d, critical <3d) are **runtime-tunable** config (§5).
- **Federation-health probe:** periodically resolve our own delegation and validate our own cert _as a peer would_ (external vantage where possible), so a broken renewal is caught as "we would fail a peer's validation" before a peer notices.
- **Escalation tie-in:** a critical cert-expiry or federation-health failure raises a `mosaic.escalation` (RFC-001 §4.2/§5) into the HIL room. A cert lapse is a fleet-visibility incident, not a silent config drift.
---
## 4. Secret backend interface
### 4.1 The `SecretBackend` contract
A single pluggable interface. The appservice and orchestrator depend on the **interface**, never on Vault or Vaultwarden directly. Chosen at install; swappable without touching callers. Illustrative contract (decomposition-ready, not frozen):
```ts
interface SecretBackend {
// --- static secret CRUD (appservice tokens, ACME EAB, DB creds) ---
get(ref: SecretRef): Promise<SecretValue>;
put(ref: SecretRef, value: SecretValue, opts?: { immutable?: boolean }): Promise<void>;
rotate(
ref: SecretRef,
next: SecretValue,
): Promise<{ previous: SecretVersion; current: SecretVersion }>;
list(prefix: SecretRef): Promise<SecretRef[]>;
delete(ref: SecretRef): Promise<void>;
// --- agent-credential lifecycle (the fleet-identity part) ---
enrollAgent(input: {
agentSlug: string;
scope: CredentialScope; // which rooms/secrets this agent may read
ttl?: Duration; // ephemeral-by-default per RFC-001 §8
}): Promise<AgentCredentialHandle>; // wraps the per-agent access_token + optional pubkey record
revokeAgent(agentSlug: string): Promise<void>; // must be authoritative & immediate
// --- health / bootstrap ---
health(): Promise<BackendHealth>;
authenticateSelf(bootstrap: BootstrapAuth): Promise<void>; // how the appservice/orchestrator logs into the backend
}
```
Design intent: **`get/put/rotate`** cover the static crown-jewel secrets (appservice `hs_token`/`as_token`, ACME account/EAB keys, DB DSN). **`enrollAgent/revokeAgent`** cover the _fleet-identity_ lifecycle — this is where RFC-001's "mint per-agent token at enroll, discard on teardown" (RFC-001 §4.1, §8) actually lands.
### 4.2 How appservice / agent tokens map onto it
RFC-001 defines three tiers of Matrix secret. They map cleanly:
| RFC-001 secret | Sensitivity | `SecretBackend` treatment |
| ------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`as_token`** (AS→HS, acts as any namespaced user) | crown jewel | `put(immutable-ish)` + `rotate`; **only** the appservice may `get` it; stored under a fleet-admin scope; never handed to an agent |
| **`hs_token`** (HS→AS callback auth) | crown jewel | same as `as_token`; both live only in appservice scope (RFC-001 §8) |
| **enroll bootstrap secret / orchestrator-signed nonce** | high | `get` by orchestrator + appservice only; used to authenticate `POST /enroll` so a rogue process can't enroll a rogue agent (RFC-001 §8, B5) |
| **per-agent `access_token`** | transient runtime | minted via `enrollAgent`, scoped to that agent, **short-TTL / re-mintable**, discarded on teardown; **not** durably persisted (RFC-001 §8 says per-agent tokens are transient) — the backend may hold a short-lived handle or a personal-vault entry for the agent's own lifetime |
| **Ed25519 signed-authorship keypair** | high (private key) | private key generated **agent-side**, only pubkey leaves the agent (RFC-001 §4.4/§8); the SecretBackend stores the **pubkey record** for audit; per-spin keys need no at-rest custody |
The key blast-radius property (RFC-001 §8) is preserved: agents receive **only their own** credential via `enrollAgent`; the `as_token` never leaves appservice scope.
### 4.3 Vault implementation
Vault maps naturally:
- Static secrets → **KV v2** at a mount/namespace (`mosaic-fleet/`), with versioning giving `rotate` semantics for free.
- **`enrollAgent`** → issue a scoped, TTL'd token or use **AppRole** / a scoped policy per agent; Vault's native TTL + revocation is exactly the transient per-agent model. `revokeAgent` → Vault token/lease revoke (authoritative, immediate).
- **`authenticateSelf`** → the appservice authenticates to Vault via AppRole (role_id from config, secret_id injected at deploy) or a platform auth method; consistent with how Gateway/DB secrets are handled today (RFC-001 §8). **[VERIFY]** align with whatever KBN-101 lands for Mosaic secret management (CLAUDE.md flags secrets work in flight).
- **Trade-off (honesty):** Vault is the most capable backend but is **not** the free-and-simple default for a hobbyist stranger; hence it must not be _forced_ (G6).
### 4.4 Vaultwarden implementation + the org/enroll/revoke agent-account model
Vaultwarden (self-hostable Bitwarden-compatible server) is the **open-source-ethos default candidate** — free, self-hostable, familiar. Jason's model, mapped onto Bitwarden/Vaultwarden's org primitives:
1. **Operator creates one or more Bitwarden orgs** at install (e.g. `mosaic-fleet`).
2. **The orchestrator is enrolled into the org and granted authority** to enroll/revoke agent sessions — it is the org's automation principal (admin/manager over an agents **collection**).
3. **Agents get scoped credential access:** each agent gets access to a **collection** (or a personal vault provisioned for its spin) holding exactly the secrets its scope allows. `enrollAgent` = grant the agent principal access to its collection + provision its per-agent Matrix token entry; `revokeAgent` = remove the agent principal / revoke its access, immediately.
4. **User + agents share scoped access:** the human operator and the agents both hold credentials in the same org, scoped by collection — humans and agents on one secret surface, mirroring RFC-001's "humans and agents on one comms surface" pattern.
Mapping to the interface:
| Interface op | Vaultwarden mechanism |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `get/put/rotate` (static) | items in an org **collection**; rotate = new item version / replace + old-version audit |
| `enrollAgent` | create/attach agent principal to its **collection**; provision per-agent token item, TTL enforced by our teardown (Vaultwarden itself is not TTL-native — see [VERIFY]) |
| `revokeAgent` | revoke the agent principal's org membership / collection access |
| `authenticateSelf` | orchestrator authenticates as the org automation principal |
**HONEST MATURITY FLAG — [VERIFY]:** Bitwarden's clean **machine-account / service-account** primitive lives in **Bitwarden Secrets Manager**, and **Vaultwarden's coverage of Secrets Manager / machine accounts is partial and evolving.** What is known to work today on Vaultwarden: **orgs, collections, per-user (incl. a per-agent "user") vaults, and collection-scoped sharing.** What **may not** be fully there: the polished **machine-account API**, native short-TTL service credentials, and fine-grained programmatic access-token issuance equivalent to hosted Bitwarden Secrets Manager. **[VERIFY]** current Vaultwarden version's Secrets Manager / machine-account support before P-level commitment.
**Why this is not a blocker:** the interface is designed so **either backend is viable**. If Vaultwarden's machine-account API isn't ready, the Vaultwarden adapter implements `enrollAgent` via the **personal-vault-per-agent + org-collection** model that works _today_ (create an agent principal, share the scoped collection, we enforce TTL via orchestrator teardown rather than backend-native TTL). If an operator needs backend-native short-TTL machine credentials now, they choose the **Vault** adapter. **We are not blocked on Vaultwarden maturing**, because the `SecretBackend` abstraction lets the polished-machine-account behavior land later without changing any caller.
---
## 5. Config system
### 5.1 Storage & precedence
Config is **DB-backed** (Postgres, per the stack — CLAUDE.md/RFC-001), with sane defaults compiled into the product and install-time overrides. **Precedence, highest wins:**
```
install-time value > DB override (runtime) > compiled default
```
- **Compiled default** — ships in the product; what a stranger gets with zero config for every non-topology-critical key.
- **Install-time value** — captured by the installer (§6), written to DB, and for **install-time-immutable** keys, **locked** (marked non-overridable).
- **DB override** — runtime tuning via admin surface, allowed **only** for keys classified runtime-tunable.
> Nuance: "install-time > DB override" applies to **immutable** keys — the install-time value is frozen and a DB override of it is rejected. For **tunable** keys, the DB override is the live value and the install-time value is just the initial seed. The classification (§5.3) is what makes the precedence unambiguous per key.
### 5.2 DB schema shape
Illustrative (Drizzle/Postgres, per stack conventions):
```sql
-- one row per config key
CREATE TABLE comms_config (
key text PRIMARY KEY, -- e.g. 'topology.identity.server_name'
value jsonb NOT NULL, -- current effective value
source text NOT NULL, -- 'install' | 'db-override' | 'default'
mutability text NOT NULL, -- 'install-immutable' | 'runtime-tunable'
set_by text, -- operator/agent/system that set it
set_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT immutable_not_overridable
CHECK (NOT (mutability = 'install-immutable' AND source = 'db-override'))
);
-- append-only audit of every change (esp. attempted immutable changes)
CREATE TABLE comms_config_audit (
id bigserial PRIMARY KEY,
key text NOT NULL,
old_value jsonb,
new_value jsonb,
actor text NOT NULL,
action text NOT NULL, -- 'set' | 'override' | 'rejected-immutable'
at timestamptz NOT NULL DEFAULT now()
);
```
The DB `CHECK` is a belt-and-braces backstop; the application-layer config service enforces mutability and records rejected immutable-change attempts in the audit table. Secrets are **referenced** here (a `SecretRef`), never stored inline — actual secret values live in the `SecretBackend` (§4).
### 5.3 Install-time-immutable vs runtime-tunable — the key table
The single most important classification: **what can never change after install** vs **what an operator tunes anytime.** Getting `server_name` on the wrong side of this line is a foot-gun that orphans every identity.
| Config key | Mutability | Rationale / cost of change |
| ------------------------------------------------ | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `topology.mode` (A/B/C) | **install-immutable** | changing mode changes identity/federation semantics; a mode change is effectively a reinstall/re-home |
| `topology.identity.server_name` | **install-immutable** | **baked into every MXID and room alias.** Changing it re-homes every identity — see re-home note below. **This is THE immutable value.** |
| `topology.identity.server_name_kind` | **install-immutable** | domain↔ip change is a re-home (Mode C→A/B) |
| `topology.homeserver.host` | install-immutable (**delegation-tunable**) | in Mode A you _can_ move the host if you update delegation to match — the identity is unchanged; treat as immutable-with-migration, not free |
| `topology.homeserver.bind_ip` / `port` | runtime-tunable (ops) | operational network binding; no identity impact |
| `topology.delegation.method` / records | tunable-with-care | can switch well-known↔SRV as long as both still resolve to the same host; validated on change |
| `topology.federation.enabled` | **runtime-tunable (gated)** | can flip **on** only if DNS+cert preconditions pass (§2.4); flipping standalone→federated is the §7 upgrade path |
| `topology.federation.domain_whitelist` / `peers` | runtime-tunable | add/remove peers over time; each add re-validated |
| `tls.acme.directory_url` / `ca_kind` | runtime-tunable | can switch CA (e.g. LE→step-ca); triggers re-issue; monitor for trust-chain change |
| `tls.acme.challenge` | runtime-tunable | switch challenge type if DNS/HTTP reachability changes |
| `tls.client_tls_mode` | tunable-with-care | self-signed→acme is fine; acme→self-signed weakens trust (§8) |
| `secrets.backend` | install-immutable (**migration-only**) | switching Vault↔Vaultwarden requires a secret migration; not a live flip |
| `secrets.connection.*` | runtime-tunable | rotate backend address/auth without changing which backend |
| `presence.heartbeat_interval_ms` | runtime-tunable | RFC-001 §4.5 default ~30s; pure tuning |
| `presence.miss_tolerance` | runtime-tunable | RFC-001 §4.5 default 2 |
| `escalation.dark_threshold_min` | runtime-tunable | RFC-001 §5/§11 — default 10min → fallback |
| `escalation.hil_threshold_min` | runtime-tunable | RFC-001 §5 — default +5min → HIL |
| `cert.expiry_warn_days` / `expiry_critical_days` | runtime-tunable | §3.4 alarm thresholds (default 14 / 3) |
| `federation.health_probe_interval` | runtime-tunable | §3.4 |
**Identity re-home note (the cost of changing `server_name`):** because every MXID (`@mosaic_*:server_name`) and room alias (`#…:server_name`) embeds `server_name`, changing it means: every agent gets a **new identity**, all rooms must be **recreated/re-aliased**, signed-authorship pubkey records re-published, and federation peers re-pointed. There is **no in-place rename** in Matrix. Hence `server_name` is install-immutable and the installer gates it behind an explicit warning (§6.7). Changing it is a **migration/reinstall**, honestly (§7).
### 5.4 How the appservice / homeserver read config
- **Synapse (homeserver)** reads a _rendered_ `homeserver.yaml`. The config service **renders** Synapse's config (server_name, listeners, `federation_domain_whitelist`, `enable_registration: false`, appservice registration path, TLS/delegation) from the DB-backed config at deploy/reconfigure time. Synapse itself is not DB-config-aware; the source of truth is the product DB, and Synapse config is a **generated artifact**. Certain Synapse values (notably `server_name`) require a **Synapse restart** and are exactly the immutable ones — reinforcing §5.3.
- **The appservice** reads config **live** from the DB config service for runtime-tunable values (thresholds, whitelist changes, cert alarm thresholds) and from the `SecretBackend` for secrets. Immutable topology values are read once at boot (they can't change under it).
- **`packages/comms` SDK** receives the values it needs (homeserver client URL, presence intervals) from the appservice at enroll (RFC-001 §4.1 returns `{mxid, access_token, homeserver, rooms[]}`), so agents never read the config DB directly.
- **Delegation artifacts** (`.well-known/matrix/server`, `.well-known/matrix/client`) are likewise **rendered** from config and served by the reverse proxy / homeserver.
---
## 6. Installer UX flow
A guided installer (`mosaic comms install` or equivalent) that captures topology, provisions certs, wires secrets, and **validates before declaring success.** It must never report success it hasn't proven. Steps:
**6.1 — Preflight & detection.** Detect existing DNS records for a candidate domain, existing certs, an existing reachable Synapse, an existing Vault/Vaultwarden. Offer detected values as suggestions (never as silent defaults). Detect whether the host has public inbound :80/:443 (informs challenge-type guidance, §3.2).
**6.2 — Primary instance (ALWAYS).** The PRIMARY/home instance is always configured — there is no "skip primary." Prompt for its identity. This is non-optional and is what makes standalone work out of the box.
**6.3 — Pick topology mode (A/B/C).** Ask the shape:
- Do you have a domain? **No →** Mode C (IP-only standalone); warn federation is impossible here (§2.4) and that `server_name` will be an IP (re-home cost if they later want federation).
- Yes, and identity domain differs from the homeserver host? **Yes →** Mode A (split-domain); capture `server_name` + host + delegation method.
- Yes, one domain does everything? **→** Mode B (single-domain).
**6.4 — Pick CA (ACME directory).** step-ca vs Let's Encrypt → capture `directory_url`, account email, and challenge type with the §3.2 guidance surfaced (e.g. "private/internal domain? → DNS-01"). Capture EAB if the CA requires it (→ SecretBackend). For step-ca, offer to point at an existing step-ca or document standing one up.
**6.5 — Pick secret backend.** Vault vs Vaultwarden → capture connection (address, org/namespace, bootstrap auth). If Vaultwarden, walk the org/collection setup (§4.4) and **surface the machine-account maturity [VERIFY]** honestly so the operator chooses eyes-open.
**6.6 — Federation (OPTIONAL).** Only offered if Mode A/B. Ask whether to enable federation now; if yes, capture peer `server_name`s and build the `domain_whitelist`. If Mode C, federation is not offered (greyed out with the explanation). Federation-off is a first-class, fully-supported outcome.
**6.7 — The "what can't be changed later" warning gate.** Before writing immutable config, present an explicit confirmation:
```
⚠ IMMUTABLE CHOICES — read before confirming
server_name = "mosaic.woltje.com"
This becomes part of every agent identity (e.g. @mosaic_coordinator-1:mosaic.woltje.com)
and every room alias. It CANNOT be changed later without re-homing every identity
(new MXIDs for all agents, recreating all rooms). There is no in-place rename in Matrix.
topology.mode = "split-domain" — changing modes later is a reinstall.
secrets.backend = "vaultwarden" — switching backends later requires a secret migration.
Type the server_name to confirm you understand it is permanent: ____________
```
The operator must **retype `server_name`** to proceed — a deliberate friction gate on the one truly permanent value.
**6.8 — Provision & validate (no success claim until proven).** The installer then:
1. Renders Synapse config + delegation artifacts; brings up Synapse with `enable_registration: false`.
2. Runs the ACME flow; obtains cert(s); verifies they're valid and installed.
3. Authenticates to the SecretBackend; stores `hs_token`/`as_token`, enroll bootstrap; runs `health()`.
4. **Reachability & cert validation** appropriate to mode:
- Mode A: fetch our own `.well-known`/SRV, confirm it points at host; TLS-probe host cert as a peer would; confirm C-S discovery.
- Mode B: TLS-probe the single domain; confirm C-S + (if federation) S2S.
- Mode C: confirm local C-S reachability over the bind IP; confirm (self-signed or private-CA) client TLS; **explicitly report "standalone — federation not available."**
- If federation enabled: validate each peer resolves + presents a peer-trusted cert; confirm `domain_whitelist` mutual consistency. If any peer fails, **federation is reported NOT-ready** — the primary still succeeds standalone.
5. **Only now** declare success, with a per-capability report: `PRIMARY: ✅ | CERT: ✅ (expires in 90d, auto-renew on) | SECRETS: ✅ (vaultwarden) | FEDERATION: ✅ 1 peer / ⚠ not-ready / ⛔ n-a (Mode C)`.
**6.9 — Post-install.** Emit the cert-expiry monitor + federation-health probe (§3.4) into OTEL; write config to DB with correct mutability flags; print the immutable-values summary again for the record.
---
## 7. Standalone → federated upgrade path
An operator who started standalone later wants federation. The path depends on **whether they started with a domain**:
**Case 1 — started Mode A or B (had a domain), federation was just off.** _Cheap, no re-home._ `server_name` is already a real domain and identities are already minted against it. To federate:
1. Ensure DNS resolves for peers (their `server_name`s and yours) — likely already true.
2. Ensure a **valid, peer-trusted cert** on the federation endpoint (if they were running client-only self-signed, they now need a real ACME cert; if already ACME, done).
3. Set `federation.enabled = true`, populate `domain_whitelist` + `peers` (all runtime-tunable, §5.3).
4. Re-run the installer's **federation validation** (§6.8 step 4) against each peer. On green, federation is live. **No identity change** — existing MXIDs simply become reachable cross-site. This is the intended, low-friction upgrade.
**Case 2 — started Mode C (IP-only), now wants federation.** _Expensive — an identity re-home, and we say so plainly._ Federation requires DNS + a valid cert (§2.4), which an IP `server_name` can never satisfy. So the operator must:
1. **Acquire a domain** and DNS, and provision a **valid ACME cert** (LE public, or step-ca if the domain is private — but note a _private_ domain can only federate with peers who trust that private CA root, §8).
2. **Change `server_name` from the IP literal to the domain** — this is the **install-immutable value**, so this is a **re-home, not a config tweak**:
- Every agent identity `@mosaic_*:192.168.1.50:8448` becomes `@mosaic_*:newdomain` — **all new MXIDs.**
- Every room + alias must be **recreated** under the new `server_name`.
- Signed-authorship pubkey records re-published under the new identities.
- Any durable references to old MXIDs (escalation policies, fallback-coordinator targets, RFC-001 §5) must be re-pointed.
3. Effectively: **treat it as a fresh install in Mode A/B with a data migration of rooms/history**, not an in-place flip. Matrix has **no in-place `server_name` rename**; this cost is intrinsic to Matrix, not to our design.
**Honest guidance the installer gives Mode C operators up front (§6.3):** "If there is _any_ chance you'll want to federate later, start with a domain (Mode A/B) even if you keep federation off — flipping federation on later is free, but changing an IP `server_name` to a domain later is a full identity re-home." This lets an informed operator avoid the expensive path by choosing Mode B-with-federation-off instead of Mode C.
---
## 8. Security
**8.1 — Cert trust model per CA choice.**
- **Let's Encrypt (public):** chains to a universally-trusted root (ISRG). Peers, humans' browsers, and Element trust it with no extra distribution. Best for public domains; nothing to distribute.
- **step-ca (private):** chains to a **root you operate**. Nothing trusts it by default. Therefore the **step-ca root must be distributed** to everyone who validates certs: peer homeservers (so cross-site S2S validates — a peer must add your root to its federation trust store, **[VERIFY]** Synapse's mechanism for trusting a custom federation CA), agent hosts, and any human client. This is the price of "total control" and airgap capability. For **federation between two private-CA sites**, both sites must trust each other's roots (or a shared root). Getting this wrong reproduces the silent-defederation failure (§3.4) — a peer that doesn't trust your root silently refuses your S2S.
- **Mode C self-signed client TLS:** weakest — see 8.4.
**8.2 — Federation whitelist.** `federation_domain_whitelist` is a **hard allowlist** (RFC-001 §6/NG5): only listed Mosaic site domains may federate; no public-network federation. The installer/config validator keeps the whitelist consistent with the declared peer list. Adding a peer is an explicit, audited config change.
**8.3 — Secret-backend auth.** The appservice/orchestrator authenticate to the `SecretBackend` via a **bootstrap credential injected at deploy** (Vault AppRole secret_id, or Vaultwarden org automation principal), never committed, consistent with existing Gateway/DB secret handling (RFC-001 §8). The `as_token`/`hs_token` live **only** in backend + appservice memory; agents get only their own scoped, re-mintable token (§4.2). Enroll is authenticated (RFC-001 B5) so a rogue local process can't enroll a rogue agent. Backend access is scoped: an agent's credential can read only its collection/policy, never the fleet-admin scope holding the crown jewels.
**8.4 — Honest note: IP-only standalone with self-signed client TLS is a weaker-trust local mode.** In Mode C, client TLS may be self-signed (or a local private CA). This means: no third party vouches for the endpoint; clients must be told to trust the self-signed cert (TOFU or manual root import); there is no external validation of who's on the other end. This is **acceptable and supported for local/airgapped/homelab** use where the network is already trusted, but it is **explicitly a weaker trust posture** than a real CA. The installer states this plainly at install (§6.8 Mode C). It is one more reason Mode C cannot federate: we will not extend this weaker-trust local posture across sites (NG5).
**8.5 — Homeserver hardening** (inherited from RFC-001 §8, config-rendered here): `enable_registration: false` always (agents come only via the appservice), rate-limiting on, admin API bound to localhost/behind auth, media repo locked/disabled if unused, TLS terminated at our controlled proxy. These are **rendered from config** (§5.4) so a stranger gets them by default, not by remembering to set them. **[VERIFY]** current recommended Synapse hardening flags at implementation.
---
## 9. How RFC-002 integrates with RFC-001's P1–P5
RFC-002 is the **substrate**. Each RFC-001 phase consumes a subset of it. Critically, **P1 does not need the hard parts** — presence ships on a single clean-domain instance with no federation, no IP-only, and no secret-rotation story resolved.
| RFC-001 phase | RFC-002 pieces it NEEDS | RFC-002 pieces it does NOT need yet |
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **P1 — Presence (first shippable slice)** | **Mode A or B, single-instance, clean domain** (§2.3). One ACME cert (LE or step-ca) via the single integration (§3). Config system minimal: `server_name` immutable + presence thresholds tunable (§5.3). Installer path 6.1–6.3, 6.4 (cert), 6.7 (immutable gate), 6.8 (validate single-instance). A **minimal** SecretBackend just holding the appservice token. | **No federation** (§2.4 gate irrelevant — single site). **No IP-only** needed for P1 (P1 wants a clean domain so Element/humans connect cleanly). **No agent-credential rotation** maturity. **No peer/whitelist** config. Vaultwarden machine-account [VERIFY] does **not** block P1. |
| **P2 — Appservice + auto-enroll** | Full `SecretBackend` **`enrollAgent`/`revokeAgent`** (§4.1), `hs_token`/`as_token` custody (§4.2), enroll-bootstrap secret. Config: room taxonomy, per-agent token classification. Chosen backend (Vault or Vaultwarden) real. | Federation, cross-site, IP-only. |
| **P3 — MACP v1 spec** | Config keys for MACP versioning/thresholds as runtime-tunable (§5.3); nothing new topology-wise. | Federation, secret rotation-in-anger. |
| **P4 — Federation** | **The whole federation half of RFC-002:** Mode A delegation (§3.3), the hard DNS+cert gate (§2.4), `domain_whitelist`+peers config, cert monitoring/silent-defederation alarm (§3.4), per-CA trust distribution for cross-site (§8.1). This is where standalone→federated (§7 Case 1) and Jason's `mosaic.woltje.com`↔`mosaic.uscllc.com` shape land. | IP-only (federation excludes it by construction). |
| **P5 — Hardening + signed-authorship + Hermes retired** | Secret **rotation runbooks** executed in anger (§4, RFC-001 E2), pubkey-record custody for Ed25519 (§4.2), cert-rotation runbook (§3.4), full homeserver hardening validated (§8.5), backend auth review (§8.3). | — |
**One-line integration statement:** P1 rides on the _smallest_ slice of RFC-002 (single clean-domain instance + one cert + minimal config + minimal secret storage); the federation/IP-only/backend-maturity complexity is deferred to exactly the phases that need it (mostly P4/P5). RFC-002 therefore does not gate P1.
---
## 10. Open questions
Deliberately few — most topology/cert/secret decisions are resolved by Jason's rulings and baked in above.
1. **[VERIFY] IP-only `server_name` acceptance.** Does the deployed Synapse version accept an `ip:port` `server_name` and mint usable MXIDs for standalone (§2.3)? If not, Mode C uses a fabricated local domain (`mosaic.local` via local resolver) — confirm which, since it affects the re-home wording for Mode C→A/B (§7).
2. **[VERIFY] Vaultwarden machine-account maturity.** Confirm the current Vaultwarden version's Secrets Manager / machine-account coverage (§4.4). Determines whether the Vaultwarden adapter's `enrollAgent` uses native machine accounts or the personal-vault-per-agent + collection model. Does **not** block (interface absorbs either), but sets P2 expectations.
3. **[VERIFY] step-ca root distribution for cross-site federation.** Confirm Synapse's supported mechanism for trusting a **custom federation CA root** (§8.1) so two private-CA sites can federate. If Synapse won't easily trust a private federation CA, private-domain federation may in practice require public certs (LE) on the federation SANs even when internal traffic uses step-ca.
4. **Default secret backend for the published installer.** Given the open-source ethos (G6), should the installer _default-suggest_ Vaultwarden (free, self-hostable) while clearly offering Vault, or present them neutrally? Recommendation: suggest Vaultwarden as the zero-cost path with the maturity caveat surfaced, Vault as the "I need native short-TTL machine creds now" path. Jason to confirm the framing.
5. **Single multi-SAN cert vs two certs in Mode A** (§3.3) — operational preference for identity-domain + host coverage. Minor; validate during P4.
6. **Reconfigure-time Synapse restart policy.** Which rendered-config changes (§5.4) require a Synapse restart vs hot-reload on the deployed version, so the config service knows when a tunable change needs a bounce. **[VERIFY]** at implementation.
---
## Appendix A — Real mechanics this RFC leans on (quick reference)
- **`server_name`** — Synapse identity domain; the `:suffix` of every MXID/alias; install-immutable; changing it = re-home (no in-place rename). Distinct from where Synapse _listens_.
- **Delegation** — `https://<server_name>/.well-known/matrix/server` → `{"m.server":"host:port"}` and/or `_matrix._tcp.<server_name>` **SRV**; how identity-domain ≠ host is reconciled (Mode A). **[VERIFY]** precedence on deployed Synapse.
- **`.well-known/matrix/client`** — C-S discovery so agents/Element find the homeserver host from the identity domain.
- **`federation_domain_whitelist`** — Synapse allowlist; only listed domains federate; our hard no-public-federation boundary.
- **ACME** — single provisioning protocol for both CAs; operator supplies **directory URL** (step-ca vs Let's Encrypt) + **challenge type**.
- **HTTP-01 / DNS-01 / TLS-ALPN-01** — challenge types; **DNS-01 is the one for private/split-horizon/wildcard**; HTTP-01 for public single host with :80; TLS-ALPN-01 for :443-only public.
- **step-ca ACME provisioner** — Smallstep's self-hosted CA exposing an ACME directory; enables private/internal-domain certs and total control; may require **EAB**; root must be distributed to validators.
- **Let's Encrypt** — public ACME CA; universally-trusted chain; 90-day certs; staging endpoint for testing.
- **Bitwarden/Vaultwarden org + collection + machine/service-account** — org holds collections; collections scope access; machine/service accounts (Bitwarden **Secrets Manager**) are the clean automation primitive but **Vaultwarden coverage is partial/evolving [VERIFY]**; personal-vault-per-agent + org-collection works today.
- **Vault KV v2 / AppRole / lease-TTL / revoke** — the capable backend; native versioning=rotate, TTL+revoke=transient per-agent creds.
- **Silent defederation** — a lapsed/renewal-failed federation cert causes peers to stop trusting S2S with no local error; must be monitored + alarmed (§3.4).
_All Matrix/ACME/secret-backend mechanics above are cited from architecture knowledge and MUST be re-verified against the actually deployed versions during implementation — every **[VERIFY]** is a checkpoint, not an assumption. Every illustrative domain (`mosaic.woltje.com`, `mosaic.uscllc.com`, `matrix.woltje.com`) is an operator-supplied example, never a product default or literal._
docs/archive/planning/monorepo-consolidation/README.md
+19
View File
@@ -0,0 +1,19 @@
# Monorepo consolidation planning bundle
> **Status:** Historical planning evidence. The five source records were moved byte-identically from migration quarantine on 2026-08-10.
This bundle records the decision and proposed work packages for consolidating prior Forge, MACP, and OpenClaw framework work into this monorepo.
## Records
- [Consolidation brief](brief.md) — original scope, target layout, constraints, and success criteria.
- [Board review](board-review.md) — historical deliberation and conditional approval.
- [WP1: Forge package](wp1-forge-package.md) — proposed TypeScript Forge implementation.
- [WP2: MACP package](wp2-macp-package.md) — proposed protocol, gate, credential, and event implementation.
- [WP3: Mosaic framework plugin](wp3-mosaic-framework-plugin.md) — proposed OpenClaw framework plugin port.
## Current boundary
`packages/forge`, `packages/macp`, and `plugins/mosaic-framework` exist in the current checkout. That existence is sufficient to classify this bundle as historical planning, but it does **not** prove that every stated success criterion, integration, coverage target, or behavior remains satisfied.
Use current package source, manifests, and tests for implementation truth. Do not use this bundle as an active task ledger or as authority to change package behavior.
docs/archive/planning/monorepo-consolidation/board-review.md
+1256
View File
File diff suppressed because it is too large Load Diff
docs/archive/planning/monorepo-consolidation/brief.md
+231
View File
@@ -0,0 +1,231 @@
# Brief: Monorepo Consolidation — mosaic/stack → mosaic/mosaic-stack
## Source
Architecture consolidation — merge the mosaic/stack repo (Forge pipeline, MACP protocol, framework tools) into mosaic/mosaic-stack (Harness Foundation platform). Two repos doing related work that need to converge.
## Context
**mosaic/stack** (OLD) contains:
- Forge progressive refinement pipeline (stages, agents, personas, rails, debate protocol, brief classification)
- MACP protocol (JSON schemas, deterministic Python controller, dispatcher, event system, gate runner)
- Credential resolver (Python — OC config, mosaic files, ambient env, JSON5 parser)
- OC framework plugin (injects Mosaic rails into all agent sessions)
- Profiles (runtime-neutral context packs for tech stacks and domains)
- Stage adapter (Forge→MACP bridge)
- Board tasks (multi-agent board evaluation)
- OpenBrain specialist memory (learning capture/recall)
- 17 guides, 5 universal skills
**mosaic/mosaic-stack** (NEW) contains:
- Harness Foundation platform (NestJS gateway, Next.js web, Drizzle ORM, Pi SDK runtime)
- 5 provider adapters, task classifier, routing rules, model capability matrix
- MACP OC plugin (ACP runtime backend with Pi bridge)
- TS coord package (mission runner, tasks file manager, status tracker — 1635 lines)
- BullMQ job queue, OTEL telemetry, channel plugins (Discord, Telegram)
- CLI with TUI, 65/65 tasks done, v0.2.0
**Decision:** NEW repo is the base. All unique work from OLD gets ported into NEW as packages.
## Scope
### Work Package 1: Forge Pipeline Package (`packages/forge`)
Port the entire Forge progressive refinement pipeline as a TypeScript package.
**From OLD:**
- `forge/pipeline/stages/*.md` — 11 stage definitions
- `forge/pipeline/agents/{board,generalists,specialists,cross-cutting}/*.md` — all persona definitions
- `forge/pipeline/rails/*.md` — debate protocol, dynamic composition, worker rails
- `forge/pipeline/gates/` — gate reviewer definitions
- `forge/pipeline/orchestrator/run-structure.md` — file-based observability spec
- `forge/templates/` — brief and PRD templates
- `forge/pipeline/orchestrator/board_tasks.py` → rewrite in TS
- `forge/pipeline/orchestrator/stage_adapter.py` → rewrite in TS
- `forge/pipeline/orchestrator/pipeline_runner.py` → rewrite in TS
- `forge/forge` CLI (Python) → rewrite in TS, integrate with `packages/cli`
**Package structure:**
```
packages/forge/
├── src/
│ ├── index.ts # Public API
│ ├── pipeline-runner.ts # Orchestrates full pipeline run
│ ├── stage-adapter.ts # Maps stages to MACP/coord tasks
│ ├── board-tasks.ts # Multi-agent board evaluation task generator
│ ├── brief-classifier.ts # strategic/technical/hotfix classification
│ ├── types.ts # Stage specs, run manifest, gate results
│ └── constants.ts # Stage sequence, timeouts, labels
├── pipeline/
│ ├── stages/ # .md stage definitions (copied)
│ ├── agents/ # .md persona definitions (copied)
│ │ ├── board/
│ │ ├── cross-cutting/
│ │ ├── generalists/
│ │ └── specialists/
│ │ ├── language/
│ │ └── domain/
│ ├── rails/ # .md rails (copied)
│ ├── gates/ # .md gate definitions (copied)
│ └── templates/ # brief + PRD templates (copied)
└── package.json
```
**Key design decisions:**
- Pipeline markdown assets are runtime data, not compiled — ship as-is in the package
- `pipeline-runner.ts` calls into `packages/coord` for task execution (not a separate controller)
- Stage adapter generates coord-compatible tasks, not MACP JSON directly
- Board tasks use `depends_on_policy: "all_terminal"` for synthesis
- Per-stage timeouts from `STAGE_TIMEOUTS` map
- Brief classifier supports CLI flag, YAML frontmatter, and keyword auto-detection
- Run output goes to project-scoped `.forge/runs/{run-id}/` (not inside the Forge package)
**Persona override system (new):**
- Base personas ship with the package (read-only)
- Project-level overrides in `.forge/personas/{role}.md` extend (not replace) base personas
- Board composition configurable via `.forge/config.yaml`:
```yaml
board:
additional_members:
- compliance-officer.md
skip_members: []
specialists:
always_include:
- proxmox-expert
```
- OpenBrain integration for cross-run specialist memory (when enabled)
### Work Package 2: MACP Protocol Package (`packages/macp`)
Port the MACP protocol layer, event system, and gate runner as a TypeScript package.
**From OLD:**
- `tools/macp/protocol/task.schema.json` — task JSON schema
- `tools/macp/protocol/` — event schemas
- `tools/macp/controller/gate_runner.py` → rewrite in TS as `gate-runner.ts`
- `tools/macp/events/` — event watcher, webhook adapter, Discord formatter → rewrite in TS
- `tools/macp/dispatcher/credential_resolver.py` → rewrite in TS as `credential-resolver.ts`
- `tools/macp/memory/learning_capture.py` + `learning_recall.py` → rewrite in TS
**Package structure:**
```
packages/macp/
├── src/
│ ├── index.ts # Public API
│ ├── types.ts # Task, event, result, gate types
│ ├── schemas/ # JSON schemas (copied)
│ ├── gate-runner.ts # Mechanical + AI review quality gates
│ ├── credential-resolver.ts # Provider credential resolution (mosaic files, OC config, ambient)
│ ├── event-emitter.ts # Append events to ndjson, structured event types
│ ├── event-watcher.ts # Poll events.ndjson with cursor persistence
│ ├── webhook-adapter.ts # POST events to configurable URL
│ ├── discord-formatter.ts # Human-readable event messages
│ └── learning.ts # OpenBrain capture + recall
└── package.json
```
**Integration with existing packages:**
- `packages/coord` uses `packages/macp` for event emission, gate running, and credential resolution
- `plugins/macp` uses `packages/macp` for protocol types and credential resolution
- `packages/forge` uses `packages/macp` gate types for stage gates
### Work Package 3: OC Framework Plugin (`plugins/mosaic-framework`)
Port the OC framework plugin that injects Mosaic rails into all agent sessions.
**From OLD:**
- `oc-plugins/mosaic-framework/index.ts` — `before_agent_start` + `subagent_spawning` hooks
- `oc-plugins/mosaic-framework/openclaw.plugin.json`
**Structure:**
```
plugins/mosaic-framework/
├── src/
│ └── index.ts # Plugin hooks
└── package.json
```
**This is separate from `plugins/macp`:**
- `mosaic-framework` = injects Mosaic rails/contracts into every OC session (passive enforcement)
- `macp` = provides an ACP runtime backend for MACP task execution (active runtime)
### Work Package 4: Profiles + Guides + Skills
Port reference content as a documentation/config package or top-level directories.
**From OLD:**
- `profiles/domains/*.json` — HIPAA, fintech, crypto context packs
- `profiles/tech-stacks/*.json` — NestJS, Next.js, FastAPI, React conventions
- `profiles/workflows/*.json` — API development, frontend component, testing workflows
- `guides/*.md` — 17 guides (auth, backend, QA, orchestrator, PRD, etc.)
- `skills-universal/` — jarvis, macp, mosaic-standards, prd, setup-cicd skills
**Destination:**
```
profiles/ # Top-level (same as OLD)
guides/ # Top-level (same as OLD)
skills/ # Top-level (renamed from skills-universal)
```
These are runtime-neutral assets consumed by any agent or profile loader — they don't belong in a compiled package.
## Out of Scope
- Rewriting the NestJS orchestrator app from OLD (`apps/orchestrator/`) — its functionality is subsumed by `packages/coord` + `apps/gateway`
- Porting the FastAPI coordinator from OLD (`apps/coordinator/`) — its functionality (webhook receiver, issue parser, quality orchestrator) is handled by `packages/coord` + `apps/gateway` in the new architecture
- Porting the Prisma schema or OLD's `apps/api` — Drizzle migration is complete
- Old Docker Compose configs (Traefik, Matrix, OpenBao) — NEW has its own infra setup
## Success Criteria
1. `packages/forge` exists with all 11 stage definitions, all persona markdowns, all rails, and TS implementations of pipeline-runner, stage-adapter, board-tasks, and brief-classifier
2. `packages/macp` exists with gate-runner, credential-resolver, event system, and learning capture/recall — all in TypeScript
3. `plugins/mosaic-framework` exists and registers OC hooks for rails injection
4. Profiles, guides, and skills are present at top-level
5. `packages/forge` integrates with `packages/coord` for task execution
6. `packages/macp` credential-resolver is used by `plugins/macp` Pi bridge
7. All existing tests pass (no regressions)
8. New packages have test coverage ≥85%
9. `pnpm lint && pnpm typecheck && pnpm build` passes
10. `.forge/runs/` project-scoped output directory works for at least one test run
## Technical Constraints
- All new code is ESM with NodeNext module resolution
- No Python in the new repo — everything rewrites to TypeScript
- Pipeline markdown assets (stages, personas, rails) are shipped as package data, not compiled
- Credential resolver must support: mosaic credential files, OC config (JSON5), ambient environment — same resolution order as the Python version
- Must preserve `depends_on_policy` semantics (all, any, all_terminal)
- Per-stage timeouts must be preserved
- JSON5 stripping must use the placeholder-extraction approach (not naive regex on string content)
## Estimated Complexity
High — crosses 4 work packages with protocol porting, TS rewrites, and integration wiring. Each work package is independently shippable.
**Suggested execution order:**
1. WP4 (profiles/guides/skills) — pure copy, no code, fast win
2. WP2 (packages/macp) — protocol foundation, needed by WP1 and WP3
3. WP1 (packages/forge) — the big one, depends on WP2
4. WP3 (plugins/mosaic-framework) — OC integration, can parallel with WP1
## Dependencies
- `packages/coord` must be stable (it is — WP1 integrates with it)
- `plugins/macp` must be stable (it is — WP2 provides types/credentials to it)
- Pi SDK (`@mariozechner/pi-agent-core`) already in the dependency tree
docs/archive/planning/monorepo-consolidation/wp1-forge-package.md
+265
View File
@@ -0,0 +1,265 @@
# WP1: packages/forge — Forge Pipeline Package
## Context
Port the Forge progressive refinement pipeline from Python (~/src/mosaic-stack/forge/) to TypeScript as `packages/forge` in this monorepo. The pipeline markdown assets (stages, agents, personas, rails, gates, templates) are already copied to `packages/forge/pipeline/`. This task is the TypeScript implementation layer.
**Board decisions that constrain this work:**
- Abstract TaskExecutor interface — packages/forge must NOT hard-import packages/coord. Define an abstract interface; coord satisfies it.
- Clean index.ts exports, no internal path leakage, no hardcoded paths
- 85% test coverage on TS implementation files (markdown assets excluded)
- Test strategy for non-deterministic AI orchestration: fixture-based integration tests
- OpenBrain is OUT OF SCOPE
- ESM only, zero Python
**Dependencies available:**
- `@mosaicstack/macp` (packages/macp) is built and provides: GateEntry, GateResult, Task types, credential resolution, gate running, event emission
## Source Files (Python → TypeScript)
### 1. types.ts
Define all Forge-specific types:
```typescript
// Stage specification
interface StageSpec {
number: string;
title: string;
dispatch: 'exec' | 'yolo' | 'pi';
type: 'research' | 'review' | 'coding' | 'deploy';
gate: string;
promptFile: string;
qualityGates: (string | GateEntry)[];
}
// Brief classification
type BriefClass = 'strategic' | 'technical' | 'hotfix';
type ClassSource = 'cli' | 'frontmatter' | 'auto';
// Run manifest (persisted to disk)
interface RunManifest {
runId: string;
brief: string;
codebase: string;
briefClass: BriefClass;
classSource: ClassSource;
forceBoard: boolean;
createdAt: string;
updatedAt: string;
currentStage: string;
status: 'in_progress' | 'completed' | 'failed' | 'interrupted' | 'rejected';
stages: Record<string, StageStatus>;
}
// Abstract task executor (decouples from packages/coord)
interface TaskExecutor {
submitTask(task: ForgeTask): Promise<void>;
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
}
// Persona override config
interface ForgeConfig {
board?: {
additionalMembers?: string[];
skipMembers?: string[];
};
specialists?: {
alwaysInclude?: string[];
};
}
```
### 2. constants.ts
**Source:** Top of `~/src/mosaic-stack/forge/lib` (ALL_STAGES, LABELS, STAGE_SPECS equivalent) + `~/src/mosaic-stack/forge/pipeline/orchestrator/stage_adapter.py` (STAGE_TIMEOUTS)
```typescript
export const STAGE_SEQUENCE = [
'00-intake',
'00b-discovery',
'01-board',
'01b-brief-analyzer',
'02-planning-1',
'03-planning-2',
'04-planning-3',
'05-coding',
'06-review',
'07-remediate',
'08-test',
'09-deploy',
];
export const STAGE_TIMEOUTS: Record<string, number> = {
'00-intake': 120,
'00b-discovery': 300,
'01-board': 120,
'02-planning-1': 600,
// ... etc
};
export const STAGE_LABELS: Record<string, string> = {
'00-intake': 'INTAKE',
// ... etc
};
```
Also: STRATEGIC_KEYWORDS, TECHNICAL_KEYWORDS for brief classification.
### 3. brief-classifier.ts
**Source:** `classify_brief()`, `parse_brief_frontmatter()`, `stages_for_class()` from `~/src/mosaic-stack/forge/lib`
- Auto-classify brief by keyword analysis (strategic vs technical)
- Parse YAML frontmatter for explicit `class:` field
- CLI flag override
- Return stage list based on classification (strategic = full pipeline, technical = skip board, hotfix = skip board + brief analyzer)
### 4. stage-adapter.ts
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/stage_adapter.py`
- `mapStageToTask()`: Convert a Forge stage into a task compatible with TaskExecutor
- Stage briefs written to `{runDir}/{stageName}/brief.md`
- Result paths at `{runDir}/{stageName}/result.json`
- Previous results read from disk at runtime (not baked into brief)
- Per-stage timeouts from STAGE_TIMEOUTS
- depends_on chain built from stage sequence
### 5. board-tasks.ts
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/board_tasks.py`
- `loadBoardPersonas()`: Read all .md files from `pipeline/agents/board/`
- `generateBoardTasks()`: One task per persona + synthesis task
- Synthesis depends on all persona tasks with `depends_on_policy: 'all_terminal'`
- Persona briefs include role description + brief under review
- Synthesis script merges independent reviews into board memo
### 6. pipeline-runner.ts
**Source:** `~/src/mosaic-stack/forge/pipeline/orchestrator/pipeline_runner.py` + `~/src/mosaic-stack/forge/lib` (cmd_run, cmd_resume, cmd_status)
- `runPipeline(briefPath, projectRoot, options)`: Full pipeline execution
- Creates run directory at `{projectRoot}/.forge/runs/{runId}/`
- Generates tasks for all stages, submits to TaskExecutor
- Tracks manifest.json with stage statuses
- `resumePipeline(runDir)`: Pick up from last incomplete stage
- `getPipelineStatus(runDir)`: Read manifest and report
**Key difference from Python:** Run output goes to PROJECT-scoped `.forge/runs/`, not inside the Forge package.
### 7. Persona Override System (NEW — not in Python)
- Base personas read from `packages/forge/pipeline/agents/`
- Project overrides read from `{projectRoot}/.forge/personas/{role}.md`
- Merge strategy: project persona content APPENDED to base persona (not replaced)
- Board composition configurable via `{projectRoot}/.forge/config.yaml`
- If no project config exists, use defaults (all base personas, no overrides)
## Package Structure
```
packages/forge/
├── src/
│ ├── index.ts
│ ├── types.ts
│ ├── constants.ts
│ ├── brief-classifier.ts
│ ├── stage-adapter.ts
│ ├── board-tasks.ts
│ ├── pipeline-runner.ts
│ └── persona-loader.ts
├── pipeline/ # Already copied (WP4) — markdown assets
│ ├── stages/
│ ├── agents/
│ ├── rails/
│ ├── gates/
│ └── templates/
├── __tests__/
│ ├── brief-classifier.test.ts
│ ├── stage-adapter.test.ts
│ ├── board-tasks.test.ts
│ ├── pipeline-runner.test.ts
│ └── persona-loader.test.ts
├── package.json
├── tsconfig.json
└── vitest.config.ts
```
## Package.json
```json
{
"name": "@mosaicstack/forge",
"version": "0.0.1",
"type": "module",
"exports": {
".": "./src/index.ts"
},
"dependencies": {
"@mosaicstack/macp": "workspace:*"
},
"devDependencies": {
"vitest": "workspace:*",
"typescript": "workspace:*"
}
}
```
Only dependency: @mosaicstack/macp (for gate types, event emission).
## Test Strategy (Board requirement)
**Deterministic code (brief-classifier, stage-adapter, board-tasks, persona-loader, constants):**
- Standard unit tests with known inputs/outputs
- 100% of classification logic, stage mapping, persona loading covered
**Non-deterministic code (pipeline-runner):**
- Fixture-based integration tests using a mock TaskExecutor
- Mock executor returns pre-recorded results for each stage
- Tests verify: manifest progression, stage ordering, dependency enforcement, resume behavior, error handling
- NO real AI calls in tests
**Markdown assets:** Excluded from coverage measurement (configure vitest to exclude `pipeline/` directory).
## ESM Requirements
- `"type": "module"` in package.json
- NodeNext module resolution in tsconfig
- `.js` extensions in all imports
- No CommonJS
## Key Design: Abstract TaskExecutor
```typescript
// In packages/forge/src/types.ts
export interface TaskExecutor {
submitTask(task: ForgeTask): Promise<void>;
waitForCompletion(taskId: string, timeoutMs: number): Promise<TaskResult>;
getTaskStatus(taskId: string): Promise<TaskStatus>;
}
// In packages/coord (or wherever the concrete impl lives)
export class CoordTaskExecutor implements TaskExecutor {
// ... uses packages/coord runner
}
```
This means packages/forge can be tested with a mock executor and deployed with any backend.
## Asset Resolution
Pipeline markdown assets (stages, personas, rails) must be resolved relative to the package installation, NOT hardcoded paths:
```typescript
// Use import.meta.url to find package root
const PACKAGE_ROOT = new URL('..', import.meta.url).pathname;
const PIPELINE_DIR = path.join(PACKAGE_ROOT, 'pipeline');
```
Project-level overrides resolved relative to projectRoot parameter.
docs/archive/planning/monorepo-consolidation/wp2-macp-package.md
+150
View File
@@ -0,0 +1,150 @@
# WP2: packages/macp — MACP Protocol Package
## Context
Port the MACP protocol layer from Python (in ~/src/mosaic-stack/tools/macp/) to TypeScript as `packages/macp` in this monorepo. This package provides the foundational protocol types, quality gate execution, credential resolution, and event system that `packages/coord` and `plugins/macp` depend on.
**Board decisions that constrain this work:**
- No Python in the new repo — everything rewrites to TypeScript
- OpenBrain learning capture/recall is OUT OF SCOPE (deferred to future brief)
- 85% test coverage on TS implementation files
- Credential resolver behavior must be captured as test fixtures BEFORE rewrite
- Clean index.ts exports, no internal path leakage
## Source Files (Python → TypeScript)
### 1. credential-resolver.ts
**Source:** `~/src/mosaic-stack/tools/macp/dispatcher/credential_resolver.py`
Resolution order (MUST preserve exactly):
1. Mosaic credential files (`~/.config/mosaic/credentials/{provider}.env`)
2. OpenClaw config (`~/.openclaw/openclaw.json`) — env block + models.providers.{provider}.apiKey
3. Ambient environment variables
4. CredentialError (failure)
Key behaviors to preserve:
- Provider registry: anthropic, openai, zai → env var names + credential file paths + OC config paths
- Dotenv parser: handles single/double quotes, comments, blank lines
- JSON5 stripping: placeholder-extraction approach (NOT naive regex) — protects URLs and timestamps inside string values
- OC config permission check: warn on world-readable, skip if wrong owner
- Redacted marker detection: `__OPENCLAW_REDACTED__` values skipped
- Task-level override via `credentials.provider_key_env`
### 2. gate-runner.ts
**Source:** `~/src/mosaic-stack/tools/macp/controller/gate_runner.py`
Three gate types:
- `mechanical`: shell command, pass = exit code 0
- `ai-review`: shell command producing JSON, parse findings, fail on blockers
- `ci-pipeline`: placeholder (always passes for now)
Key behaviors:
- `normalize_gate()`: accepts string or dict, normalizes to gate entry
- `run_gate()`: executes single gate, returns result with pass/fail
- `run_gates()`: executes all gates, emits events, returns (all_passed, results)
- AI review parsing: `_count_ai_findings()` reads stats.blockers or findings[].severity
- `fail_on` modes: "blocker" (default) or "any"
### 3. event-emitter.ts
**Source:** `~/src/mosaic-stack/tools/macp/controller/gate_runner.py` (emit_event, append_event functions) + `~/src/mosaic-stack/tools/macp/events/`
- Append structured events to ndjson file
- Event types: task.assigned, task.started, task.completed, task.failed, task.escalated, task.gated, task.retry.scheduled, rail.check.started, rail.check.passed, rail.check.failed
- Each event: event_id (uuid), event_type, task_id, status, timestamp, source, message, metadata
### 4. types.ts
**Source:** `~/src/mosaic-stack/tools/macp/protocol/task.schema.json`
TypeScript types for:
- Task (id, title, status, dispatch, runtime, depends_on, depends_on_policy, quality_gates, timeout_seconds, metadata, etc.)
- Event (event_id, event_type, task_id, status, timestamp, source, message, metadata)
- GateResult (command, exit_code, type, passed, output, findings, blockers)
- TaskResult (task_id, status, completed_at, exit_code, gate_results, files_changed, etc.)
- CredentialError, ProviderRegistry
### 5. schemas/ (copy)
Copy `~/src/mosaic-stack/tools/macp/protocol/task.schema.json` as-is.
## Package Structure
```
packages/macp/
├── src/
│ ├── index.ts
│ ├── types.ts
│ ├── credential-resolver.ts
│ ├── gate-runner.ts
│ ├── event-emitter.ts
│ └── schemas/
│ └── task.schema.json
├── __tests__/
│ ├── credential-resolver.test.ts
│ ├── gate-runner.test.ts
│ └── event-emitter.test.ts
├── package.json
├── tsconfig.json
└── vitest.config.ts
```
## Package.json
```json
{
"name": "@mosaicstack/macp",
"version": "0.0.1",
"type": "module",
"exports": {
".": "./src/index.ts"
},
"dependencies": {},
"devDependencies": {
"vitest": "workspace:*",
"typescript": "workspace:*"
}
}
```
Zero external dependencies. Uses node:fs, node:path, node:child_process, node:crypto only.
## Test Requirements
Port ALL existing Python tests as TypeScript equivalents:
- `test_resolve_from_file` → credential file resolution
- `test_resolve_from_ambient` → ambient env resolution
- `test_resolve_from_oc_config_env_block` → OC config env block
- `test_resolve_from_oc_config_provider_apikey` → OC config provider
- `test_oc_config_precedence` → mosaic file wins over OC config
- `test_oc_config_missing_file` → graceful fallback
- `test_json5_strip` → structural transforms
- `test_json5_strip_urls_and_timestamps` → URLs/timestamps survive
- `test_redacted_values_skipped` → redacted marker detection
- `test_oc_config_permission_warning` → file permission check
- `test_resolve_missing_raises` → CredentialError thrown
- Gate runner: mechanical pass/fail, AI review parsing, ci-pipeline placeholder
- Event emitter: append to ndjson, event structure validation
## ESM Requirements
- `"type": "module"` in package.json
- NodeNext module resolution in tsconfig
- `.js` extensions in all imports
- No CommonJS (`require`, `module.exports`)
## Integration Points
After this package is built:
- `packages/coord` should import `@mosaicstack/macp` for event emission and gate types
- `plugins/macp` should import `@mosaicstack/macp` for credential resolution and protocol types
docs/archive/planning/monorepo-consolidation/wp3-mosaic-framework-plugin.md
+63
View File
@@ -0,0 +1,63 @@
# WP3: plugins/mosaic-framework — OC Rails Injection Plugin
## Context
Port the OpenClaw framework plugin from ~/src/mosaic-stack/oc-plugins/mosaic-framework/ to `plugins/mosaic-framework` in this monorepo. This plugin injects Mosaic framework contracts (rails, completion gates, worktree requirements) into every OpenClaw agent session.
**This is SEPARATE from plugins/macp:**
- `mosaic-framework` = passive enforcement — injects rails into all OC sessions
- `macp` = active runtime — provides ACP backend for MACP task execution
## Source Files
**Source:** `~/src/mosaic-stack/oc-plugins/mosaic-framework/`
- `index.ts` — plugin hooks (before_agent_start, subagent_spawning)
- `openclaw.plugin.json` — plugin manifest
- `package.json`
## What It Does
### For OC native agents (before_agent_start hook):
- Injects Mosaic global hard rules via `appendSystemContext`
- Completion gates: code review ✓ | security review ✓ | tests GREEN ✓ | CI green ✓
- Worker completion protocol: open PR → fire system event → EXIT — never merge
- Worktree requirement: `~/src/{repo}-worktrees/{task-slug}`, never `/tmp`
- Injects dynamic mission state via `prependContext` (reads from project's `.mosaic/orchestrator/mission.json`)
### For ACP coding workers (subagent_spawning hook):
- Writes `~/.codex/instructions.md` or `~/.claude/CLAUDE.md` BEFORE the process starts
- Full runtime contract: mandatory load order, hard gates, mode declaration
- Global framework rules + worktree + completion gate requirements
## Implementation
Port the TypeScript source, updating hardcoded paths to be configurable. The OC plugin SDK imports should reference the installed OpenClaw location dynamically (not hardcoded `/home/jarvis/` paths like the OLD version).
**Structure:**
```
plugins/mosaic-framework/
├── src/
│ └── index.ts
├── openclaw.plugin.json
├── package.json
└── tsconfig.json
```
## Key Constraint
The plugin SDK imports in the OLD version use absolute paths:
```typescript
import type { OpenClawPluginApi } from '/home/jarvis/.npm-global/lib/node_modules/openclaw/dist/plugin-sdk/index.js';
```
This must be resolved dynamically or via a peer dependency. Check how `plugins/macp` handles this in the new repo and follow the same pattern.
## Tests
Minimal — plugin hooks are integration-tested against OC runtime. Unit test the context string builders and config resolution.
docs/archive/tui/PRD-TUI_Improvements.md
+70
View File
@@ -0,0 +1,70 @@
# PRD: TUI Improvements — Phase 7
**Branch:** `feat/p7-tui-improvements`
**Package:** `packages/cli`
**Status:** In Progress
---
## Problem Statement
The current Mosaic CLI TUI (`packages/cli/src/tui/app.tsx`) is a minimal single-file Ink application with:
- Flat message list with no visual hierarchy
- No system context visibility (cwd, branch, model, tokens)
- Noisy error messages when gateway is disconnected
- No conversation management (list, switch, rename, delete)
- No multi-panel layout or navigation
- No tool call visibility during agent execution
- No thinking/reasoning display
The TUI should be the power-user interface to Mosaic — informative, responsive, and visually clean.
---
## Goals
### Wave 1 — Status Bar & Polish (MVP)
Provide essential context at a glance and reduce noise.
1. **Top status bar** — shows: connection indicator (●/○), gateway URL, agent model name
2. **Bottom status bar** — shows: cwd, git branch, token usage (input/output/total)
3. **Better message formatting** — distinct visual treatment for user vs assistant messages, timestamps, word wrap
4. **Quiet disconnect** — single-line indicator when gateway is offline instead of flooding error messages; auto-reconnect silently
5. **Tool call display** — inline indicators when agent uses tools (spinner + tool name during execution, ✓/✗ on completion)
6. **Thinking/reasoning display** — collapsible dimmed block for `agent:thinking` events
### Wave 2 — Layout & Navigation
Multi-panel layout with keyboard navigation.
1. **Conversation sidebar** — list conversations, create new, switch between them
2. **Keybinding system** — Ctrl+N (new conversation), Ctrl+L (conversation list toggle), Ctrl+K (command palette concept)
3. **Scrollable message history** — viewport with PgUp/PgDn/arrow key scrolling
4. **Message search** — find in current conversation
### Wave 3 — Advanced Features
1. **Project/mission views** — show active projects, missions, tasks
2. **Agent status monitoring** — real-time agent state, queue depth
3. **Settings/config screen** — view/edit connection settings, model preferences
4. **Multiple agent sessions** — split view or tab-based multi-agent
---
## Technical Approach
- **Ink 5** (React for CLI) — already in deps
- **Component architecture** — break monolithic `app.tsx` into composable components
- **Typed Socket.IO events** — leverage `@mosaicstack/types` `ServerToClientEvents` / `ClientToServerEvents`
- **Local state only** (Wave 1) — cwd/branch read from `process.cwd()` and `git` at startup
- **Gateway metadata** (future) — extend socket handshake or add REST endpoint for model info, token usage
---
## Non-Goals (for now)
- Image rendering in terminal
- File editor integration
- SSH/remote gateway auto-discovery
docs/archive/tui/TASKS-TUI_Improvements.md
+105
View File
@@ -0,0 +1,105 @@
# Tasks: TUI Improvements
**Branch:** `feat/p7-tui-improvements`
**Worktree:** `/home/jwoltje/src/mosaic-mono-v1-worktrees/tui-improvements`
**PRD:** [PRD-TUI_Improvements.md](./PRD-TUI_Improvements.md)
---
## Wave 1 — Status Bar & Polish ✅
| ID | Task | Status | Notes |
| -------- | ----------------------------------------------------------------------------------------------------- | ------- | ------- |
| TUI-001 | Component architecture — split `app.tsx` into `TopBar`, `BottomBar`, `MessageList`, `InputBar`, hooks | ✅ done | 79ff308 |
| TUI-002 | Top status bar — branded mosaic icon, version, model, connection indicator | ✅ done | 6c2b01e |
| TUI-003 | Bottom status bar — cwd, git branch, token usage, session ID, gateway status | ✅ done | e8d7ab8 |
| TUI-004 | Message formatting — timestamps, role colors (❯ you / ◆ assistant), word wrap | ✅ done | 79ff308 |
| TUI-005 | Quiet disconnect — single indicator, auto-reconnect, no error flood | ✅ done | 79ff308 |
| TUI-006 | Tool call display — inline spinner + tool name during execution, ✓/✗ on completion | ✅ done | 79ff308 |
| TUI-007 | Thinking/reasoning display — dimmed 💭 block for `agent:thinking` events | ✅ done | 79ff308 |
| TUI-007b | Wire token usage, model info, thinking levels end-to-end (gateway → types → TUI) | ✅ done | a061a64 |
| TUI-007c | Ctrl+T to cycle thinking levels via `set:thinking` socket event | ✅ done | a061a64 |
## Wave 2 — Layout & Navigation ✅
| ID | Task | Status | Notes |
| ------- | --------------------------------------------------------- | ------- | ------- |
| TUI-010 | Scrollable message history — viewport with PgUp/PgDn | ✅ done | 4d4ad38 |
| TUI-008 | Conversation sidebar — list, create, switch conversations | ✅ done | 9ef578c |
| TUI-009 | Keybinding system — Ctrl+L, Ctrl+N, Ctrl+K, Escape | ✅ done | 9f38f5a |
| TUI-011 | Message search — find in current conversation | ✅ done | 8627827 |
## Wave 3 — Advanced Features
| ID | Task | Status | Notes |
| ------- | ----------------------- | ----------- | ----- |
| TUI-012 | Project/mission views | not-started | |
| TUI-013 | Agent status monitoring | not-started | |
| TUI-014 | Settings/config screen | not-started | |
| TUI-015 | Multiple agent sessions | not-started | |
---
## Handoff Notes
### File Structure
```
packages/cli/src/tui/
├── app.tsx ← Shell composing all components + global keybindings
├── components/
│ ├── top-bar.tsx ← Mosaic icon + version + model + connection
│ ├── bottom-bar.tsx ← Keybinding hints + 3-line footer: gateway, cwd, tokens
│ ├── message-list.tsx ← Messages, tool calls, thinking, streaming, search highlights
│ ├── input-bar.tsx ← Bordered prompt with context-aware placeholder
│ ├── sidebar.tsx ← Conversation list with keyboard navigation
│ └── search-bar.tsx ← Message search input with match count + navigation
└── hooks/
├── use-socket.ts ← Typed Socket.IO + switchConversation/clearMessages
├── use-git-info.ts ← Reads cwd + git branch at startup
├── use-viewport.ts ← Scrollable viewport with auto-follow + PgUp/PgDn
├── use-app-mode.ts ← Panel focus state machine (chat/sidebar/search)
├── use-conversations.ts ← REST client for conversation CRUD
└── use-search.ts ← Message search with match cycling
```
### Cross-Package Changes
- **`packages/types/src/chat/events.ts`** — Added `SessionUsagePayload`, `SessionInfoPayload`, `SetThinkingPayload`, `session:info` event, `set:thinking` event
- **`apps/gateway/src/chat/chat.gateway.ts`** — Emits `session:info` on session creation, includes `usage` in `agent:end`, handles `set:thinking`
### Key Design Decisions
#### Wave 1
- Footer is 3 lines: (1) gateway status right-aligned, (2) cwd+branch left / session right, (3) tokens left / provider+model+thinking right
- Mosaic icon uses brand colors in windmill cross pattern with `GAP` const to prevent prettier collapsing spaces
- `flexGrow={1}` on header text column prevents re-render artifacts
- Token/model data comes from gateway via `agent:end` payload and `session:info` events
- Thinking level cycling via Ctrl+T sends `set:thinking` to gateway, which validates and responds with `session:info`
#### Wave 2
- `useViewport` calculates scroll offset from terminal rows; auto-follow snaps to bottom on new messages
- `useAppMode` state machine manages focus: only the active panel handles keyboard input via `useInput({ isActive })`
- Sidebar fetches conversations via REST (`GET /api/conversations`), not socket events
- `switchConversation` in `useSocket` clears all local state (messages, streaming, tool calls)
- Search uses `useMemo` for reactive match computation; viewport auto-scrolls to current match
- Keybinding hints shown in bottom bar: `^L sidebar · ^N new · ^K search · ^T thinking · PgUp/Dn scroll`
### How to Run
```bash
cd /home/jwoltje/src/mosaic-mono-v1-worktrees/tui-improvements
pnpm --filter @mosaicstack/cli exec tsx src/cli.ts tui
# or after build:
node packages/cli/dist/cli.js tui --gateway http://localhost:14242
```
### Quality Gates
```bash
pnpm --filter @mosaicstack/cli typecheck && pnpm --filter @mosaicstack/cli lint
pnpm --filter @mosaicstack/gateway typecheck && pnpm --filter @mosaicstack/gateway lint
pnpm --filter @mosaicstack/types typecheck
```
docs/archive/work-records/1000-rm-61-ci-contract-exemption.md
+118
View File
@@ -0,0 +1,118 @@
# RM-61 — CI contract exemption for #1000 teardown artifact
**Tracking:** RM-61 / issue #1000
**Branch:** `fix/rm-61-ci-contract-exemption`
**Owner:** `coder-mos1`
## Objective
Determine, by red-first provider controls, whether the `ci-postgres` pod-not-found teardown signature discriminates from a real PostgreSQL failure. Only if it discriminates may a named, bounded CI-contract exemption be implemented. The exemption must retire when #1000 is fixed; fixing #1000 is the closure path.
## Pre-registered kill criterion
If an injected real `ci-postgres` failure also yields `pods "wp-svc-<ULID>-ci-postgres" not found` as the service's provider-visible failure, the signature does not discriminate. Option B is unsafe; stop exemption implementation and fall to Option A (#1000).
## Plan
1. Capture full `-f json` records for the 11 supplied observations and state counts.
2. Run one startup-failure control using the real pgvector/PostgreSQL image with an invalid `initdb` argument.
3. Run one post-readiness crash control using real PostgreSQL, `pg_isready`, and a deliberate postmaster kill while a DB-dependent probe is active.
4. Compare the raw `ci-postgres` service record independently of failures in dependent steps.
5. Investigate runner/time/head clustering only as a hypothesis; never encode incidental correlates or retries into policy.
6. If and only if the controls discriminate, implement and test the exact exemption, document its two-way boundary, and track retirement at #1000.
## Budget
No explicit token cap supplied. Working estimate: 20K–30K tokens. Limit provider controls to the two pre-registered runs; no retries or re-roll policy.
## Initial evidence
Historical JSON saved locally under `.evidence/rm-61/` (not for commit). Supplied pipelines: 11 total. Child-step counts: five pipelines with 9 children and six with 10 children. Seven contain the `ci-postgres` pod-not-found failure (#2170, #2175, #2180, #2181, #2182, #2187, #2188); four do not (#2158, #2167, #2184, #2186). Every observed workflow reports `agent_id=44`, so the available JSON does not separate clean and artifact runs by runner. This refutes runner identity as a discriminator in the sampled record.
## Progress
- [x] Requirements and kill criterion recorded before control implementation.
- [x] Historical full-JSON records captured.
- [x] Startup-failure control observed terminal.
- [x] Post-readiness crash control observed terminal.
- [x] Discrimination verdict recorded: Option B may proceed.
- [x] Conditional exemption implementation.
## Tests / evidence
### Control 1 — real startup failure
- Commit: `3931b0e29eb834914f7b17e4db7e221481d436fa`
- Pipeline: #2189, exact commit match.
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
- `ci-postgres`: `state=failure`, `exit_code=1`, `error=null`, with a five-second execution window.
- `test`: `state=failure`, `exit_code=1` after the readiness budget expired.
- Pipeline/workflow: terminal `failure`.
This control is red and its service record differs from #1000 (`exit_code=0` plus pod-not-found). It proves the startup-failure direction only. It does not settle the dangerous post-readiness crash/garbage-collection path.
### Control 2 — real post-readiness crash
- Commit: `25ac59715a94dd1b52ef42577472eb44ecc4b446`
- Pipeline: #2191, exact commit match.
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
- Service log proves PostgreSQL reached `database system is ready to accept connections`, the test created the arm table, and the service then killed postmaster PID 7.
- Test log proves a successful `SELECT 1` followed by `Connection refused`; it exited the pre-registered control code 61.
- `ci-postgres`: `state=failure`, `exit_code=137`, `error=null`, with a 203-second execution window.
- `test`: `state=failure`, `exit_code=61`.
- Pipeline/workflow: terminal `failure`.
This is the dangerous post-readiness crash path. Its service record is not pod-not-found and therefore differs from #1000 independently of the dependent test failure.
### Discrimination verdict
Both real failures are provider-visible as process exits (`exit_code=1` startup; `exit_code=137` crash) with no pod-not-found error. The seven observed #1000 artifacts are provider reconciliation misses (`exit_code=0` plus the exact pod-not-found error). The declared kill criterion did not fire, so Option B may proceed with a matcher requiring the full conjunction. This evidence does **not** prove every future Kubernetes failure is distinguishable; it proves these two concrete real-failure classes remain blocking and bounds the exemption to the observed reconciliation shape.
### Unit red-first checkpoint
The nine-case contract harness was written before the verifier. First execution exited 1 because `verify-terminal-green.py` did not exist; no exemption implementation was live. Cases pre-register ordinary green, the exact artifact, both provider controls, near-miss signatures, an independent failure, and a skipped step.
### Control 2 setup attempt — invalid, excluded from evidence
- Commit: `9455cd6a2650b2b7e70f746c07933d96e5cb3d20`
- Pipeline: #2190, exact commit match.
- Full JSON child scan: 9 total — 7 success, 2 failure, 0 skipped/pending/running.
- Service log: `/bin/sh: 0: -c requires an argument`.
- Root cause: Woodpecker service `commands` did not become the third `sh -c` argument. PostgreSQL never started, so this run is **not** the post-readiness crash control and provides no discrimination evidence.
- Focused remediation: place the script directly in the third `entrypoint` element and supply `PGPASSWORD` for the marker query. This is a control-fixture correction, not a retry of #1000 and not evidence for either verdict.
## Implementation evidence
- `verify-terminal-green.py` consumes only the full JSON/API record; it performs no fetch, retry, or trigger.
- Exact #2188 record: exit 0, 10 children, 9 success + 1 named exemption.
- Historical set: #2158/#2167/#2184/#2186 pass with no exemption; #2170/#2175/#2182/#2187/#2188 pass with one named exemption; #2180/#2181 remain red because independent failures exist.
- Provider controls: #2189 and #2191 both exit 1 under the verifier; neither is exempted.
- Unit harness: initial 9/9 cases passed after the red-first checkpoint; review remediation expands this to 12 cases with expected-head match/missing/mismatch coverage.
- Test-membership guard: PASS, population 45; 26 enumerated, 19 signed exclusions; all 39 surface paths present.
- Python compile: PASS.
- `pnpm typecheck`: PASS, 45/45 tasks.
- `pnpm lint`: PASS, 25/25 tasks.
- `pnpm format:check`: PASS after moving local evidence outside the repository tree.
- `test:framework-shell`: RM-61 and all preceding suites passed, then the pre-existing wake assertion aborted with exit 97 because this host's Bash 5.2.15 reports `BASH_LINENO [3 5]` where that suite requires `[3 4]`. RM-61 does not modify the wake suite; the command is not fully runnable on this host as written and no substitute result is claimed.
## Independent review
- Review 67 / comment 20403 at exact head `e7b29219e11efd0a19395156ac0b154bec0c3a73`: **REQUEST CHANGES**.
- Blocker: the verifier echoed the pipeline commit but did not bind it to the current PR head; mutating only #2188's commit still returned terminal-green.
- Remediation: require `--expect-commit <full-40>`, add a pipeline anomaly on missing/mismatched record commits, emit expected and observed values, wire both CI documentation and the merge-gate baseline to pass provider PR head, and add match/missing/mismatch tests.
- This binding is not prohibited head-based clustering policy: it proves the evidence belongs to the commit under verdict. Runner/node/time/head correlation remains excluded from the teardown signature itself.
- Review 69 later approved the commit-binding remediation at exact head `033b2ffb46674b2c0bcc5197273c109b461f62d9`; pipeline #2193 was 9/9 success. Before merge-gate, an independent adjudicator found that Python treats JSON `false == 0`, allowing a non-integer exit value to match. The prior gate-ready state was withdrawn. The type-strict set distinguishes genuine red-first controls (`false`, `0.0`, which wrongly exempted) from regression guards (`true`, `"0"`, `null`, which already blocked). Remediation requires the decoded type to be exactly `int` and excludes `bool` explicitly.
## Documentation checklist
- [x] CI contract documented in the canonical framework CI/CD guide.
- [x] Operator command documented in the Woodpecker tool README.
- [x] Merge-gate baseline points to the deterministic verifier and named retirement.
- [x] Tracking and retirement cite issue #1000.
- [x] Both positive and negative guarantee boundaries are stated.
- [x] No API/auth/schema/user-facing navigation change; OpenAPI, user guide, and sitemap are not applicable.
## Risks
The controls establish discrimination for deterministic startup failure and an armed post-readiness postmaster crash on the current Woodpecker Kubernetes provider. They cannot prove that every future Kubernetes failure mode will preserve a non-zero exit before reconciliation. The exact matcher minimizes that residual risk, and issue #1000 remains the mandatory provider-seam closure and retirement trigger.
docs/archive/work-records/2026-06-20-fleet-cli-local-canary.md
+52
View File
@@ -0,0 +1,52 @@
# Fleet CLI Local Canary Dogfood — 2026-06-20
## Objective
Move the durable tmux fleet PoC into a functional local canary on this server. This is **not** production deployment. It is a canary/dogfood path for a small local agent fleet using an isolated tmux socket.
## Issue
- Gitea issue: #562 — `feat(fleet): local CLI canary dogfood`
## Scope
Implement enough product surface to use the fleet locally:
- `mosaic fleet init/install/start/stop/restart/status/verify`
- `mosaic agent roster/status/send/reset/tail`
- roster schema and examples
- local canary docs and rollback instructions
- tests for CLI behavior where practical
- canary verification on named tmux socket `mosaic-fleet`
## Non-goals
- No production rollout.
- No migration of existing default tmux sessions.
- No image build/deploy work.
- No hardcoded USC/local roster as product default.
## Acceptance Criteria
- CLI can initialize a minimal roster outside product defaults.
- CLI can install user systemd units and fleet helper scripts to a configurable Mosaic home.
- CLI can start/stop/status/verify a canary fleet using `mosaic-fleet`.
- `mosaic agent send` uses existing named-socket/exact-target tmux tooling.
- `mosaic agent reset` targets only the named agent session on the named socket.
- Verification proves default tmux sessions remain untouched.
- Baseline repo gates pass.
- PR CI is green before merge.
- Local canary evidence is captured after merge/install.
## Budget / Routing
- Agent: codex preferred.
- Estimate: 25K-40K tokens.
- Worker owns implementation/tests/docs in branch `feat/fleet-cli-local-canary`.
- Orchestrator owns `docs/TASKS.md`, issue/PR/merge, and local canary install verification.
## Progress
- 2026-06-20: #557 PoC primitives merged to `main` as `45e2c2a`.
- 2026-06-20: issue #562 created for local CLI canary dogfood.
- 2026-06-20: worktree created at `/home/jarvis/src/mosaicstack-stack-worktrees/fleet-cli-local-canary`.
docs/archive/work-records/2026-06-20-fleet-release-hardening.md
+35
View File
@@ -0,0 +1,35 @@
# Fleet release hardening
## Objective
Harden the Mosaic local fleet release path for operator sends, tmux/systemd verification, package contents, and dogfood release documentation.
## Constraints
- Do not edit `docs/TASKS.md`.
- Do not change production deployment refs.
- Keep fleet transport generic and named-socket safe.
- Preserve strict roster validation.
- Add tests first or alongside fixes.
## Plan
1. Add regression tests for deterministic `mosaic agent send` source labels.
2. Strengthen fleet status/verify/package/install-systemd coverage.
3. Implement focused CLI/source-label changes.
4. Update local canary documentation with dogfood preflight.
5. Run formatting, targeted tests, typecheck, lint, and package dry-run evidence.
## Evidence Log
- Started from existing `docs/PRD.md`; durable local fleet canary is in v0.1.0 scope.
- Loaded `mosaic-fleet-operations` skill; key constraints are isolated tmux sockets, no default tmux positive tests, and `active (exited)` is not liveness.
- TDD red: `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet.spec.ts` initially failed because `node_modules` was absent; after `pnpm install`, the new source-label tests failed on missing `-S`, missing helper, and unknown `--source-label`.
- Green implementation: `mosaic agent send` now passes `-S <hostname>:operator` by default and accepts `--source-label` / `--source` overrides.
- Test coverage added for tmux-based fleet verify liveness, package `files` allowlist containing `framework`, and explicit operator source-label command construction.
- Formatting: `pnpm exec prettier --write packages/mosaic/src/commands/fleet.ts packages/mosaic/src/commands/fleet.spec.ts docs/guides/fleet-local-canary.md docs/scratchpads/2026-06-20-fleet-release-hardening.md`.
- Targeted tests: `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet.spec.ts src/cli-smoke.spec.ts` passed with 49 tests.
- Typecheck: `pnpm typecheck` passed.
- Lint: `pnpm lint` passed.
- Package dry-run: `npm pack --dry-run --json` from `packages/mosaic` included `framework/fleet`, `framework/systemd/user`, `framework/tools/fleet/start-agent-session.sh`, and `framework/tools/tmux/{agent-send.sh,send-message.sh}`.
- Review: `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` approved the supplied diff with no findings; the review tool noted its read-only sandbox could not inspect files directly.
docs/archive/work-records/387-updater-wrapper-gitea-20260404.md
+54
View File
@@ -0,0 +1,54 @@
# Scratchpad — issue #387 updater simplification + Gitea wrapper repo context
- Objective: simplify updater behavior to `@mosaicstack/mosaic` only, and fix Gitea wrapper scripts so merge/CI/issue operations work reliably when tea needs explicit repo/login context.
- Scope:
- `packages/mosaic/src/runtime/update-checker.ts`
- `packages/mosaic/__tests__/update-checker.test.ts`
- any package metadata/version bumps needed
- repo-source git wrappers under `packages/mosaic/framework/tools/git/`
- Constraints:
- Jason approved breaking changes; legacy `@mosaicstack/cli` support is out of scope.
- Keep changes focused and mergeable.
- Acceptance:
- updater only targets `@mosaicstack/mosaic`
- wrapper path works on Gitea in this environment without manual repo guessing
- PR merges squash-only after green CI
## Progress
### 2026-04-04
#### Update Checker Simplification (DONE)
- Removed `LEGACY_PKG`, `INSTALLED_PACKAGE_ORDER` constants — only `PKG = '@mosaicstack/mosaic'` remains
- `getInstalledVersion()` — removed loop over multiple packages, only checks `@mosaicstack/mosaic`
- `getLatestVersion()` — removed `installedPackage` parameter and candidate iteration; single query for `@mosaicstack/mosaic`
- `checkForUpdate()` — removed `currentPackage`-based cache key comparison; cache is now package-agnostic
- `RegistryCache` — removed `currentPackage` and `targetPackage` fields
- `formatUpdateNotice()` — removed `targetChanged` branch (package migration notice no longer relevant)
- All legacy fallback/compatibility logic removed
#### Test Updates (DONE)
- Removed legacy `@mosaicstack/cli` fallback test
- Removed cache cross-contamination test (was testing legacy→modern package transition)
- Added `does not query legacy @mosaicstack/cli package` test — asserts no `@mosaicstack/cli` npm commands are issued
- Added `returns empty result when package is not installed` test
- All 8 tests pass ✅
#### Gitea Wrapper Fixes (DONE)
- `pr-merge.sh`:
- Added `OWNER=$(get_repo_owner)` / `REPO=$(get_repo_name)` before case block
- tea merge command now includes `--repo $OWNER/$REPO --login ${GITEA_LOGIN:-mosaicstack}`
- `issue-close.sh`:
- Added `OWNER=$(get_repo_owner)` / `REPO=$(get_repo_name)` after detect_platform
- Both `tea issue comment` and `tea issue close` now include `--repo "$OWNER/$REPO" --login "${GITEA_LOGIN:-mosaicstack}"`
- `pr-ci-wait.sh`: No changes needed — uses curl API calls (not tea), already passes owner/repo correctly
- `detect-platform.sh`: No changes needed — provides the `get_repo_owner`/`get_repo_name` functions used by wrappers
#### Verification
- `vitest run` — 8/8 tests pass
- `tsc --noEmit` — no errors in update-checker.ts (pre-existing workspace dep errors unrelated)
- `eslint` — clean, no warnings
docs/archive/work-records/41-plugin-host.md
+30
View File
@@ -0,0 +1,30 @@
# Scratchpad — P5-001 Plugin Host
- Task: P5-001 / issue #41
- Branch: feat/p5-plugin-host
- Objective: add global NestJS plugin host module, wire Discord import, register active plugins from env, and attach to AppModule.
- TDD: skipped as optional for module wiring/integration work; relying on targeted typecheck/lint and module-level situational verification.
- Constraints: ESM .js imports, explicit @Inject(), follow existing gateway patterns, do not touch TASKS.md.
## Progress Log
- 2026-03-13: session started in worktree; loading gateway/plugin package context.
- 2026-03-13: implemented initial plugin module, service, interface, and AppModule wiring; pending verification.
- 2026-03-13: added `@mosaicstack/discord-plugin` as a gateway workspace dependency and regenerated `pnpm-lock.yaml`.
- 2026-03-13: built gateway dependency chain so workspace packages exported `dist/*` for clean TypeScript resolution in this fresh worktree.
- 2026-03-13: verification complete.
## Verification
- `pnpm --filter @mosaicstack/gateway... build` ✅
- `pnpm --filter @mosaicstack/gateway typecheck` ✅
- `pnpm --filter @mosaicstack/gateway lint` ✅
- `pnpm format:check` ✅
- `pnpm typecheck` ✅
- `pnpm lint` ✅
## Review
- Automated review: `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted`
- Manual review: diff inspection of gateway plugin host changes
- Result: no blocker findings
docs/archive/work-records/462-fed-m3-04-scope-service.md
+60
View File
@@ -0,0 +1,60 @@
# Scratchpad — FED-M3-04 Scope Service
## Objective
Implement `apps/gateway/src/federation/server/scope.service.ts` for the M3 inbound federation scope-enforcement pipeline.
## Scope / Constraints
- Task: FED-M3-04, issue #462.
- Branch: `feat/federation-m3-scope-service` from `origin/main` @ 0.0.48.
- Pure service: no direct DB access; native RBAC/data access is injected per evaluation call.
- Reuse `parseFederationScope` from M2-03.
- Workers do not edit `docs/federation/TASKS.md` per repo AGENTS.md.
## Acceptance Criteria
1. Resource allowlist and `excluded_resources` enforced.
2. Native RBAC evaluated as `subjectUserId` through an injected evaluator.
3. Scope filter intersection supports `include_teams` and `include_personal` without widening native RBAC.
4. `max_rows_per_query` caps requested limits.
5. Service returns `{ allowed: true, filter }` or a structured deny reason usable by M4 audit.
6. Unit tests cover every deny path.
## Plan
1. Inspect existing federation scope/schema/auth guard contracts.
2. Add pure `FederationScopeService` plus typed result/filter/deny interfaces.
3. Add focused unit tests for happy paths, filter intersection, row cap, and deny paths.
4. Export/register service for future verb controllers.
5. Run situational tests, baseline gates, code review, then PR.
## Budget
- Provided model tier: sonnet.
- Estimate from task row: 10K tokens.
- Working cap assumption: keep implementation focused to FED-M3-04 surfaces only.
## Progress
- Intake complete; dirty base worktree avoided by creating isolated worktree at `/home/jarvis/src/mosaic-mono-v1-fed-m3-04`.
- Project PRD and federation task spec reviewed.
- Added `FederationScopeService` with structured allow/deny result types and injected native RBAC evaluator contract.
- Added unit coverage for happy path, row cap, filter intersection, and every deny path.
- Exported/registered the service for upcoming M3 verb controllers.
## Verification Evidence
- `pnpm --filter @mosaicstack/gateway test -- src/federation/server/__tests__/scope.service.spec.ts` — pass (10 tests before review update; 11 tests after adding include_personal no-leak coverage).
- `pnpm build` — pass (23 successful tasks).
- `pnpm typecheck` — pass (41 successful tasks; re-run after review update).
- `pnpm lint` — pass (23 successful tasks; re-run after review update).
- `pnpm format:check` — pass (re-run after review update).
- `pnpm test` — pass after starting local `postgres`/`valkey` and running `pnpm --filter @mosaicstack/db db:push` for the DB-backed cross-user isolation suite (41 successful tasks; gateway 477 passed / 11 skipped).
- Code review: `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — approve, 0 findings.
- Security review: `~/.config/mosaic/tools/codex/codex-security-review.sh --uncommitted` — risk none, 0 findings.
## Risks / Blockers
- Issue #462 is already closed in provider output; likely milestone tracking mismatch. Will still reference #462 in PR body unless orchestrator redirects.
- Local full-test setup required `docker compose up -d postgres valkey` + `db:push`; containers were stopped with `docker compose down` after verification.
docs/archive/work-records/462-fed-m3-06-get-verb.md
+38
View File
@@ -0,0 +1,38 @@
# Scratchpad — FED-M3-06 get verb
## Objective
Implement `POST /api/federation/v1/get/:resource/:id` for M3 inbound federation reads.
## Scope
- `apps/gateway/src/federation/server/verbs/get.controller.ts`
- `apps/gateway/src/federation/server/verbs/get-query.service.ts`
- Unit coverage for controller pipeline + query service RBAC guardrails
- Register controller/service in `FederationModule`
## Plan
1. Mirror the list verb pipeline: `FederationAuthGuard` → `FederationScopeService` → read-only query service.
2. Return one `_source: "local"` tagged item on success.
3. Return federation error envelopes:
- `404 not_found` when the resource id does not exist.
- `403 scope_violation` when the row exists but falls outside native RBAC/scope intersection.
- `400 invalid_request` for malformed ids/scope requests.
4. Keep read audit persistence deferred to M4; no body or response persistence in M3.
## Verification Evidence
- Rebased onto `origin/main` at `86e106fcc9a1dfa3a18f7846bb477be128794aad` after M3-05 merged; resolved `FederationModule` by registering both list and get verb controllers/services.
- Review-change coverage added for comment 15971:
- get note access now requires subject ownership AND authorized mission intersection.
- missing federation context returns structured `401 unauthorized` envelope.
- unsupported get resources fail closed with structured denial.
- PGlite regressions cover cross-user note exclusion and subject-note unauthorized-mission exclusion.
- `pnpm --filter @mosaicstack/gateway test -- src/federation/server/verbs/__tests__/get.controller.spec.ts src/federation/server/verbs/__tests__/get-query.service.spec.ts` — pass (2 files / 17 tests; re-run after review changes).
- `pnpm --filter @mosaicstack/gateway build` — pass (re-run after review changes).
- `pnpm build` — pass (23 successful tasks before review changes).
- `pnpm typecheck` — pass (41 successful tasks; re-run after review changes).
- `pnpm lint` — pass (23 successful tasks; re-run after review changes).
- `pnpm format:check` — pass (re-run after review changes).
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — approve, 0 findings after review changes.
docs/archive/work-records/536-wrapper-login-pin.md
+50
View File
@@ -0,0 +1,50 @@
# Issue 536 Wrapper Login Pin Scratchpad
## Metadata
- Date: 2026-06-12
- Worktree: `/home/hermes/agent-work/536-wrapper-audit`
- Branch: `fix/536-wrapper-login-pin`
- Coordinator: `mos-claude`
- Issue: `mosaicstack/stack#536`
- Scope: Audit and fix Gitea git wrappers that hardcode or incorrectly inherit tea login/instance selection.
## Objective
Fix the framework git wrappers so Gitea issue/PR operations resolve the tea login from the target repository host instead of pinning `mosaicstack`. The fix must cover the class of bug across `packages/mosaic/framework/tools/git/`, not only `issue-close.sh`.
## Acceptance Criteria
1. `issue-close.sh` no longer uses `--login mosaicstack` for non-mosaic hosts.
2. All wrappers in `packages/mosaic/framework/tools/git/` avoid hardcoded Gitea login fallback where host-specific resolution is available.
3. Host-specific resolution works for `git.mosaicstack.dev` and `git.uscllc.com` using configured credentials / tea login data.
4. Read-only verification runs against both Gitea instances where possible.
5. Queue guard passes before push, PR is opened referencing #536, and merge is left to the coordinator.
## Progress Log
- Read required Mosaic hard-gate docs and coordinator briefing.
- Read issue #536 via Gitea API with mosaicstack credentials.
- Initial audit found hardcoded `${GITEA_LOGIN:-mosaicstack}` in issue and PR wrappers, plus shared `get_gitea_repo_args`.
- Added host-aware Gitea login resolution in `detect-platform.sh`, including exact host matching for `tea login list` entries and HTTPS remotes with embedded credentials.
- Updated Gitea issue, PR, milestone, and CI wrappers to use resolved host-specific tea login arguments instead of defaulting to `mosaicstack`.
- Added authenticated API fallbacks for close/reopen paths so wrappers can still operate when a matching `tea` login is absent but token credentials are available.
- Added regression coverage for stale `GITEA_LOGIN`, exact host matching, `--repo` override flows, USC issue close routing, mosaicstack API fallback, and PR metadata/merge fallbacks.
- Delta after PR #538 review: extended host-aware login/repo resolution to PowerShell wrappers, Bash milestone wrappers, and API-only `--repo` fallback paths.
- Delta after live USC `pr-create.sh` repro: tightened `GITEA_LOGIN` trust so stale login names are ignored unless the tea login itself matches the target host, and added USC API fallback coverage for `pr-create.sh`.
## Verification
- `bash -n packages/mosaic/framework/tools/git/*.sh`
- `packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh`
- `packages/mosaic/framework/tools/git/test-pr-metadata-gitea.sh`
- `packages/mosaic/framework/tools/git/test-pr-merge-gitea-empty-uid.sh`
- `pwsh -NoProfile` parse check for all `packages/mosaic/framework/tools/git/*.ps1`
- `pnpm typecheck`
- `pnpm lint`
- `pnpm format:check`
- `pnpm --filter @mosaicstack/mosaic test -- src/commands/git-wrapper-redirects.spec.ts`
- `pnpm test` progressed past wrapper redirect assertions; local run then stopped on `apps/gateway` Postgres connection refused at `localhost:5433`, which CI provides as a service.
- Live read-only: direct Gitea API read of `mosaicstack/stack#536` with `User-Agent: curl/8`.
- Live read-only: USC temporary repo remote to `https://git.uscllc.com/USC/uconnect.git`; `issue-list.sh -n 1` resolved the USC login and returned USC issues.
- Independent Codex review final verdict: approve, no findings.
docs/archive/work-records/544-agent-reflection-loop.md
+55
View File
@@ -0,0 +1,55 @@
# Scratchpad — #544 Agent Reflection Loop (durable kernel)
**Started:** 2026-06-16 · **Branch:** `feat/agent-reflection-loop` · **Base:** `main` @ c461380
## Goal
Bake the durable kernel of the agent reflection loop into the Mosaic Stack
monorepo through full delivery gates. Kernel only; closed loop (§7–§8) gated on
Phase-0. Authoritative spec: `docs/plans/agent-reflection-loop-PRD.md`. Task
breakdown: `docs/tasks/544-agent-reflection-loop.md`.
## Timeline / decisions
- Mapped house style against `main` truth (the earlier recon had mapped a dirty
feature branch and returned non-existent paths; re-cloned `main` clean).
- macp uses co-located `*.spec.ts`; types uses `src/<mod>/{*.ts, *.dto.ts, __tests__/*.spec.ts}`.
- zod v4 + class-validator/class-transformer present in `@mosaicstack/types`;
`packages/types/tsconfig.json` enables `experimentalDecorators`/`emitDecoratorMetadata`.
- **Gotcha (fixed):** `class-transformer`'s `@Type` calls `Reflect.getMetadata`
at module-load time; the types vitest env has no `reflect-metadata`, so any test
importing the reflection barrel crashed on import. `chat.dto.ts` avoids this by
using class-validator only. Fix: dropped `@Type`/`@ValidateNested` from the DTO;
zod owns deep nested validation.
- **Gotcha (fixed):** Stop hook `EXIT` trap referenced a `main`-local `lock` →
`unbound variable` under `set -u` at exit. Promoted to a global `LOCKFILE`.
- **Gotcha (fixed):** the hook's own lock + `.mosaic/` scratch leaked into
`files_changed`. Excluded `^\.mosaic/` from the change-surface scan.
## Verification evidence
- macp: typecheck OK, lint OK, **88 tests pass** (15 new risk-floor).
- types: typecheck OK, lint OK, **64 tests pass** (10 new reflection).
- Root: `pnpm typecheck` (41 tasks), `pnpm lint` (23), `pnpm format:check`, `pnpm build` (23) — all green.
- Stop hook smoke (throwaway git repo): TEST1 no-op (mode unset, 0 files);
TEST2 solo degraded, `.mosaic/` excluded, auth→needs_review; TEST3 self-report
merged, degraded=false; TEST4 lock suppresses re-fire. All pass, always exit 0.
- shellcheck clean: hook + `reflect-{git-history,board-history,calibration}.sh`.
- Phase-0 smoke: P2 on this repo (142 failures classified), P1 AUC=0.875 on a
synthetic fixture, P3 base-rate on a synthetic board — all emit structured output
- kill conditions.
## Open risks / follow-ups
- Full `pnpm test` (DB-bound packages) validated via CI's postgres service, not
locally; affected packages (macp, types) are DB-independent and green here.
- sequential-thinking MCP was registered mid-session (effective next session);
this session compensated with the written PRD as the planning artifact.
- Phase-0 corpora are not yet wired — scripts are harnesses + pre-registered
rubrics (P1/P2/P3 tasks tracked in jarvis-brain `agent-reflection-loop` project).
## Gate status
- [x] PRD authored · [x] issue #544 created + linked · [x] code + tests
- [x] local gates green · [ ] independent code review · [ ] PR opened
- [ ] CI terminal green · [ ] merged to main · [ ] issue closed
docs/archive/work-records/559-560-wrapper-eval-login-20260620.md
+87
View File
@@ -0,0 +1,87 @@
# Wrapper hardening fold-in: #559 (eval removal) + #560 (host-derived login)
**Branch:** `fix/wrapper-hardening-tls-credpath-cicwait` (PR #551)
**Worker:** coderlite0 (Sonnet lane) · coordinated by mos-claude
**Date:** 2026-06-20
**Scope:** `packages/mosaic/framework/tools/git/*.sh` only
## What the issues asked for vs. what was already landed
Both issues were largely satisfied by prior merged work; this fold-in closes the
remaining gaps (regression tests + a loud diagnostic + one residual word-split site)
rather than re-implementing finished functionality.
### #559 — remove `eval` from issue-create.sh (and siblings)
- `eval`-based command construction was already removed across the wrapper surface
(landed in #549). A full scan of `tools/git/*.sh` finds **zero** `eval` usages.
- `issue-create.sh`, `pr-create.sh`, `issue-edit.sh`, `issue-assign.sh` already build
their `tea`/`gh` invocations as argv arrays (`CMD=(...)`, `"${CMD[@]}"`), so Markdown
bodies pass through verbatim.
- **Residual found & fixed:** `issue-comment.sh` still used unquoted
`$(get_gitea_repo_args)` word-splitting (the comment body itself was already safely
quoted, so no injection bug — but it was the inconsistent, fragile pattern #559 targets,
and it failed silently when no login resolved). Converted to an argv array with an
explicit, loud login-resolution error.
- **Added regression test:** `test-issue-create-body-safety.sh` — feeds a hostile
Markdown body (`$(touch SENTINEL)`, backticks, single/double quotes, `$HOME`/`${PATH}`,
pipes/`&&`/`;`) through `issue-create.sh` and asserts (1) no command substitution
executes (sentinel file never created) and (2) the `--description` `tea` receives is
byte-for-byte the original body.
### #560 — auto-detect Gitea `--login` from repo origin host
- Centralized host→login resolution already exists in `detect-platform.sh`
(`get_gitea_login_for_host` → `find_tea_login_for_host`, matching `urlparse(url).hostname`).
Every wrapper routes through it (or `get_gitea_login` / `get_gitea_login_for_repo_override`);
**no wrapper hardcodes `${GITEA_LOGIN:-mosaicstack}`**. Explicit `GITEA_LOGIN` wins only
when it matches the host (`tea_login_matches_host`), so stale overrides are rejected.
- **Gap fixed — silent failure → loud diagnostic:** the failure path of
`get_gitea_login_for_host` returned non-zero with no message. Added
`print_gitea_login_diagnostic`, emitted to **stderr** on resolution failure: names the
unresolved host, lists available tea logins (name + host), and gives the `GITEA_LOGIN`
override + `tea login add` fix. Stderr-only, so it never contaminates stdout (the
resolved login name) or the log-grep assertions in the existing harnesses. Callers with
an API fallback (pr-merge, issue-close, pr-create, issue-create) still follow with their
own "using API fallback" line, giving a clear "no login → fallback" trail.
- **Extended test:** `test-gitea-login-resolution.sh` now also asserts (a) the loud
diagnostic fires and lists available logins for an unresolved host, (b) login is derived
from origin host for **both** instances (mosaicstack + usc) via a scoped second `tea`
mock, and (c) a valid `GITEA_LOGIN` override is honored. The scoped mock keeps the
existing API-fallback assertions (which require mosaicstack to have _no_ tea login) valid.
## Files changed (wrapper surface only)
- `detect-platform.sh` — add `print_gitea_login_diagnostic`; call it on the
`get_gitea_login_for_host` failure path.
- `issue-comment.sh` — argv array + loud login-resolution error (was unquoted
`$(get_gitea_repo_args)`).
- `test-issue-create-body-safety.sh` — **new** (#559 regression).
- `test-gitea-login-resolution.sh` — extended (#560 diagnostic + both-host + override).
## Verification
All wrapper harnesses pass locally:
- `test-issue-create-body-safety.sh` — PASS
- `test-gitea-login-resolution.sh` — PASS
- `test-pr-merge-gitea-empty-uid.sh` — PASS
- `test-pr-metadata-gitea.sh` — PASS
- `test-lane-brief-pr-linkage.sh` — PASS
## Open items flagged to mos-claude (orchestrator decisions)
1. **CHANGELOG absent.** The task said "update CHANGELOG (append-only), keep the existing
#550/#551 entry." No CHANGELOG file exists anywhere in the repo, and #550/#551 are not
recorded in one. **ASSUMPTION:** documenting #559/#560 in this scratchpad + the PR
description (`Closes #559 Closes #560`) follows the repo's actual convention
(`docs/scratchpads/`). Did not invent a new CHANGELOG structure.
2. **`docs/TASKS.md` is orchestrator single-writer.** It carries a "Workers read but never
modify" banner. As a worker I did **not** edit it; task tracking is via the linked Gitea
issues #559/#560 + this scratchpad. Orchestrator may add a rollup row if desired.
3. **Wrapper `test-*.sh` are not CI-wired.** `.woodpecker/ci.yml` runs `pnpm
typecheck/lint/format:check/test` (`turbo run test`); the framework dir has no
`package.json`, so these shell harnesses run **locally/manually only** — they do not gate
the PR in Woodpecker. **ASSUMPTION:** out of scope to wire a shell-test step into CI in
this PR (would broaden the diff beyond the wrapper surface). Flagging for a follow-up if
the fleet wants these gated.
docs/archive/work-records/561-python-is-python3.md
+33
View File
@@ -0,0 +1,33 @@
# Issue #561 — Bare python on agent hosts
## Objective
Make the durable bootstrap/provisioning guidance ensure agent hosts provide a bare `python` command that resolves to Python 3.
## Scope
- Add Debian/Ubuntu `python-is-python3` to agent-host prerequisites in bootstrap docs.
- Check for actual OS package provisioning scripts and update only if an existing agent-host package install path exists.
- Do not touch live host state.
- Do not update `docs/TASKS.md`; repo guidance says workers read it but never modify it.
## Recon
- Issue #561 confirms repeated `python: command not found` failures from fleet agents that emit `python foo.py`.
- `guides/BOOTSTRAP.md` and `packages/mosaic/framework/guides/BOOTSTRAP.md` are the source and packaged framework copies of the bootstrap guide.
- Targeted repo sweep found no agent-host Debian package provisioning script. Existing `apt-get install` hits are CI/test helper paths or unrelated deployment docs.
## Plan
1. Add a host prerequisite section to both bootstrap guide copies.
2. Include `python-is-python3` in the Debian/Ubuntu package list with an issue comment.
3. Note the non-Debian equivalent as a `/usr/bin/python -> python3` symlink.
4. Validate markdown/diff, run shell syntax checks where applicable, run required review, commit, queue guard, and push.
## Validation Log
- `rg` recon: no existing agent-host Debian package provisioning script; only CI/test helper `apt-get install` paths and unrelated deployment docs.
- `git diff --check`: passed.
- `bash -n packages/mosaic/framework/install.sh tools/install.sh packages/mosaic/framework/tools/bootstrap/init-project.sh packages/mosaic/framework/tools/_scripts/mosaic-bootstrap-repo`: passed. No touched shell scripts.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted`: approved, 0 findings.
- `pnpm format:check`: initially blocked because `node_modules` was absent and `prettier` was unavailable; `pnpm install --frozen-lockfile` initially hit an invalid `/root` pnpm store path. Reran install with `--store-dir /home/hermes/agent-work/.pnpm-store`, then `pnpm format:check` passed.
docs/archive/work-records/672-fleet-personas-timeout.md
+25
View File
@@ -0,0 +1,25 @@
# Scratchpad — fleet-personas spec timeout
## Objective
Raise the `@mosaicstack/mosaic` Vitest timeout to 30s at config level so filesystem-backed fleet drift-guard specs (`fleet-personas`, `fleet-profiles`, and siblings) stop false-reding under contended CI.
## Plan
1. Move timeout policy into `packages/mosaic/vitest.config.ts` with `testTimeout: 30_000`.
2. Remove the narrower `fleet-personas.spec.ts` local override so PR #677 fixes the suite class, not one file.
3. Run targeted fleet specs plus typecheck/lint/format gates.
4. Commit, queue guard, push, PR update.
## Evidence
- `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet-personas.spec.ts` — pass (8 tests; initial narrow fix).
- `pnpm typecheck` — pass (41 tasks; initial narrow fix).
- `pnpm lint` — pass (23 tasks; initial narrow fix).
- `pnpm format:check` — pass after formatting this scratchpad (initial narrow fix).
- Package-wide timeout follow-up:
- `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet-personas.spec.ts src/commands/fleet-profiles.spec.ts` — pass (24 tests).
- `pnpm --filter @mosaicstack/mosaic test` — pass (44 files / 618 tests).
- `pnpm typecheck` — pass (41 tasks).
- `pnpm lint` — pass (23 tasks).
- `pnpm format:check` — pass.
docs/archive/work-records/703-wrapper-interactive-auth.md
+49
View File
@@ -0,0 +1,49 @@
# #703 Git Wrapper Interactive and Auth Resilience
## Objective
Restore the deployed Git wrapper contract: issue-create supports interactive invocation and Gitea mutation behavior tolerates a stale Tea authenticated user by validating current identity and using the existing host-scoped API fallback.
## Scope
- `packages/mosaic/framework/tools/git/issue-create.sh`
- `packages/mosaic/framework/tools/git/detect-platform.sh`
- Git wrapper regression harnesses
- This scratchpad
## Requirements / acceptance evidence
1. `issue-create -i` and `--interactive` prompt for missing issue fields without exposing credentials.
2. Explicit command-line fields retain precedence and do not trigger prompt input.
3. Gitea wrapper resolves the current user dynamically from the target host and does not rely on the saved Tea user identity.
4. A Tea `GetUserByName` failure falls back to authenticated API creation.
5. Existing body-safety, login-resolution, issue-create, and pr-create paths remain green.
6. Source framework is re-seeded to deployed `~/.config/mosaic`, then deployed wrappers are verified end to end.
## Plan
1. Add failing shell regression harness for interactive input and stale Tea user fallback.
2. Implement minimal helper and parser changes.
3. Run wrapper harnesses, syntax checks, and repository baseline checks.
4. Re-seed deployed framework and run live wrapper verification.
5. Commit, queue guard, push, open PR, and stop for independent review.
## Progress
- Issue #703 filed before code; issue comment records #536 root cause and stale-login trigger.
- Deployed wrapper `issue-create.sh -i` reproduced: `Unknown option: -i` (exit 1).
- Live Tea mutation did not reproduce `GetUserByName` on this host because the current mosaicstack Tea login is valid. The test harness models the reported stale authenticated-user condition.
- Implemented `-i` / `--interactive` prompt collection and a dynamic Tea `/user` validation. A stale Tea identity now selects the existing host-scoped Gitea API fallback before mutation for both issue and PR creation.
- Re-seeded the framework with `MOSAIC_INSTALL_MODE=keep MOSAIC_SYNC_ONLY=1 bash packages/mosaic/framework/install.sh`. Installed and source wrapper SHA-256 values matched.
- Live deployed verification: interactive issue-create opened then closed #704; installed dynamic identity resolved `jason.woltje`.
## Verification
- PASS: `packages/mosaic/framework/tools/git/test-issue-create-interactive-auth.sh`
- PASS: `packages/mosaic/framework/tools/git/test-issue-create-body-safety.sh`
- PASS: `packages/mosaic/framework/tools/git/test-gitea-login-resolution.sh`
- PASS: `packages/mosaic/framework/tools/git/test-pr-metadata-gitea.sh`
- PASS: `packages/mosaic/framework/tools/git/test-pr-merge-gitea-empty-uid.sh`
- PASS: `bash packages/mosaic/framework/tools/git/test-lane-brief-pr-linkage.sh`
- PASS: `bash -n packages/mosaic/framework/tools/git/*.sh`
- PASS: Prettier check for this scratchpad
docs/archive/work-records/747-wsa-dehardcode.md
+43
View File
@@ -0,0 +1,43 @@
# #747 — De-hardcode orchestrator and interaction agent names
## Objective
Replace branded Mos/Tess symbols, filenames, DI tokens, and error prose with role-neutral orchestrator/interaction vocabulary without changing env-driven runtime identity behavior. Add optional roster `alias` and `provider` fields and show aliases in `mosaic fleet ps` with name fallback.
## Scope and constraints
- Requirements: `/home/hermes/agent-work/reviews/747-wsa-dehardcode-brief.md`.
- Branch: `feat/747-dehardcode-orchestrator-interaction-names` from `main` at `e72388b2`.
- Keep `MOSAIC_AGENT_NAME` and `MOSAIC_ORCHESTRATOR_AGENT_NAME` unchanged.
- Sample/test data may retain operator display names.
- No behavior change beyond optional roster metadata and alias display.
- Budget: no explicit cap; conservative mechanical-rename scope only.
- TDD: optional and skipped because this is a mechanical rename with existing focused coverage; add focused alias/schema regression coverage before completion.
## Plan
1. Rename coordination and durable-session files and symbols using canonical vocabulary.
2. Scrub branded symbol names and error prose in the assigned source trees while preserving allowed sample data.
3. Extend roster schema with optional `alias` and `provider`; update fleet roster typing/rendering and focused tests.
4. Run grep-clean verification, build, typecheck, lint/format, focused coord/durable-session/fleet tests, and roster validation.
5. Commit, queue-guard, push, open a Gitea PR closing #747, and report to the coordinator.
## Progress
- 2026-07-13: Task resumed from coordinator brief; repository clean at `e72388b2`.
- Renamed coordination and durable-session files, exports, gateway DI symbols, DTOs, services, repositories, and tests.
- Replaced branded authority/error prose while preserving the existing `/api/coord/mos` compatibility route and env-variable identity inputs.
- Added optional roster `alias`/`provider` support and alias-first `fleet ps` display with canonical-name fallback.
## Verification
- `pnpm typecheck`: passed (42 tasks).
- `pnpm build`: passed (23 tasks).
- `pnpm lint`: passed (23 tasks).
- `pnpm format:check`: passed.
- Coordination tests: 7 passed.
- Agent durable-session/runtime tests: 23 passed.
- Gateway coordination/durable-session/integration tests: 20 passed.
- Full `fleet.spec.ts`: 192 passed, including alias/provider parsing and alias display.
- JSON Schema 2020 validation: legacy minimal roster and extended alias/provider roster passed; `alias` and `provider` remain absent from `required`.
- Grep verification: no branded symbol/type/file/DI names or error prose remain in assigned source trees; one allowed `Tess Owner` test-data display name remains.
docs/archive/work-records/751-native-kanban-canon.md
+152
View File
@@ -0,0 +1,152 @@
# Issue #751 — Native Kanban/SOT canonical publication
## Objective
Publish the owner-ratified P0–P3 requirements, mission manifest, task decomposition, and frozen shared contracts before feature implementation.
## Authority and decisions
- Owner: Jason
- Plan owner/orchestrator: web1 control plane; takeover by mosaic-100 during Claude quota outage
- Tracking: Mosaic Stack issue #751
- Foundation: current Stack main + Drizzle/PostgreSQL
- Fixed invariants: PostgreSQL sole writable SOT; writes fail closed; exports never import; outage notes become attributable proposals; mechanical Coordinator has no scope/gate/certify/merge authority; Certifier has no merge authority.
- Recovery posture only is configurable through Lite, Standard, and High-assurance profiles.
## Execution log
- 2026-07-14: Existing planner-sol canon remediation reviewed from staging. KCR-001–016 claimed resolved; static checks passed.
- 2026-07-14: Independent GPT/Terra re-review dispatched to rev1.
- 2026-07-14: Re-review returned NO-GO: proposal audit-event IDs were not workspace-bound, leaving attribution forgeable; formatter evidence was not reproducible. Focused remediation round 2 routed to planner-sol.
- 2026-07-14: Remediation bound proposal audit links to `task_events(workspace_id,id)`, froze same-transaction semantic validation and negative tests, and made formatter/type/static checks reproducible.
- 2026-07-14: Independent rev1 re-review returned GO with KCR-001–016 closed and no new blocker. Canon copied into the issue #751 publication worktree; feature implementation remains held until merge.
- 2026-07-14: Independent publication validation returned FAIL on formatting/trailing whitespace, stale staging wording, ignored review evidence, and missing worktree dependencies. Bounded publication remediation routed to planner-sol; no runtime source change authorized.
- 2026-07-14: Publication remediation installed locked dependencies outside the repository cache, fixed formatting and wording, and preserved docs-only scope. Independent gaterun revalidation returned PASS across staged scope, formatting, lint, typecheck, strict contract compile, links, rollups, review artifacts, and fixed invariants.
- 2026-07-14: Ultron final gate returned GO with zero BLOCKER/HIGH findings; residual LOW items remain explicit implementation obligations.
- 2026-07-14: First commit attempt was correctly blocked by the lint-staged hook because docs contract `.ts` files were outside TypeScript project-service scope. Added a strict no-emit workstream `tsconfig.json` with exact Drizzle declaration paths; targeted contract TSC, contract ESLint, format, full lint/typecheck, strictness, and docs-only scope independently passed.
## Verification evidence
- Initial review: `docs/reports/native-kanban-sot/canon-initial-review-no-go.md`
- Final GO: `docs/reports/native-kanban-sot/canon-final-rereview-go.md`
- Ultron GO: `docs/reports/native-kanban-sot/ultron-final-go.md`
- Pending: PR gates, squash merge, terminal-green CI, and issue closure.
## Publication remediation session — planner-sol
**Assignment:** Remediate only issue #751 publication blockers from `/home/hermes/agent-work/reviews/751-kanban-canon-publication-validation.md`; no source, package, lock, provider, CI, build, deploy, commit, or push action.
**Plan:**
1. Install the locked pnpm dependencies without modifying package metadata.
2. Replace only stale publication/staging wording while retaining the canon-merge and KBN prerequisite implementation hold.
3. Format all changed candidate Markdown and contract TypeScript; remove trailing whitespace.
4. Run worktree `format:check`, `lint`, `typecheck`, strict no-emit contract compilation, tracked/untracked whitespace checks, and static invariants.
5. Verify package/lock/source paths are unchanged and append exact evidence here.
**Budget:** No explicit token cap; bounded docs-only remediation, no exploratory/runtime work.
**TDD:** Not applicable—documentation/contract-publication formatting and wording only; strict/static validation is the primary gate.
## Publication remediation results
### Changes
- `docs/native-kanban-sot/INDEX.md`: replaced staging/pending-GO wording with current publication and independent-GO wording; retained the merge hold and dependency-ordered KBN prerequisite hold.
- `docs/native-kanban-sot/TASKS.md`: replaced “Mos using this staging set” with “Mos / publication control plane”; made the post-merge KBN prerequisite hold explicit.
- Formatted all changed candidate Markdown and four contract TypeScript files with current-worktree Prettier 3.8.1.
- Removed trailing whitespace from candidate Markdown, including both linked review reports.
- Preserved both review reports and their links; they remain ignored by `.gitignore:11` for coordinator force-tracking.
### Dependency installation
The first target-worktree install attempt used the environment's default root-owned pnpm store and failed without changing package metadata:
```text
cd /home/hermes/agent-work/stack-kanban-canon && pnpm install --frozen-lockfile
EACCES: permission denied, open '/root/.local/share/pnpm/store/v10/server/server.json'
```
Successful locked install using an authorized cache outside the repository:
```bash
cd /home/hermes/agent-work/stack-kanban-canon
pnpm install --frozen-lockfile --store-dir /home/hermes/agent-work/pnpm-store
```
Result: PASS, 1,240 packages installed; lockfile resolution skipped as up to date. `node_modules` remains ignored. Tool versions: pnpm 10.6.2, Prettier 3.8.1, TypeScript 5.9.3, Drizzle ORM 0.45.1, Turbo 2.8.16.
### Exact quality-gate results
```text
pnpm format:check
PASS — All matched files use Prettier code style.
pnpm lint
PASS — 23 successful lint tasks.
pnpm typecheck
PASS — 42 successful tasks. Turbo invoked configured dependency build prerequisites as part of the repository's exact typecheck graph; no standalone build command was run.
```
Candidate formatting commands:
```bash
pnpm exec prettier --write <3 tracked rollups + 9 native-kanban artifacts + requirements + scratchpad>
pnpm exec prettier --check <same files>
pnpm exec prettier --ignore-path /dev/null --write \
docs/reports/native-kanban-sot/canon-initial-review-no-go.md \
docs/reports/native-kanban-sot/canon-final-rereview-go.md
pnpm exec prettier --ignore-path /dev/null --check \
docs/reports/native-kanban-sot/canon-initial-review-no-go.md \
docs/reports/native-kanban-sot/canon-final-rereview-go.md
```
Result: PASS. The explicit `/dev/null` ignore path is required because `docs/reports/` is intentionally ignored pending coordinator force-tracking.
Tracked and untracked whitespace checks:
```text
git diff --check
PASS
git diff --no-index --check /dev/null <each untracked/ignored candidate>
PASS for all candidates
```
Strict contract compilation initially could not resolve pnpm-isolated `drizzle-orm` from the external docs directory. A temporary, removed dependency-context symlink made current-worktree resolution explicit:
```bash
LINK=docs/native-kanban-sot/node_modules
ln -s ../../packages/db/node_modules "$LINK"
trap 'unlink "$LINK"' EXIT
pnpm exec tsc \
--noEmit \
--strict \
--skipLibCheck \
--target ES2022 \
--module NodeNext \
--moduleResolution NodeNext \
docs/native-kanban-sot/contracts/*.ts
```
Result: `strict-contract-noemit=PASS`; temporary link removed.
Static result:
```text
proposal-audit-links=PASS
kcr-invariant-regression=PASS
publication-wording=PASS
vocabulary-alignment=PASS
```
### Scope-integrity evidence
Baseline and final hashes are identical:
```text
package.json 93a50eaefc7a0446a56234e427df03f6a2256f8da17c0bede17c22206928c8c0
pnpm-lock.yaml 8b6448d51ac7797c8f782af52a080c0e38ab8bf364f32624f94e636bf5743229
```
`tracked-package-lock-source-unchanged=PASS`: every tracked/untracked nonignored change remains under `docs/`; no package, lock, application source, plugin source, configuration, CI trigger, standalone build/deploy, container, provider, commit, or push action occurred.
docs/archive/work-records/753-kbn010-threat-gate.md
+101
View File
@@ -0,0 +1,101 @@
# Issue #753 — KBN-010 threat, authorization, and constraint-impact gate
## Objective
Complete the mandatory threat/auth/constraint-impact analysis that gates KBN-100 schema implementation.
## Scope
- In: threat matrix, frozen-control mapping, schema/API/test impact inventory, security evidence plan.
- Out: runtime, schema, migration, API, dependency, CI, deployment, and secret changes.
- Canonical requirements: `docs/requirements/native-kanban-sot.md`.
- Frozen contract: `docs/native-kanban-sot/SHARED-CONTRACT.md` and `contracts/*.v1.ts`.
- Tracking: Mosaic Stack issue #753.
## Plan
1. Independently inspect current-main implementation and frozen canon.
2. Enumerate tenant, identity, health-proof, approval, fencing, proposal, audit, token, and outage threats.
3. Map every threat to required constraints, command behavior, negative tests, and owning future slice.
4. Surface any unresolved schema impact as a blocker; do not silently amend the frozen contract.
5. Run documentation/static validation and submit for independent SecReview.
## Execution log
- 2026-07-14: KBN-000 completed through PR #752 and post-merge pipeline #1798.
- 2026-07-14: KBN-010 issue #753 created; task marked in progress; fresh GPT worker pending dispatch.
## Verification evidence
Pending worker validation, independent SecReview, Ultron gate, PR merge, post-merge CI, and issue closure.
## 2026-07-14 worker analysis checkpoint
- Inspected issue #753, canonical requirements, all frozen v1 contracts, and actual `origin/main` at `49e8a54` across DB schema, Better Auth scope, project/task/mission/team repositories/controllers, fleet backlog, and `TASKS.md` parsing/writing.
- Confirmed the worker branch has no source/runtime/schema delta from `origin/main`; orchestrator-owned `.mosaic` state remains dirty and untouched.
- Authored the threat, authorization, constraint-impact, negative-test, and requirements-traceability analysis in `docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md`.
- Gate decision: **BLOCKED** by `KBN010-SI-001`. `missionsV1` lacks a unique candidate key on `(workspace_id, id)`, while `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk` both reference that exact pair. PostgreSQL cannot create the frozen composite foreign keys as declared.
- Decision: do not select or apply a schema fix. Contract authority must version either a `(workspace_id, id)` mission candidate key or project-congruent child keys, then obtain independent re-review before KBN-100.
- Additional risks are controlled by frozen transaction/API behavior but require the exact future negative tests cataloged in the deliverable, especially active membership, service-token revocation, same-workspace semantic evidence checks, no-oracle behavior, and serialized parent/DAG cycle checks.
- OpenBrain startup recall was attempted but unavailable because `/home/hermes/.config/mosaic/credentials.json` is absent; no project state was written to an alternate memory silo.
- TDD: not applicable; this slice changes documentation/test-plan analysis only and implements no runtime behavior.
## Validation log
- `pnpm exec prettier --check docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md docs/scratchpads/753-kbn010-threat-gate.md` — PASS.
- Changed-doc link validator — PASS (`relative_links=0`, one external issue link); `curl -fsSIL https://git.mosaicstack.dev/mosaicstack/stack/issues/753` — PASS.
- The first inline link-validator invocation had a Python f-string syntax error; corrected once and rerun successfully without changing the deliverable.
- `pnpm format:check` — PASS.
- `pnpm lint` — PASS (23 tasks successful).
- `pnpm typecheck` — PASS (42 tasks successful).
- `pnpm exec tsc -p docs/native-kanban-sot/tsconfig.json --noEmit` — PASS.
- Scoped diff review — PASS: authored delta is limited to the exclusive deliverable and this scratchpad; no runtime/schema/config/dependency/CI/deployment file is changed, and `docs/native-kanban-sot/TASKS.md` has no worker worktree delta from tracking commit `9b55de0`.
- Independent SecReview remains pending and cannot return PASS until contract authority resolves `KBN010-SI-001`.
- Final formatting/diff/test-ID completeness review — PASS (all catalog prefixes contiguous with no duplicate IDs).
- Remaining: scoped commit, queue guard, and push.
## 2026-07-14 KBN010-SI-001 contract-authority amendment
- Authority: `web1:mosaic-100` directed the minimal rc.4 amendment under issue #753: add a non-partial unique candidate key on `missions(workspace_id, id)` while retaining global `missions.id` uniqueness and the project-congruent `(workspace_id, project_id, id)` key.
- Rationale: artifacts and approval decisions are polymorphic exactly-one-target records and do not consistently carry `project_id`; widening both children would broaden frozen v1 semantics without improving tenant safety.
- Plan: amend only the frozen schema contract and shared contract, freeze exact DDL ordering and future migration negatives, run scoped/full validation and independent review, then commit and push without opening/merging a PR or closing #753.
- TDD: not applicable because this is a design-contract amendment with no runtime schema or migration implementation; rc.4 freezes future executable empty/prod/N-1/rollback/foreign-workspace and duplicate-key-feasibility tests.
- Budget: no explicit cap supplied; use a 20K-equivalent soft working cap and one bounded worker lane.
- Read-only #757 boundary: PR #757 adds separate logical-agent connector lease/CAS fencing (`logical_agent_connector_leases.lease_epoch`) in runtime schema and connector contracts. SI-001 changes only the frozen mission candidate key; it does not consume, alter, or reinterpret connector fencing, task fencing, lease authority, or #757 ownership.
### Amendment verification evidence
- Frozen schema: added exactly one non-partial `missions_workspace_id_uidx` on `(workspace_id, id)`; retained the global `id` primary key and `missions_workspace_project_id_uidx` on `(workspace_id, project_id, id)`.
- FK reconciliation: targeted static checks prove the candidate key precedes both `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk`; each continues to reference exact ordered columns `(workspace_id, mission_id)` → `missions(workspace_id, id)` with RESTRICT deletion.
- Shared contract: candidate version is `1.0.0-rc.4`; authority, rationale, exact effect/non-effect, candidate-before-FK DDL order, duplicate feasibility, empty/prod/N-1/rollback/foreign-workspace future tests, and unchanged KCR/SOT/tenant/proposal/fencing/no-cascade invariants are explicit.
- Changed-file Prettier, contract ESLint, `pnpm exec tsc -p docs/native-kanban-sot/tsconfig.json`, and targeted SI-001 static checks — PASS.
- Full `pnpm format:check && pnpm lint && pnpm typecheck` — PASS (23 lint tasks and 42 typecheck/build tasks).
- Independent Codex security review — PASS, zero critical/high/medium/low findings; tenant isolation is preserved.
- Independent Codex code review confirmed the candidate key repairs both FK targets and reported no blocker on the authorized delta. Its initial request to rewrite the historical KBN-010 verdict conflicts with the exclusive scope and was resolved by documenting the immutable-evidence boundary in rc.4; its remaining finding concerns pre-existing live `.mosaic` session files, which are untouched and excluded from this commit.
- Scoped diff: only the two frozen contract files and this append-only scratchpad amendment are staged for delivery; no runtime schema, migration, task plan, gate verdict, provider artifact, or #757-owned file is included.
## 2026-07-14 final rc.4 KBN-010 disposition
- Control-plane direction authorized final disposition edits only to `docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md` and this append-only scratchpad; contract author `web1:kbn-contract` remained idle and undisturbed.
- Exact reviewed contract object: commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`, tree `7ebab8fa530a7180036928cea9527f808548aa14`.
- Corroborating review identities: full-index SHA-256 `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`; stable patch-id `058cf98026fcd1043703c866aee047c8bb144740`. A command-rendered patch digest varied by Git rendering command/options and is non-authoritative; commit+tree+exact file content are canonical.
- Independent Homelab non-author schema/security review verdict: **APPROVE**. It confirmed the rc.4 `missions_workspace_id_uidx(workspace_id,id)` repairs both dependent FKs while retaining the global PK and project-congruent key; tenant, polymorphic exactly-one-target, RESTRICT/no-cascade, N-1/rollback semantics remain valid.
- Read-only #757 cross-check: no shared table, index, FK, identity, fence, or authority collision with connector lease/CAS fencing.
- Final KBN-010 gate decision: **PASS / GO** at rc.4 with `UNRESOLVED SCHEMA IMPACTS` equal to exact `none`. Historical SI-001 detection remains in the gate document as evidence that rc.3 was invalid.
- No runtime schema/migration/API/config/dependency/CI/deployment implementation is claimed. KBN-100 must still implement candidate-before-dependent-FK ordering, duplicate feasibility, empty/prod/N-1/rollback evidence, exact FK reconciliation, and separate artifact/approval foreign-workspace negatives (N100-45..50).
- TDD remains not applicable because this continuation changes only documentation/evidence disposition and no runtime behavior.
- Remaining orchestrator-owned sequence: worker validation/commit/push → PR open/update → Ultron review → squash merge → terminal-green post-main CI → close #753 → release KBN-100. KBN-100 is not released earlier.
### Final worker validation
- Changed-doc Prettier and link checks — PASS; issue #753 external link returned successfully.
- Static future-test catalog check — PASS: all prefixes are individually enumerated, contiguous, and duplicate-free; N100 now spans N100-01..50.
- rc.4 disposition assertions — PASS: gate status PASS/GO, frozen target rc.4, exact `none` unresolved section, independent APPROVE, review identities, and N100-50 are present.
- Review identity reproduction — PASS: commit tree `7ebab8fa530a7180036928cea9527f808548aa14`, full-index SHA-256 `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`, and stable patch-id `058cf98026fcd1043703c866aee047c8bb144740` match.
- rc.4 frozen-contract static check — PASS: the candidate key is unique in the declaration and precedes both dependent FKs; frozen contract files remain byte-identical to commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`.
- `pnpm exec tsc -p docs/native-kanban-sot/tsconfig.json --noEmit` — PASS.
- `pnpm format:check` — PASS.
- `pnpm lint` — PASS (23 tasks successful).
- `pnpm typecheck` — PASS (42 tasks successful).
- Scoped diff — PASS: only the gate document and this scratchpad are authored changes; contracts, TASKS, requirements, runtime/schema/migration/config/dependency/CI/deployment and #757-owned files are unchanged. Live `.mosaic` session state remains untouched and excluded.
- Remaining worker steps: final formatting/scoped staging, commit `docs(#753): clear KBN-010 schema gate`, queue guard, push, and control-plane notification.
docs/archive/work-records/755-mos-logical-identity-fencing.md
+148
View File
@@ -0,0 +1,148 @@
# Issue #755 — Logical Mos identity and connector lease fencing
- Task: `MOS-PORT-M1-001`
- Branch: `feat/mos-logical-identity-fencing`
- Base: `origin/main`
- Started: 2026-07-14
- Working budget: 38K tokens (task ledger estimate); one implementation lane, bounded to M1.
## Objective
Implement the first runtime-portability security boundary: normalized logical-agent identity plus a PostgreSQL-durable exclusive connector lease and server-validated fencing grants.
## Scope
- Normalized identity contract independent of harness/provider-native session IDs.
- DB migration/schema/repository for one lease per tenant/logical-agent/binding.
- CAS acquire/takeover, monotonic epoch, TTL, heartbeat, release, expiry handling.
- Server-derived grants bound to tenant, logical agent, binding, connector, scopes, expiry, and lease epoch.
- Reject and credential-safely audit stale, expired, forged, unauthorized, cross-tenant, and cross-binding grants before adapter side effects.
- Runtime adapter boundary consumes normalized lease context.
- Unit, migration, close/reopen, concurrency, abuse, and gateway integration tests.
- Required developer/operations documentation for schema and security behavior.
## Explicit exclusions
No checkpoint/handoff payloads, exactly-once journal/receipts, concrete Claude/Pi/Codex harness adapter, channel cutover, or full cross-harness failover E2E.
## Reconciliation baseline
- Prospective remediation handoff: `web1:coder1`; PR [#757](https://git.mosaicstack.dev/mosaicstack/stack/pulls/757), issue [#755](https://git.mosaicstack.dev/mosaicstack/stack/issues/755).
- Before rebase: `dff8ce4f79ef90370c29d925002118a708010091`; required base: `origin/main` at `2e2280070ae67288be45f41743cf67052a8ca5a6`; original branch base: `d0771835542deab048ad8e79f271e3abdb6151f7`.
- Provider metadata: #757 is open, targets `main`, head is `feat/mos-logical-identity-fencing`, prior CI is green, and the provider reports it is not mergeable because of conflicts.
- Affected delivery paths: `apps/gateway/src/agent/agent.module.ts`; connector-lease gateway repository/service and three focused tests; `packages/agent/src/connector-lease.ts` plus test/export; `packages/types/src/agent/connector-lease.dto.ts` plus test/export; `packages/db/src/schema.ts`, migration `0016_salty_morlocks.sql`, Drizzle snapshot/journal; `docs/PRD.md`, `docs/SITEMAP.md`, MOS architecture/operations pages, this scratchpad, and `docs/tess/TASKS.md`.
- Read-only merge-tree inspection found only `docs/PRD.md` and `docs/SITEMAP.md` conflicts. Current-main #756 channel contracts, #758 roster-v2 structural compiler, and Native Kanban SOT use distinct contract domains; no substantive architecture collision was identified before mechanical reconciliation.
- Reconciliation constraints: retain all current-main #752/#756/#758/KBN content; add only nonduplicative #755 references; do not alter semantics or conflate connector leases/grants with Kanban task leases/fences, local Fleet leases, auth sessions, ResetSession generations, or federation grants.
## Plan (TDD RED → GREEN → REFACTOR)
1. Map existing contracts, DB/migration conventions, gateway authorization/audit boundaries, and test infrastructure.
2. Add failing contract/repository/concurrency/restart/abuse/gateway tests and capture RED evidence.
3. Implement the smallest normalized contracts, schema/migration/repository, grant validator, audit sink, and gateway service/adapter boundary needed to pass.
4. Refactor for clear invariants and credential-safe observability; rerun focused suites.
5. Run package/repo typecheck, lint, format, and appropriate tests.
6. Run independent code + security review, remediate, and re-review.
7. Inspect the final diff for security/scope drift; commit; queue guard; push; open PR with `Refs #755` and exact verification; stop without merge/issue closure.
## Constraints and safety notes
- `docs/tess/TASKS.md` is orchestrator-only and will not be edited.
- Existing dirty `.mosaic/orchestrator/mission.json` and `.mosaic/orchestrator/session.lock` are launcher/orchestrator state and will not be staged or altered intentionally.
- No client-supplied identity may confer authority.
- No credential, token, or raw grant material may be persisted to audit/log output.
- Existing authorization checks remain intact; fencing is an additional fail-closed layer.
## Assumptions resolved from existing architecture
- `ASSUMPTION:` M1 exposes no public lease endpoint. The gateway service is an internal policy surface with deny-all default policy because concrete connector activation/cutover is explicitly deferred.
- `ASSUMPTION:` Fencing epochs use PostgreSQL `bigint` and cross-module decimal strings, preserving JSON portability without JavaScript number precision loss.
- `ASSUMPTION:` Process-local grant provenance intentionally fails closed across restart; durable lease/epoch state survives and fresh grants require current policy + lease validation.
## TDD evidence
RED observed before implementation:
- `corepack pnpm --filter @mosaicstack/types exec vitest run src/agent/connector-lease.dto.spec.ts` → failed to load missing `connector-lease.dto.js`.
- `corepack pnpm --filter @mosaicstack/agent exec vitest run src/connector-lease.test.ts` → failed to load missing `connector-lease.js`.
- Gateway focused tests failed before implementation because the new repository/service boundaries did not exist (workspace dependencies were then built before behavioral GREEN runs).
GREEN to date:
- Types contract: 6/6 passed.
- Agent grant/fencing unit suite: 5/5 passed.
- Gateway PGlite repository + policy/side-effect integration: 7/7 passed; 1 real-PostgreSQL test skipped when `DATABASE_URL` absent.
- Real PostgreSQL focused run with configured `DATABASE_URL`: 1/1 passed (credential value not emitted in reports).
## Documentation checklist
- [x] `docs/PRD.md` contains current MOS-PORT M1 scope and acceptance criteria.
- [x] Developer architecture: `docs/architecture/mos-runtime-portability-m1.md`.
- [x] Admin/operations guidance: `docs/guides/mos-connector-lease-operations.md`.
- [x] `docs/SITEMAP.md` links both pages.
- [x] No user-guide change: M1 exposes no user-facing flow or channel cutover.
- [x] No OpenAPI/endpoint-index change: M1 adds no HTTP endpoint.
- [x] Migration/restart/rollback safety and credential-safe audit constraints documented.
- [x] Canonical source remains in-repo; no external publishing action is in scope.
- [x] Independent review confirms documentation matches implementation; implementation-specific findings were remediated.
## Independent review and remediation
Codex code/security review ran in multiple rounds. Findings and root-cause remediations:
1. Policy could not inspect requested scope/TTL → policy subject now receives normalized requested scopes and explicit requested TTL.
2. Unbounded authority lifetime → hard defaults cap leases at 5 minutes and grants at 30 seconds; overrides may only tighten; over-limit tests added.
3. Cross-tenant denial could audit under submitted tenant → mismatch audit uses authenticated tenant plus sanitized `untrusted` target metadata; integration assertion added.
4. Malformed forged grant could break the denial/audit path → runtime-safe shape validation with sanitized fallback audit; malformed-input test added.
5. Gateway integration test depended on prior test state → denial test now seeds a unique binding itself; isolated `-t` run passed.
6. Reviewer repeatedly identified launcher-generated `.mosaic/orchestrator/*` state; those files remain unstaged and excluded from the implementation commit.
Latest independent security review: no critical/high/medium/low findings. Final commit-level code review remains to run after the intended diff is committed without launcher state.
## Verification evidence
- Focused contracts/fencing: types 6/6; agent 9/9.
- Gateway focused PGlite repository/policy integration: 7/7; isolated denial test 1/1.
- Real PostgreSQL close/reopen/CAS test: 1/1 with configured `DATABASE_URL`.
- Root `corepack pnpm typecheck`: 42/42 Turbo tasks passed.
- Root `corepack pnpm lint`: 23/23 Turbo tasks passed.
- Root `corepack pnpm format:check`: all matched files passed.
- Root `corepack pnpm test`: 42/42 Turbo tasks passed; gateway 616 passed / 12 environment-gated skipped; DB 19 passed / 7 environment-gated skipped; Mosaic 650 passed.
## Known residual risks
- Concrete connector policies and Claude/Pi/Codex adapters are intentionally deferred; production policy defaults deny-all.
- Gateway pre-side-effect validation cannot make an external system exactly-once. Adapters must propagate/enforce the epoch at downstream effect boundaries; receipts/journaling are later #754 scope.
- Migration rollback is additive-only; dropping lease/audit tables is intentionally manual to avoid destroying authority/audit evidence.
## Commit-level review remediation
- Commit-level Codex code review found one `should-fix`: heartbeat, release, and grant issuance authorized caller-supplied lease fields before canonical normalization.
- TDD RED: the isolated gateway policy-boundary test showed mixed-case/padded logical agent, binding, connector, scope, and epoch values reaching policy unchanged.
- Remediation: exported the coordinator's canonical lease normalizer and applied it at the gateway boundary before tenant/policy checks and coordinator dispatch for heartbeat, release, and grant issuance.
- GREEN: isolated policy test 1/1; focused types 6/6, agent 9/9, gateway 8/8; root typecheck 42/42, lint 23/23, format check passed, and root tests 42/42 (gateway 617 passed / 12 environment-gated skipped).
- Commit-level security review remained clean: no critical/high/medium/low findings.
## Durable grant-expiry review remediation
- Final commit review found a second `should-fix`: grant expiry was capped against submitted lease metadata after current-authority validation, rather than the durable lease row.
- TDD RED: a crafted same-authority lease with a later submitted expiry produced a grant expiring after the durable row.
- Remediation: grant authority fields and expiry now derive from the durable current lease; submitted scopes remain an additional narrowing constraint.
- GREEN: focused agent fencing suite 10/10.
## Current-main reconciliation (2026-07-14)
- Rebased the existing PR branch from `dff8ce4f79ef90370c29d925002118a708010091` (old base `d0771835542deab048ad8e79f271e3abdb6151f7`) onto `origin/main` `2e2280070ae67288be45f41743cf67052a8ca5a6`; current uncommitted reconciliation head is `d190732a550918161b91d3eb54640f4ea0e2e499`.
- Resolved only `docs/PRD.md` and `docs/SITEMAP.md`: preserved current-main #752 Native Kanban, #756 official-channel, and #758 FCM material; retained the nonduplicative #755 M1 workstream and placed its two documentation links in the existing Runtime-neutral Mos section. No #755 source semantics changed.
- Compatibility review confirmed separate authority domains: connector lease/epoch/grant remains distinct from KBN task leases/fences, local Fleet roster lifecycle, auth sessions, ResetSession context, and federation grants. No channel cutover, adapter activation, checkpoint/exactly-once behavior, UI convergence, or #754 expansion was added.
- Focused verification after building the required workspace dependencies: types contract 6/6; agent fencing/grant 10/10; gateway repository/PGlite and policy integration 8/8. The focused real-PostgreSQL test was skipped because `DATABASE_URL` was not configured; no credentials were inspected or emitted.
- Generated schema check: `pnpm --filter @mosaicstack/db db:generate` reported no schema changes; migration `0016_salty_morlocks`, snapshot, and journal were unchanged by generation.
- Full verification: `pnpm typecheck` 42/42; `pnpm lint` 23/23; `pnpm format:check` passed; `pnpm test` 42/42 (gateway 627 passed / 12 environment-gated skipped). Scoped diff check and local documentation-link scan passed.
- Pending only: commit this reconciliation record, queue guard, force-with-lease push of the rebased existing branch, then fresh independent DB/code/security review and Ultron. Do not claim merge or issue closure.
## Durable lifecycle-authority remediation (2026-07-14)
- Independent review found that heartbeat and release authorized the submitted lease, allowing forged lifecycle scope data to influence policy before the durable row was consulted.
- Remediation: lifecycle operations tenant-check the submitted lease, load the durable current lease, compare tenant, logical agent, binding, lease UUID, connector, epoch, and canonical ordered scopes, audit and deny any absent/mismatched authority, then run policy and coordinator lifecycle calls with the durable lease. Coordinator CAS/fencing checks remain unchanged.
- Adversarial gateway integration coverage proves forged `tool.execute` scopes on a durable `runtime.send` lease deny before policy or mutation, preserve heartbeat/release state, and write a denial audit for both lifecycle actions; canonical heartbeat and release remain accepted.
- Verification: focused types 6/6; agent 10/10; gateway PGlite integration/repository 9/9 with one `DATABASE_URL`-gated PostgreSQL test skipped; Drizzle check passed; root typecheck 42/42; lint 23/23; format check passed; root test 42/42 (gateway 628 passed / 12 environment-gated skipped).
- Pending: commit, queue-guard, force-with-lease push, then independent DB/code/security rereview and CI. Do not merge or close #755.
docs/archive/work-records/756-official-discord-plugin.md
+57
View File
@@ -0,0 +1,57 @@
# Scratchpad — #756 Official Discord channel plugin
- **Task / issue:** Official Discord channel plugin / #756
- **Branch:** `feat/756-official-discord-plugin`
- **Worktree:** `/home/hermes/agent-work/stack-discord-plugin`
- **Base:** `origin/main` at `49e8a54105eddf41e8e0e44603ded616ee76044f`
- **Objective:** Deliver harness-neutral Discord routing, native mention-to-thread behavior, untagged in-channel interaction, fail-closed channel/user authorization, and transport-neutral contracts for future official channel plugins.
- **Collision boundary:** Do not modify orchestrator-to-Pi migration, logical-agent lease/fencing (#754/#755), runtime provider implementations, or orchestrator-owned `docs/TASKS.md`.
- **Working budget:** 55K tokens for requirements, implementation, tests, documentation, independent review, and delivery. No user-specified hard cap. Reduce optional refactoring before reducing acceptance coverage.
## Assumptions
1. **ASSUMPTION:** Every configured Discord channel is intentionally agent-bound, so an authorized human's untagged message is agent input and receives an in-channel response. Rationale: this satisfies the requested no-tag behavior without activating the bot in arbitrary channels.
2. **ASSUMPTION:** Mentioning the bot in a parent channel creates or reuses a public Discord thread; messages already in a thread remain in that thread without repeated mentions. Rationale: Discord does not support nested threads and the request describes tags as the thread-selection signal.
3. **ASSUMPTION:** One bot process may host multiple configured channel-to-logical-agent bindings. Rationale: bindings are already configuration-owned and this avoids per-agent Discord credentials.
4. **ASSUMPTION:** Static guild/channel/user allowlists and paired-user roles remain the administration surface for this slice. Rationale: dynamic admin UI is larger and can be added without changing adapter contracts.
5. **ASSUMPTION:** The stable conversation handle contains logical agent plus Discord channel/thread identity and never a harness/provider identifier. Rationale: runtime re-enrollment and the active lease work can change Claude/Codex/Pi/OpenCode behind the same channel connection.
6. **ASSUMPTION:** Canonical documentation remains in-repository for this slice; no external publishing is performed.
## Plan
1. Update `docs/PRD.md` before code with #756 scope, constraints, assumptions, and acceptance criteria.
2. Add failing Discord behavior and authorization tests first.
3. Add transport-neutral channel DTO/contracts in `@mosaicstack/types`.
4. Implement mention-to-thread, untagged in-channel, existing-thread, stable conversation, and adapter health behavior without runtime-specific imports.
5. Update admin/developer/channel protocol docs and documentation checklist.
6. Run focused tests, typecheck, lint, formatting, full applicable baseline, and coverage.
7. Run independent code and security review; remediate and re-review.
8. Commit, queue-guard, push, open PR to `main`, wait for green CI, squash merge, verify merged CI, and close #756.
## Progress
- 2026-07-14: Loaded mission state (none active), global/project guidance, current `origin/main`, existing Discord/Tess/fleet connector architecture, issue #709 history, and active portability issues #754/#755.
- 2026-07-14: Created issue #756 through the Mosaic wrapper.
- 2026-07-14: Created isolated worktree/branch from current `origin/main`; the stale root checkout and unrelated QA artifacts remain untouched.
## TDD decision
Required and applied. This change modifies authorization-sensitive remote ingress and routing behavior. Failing permission and routing tests will be captured before implementation.
## Risks / blockers
- Active logical-agent lease work may later add stronger fencing fields. This slice must expose a clean, harness-neutral seam without duplicating that schema.
- Discord thread creation is an external side effect. Unit tests use a typed fake; live credential smoke testing is out of scope and must not use committed secrets.
- Repository-wide checks may expose unrelated baseline debt; changed-scope evidence and CI remain mandatory.
## Verification evidence
- `pnpm format:check` — passed.
- `git diff --check` — passed.
- `pnpm typecheck` — passed (42 Turbo tasks).
- `pnpm lint` — passed (23 Turbo tasks).
- `pnpm build` — passed (23 Turbo tasks).
- `pnpm --filter @mosaicstack/discord-plugin test` — passed: 44 tests; V8 coverage 92.18% statements/lines, 86.55% branches, 100% functions (all configured thresholds ≥85%).
- Focused gateway verification passed: Discord ingress/security, cross-surface, ownership, redaction/concurrency, and agent attachment tests.
- `pnpm test` — changed-scope suites passed; repository baseline remains blocked by `apps/gateway/src/__tests__/cross-user-isolation.test.ts` requiring PostgreSQL at localhost port 5433 (`ECONNREFUSED`). The failure is unrelated to this change.
- Independent code/security re-reviews requested after final remediation; reports are stored under ignored `docs/reports/` evidence paths.
docs/archive/work-records/758-fcm-m1-002-shared-role-resolution.md
+148
View File
@@ -0,0 +1,148 @@
# FCM-M1-002 — Shared role resolution
- **Task:** `FCM-M1-002`
- **Issue:** `mosaicstack/stack#758`
- **Branch:** `feat/758-shared-role-resolution`
- **Starting head:** `32e75c67b094de443d37fe7d5ff8d25cdfc8b39d`
- **Role:** implementation worker; independent review and merge remain outside this worker
## Objective
Reuse the existing baseline-plus-`roles.local` persona resolver as the sole class authority for roster-v2 semantics, profile validation, provisioning, and launch/persona resolution. Add exact approved alias canonicalization, fail-closed semantic validation, immutable canonical-class authority contracts, required baseline roles, and operator documentation without implementing lifecycle, mutation, credentials, certificate workflow, or later FCM cards.
## Budget
- Soft budget: **25K tokens**.
- Strategy: inspect once, implement in small TDD units, run focused suites before the full package gate, and avoid unrelated refactors or M1-003/M2/M4 scope.
## Plan
1. Map the existing persona resolver, roster-v2 compiler, profile/provision consumers, launch resolution, role library, and focused tests.
2. Write denial/invariant tests first for aliases, canonicalization-before-override, unreadable roles, authority boundaries, policy mismatch, canonical provision output, and resolver parity.
3. Run the focused suites and record the expected red evidence.
4. Implement one shared canonical resolution and authority contract in/through `fleet-personas.ts`; delegate roster semantic validation and profile/provision paths to it.
5. Add baseline `validator`, `team-leader`, and `interaction` role contracts plus `LIBRARY.md` entries while retaining `operator-interaction` compatibility.
6. Add the required role reference, alias migration, customization guide, and roster-v2 semantic handoff documentation.
7. Run focused tests, the full `@mosaicstack/mosaic` suite, typecheck, lint, Prettier, `git diff --check`, situational verification, independent code/security review, and remediation.
8. Commit with the required co-author trailer, run the CI queue guard, push the existing branch, and create/update exactly one PR to `main` with `Refs #758`.
## TDD evidence
### Red
After installing worktree-local dependencies and building `@mosaicstack/db`, the pre-implementation
focused run collected the intended tests and failed as expected:
```text
2 test files failed; 32 tests failed; 32 tests passed
```
Expected failures named the missing `canonicalizeRoleClass`,
`authorityForCanonicalClass`, and `validateRosterV2Semantics` APIs, absent requested/canonical typed
output, and unresolved required canonical role contracts. An earlier run that failed before test
collection on an unresolved `yaml` dependency was treated as environment setup, not TDD evidence.
### Green
Focused role-resolution and affected service fixtures:
```text
6 test files passed; 109 tests passed
```
The focused set covers personas, profiles, provision, launch persona contract, roster-v2 semantics,
and the operator-interaction service fixture. The final profile tests also cover readable lead/floor
compatibility and canonical collision denial.
## Tests and gates
- Focused suites: pass, **6 files / 109 tests**.
- Full `@mosaicstack/mosaic` suite: pass, **50 files / 713 tests**. Workspace package build outputs
were prepared first because a clean worktree has no dependency `dist` entries.
- `pnpm --filter @mosaicstack/mosaic typecheck`: pass.
- `pnpm --filter @mosaicstack/mosaic lint`: pass.
- Prettier check over every changed file: pass.
- `git diff --check`: pass.
- Runtime/file-boundary evidence: real role library, profile/provision filesystem integration,
launch-time synchronous contract injection, v1 roster parser round-trip, roster-v2 semantic
filesystem checks, and operator-interaction service fixtures all pass without live mutation.
- Independent code review: **APPROVE**, no blocking or non-blocking findings; reviewed complete
tracked/untracked delta including the canonical collision guard. Residual: roster-v2 semantic
validation is an explicit async handoff with production caller wiring owned by later work.
- Independent security review: **APPROVE**, no verified authority/security findings on the final
delta.
### Post-PR fail-closed remediation
Independent rereview found resolver fail-open edges that the original green PR head did not cover. The
remediation remained uncommitted until every finding was reproduced red-first and the same reviewer
approved the complete two-file delta.
Final regression evidence:
```text
persona resolver: 47/47
focused affected suites: 6 files / 138 tests
root-container resolver suites: 86/86
full canonical run: 42/42 Turbo tasks; Mosaic 50 files / 733 tests
```
DB migration, typecheck, lint, Prettier, and `git diff --check` also passed. Coverage now proves:
- unreadable, unscannable, direct-dangling, ancestor-dangling, and literal `..` traversal override paths
fail closed across async, sync, listing, and status APIs;
- genuinely missing override directories still permit baseline fallback;
- cached missing scans are revalidated before fallback;
- marker-defined identity and domain metadata are revalidated on the second read;
- `LIBRARY.md` rows and incidental later markers cannot define, shadow, or advertise personas.
Exact-head pipeline `1819` passed for rebased head `4d990eee…`, but the independent reviewer-of-record
returned **REQUEST CHANGES** after reproducing three additional edge failures: a canonical filename could
inherit protected authority despite a conflicting explicit first marker, cached `scanned` absence could
miss an override created before baseline fallback, and inherited plain-object names such as `constructor`
could corrupt alias/authority lookup. Merge remained held.
Each failure was reproduced red-first in the persona suite (4 failing assertions), then remediated without
expanding card scope. Explicit first markers now own identity and filename fallback applies only to
markerless contracts; every second read rejects a newly introduced conflicting marker regardless of
cached classification; cached async resolution re-scans the override layer immediately before every
baseline fallback; alias and authority registries require own-property matches. Current uncommitted
evidence is persona **52/52**, focused affected suites **6 files / 143 tests**, and full Mosaic package
**50 files / 738 tests**, plus typecheck, lint, Prettier, and `git diff --check`. Independent
finding-specific rereview **APPROVED** the complete uncommitted three-file remediation after direct
adversarial reproduction of all three findings and the follow-up markerless TOCTOU. All post-commit
exact-head gates remain required.
## Risks and boundaries
- **Security-sensitive authority:** authority must derive only from canonical class, never role prose, aliases, display names, or tool-policy text.
- **Resolver divergence:** no second regex, registry, scanner, or prose parser may be introduced.
- **Alias capture:** aliases must canonicalize before baseline/`roles.local` lookup so local files cannot redefine legacy aliases as separate authority.
- **Readable persona requirement:** semantic success requires a resolved readable persona, not class-set membership.
- **Scope control:** no roster mutation, lifecycle, lease issuance, certificate workflow/storage, credentials, remote reconciliation, provision-v2 conversion, or shipped-example disposition execution.
- **Coordination:** `docs/TASKS.md` is read-only and remains orchestrator-owned.
## Acceptance-evidence mapping
| Requirement / criterion | Verification evidence |
| ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FCM-REQ-02` shared semantic resolver | Async/sync resolver parity; roster-v2 delegates batched scans and resolution; profiles/provision and launch reuse `fleet-personas.ts`; no second scanner or class-marker regex added. |
| `FCM-REQ-07` canonical classes and authority boundaries | Exact alias and non-alias tests; immutable authority invariant tests; all required canonical contracts resolve through the real role library. |
| `AC-FCM-01` structural + semantic roster validation | Synchronous parser/normalizer tests remain intact; async semantic tests cover aliases, custom roles, unreadable/unresolved roles, `LIBRARY`-only rejection, and bidirectional protected policy mismatch. |
| `AC-FCM-07` protected authority invariants | Denial tests prove merge-gate-only merge, validator certificate-only, orchestrator/team-leader/interaction limits, no implicit custom-role authority, and canonical tool-policy matching. |
## Documentation
- `docs/fleet/reference/role-classes.md`
- `docs/fleet/migration/legacy-class-aliases.md`
- `docs/fleet/how-to/customize-roles.md`
- `docs/fleet/reference/roster-v2-fields.md` semantic handoff
- Baseline role contracts and `LIBRARY.md` rows for `validator`, `team-leader`, and `interaction`
## Residual risks
- Provisioning remains intentionally v1 and does not emit `reports_to`; canonical topology is retained
in its typed seat/summary path only, matching the existing v1 parser boundary.
- Alias support remains for compatibility; new configuration should emit canonical identities.
- This card defines authority metadata and validation only. Enforcement workflows for leases,
certificates, lifecycle, and mutation remain owned by later FCM cards.
docs/archive/work-records/758-fcm-m1-003-example-profile-dispositions.md
+37
View File
@@ -0,0 +1,37 @@
# FCM-M1-003 — Executable example/profile/service-preset dispositions
- **Task / issue:** FCM-M1-003 / #758
- **Branch / base:** `test/758-example-profile-dispositions` from `origin/main` `a5e8e554012f27898e035d2882a8e47e1a02fe97`
- **Objective:** Make every artifact in the M0 legacy disposition inventory executable evidence: it must validate canonically, be explicitly retained as a v1 fixture, or be retired with a replacement/deprecation link.
- **Scope:** Validation and explicit version/retirement metadata only for shipped examples, profiles, and the operator-interaction service preset. Reuse the central resolver and existing v2 roster compiler.
- **Out of scope:** Generated environment boundaries, CRUD, reconciliation/apply, migration, live fleet mutation, and `docs/TASKS.md`.
- **Budget:** 20K card allocation; use focused package tests before full package validation.
## Plan
1. Inventory exact shipped artifacts and existing compiler/resolver/profile tests.
2. Add failing behavior tests covering all listed artifacts and their documented disposition.
3. Implement minimal declarative fixture/disposition validation; do not add a role/class resolver.
4. Run focused and package quality gates; obtain independent code and security review.
5. Commit, queue-guard, push, open one `main` PR with `Refs #758`.
## Progress
- Intake complete: verified no branch, worktree, or open PR for this card before creating this isolated worktree.
- Requirements read: FCM PRD, FCM-M1-003 task row, M0 disposition inventory, delivery/QA/documentation guides.
- TDD: RED recorded with `pnpm --filter @mosaicstack/mosaic test -- example-profile-dispositions.spec.ts` failing because the new module did not exist; GREEN recorded after the minimal guard implementation. The focused suite now has 4 passing tests, including undeclared-artifact and missing-explicit-v1-version denials.
- Independent review: initial code review found the service policy path was hardcoded; remediation iterates declared `canonical-service-policy` artifacts. Exact-head code review approved and exact-head security review found no issues.
## Risks / decisions
- The M0 inventory permits unresolved legacy roles only when explicitly v1-versioned or retired. Do not infer aliases beyond the three approved by FCM-M1-002.
- `docs/TASKS.md` is orchestrator-owned and will not be edited.
## Verification evidence
- Focused TDD guard: `pnpm --filter @mosaicstack/mosaic test -- example-profile-dispositions.spec.ts` — PASS (4 tests).
- Full package suite: `pnpm --filter @mosaicstack/mosaic test` — PASS (51 files, 742 tests).
- Static gates: `pnpm --filter @mosaicstack/mosaic typecheck`, `pnpm --filter @mosaicstack/mosaic lint`, and `pnpm format:check` — PASS.
- Diff gate: `git diff --check` — PASS.
- Exact-head reviews: `codex-code-review.sh --uncommitted` — APPROVE; `codex-security-review.sh --uncommitted` — no findings.
- Delivery: committed as `9a9ad1a`, pushed after `ci-queue-wait.sh --purpose push`, and opened PR [#770](https://git.mosaicstack.dev/mosaicstack/stack/pulls/770) to `main` with `Refs #758`. `pr-ci-wait.sh -n 770` reported terminal-green Woodpecker pipeline [#1823](https://ci.mosaicstack.dev/repos/47/pipeline/1823/1).
docs/archive/work-records/758-fcm-m2-002-fleet-agent-crud.md
+46
View File
@@ -0,0 +1,46 @@
# FCM-M2-002 — Generation-Guarded Fleet Agent CRUD
- **Task / issue:** FCM-M2-002 / #758
- **Branch / base:** `feat/758-fleet-agent-crud` from `origin/main` `191efaefeb5c0c6bb218c1292d12ce8e73ace12b`
- **Budget:** 30K card allocation; no deployment or live-fleet actions.
## Objective
Provide local roster-owned create, get, update, and delete mutations with a generation precondition, deterministic dry-run plan, complete-state structural/semantic/projection validation before writes, atomic roster persistence, and redacted recovery output on a late projection failure.
## Scope and exclusions
- Roster v2 is the sole desired-state authority. Reuse `parseRosterV2`, `renderRosterV2Yaml`, `validateRosterV2Semantics`, and the generated-environment boundary.
- Fresh create defaults to `enabled: true` and `desired_state: stopped`; this card never starts a runtime.
- Excluded: reconcile/apply; lifecycle/session/systemd/tmux actions; migration/canary; remote/connector/gateway mutation; arbitrary commands/channels/secrets; generated files as authority; `docs/TASKS.md` and orchestration ledger changes.
## Red-first plan
1. Add failing tests for dry-run non-mutation, stale generation, concurrent writer locking, stopped default create, equivalent idempotency, complete proposed-state semantic/boundary validation, atomic roster write, and injected late projection failure with redacted recovery details.
2. Implement only a roster-v2 CRUD service and file adapter; no legacy `fleet add/remove` behavior expansion.
3. Add operator/reference docs for JSON outcomes, generation retries, recovery, and no-runtime-action boundary.
## Progress
- Preflight: clean exact base and no duplicate PR confirmed.
- Intake read: FCM PRD/AC-FCM-03, task row, launch and generated-env boundaries, roster v2/resolver contracts, legacy fleet command behavior, delivery/QA/TypeScript/security/documentation guidance.
- TDD: RED observed for the missing CRUD module. GREEN: focused suite passes 7 tests covering stopped-default create, stale generation, idempotency, dry-run non-mutation, concurrent lock denial, exact stale/absent generated-projection delete cleanup, and redacted late-projection recovery.
- REVIEW-1 remediation: RED observed for absent CLI create/get/update/delete/plan wiring. Added roster-v2-only JSON commands, including safe `get`, read-only planning/dry-run, explicit persisted-start recording (never runtime start), stable handled error codes, and focused CLI coverage.
- REVIEW-2 remediation: RED observed for missing direct fleet-control-plane registration and ambiguous partial late-I/O result. The public surface is now direct `mosaic fleet {create,get,update,delete,plan}` (not root gateway `mosaic agent`); a late filesystem projection failure returns non-zero redacted JSON with `authoritativeRoster: committed` and `projections: incomplete`, proving no rollback/no-op claim.
- REVIEW-3 remediation: RED observed for unnamed update/delete plans and deleted-agent quarantine conflict. `plan <operation> [name]` now requires target names only for update/delete; delete validates/removes only exact generated state while retaining local/legacy/quarantine/unrelated files. Actual CLI/filesystem tests cover create/update/delete plans, plan validation/non-mutation, retained artifacts/dry-run bytes, unsafe permission/symlink rejection, and post-roster delete recovery. Added the required operator how-to.
- REVIEW-4 remediation: RED observed that actual Commander create accepted and silently dropped disallowed `command`, `channel`, and `secretRef` keys. The `--agent` object and nested `launch` now use strict own-property allowlists; non-plain/prototype-sensitive shapes and unknown keys fail `invalid-request` before resolver, roster, or projection mutation. Actual Commander plan/create/update tests cover top-level command/channel/secretRef/constructor/prototype/`__proto__` shapes and nested launch unknown fields, byte-identical retained artifacts, non-zero exit, and diagnostics that never echo rejected values. Docs state the same boundary.
## Risks / assumptions
- **ASSUMPTION:** M2 mutations operate exclusively on the existing v2 roster contract because generation/lifecycle fields are v2-only; legacy v1 add/remove commands remain unchanged compatibility paths.
- A multi-file roster/projection write cannot be one filesystem rename. The roster is authoritative; a post-roster projection failure returns a redacted recovery plan naming only safe paths/actions, never environment values.
## Verification evidence
- `pnpm --filter @mosaicstack/mosaic test -- fleet-agent-crud-command.spec.ts fleet-agent-crud.spec.ts generated-env-boundary.spec.ts` — PASS (48 tests after REVIEW-4 remediation).
- `pnpm --filter @mosaicstack/mosaic test` — PASS (54 files, 794 tests after REVIEW-4 remediation).
- `pnpm --filter @mosaicstack/mosaic lint` — PASS.
- `pnpm --filter @mosaicstack/mosaic typecheck` — PASS.
- `pnpm format:check` and `git diff --check` — PASS.
- Root `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, and `git diff --check` — PASS after REVIEW-4 remediation; full package test is 54 files / 794 tests.
- REVIEW-4 remediation focused checks are green; fresh full-delta independent review is required before author-green. No commit, push, or PR opened.
docs/archive/work-records/758-fcm-m3-001-local-reconciler.md
+39
View File
@@ -0,0 +1,39 @@
# FCM-M3-001 — Local roster-owned reconciliation and lifecycle
- **Task / issue:** FCM-M3-001 / #758
- **Branch / base:** `feat/758-local-reconciler` from `origin/main` `bc5e73629e92c56a80fa6a769ebad17c0177f504`
- **Base tree:** `1b9ebe4fa1a90734b6f81118e120bae5290cd350`
- **Scope:** source, isolated fake-adapter tests, and card documentation only. No live fleet/systemd/tmux action.
## Objective
Provide local roster-v2 `apply`/`reconcile` and lifecycle/status contracts. The roster remains desired-state authority; projections and runtime observations are derived state.
## Red-first evidence
The initial focused reconciler test failed because `fleet-reconciler.ts` did not exist. The initial new-worktree test invocation also exposed absent dependencies; `pnpm install --frozen-lockfile --store-dir /home/jarvis/.local/share/pnpm/store` restored local workspace dependencies without changing source.
## Design
- A new `fleet-reconciler.ts` accepts only typed roster-v2 input plus injected command and projection adapters.
- It targets only exact `mosaic-agent@<roster-name>.service` units and the exact configured tmux socket/session.
- It classifies unowned/unmanaged state and fails mutation closed rather than adopting or killing it.
- It validates the private install-derived holder identity and complete expected tmux global environment before mutating lifecycle state.
- REVIEW-1 remediation: RED review evidence found service-level apply could omit generation and had no mutation lock. Every non-observational command now requires an expected generation; a private exclusive roster-adjacent lock is acquired before effects and released on success or partial failure. Tests cover missing/stale values, concurrent denial, no effects, release, and lock-free observation.
- REVIEW-2 remediation: lock acquisition now validates private real `MOSAIC_HOME`/`fleet` ancestors, rejects symlink or unsafe leaves, distinguishes `EEXIST` concurrency from other I/O, and binds release to the created inode plus random ownership token. A replacement lock is retained and reported, not unlinked. A crash may leave a stale lock for inspection; no stale-lock break is claimed.
- REVIEW-3 remediation: a lock cleanup failure now adds bounded `cleanup` diagnostics to a known successful or partial effect result without replacing its projection/lifecycle/recovery truth. Cleanup is not claimed as complete, and the retained lock requires inspection before retry.
- REVIEW-4 remediation: command JSON with an additive cleanup diagnostic now exits non-zero even where known effects completed; clean effect and observational JSON remain zero-exit.
- REVIEW-5 remediation: mutating operations acquire the private lock before rereading canonical `roster.yaml`; the fenced reread, not a caller snapshot, supplies generation validation, plan, projection, and lifecycle authority.
- `apply` starts only enabled agents whose persisted desired state is `running`; stopped/default agents are never started by reconciliation.
- Observational commands produce JSON classification only. Partial projection or lifecycle effects report explicit recovery without values.
## Boundaries
Excluded: live host actions, remote/SSH/connector lifecycle mutation, migrations, canaries, deployment, gateway changes, arbitrary command/channel/secret inputs, `docs/TASKS.md`, and orchestration ledgers.
## Verification
- Focused reconciler/Commander/CRUD/fleet tests: 4 files / 229 tests passed.
- Full `@mosaicstack/mosaic` suite: 56 files / 820 tests passed after REVIEW-5 canonical roster fencing remediation.
- Package and root typecheck/lint, root format check, and `git diff --check`: passed.
- Isolated launcher and systemd template harnesses passed; they use fixtures only. No live fleet, systemd, tmux, session, remote, connector, or runtime action occurred.
docs/archive/work-records/758-fcm-m3-002-reconciler-lifecycle-gates.md
+84
View File
@@ -0,0 +1,84 @@
# FCM-M3-002 — Reconciler lifecycle acceptance gates
- **Task / issue:** FCM-M3-002 / mosaicstack/stack#758
- **Branch:** `test/758-reconciler-lifecycle-gates`
- **Required starting head:** `499090508ef1d768660e4d54e7934cbcf13cb1cd`
- **Required starting tree:** `2f1bb7fed48291f3f7ba8b21c2b52491aa14fe2b`
- **Scope:** isolated acceptance coverage and card-required evidence/tracking only; no live fleet, systemd, tmux, session, site, migration, canary, deployment, runtime, connector, or remote action.
- **Budget:** use the task estimate of 25K as the working cap; keep the delta to one coherent acceptance suite plus required task/scratchpad evidence. No production change unless a failing reproducer proves an in-scope defect.
## Intake evidence
- Clean exact local branch/head/tree verified before editing.
- `origin/test/758-reconciler-lifecycle-gates` fetched and verified at the same required head.
- The Mosaic PR wrapper reported no open pull requests, so no open-PR branch collision exists.
- Parent issue #758 is open and remains intentionally open through M5.
- Requirements loaded from `docs/PRD.md` FCM requirements and `AC-FCM-05`, `docs/TASKS.md` FCM DAG, the M3 rows in `docs/fleet/FLEET-CONFIG-DOCS-IA-CHECKLIST.md`, and the FCM-M3-001 implementation scratchpad.
## Objective
Add broad, behavior-oriented acceptance evidence around the shipped local reconciler contracts. Exercise only injected fake systemd/tmux adapters and temporary filesystem fixtures. Prove exact ownership/targeting, persisted stopped-state safety, truthful partial-failure recovery, rollback behavior, and stable command JSON/exit outcomes without touching live services or sessions.
## Acceptance mapping and evidence
| FCM-M3-002 acceptance concern | Delivered isolated evidence |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Systemd/tmux lifecycle | `fleet-reconciler.acceptance.spec.ts` drives apply, reconcile, stop, restart, status, and recovery reconcile through one stateful injected fake host. The fake models exact systemd effects and tmux session observations; no host commands run. |
| Drift | Canonical roster-v2 YAML drives the Commander `fleet status` boundary and classifies `missing-session`, `unexpected-session`, and `disabled-running`, including combined drift, while asserting observation emits no lifecycle mutation. |
| Exact default/named socket targeting | Canonical v2 requires an explicit non-empty named socket; the parser rejects missing/empty values and the Commander acceptance path asserts exact `-L mosaic-fleet` targeting. A separate canonical legacy-v1 roster loader plus runtime-transport path proves a socket-less compatibility roster targets the literal tmux default server with no `-L`. No unreachable empty-socket v2 fixture is used. |
| Unmanaged-session classification | Stateful fixtures report sorted `coder0-shadow`/`unmanaged` sessions, then prove an exact roster stop leaves both sessions and the near-collision service intact. |
| Crash/partial failure | Injected restart failure is applied after the fake effect to model crash/partial truth: result is `lifecycle: incomplete`, the roster is unchanged, and observed runtime may be active. |
| Rollback/recovery semantics | M3 has no rollback command and explicitly does not claim automatic rollback. The acceptance workflow proves the bounded recovery contract: inspect, then exact reconcile restores the persisted stopped target without a start or fuzzy effect. M4 migration/canary rollback remains outside this card. |
| Stopped-state preservation | Stateful apply and reconcile both stop an initially running observed agent whose persisted target is stopped; failed explicit restart leaves desired state stopped; recovery reconcile restores stopped state. No start call is emitted. |
| Zero fuzzy destructive targeting | Near-collision `coder0-shadow` service/session plus `unmanaged` session remain untouched. The recorded destructive calls contain only exact `[email protected]`; no tmux kill action is emitted. |
| Stable JSON/exit behavior | Temporary canonical roster fixture invokes the CLI boundary and asserts exactly one JSON line, exact partial-result shape, and exit code 1. Existing focused command specs continue to cover clean zero-exit and stable error JSON. |
| Redacted truthful recovery | Fake stderr includes `PASSWORD=acceptance-secret`; exact CLI JSON contains only bounded recovery metadata and excludes the key, value, and raw diagnostic. |
## Plan
1. Inventory existing reconciler and command specs against the table above; avoid duplicating narrow assertions already present.
2. Add one acceptance-level spec using only fake/injected adapters and temporary files.
3. If a real defect is exposed, preserve the failing reproducer and make only the smallest FCM-M3-002-required fix; otherwise leave production unchanged.
4. Reconcile `docs/TASKS.md` only for delivered M1/M2/M3-001 truth and mark FCM-M3-002 in progress.
5. Run focused tests, full `@mosaicstack/mosaic` tests, package/root typecheck and lint, Prettier/format and diff checks, plus adversarial fake-runner cases.
6. Record exact evidence and leave the tree uncommitted for independent synthetic-tree review.
## TDD decision
This card adds acceptance coverage to already-delivered behavior. Test-first applies to any product defect discovered: retain a failing reproducer before an in-scope fix. If the shipped behavior already satisfies the acceptance contract, no production code will be changed and the acceptance suite itself is the deliverable.
## Progress
- Intake and immutable baseline verification complete.
- Existing coverage inventory confirmed strong unit coverage but no stateful cross-command lifecycle acceptance harness.
- Added `packages/mosaic/src/fleet/fleet-reconciler.acceptance.spec.ts`: one injected stateful fake systemd/tmux host, temporary canonical v2 and legacy-v1 roster fixtures, and seven acceptance tests.
- Production source is unchanged; no product defect requiring an FCM-M3-002 fix was found.
- `docs/TASKS.md` reconciles only merged M1/M2/M3-001 truth and marks FCM-M3-002 in progress.
## Verification evidence
All commands ran from `/home/jarvis/src/mosaic-stack-local-reconciler` and passed unless explicitly noted.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/fleet-reconciler.acceptance.spec.ts` — final remediation run: 1 file, 7 tests passed; canonical v2 named-socket parsing/Commander status, missing/empty v2 rejection, and canonical legacy-v1 default-server runtime targeting are distinct reachable cases.
- Focused reconciler/roster/transport command covering acceptance, reconciler, command, CRUD, v2 parser, and runtime transport specs — 8 files, 304 tests passed.
- `pnpm --filter @mosaicstack/mosaic test` — final remediation run: 57 files, 827 tests passed.
- `pnpm --filter @mosaicstack/mosaic typecheck` — passed.
- `pnpm --filter @mosaicstack/mosaic lint` — passed.
- `pnpm typecheck` — 42/42 Turbo tasks successful.
- `pnpm lint` — 23/23 Turbo tasks successful.
- `pnpm exec prettier --check docs/TASKS.md docs/scratchpads/758-fcm-m3-002-reconciler-lifecycle-gates.md packages/mosaic/src/fleet/fleet-reconciler.acceptance.spec.ts` — passed.
- `pnpm format:check` — all matched files use Prettier style.
- `git diff --check` — passed with no output.
- Initial scoped Prettier check found style drift in the new spec and tracking table; `pnpm exec prettier --write ...` remediated it before all final gates above.
- No live fleet, systemctl, tmux, process, site, migration, canary, deploy, runtime, connector, or remote command was invoked.
## Review boundary
This is an author handoff. No self-review is represented as reviewer-of-record. The uncommitted synthetic tree is intended for independent review.
## Risks / blockers
- M3 truthfully reports incomplete lifecycle effects and bounded recovery; it does not implement or claim an automatic rollback command. This suite proves stopped-state restoration by the documented exact recovery reconcile. M4 retains migration/canary rollback ownership.
- The fake host models only the public systemd/tmux runner contract and temporary roster filesystem boundary. This is intentional under the no-live-effects hold.
- Parent issue closure, commit, push, PR, merge, deployment, and branch cleanup remain explicit holds.
- No residual implementation blocker.
docs/archive/work-records/758-fcm-m4-001-v1-v2-migrator.md
+80
View File
@@ -0,0 +1,80 @@
# FCM-M4-001 — v1-to-v2 inventory, preview, and migrator
- **Task / issue:** FCM-M4-001 / mosaicstack/stack#758
- **Branch / base:** `feat/758-v1-v2-migrator` from `origin/main` `c1aecfabe97a5dc81a72f44910cd4e626f41863f`
- **Base tree:** `46cdfbcdc1d1ff9c7b8b2b9cf3841086590bf774`
- **Scope:** field-complete inventory, non-mutating preview, canonical v2 migration output, and migration/recovery disposition evidence. All effects use injected fakes or temporary fixtures.
- **Budget:** 35K task estimate is the hard working cap. Keep one card/one PR and prefer focused reuse of the v2 compiler, shared role resolver, generated-env boundary, M1 executable disposition inventory, and reconciler observations.
## Objective
Implement preview-first v1 migration that never infers unresolved classes or lifecycle, preserves observed running/stopped state, quarantines forbidden legacy environment inputs with key-name/SHA-256-only diagnostics, inventories remote/connector/schema-only entries without reconciling them, covers every M1-classified shipped artifact, and emits deterministic recovery disposition evidence for the later M4-002 canary/rollback gate.
## Acceptance mapping
1. Field-by-field v1 inventory and no-mutation preview.
2. Canonical output compiled by `roster-v2.ts` and semantically validated by the existing baseline-plus-`roles.local` resolver.
3. Only approved deterministic aliases; every other noncanonical class requires an explicit disposition.
4. Observed stopped/running maps explicitly to persisted lifecycle; stopped observations never produce running targets.
5. Generated env is regenerated; strict local data is relocated; forbidden keys are quarantine inputs reported only by key name and SHA-256.
6. Remote/connector/schema-only entries are inventory-only and excluded from local reconciliation output.
7. Every shipped M1 example/profile/service preset has executable migration disposition evidence.
8. Deterministic migration/recovery evidence records source, output, exclusions, quarantine, and restore prerequisites without executing a canary or rollback.
## Boundaries
Out of scope: FCM-M4-002 executable canary/rollback and host fixture, #766 communications, #636 commands/channels, live fleet/systemd/tmux/session/migration/deploy/connector/remote/gateway effects, `docs/TASKS.md`, parent issue mutation, commit, push, and PR operations.
## TDD plan
Migration rules and redaction are critical data-mutation/security logic, so tests are written red-first for inventory completeness, explicit class disposition, observed-state preservation, quarantine redaction, inventory-only remote/schema entries, compiler/resolver reuse, artifact coverage, and recovery evidence. Production code follows only after the focused tests fail for the missing behavior.
## Plan
1. Map existing v1 loader, v2 compiler/resolver, env quarantine, reconciler observation, and M1 disposition guard.
2. Add behavior-oriented failing migration tests with temporary fixtures and injected observation/filesystem adapters only.
3. Implement the narrow migration module and CLI boundary without a second resolver or live command runner.
4. Add scoped M4 migration/recovery documentation and executable shipped-artifact evidence.
5. Run focused tests, full package tests, package/root typecheck and lint, formatting, diff checks, and adversarial redaction/no-effect verification.
6. Run independent code/security review, remediate findings, reconstruct the synthetic tree using a temporary index, and stop uncommitted.
## Progress
- Collision checks passed: no local/remote branch, worktree, target path, or open PR owned `feat/758-v1-v2-migrator`.
- Dedicated worktree created at the exact green `origin/main` base.
- Required global/repository guides and FCM requirements/evidence loaded.
- No matching migration skill exists under the configured skill directories; no unrelated skill loaded.
- Added a preview-only CLI and migration module that compile with the existing v2 parser/renderer and validate through the shared persona resolver.
- Added value-free raw-v1 inventory, strict unknown-field/synonym/duplicate detection, inventory-only remote and connector handling, and explicit class/tool-policy decisions.
- Added separate reviewed lifecycle observations with only unambiguous running/stopped mappings.
- Added sanitized, non-mutating environment preflight and recovery evidence explicitly marked non-executable.
- Added executable disposition evidence derived from the exact 13-entry M1 inventory and operator documentation.
- Tightened untrusted decisions/observations to reject unknown keys, invalid types/enums, extra local records, and competing automatic-alias dispositions.
- Remediated independent review findings: v1 runtime/reset defaults are preserved, `~` workdirs expand only at env preflight, malformed/required agent fields fail closed before remote exclusion, and all seven shipped v1 fixtures now execute real previews with explicit evidence.
- Remediated socket and locality authority blockers: socket-only agents stay local; `host == fleetHost` stays local; only `host != fleetHost` is inventory-only; ssh-only, missing reviewed fleet-host identity, and contradictory host/ssh targets block explicitly without lifecycle omission.
- Remediated final exact-tree blockers: a declared v1 root socket cannot be overridden; matching/conflicting socket decisions retain reviewed running evidence; canonical ordering uses a shared locale-independent Unicode code-point comparator; migration evidence preserves all four legacy environment dispositions; backup documentation no longer claims validation that M4-001 does not perform.
- Remediated immutable-review socket-presence blocker: both `socket_name` and `socketName` are field-presence-aware, so explicit empty/default-server declarations remain authoritative and incompatible named decisions block rather than replacing them.
- Remediated the replacement-tree blockers: the shared v2 compiler and reconciler accept an explicit empty socket as literal default-server identity; present-empty holder session, default/agent work directory, runtime reset command, and alias values block rather than defaulting; missing preview inputs emit one stable blocked JSON object with non-zero status. Snake/camel aliases and whitespace-only input have adversarial coverage.
- Remediated the subsequent authority blockers: each present-empty CLI path emits exactly one stable blocked JSON object with exit 1 before file reads, and reconciler `start`/`restart` or desired-state `apply`/`reconcile` fail closed before fixed `mosaic-fleet` systemd services can act on a default-server roster.
- Remediated committed-head review blockers: explicitly declared empty runtime objects use the production v1 `/clear` reset fallback while omitted `pi` retains `/new`; lifecycle observations are sorted by canonical agent name; and bare CLI path flags reach preview validation, emit one stable blocked JSON object with exit 1, and perform zero reads. Built production-CLI subprocess tests cover all three bare flags.
- Remediated late-audit blockers: the documented M4 guard invokes the 13-artifact validator and all seven v1 previews; canonical `~`/`~/...` workdirs remain unchanged in migration evidence and traversal-free forms expand at the shared production projection boundary while ordinary relative and home-relative traversal paths remain rejected; remote inventory and exclusion evidence sort canonically; and every default-server lifecycle-mutating reconciler path fails before observation, projection preparation/application, or fixed-unit effects. Explicit regressions preserve `plan`, `status`, `doctor`, and `verify` as observational default-server commands.
- Remediated exact-tree traversal review: `~/../escape` and `~/src/../../escape` remain unexpanded and fail the unchanged shared `unsafe-path` validation. Both the shared generated-environment boundary and the production v1 environment caller have red-first regressions, preventing earlier caller normalization from bypassing the boundary.
## Verification evidence
- Focused migration/compiler/environment/reconciler/CLI: 8 files, 372 tests passed.
- Documented 13-artifact guard: 1 matching test passed and executed all seven v1 previews.
- Full `@mosaicstack/mosaic`: 59 files, 902 tests passed.
- Workspace build: 23 tasks passed.
- Root typecheck: 42 tasks passed.
- Root lint: 23 tasks passed.
- Root format check and `git diff --check`: passed.
- Built production CLI: canonical `~/src` remains in ready roster/YAML evidence; generated projection preflight succeeds with no blockers; a bare path flag emits one blocked JSON object, exit 1, and no stderr.
- Independent high-effort late-audit review: no blocker remained in the four assigned repair surfaces; separate exact-tree security audits found no qualifying newly introduced vulnerability.
- Exact temporary-index synthetic tree includes every tracked changed path; immutable SHA and duplicate reconstruction are recorded in the final handoff.
## Risks / blockers
- M4-001 emits rollback prerequisites/evidence only; executable rollback/canary and the managed/unmanaged host fixture remain owned by M4-002.
- Remote/connector entries remain inventory-only; later federation or connector reconciliation requires separately reviewed work.
- Existing environment data is only preflighted. Cutover backup, quarantine write, legacy removal, and generated projection application remain later reviewed effects.
docs/archive/work-records/791-upgrade-config-protection.md
+556
View File
@@ -0,0 +1,556 @@
# Scratchpad — #791 Upgrade config protection (ms-791 worker lane)
**Lane:** web1:ms-791 → reports to MS-LEAD (web1:mosaic-100). Do NOT contact Jason/Mos directly.
**Worktree:** `/home/hermes/agent-work/stack-agents-dir-791`, branch `feat/791-upgrade-config-protection`
off `origin/main` `9745bc3f` (verified exact head).
## Mission prompt (verbatim intent)
Protect operator-owned config under `~/.config/mosaic` from framework-upgrade wipes. Ratified
combination (Mos-approved, do NOT re-litigate): (b) strict ownership separation [PRIMARY] + (a)
transactional pre-update snapshot [safety net] + (d) regeneration-from-SSOT [recovery]. (c) periodic
timer DEFERRED. HARD GATE: unit test that an upgrade run touches NO path outside the manifest.
Design-first: write design doc, send to MS-LEAD, WAIT for confirmation before impl.
## Session 1 (2026-07-16) — Phase 1 design
### Evidence gathered (wipe mechanism, file/line)
- `mosaic update` → `update-checker.ts:509` `buildReseedCommand` → `bash install.sh`
(`MOSAIC_SYNC_ONLY=1`, `MOSAIC_INSTALL_MODE=keep`).
- Wipe = `packages/mosaic/framework/install.sh:199` `rsync -a --delete` + `PRESERVE_PATHS` denylist
(`install.sh:47`). cp-fallback `install.sh:223` `find ... -exec rm -rf`.
- Denylist gaps → WIPED: `agents/*.conf`, `policy/*.md`, `*.local.md`, harvester/SOP,
`tools/_lib/credentials.json`.
- Stale comment `update-checker.ts:492` claims `*.local` preserved — PRESERVE_PATHS has no such entry.
- TS path `file-adapter.ts:157` → `file-ops.ts:66` `syncDirectory` = non-destructive copy-overlay, BUT
its preserve list (`file-adapter.ts:164`) already DRIFTED from install.sh (missing `fleet/backlog`,
`fleet/roles.local`). Evidence for single shared manifest SSOT.
- Existing snapshot (`install.sh:76`) = /tmp, crash-trap only, deleted on success → inadequate; no
`mosaic restore`.
- `fleet-reconciler.ts:93,234` already has `regenerate-projections-from-roster` phase separate from
lifecycle → `mosaic fleet regen` = thin projection-only wrapper (no restart), no FCM-M4/M5 preemption.
### Design decisions
- **(b)** Invert to allow-list: shared `framework/framework-manifest.json` (framework globs +
operatorReserved carve-outs); resolve per-path, deny-wins; **UNKNOWN ⇒ operator (fail-safe)**.
Mechanism: drop `--delete`; non-deleting bulk copy + explicit manifest-scoped prune pass (iterate
framework globs only → operator/unknown structurally unreachable). Pure prune-planner fn for tests.
- **(a)** Snapshot to `~/.local/state/mosaic/backups/pre-update-<ts>/` 0700/0600, retention N=5,
post-sync verify+restore, `mosaic restore --list/--from`. No secret values in output.
- **(d)** `mosaic fleet regen` projection-only, preview-first, never restart.
- HARD GATE test includes a deliberately-unanticipated operator path to prove fail-safe default.
- **PR split:** PR1 manifest+guard (root fix, ships alone) → PR2 snapshot/restore (secrev) → PR3
regen+docs. PR2/PR3 depend on PR1.
### Status
Design doc written: `docs/design/791-upgrade-config-protection.md`. Sent to MS-LEAD.
## Session 1 (cont.) — MS-LEAD CONFIRMED → Phase 2 GO
All 4 asks approved. Binding conditions:
- TDD tests-first, red-first proof per PR; ≥85% new-code; co-located `*.spec.ts`; never `--no-verify`.
- HARD GATE test (§2.4, unanticipated sentinel survives byte-identical + mtime unchanged) = MERGE-BLOCKING for PR1.
- Manifest-completeness test (§6.2) required.
- Bash+TS read ONE shared `framework-manifest.json`; parity test (§6.1) required (closes #631 drift class).
- UNKNOWN⇒operator (rule 3) non-negotiable. Keep prune-planner PURE.
- `fleet regen`: NEVER restart; dry-run default, `--write` to apply; "never issues restart" test mandatory.
- Independent review every PR; PR2 dedicated secrev.
- One PR at a time through DAG. Report PR1 exact head + red→green evidence for review commission.
### Now: implementing PR1 (manifest + resolver + non-deleting sync + scoped prune + guard tests).
## Session 2 (2026-07-16) — PR1 built, tests-first, red→green proven
Deviation noted to MS-LEAD in PR: manifest is `framework-manifest.txt` (line-oriented), NOT `.json`.
Rationale: keep the bash installer free of a python3/jq dependency. The "ONE shared file, parity-
tested" requirement is honored — `manifest-parity.spec.ts` drives the bash resolver as a subprocess
and asserts byte-identical ownership vs the TS resolver over 34 probe paths spanning every class.
### PR1 artifacts
- SSOT: `packages/mosaic/framework/framework-manifest.txt` ([framework]/[operator], deny-wins, fail-safe).
- TS resolver: `src/framework/manifest.ts` (pure: parse/matchGlob/resolveOwnership/frameworkSubtreeRoots/
planPrune) + `manifest.spec.ts` (18 tests incl. planPrune property test + §6.2 completeness).
- Bash resolver: `framework/tools/_lib/manifest.sh` (compiled globs → fork-free `manifest_is_framework`;
CLI `resolve|subtree-roots|classify`). Sourced by install.sh.
- HARD GATE (§2.4): `framework/tools/quality/scripts/test-upgrade-manifest-guard.sh` — keep-mode reseed,
10 operator sentinels (incl. unanticipated `unknown-operator-dir/x`, `harvester/sop.md`,
`fleet/my-fleet.yaml`) survive byte-identical + mtime-unchanged; retired framework file pruned;
secret value absent from output. RED=31 fail (orig install.sh) → GREEN=48 pass (fixed).
- install.sh: keep mode now manifest-driven (`sync_framework_keep`, no `--delete`); overwrite unchanged.
PRESERVE_PATHS denylist deleted.
- TS sync: `file-ops.syncDirectory` gains `isOperatorOwned` guard; `file-adapter.syncFramework` derives
it from `loadManifest` — hardcoded (drifted) preservePaths deleted. Fixture uses the REAL manifest.
- Parity: `manifest-parity.spec.ts` (§6.1) — bash↔TS agree on 34 paths + subtree roots.
- Migration matrix `test-install-migration.sh`: F6 flipped — `my-fleet.yaml` now MUST survive (fail-safe).
- CI: new merge-blocking `upgrade-guard` step (`.woodpecker/ci.yml`) runs both bash suites (adds rsync).
- update-checker.ts reseed comment corrected to the manifest model.
### Gates (all green)
- `pnpm typecheck` ✓ · `pnpm lint` ✓ · `pnpm format:check` ✓
- Full mosaic vitest: 1062 passed (cli-smoke needs `pnpm build` first — build-artifact dep, not this change).
- HARD GATE 48/48 · migration 21/21 · parity 3/3 · manifest 18/18 · file-adapter 8/8.
### PR opened + reported (2026-07-16)
- **PR #802** http://git.mosaicstack.dev/mosaicstack/stack/pulls/802 — base `main`@`9745bc3f`,
head `34e55d4a` (commit `feat(mosaic): manifest-owned upgrade guard…`). 15 files, +1160/-142.
- Reported PR head + red→green evidence to MS-LEAD (web1:mosaic-100); queued (lead busy).
Standing by for the independent-review commission at head `34e55d4a`.
- **TWO items flagged to MS-LEAD for decision (awaiting reply):**
1. Deviation `.txt` vs `.json` — confirm accept (parity-tested) or convert to `.json`+jq.
2. `pr-create -i 791` appended `Fixes #791` → would auto-close the tracking issue on PR1 merge
while PR2/PR3 remain. Recommended edit to `Part of #791`; awaiting go-ahead to patch PR body.
- DO NOT start PR2/PR3 until PR1 merges (DAG; one PR at a time).
### MS-LEAD ruling → #797 ledger-survival sentinel folded into PR1 (2026-07-16)
MS-LEAD ruled both my decisions: (1) `.txt` format ACCEPTED (parity must be strict/merge-blocking incl.
format edge cases + negative probe); (2) trailer `Fixes #791`→`Part of #791` APPROVED (patched PR #802
body via Gitea API — tracking issue no longer auto-closes on PR1 merge). Plus Mos-ELEVATED merge-blocker
(spec `~/agent-work/planning/epic-796/791-ledger-survival-sentinel-SPEC.md`): #797 Runtime Session Ledger
must survive upgrade. Two coupled deliverables landed in PR1:
- (i) Carve-out: `fleet/run/**` was ALREADY an explicit `[operator]` entry — glob matches the spec's
pinned `fleet/run/**` EXACTLY, so NO divergence to route back to planner-opus. Strengthened its comment
to name the ledger (`fleet/run/sessions/` events.ndjson + ledger.json) so it is unmistakably load-bearing.
- (ii) HARD-GATE sentinel: seeded populated ledger (events.ndjson 3 events + ledger.json node+edge+gen,
0600 under 0700) into test-upgrade-manifest-guard.sh sentinels; asserts byte-identical + mtime-unchanged
- dir-perms unchanged. Negative control (retired framework file IS pruned) relabeled explicitly.
HARD GATE now 58/58 (was 48).
- Decision-1 parity hardening: format-edge fixtures (comments/blanks/whitespace, duplicate+overlapping
globs deny-wins, section/glob-ordering independence) + explicit UNKNOWN→operator negative probe, driven
through BOTH resolvers via MANIFEST_FILE override. Parity 7/7 (was 3).
- RED-FIRST honesty note: the bash ledger sentinel stays GREEN even against the pre-fix installer (the
ledger was incidentally safe from the rsync --delete bug; overall pre-fix run 30/58 as expected). The
carve-out's TRUE load-bearing value (deny-wins if framework ownership ever broadens to `fleet/**`) is
isolated by a dedicated resolver-seam red→green in manifest.spec.ts: WITHOUT `fleet/run/**` operator
entry + hypothetical `fleet/**` framework → ledger resolves framework and planPrune DELETES it (RED);
WITH the carve-out → deny-wins → operator, unprunable (GREEN). manifest.spec.ts 21/21 (was 18).
- Gates all green: typecheck ✓ lint ✓ format:check ✓ · full mosaic vitest 1069 passed · HARD GATE 58/58
· migration 21/21. Committing FORWARD on the branch (NOT rebasing 34e55d4a out from under review).
### MS-LEAD REQUEST CHANGES @ 0a5e703a → B1/B2/B3 fixed red-first (2026-07-16)
MS-LEAD returned REQUEST CHANGES (routed merge-blockers satisfied; 2 CRITICAL reliability defects from
the commissioned independent review). Fixed forward on the branch, red-first:
- **B1 (CRITICAL) — dead ERR trap.** install.sh had `set -euo pipefail` (no `-E`), so the
`trap restore_snapshot ERR` never fired for a failure inside sync_framework_keep() (function body) —
a mid-sync abort left a half-written target with NO rollback. Fix: `set -Eeuo pipefail` (errtrace) +
disarm the trap at the top of restore_snapshot() to prevent re-entrancy. New gate
`test-upgrade-rollback.sh`: injects a mid-sync `cp` EACCES (read-only divergent framework file);
Part A asserts the shipped installer rolls back (restore message fires AND target byte-identical to
pre-upgrade); Part B control strips `-E` and asserts the rollback message does NOT fire (dead trap) —
self-verifying red→green. 7/7.
- **B2/B3 (CRITICAL) — empty/unreadable/malformed manifest divergence.** Pre-fix: TS `parseManifest('')`
returned `{framework:[],operator:[]}` (NO throw) → silent no-op "Installation complete"; bash aborted
fragilely (the `_manifest_compile` `"${MANIFEST_OPERATOR[@]:-}"` artifact returned 1 with no message)
AND the CLI dispatch swallowed manifest_load's rc (no `|| exit`) so `resolve` exited 0 resolving
everything operator. Fix (fail-loud + identical both langs):
- TS `parseManifest`: throw on zero framework entries; `loadManifest`: wrap read error →
"Cannot read framework manifest …".
- bash `manifest_load`: explicit unreadable guard (`[[ ! -r ]]`) + zero-`[framework]` guard, both loud
stderr + return 1; `_manifest_compile` gets explicit `return 0` (kills the empty-array artifact);
CLI dispatch `manifest_load … || exit 1`.
- `finalize.ts`: wrap syncFramework → `spin.stop('Framework sync aborted …')` + rethrow (never falls
through to "Installation complete").
Tests: manifest.spec.ts +5 fail-closed (empty/comment-only/operator-only/empty-section/missing);
manifest-parity.spec.ts +7 failure-mode parity (both reject empty/comment-only/operator-only/
empty-section/entry-before-header/unknown-header/missing — TS throws, bash CLI exits non-zero+stderr);
HARD GATE +4 end-to-end fail-closed matrices (empty/operator-only/malformed/missing → abort non-zero,
manifest error surfaced, every operator sentinel byte-identical). RED proven by reverting
manifest.ts+manifest.sh to HEAD → 12 new tests fail; restore → 40/40 green.
- **Non-blocking addressed.** MEDIUM install.sh:222 find-empty now warns on a real failure instead of
blanket `|| true`. LOW: corrected the "both destructive paths rsync vs cp" overstatement in the HARD
GATE header + cp-fallback comment + ci.yml (keep mode is a single cp-based path; the rsync-present vs
-absent runs prove rsync-independence). `.pre-constitution.bak` triage: single-shot backup is
intentional (reconcile_framework_files backs up once), no change.
- Gates: typecheck ✓ lint ✓ format:check ✓ · full mosaic vitest 1081 passed (was 1069, +12) · HARD GATE
118/118 (was 58) · rollback 7/7 (new) · migration 21/21. No --no-verify. Rollback test wired into
ci.yml upgrade-guard. Committing FORWARD (no rebase of 34e55d4a/0a5e703a).
### Codex round 2 (pre-push self-review) → blockers A/B + should-fix C fixed red-first (2026-07-16)
Before committing round 1 I re-ran codex on the change set; it surfaced two fresh reliability defects
and one messaging defect on the SAME rollback/manifest path. Fixed forward, red-first:
- **Blocker-A (CRITICAL) — signal trap resumed instead of terminating.** A bash INT/TERM handler that
merely `restore_snapshot` (returns) does NOT terminate the script — execution RESUMES past the
interrupt, cleans the snapshot and reports success, leaving a partial post-interrupt update. Fix:
`trap 'restore_snapshot; exit 1' ERR INT TERM` so both the errtrace (ERR) and signal (INT/TERM) paths
exit non-zero. Rollback test Part C: a `cp` shim that `kill -TERM $PPID` mid-sync then succeeds (so
set -e never fires and only the signal path governs) → asserts abort non-zero + restore fires + does
NOT print "file phase complete"; control strips `exit 1` and asserts the buggy resume-to-success.
- **Blocker-B (CRITICAL) — degenerate `[framework]` section resolved everything operator.** A manifest
whose framework entries are all empty / bare-dot (`/`, `./`, `.`, `..`) passed the non-empty guard yet
yielded zero usable globs → nothing is framework → a keep-mode sync silently no-ops (bash resolved
`operator`, exit 0). Fix (both langs, parity): reject when no entry has a char other than `/`/`.` —
TS `isUsableFrameworkGlob` = `/[^/.]/.test(normalizeRel(glob))`, throws `ManifestError`; bash mirror
loops `[[ "$(_manifest_norm "$_g")" =~ [^/.] ]]`, loud stderr + return 1. Tests: manifest.spec.ts
`it.each(['/','./','.','..','/\n./'])` throw; parity +3 `expectBothReject` (root-slash/dot-slash/
bare-dot). RED: reverting the guard makes `[framework]\n/` resolve `operator` exit 0.
- **Should-fix-C — misleading abort message.** finalize.ts printed one generic "may be partially
applied" for every sync failure. A `ManifestError` is a PRE-sync validation abort (manifest is
validated before any copy) → nothing was written; conflating it with a mid-copy failure misdirects
recovery. Fix: introduce `ManifestError` (exported from manifest.ts, thrown by every fail-closed
parse/load path), and classify in finalize.ts — ManifestError → "no files were changed"; any other →
"may be partially applied". New co-located `finalize-sync-abort.spec.ts` (3 tests) asserts both
branches re-throw the original error + the correct message, and that config writes are never reached.
RED proven by collapsing the classification → the ManifestError test fails.
- Gates: typecheck ✓ lint ✓ format:check ✓ · full mosaic vitest 1094 (was 1081, +3 finalize-abort;
manifest specs already counted) · HARD GATE 193/193 · rollback 14/14 · migration 21/21.
### Codex round 3 (pre-push self-review) → blockers D1/D2 fixed red-first (2026-07-16)
Re-ran codex again; it found two more rollback-path gaps `set -E` cannot catch. Fixed forward, red-first:
- **Blocker-D1 (CRITICAL) — `find` scan failures swallowed by process substitution.** Both the overlay
copy and the scoped prune consumed `< <(find … -print0)`. Bash does NOT propagate the producer's exit
status to the `while`, so an EACCES/I/O failure mid-scan truncates the file list yet leaves the loop
exiting 0 → a partial upgrade commits and reports success; the ERR/restore trap never fires. Fix:
`_scan_or_die` runs `find … -print0 > "$tmp"` to completion, checks its status, and returns non-zero
(→ ERR trap → restore) on failure; both loops now read from the checked temp file. Rollback test
Part D: a `find` shim that fails every `-print0` scan → shipped installer aborts non-zero + restores +
emits "Could not enumerate framework files" + target byte-identical; control neuters the `# D1-GUARD`
`return 1` → find failure swallowed, upgrade wrongly reports "file phase complete", no rollback.
- **Blocker-D2 (CRITICAL) — silent `set -e` exit on a failed target reset.** restore_snapshot did a bare
`rm -rf "$TARGET_DIR"; mkdir -p "$TARGET_DIR"` (trap disarmed, under set -e). If `rm`/`mkdir` fails —
possibly after `rm` deleted part of the target — the script exits immediately, skipping the cp AND the
recovery pointer, leaving a half-removed target and an orphaned snapshot the operator can't locate.
Fix: `if ! rm -rf … || ! mkdir -p …; then fail "Snapshot restore could not reset … preserved at:
$SNAPSHOT_DIR — copy it back …"; return 1; fi` (tested like the cp -a check; snapshot NOT deleted).
Rollback test Part E: cp-poison triggers restore + an `rm` shim fails `rm -rf <TARGET>` → shipped
emits the recovery pointer, the named snapshot dir survives, secret value never leaked; control deletes
the recovery line → operator gets no pointer. RED: reverting D1+D2 → 7 shipped/control assertions fail.
- Gates: typecheck ✓ lint ✓ format:check ✓ · full mosaic vitest 1094 · HARD GATE 193/193 ·
rollback 28/28 (was 14, +14 for D1/D2 with controls) · migration 21/21. shellcheck clean on new lines.
No --no-verify. Committing FORWARD (no rebase of 34e55d4a/0a5e703a).
## Session 3 (2026-07-16) — PR1 MERGED, starting PR2 (durable snapshot + restore + secrev)
PR1 (#802) squash-merged → main `32a0ffba`; issue #791 stays open (3-PR DAG umbrella). Independent Opus
adversarial/security review APPROVED at head `af627e75` (Gitea RoR cmt 17892); lead ran rollback 28/28 +
HARD GATE 193/193 green; CI #1877 green. PR2 UNBLOCKED.
PR2 branch: `feat/791-pr2-snapshot-restore` off `origin/main` 32a0ffba. Same treatment applies:
tests-first red-first, independent review + durable Gitea Reviewer-of-Record comment BEFORE MS-LEAD runs
the queue guard/merge. Report PR2 number + exact head when ready. PR body: `Part of #791` (NOT Fixes).
### PR2 scope (ratified §3/§5 of design doc, Mos-approved — do NOT re-litigate)
- **(a) Durable pre-update snapshot** to `${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-ts>/`
— OUTSIDE ~/.config/mosaic and any repo. Perms dir 0700 / files 0600 (umask 077 + explicit chmod).
Scope = operator-owned surface that EXISTS (operatorReserved paths), not the framework tree. Taken
BEFORE any mutation. Retention N=5 (`MOSAIC_BACKUP_RETENTION`), prune older.
- **Post-sync verify + selective restore**: diff operator surface vs snapshot; (b) should never touch
operator paths, so ANY diff = manifest bug → restore affected paths + warn loudly. (a) catches a (b) miss.
- **`mosaic restore`** (TS CLI): `--list` (default, dry-run) enumerates snapshots by ts; `--from <ts>`
restores over operator surface, confirmation-gated. Counts/paths only.
- **Secret-safety (secrev)**: snapshot/restore NEVER emit file contents; only paths/counts. Tests assert
0700/0600 AND that a secret value seeded in tools/\_lib/credentials.json never appears in any output.
### PR2 implementation status (2026-07-16, ready-for-review)
All three tasks implemented, red-first proven, unit-green:
- **Task #10 — durable snapshot (install.sh)**: `backup_root()`/`enumerate_operator_files()`/
`prune_durable_snapshots()`/`make_durable_snapshot()` wired into keep-mode main() after `manifest_load`,
before any mutation. umask 077 + explicit chmod 700/600. UTC ts, collision suffix. FAIL-OPEN (a backup
failure never aborts the upgrade it protects). Retention `MOSAIC_BACKUP_RETENTION` (default 5), in-place
`sort -r -o` prune (no `mv` — stays inside the rsync-absent coreutils whitelist).
- **Task #11 — post-sync verify net (install.sh)**: `verify_operator_surface()` runs after sync (trap
disarmed), `cmp -s` each snapshot file vs target; restores any diverged/missing operator file + warns
loudly (a divergence = manifest bug). VERIFY-NET wired before `cleanup_snapshot`.
- **Task #12 — `mosaic restore` (TS)**: `src/commands/restore.ts` + co-located spec (19 tests).
`--list` default (dry-run enumerate), `--from <ts>` confirmation-gated restore, `--dry-run`, `--yes`/
`MOSAIC_ASSUME_YES`. Injectable `confirm` for testability (proceed/decline/env-bypass covered). Restored
files forced 0600. Registered in `cli.ts`. Path convention mirrors install.sh `backup_root()`.
- **CI**: `.woodpecker/ci.yml` upgrade-guard runs the new `test-upgrade-durable-snapshot.sh` gate.
- **Gates green**: typecheck ✓ lint ✓ format:check ✓ · full mosaic vitest 1241 (+5) ·
durable-snapshot 26/26 · manifest-guard 193/193 · rollback 28/28 · migration 21/21.
Est. new-code coverage ≈93% (only the interactive readline default + process.exit-on-error uncovered).
- Regression fixed: PR2's `date`/`sort`/`mv` broke the rsync-absent manifest-guard PATH whitelist →
made date/sort fail-open, replaced `mv` with in-place `sort -o`, added `date sort` to the test whitelist
- isolated `XDG_STATE_HOME`. All 193 manifest-guard assertions green under restricted PATH.
- Codex code-review + security-review (secrev) run on the uncommitted diff before commit.
### PR2 review round 1 — findings + remediations (2026-07-16, pre-PR)
Codex code-review returned **request-changes** (1 blocker + 3 should-fix); Codex security-review returned
**high** (1 high + 1 medium). Deduped to 5 distinct defects, ALL legitimate, ALL fixed FORWARD, each with
a red-first regression test whose control neuters exactly the guard under test:
- **A · BLOCKER — verify net undid the legacy bin/ migration (install.sh).** On a pre-v2 install `bin/**`
is operator-classified, so the durable snapshot captured it; `run_migrations()` deletes bin/ on purpose,
but `verify_operator_surface()` then saw it "missing" and healed it back — the migration would be silently
undone forever once the version stamps. **Fix:** `MIGRATION_REMOVED_PATHS[]` recorded by run_migrations
(`bin`,`rails`) + `is_migration_removed()` skip in the verify loop (`# MIGRATION-SKIP-GUARD`).
**Test:** Part 6 — v1 fixture with bin/; shipped keeps it removed + stamps v3; control (guard stripped)
wrongly restores bin/tool.sh.
- **B · HIGH (CWE-59) — restore/verify wrote secrets THROUGH a symlink (install.sh + restore.ts).** An
attacker swapping an operator path (e.g. tools/\_lib/credentials.json) for a symlink after the snapshot
would make `cp`/`copyFileSync` write the snapshot's secret out through the link. **Fix (bash):** refuse a
symlinked ancestor (`has_symlinked_parent`), drop a symlinked leaf before restore
(`# SYMLINK-LEAF-GUARD`). **Fix (TS):** reuse audited `secure-file.ts` — `assertCanonicalContainment`
- `ensureManagedDirectory` on every dst, open the leaf `O_NOFOLLOW|O_CREAT|O_TRUNC` 0600 (ELOOP =
fail-closed). **Tests:** Part 7 (shipped leaves external exfil target untouched, restores a real 0600
file; control leaks the secret through the link) + restore.spec symlinked-leaf/ancestor cases (red-first).
- **C · MEDIUM/should-fix (CWE-22) — `--from` traversal escaped the backup root (restore.ts).**
`join(root, from)` accepted `../poison`. **Fix:** validate the selector against
`^\d{8}T\d{6}Z(?:-\d+)?$`, build exactly `join(root,'pre-update-'+ts)`, `lstat` (reject symlinked snap
dir). **Test:** restore.spec `it.each` of 6 malformed selectors + `--from ../poison` fail-closed (red-first).
- **D · should-fix — verify `mkdir -p` unguarded under set -e (install.sh).** A parent replaced by a
regular file aborted the installer before the recovery pointer printed. **Fix:** guard `mkdir -p`, warn
- `continue` on failure (keeps healing remaining files).
- **E · should-fix — snapshot `umask 077` leaked process-global (install.sh).** Later sync copies/dirs
inherited 0600/0700. **Fix:** save `old_umask`, restore on EVERY return path (`# UMASK-RESTORE-NORMAL`).
**Test:** Part 8 — synced framework file is 0644 while the secret backup stays 0600; control (restore
stripped) makes the synced file 0600.
**Full gate suite re-run after fixes (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic
vitest **1252** · restore.spec **30** · durable-snapshot **41** · manifest-guard 193 · rollback 28 ·
migration 21. shellcheck clean on all new lines; new test markers mirror the existing `# VERIFY-NET`
anchor convention. NOTE: codex self-review does NOT satisfy the independent-review gate — an independent
(author≠reviewer) review + durable Gitea Reviewer-of-Record comment is still required before MS-LEAD merges.
## Session 4 (2026-07-16) — PR2 MERGED, PR3 built (fleet regen — recovery layer)
PR2 (#811) squash-merged → main `31607a4a`; issue #791 stays open (final PR of the 3-PR DAG). Independent
exact-head RoR at `d12c5f78` APPROVE (Gitea cmt 17904); #1882 green; busybox-portable Part 7 control fix
verified in-Alpine. PR3 UNBLOCKED.
PR3 branch: `feat/791-pr3-fleet-regen` off `origin/main` 31607a4. Same discipline: tests-first red-first,
independent review + durable Gitea RoR BEFORE MS-LEAD runs the queue guard/merge. PR body `Part of #791`.
### PR3 scope (ratified §4/§7 of design doc) — `mosaic fleet regen`
Projection-only recovery command: rebuilds each `fleet/agents/<name>.env.generated` from `roster.yaml`
(SSOT). Dry-run default; `--write` applies; `--json` machine output. Structural guarantee: NO code path to
systemd lifecycle — **never restarts an agent**. Single-SSOT: reuses `projectRosterV2AgentGeneratedEnv`
(extracted, shared with the reconciler apply path) so regen and reconcile cannot drift. Secrev: paths +
counts only, never the rendered KEY=value body.
New files: `commands/fleet-regen-command.ts` (+ `.spec.ts`), guide `docs/guides/upgrade-safety-and-recovery.md`
(three-layer model: PR1 manifest ownership → PR2 snapshot/restore → PR3 regen; do-NOT-restart-before-verify
runbook), regen reference added to `docs/guides/fleet-local-canary.md`. Wired in `commands/fleet.ts`.
### Independent review (3 reviewers: subagent code-reviewer + codex code-review + codex security) → 4 fixes, red-first
- **A · BLOCKER (codex) — regen mutated/deleted legacy operator env.** `applyPreparedAgentEnvironmentProjection`
also writes `.env.local`/`.env.quarantine` and unlinks legacy `.env`. Violated projection-only contract.
**Fix:** NEW generated-only boundary primitives `prepareGeneratedAgentEnvironmentProjection` +
`applyPreparedGeneratedAgentEnvironmentProjection` (write ONLY `<name>.env.generated`). regen now has no
code path that touches `.env`/`.env.local`/`.env.quarantine`. **Test:** projection-only leaves legacy `.env`
verbatim, no local/quarantine fabricated.
- **B · should-fix (codex + subagent + security) — partial write on mid-loop failure.** Interleaved
prepare/apply left earlier agents written when a later agent failed prepare. **Fix:** PREPARE ALL agents
before writing ANY (mirrors reconciler `defaultPrepareProjections`). **Test:** 2nd agent's projection
pre-seeded 0644 → prepare rejects → coder0 NOT written, exit 1.
- **C · subagent — semantic-validation bypass.** Default readRoster skipped `validateRosterV2Semantics`, so
a tampered protected-class `tool_policy` would be silently projected. **Fix:** default readRoster now runs
`validateRosterV2Semantics` (persona resolution + protected-class match), rolesDir/overrideDir defaults
mirroring the reconciler. **Test:** merge-gate agent w/ tool_policy=code → fails closed, no write.
- **D · MEDIUM (codex security, CWE-362) — concurrent-reconcile race.** regen `--write` wrote without the
reconcile lock. **Fix:** `--write` acquires `acquirePrivateReconcileLock(mosaicHome)` for the whole
read-prepare-apply sequence, released in `finally`; dry-run stays lock-free. **Test:** pre-held lock →
regen fails closed, no write.
**Gate suite after fixes (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest **1265**
(regen spec 13, incl. 4 new red-first regressions). NOTE: codex self-review does NOT satisfy the
independent-review gate — an independent (author≠reviewer) review + durable Gitea RoR is still required
before MS-LEAD merges. STOP at PR-open for MS-LEAD's exact-head review; do NOT self-merge.
## Session 5 — PR3 review round 2 (finding L + M1/M2/M3), red-first fixes
Second review pass on the lock-cleanup plumbing surfaced one round-1 residual (L) and three round-2
findings (M1 blocker, M2/M3 should-fix). All fixed red-first (RED proven per-finding, then GREEN).
- **L · should-fix (codex r1) — mutation-lock release swallowed unlink failures.** regen's
`acquirePrivateRosterMutationLock` release copied CRUD's `unlink().catch(()=>{})`, hiding a stale
`roster.yaml.mutation.lock`. **Fix:** its release PROPAGATES the unlink fault (finding-J stale-lock
warning then fires for this lock too). **Test:** acquire real lock, `rm` it, assert `release()` rejects.
- **M1 · BLOCKER (codex r2) — replacement-lock race.** The propagating release from L did an
UNCONDITIONAL `unlink(lockPath)` without proving ownership. If the lock is cleared + re-created by
another writer mid-op, regen deletes the STRANGER's live lock → a third writer enters → mutual
exclusion defeated. **Fix (reuse, not reimplement):** generalized the reconciler's ownership-proving
lock body into shared `acquirePrivateManagedRosterLock(mosaicHome, lockLeaf, busyMessage, openLock)`;
`acquirePrivateReconcileLock` delegates to it (behavior-identical: same leaf/codes/messages), and a NEW
hardened `acquirePrivateRosterMutationLock` (now in fleet-reconciler.ts, leaf `roster.yaml.mutation.lock`)
records dev/ino + ownership token and RE-PROVES ownership (`assertLockOwnership`) before unlinking —
fails closed as `lock-cleanup-failed` if replaced. Removed the crud-based export; reverted
`acquireMutationLock` (fleet-agent-crud.ts) to its original inline empty-file/swallowing-release form
(CRUD behavior intentionally unchanged). Compatibility: CRUD empty-file `wx` and regen tokened `wx`
contend on the same path but never co-own (wx winner owns; loser → concurrent-mutation), so the token
is only ever read back by the same regen invocation. **Test:** acquire, `rm`+recreate lock (new inode),
assert `release()` rejects AND the replacement survives (not unlinked).
- **M2 · should-fix (codex r2) — acquire-unwind fault dropped.** The acquire-failure catch discarded
`releaseFleetLocks`' return (a possible fault on the already-held first lock). **Fix:** capture and
augment — `const releaseFault = await releaseFleetLocks(releases); throw augmentWithLockCleanupFault(error, releaseFault);`
(symmetric to finding J). **Test:** mutation lock acquires w/ faulting release + reconcile acquire
throws → thrown error mentions stale/lock, nothing written.
- **M3 · should-fix (codex r2 + subagent REQUEST-CHANGES) — cleanup warning named only reconcile lock.**
Finding L made the mutation-lock release fault reachable, so the `cleanup` marker can originate from
EITHER lock. **Fix:** `formatFleetRegenReport`'s WARNING now names BOTH `roster.yaml.mutation.lock` and
`roster.yaml.reconcile.lock`, matching `augmentWithLockCleanupFault`. **Test:** fault the mutation-lock
release specifically → report names both lock files.
**Refactor note (no cycle):** neither fleet-reconciler nor fleet-agent-crud imports the other; regen
imports lock acquirers from fleet-reconciler and the projection mapping from fleet-reconciler. The two
reconcile-lock reviewers reconciled: independent reviewer validated acquire-time empty-file compatibility
(preserved), codex flagged RELEASE-time replacement race (closed by ownership proof) — non-contradictory.
**Gate suite after fixes (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest **1275**
(regen spec 23, incl. 7 red-first lock regressions E/F/G/K/L/M1/M2/M3). RED proven per-finding by
temporary revert before re-applying each fix. Independent (author≠reviewer) review of M1/M2/M3 + codex
code/security re-run in flight. STOP at PR-open for MS-LEAD's exact-head review + durable Gitea RoR; do
NOT self-merge; #791 umbrella stays OPEN; PR body `Part of #791`.
### Round 3 review (after M1/M2/M3) — independent review PASS + codex residual-TOCTOU disposition
Three reviewers on the post-M1/M2/M3 head:
- **Independent (subagent, author≠reviewer) — PASS.** Verified M1/M2/M3 all correctly fixed; "never
restarts" is STRUCTURAL (runner never referenced in executable code); no secrets; no deadlock (only
regen holds both locks); tests meaningful (assert inode preservation + exact lock-file names). Raised:
- **should-fix #1 (fixed, red-first):** generalizing the lock helper left `assertSafeLockLeafIfPresent`/
`assertLockOwnership` hardcoding "reconciliation lock" in thrown messages → a MUTATION-lock fault
misreported as the reconcile lock, undercutting M3's accurate-diagnosis goal. **Fix:** thread
`lockLabel = fleet/<leaf>` through both helpers + the generic lock-io messages, so every fault names
the actual lock file. Red-first: strengthened the M1 test to assert `/roster\.yaml\.mutation\.lock/`
(RED: got "reconciliation lock"; GREEN after). Also resolves nit #3 (generic-message drift).
- **nit #2 (fixed):** `FleetRegenResult.cleanup` JSDoc still said "the shared reconcile lock"; now names
both locks (regen holds both).
- **nit #4 (fixed):** removed the redundant duplicate `assertLockOwnership` call before unlink
(pre-existing in merged main; harmless but dead — dropped since the fn was already being touched).
- **Codex security — clean (risk: none).** Validates roster semantics, constrains env values, no shell
eval, no secret output, generated-only writes, serialized against both locks.
- **Codex code — request-changes, 1 "blocker": residual check-then-unlink TOCTOU.** Between the final
`assertLockOwnership` and the path-based `unlink`, an external actor could vacate our inode and a new
writer grab the path, so the unlink deletes the stranger's lock. **Disposition: documented known
limitation, NOT fixed in PR3.** Rationale: (1) byte-identical to the MERGED, shipped reconcile-lock
release on origin/main (fleet-reconciler.ts L654-659) — not introduced here; (2) UNREACHABLE within the
`wx` writer protocol — no Mosaic writer removes a lock it doesn't own (wx fails EEXIST while our inode
exists), so only external interference can vacate our inode in the sub-instruction window; (3) the
ownership guard DOES close the reachable case (stale-lock reaper/operator cleared our lock + another
writer took it BEFORE release began → fail closed, don't delete stranger's lock); (4) the true atomic
fix — fd-held advisory lock (flock/lockf) adopted by ALL fleet writers (CRUD + reconcile + regen) — is
a cross-cutting mechanism change touching merged CRUD + reconciler, out of scope for a projection-only
recovery PR. Documented honestly in the acquirer doc + M1 test comment. **The binding independent
review did NOT treat this as a blocker.** Recommendation to MS-LEAD: proceed to PR-open + spin a
SEPARATE follow-up issue for the fd-advisory-lock migration; MS-LEAD adjudicates scope at exact-head
review (merge authority).
**Gates after round-3 fixes (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest
**1275** (regen spec 23). Fresh codex code re-run in flight to confirm no NEW issues from the label fix.
---
## Session 6 — Round 4/5 convergence (stranded-lock robustness)
**Two independent reviewers converged on the SAME should-fix** on the init-failure cleanup path,
strengthening confidence it was real:
- **Codex code-review-5 — 0 blockers, 1 should-fix.** "Stat failure after lock creation strands the new
lock." When `handle.stat()` ITSELF fails right after the `wx` create (transient EIO/EBADF), `created`
is `undefined`, so `removeOwnedLockLeafBestEffort` had `if (!created) return;` → no cleanup → the
just-created `roster.yaml.mutation.lock`/`reconcile.lock` is stranded, permanently blocking future
regen + CRUD. (Notably NO blocker, and the TOCTOU is no longer flagged in code-review as of r5.)
- **Independent delta reviewer (author≠reviewer, pr-review-toolkit) — no blockers, same should-fix.**
Independently flagged the identical `!created` gap; validated FIX 1 (label threading — no call site
missed, codes unchanged, no test depended on old text) and FIX 2 (dev/ino-guarded cleanup, best-effort,
happy-path release reuses captured dev/ino) as correct. Suggested an unconditional best-effort unlink
in the `!created` branch; I took the **safer** variant below.
- **Codex security-review-5 — 0 crit / 0 high / 1 medium.** The single medium is the SAME residual
check-then-unlink TOCTOU already dispositioned in round 3 (its own remediation = "migrate every writer
to an fd-held advisory lock" = the follow-up issue). No new security finding. No secrets.
**Fix (red-first, safer than an unconditional unlink):** thread the persisted random `token` into
`removeOwnedLockLeafBestEffort`. Two independent ownership proofs now: primary dev/ino (unchanged), and a
**fallback** when the post-create stat failed — read the leaf and unlink ONLY if its content equals our
`randomUUID()` token. Only OUR lock carries that token, so a CRUD (empty) or differently-tokened
replacement is never deleted. `tokenPersisted` guards passing the token (only after `writeFile` lands).
Doubly-degenerate case (stat fails AND token write never landed) leaves the lock in place rather than
risk deleting a stranger's file — requires two independent fs faults on a just-created fd; documented.
- **Red-first proof:** new test `does not strand the lock file when the post-create stat itself fails`
injects a real `wx` create + a Proxy handle whose `stat()` rejects (writeFile/close succeed), asserts
`exists(lockPath) === false`. RED before fix (`expected true to be false` — lock stranded); GREEN after.
- **Also fixed (delta nit #3):** `fleet-regen-command.ts` `acquireRosterMutationLock` JSDoc said "CRUD's
private lock"; the default is the reconciler's hardened ownership-proving acquirer for the same
`fleet/roster.yaml.mutation.lock` path. Corrected.
- **PR-description note (delta nit #2):** FIX 1 also collapsed a pre-existing duplicate back-to-back
`assertLockOwnership` call in the release closure (identical args, no intervening logic) into one — a
no-op simplification of merged code, not a behavior change. Called out so a future reader doesn't
wonder if the duplicate had a purpose.
**Gates after round-4 fixes (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest
**1277** (regen spec now 25: +1 stat-failure stranded-lock regression). Residual TOCTOU still deferred to
the fd-advisory-lock follow-up issue; MS-LEAD adjudicates scope at exact-head review (merge authority).
---
## Session 6 — Round 6 (persona-root wiring)
**Codex code-review-6 — 0 blockers, 1 should-fix (NEW, distinct from the lock work).** "Forward
configured persona directories to regen." `registerFleetRegenCommand` was registered at
`fleet.ts:2069` with only `{ runner, mosaicHome }`, discarding `deps.reconcileDeps.rolesDir` /
`overrideDir`. The regen command ALREADY has those seams (validates roster semantics via
`validateRosterV2Semantics({ rolesDir, overrideDir })`, defaulting to `<mosaicHome>/fleet/roles{,.local}`),
but the top-level wiring never forwarded the configured roots. **Impact:** in a deployment with custom
persona roots, `fleet reconcile` (which honors the overrides) would ACCEPT a roster while `fleet regen`
REJECTS the same roster (persona resolution against the wrong default dir) — blocking the recovery
command and violating the documented "resolves personas the SAME way reconcile does" contract.
**Fix (red-first):** forward `rolesDir`/`overrideDir` from `deps.reconcileDeps` into
`registerFleetRegenCommand` at `fleet.ts:2069`. Red-first test `forwards configured persona roots
(rolesDir/overrideDir) from reconcileDeps into regen`: seeds personas ONLY under a custom root, leaves
the default `<home>/fleet/roles` empty, registers with `reconcileDeps: { rolesDir, overrideDir }`, and
requires `fleet regen` to SUCCEED. RED before fix (`expected 1 not to be 1` — regen validated against the
empty default and exited 1); GREEN after.
**Codex security-review-6 — 0 crit / 0 high / 1 medium.** Same residual check-then-unlink TOCTOU, now
noted at BOTH the release closure and the init-cleanup path; remediation = fd-held advisory lock across
all writers = the SAME deferred follow-up item. No new security finding, no secrets.
**Independent confirmation review of the token-fallback fix (Session 6/round 4) — PASS, no findings.**
All 7 verification points confirmed; reviewer mechanically reverted `removeOwnedLockLeafBestEffort` to
the pre-fix `if (!created) return;` and re-ran the new test → RED (`expected true to be false`),
confirming the test genuinely pins the fix; restored after. No lint/type issues; doc-comment accurate.
**Gates after round-6 fix (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest
**1278** (regen spec now 26: +1 persona-root wiring regression).
---
## Session 6 — Round 7 convergence (review CLOSED for PR-open)
- **Codex code-review-7 — 0 blockers, 1 should-fix = the residual TOCTOU** (previously a "blocker" in r3,
dropped in r4/r5, now re-surfaced as a should-fix). **Codex security-review-7 — 0 crit / 0 high /
1 medium = the SAME residual TOCTOU.** Codex has CONVERGED: the only remaining finding across both
streams is that one race, whose own remediation is "fd-held advisory lock shared by all fleet writers"
= the deferred follow-up. No new distinct finding; the wiring fix introduced nothing.
- **Independent confirmation review of the persona-root wiring fix — PASS, no findings.** Reviewer
mechanically reverted the two forwarded lines → RED (`Roster v2 agent "coder0" class "code" does not
resolve to a readable persona` → exit 1), restored → GREEN (26 regen + 204 fleet tests). Confirmed the
optional-chaining fallback preserves default-deployment behavior and no type/lint issue.
**Review disposition for PR-open:** ALL actionable findings fixed red-first across rounds 3–6 (label
threading, stranded-lock on init failure, stat-failure strand, persona-root wiring). The residual
check-then-unlink TOCTOU is the ONLY open item and is DEFERRED to a follow-up issue (fd-advisory-lock
migration across CRUD + reconcile + regen) — byte-identical to merged origin/main's reconcile-lock
release, unreachable within the `wx` writer protocol (no Mosaic writer removes a lock it doesn't own;
only external `rm`/a stale-lock reaper can vacate the inode mid-release), and its true fix is a
cross-cutting mechanism change out of scope for a projection-only recovery PR. Two independent human-agent
reviews (author≠reviewer) treated it as non-blocking. MS-LEAD adjudicates scope at exact-head review
(merge authority); recommendation = proceed to PR-open + spin the follow-up issue.
**Final gates (all green):** typecheck ✓ · lint ✓ · format:check ✓ · full mosaic vitest **1278**
(regen spec 26). No secret values in any snapshot/projection/report output (counts + paths only). Regen
NEVER issues a lifecycle/restart call (load-bearing recordingRunner gate). STOP at PR-open for MS-LEAD's
exact-head review + durable Reviewer-of-Record before any merge; do NOT self-merge.
docs/archive/work-records/804-install-unknown-flags.md
+86
View File
@@ -0,0 +1,86 @@
# Issue #804 — fail closed on unknown installer arguments
## Objective
Implement Part 1 of Gitea issue #804 only: `tools/install.sh` must reject every unrecognized flag or argument with an actionable STDERR error and nonzero exit before installation starts.
## Scope and constraints
- Preserve all currently recognized options and behavior, including `-y` and `--ref <branch>`.
- No positional arguments are currently accepted by the parser.
- Do not add `--next`, `MOSAIC_NEXT`, prerelease routing, or any Part 2 behavior.
- TDD is mandatory: add and observe a failing process-level regression test before changing `tools/install.sh`.
- Worker lifecycle ends after branch push, PR creation, and coordinator notification; do not merge or close #804.
- Existing launcher-owned changes in `.mosaic/orchestrator/mission.json` and `.mosaic/orchestrator/session.lock` are out of scope and must not be committed.
## Requirements and acceptance criteria
- Unknown input names the offending argument on STDERR.
- STDERR includes a short installer usage hint.
- Exit status is nonzero.
- The installer does not invoke npm or otherwise proceed into installation.
- Existing recognized flags remain unchanged.
## Plan
1. Add a process-level Vitest regression using the installer test location under `packages/mosaic/src/commands/`.
2. Run the focused test and record the expected RED failure.
3. Commit the RED test as `test(#804): ...`.
4. Replace the parser catch-all with a fail-closed STDERR error and usage hint.
5. Update concise installer-facing documentation without introducing prerelease behavior.
6. Run focused tests, shell syntax validation, package tests, lint, typecheck, and format checks.
7. Run independent review tooling and remediate findings.
8. Commit as `fix(#804): ...`, queue-guard, push, open a PR containing `Closes #804.`, notify the coordinator, and exit.
## Budget
- No explicit token cap supplied.
- Working estimate: 8K tokens; narrow two-file behavior/test change plus concise docs and delivery gates.
## Progress
- 2026-07-17: Loaded mission state, issue #804, delivery/QA/documentation rails, and relevant TDD/Vitest/pnpm/Gitea skills.
- 2026-07-17: Confirmed the parser has no legitimate positional arguments and currently drops all unmatched input via `*) shift ;;`.
- 2026-07-17: Installed locked workspace dependencies with a worktree-local pnpm store; no lockfile changes.
- 2026-07-17: Added the process-level unknown-argument regression with an isolated `$HOME` and npm shim.
- 2026-07-17: Replaced the silent catch-all with STDERR error + usage output and exit 2 before preflight or installation.
- 2026-07-17: Initial Codex code review found an unknown option could still be consumed as the `--ref` value. Added a second RED reproducer, then rejected option-shaped/missing `--ref` values without changing valid `--ref <branch>` behavior. The review's launcher-state note is handled by excluding both `.mosaic/orchestrator/` files from commits.
- 2026-07-17: Updated README, user guide, and packaged framework README with the fail-closed argument contract. No API, auth, admin, sitemap/navigation, or publishing surface changed.
## Verification
- RED: `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/install-arguments.spec.ts` — expected failure: installer exited `0` instead of nonzero at the exit-status assertion; confirms the test reproduces the silent-drop defect before production changes.
- Remediation RED: the added `--cli --ref --bogus` case exited `0`, proving `--ref` could swallow an unknown option before the guard was added.
- GREEN: `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/install-arguments.spec.ts src/commands/install-heading.spec.ts` — 2 files, 3 tests passed.
- Situational process check: unknown positional input exited 2, named the input on STDERR, printed usage, and did not call the npm shim.
- `bash -n tools/install.sh` — passed.
- Bare `--ref` process check — exited 2 with `Missing value for --ref` and usage.
- `pnpm --filter @mosaicstack/mosaic test` — 69 files, 1,287 tests passed; framework shell checks passed. The first attempt lacked generated `dist/cli.js`; `pnpm --filter @mosaicstack/mosaic build` restored the required test precondition and the full rerun passed.
- `pnpm lint` — 23/23 tasks passed.
- `pnpm typecheck` — 42/42 tasks passed.
- `pnpm format:check` — passed.
- Codex code re-review against `origin/main` — `approve`, 0 blockers/should-fix/suggestions.
- Codex security re-review against `origin/main` — risk `none`, 0 findings.
## Acceptance evidence
| Criterion | Evidence |
| ------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Unknown input is named on STDERR | Process-level Vitest assertions for `--bogus`, including after `--ref` |
| Short usage hint is printed on STDERR | Vitest usage regex + manual process output |
| Exit is nonzero | Vitest status assertions and manual exit 2 |
| Installation does not proceed | Isolated npm shim marker remains absent |
| Recognized behavior is preserved | Parser cases are unchanged except validation of malformed `--ref`; full Mosaic package suite passed |
| Part 2 is excluded | No `--next`, `MOSAIC_NEXT`, dist-tag, or prerelease routing changes |
## Documentation checklist
- Current canonical `docs/PRD.md` remains unchanged; issue #804 and the coordinator brief supply this bounded defect's acceptance contract.
- Updated installer behavior in root README, user guide, and packaged framework README in the same logical change set.
- API/OpenAPI, auth/permissions, admin operations, developer architecture, sitemap/navigation, and external publishing are not affected.
- Scratchpad remains under `docs/scratchpads/`; no root-hygiene changes.
## Risks and blockers
- Part 2 remains owner-gated under #805 and is intentionally excluded.
- No implementation blocker remains. Independent coordinator RoR, CI, merge, and issue closure remain pending after worker handoff.
docs/archive/work-records/807-glpi-partial-content.md
+64
View File
@@ -0,0 +1,64 @@
# Issue #807 — GLPI list wrappers accept HTTP 206
- **Branch:** `fix/807-glpi-206`
- **Task:** Gitea issue #807
- **Role:** Author-only worker reporting to `mosaic-100`; no self-review or merge
- **Started:** 2026-07-16
## Objective
Fix the shipped GLPI ticket, computer, and user list wrappers so ranged responses with HTTP 206 Partial Content render successfully while genuine HTTP failures remain non-zero errors.
## Scope
- Modify only the three affected list wrappers and a focused shell regression test.
- Do not touch `session-init.sh`, `ticket-create.sh`, or `docs/TASKS.md`.
- Add task-local delivery evidence here as required by the mission protocol.
## Plan
1. Add a deterministic shell harness that copies each wrapper beside stubbed `session-init.sh`, credentials, and `curl` boundaries.
2. Prove RED against the current 200-only gates: 206 must fail before the implementation change.
3. Update all three status gates to accept exactly 200 or 206.
4. Prove GREEN for 206 rendering and genuine 401/500 failures, then run repository quality gates.
5. Commit with co-author attribution, run the push queue guard, push, and open a PR for independent review and merge by the team lead.
## Budget
- No explicit token cap supplied.
- Soft estimate: 8K tokens; narrow single-worker execution with no exploratory scope.
## Progress
- [x] Mission, task, PRD, QA, documentation, and code-review guidance loaded.
- [x] RED regression evidence captured: `test-list-http-status.sh` exited 1; all three wrappers rejected 206 while retaining 401 failures.
- [x] Implementation complete.
- [x] Relevant tests and repository gates green.
- [ ] Commit pushed and PR opened.
## Tests and evidence
- RED (before source fix): `packages/mosaic/framework/tools/glpi/test-list-http-status.sh` → exit 1; ticket/computer/user 206 assertions failed, all 401 assertions passed.
- GREEN: `bash -n packages/mosaic/framework/tools/glpi/{ticket-list.sh,computer-list.sh,user-list.sh,test-list-http-status.sh}` → pass.
- GREEN: `shellcheck packages/mosaic/framework/tools/glpi/test-list-http-status.sh` → pass.
- GREEN: `packages/mosaic/framework/tools/glpi/test-list-http-status.sh` → 6 assertions pass (206 renders and 401 errors for all three wrappers).
- GREEN: `pnpm typecheck` → 42/42 tasks pass.
- GREEN: `pnpm lint` → 23/23 tasks pass.
- GREEN: `pnpm format:check` → all matched files pass.
- Setup note: initial gate attempts could not start because the fresh worktree lacked dependencies; `pnpm install --frozen-lockfile --store-dir /home/hermes/.local/share/pnpm/store` restored the locked workspace dependencies without lockfile changes.
## Acceptance criteria mapping
| Criterion | Evidence |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| HTTP 206 succeeds and renders each ranged list | Focused test's three 206 render assertions pass |
| Genuine HTTP failures remain non-zero with existing diagnostics | Focused test's three HTTP 401 assertions pass |
| Only affected list wrappers change | Diff contains the three status predicates plus focused test/evidence; session and create wrappers untouched |
## Documentation decision
No operator/API documentation change is needed: this restores documented list behavior for a healthy GLPI response without changing command syntax, output, configuration, or public contracts. This task scratchpad records delivery evidence.
## Risks / blockers
- Existing dirty `.mosaic/orchestrator/mission.json` and `.mosaic/orchestrator/session.lock` are runtime-owned and will not be edited or committed.
docs/archive/work-records/808-agent-send-sender-identity.md
+37
View File
@@ -0,0 +1,37 @@
# Issue #808 — agent-send sender identity
## Objective
Fix cross-socket `agent-send.sh` preambles so replies route to the real sender rather than a destination-socket holder session.
## Scope and acceptance criteria
- Prefer exported `MOSAIC_AGENT_NAME` as the authoritative sender session name.
- If it is unset, query the sender's local/default tmux socket for `#S` without destination `-L` arguments.
- Preserve `?` when sender identity cannot be determined.
- Do not alter destination socket dispatch.
- Add red-first regressions for all three identity paths.
## Plan
1. Extend `agent-send.test.sh` with deterministic fake-tmux coverage.
2. Run the test against the unpatched implementation and record RED evidence.
3. Apply the minimal sender lookup fix only.
4. Run the focused suite and repository quality gates.
5. Commit, queue-guard, push, and open an author-only PR for independent review.
## Constraints and risks
- Worker lane is author-only: no self-review or merge.
- `docs/TASKS.md` and mission state are orchestrator-owned and will not be modified.
- Pre-existing runtime changes under `.mosaic/orchestrator/` are excluded from this work.
- Budget: no explicit token cap; keep changes limited to the shell tool, sibling regression test, and this scratchpad.
## Evidence
- RED: `bash packages/mosaic/framework/tools/tmux/agent-send.test.sh` failed on the unpatched implementation with `PASS=12 FAIL=3`; it selected `destination-holder` instead of both `MOSAIC_AGENT_NAME=authoritative-agent` and local session `local-agent`. The genuinely unavailable sender case already exercised and preserved `?`.
- GREEN: `bash packages/mosaic/framework/tools/tmux/agent-send.test.sh` passed with `PASS=15 FAIL=0`; coverage includes env authority, local/default tmux fallback across a destination `-L`, explicit rejection of the destination holder, and `?` fallback.
- Syntax: `bash -n packages/mosaic/framework/tools/tmux/agent-send.sh packages/mosaic/framework/tools/tmux/agent-send.test.sh` passed.
- Quality gates: `pnpm typecheck` (42/42 tasks), `pnpm lint` (23/23 tasks), and `pnpm format:check` all passed after installing the frozen lockfile dependencies. The first install attempt failed because pnpm's configured store pointed at `/root`; retrying with the existing user-owned store (`--store-dir /home/hermes/.local/share/pnpm/store`) succeeded without changing tracked dependency files.
- Documentation: no public API or operator workflow changed; the source comment, regression-test contract, and this implementation record cover the internal bug fix.
- Independent review: intentionally pending for the reviewer assigned by `mosaic-100`; this author-only lane will not self-review or merge.
docs/archive/work-records/812-pr-review-comment.md
+58
View File
@@ -0,0 +1,58 @@
# Issue #812 — durable Gitea PR review comments
- **Lane:** ms-812
- **Branch:** `fix/812-pr-review-comment`
- **Issue:** mosaicstack/stack#812
- **Budget:** 15K working estimate; single focused shell-wrapper/test/docs change.
## Objective
Make the Gitea `comment` action in `packages/mosaic/framework/tools/git/pr-review.sh` use the supported Gitea comments REST API and report success only after provider read-back verifies the created comment against the intended repository, PR, and exact body.
## Plan
1. Add and commit a failing shell regression harness before production changes.
2. Verify RED against the nonexistent `tea pr comment` fallback false-positive.
3. Implement the minimal supported write plus ID-based provider read-back.
4. Document that wrapper write output is not durable provenance until read-back succeeds.
5. Run focused regression tests, touched-package tests, and repository quality gates.
6. Remediate review findings, queue-guard, and push for coordinator-owned independent review. Do not open or merge a PR.
## Progress checkpoints
- [x] RED regression committed and reported to mosaic-100 (rebased commit `770e3f57`)
- [x] Initial minimal fix implemented (rebased commit `ea7f8c57`)
- [x] Rebased cleanly onto main `627cf2bb387f7c84a532d88819903a7679ce0d72`
- [x] Codex blocker remediated by replacing unsupported `tea api` with authenticated REST write/read-back
- [x] Focused, package, and repository gates green
- [ ] Coordinator-owned independent review pending after push
- [x] No PR opened; no self-review or self-merge
## Tests run
- RED after rebase: the regression harness failed against `origin/main` with status 1 after reproducing the old `tea pr comment` zero-exit fallback and false success echo.
- GREEN at resumed head: the same harness passed with REST POST 201 plus GET 200 read-back.
- All `packages/mosaic/framework/tools/git/test-*.sh` harnesses passed.
- `shellcheck -x` passed for the changed scripts; `bash -n` passed.
- Manifest resolver returned `framework` for `tools/git/test-pr-review-gitea-comment.sh`.
- `pnpm test` passed (43/43 Turbo tasks; Mosaic 75 files/1434 tests; Gateway 56 files/628 tests plus documented skips).
- `pnpm typecheck` passed (42/42 tasks), `pnpm lint` passed (23/23), and `pnpm format:check` passed.
- Firewall checks found no user-home paths or operator identities in changed shipped files; no token value is logged or echoed.
## Risks / blockers
- No active implementation blocker. #789 reached terminal merged state and the coordination hold was lifted.
- Review round 1 found one portability blocker: the API base reconstructed `https://$host` and discarded configured schemes/path prefixes.
- Review round 2 found a second subpath portability blocker: clone-derived `get_repo_slug` retained the deployment prefix, duplicating it under `/api/v1/repos/`.
- Round 3 resolves owner/repo relative to the configured Gitea base path for HTTP(S) clones while preserving root-mounted and SSH clone forms. Host matching now compares non-default ports consistently.
- REST transport failures, non-201 writes, malformed/missing created IDs, non-200 read-backs, and read-back mismatches all fail closed.
- Existing approve/request-changes behavior remains covered.
- Independent exact-head re-review remains coordinator-owned.
## Final verification evidence
- URL-portability regression was RED before remediation at the new `http://git.mosaicstack.dev` case and GREEN afterward.
- Round-3 genuine subpath regression was RED against round-2 head `1b190201` and GREEN after the fix: `https://git.example/gitea/owner/repo.git` maps to API repository `owner/repo` under configured base `/gitea`.
- Regression coverage verifies POST and read-back GET for root-mounted HTTP(S), path-prefixed HTTP(S), non-default HTTP port, scp-style SSH, and `ssh://` clone forms.
- Focused shell checks, all git-wrapper harnesses, and full repository test/typecheck/lint/format gates passed after remediation.
- Branch will be force-pushed with lease for coordinator re-verification; no PR opened.
docs/archive/work-records/824-mosaic-skill-cli.md
+112
View File
@@ -0,0 +1,112 @@
# Issue #824 — Mosaic skill CLI and Claude bridge auto-sync
## Objective
Deliver `mosaic skill register|unregister|list` plus install/upgrade reconciliation of every canonical `~/.config/mosaic/skills/*` entry into `~/.claude/skills/`, without clobbering runtime-owned files or directories.
## Scope and constraints
- Issue: mosaicstack/stack#824
- Branch: `feat/824-mosaic-skill-cli`
- M1 runtime: Claude Code only.
- Pi/Codex parity is documentation-only; no non-Claude bridge implementation.
- Do not author the downstream `mosaic-context-refresh` skill.
- Workers do not modify `docs/TASKS.md`, merge, close #824, or touch `main`.
- TDD is mandatory and red-first; filesystem tests use temporary directories only.
- Budget: no explicit token cap supplied; use a focused single-worker implementation with no new dependencies.
## Requirements mapping
1. Register creates the canonical Claude symlink and is idempotent.
2. Names are untrusted: reject empty/escaping/absolute/separator/`..`/leading-dash names before filesystem mutation, with clear CLI stderr and nonzero status.
3. Register repairs only Mosaic-owned dangling symlinks and refuses foreign files, directories, and symlinks.
4. Unregister removes only symlinks pointing inside the canonical Mosaic skills root and is idempotent when absent.
5. List reports registered, dangling, foreign, and canonical-but-unregistered skills.
6. Install and upgrade generically reconcile all canonical skills after framework sync/re-seed, continuing past foreign conflicts without clobbering them.
7. User/developer documentation describes commands, status meanings, security boundaries, and Claude-only M1 scope.
## Plan
1. Add co-located failing Vitest coverage for all filesystem behaviors and auto-sync.
2. Run the focused spec and record the expected RED failure.
3. Commit the red contract as `test(#824): ...`.
4. Implement the skill bridge and Commander command registration.
5. Wire reconciliation into wizard finalize and `mosaic update` re-seed, preserving non-clobber behavior.
6. Update canonical docs and sitemap if navigation changes.
7. Run focused tests, package tests, typecheck, lint, and formatting.
8. Commit implementation/docs as `feat(#824): ...`, queue-guard, push, open PR with `Closes #824.`, fire completion event, and notify the coordinator.
## Progress
- 2026-07-17: Loaded mission/delivery/TDD/documentation rails, issue #824, active mission state, and relevant installer/update paths.
- 2026-07-17: Confirmed `mosaic update` invokes `framework/install.sh` with `MOSAIC_SYNC_ONLY=1`; that path exits before existing post-install skill linking, leaving newly present canonical skills unregistered.
- 2026-07-17: Coordinator addendum classified the user-supplied skill name and runtime symlink target as a path-traversal/symlink-injection surface. Expanded the initial red contract to reject traversal before mutation, preserve every foreign entry, and unregister Mosaic-owned links only.
- 2026-07-17: Implemented the Commander command group and secure generic bridge; wired wizard finalize and successful framework re-seed reconciliation; updated user/developer/installed/root docs and sitemap.
- 2026-07-17: Focused, package-wide, repository baseline, temp-home situational, and independent review gates completed. Ready for scoped feature commit, queue guard, push, and PR handoff.
## Tests and evidence
### TDD evidence
- RED environment attempt: `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/skill.spec.ts` initially could not locate Vitest because this fresh worktree had no dependencies.
- Dependency setup: `pnpm install --frozen-lockfile --store-dir /home/hermes/.local/share/pnpm/store` succeeded. The explicit store was required because machine pnpm config incorrectly resolves the default store under `/root`.
- RED behavior: focused Vitest failed with `Failed to load url ./skill.js ... Does the file exist?`, proving the bridge API was absent.
- RED integration: finalize/update specs failed because no Claude links or `skillSync` result existed.
- RED symlink injection: symlinked Claude/canonical root tests failed because the initial implementation followed ancestor links.
- GREEN after review remediation: `skill.spec.ts` 36/36, `finalize-skills.spec.ts` 6/6, and `update-checker.reseed.spec.ts` 30/30.
### Baseline gates
- `pnpm --filter '@mosaicstack/mosaic...' run build` — pass (fresh-worktree dependency outputs built).
- `pnpm --filter @mosaicstack/mosaic run typecheck` — pass.
- `pnpm --filter @mosaicstack/mosaic run lint` — pass.
- `pnpm --filter @mosaicstack/mosaic test` — pass: 69 files, 1,325 Vitest tests plus framework shell suite.
- `pnpm typecheck` — pass: 42/42 Turbo tasks.
- `pnpm lint` — pass: 23/23 Turbo tasks.
- `pnpm format:check` — pass.
### Situational evidence
A built-CLI temp-home smoke test (no real `~/.claude` or Mosaic config touched) proved:
- register creates the exact link and a second run reports `already registered`;
- list reports registered and unregistered canonical skills;
- `../../etc` exits 1 with `Invalid skill name` and creates no escaped path;
- unregister removes the managed link and a second run reports `already unregistered`;
- a fake successful framework re-seed generically registered both `added-after-setup` and `second-skill` from runtime directory enumeration.
### Review evidence
- Initial uncommitted Codex code/security review described name validation/clobber protection as strong; its only finding was the harness-owned, unrelated `.mosaic/orchestrator/session.lock`, which is excluded from all commits and the PR.
- Exact branch review then identified two remediations: preserve successful framework re-seed status when bridge-wide reconciliation fails, and reject/escape control-character names to prevent terminal/log injection.
- Both findings were reproduced red-first and remediated. A subsequent exact review identified one finalize failure-isolation blocker; a root-wide bridge error now warns and allows wizard doctor/summary/next-steps completion, with a red-first regression.
- All remediations passed the full package and repository gates. Final exact-head review is rerun after amending the feature commit.
### Acceptance mapping
| Acceptance criterion | Evidence |
| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| register/unregister/list, idempotent | `skill.spec.ts` and built-CLI temp-home smoke |
| traversal/symlink-injection protection | invalid-name matrix, foreign file/dir/link tests, symlinked-root tests |
| list flags dangling and foreign entries | deterministic list status test |
| install and upgrade auto-sync every canonical directory | finalize + framework re-seed integration specs; two-skill built-module smoke |
| newly added skill becomes discoverable without manual link | `added-after-setup` auto-sync creates exact Claude link; Claude can rescan with `/reload-skills` or a new session |
| Pi/Codex parity captured as scope note | user guide, developer guide, installed framework README |
| documentation gate | root README, user guide, developer guide, framework README, sitemap |
## Risks
- Symlink replacement uses `lstat` semantics so dangling links are detectable without following them.
- Link ownership is determined lexically against the canonical skills root, and existing symlink ancestors in either managed root are rejected before mutation.
- Auto-sync continues across per-skill conflicts while never deleting real files/directories or foreign symlinks.
- Claude Code discovers filesystem skills at session launch/reload boundaries; bridge creation makes a later `/reload-skills` or new session able to discover the skill, but cannot mutate an already-cached in-process registry by itself.
- Pi does not need this Claude bridge because its Mosaic launcher can consume the canonical root. Codex lifecycle parity remains explicitly deferred.
- No deployment surface is affected.
## PR #826 review remediation
- 2026-07-17: Exact-head RoR requested changes for two ownership bugs: installer pruning deleted foreign-name links under `MOSAIC_HOME` outside canonical skills, and unregister deleted a same-root link targeting a different skill. It also requested trailing-dot rejection and executable coverage support.
- RED evidence: focused regression run failed 4 tests: register/unregister accepted `safe.`, misdirected unregister did not throw, and the install linker deleted the foreign-name link.
- GREEN evidence: `skill.spec.ts` passes 43/43, including live and dangling foreign-name links in a temp HOME/MOSAIC_HOME and the misdirected unregister invariant.
- Coverage: `vitest run src/commands/skill.spec.ts --coverage` passes configured 85% thresholds for `skill.ts`: 91.05% statements/lines, 86.27% branches, 95.23% functions.
- Full gates: package build passed; package tests passed 69 files / 1,332 tests plus framework shell suite; repository typecheck 42/42, lint 23/23, and format check passed.
docs/archive/work-records/828-lease-broker.md
+86
View File
@@ -0,0 +1,86 @@
# WI-1 Scratchpad — Authenticated external lease broker
- **Issue:** Gitea #828
- **Milestone:** 188 — Compaction-Refresh Mechanism (M1: Claude + Pi)
- **Branch:** `feat/828-lease-broker`
- **Starting HEAD:** `d801d6c4c8a984d6a95033c49714210018d3d9a8`
- **Session role:** Orchestrator coordinating implementation; Mos retains merge authority.
## Objective
Implement the ratified WI-1 product lease broker under `packages/mosaic/`: Linux `SO_PEERCRED` identity, broker-minted logical session IDs, `(pid,starttime)` launcher anchors with per-hop `/proc` starttime revalidation, sibling-substitution rejection, same-PID runtime-generation revocation, crypto-RNG single-use token persistence, and protected Unix-socket posture.
## Authority verification
Verified before code on session start; all exact SHA-256 values matched:
- BUILD-BRIEF: `89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b`
- SPEC-v5: `a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433`
- Ratification: `bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67`
- P6 planner ruling: `b7bbb6ea6e8d9a5c3366993642ab4e4f65b961af04936dcac20bfbcdcbaf1a09`
- WI-0 Gate0 evidence: `5d418306fcc597fd514e500bee40d1509f0bf467e46ee13fc5c280ed8274759d`
## Locked constraints
- Build against the ratified design; do not re-derive it.
- Product code only in `packages/mosaic`; Gate0 Python probes are reference prototypes and are not shipped.
- Caller-supplied/asserted `session_id` is refused.
- Tokens use the operating-system CSPRNG via Python `secrets`; never `Math.random` or model output.
- Socket parent directory mode `0700`, socket mode `0600` minimum; document distinct-principal deployment as the stronger T-C-closing posture.
- Red-first TDD for six named cases; new-code coverage >=85%.
- No merge. PR must say `closes #828`; exact 40-character head handed to Mos for Opus-SECREV and independent review.
## Plan
1. Load security/testing/docs guidance and inspect existing `packages/mosaic` architecture.
2. Write the six required tests first and capture RED evidence.
3. Implement minimal broker modules and CLI/runtime integration necessary for product use.
4. Run focused tests with coverage, package gates, then full repository gates/suite.
5. Run author-side review/remediation, commit `closes #828`, queue guard, push, and open PR through Mosaic wrappers.
6. Send PR number + exact head SHA to `web1:mosaic-100`; stop without merging.
## Risks / boundaries
- Same-UID counterfeit socket replacement remains the disclosed T-C residual unless broker runs under a distinct principal; filesystem modes alone are minimum hardening, not a complete authenticity proof.
- `.mosaic/orchestrator/mission.json` and `.mosaic/orchestrator/session.lock` were already modified at session start and must not be included in this PR.
- Repository Woodpecker pipelines exist; CI is the canonical build path. No manual image build/deploy is in scope.
## Progress / evidence
- 2026-07-18 session start: mandatory mission files and orchestration guides loaded.
- STEP 0: all four authority hashes matched; artifacts read in full.
- Branch/HEAD confirmed; issue #828 open; Gate0 evidence hash confirmed.
- Initial RED: focused Vitest acceptance suite failed 11/11 because the product daemon did not exist; the expected missing-product failure was observed before implementation.
- Review-remediation RED: partial/zero-progress state writes, nested corrupt state, symlink state, canonical starttime, and duplicate-anchor generation behavior failed before their fixes. Real socket RED/GREEN runs were executed by the unrestricted parent harness because the delegated worker sandbox denies `AF_UNIX.bind()`.
- Product implementation added at `packages/mosaic/framework/tools/lease-broker/daemon.py`; Gate0 probe scripts were read as references but not copied or shipped.
- Independent Codex code review round 1 found 2 blockers + 1 should-fix (connection stall/crash, partial writes, packet-dependent framing); all were remediated with tests.
- Independent Codex code review round 2 found 2 blockers + 1 relevant should-fix (half-close contract ambiguity, incomplete persisted-state validation, symlink/non-regular state); all were remediated with tests and documentation. Pre-existing `.mosaic/*` session dirt remains excluded from the PR.
- Unrestricted focused situational suite: `35/35` GREEN.
- New Python product module coverage: `90%` (`356` statements, `36` missed), above the user-required 85%.
- Root typecheck: `42/42` Turbo tasks GREEN.
- Root lint: `23/23` Turbo tasks GREEN.
- Root format check: GREEN.
- Package build + suite: `71/71` files and `1,369/1,369` tests GREEN, including framework shell tests.
- Full root suite: `43/43` Turbo tasks GREEN after the oversized-frame production race fix.
- Focused acceptance suite: `35/35` GREEN in three consecutive unrestricted runs; exact-head instrumented run also `35/35` GREEN.
- Exact-head Python product coverage: `90%` (`365` statements, `37` missed), above the required 85%.
- Review-triggered oversized-frame race was fixed in production by bounded drain-to-EOF; tests were not changed.
- Commits banked in red/green cadence: `d61c5441` (RED contract), `deb11df7` (GREEN implementation/docs), `57770e34` (oversized-frame production fix).
- Final-review blocker remediated: added a 256-token pending-state cap, deletion on consume/generation revocation, pre-open serialized-size enforcement, and request-wide in-memory rollback for every broker mutation/commit failure while retaining the v1 live-token schema.
- Distinct-principal docs now state built-in `0700`/`0600` is same-principal only; WI-1 does not provide the external identity-preserving proxy/ACL/service boundary needed for the stronger deployment.
- Exact Python unit suite: `8/8` GREEN. Unrestricted focused acceptance: `35/35` GREEN.
- Exact-head package build/suite: `71/71` files and `1,369/1,369` tests GREEN.
- Exact-head Python product coverage: `90%` (`376` statements, `36` missed), above required 85%.
- Root typecheck: `42/42` GREEN. Root lint: `23/23` GREEN. Root format check and `git diff --check`: GREEN.
- Final exact-head rereview found two persistence blockers: post-rename directory-fsync uncertainty and acceptance of impossible persisted token records. RED was captured as three invariant failures plus one missing fail-stop error; commits `a94b1220` (RED) and `d05465e5` (GREEN) remediate both without weakening tests.
- Post-remediation evidence: Python unit suite `10/10`, focused real-socket acceptance `35/35`, full root suite `43/43` Turbo tasks, broker coverage `90%` (`395` statements, `38` missed), lint `23/23`, typecheck `42/42`, format check and `git diff --check` GREEN.
- Independent Codex review of remediation commit `d05465e54736c4966294c4af8fbd6a4ad8fe81aa`: APPROVE, confidence `0.94`, zero findings. Reviewer sandbox could not allocate temp directories; unrestricted parent test evidence above is canonical.
- Remediation session: terra review comment `18072` reproduced a SERIAL-ACCEPT DoS; scope is RED regressions plus bounded concurrent connection handling on PR #836, preserving all existing broker security properties.
- RED evidence against reviewed daemon: four silent peers delayed registration `3920 ms` beyond the `1500 ms` bound; 16 silent peers were not reaped within `2500 ms`. The first bounded implementation then exposed slot exhaustion by rejecting the valid queued caller with `EPIPE`; admission was corrected to wait for a reclaimed bounded slot. GREEN evidence: queued-peer test `211 ms`; strengthened cap/reap/reclaim test `1118 ms`; complete real-socket acceptance `37/37` and Python persistence suite `10/10`.
## Coordinator handoff requirements
1. Mandatory Opus-SECREV on the exact PR head; no GPT/terra substitute.
2. Independent exact-head code review and exact-head RoR before Mos-authorized merge.
3. Mos retains merge authority; this WI author stops after PR + full 40-character head handoff.
docs/archive/work-records/829-mutator-gate.md
+130
View File
@@ -0,0 +1,130 @@
# WI-2 Scratchpad — Whole mutator-class gate
- **Issue:** Gitea #829
- **Branch:** `feat/829-mutator-gate`
- **Base HEAD:** `8ec67a1126adb0dcd4c3a2bf5525f3e239c0b201`
- **Role:** sol author/build lane only; terra code review and Opus security review are coordinator-owned.
## Mission prompt
Implement BUILD-BRIEF Deliverable 2 as a framework-native whole mutator-class gate under `packages/mosaic/`, building against the merged WI-1 lease broker. No consequential mutator may succeed while UNVERIFIED after a compaction observer fires or after TTL. Carry the T-B compromised-tool acceptance criteria. Enforce revoke-first and promote-last structurally. A receipt is only a promotion prerequisite; the mutator-class gate remains the safety mechanism. M1 is Claude + Pi only.
## Authority verification
Verified exact SHA-256 before design/code:
- BUILD-BRIEF: `89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b`
- SPEC-v5: `a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433`
- Ratification: `bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67`
- sol final red-team: `3da326a4ea91767b731e128a93b13194e8002358101e30de3fcb8ca2f8f54faa`
Carried authority chain also verified/read for the locked T-B gate contract: SPEC-v4 `a5e9c261…`, v4 sol `1e76ee59…`, SPEC-v3 `e0830ba0…`, v3 sol `9f321ade…`.
## Plan
1. RED real-socket acceptance tests for default-deny whole classes, T-B raw-tool bypass, observer/TTL revocation, and structural revoke-first/promote-last.
2. Extend the merged WI-1 broker as the sole lease authority; authenticate every transition through existing peercred/ancestry/session logic and consume WI-1 single-use cycle tokens atomically before promotion.
3. Add one broker-backed runtime gate executable and wire it across all Claude `PreToolUse` tools and Pi `tool_call`; unknown/custom tools deny by default.
4. Add proportional protocol/security/operations documentation and requirements-to-evidence mapping.
5. Run focused coverage, package/full suites, lint/typecheck/format, then queue-guard, push, open an unmerged PR with `closes #829`, and hand off the exact head.
## Risks and bounds
- Receipt parsing/builders and compaction observers are later WIs; WI-2 exposes the promotion prerequisite boundary but does not treat a receipt as safety authority.
- Broker restart intentionally loses volatile VERIFIED leases and therefore restarts UNVERIFIED; persistent WI-1 identity/token state remains unchanged.
- The gate is whole-class and does not parse shell command strings. T-C extension/hook absence and same-UID broker replacement remain outside the client guarantee and server branch protection remains the backstop.
- Initial lease TTL is capped at ratified 300 seconds; callers may only shorten it.
- Working budget assumption: 35K tokens; reduce documentation/refactor breadth before touching locked scope if pressure rises.
## Progress and verification
- RED #1: all 5 initial real-socket contract tests failed on WI-1 with `UNKNOWN_ACTION` or missing adapter behavior.
- RED #2: register-before-exec runtime test failed because `launch-runtime.py` did not exist.
- GREEN: broker-owned volatile lease state, 300-second maximum monotonic TTL, WI-1 token-backed promotion, all-tools runtime gate, Claude/Pi wiring, and register-before-exec launcher delivered without changing WI-1 peercred/ancestry authority.
- Focused broker + gate acceptance: `43/43` GREEN.
- Instrumented Python coverage: `88%` total — daemon `89%`, register/exec launcher `86%`, runtime gate `86%`.
- Full repository suite: `43/43` Turbo tasks GREEN; `@mosaicstack/mosaic` `72/72` files and `1,377/1,377` tests GREEN.
- Root typecheck: `42/42`; lint: `23/23`; format check and `git diff --check`: GREEN.
- No author self-review was run. Exact-head terra CODE and Opus SECREV remain coordinator-owned gates.
## Acceptance mapping
| Acceptance criterion | Evidence |
| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No consequential mutator succeeds while UNVERIFIED after observer revoke or TTL | `observer revocation and monotonic TTL expiry deny the next mutator` real-socket acceptance test |
| T-B compromised-tool bypass is covered by the whole gate | `T-B raw and custom mutator tools are default-denied without shell parsing` across Claude/Pi built-ins, raw Bash class, MCP/custom/unknown tools |
| Revoke-first / promote-last is structural | `revoke-first and promote-last structurally bracket mutator authority`; direct promotion rejected, pending remains denied, token consumption commits before VERIFIED |
| Consume WI-1 auth/lease substrate | All transitions and decisions traverse merged peercred/ancestry authentication; promotion consumes the exact WI-1 CSPRNG cycle token |
| M1 Claude + Pi | Claude `.*` PreToolUse and Pi `tool_call` invoke the same broker gate; register-before-exec test proves broker-minted parent identity reaches runtime descendants |
## Locked discipline
- Re-verify and read the four authority artifacts before design or code.
- RED-first tests must cover unverified mutation refusal, T-B compromised-tool refusal, and revoke-first/promote-last ordering.
- Consume WI-1 VERIFIED-lease state; do not re-derive kernel identity, ancestry, sessions, or token authentication.
- Minimum 85% new-code coverage; full suite, lint, typecheck, and format checks green.
- Build only: no self-review and no merge. Open a PR containing `closes #829`, report its exact 40-character head, then exit.
## Remediation — terra CODE comment 18091
- Coordinator correction: terra returned REQUEST CHANGES at head `77b137ccc04b5be035cac5ca21bbbf3df8b94f97`; Opus SECREV was GO and CI green, but no evidence transfers to the remediated head.
- BLOCKER 1 verified: first-class Claude/Pi route through `execLeaseGatedRuntime`, while the supported Claudex path preserves isolation but directly invokes `claude`; it therefore registers no anchor, injects no lease session, and the isolated config has no guaranteed all-tools gate hook.
- BLOCKER 2 accepted: prior 88% was aggregate evidence. Remediation must produce independently measured branch coverage of at least 85% for each new executable (`launch-runtime.py`, `mutator-gate.py`, and daemon delta evidence), including successful exec-boundary collection and validation/error branches.
- Remediation discipline: RED tests first; preserve the reviewed-good broker lock/state-transition ordering; update PR #837 on the same branch; no self-review or merge.
### Remediation evidence
- RED commit `046896c6`: both `mosaic claudex` and `mosaic yolo claudex` behavioral probes exited 1 because the direct path supplied neither a broker session nor the isolated all-tools hook; branch-focused Python tests failed on the absent injectable boundaries. Fresh WI-2 daemon-delta instrumentation also failed the ≥85% branch gate at 75%.
- GREEN: Claudex now exposes only `execLeaseGated`, passes the preserved isolated proxy environment through the shared register-before-exec wrapper, and merges the exact `.*` mutator hook into isolated `settings.json` with mode `0600`. Missing broker/identity, malformed or symlinked settings, and an unverified consequential tool all fail closed. The broker transition/lock implementation was not changed.
- Behavioral regression: normal and YOLO Claudex both receive a 64-hex broker session, retain their mode-specific arguments, observe the exact all-tools hook, and receive status 2 for unverified `Bash`.
- Independent branch coverage: `launch-runtime.py` 16/18 = **89%** (statements 98%); `mutator-gate.py` 21/22 = **95%** (statements 99%); `daemon.py` WI-2 delta 35/40 = **88%** (whole-file branch 80%, statements 90%).
- Fresh focused real-socket coverage run: WI-1 + WI-2 acceptance `46/46`; persistence `10/10`; branch unit suite `10/10`.
- Fresh full repository suite: `43/43` Turbo tasks; `@mosaicstack/mosaic` `72/72` files and `1,381/1,381` tests.
- Fresh root gates: typecheck `42/42`; lint `23/23`; format and `git diff --check` GREEN.
- PR #837 remains open and unmerged. Terra CODE and Opus SECREV must both rerun from zero on the exact remediated head before coordinator-owned merge authorization.
## Remediation round 3 — terra CODE comment 18099 + binding upgrade
- Locked-good surfaces: Claudex gating and B2 per-executable coverage are verified; do not regress them. Broker state-transition/lock ordering remains untouched.
- Mechanical repository sweep found direct executing Claude entries in PRDY init, PRDY update, QA remediation, and `@mosaicstack/coord` task launch. It also found a direct Claude command rendered into the QA report template and documentation examples. Existing Mosaic CLI Claude/Pi/Claudex, orchestrator session-run, and fleet starts already reach the gated boundary.
- Elevated hard requirements: ship a permanent suite/CI guard that scans production source and fails on any direct Claude/Pi launch; route every executing entry through one common gated wrapper; add real-broker RED/GREEN tests for PRDY init/update and QA; preserve each environment and denial behavior; independently measure all new executable coverage at ≥85%.
- Round-3 plan: first commit RED behavioral and scanner-contract tests; then add one framework `launch-runtime.sh` choke-point over `launch-runtime.py`, make Mosaic CLI and shell launchers use it, make coord route through `mosaic`, and wire the permanent guard into package tests. Update all discovered operator-facing direct-launch examples so the scanner inventory remains complete.
### Round-3 outcome
- RED commit `7f3418fa`: PRDY init, PRDY update, and QA remediation all reached the fake Claude binary without a broker session even when the configured socket did not exist; the permanent-guard contract initially failed because its executable was absent, then failed against the five discovered direct entries (four executing plus the QA command template).
- Choke-point decision: a new shell layer was unnecessary. Every executing repository entry now converges directly or through `mosaic`/`execLeaseGatedRuntime` on the existing single `launch-runtime.py` register-then-exec wrapper. PRDY and QA preserve their working directories, prompts, flags, logging pipe, and environment. Coord rewrites direct Claude commands to `mosaic claude` and rejects unknown custom Claude launchers fail-closed.
- Permanent guard: `packages/mosaic/framework/tools/lease-broker/check-runtime-launches.py`, invoked by `packages/mosaic/package.json` `test:framework-shell` and therefore root `pnpm test`/CI. It scans production code under `packages/`, `apps/`, `plugins/`, and `tools/` and rejects literal, absolute-path, process-API, command-array, and dynamic Claude/Pi launch forms. Synthetic bypass tests are permanent at `runtime_launch_guard_unittest.py`.
- Mechanical inventory: **14 gated / 14 total** — coord 2, fleet 1, QA 2, orchestrator 3, PRDY 2, Mosaic Claude/Pi/Claudex adapter/boundary 4. No verification-layer fallback or follow-up issue is needed because the single code-level wrapper was achieved.
- Real-socket behavioral evidence: PRDY init, PRDY update, and QA remediation each fail before runtime execution when the broker is absent; with the broker present they receive a broker-minted 64-hex session and the unverified `Bash` authorization exits 2. Claudex normal/YOLO and the broker state machine remain GREEN.
- Fresh branch coverage: `launch-runtime.py` **18/18 = 100%**; `mutator-gate.py` **22/22 = 100%**; permanent guard **36/38 = 95%**; `daemon.py` WI-2 delta **35/40 = 87.5%**. All attributable executable statement coverage is at least 98%.
- Fresh focused suites: broker + mutator real-socket acceptance `49/49`; persistence `10/10`; launcher/gate branch suite `13/13`; permanent guard suite `7/7`; coord `19/19`.
- Fresh full repository suite: `43/43` Turbo tasks; `@mosaicstack/mosaic` `72/72` files and `1,384/1,384` tests. Root typecheck `42/42`, lint `23/23`, format, and diff checks GREEN.
- PR #837 remains open and unmerged. Terra CODE and Opus SECREV must both rerun from zero on the exact round-3 head before coordinator-owned merge authorization.
## Remediation round 4 — terra CODE comment 18104
- Locked-good surfaces: the 14/14 launch inventory, single `launch-runtime.py` choke-point, real-socket launcher behavior, coverage, Claudex gating, and broker state machine must not change.
- Reproduced RIDER E exactly at head `1792b7934dda7eff64a207b8b0edb9c460d4164b`: a temporary production file containing `exec claude --dangerously-skip-permissions "terra-r3" # launch-runtime.py` made the guard exit 0 and report `1 gated/1 total`.
- Root cause: classification searched the unparsed physical line, and the broad gated regex treated any `launch-runtime.py` substring—including comments and inert arguments—as an invocation before the direct-launch finding was evaluated.
- Round-4 plan: add permanent RED cases for the exact comment evasion plus string-argument, echo, and unrelated-variable marker evasions; tokenize/strip comments by launcher syntax; recognize only command-position wrapper invocations with `--runtime` and the gated command separator; retain 14/14 real inventory; rerun guard coverage and all gates fresh.
- Mos Rider A/B decision: adopt **both** defenses. Command-position parsing remains necessary because a normal `claude -p` launch is consequential even without the dangerous flag. The primitive-location invariant is more mechanically robust for dangerous mode because it does not need to recognize a wrapper marker at all. Move the sole raw `--dangerously-skip-permissions` literal into `launch-runtime.py`; any occurrence in another production file is independently RED.
- Rider-A RED matrix adds heredoc body, backslash continuation, non-first `;`/`&&`/pipe commands, command substitution, `eval`, and variable-execution indirection in addition to the six marker/comment evasions. Before the augmented implementation, primitive ownership, command substitution, `eval`, variable execution, and the preserved 14-site inventory all fail.
- Round-4 GREEN uses both defenses. Quote-aware comment stripping removes shell/Python `#` and JS/TS line/block comments; shell command prefixes are segmented with `shlex`; validated wrappers require `launch-runtime.py` in command position, `--runtime`, and the `--` command separator; multiline TypeScript wrapper calls are validated as complete invocations. Direct command syntax wins over markers, while tracked runtime assignments plus `eval`/variable execution, command substitution, chained commands, heredocs, continuations, and `env`/`command`/`nohup` prefixes are rejected.
- Primitive ownership is independently load-bearing: `launch-runtime.py` is the sole production owner of the raw Claude dangerous flag. Mosaic, Claudex, and PRDY request semantic `--dangerous`; the wrapper validates Claude and injects the primitive immediately before register/exec. This preserves actual YOLO argv behavior while making any raw primitive elsewhere fail without relying on wrapper-name recognition.
- Permanent guard suite now has 10 tests and 31 direct-launch forms, including 18 new round-4 marker/comment/indirection/prefix evasions plus harmless-marker and multiline-wrapper controls. Terra's exact add-ungated source is exercised through the CLI effectiveness test. Repository inventory remains exactly **14 gated / 14 total**.
- Fresh round-4 coverage: guard **97%** branch-aware aggregate (241 statements, 110 branches); `launch-runtime.py` **100%**; `mutator-gate.py` **100%**. The daemon is byte-unchanged from the round-3 head whose WI-2 delta is **87.5%**.
- Fresh round-4 gates: real-socket acceptance **49/49**; persistence **10/10**; launcher/gate **14/14**; guard **10/10**; coord **19/19**; Mosaic **1384/1384**; root **43/43**; typecheck **42/42**; lint **23/23**; format and diff checks green.
## Remediation round 5 — terra 18116 + Opus 18114
- Both independent gates converged on one guard-only completeness gap at round-4 head `1eb77c17f3147d4fa9944f77f1826243135b9cc0`; all round-4 primitive anchoring, command-position parsing, 14/14 inventory, broker ordering, and coverage remain locked-good.
- Reproduced exactly: a temporary production source containing `launcher=claude` followed by `exec "$launcher" -p x` exits 0 with `0 gated / 0 total`. The literal command resolver skips prefixes but cannot resolve a tracked variable; the variable resolver handles only bare/eval references and cannot skip prefixes.
- Round-5 plan: add 10 permanent RED forms (quoted/unquoted `exec`, `command`, `nohup`, and `env` with assignment, each multiline and same-line), then unify shell command-position resolution so literal and tracked-variable terminal tokens traverse the same prefix parser. Retain an independent variable-reference backstop, the 14/14 inventory, and every round-4 regression.
- Mos stopping-criterion augment: command parsing is explicitly best-effort rather than a complete shell interpreter. Add B1 proving a parser-exotic alias launch with the raw dangerous flag is still RED by primitive anchoring, and B2 proving a parser-missed non-dangerous alias launch reaches the global `.*` hook and fails closed with `GATE_UNAVAILABLE` when no lease session exists. Document A (realistic parser matrix) + B (robust residual backstops); fresh reviewers supply criterion C (no new non-overlapping finding).
- RED commit `91a4a983`: all 10 prefix×variable cases failed as expected before the fix—quoted/unquoted `exec`, `command`, `nohup`, and `env A=1`, each in multiline and same-line assignment shapes.
- GREEN structural resolution: `shell_command_tokens()` now owns command-position prefix skipping for both literal and variable callers, including nested `exec`/`command`/`nohup`/`env` ordering. `runtime_variables` is threaded into `is_shell_direct_invocation()` and the same terminal-token resolver backs `executes_runtime_variable()`; exact `$v` and `${v}` references are resolved after `shlex` removes quoting. Same-line runtime assignment delimiters include shell operators.
- Residual backstops: B1 proves alias-indirected dangerous mode is classified `dangerous-primitive` even though the parser does not resolve the alias. B2 proves a non-dangerous alias residual remains parser-missed, then verifies the shipped global `.*` Claude hook and status-2 `GATE_UNAVAILABLE` denial for representative read, mutator, and custom/MCP tools without a lease session.
- Stopping-criterion evidence A+B is committed in tests and architecture docs; C remains the fresh exact-head terra/Opus determination. Repository inventory remains exactly **14 gated / 14 total**.
- Fresh round-5 coverage: permanent guard remains **97%** branch-aware aggregate (251 statements, 112 branches). Locked-good launcher and mutator-gate executables remain unchanged at their round-4 **100% / 100%** evidence.
- Fresh round-5 gates: real-socket acceptance **50/50**; persistence **10/10**; launcher/gate **14/14**; guard **12/12**; coord **19/19**; Mosaic **1385/1385**; root **43/43**; typecheck **42/42**; lint **23/23**; format and diff checks green.
docs/archive/work-records/830-compaction-revoke.md
+95
View File
@@ -0,0 +1,95 @@
# WI-3 Scratchpad — Compaction revocation and runtime-generation rollover
- **Issue:** Gitea #830
- **Branch:** `feat/830-compaction-revoke`
- **Base HEAD:** `abd2791f59b3f06f46dd08e55298ced72f6aa7c2`
- **Role:** sol author/build lane only; terra CODE and Opus SECREV are coordinator-owned.
## Mission prompt
Implement BUILD-BRIEF Deliverable 3.3 and D4 on merged WI-1/WI-2 under `packages/mosaic/`. Claude `PreCompact` and `SessionStart(matcher=compact)` plus Pi `session_before_compact`/`context` equivalents must revoke the active lease through the existing broker state machine. Any `runtime_generation` bump—including same-PID reload/resume/fork—must auto-revoke the prior incarnation so the new generation inherits no prior lease. M1 is Claude + Pi only.
Honor amended D2-v5 exactly: hard fail-closure when at least one observer fires or after lease expiry; both observers missing within TTL is an explicitly named bounded residual stale window (maximum 300 seconds, soak-tighten only), with no claim that the mutator gate bounds actions inside that window; total gate-hook miss is T-C. T12b/T30 must report both the within-TTL ALLOWED outcome and after-TTL DENIED outcome.
## Session start verification
- Worktree is clean on `feat/830-compaction-revoke` at exact required base `abd2791f59b3f06f46dd08e55298ced72f6aa7c2`; `origin/main` is the same SHA and includes merged WI-2 atop WI-1.
- Authority SHA-256 verified:
- BUILD-BRIEF: `89fdbc27ed0e5050dc7b52f3ef2ddaea691edf17fd89d51b15e26fb5ed47171b`
- SPEC-v5: `a6d07ade835758e8488ca10d3b0631caf0beb93ea3a6733631f151b0c2f01433`
- Ratification: `bac58319c9c4028b5b40e1129e0033cdb5a6b7b02033c25f06f4cb77d7779c67`
- sol red-team: `3da326a4ea91767b731e128a93b13194e8002358101e30de3fcb8ca2f8f54faa`
- WI-0 evidence pack SHA-256 `5d418306fcc597fd514e500bee40d1509f0bf467e46ee13fc5c280ed8274759d` read directly. Probe P3 is **PASS**: real Pi retained the same PID/starttime through reload/fork/new/resume while generations advanced and a prior VERIFIED generation was revoked.
- P6 planner-return ruling SHA-256 `b7bbb6ea6e8d9a5c3366993642ab4e4f65b961af04936dcac20bfbcdcbaf1a09` read directly: feature WI admission is GO with the exact-delivery empirical compatibility fact and disclosed T-C middle-drop residual; no receipt redesign.
## Plan and budget
1. RED real-socket acceptance for T12b/T30, each Claude observer, same-PID generation rollover, Claude/Claudex hook wiring, and Pi lifecycle wiring.
2. Add one broker client executable for observer revocation plus a private monotonic generation-file helper shared by launcher, gate, and revoker.
3. Wire Claude `PreCompact`, `SessionStart(compact)`, and resume/clear generation rollover; merge equivalent mandatory hooks into isolated Claudex settings.
4. Wire Pi pre/post compaction observers and reload/new/resume/fork generation rollover with local fail-closed tool blocking if lifecycle revocation fails.
5. Document the D2-v5 bounded stale window without claiming the mutator gate bounds within-TTL actions; update protocol/security/operations/sitemap/checklist.
6. Run focused real tests, independently measured executable coverage ≥85%, full repository gates, commit/push, open an unmerged `closes #830` PR, and hand off for terra CODE + mandatory Opus SECREV.
Working estimate: **35K tokens**. No explicit hard cap was supplied; reduce refactor breadth before touching locked broker authority/state-machine semantics.
## RED evidence
- New T12b/T30 test already reports the inherited primitive honestly: within-TTL **ALLOWED**, after-TTL **DENIED**. The complete AC remains RED because the mandatory threat-contract document is absent.
- Focused real-socket suite is RED with 7 expected failures: missing revoker executable (both Claude observers + generation bump), missing Claude/Pi wiring, missing isolated Claudex observers, and missing D2-v5 disclosure.
- Branch-focused Python suite is RED on the wished generation initializer/resolver interfaces and missing `lease_generation.py` / `revoke-lease.py`.
- Pi lifecycle suite is RED because the wished standalone `lease-lifecycle.ts` observer/generation module does not exist.
## Locked discipline
- RED-first T12b/T30 and observer/generation tests; test commit precedes implementation.
- Reuse broker `revoke_lease`; do not fork identity, lease, or transition authority.
- Preserve revoke-first/promote-last and WI-1/WI-2 reviewed state machine.
- ≥85% attributable executable coverage with real tests.
- No author self-review, no merge, no `--no-verify`.
## Local implementation complete (push held)
Implemented on the WI-3 base `abd2791f59b3f06f46dd08e55298ced72f6aa7c2` without changing the reviewed broker state machine:
- Added `revoke-lease.py`, which authenticates through the existing broker session/generation and invokes `revoke_lease`. A fired observer that cannot confirm broker revocation advances the private generation as a local fence before returning non-zero.
- Added `lease_generation.py`: owner/type/mode/size validation, no-follow opens, exclusive bump lock, monotonic `int64` generation, write-all + `fsync`, and fail-closed exhaustion/corruption handling.
- `launch-runtime.py` creates `generation-<broker-session>.state` mode `0600` beside the socket before `exec`; `mutator-gate.py` resolves that current file value on every tool check.
- Claude settings and isolated Claudex settings now preserve/install `PreCompact`, `SessionStart(compact)`, and resume/clear rollover hooks in addition to the global all-tools gate.
- Pi now registers tested `session_before_compact`, `session_compact`→first `context`, and `session_start(reload|new|resume|fork)` handlers. Failed pre-compact revocation cancels compaction; failed post-compact/rollover revocation latches local all-tool denial.
- Added PRD requirements, architecture/security/protocol/operations updates, sitemap entry, and the ignored-by-default documentation checklist (force-add required at commit).
### Acceptance and coverage evidence
- Focused acceptance: `19/19`; T12b/T30 prints within-TTL **ALLOWED** and after-TTL **DENIED**.
- Pi lifecycle: `8/8`, with **100% statements/branches/functions/lines** attributable coverage.
- New Python generation/revoker: `24/24`, **99% branch-aware aggregate coverage** (`lease_generation.py` 98%, `revoke-lease.py` 100%).
- Mosaic package: `1399/1399`; framework shell Python `24/24`, launch guard `12/12`, permanent launch inventory `14 gated/14 total`.
- Existing lease-broker real-socket acceptance: `37/37` within the package run.
- Full repository: `43/43` Turbo tasks green; gateway `628 passed / 12 skipped`; Mosaic `1399/1399`.
- Root typecheck: `42/42`; lint: `23/23`; format and `git diff --check` green.
- Initial direct package test without first building the package reproduced the known missing-`dist/cli.js` harness condition; the canonical root Turbo test (which schedules `@mosaicstack/mosaic#build`) and explicit package build+test are green. No test was weakened.
### Review evidence
- Codex uncommitted code review: **APPROVE**, confidence `0.88`, zero findings. Its read-only sandbox could not rerun Vitest, but the author-side focused and full suites above were green.
- Codex uncommitted security review: risk **none**, confidence `0.91`, zero findings.
- Coordinator-mandated fresh exact-head terra CODE and Opus SECREV remain pending after rebase/push clearance; these local reviews do not replace that final gate.
### Hold and residuals
- **DO NOT PUSH OR OPEN A PR YET.** Coordinator requires flake-fix #838 to land, then WI-3 must rebase onto deterministic-green `main` before push.
- Merge remains gated on #838, #827 Probe 3, and combined GO.
- Named residual retained verbatim: when both observers are entirely missed, within-TTL consequential actions remain allowed; only lease expiry denies after the bounded stale window. No within-window mutator-action bound is claimed.
## Deterministic-main rebase evidence
- Fetched and confirmed `origin/main` at `8dfcf1903e385f977121069f798f476eb671fffc` (`#838` bounded broker deadlines, empty-read fail-closure, and de-flaked acceptance client).
- Linear rebase completed. The only content conflict was `packages/mosaic/src/mutator-gate/runtime_tools_unittest.py`; resolution retained #838's `subprocess`/`threading` deadline regressions and WI-3's `stat` generation-state coverage. No authority/state-machine choice was ambiguous.
- `packages/mosaic/src/mutator-gate/mutator-gate.acceptance.spec.ts` auto-merged on top of #838's shared `requestBrokerReply` helper. No inline socket/`JSON.parse` client was resurrected.
- Verified WI-3 has zero diff from `origin/main` for #838-owned `daemon.py`, `broker-test-client.ts`, `lease-broker.acceptance.spec.ts`, `vitest.config.ts`, and `packages/mosaic/package.json`; bounded deadlines and the de-flaked harness are preserved byte-for-byte.
- Required verbose acceptance command: **2 files / 56 tests green**. T12b/T30 still prints within-TTL **ALLOWED** and after-TTL **DENIED**.
- Full Mosaic package after explicit build: **74 files / 1408 tests green**; deadline unit `2/2`, runtime tools `25/25`, launch guard `12/12`, inventory `14/14`.
- Full repository: **43/43 Turbo tasks green**. Root typecheck `42/42`, lint `23/23`, format and diff checks green.
- Attributable coverage remains Python **99%** branch-aware and Pi lifecycle **100%** statements/branches/functions/lines.
- Push and PR remain held pending combined GO and all WI-3 merge gates. The coordinator-owned promote-lease-lost-ACK SPEC amendment/backstop is acknowledged as a future merge prerequisite and was not retro-expanded into this core rebase/build.
docs/archive/work-records/832-receipt-challenge-protocol.md
+13
View File
@@ -0,0 +1,13 @@
# #832 Receipt-challenge protocol — build scratchpad
- **Objective:** Deliver WI-5 receipt-challenge protocol ACs T25, T26, T28, and T29 only.
- **Authority:** BUILD-BRIEF, SPEC-v5, ratification, and red-team hashes verified in STEP-0.
- **Base:** `e522b22fa4492861b0fcd4a956a8795c54eb9bfe` (`origin/main`).
- **Constraints:** Byte-build only: no live broker/socket/systemd/tmux mutation. No PR, self-review, or probe fire. T27/T30 are out of scope.
- **Plan:**
1. Add red-first deterministic T26/T29 in-build tests that call shipped normative construction and broker path.
2. Add an unexecuted, isolated P5 out-of-process replay harness that drives the shipped daemon and asserts consume-before-promote for T25/T28.
3. Implement the broker-minted receipt challenge and exact receipt observation/consume/promote path.
4. Run unit, framework-shell, compile, lint, and type checks; push after the required queue guard; report to `mosaic-100`.
- **Risks:** The standalone harness must drive the real daemon without a divergent fixture. If that is impossible, stop and flag Mos.
- **Evidence:** Initial RED recorded in `/home/hermes/agent-work/reviews/832-wi5-red-receipt-challenge.log`; initial green checks passed. Remediation RED recorded in `/home/hermes/agent-work/reviews/832-wi5-remediation-red.log` before observer/payload implementation; remediation green passed. Remediation-2 RED recorded in `/home/hermes/agent-work/reviews/832-wi5-remediation2-red.log`: each rejected begin restored prior VERIFIED authority. Remediation-2 GREEN: receipt unittest (5: all `INVALID_CONSTRUCTION`, `PAYLOAD_CONSTRUCTION_REFUSED`, and `PAYLOAD_BINDING_MISMATCH` cases preserve UNVERIFIED and deny the next mutator), normative-fragments unittest (5), state-store regression (10), full mutator-gate acceptance (20, including the real begin → observer → consume → promote path), `py_compile`, Mosaic package lint/typecheck, and targeted Prettier check. The P5 harness remains unfired. Coverage tooling remains unavailable (`python3 -m coverage`: module not installed). Push pending.
docs/archive/work-records/833-constrained-recovery-command.md
+15
View File
@@ -0,0 +1,15 @@
# #833 constrained recovery command — build scratchpad
- **Objective:** Deliver WI-6 plus Mos-ruled B1/B2 and R2 Claude-only literal-argv repair: no shell-active recovery mapping bypass, unchanged Pi gate/B2 observer, AC-1/C4 preservation, and an unfired P6 probe.
- **Authority:** STEP-0 SHA-256 verified 4/4 against the supplied BUILD-BRIEF, SPEC-v5, ratification, and red-team records.
- **Base:** exact `07553ead337a70a9241f826d27571650262b289c`; new branch `feat/833-constrained-recovery-command`; merge-base assertion passed before any commit.
- **Constraints:** No rebase/pull during build; no live install/symlink or live broker/socket/tmux/systemd/model-stream activation; no self-review, PR, merge, push, or P6 fire. `docs/TASKS.md` is orchestrator-owned and will not be modified.
- **Plan:**
1. Add red-first unit tests against the recovery broker entrypoint for fresh recovery challenge, normal-receipt replay refusal, observable partial-delivery refusal, and the explicit middle-drop negative capability; commit the RED test and preserve its command output.
2. Implement the recovery command as a thin driver over shared WI-5 broker transitions and the trusted observer seam; it never accepts caller receipt text.
3. Add the source-resident skill under `packages/mosaic/framework/skills/`, plus a tmp-only #824 bridge projection test.
4. Build an unfired, default-3-run P6 standalone real-socket driver; it is not added to package scripts and will not be run.
5. Run targeted broker/mutator/receipt suites, lint, and format check; report head to `mosaic-100` and stop.
- **Risks:** The observer can only represent an exact latest message. Tail-preserving middle-drop is intentionally not claimed receipt-detectable (T-C residual deferred to WI-7 server evidence).
- **Evidence:** Original RED test committed at `3b5513bd6efad0c06b599fc759a66cff4286db04`; expected `UNKNOWN_ACTION` is logged in `/home/hermes/agent-work/reviews/833-wi6-red.log`. B1/B2 repair RED is `f4beedc3e7ac2e142dcbdeefb0e5ee40c20d9b86` in `/home/hermes/agent-work/reviews/833-wi6-repair-red.log`. R2 adversarial RED is committed at `65e2bd71cf360b6f45c86eec0b06ae24494832d8` in `/home/hermes/agent-work/reviews/833-wi6-repair-R2-red.log` before the literal-only gate source: the private real-gate/real-daemon battery covers every argv position (executable, path, phase, each flag, each value) for command substitution, backticks, parameter/arithmetic expansion, brace/tilde, process substitution, glob, redirects, control operations, embedded newline, and quotes. P6 remains rebuilt and unfired. The ordinary package Vitest/lint/typecheck/root-format commands cannot resolve their executables in this intentionally dependency-free fresh worktree (`node_modules` absent); no install/symlink workaround was used.
- **Budget:** No explicit task token cap was supplied; scope is fixed to WI-6 and no unrelated behavior will be added.
docs/archive/work-records/838-broker-acceptance-flake.md
+44
View File
@@ -0,0 +1,44 @@
# Issue #838 — Broker acceptance socket flake
## Objective
Eliminate high-contention uncaught JSON parse failures in lease-broker and mutator-gate acceptance helpers without laundering malformed/empty replies into passing assertions.
## Constraints
- Branch: `fix/838-broker-acceptance-flake` in `/home/hermes/agent-work/stack-838-flakefix`.
- Red-first TDD with a forced empty/truncated reply path and deterministic rejection or documented retry.
- Read newline-framed replies completely; reject malformed broker replies with byte length/content context.
- Determine RIDER2 branch before finalizing: test-harness-only `(a)` or daemon write truncation `(b)`.
- If product-side, prove the real Claude/Pi adapter read path fails closed; fix any allow-risk.
- Coverage >=85% per changed executable through real tests.
- Full suite and repository gates green; independent exact-head review required.
- Push and open a PR containing `closes #838`; do not merge.
## Progress
- 2026-07-17: Reclaimed from #824. Confirmed clean worktree on `fix/838-broker-acceptance-flake` at main base `abd2791f59b3f06f46dd08e55298ced72f6aa7c2`.
- RED evidence: after dependency setup, `broker-test-client.spec.ts` failed to load the intentionally absent shared client module. Its contract forces empty-close retry, repeated truncated-close rejection with byte context, and newline-terminated malformed-reply rejection.
- RIDER2 verdict: branch **(b)**. `daemon.py` starts one connection deadline before request reading, then may spend that budget waiting for `broker_lock`; after handling, `remaining <= 0` returns without writing. A timed `sendall` failure can likewise close after a partial write. A deterministic socketpair probe held the lock past `CONNECTION_DEADLINE_SECONDS` and observed `deadline_probe_reply_length=0`.
- Adapter tripwire: real subprocess executions of `mutator-gate.py` for both `--runtime claude` and `--runtime pi` against actual Unix servers returning empty and truncated replies all exited 2 with `GATE_UNAVAILABLE`. Adapter fail-close is proven; there is no ALLOW risk.
- Residual: production broker reply loss remains an availability-denial path under extreme contention, but cannot grant mutator authority. The acceptance-only client retries early closes and otherwise rejects with response byte length, escaped bytes, and hex; malformed newline-framed replies are never retried or converted to reply objects.
- Shared client now owns newline framing and parsing for both acceptance suites. Focused suites pass 60 tests, including the real adapter tripwire.
- Coverage gate: changed helper is 100% statements/lines/functions and 90% branches; existing `skill.ts` remains above 85% per-file thresholds.
## Scope corrections and bounded product repair
- Coordinator correction superseded the initial push/PR and retry language: this lane is BUILD-ONLY, and early-close retries are forbidden because they can mask a committed broker transaction. Nothing may be pushed or opened until explicitly cleared.
- Revised RED evidence: the no-retry empty/truncated tests failed because the first failed exchange was retried into `{ ok: true }`; the daemon regression failed because a slow completed `broker.handle()` produced `b''` instead of a newline-framed reply.
- Client repair: both former duplicated helpers use one shared reader. Empty, truncated, malformed, oversized, timed-out, and socket-error replies reject `BrokerTransportError` with a typed `kind`, attempt count fixed at one, response length, escaped bytes, and hex. No retry and no catch-to-reply conversion exists.
- Product repair: `daemon.py` now has independent bounded read, broker-lock queue, and send budgets. Lock queue exhaustion returns explicit `BROKER_BUSY` before `broker.handle()` can mutate state. Once handling starts it finishes atomically, and its reply always receives a fresh send timeout instead of being skipped because read/lock/fsync consumed a shared deadline.
- Product GREEN evidence: deterministic socketpair tests prove lock saturation returns framed `BROKER_BUSY` without invoking `handle()`, and a handle that completes after the former one-second shared deadline still returns its complete framed reply.
- RIDER2b remains **fail-closed**: real Claude and Pi `mutator-gate.py` subprocesses against empty and truncated Unix-socket replies exit 2 with `GATE_UNAVAILABLE`; no malformed/default ALLOW was observed.
- Residual/tripwire: an unavoidable peer disconnect or send failure can still lose acknowledgement after a valid transaction commits. The affected adapter call fails closed. A valid `promote_lease` may nevertheless remain VERIFIED after its acknowledgement is lost; that is authority-observability divergence requiring WI-3/Opus security review rather than expansion of #838. #838 does not add retries or attempt a protocol redesign.
- Final package evidence: recursive Mosaic dependency build passed; 73 Vitest files / 1,392 tests passed; deadline unit tests 2/2, real runtime tool tests 15/15, launch guard tests 12/12, inventory 14/14, and shell regressions passed.
- Final coverage: `broker-test-client.ts` 99.24% statements/lines, 86.11% branches, 100% functions under per-file >=85% thresholds. Repository typecheck 42/42, lint 23/23, and format check passed.
- Independent Codex exact-head review requested one framing fix: a valid frame followed by trailing bytes in a later socket chunk could resolve before the garbage arrived. Security review also flagged complete token-bearing reply bodies in diagnostic properties/logs.
- Review RED evidence: delayed cross-chunk garbage resolved `{ ok: true }` instead of rejecting, and a truncated `promotion_token` remained in the typed error. The tests use separate timed writes to prevent kernel/event-loop coalescing.
- Review remediation: the shared client now accumulates through EOF, requires exactly one terminal newline, then parses inside a rejecting error boundary. Diagnostic bodies are capped at 256 bytes, sensitive broker fields are fully redacted, and a SHA-256 digest preserves correlation without credential disclosure.
- Post-review coverage: `broker-test-client.ts` 99.35% statements/lines, 86.66% branches, 100% functions. Final full suite is 73 files / 1,394 tests plus all Python/shell gates; recursive build, typecheck, lint, and format are green.
- Fresh exact-head Codex security review: risk `none`, no findings. Fresh code review found only that the real-adapter subprocess proof lacked a timeout; it now has a five-second bound and fails with runtime/wire context while closing the fake server. The focused Python suite remains 15/15 green.
- Mandatory Opus SECREV remains coordinator-owned and pending before any push/PR decision; this build is intentionally local-only.
docs/archive/work-records/B1-next-durable-publish-design.md
+82
View File
@@ -0,0 +1,82 @@
# B1 / @next Durable Publish Pipeline — Design
## Objective
Make `next` a durable integration line that publishes the artifacts required by downstream federation boot tests without manual builds.
Every merge to `next` publishes:
1. **npm prerelease packages** to the Gitea npm registry with dist-tag `next`.
2. **Gateway container image** tagged only as `gateway:sha-<short>`.
The existing stable release behavior remains isolated to `main` / tags.
## Registry verification
Target registry: `https://git.mosaicstack.dev/api/packages/mosaicstack/npm/`.
Pre-implementation checks:
- `npm view @mosaicstack/mosaic dist-tags --registry https://git.mosaicstack.dev/api/packages/mosaicstack/npm/ --json` returned a dist-tags object (`latest: 0.0.48`).
- `npm view @mosaicstack/mosaic@latest version --registry https://git.mosaicstack.dev/api/packages/mosaicstack/npm/` resolved `0.0.48`.
- `@next` currently returns 404 because no `next` dist-tag exists yet; this is expected before the first next prerelease publish.
Pipeline design includes a post-publish verification that `npm view @mosaicstack/mosaic@next version` resolves to the exact CI-computed prerelease version. If Gitea fails to honor the `next` dist-tag, the pipeline fails closed.
## Version scheme
The prerelease version is computed at publish time only; no `package.json` version changes are committed.
For each non-private `@mosaicstack/*` package:
```text
<target-stable>-next.<CI_PIPELINE_NUMBER>
```
Where:
- `CI_PIPELINE_NUMBER` is Woodpecker's monotonic pipeline number.
- `target-stable` is the package's current committed stable version with the patch component incremented.
- Example: `@mosaicstack/mosaic` `0.0.48` publishes as `0.0.49-next.1626`.
- Example: `@mosaicstack/gateway` `0.0.6` publishes as `0.0.7-next.1626`.
Rationale:
- npm semver sorts `0.0.49-next.1627` above `0.0.49-next.1626`.
- The prerelease does not overtake the future stable `0.0.49`.
- The monotonic pipeline number avoids conflicts across repeated `next` merges.
## Branch and tag guardrails
| Pipeline path | Branch/event | Publishes | Forbidden |
| --------------------- | ------------------------------ | ------------------------------------------------------- | ---------------------- |
| stable npm publish | `main` push/manual or tag | package versions already committed in package manifests | `@next` dist-tag |
| next npm publish | `next` push/manual only | CI-computed prereleases with `--tag next` | `latest` dist-tag |
| gateway image | `main` push/manual or tag | `sha-<short>` + `latest` on main + tag on tag events | next prerelease npm |
| gateway image | `next` push/manual only | `sha-<short>` only | `latest` |
| appservice/web images | `main` push/manual or tag only | existing stable image behavior | next image publication |
The pipeline has explicit branch checks inside the publish commands as a second fail-closed layer beyond Woodpecker `when` clauses.
## Implementation plan
1. Widen `.woodpecker/publish.yml` top-level `when` to include `next` so the publish pipeline runs on next merges.
2. Keep existing `publish-npm` on `main` / tags only.
3. Add `publish-next-npm` for `next` push/manual only:
- configure Gitea npm auth from existing `gitea_token` secret as `NPM_TOKEN`;
- preflight registry dist-tag metadata;
- compute prerelease versions in CI by temporarily editing package manifests in the workspace;
- run `pnpm publish ... --tag next` against non-private `@mosaicstack/*` packages;
- verify `@mosaicstack/mosaic@next` resolves to the computed version.
4. Split image `when` anchors:
- `image_build_when` includes `next` and is used by `build-gateway`;
- `main_image_build_when` keeps appservice/web on main/tags only.
5. Keep gateway next image destinations to `sha-<short>` only; no `latest` on next.
## Risk controls
- Auth/registry failures are fatal.
- No manual image build/push path is introduced.
- No production `latest` tags are touched from `next`.
- No `@latest` npm dist-tags are touched from `next`.
- All changes live in CI config and docs; no runtime source behavior changes.
docs/archive/work-records/B2-skills-sync-path.md
+34
View File
@@ -0,0 +1,34 @@
# B2 — Fresh-install skills sync path
## Problem
Greenfield wizard on `next` reported:
```text
Skills sync script not found at ~/.config/mosaic/bin/mosaic-sync-skills
Skills: install failed
```
## Diagnosis
The framework install migration removed the legacy `~/.config/mosaic/bin/` directory and now installs framework helper scripts under:
```text
~/.config/mosaic/tools/_scripts/
```
`packages/mosaic/src/stages/finalize.ts` still resolved wizard helper scripts from `mosaicHome/bin`, so wizard-selected skills failed even though `mosaic-sync-skills` was present in the current framework layout.
## Fix
- Resolve framework helper scripts through `tools/_scripts/<name>` first.
- Keep a legacy `bin/<name>` fallback for pre-migration installs.
- Point missing-script warnings at the current `tools/_scripts` layout.
- Update the finalize skills test fixture to model the fresh framework layout.
- Update framework README examples from legacy `bin/` helper paths to `tools/_scripts/`.
## Verification
- Unit: `pnpm --filter @mosaicstack/mosaic test -- finalize-skills`
- Gates: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, `pnpm build`
- Fresh path: ran `packages/mosaic/framework/install.sh` with a temp `MOSAIC_HOME` and `MOSAIC_SYNC_ONLY=1`; verified `tools/_scripts/mosaic-sync-skills` exists, legacy `bin/mosaic-sync-skills` does not, and the script installs a selected fake `lint` skill into Mosaic + Pi runtime skill directories.
docs/archive/work-records/B3-wizard-gateway-health-order.md
+36
View File
@@ -0,0 +1,36 @@
# B3 — Wizard completion ordering
## Problem
The wizard printed the success summary / `Mosaic is ready.` during `finalizeStage`, before the gateway configuration stage had completed its daemon health check. If the gateway health gate later failed, the user could see a success claim followed by a gateway failure.
## Diagnosis
`finalizeStage` handled both mutation work and terminal success messaging. Wizard paths then ran `gatewayConfigStage` and `gatewayBootstrapStage` afterward:
1. finalize writes config, links runtime assets, syncs skills, runs doctor;
2. finalize prints `Installation Summary` + `Mosaic is ready.`;
3. gateway config starts/waits for daemon health;
4. gateway bootstrap runs.
The summary needed to be deferred until after the gateway readiness gates.
## Fix
- `finalizeStage` now returns a `showSummary()` callback and supports `deferSummary`.
- Wizard/quick-start paths call finalize with `deferSummary: true`.
- `showSummary()` is called only after gateway config reports ready and bootstrap completes, or immediately when the caller explicitly skips gateway setup.
- If gateway health/config reports not ready, the wizard returns/aborts without printing the success summary.
- Folded in adjacent runtime install hint fix for Pi: `curl -fsSL https://pi.dev/install.sh | sh`.
## Verification
- Added unified-wizard coverage for summary-after-health and no-summary-on-health-failure.
- Targeted: `pnpm --filter @mosaicstack/mosaic test -- unified-wizard finalize-skills`
- `pnpm format:check`
- `pnpm typecheck`
- `pnpm lint`
- `pnpm build`
- `pnpm test`
- Codex code review: approve.
- Codex security review: one low finding on the requested Pi `curl | sh` install hint; no security finding in the wizard completion-ordering change.
docs/archive/work-records/B4-wizard-step-dedup.md
+36
View File
@@ -0,0 +1,36 @@
# B4 — Wizard step deduplication
## Problem
Greenfield wizard testing showed completed wizard steps could be executed again after the menu marked them `[done]`. In practice this made the Providers/API-key flow and Skills flow appear twice in one wizard run.
There was a second related API-key duplication path: when the Providers step was completed with no key, `gatewayConfigStage` still prompted for `ANTHROPIC_API_KEY` during Finish because it only skipped the gateway API-key prompt when `providerKey` was non-empty.
## Diagnosis
- `runMenuLoop` labeled completed sections with `[done]`, but still dispatched the selected step again if the user selected that row.
- Quick Start ran Providers and Skills but did not mark those sections complete in `completedSections`.
- `runFinishPath`/`quickStartPath` defaulted `providerType` to `none` for gateway config, which made it impossible for `gatewayConfigStage` to distinguish:
- provider step completed and user intentionally skipped the key, vs.
- provider step was never run.
## Fix
- Added a shared menu section key helper and a completed-step guard in `runMenuLoop`.
- Completed menu steps now log a skip message instead of re-running their stage.
- Quick Start marks Providers and Skills complete after running them.
- Finish/Quick Start now pass `state.providerType` as-is to gateway config instead of defaulting to `none`.
- `gatewayConfigStage` treats `providerType: 'none'` as an explicit completed provider setup with no key and skips the second gateway API-key prompt.
## Verification
- Added unified wizard regression coverage asserting repeated Providers/Skills menu selections only execute each stage once.
- Added gateway config coverage asserting `providerType: 'none'` does not prompt for a gateway API key and writes no API key env var.
- Targeted: `pnpm --filter @mosaicstack/mosaic test -- unified-wizard gateway-config`
- `pnpm format:check`
- `pnpm typecheck`
- `pnpm lint`
- `pnpm build`
- `pnpm test`
- Codex code review: approve.
- Codex security review: no findings.
docs/archive/work-records/BUG-CLI-scratchpad.md
+47
View File
@@ -0,0 +1,47 @@
# BUG-CLI Scratchpad
## Objective
Fix 4 CLI/TUI polish bugs in a single PR (issues #192, #193, #194, #199).
## Issues
- #192: Ctrl+T leaks 't' into input
- #193: Duplicate React keys in CommandAutocomplete
- #194: /provider login false clipboard claim
- #199: TUI shows hardcoded version "0.0.0"
## Plan and Fixes
### Bug #192 — Ctrl+T character leak
- Location: `packages/cli/src/tui/app.tsx`
- Fix: Added `ctrlJustFired` ref. Set synchronously in Ctrl+T/L/N/K handlers, cleared via microtask.
In the `onChange` wrapper passed to `InputBar`, if `ctrlJustFired.current` is true, suppress the
leaked character and return early.
### Bug #193 — Duplicate React keys
- Location: `packages/cli/src/tui/components/command-autocomplete.tsx`
- Fix: Changed `key={cmd.name}` to `key={`${cmd.execution}-${cmd.name}`}` for uniqueness.
- Also: `packages/cli/src/tui/commands/registry.ts` — `getAll()` now deduplicates gateway commands
that share a name with local commands. Local commands take precedence.
### Bug #194 — False clipboard claim
- Location: `apps/gateway/src/commands/command-executor.service.ts`
- Fix: Removed the `\n\n(URL copied to clipboard)` suffix from the provider login message.
### Bug #199 — Hardcoded version "0.0.0"
- Location: `packages/cli/src/cli.ts` + `packages/cli/src/tui/app.tsx`
- Fix: `cli.ts` reads version from `../package.json` via `createRequire`. Passes `version: CLI_VERSION`
to TuiApp in both render calls. TuiApp has new optional `version` prop (defaults to '0.0.0'),
passes it to TopBar instead of hardcoded `"0.0.0"`.
## Quality Gates
- CLI typecheck: PASSED
- CLI lint: PASSED
- Prettier format:check: PASSED
- Gateway lint: PASSED
docs/archive/work-records/FED-M3-05-list-verb.md
+52
View File
@@ -0,0 +1,52 @@
# FED-M3-05 — Federation List Verb Scratchpad
## Objective
Implement `POST /api/federation/v1/list/:resource`.
## Scope
- Wire `FederationAuthGuard` → `FederationScopeService` → read-only list query layer.
- Apply `max_rows_per_query` row cap and return pagination metadata when truncated.
- Tag returned rows with `_source: "local"`.
- Keep audit writes deferred to M4.
- No request/response body persistence.
## Base / branch
- Branch: `feat/federation-m3-verb-list`
- Base: `main` after M3-04 scope service merged via PR #672 (`c739256a`).
## Implementation notes
- Added `ListController` under `apps/gateway/src/federation/server/verbs/`.
- Added `FederationListQueryService` as the read-only query layer and native RBAC evaluator.
- Query resources supported in M3 list path:
- `tasks`: project/mission scoped tasks visible through personal/team project access.
- `notes`: non-empty `mission_tasks.notes` rows visible through personal/team mission access.
- `memory`: user-owned `insights` and `preferences` rows.
- `credentials` / `api_keys`: denied by native RBAC in M3 even if present in scope; sensitive-resource implementation is not part of FED-M3-05.
- Cursor pagination uses an opaque base64url keyset cursor over `(createdAt, id)`; DB reads fetch at most `limit + 1` rows per resource query.
- Reviewer isolation fix: `mission_tasks.notes` rows are always constrained by `missionTasks.userId = subjectUserId` and accessible mission IDs; team scope narrows missions but never widens to other users' mission task notes.
- Follow-up review fix: memory listing now uses deterministic table-block pagination (`insights` first, then `preferences`) with cursor source metadata, so one table's cursor is never applied to the other.
- Follow-up hardening: missing auth-guard context returns a structured federation `unauthorized` envelope; unsupported resources and non-encodable truncated cursors throw instead of silently crashing/truncating.
## Tests
- `pnpm --filter @mosaicstack/gateway test -- list.controller.spec.ts list-query.service.spec.ts` — PASS (16 tests, including PGlite regression coverage for team-scoped notes isolation, unauthorized mission notes exclusion, `includePersonal: false`, deterministic memory pagination, missing context envelope, unsupported resource, and cursor encode failure).
- `pnpm --filter @mosaicstack/gateway typecheck` — PASS.
- `pnpm --filter @mosaicstack/gateway lint` — PASS.
- `pnpm format:check` — PASS.
- `pnpm typecheck` — PASS (41/41 turbo tasks).
- `pnpm lint` — PASS (23/23 turbo tasks).
- `pnpm --filter @mosaicstack/gateway test` — FAIL in pre-existing/live-DB integration suite: `apps/gateway/src/__tests__/cross-user-isolation.test.ts` cleanup cannot connect to local PostgreSQL on `localhost:5433`. New list tests pass; failure is outside FED-M3-05.
## Review evidence
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — PASS after follow-up remediation; approve, no findings.
- `~/.config/mosaic/tools/codex/codex-security-review.sh --uncommitted` — PASS after follow-up remediation; risk level none, no findings.
- Security-review note: read-path audit logging remains intentionally deferred to M4 per orchestrator clarification and FED-M3-05 scope.
## Risks / follow-up
- Read-path audit logging remains intentionally deferred to M4.
docs/archive/work-records/FED-M3-07-capabilities.md
+65
View File
@@ -0,0 +1,65 @@
# FED-M3-07 — Capabilities Verb Scratchpad
## Objective
Implement `GET /api/federation/v1/capabilities` in `apps/gateway/src/federation/server/verbs/capabilities.controller.ts`.
## Scope
- Add read-only capabilities controller under federation server verbs.
- Use `FederationAuthGuard` only; active grant is sufficient and no native RBAC/scope-service eval runs.
- Response shape: `{ resources, excluded_resources, max_rows_per_query, supported_verbs }` derived from grant scope.
- Register controller in `FederationModule`.
- Unit-test happy path, defaults, no-context guard seam, and invalid scope handling.
## Constraints / assumptions
- Issue: #462.
- Branch: `feat/federation-m3-verb-capabilities` from `origin/main` (`3eeed04e`).
- Depends on M3-03 auth guard; guard attaches `request.federationContext.scope` after active-grant validation.
- ASSUMPTION: `supported_verbs` is the M3 verb set from `@mosaicstack/types` (`list`, `get`, `capabilities`).
- ASSUMPTION: `filters`/`rate_limit` are intentionally omitted for FED-M3-07 because the card’s response shape lists only the four required fields.
- Budget: no explicit hard cap from orchestrator; working cap ~4K-8K tokens for card implementation + tests + PR cycle.
## Plan
1. Write controller unit tests first.
2. Implement controller and module registration.
3. Run scoped tests + typecheck/lint/format.
4. Run Codex code/security review and remediate.
5. Commit, queue guard, push, PR via wrapper.
## Progress
- 2026-06-24: Intake complete; fresh worktree created from origin/main.
- 2026-06-24: Added `CapabilitiesController`, registered it in `FederationModule`, and added 5 unit tests.
- 2026-06-24: Code/security reviews passed with no findings.
## Tests run
- `pnpm --filter @mosaicstack/gateway test -- capabilities.controller.spec.ts` — PASS (5 tests).
- `pnpm --filter @mosaicstack/gateway typecheck` — PASS.
- `pnpm --filter @mosaicstack/gateway lint` — PASS.
- `pnpm format:check` — PASS.
- `pnpm typecheck` — PASS (41/41 turbo tasks).
- `pnpm lint` — PASS (23/23 turbo tasks).
- `pnpm test` — FAIL in pre-existing/live-DB integration suite: `apps/gateway/src/__tests__/cross-user-isolation.test.ts` cleanup hit PostgreSQL connection/schema state for the `messages` table. Changed capabilities tests passed; failure is outside FED-M3-07 surface. No `fleet-personas.spec` flake encountered.
## Review evidence
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — PASS/approve, no findings.
- `~/.config/mosaic/tools/codex/codex-security-review.sh --uncommitted` — PASS, risk level none, no findings.
## Risks / blockers
- Full repo `pnpm test` may hit known `fleet-personas.spec` flake per orchestrator; ignore that specific flake if encountered.
- Previous card saw local DB schema issue in `cross-user-isolation.test.ts`; scoped capabilities tests should be authoritative for this surface.
## Acceptance evidence mapping
| Acceptance criterion | Evidence |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| GET `/api/federation/v1/capabilities` exists | Route metadata test in `capabilities.controller.spec.ts`; scoped test PASS |
| Uses active-grant auth guard and no RBAC eval | Guard metadata test confirms only `FederationAuthGuard`; controller has no service injections/RBAC calls; scoped test PASS |
| Response enumerates resources/excluded/max rows/supported verbs from scope | Happy-path/default scope tests + response schema parse; scoped test PASS |
| Read-only/no persistence side effects | Controller only parses request `federationContext.scope` and returns a DTO; no DB/service dependency; code review PASS |
docs/archive/work-records/FED-M3-09-query-source.md
+67
View File
@@ -0,0 +1,67 @@
# FED-M3-09 — Query Source Service Scratchpad
## Objective
Implement `apps/gateway/src/federation/client/query-source.service.ts` for `source: "local" | "federated:<host>" | "all"` routing.
## Scope
- Add QuerySourceService in gateway federation client layer.
- Unit-test local-only, single federated peer, all-source fan-out/merge, and per-peer partial failures.
- Keep `docs/federation/TASKS.md` read-only per project agent guidance.
## Constraints / assumptions
- Issue: #462.
- Branch: `feat/federation-m3-query-source` from `origin/main` (`e0e7be70`).
- ASSUMPTION: `federated:<host>` should match active outbound peers by `commonName` first and by `endpointUrl` host/hostname as compatibility fallback; source tags use `peer.commonName` per `@mosaicstack/types` source-tag docs.
- ASSUMPTION: QuerySourceService provides list/fan-out behavior; get/source routing can be layered later because card acceptance says merge rows.
- ASSUMPTION: `source: "all"` cannot safely return a single continuation cursor for multiple sub-sources; any subquery cursor marks the merged response `_partial: true` + `_truncated: true` while omitting `nextCursor`.
- Budget: no explicit hard cap from orchestrator; working cap ~8K-12K tokens for card 1 implementation + tests + PR cycle.
- OpenBrain unavailable: credential loader failed with missing `/home/jarvis/.config/mosaic/credentials.json`; not blocking code delivery.
## Plan
1. Review federation client/types/db patterns.
2. Write unit tests for source behavior.
3. Implement QuerySourceService and export/register it in FederationModule.
4. Run scoped tests, typecheck, lint, format.
5. Run codex uncommitted review and remediate.
6. Commit, queue guard, push, PR via wrapper.
## Progress
- 2026-06-24: Intake complete; using isolated worktree to avoid dirty orchestrator files in original checkout.
- 2026-06-24: Added QuerySourceService, module export, barrel export, and 7 unit tests.
- 2026-06-24: First Codex review found pagination and port-host matching issues; both remediated with tests.
## Tests run
- `pnpm --filter @mosaicstack/gateway test -- query-source.service.spec.ts` — PASS (7 tests).
- `pnpm --filter @mosaicstack/gateway typecheck` — PASS.
- `pnpm --filter @mosaicstack/gateway lint` — PASS.
- `pnpm format:check` — PASS.
- `pnpm typecheck` — PASS (41/41 turbo tasks).
- `pnpm lint` — PASS (23/23 turbo tasks).
- `pnpm test` — FAIL in pre-existing/live-DB integration suite: `apps/gateway/src/__tests__/cross-user-isolation.test.ts` cleanup hit `relation "messages" does not exist` against local PostgreSQL. Changed QuerySource unit tests passed; failure is outside FED-M3-09 surface and appears tied to local DB schema state.
## Review evidence
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — first pass request-changes, 2 should-fix findings (all-source cursor handling; endpoint port host matching).
- Remediation: `_partial` + `_truncated` when any all-source subquery has `nextCursor`; endpoint match accepts URL `host` and `hostname`; added tests for both.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — PASS/approve, no findings.
- `~/.config/mosaic/tools/codex/codex-security-review.sh --uncommitted` — PASS, risk level none, no findings.
## Risks / blockers
- Federation query layer is not yet wired; service API needs to be stable and easy to compose.
- Must avoid hard-failing `source: all` on remote peer failures.
## Acceptance evidence mapping
| Acceptance criterion | Evidence |
| ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| local source returns local rows tagged `_source: local` | `query-source.service.spec.ts` local test; scoped test PASS |
| `federated:<host>` queries selected peer and tags rows with peer source | `query-source.service.spec.ts` commonName/endpoint-host tests; scoped test PASS |
| `all` fans out local + active outbound peers in parallel and merges tagged rows | `query-source.service.spec.ts` all-source call-order/merge test; scoped test PASS |
| per-peer failure on `all` returns `_partial: true`, not throw | `query-source.service.spec.ts` peer failure test; scoped test PASS |
docs/archive/work-records/FED-M3-10-integration-tests.md
+60
View File
@@ -0,0 +1,60 @@
# FED-M3-10 — Federation M3 Integration Tests
## Objective
Add single-gateway gateway integration tests for M3 acceptance #6 and #7.
## Branch / base
- Branch: `feat/federation-m3-integration`
- Base: `origin/next` (`838701bd` after M3-06/#683 merge)
- PR base when unblocked: `next`
## Scope
- Real PostgreSQL via `@mosaicstack/db`.
- Mocked TLS context / Fastify request shim for `FederationAuthGuard`.
- Direct controller calls using the real M3 route contract: `POST /api/federation/v1/list/:resource` with body `{ limit?, cursor? }`.
- Gated by `FEDERATED_INTEGRATION=1`.
- No federation harness dependency.
## Fixture notes
Aligned with the B2 seed design vocabulary:
- `tasks` visibility uses personal `projects` + `missions` chain.
- `notes` are `mission_tasks.notes`; the integration suite asserts subject-only note visibility on an authorized mission.
- Seed includes a second user and unauthorized team/project tasks to prove exclusion from the max-row-cap list result.
- Grants/peers are direct DB fixtures; cert auth still runs through `FederationAuthGuard` using real X.509 certs generated by existing test helpers.
## Current implementation
Added `apps/gateway/src/__tests__/integration/federation-m3-list.integration.test.ts` covering:
1. M3 #6 — cert missing Mosaic OIDs returns 401 federation `unauthorized` envelope.
2. M3 #6 — valid cert whose grant row is `revoked` returns 403 federation `forbidden` envelope.
3. M3 #7 — active grant with `max_rows_per_query: 2` caps `list tasks`, returns `_truncated` + `nextCursor`, source-tags rows, and excludes other-user / unauthorized-team tasks.
4. Cross-user notes invariant — subject can list their own `mission_tasks.notes` row while another user's note on the same authorized mission is excluded.
5. Unsupported-resource invariant — `list widgets` fails closed with a federation `scope_violation` envelope.
## Verification
- `pnpm --filter @mosaicstack/types build` — PASS.
- `pnpm --filter @mosaicstack/db build` — PASS.
- `pnpm --filter @mosaicstack/storage build` — PASS.
- `pnpm --filter @mosaicstack/brain build` — PASS.
- `pnpm --filter @mosaicstack/queue build` — PASS.
- `pnpm --filter @mosaicstack/config build` — PASS.
- `pnpm --filter @mosaicstack/auth build` — PASS.
- `pnpm --filter @mosaicstack/gateway test -- src/__tests__/integration/federation-m3-list.integration.test.ts` — PASS skipped when `FEDERATED_INTEGRATION` unset (5 skipped).
- `FEDERATED_INTEGRATION=1 pnpm --filter @mosaicstack/gateway test -- src/__tests__/integration/federation-m3-list.integration.test.ts` — PASS (5 tests) after local `docker compose up -d postgres` + `pnpm --filter @mosaicstack/db db:push`.
- `pnpm --filter @mosaicstack/gateway typecheck` — PASS.
- `pnpm --filter @mosaicstack/gateway lint` — PASS.
- `pnpm format:check` — PASS.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — PASS; approve, no findings.
- `~/.config/mosaic/tools/codex/codex-security-review.sh --uncommitted` — PASS; risk level none, no findings.
## Push / PR
- #683 landed in `next`; branch rebased onto `origin/next` before push.
- CI is serialized; run queue guard before push.
docs/archive/work-records/README.md
+19
View File
@@ -0,0 +1,19 @@
# Archived work records
> **Status:** Historical evidence index. These scratchpads record prior task execution and investigation; they are not active task state, current requirements, implementation contracts, or operational authority.
Ninety-four records moved byte-identically from migration quarantine on 2026-08-10 after a repository-wide consumer scan found no current path/name references and no coupled local Markdown links.
Because this collection is large, use repository search by issue, task, or topic rather than treating every file as current navigation. Validate all technical claims and commands against current source, tests, configuration, and safety policy.
## Retained in quarantine
Seventeen scratchpads were deliberately not moved because they are named by current control documents, tests/fixtures, retained mission/evidence records, or have a coupled KBN-101 report link. They must migrate with their owning workstream or consumer update.
## Boundaries
- Historical completion wording does not update `docs/TASKS.md` or mission authority.
- Old commands are not approved runbooks.
- Old security findings are not proof of current posture.
- Draft designs are not current architecture.
- Files remain byte-identical; this index supplies the lifecycle classification.
docs/archive/work-records/bug-196-admin-redirect.md
+37
View File
@@ -0,0 +1,37 @@
# BUG-196: Admin Page Redirect Issue
## Problem
Admin page redirects to /chat for users with admin role because role check fails.
## Root Cause
The `role` field is defined as an `additionalField` in better-auth's user configuration, but
better-auth v1.5.5 does not automatically include additionalFields in the session response from
the `getSession()` API. This causes the admin role check to fail:
- Frontend: `AdminRoleGuard` checks `user?.role !== 'admin'`
- Backend: `AdminGuard` checks `user.role !== 'admin'`
- When `role` is `undefined`, both checks treat the user as non-admin and deny access
## Solution
Implemented a defensive check in the backend `AdminGuard` that:
1. First tries to use the `role` field from the session (if better-auth includes it)
2. Falls back to fetching the role directly from the database if it's missing
3. Defaults to 'member' if the user has no role set
This ensures that admin users can always access the admin panel, and also protects against
the case where better-auth doesn't include the additionalField in future versions.
## Files Changed
1. `/apps/gateway/src/admin/admin.guard.ts` - Added fallback role lookup
2. `/packages/auth/src/auth.ts` - No changes needed (better-auth config is correct)
## Verification
- All three quality gates pass: `typecheck`, `lint`, `format:check`
- Backend admin guard now explicitly handles missing role field
- Frontend admin guard remains unchanged (will work once role is available)
docs/archive/work-records/ci-docker-publish-20260330.md
+30
View File
@@ -0,0 +1,30 @@
# Scratchpad: CI Docker Publish (2026-03-30)
- Objective: Add Woodpecker Docker build+push steps for gateway and web images on `main` pushes.
- Scope: `.woodpecker/ci.yml`.
- Constraints:
- Use existing Dockerfiles at `docker/gateway.Dockerfile` and `docker/web.Dockerfile`.
- Publish to `git.mosaicstack.dev` with `from_secret` credentials.
- Tag both `latest` and `${CI_COMMIT_SHA}`.
- Do not run publish steps on pull requests.
- ASSUMPTION: Publishing `latest` is required by the task for registry convenience, even though immutable tags remain the safer deployment reference.
- Findings:
- Existing pipeline already has `build` after `lint`, `format`, and `test`.
- `apps/gateway/package.json` uses `tsc` for `build`; no Prisma dependency or `prisma generate` hook is present.
- Plan:
1. Patch `.woodpecker/ci.yml` to keep `build` as the quality gate successor and add `publish-gateway` plus `publish-web`.
2. Validate YAML and run repo quality gates relevant to the change.
3. Review the diff, then commit/push/PR if validation passes.
- Verification:
- `python3 -c "import yaml; yaml.safe_load(open('.woodpecker/ci.yml'))" && echo "YAML valid"`
- `pnpm lint`
- `pnpm typecheck`
- `pnpm format:check`
- `docker compose up -d`
- `pnpm --filter @mosaicstack/db db:push`
- `pnpm test`
- `pnpm build`
- Manual review of `.woodpecker/ci.yml` diff: publish steps are main-only, depend on `build`, and use secret-backed registry auth plus dual tags.
- Risks:
- Pipeline behavior beyond YAML validation cannot be fully proven locally; remote Woodpecker execution will be the final situational check after push.
- Repo baseline required two existing `plugins/macp` files to be reformatted before `pnpm format:check` would pass.
docs/archive/work-records/fcm-m2-001-generated-env-boundary.md
+110
View File
@@ -0,0 +1,110 @@
# FCM-M2-001 — Generated Environment Boundary
- **Issue/card:** #758 / FCM-M2-001
- **Branch/base:** `feat/758-generated-env-boundary` from `origin/main` `e9c4aa3e8b3780719cd5a43c0ef3f37fc70de666`
- **Budget assumption:** 30K-card budget; implement only the deterministic generated/local environment boundary and its launch-chain/docs/tests.
## Objective
Replace the generic fleet agent `.env` authority/merge path with a deterministic roster-derived `<agent>.env.generated` projection and strict, data-only `<agent>.env.local`. The roster remains the desired-state authority. Reject bad input before the launcher creates a tmux session; never print sensitive or privileged-command values.
## Scope and non-goals
- In scope: deterministic render/write, strict generated/local parse rules, legacy `.env` disposition/quarantine, systemd/launcher boundary, permission/path checks, focused fail-closed tests, operator/reference documentation, USC interface evidence.
- Excluded: roster CRUD/mutation, v2 roster schema changes, lifecycle/reconcile/apply behavior, migration/canary rollout, connectors, remote surfaces, live-fleet actions, and M2-002.
## Plan
1. Add red tests for generated-key shadowing, malformed/duplicate/unknown/command/sensitive input, no-value diagnostics, deterministic/idempotent projection, secure file modes, and legacy disposition.
2. Implement a pure strict environment contract plus atomic projection/quarantine helper.
3. Replace the generic `.env` writer/merge path and systemd reference with `.env.generated` + `.env.local` ownership.
4. Make the shell launcher parse the files without `source`/`eval`, reject unsafe input before tmux creation, and construct only the roster-derived runtime command.
5. Add operator/reference documentation with the requested USC M1 interface evidence and M2–M4 gate statement.
6. Run focused/package/root gates and audit the USC interface packet. Per continuation scope, stop before review, commit, push, PR, or live mutation.
## Initial evidence
- No existing owner: target worktree path absent; no target local/remote branch; `pr-list.sh -s open` returned no open PRs.
- M1 compiler/API/docs and executable disposition evidence are present at the assigned base.
- Existing launch chain writes `fleet/agents/<agent>.env`, preserves arbitrary legacy lines via `mergeAgentEnv`, sources `MOSAIC_AGENT_COMMAND`, and executes it through `bash -c`; all are M2 remediation targets.
- `~/.config/mosaic/guides/SECURITY.md` is absent. Read the available security-review role contract and the vault/secrets guide instead.
## Verification log
### Continuation (2026-07-14)
- Preserved the inherited 14-file delta; no reset, stash, rebase, roster mutation, lifecycle action,
live-fleet action, commit, push, or PR action was performed.
- Focused gates passed:
- `pnpm --dir packages/mosaic test -- src/fleet/generated-env-boundary.spec.ts` — 1 file, 10 tests passed.
- `bash packages/mosaic/framework/tools/fleet/test-start-agent-session.sh` — passed.
- `bash packages/mosaic/framework/systemd/user/test-fleet-units.sh` — passed.
- `pnpm --dir packages/mosaic test -- src/commands/fleet.spec.ts` — 1 file, 192 tests passed.
- Package gates passed before final documentation/format follow-up:
- `pnpm --dir packages/mosaic typecheck` — passed.
- `pnpm --dir packages/mosaic lint` — passed.
- `pnpm --dir packages/mosaic test` — 52 files, 752 tests passed.
- `pnpm format:check` initially failed only for the new boundary reference and generated-boundary
TypeScript files; targeted Prettier normalization was applied. A final `pnpm format:check` passed.
- USC packet audit: the M1 structural compiler is `parseRosterV2` with roster `version: 2`; the
semantic resolver is `validateRosterV2Semantics`; disposition artifacts retain `version: 1` fixture
evidence. `docs/TASKS.md` records M1-001 done, M1-002 in-progress, and M1-003 not-started; no
product release version is claimed. The packet now distinguishes these statuses from checkout
artifact presence and states the M2 → M3 → M4 downstream gates.
## Review remediation (2026-07-14)
- **Blocker 1 red-first:** Added a launcher reproducer with a `0777` `fleet/agents` parent and a private generated file. Before implementation, `bash packages/mosaic/framework/tools/fleet/test-start-agent-session.sh` failed: `FAIL: generated file under a world-writable parent was accepted`. The failure occurred after the launch path reached fake tmux, proving the parent was not validated.
- **Blocker 2 red-first:** Added a `symlink()` projection-directory reproducer that preloads generated/local/quarantine/legacy target files and asserts no target mutation. Before implementation, `pnpm --dir packages/mosaic test -- src/fleet/generated-env-boundary.spec.ts` failed the new test because the existing writer followed the `agentEnvDir` symlink and parsed its target legacy input (`expected /unsafe-directory/i`, received `code=malformed-line`). The initial test-only missing `mkdir` import was corrected before recording this behavior failure.
- **Blocker 3 red-first:** Added fresh/stale/absent native-heartbeat regression coverage. Before implementation, an isolated fake-tmux launcher reproducer with a fresh `<agent>.hb.native` marker failed `FAIL: fresh native heartbeat was overwritten`; the existing sidecar immediately replaced native `status=busy`/`model` content.
- **Remediation result:** The launcher now rejects a group/world-accessible or symlinked `fleet/agents` parent before an environment read or tmux call. The projection writer uses `lstat` before chmod/write processing and rejects a symlinked directory without creating generated/local/quarantine files or deleting legacy input. The heartbeat sidecar defers to a fresh non-symlink native marker and falls back when stale/absent. Focused green evidence before independent review: `bash packages/mosaic/framework/tools/fleet/test-start-agent-session.sh` and `pnpm --dir packages/mosaic test -- src/fleet/generated-env-boundary.spec.ts` (11 tests) passed.
- **Independent-review follow-up red-first:** Codex code review returned one blocker and security review one medium CWE-732 finding: the writer repaired an already `0777` directory with `chmod` before trusting its contents. Added a reproducer with a safe local file beneath an existing `0777` directory. Before the follow-up fix, `pnpm --dir packages/mosaic test -- src/fleet/generated-env-boundary.spec.ts` failed because the promise resolved and wrote `coder0.env.generated` instead of rejecting.
- **Independent-review remediation:** Existing directories now pass non-following private-directory validation before any read or chmod; only a directory created in this call is normalized to `0700`. The fleet-add test fixture now creates its simulated trusted `fleet/agents` boundary at `0700`; this corrects fixture setup to match the new required contract rather than weakening the rejection assertion. Focused reruns passed: generated-boundary 12 tests, launcher boundary suite, and fleet suite 192 tests.
- **Final verification before re-review:** Launcher + systemd suites passed; package suite passed (52 files, 754 tests); package lint/typecheck, root typecheck (42 tasks), format check, and diff check passed. The rerun code review still reports a tmux command-arity blocker, and the security rerun reports systemd `EnvironmentFile` pre-validation injection findings for both agent units. These were discovered after the specified three-remediation scope; no additional source changes were made. Independent review therefore remains `REQUEST CHANGES` despite the requested three fixes passing their behavioral suites.
## Systemd pre-validation remediation (2026-07-14)
- **Red-first:** Updated the fleet unit contract to reject any `EnvironmentFile=` projection preload, require a cleared bootstrap environment, and require a validated exact-stop path. Before implementation, `bash packages/mosaic/framework/systemd/user/test-fleet-units.sh` failed: `FAIL: agent units must not preload projections before strict parsing`.
- **Red-first parser/stop coverage:** Added interaction-wrapper and exact-stop cases to the launcher boundary suite. Before implementation, `bash packages/mosaic/framework/tools/fleet/test-start-agent-session.sh` failed: `FAIL: interaction did not use shared strict parser first`, because the interaction wrapper consumed inherited environment before projection validation.
- **Focused green:** Both unit templates now use `env -i` with fixed `HOME`, agent instance, and PATH; neither has `Environment=`/`EnvironmentFile=`. The interaction wrapper delegates to `start-agent-session.sh --interaction`, so strict generated/local parsing precedes pinned Pi profile checks. `--stop` reuses the strict generated parser before exact `=<agent>` socket/session termination. Passed: systemd unit suite, launcher boundary suite (including malformed interaction, pinned profile, and ambient-socket stop cases), and 210 focused TypeScript tests.
- **Final verification:** `pnpm --dir packages/mosaic test` passed (52 files, 754 tests); package lint/typecheck, root typecheck (42 tasks), format/diff, and shell syntax checks passed. Security review passed with no findings. Code review repeated the previously refuted tmux argv concern and a pre-existing Claude trust-lock suggestion; per the assigned narrow follow-up, no tmux or unrelated trust-path change was made.
## Risks and next review
- This card is uncommitted and unreleased. The canonical tracker still records its dependencies as
M1-002 in progress and M1-003 not started; this continuation does not reinterpret those task states.
- Final post-documentation checks passed: `pnpm --dir packages/mosaic typecheck`,
`pnpm --dir packages/mosaic lint`, `pnpm --dir packages/mosaic test` (52 files, 752 tests),
`pnpm typecheck` (42 Turbo tasks), and `pnpm format:check`.
- Obtain independent code and security review of the complete delta next. Do not run commit, push,
PR, or live-fleet commands in this continuation.
## Fresh-install directory remediation (2026-07-14)
- **Objective:** Remediate only the fresh-install path where `installFleet` created `fleet/agents`
with host-umask permissions before the boundary writer correctly rejected it.
- **Plan:** Add a real `fleet install --no-enable` integration reproducer; prove red; let the
existing boundary writer own directory creation; run focused and full gates. No commit, push,
PR, review disposition, or live-fleet action.
- **Red evidence:** Before the one-line remediation,
`pnpm --dir packages/mosaic test -- src/commands/fleet.spec.ts` failed the new test with
`AgentEnvBoundaryError: code=unsafe-permissions` at `ensurePrivateProjectionDirectory`, after
`installFleet` pre-created the directory.
- **Change:** Removed only the recursive `mkdir(activePaths.agentEnvDir)` in `installFleet`.
`writeAgentEnvironmentProjection` remains the sole creator and retains its existing `lstat`,
private-directory, symlink, and existing-unsafe-directory fail-closed checks.
- **Focused green:** `pnpm --dir packages/mosaic test -- src/commands/fleet.spec.ts` — 193 tests
passed. The new integration executes a fresh `fleet install --no-enable`, asserts a real
non-symlink `0700` directory and a `0600` generated projection. Existing unsafe-directory
coverage remains in `generated-env-boundary.spec.ts` and asserts no chmod repair/no generated
file write.
- **Full gates green:** generated-boundary 12 tests; launcher and systemd suites; package
typecheck/lint and 52 files / 755 tests; root typecheck (42 tasks), lint, format, diff check,
and root test (42 tasks) all passed.
- **Independent review:** The complete inherited uncommitted delta still has Codex `REQUEST CHANGES`
findings outside this narrow fix (tmux command arity and Claude trust-lock regression), plus a
security-review medium finding on unvalidated writable ancestor directories. No out-of-scope
source changes were made.
- **Risk:** The writer's existing create-then-validate sequence is relied on for the creation
boundary; a concurrent substitution causes fail-closed validation rather than repair. The
review findings above remain residual risks for the complete card delta.
docs/archive/work-records/fcm-m5-001-fleet-config-operator-docs.md
+85
View File
@@ -0,0 +1,85 @@
# FCM-M5-001 — Fleet configuration operator documentation
- Task: `FCM-M5-001`
- Issue: `#758`
- Branch: `docs/758-fleet-config-operator-docs`
- Exact base: `9745bc3f29c26b021a478b7ad03cfb494f6c9de3` (tree `4da210da9a71b035130d4160a4a2e691bdfde2da`)
## Objective
Deliver the accepted fleet documentation information architecture, operator workflows, operations and migration references, comprehensive contract documentation, and deterministic link/example validation without live fleet action or product mutation.
## Scope and constraints
- Documentation, examples, documentation validation, and tracking only.
- `roster.yaml` remains the sole writable desired-state authority; generated state is derived/observed.
- No M4-002 implementation or execution; no canary, migration, rollback, deployment, systemd/tmux/session, generated projection, or product mutation.
- `mos-comms` is temporary and is not permanent architecture.
- Parent issue `#758` remains open through M5.
- No credentials, sensitive values, or privileged command content.
## Plan
1. Update tracking first with exact M4-001 evidence and mark M5-001 in progress.
2. Map the M0 checklist and current implementation behavior to documentation pages.
3. Author operator, operations, migration, schema/reference, recovery, troubleshooting, and security/authority docs.
4. Add or extend deterministic documentation/link/example validation if required, red-first.
5. Run repository documentation, link, example, and relevant package checks; review and remediate.
6. Commit, queue-guard, push one branch, and open one wrapper-created PR; stop for independent review.
## Budget
- Task estimate: `24K`.
- Working cap: stay within the card estimate by parallelizing read-only discovery and limiting edits to checklist-required artifacts.
## Progress checkpoints
- [x] Loaded repository/global delivery and documentation contracts.
- [x] Verified `origin/main` is exact required base and created isolated worktree.
- [x] Tracking updated first.
- [x] Checklist mapped and docs authored.
- [x] Validation green.
- [x] Review/remediation complete.
- [x] Commit, queue guard, push, PR #789.
- [x] Rejected exact-head RoR findings repaired on a new descendant commit candidate.
- [ ] New exact-head review and CI after repair push.
## Tests and verification
- Red-first documentation validator initially failed for the absent fleet entry point and canonical
example, then passed after the IA and example were added.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/fleet/roster-v2.spec.ts src/fleet/example-profile-dispositions.spec.ts src/fleet/fleet-documentation.spec.ts src/fleet/v1-v2-migration.spec.ts src/fleet/generated-env-boundary.spec.ts src/fleet/fleet-agent-crud.spec.ts src/fleet/fleet-reconciler.spec.ts` — 7 files, 195 tests passed after building workspace dependencies.
- `pnpm format:check` — passed.
- `pnpm lint` — 23 tasks passed.
- `pnpm typecheck` — 42 tasks passed.
- `pnpm test` — 43 tasks passed; `@mosaicstack/mosaic` contributed 61 files and 1,045 tests.
- `bash packages/mosaic/framework/tools/quality/scripts/verify-sanitized.sh` — passed.
- `bash packages/mosaic/framework/tools/quality/scripts/check-resident-budget.sh` — passed.
- `git diff --check` — passed before final review.
- Independent staged-snapshot review identified four documentation/validation blockers: reboot safety,
heartbeat observation, migration failure envelope, and example-scan coverage. All were remediated;
focused rereview approved the staged remediations with no blockers. Exact committed-head review remains
a post-PR gate.
- Post-remediation `@mosaicstack/mosaic` lint/typecheck passed; package test passed 61 files / 1,045
tests; sanitization and resident-budget gates passed again.
- Post-PR exact-head RoR on rejected head `0aee2c09819fd06e28f927384ea56fa2ef374edf`
identified five blockers: update-lifecycle overclaim, missing explicit `fleet validate` gap,
fragment-blind link validation, unsupported checklist-evidence claim, and insufficient example safety
validation. Red-first regressions failed before implementation for missing-heading, privileged-command,
and credential-format fixtures. Repairs now preserve/document implementation truth, validate heading
fragments, narrow checklist claims, and scan fenced/canonical examples for common credential formats
and privileged commands without printing fixture values.
- Repair-focused fleet contracts: 7 files, 192 tests passed after review remediation; documentation
validator contributed 11 tests. Full gates passed: format; lint 23/23; typecheck 42/42; test 43/43
tasks with `@mosaicstack/mosaic` 61 files / 1,052 tests; sanitization; resident budget; and
`git diff --check`. New exact-head review/CI remain pending until the repair commit is pushed.
## Risks/blockers
- Checklist may include behavior intentionally deferred to M4-002/M5-002; such items must be recorded as approved-existing holds rather than claimed delivered.
- Commands/examples must remain non-live and avoid privileged/sensitive content.
## Final evidence
- Pending.
docs/archive/work-records/fix-ci-migrations-20260330.md
+20
View File
@@ -0,0 +1,20 @@
# Scratchpad — fix-ci-migrations-20260330
## Objective
- Fix Drizzle migration journal ordering so fresh Postgres instances apply all schema migrations in CI.
## Plan
- Inspect migration SQL, journal, and snapshot chain.
- Patch metadata consistently.
- Run full test suite.
- Record result and risks.
## Progress
- Inspected migration files and confirmed 0001_cynical_ultimatum.sql is missing from journal while 0002 depends on it.
- Patched `packages/db/drizzle/meta/_journal.json` to insert `0001_cynical_ultimatum` at idx 1 and shift later entries to idx 2-7.
- Verified snapshot content is already semantically aligned with the fixed ordering: the missing tables exist in `0001_snapshot.json`, and `mutable` appears in later snapshots.
- Installed workspace dependencies in this worktree to make the requested test command runnable.
- Ran `pnpm test` successfully; gateway reported `347 passed (347)` and Turbo finished with `35 successful, 35 total`.
docs/archive/work-records/fix-turbo-env-passthrough.md
+41
View File
@@ -0,0 +1,41 @@
# Task Scratchpad — Turbo DATABASE_URL passthrough
## Objective
- Fix CI test execution under Turborepo by passing DB-related environment variables through the `test` task.
- Preserve the repo's documented local Postgres fallback on host port `5433`.
## Scope
- Edit `turbo.json`
- Run baseline checks requested by the user and repo rules where feasible
- Publish branch and open PR if checks are acceptable
## Assumptions
- ASSUMPTION: No PRD update is required because this is a narrow CI/config bug fix that does not change product requirements.
- ASSUMPTION: `docs/TASKS.md` remains untouched because project guidance marks it orchestrator-only.
- ASSUMPTION: Local Postgres-backed tests are not runnable in this worktree, so baseline verification will rely on static gates unless the environment already provides DB access.
## Plan
- Record current repo state and avoid touching `.mosaic/orchestrator/session.lock`
- Add Turbo DB env declarations for test execution and cache invalidation
- Run `pnpm format:check`, `pnpm typecheck`, and `pnpm lint`
- Review the diff, then commit/push/PR via provider-appropriate flow
## Verification Log
- `pnpm install` completed in this worktree so repo tooling is available
- `pnpm format:check` initially failed on `turbo.json`; fixed by formatting the file
- `pnpm format:check` passed after formatting
- `pnpm typecheck` passed
- `pnpm lint` passed
- Independent review flagged two important adjustments:
- keep `DEFAULT_DATABASE_URL` on `5433` because local Docker Compose maps host `5433 -> container 5432`
- prefer Turbo `env` over `passThroughEnv` so DB config changes also affect the test cache key
## Risks
- Existing mission/orchestrator files are dirty in the worktree and must be excluded from the scoped change.
- Local tests were not run because no Postgres service is available in this worktree; CI remains the primary situational validation for the test-task behavior.
docs/archive/work-records/fleet-cli-local-canary-review-fixes.md
+54
View File
@@ -0,0 +1,54 @@
# Fleet CLI Local Canary Review Fixes
## Objective
Fix only the two should-fix code review findings:
1. Ensure `@mosaicstack/mosaic` declares `yaml` and lockfile state is current.
2. Validate `mosaic agent status [agent]` against the fleet roster before constructing/running the tmux target.
## Constraints
- Do not modify `docs/TASKS.md`.
- Leave changes uncommitted.
- Run requested formatting and quality gates.
## Plan
1. Inspect manifest/lockfile state for `yaml`.
2. Add failing regression test for `mosaic agent status typo`.
3. Patch `registerFleetAgentCommands` status validation.
4. Format touched files.
5. Run requested tests, typecheck, and lint.
6. Review final diff.
## Progress
- Loaded required repo/global/runtime instructions.
- Confirmed `packages/mosaic/package.json` already declares `yaml`.
- Confirmed `pnpm-lock.yaml` already has `packages/mosaic` importer entry for `yaml`.
- Found `registerFleetAgentCommands` status path does not validate agent before building tmux target.
## Verification
- TDD red check: `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet.spec.ts`
failed before the production fix because `mosaic agent status typo` resolved instead of
rejecting.
- Focused green check: `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet.spec.ts`
passed after adding roster validation.
- Formatting: `pnpm exec prettier --write packages/mosaic/src/commands/fleet.ts packages/mosaic/src/commands/fleet.spec.ts docs/scratchpads/fleet-cli-local-canary-review-fixes.md`
completed with all files unchanged.
- Requested tests: `pnpm --filter @mosaicstack/mosaic test -- src/commands/fleet.spec.ts src/cli-smoke.spec.ts`
passed with 36 tests.
- Baseline typecheck: `pnpm typecheck` passed.
- Baseline lint: `pnpm lint` passed.
- Independent review: `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted`
returned approve with 0 findings. Note: reviewer reported broader context inspection was limited
by its read-only sandbox, so review was based on the supplied diff.
- `docs/TASKS.md` has no diff.
## Risks
- `docs/TASKS.md` intentionally untouched per user instruction.
- Review finding 1 required no file edit: `packages/mosaic/package.json` already declares
`yaml`, and the `packages/mosaic` importer in `pnpm-lock.yaml` already includes `yaml`.
docs/archive/work-records/fleet-observability-phase2.md
+100
View File
@@ -0,0 +1,100 @@
# Scratchpad — Fleet Phase 2: Observability (W-FLEET)
> Append-only. Mission `mvp-20260312` / workstream W-FLEET.
> Lead: Jarvis (Claude) at `W-jarvis:mos-claude-18`. Coordinating with `jwoltje@dragon-lin:coder0-0`.
## Mission prompt (2026-06-20)
Establish the north star for the Mosaic Fleet feature and prepare Phase-2 observability
for delivery. The USC tmux PoC is the proven base. Jason granted lead authority:
"The fleet is a great way to actually build the MVP — we are building the system that
builds the system." Dogfood actual agent construction + ad-hoc deployment; coordinate
with a second agent on `dragon-lin`.
## Decisions of record (with Jason, 2026-06-20)
- Agent model: config defines, session runs (gateway = definition/identity/auth; tmux = runtime).
- Tenancy: multi-tenant from the start; isolation = per-tenant Linux uid.
- Health: heartbeat required; dogfood stub implements protocol now.
- Lifecycle: hybrid (core always-on + ephemeral workers).
- Observation: read-only default, opt-in takeover.
- Multi-host: designed-for day one; control plane rides federation (W1), not a bespoke broker.
- Delivery: CLI-first, dogfood on the live stub fleet; webUI deferred to Phase 5.
- Fleet is dual-role: product AND means of production (bootstrapping the MVP).
- Code review = **dual-engine**: Claude **and** gpt-5.5/Codex, run together (Jason: the
combination produces the best results). Launch reviewers via `mosaic yolo pi` / `codex`
(proven path) or `~/.config/mosaic/tools/codex/codex-code-review.sh`. Applies to all
code-review gates incl. FLEET-OBS-008. Per Jason 2026-06-20.
- Worktree discipline: do fleet work in `~/src/mosaicstack-stack-worktrees/<branch>`, NOT
the shared main checkout — concurrent processes mutate `main` there (learned 2026-06-20).
## Environment facts (verified 2026-06-20)
- Fleet is live on `W-jarvis` (uid 1000, `jarvis`, `Linger=yes`) on tmux socket
`mosaic-fleet`: `_holder`, `canary-pi`, `dogfood-coder`, `dogfood-orchestrator`,
`dogfood-reviewer`. All panes run `~/.config/mosaic/fleet/dogfood-agent.py` (stub),
including `canary-pi` (roster says runtime=pi → **drift**).
- Holder + `mosaic-agent@*` units are `active (exited)` but `UnitFileState=disabled`
(reboot loses fleet → boot-enable gap to surface).
- Observation blocked by: isolated socket (hidden from default `tmux ls`), `capture-pane`
blank for TUIs, `attach` being read-write + resizing.
- Second agent: `jwoltje@dragon-lin`, session `coder0-0` (group `coder0`), running `node`,
default socket. ssh forward reach confirmed.
## Governance / collision-safety
- `mosaicstack-stack` has active mission `mvp-20260312` with single-writer locks on
`docs/MISSION-MANIFEST.md`, `docs/TASKS.md`, `docs/scratchpads/mvp-20260312.md`.
- This workstream touches NONE of those. All Fleet docs scoped under `docs/fleet/` +
this scratchpad. Rollup row proposed, not written.
## Session log
- 2026-06-20: Researched AI guide + fleet code + live state. Established north star with
Jason (8 forks decided). Branched `feat/fleet-observability`. Persisted
`docs/fleet/{north-star.md,PRD.md,TASKS.md}` + this scratchpad. Next: establish comms
with dragon-lin coder, commit docs, begin Phase-2 delivery (heartbeat + `fleet ps`).
- 2026-06-20 (session 2): Built Phase-2 CLI via worker (commit ab47831): `fleet ps`,
`agent watch`, `agent send --verify`, 62 tests. LIVE-verified `fleet ps` on
mosaic-fleet — correctly flagged canary-pi DRIFT + BOOT-ENABLE, tenant_id+host in JSON.
Heartbeat responder added to dogfood-agent.py (FLEET-OBS-002) — `fleet ps` HB now
`healthy` for all 4 agents.
- Coordination: dual-engine-reviewed (Claude+Codex) and merged framework PRs #572
(sanitization gate) + #575 (CONSTITUTION extraction) as Lead. Codex caught an Alpine
blocker on #572 (refuted by CI); Claude caught a CI-breaking format failure on #575.
- **FINDINGS (north-star / Phase-3 blockers):**
1. Ad-hoc `mosaic yolo {codex,pi}` via `start-agent-session.sh` DIE immediately in a
detached tmux pane (codex: "stdin is not a terminal"; pi: same). Only the python stub
survives. => Real runtimes have NEVER run durably in the fleet. Launch path (PATH/TTY
in the detached shell) must be fixed before Phase-3 real-runtime swap. `fleet ps`
caught both dead panes instantly (tool validated).
2. `MOSAIC_AGENT_NAME` (set in systemd EnvironmentFile) is NOT propagated into tmux's
global env, so agents defaulted to `unknown`. Worked around in dogfood-agent.py via
tmux session-name fallback; the systemd/tmux env handoff needs a real fix.
- Next: rebase on merged main, open Phase-2 PR, dual-engine review, merge, close
`fleet-observability-1`. Defer launch-path + env-propagation fixes to Phase 3.
- 2026-06-21 (session 3): Phase-2 PR #579 merged (3 dual-engine rounds hardened
verify+watch). Then closed the launch-path question with Jason's input — CORRECTING
earlier findings:
- The ad-hoc launch deaths were NOT a fundamental TTY blocker: (a) codex was a stale
version (Jason updated it); (b) pi was misconfigured to Claude auth (Jason removed it;
default is now Codex). The REAL durable-launch bug is **PATH**: the detached tmux
launch shell is login+non-interactive, so it misses `~/.npm-global/bin` (added only in
`~/.bashrc`) -> `mosaic: command not found` (127) -> pane dies. tmux panes inherit the
tmux _server_ env, so PATH must be baked into the pane command.
- **Durable real-agent recipe (validated live on gpt-5.5, Claude-free):**
`mosaic yolo pi --model openai-codex/gpt-5.5:high` — pi tolerates detached tmux; a raw
interactive TUI (codex CLI) exits without an attached client. Status line confirmed
`(openai-codex) gpt-5.5 • high`.
- PATH fix landed in `start-agent-session.sh` (commit 32efc13, branch
feat/fleet-launch-path): derive runtime-bin prefix (MOSAIC_RUNTIME_BIN | npm prefix |
~/.npm-global/bin | ~/.local/bin), bake `export PATH=...; exec <cmd>` into the pane;
`exec` also fixes the drift false-positive. Live-tested under stripped PATH -> durable.
- Boot-survival: Jason ran `systemctl --user enable` (+ linger). TODO: auto-enable in
**fleet init** so operators never have to remember it (agentic-enhancement cycle).
- Future custom Pi harness build: pi cannot self-report its model (track
runtime/model/effort as fleet metadata); drift detection should recognize `node` as
pi's pane command (a node-wrapped pane can currently read as drift).
- Findings recorded in AI Guide playbooks/tmux-fleet.md (aiguide PR #7, merged).
- Policy: avoid Claude outside Claude Code (API pricing for alt-harness use) — fleet
runtimes default to Codex / pi-on-Codex; Claude stays in Claude Code only.
docs/archive/work-records/gateway-install-ux-20260404.md
+47
View File
@@ -0,0 +1,47 @@
# Gateway Install UX Fixes — 2026-04-04
## Context
User hit two publish-drift bugs on `mosaic gateway install` (fixed in #386 and
#389). On top of those, the install UX itself has three concrete problems:
1. Admin API token is generated by the bootstrap step and saved to
`~/.config/mosaic/gateway/meta.json`, but never shown to the user. There is
no way to retrieve it without reading the file or running `mosaic gateway
config` afterward.
2. When install crashes mid-way (e.g. daemon fails to become healthy), the
next run of `mosaic gateway install` prompts "Reinstall? [y/N]". Saying N
aborts, leaving the user with a half-configured install and no clear path
forward. There is no resume path.
3. Health-check failure only prints "Check logs: mosaic gateway logs" — forcing
the user to run another command to see the actual error.
## Goal
Make `mosaic gateway install` a single end-to-end, resumable command that shows
the admin token on creation and surfaces errors inline.
## Plan
- `install.ts`: detect partial state (meta exists, no admin token, daemon not
running) and skip already-completed phases instead of aborting.
- `install.ts`: add a prominent "Admin API Token" banner printed right after
bootstrap succeeds. Include copy-now warning.
- `install.ts`: on health-check timeout, read the tail of `LOG_FILE` and print
the last ~30 non-empty lines before returning.
- Keep `mosaic gateway config` as-is (view/edit env vars); this is not the
setup wizard.
- No new commands. No new flags yet.
## Out of scope
- Readline/stdin piping fragility (pre-existing, not related to user complaint).
- Refactor to a phased state machine (overkill for three targeted fixes).
## Verification
- Typecheck, lint, format (mandatory gates).
- Manual end-to-end: fresh install → confirm token displayed.
- Manual resume: delete daemon.pid mid-install → re-run → confirm it resumes.
- Manual failure: point ENV_FILE port to a used port → re-run → confirm log
tail is printed.
docs/archive/work-records/git-wrapper-rollup-20260526.md
+33
View File
@@ -0,0 +1,33 @@
# Git Wrapper Rollup — 2026-05-26
## Objective
Consolidate pending Mosaic wrapper fixes after `mosaic update` reported the local framework package was already current (`@mosaicstack/mosaic 0.0.30`) but the installed `~/.config/mosaic/tools` wrappers still lacked the open Gitea/Woodpecker wrapper patches.
## Scope
Roll up the open wrapper-related Gitea PR branches into one integration branch:
- PR #513: `pr-ci-wait.sh` stdin collision fix.
- PR #518: Gitea PR metadata/merge preflight hardening.
- PR #521: Gitea merge fallback + unsafe PR-number rejection.
- PR #522: Woodpecker credential/pagination fixes and CI Postgres service collision fix.
- PR #523: explicit Gitea repo/login args and `eval` removal for PR/issue creation.
## Conflict resolutions
- Kept array-based command construction where possible instead of reintroducing `eval`.
- Kept explicit `--repo OWNER/REPO --login mosaicstack` Gitea arguments for `tea` calls.
- Combined PR merge API fallback behavior from metadata hardening and empty-identity fallback branches.
- Preserved numeric PR-number validation for `pr-merge.sh`.
## Verification checklist
- `bash -n` on changed shell scripts.
- Wrapper smoke checks from a clean worktree.
- Gitea PR verification after push.
- CI status checked through Gitea/Woodpecker.
## Notes
`mosaic update` did not install these fixes because the package registry still reports `@mosaicstack/mosaic 0.0.30` as current. The source patches must merge/release before normal framework update will carry them.
docs/archive/work-records/h1-heartbeat-readiness.md
+66
View File
@@ -0,0 +1,66 @@
# H1 — heartbeat readiness detection
## Objective
Add runtime-agnostic readiness classification to `mosaic fleet ps` so an agent can be reported as working/idle/stuck/stale/dead/unknown instead of treating pane liveness as progress.
## Scope
- `packages/mosaic/src/commands/fleet.ts`
- exported readiness state/types/default thresholds/helpers/classifier
- `AgentPsRow.readiness` additive JSON field
- table HB column and IDLE/STUCK flags
- `packages/mosaic/src/commands/fleet.spec.ts`
- pure classifier branch/boundary coverage
- threshold helper coverage
- legitimate render/JSON assertion updates for new HB text
## Acceptance Criteria
- Branches covered: dead, unknown, stale, busy working, null-idle working, stuck boundary, idle boundary, working below idle.
- Threshold env helpers default to 300s/900s and honor positive integer env values.
- `fleet ps` rows populate `readiness` for roster and unmanaged socket sessions.
- Table HB text becomes `<age>s/<readiness>` when heartbeat age exists; remains `unknown` when absent.
- Flags include `IDLE`/`STUCK` for matching readiness.
- Local gates green: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, fleet vitest.
- Pre-push queue guard passes; PR opened off `origin/main`; no merge by worker.
## Constraints / Assumptions
- Source branch: `origin/main` @ `e3adc6a`.
- No scope creep beyond readiness detection.
- `docs/TASKS.md` and `docs/fleet/TASKS.md` are orchestrator-owned; worker will not modify them.
- PRD alignment source: `docs/fleet/PRD.md` Phase 2 observability; this is a refinement of heartbeat observability, preserving existing unknown/stale behavior.
## Plan
1. Install dependencies with requested PNPM environment.
2. Add readiness types/helpers/classifier near heartbeat constants.
3. Add `readiness` to `AgentPsRow` and populate both row paths.
4. Update table render and flags.
5. Add unit tests and update affected ps render/JSON assertions.
6. Run build precheck + required gates.
7. Run automated independent review, remediate findings.
8. Queue guard, push, open PR.
## Progress
- 2026-06-24: Branch created from `origin/main` @ `e3adc6a`.
- 2026-06-24: Implemented readiness thresholds/classifier, JSON row field, HB column label, and IDLE/STUCK flags.
- 2026-06-24: Added classifier branch/boundary tests, threshold helper tests, JSON shape assertions, and readiness table rendering assertions.
## Verification Evidence
- `pnpm install --store-dir "$HOME/.pnpm-store"` — pass.
- `npx turbo build --filter=@mosaicstack/mosaic^...` — pass, 12/12 tasks successful.
- `pnpm typecheck` — pass, 41/41 tasks successful.
- `pnpm lint` — pass, 23/23 tasks successful.
- `pnpm format:check` — pass, all matched files use Prettier style.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts` — pass, 171 tests.
- `pnpm --filter @mosaicstack/mosaic test` — pass, 39 files / 547 tests; `fleet.spec.ts` 171 tests.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — approve, 0 findings (reviewed supplied diff; sandbox file-inspection limitation noted by tool).
## Risks / Blockers
- No current blocker.
- Review tool could not inspect repo files directly due sandbox wrapper limitation, but it reviewed the supplied diff and approved with no findings.
docs/archive/work-records/h1b-pane-idle-signal.md
+53
View File
@@ -0,0 +1,53 @@
# H1b — tmux pane idle signal wiring
## Objective
Feed `classifyReadiness()` a real idle signal on tmux 3.4 by deriving `idleSeconds` from the first available tmux timestamp source: pane activity, then window activity, then session activity.
## Scope
- `packages/mosaic/src/commands/fleet.ts`
- Extend `buildTmuxListPanesCommand()` format to include `#{window_activity}` and `#{session_activity}` after the existing fields.
- Update `parseTmuxListPanes()` to choose the first non-empty finite positive timestamp and clamp future idle values to 0.
- `packages/mosaic/src/commands/fleet.spec.ts`
- Cover pane/window/session activity parsing behavior, empty-field index alignment, null idle, future clamping, math correctness, and exact tmux format.
## Out of Scope
- No changes to `classifyReadiness()`, thresholds, `AgentPsRow`, or `fleet ps` rendering.
- No merge by worker; orchestrator routes review/merge.
- Workers do not modify `docs/TASKS.md`.
## PRD Alignment
Aligned with `docs/fleet/PRD.md` FR-1 and acceptance criteria for truthful `mosaic fleet ps` pane/pid/idle observability.
## Plan
1. Sync branch from latest `origin/main` and install dependencies with required pnpm env.
2. Add/confirm reproducer tests for tmux 3.4 empty `pane_activity` and new fallback behavior.
3. Implement the focused parser/format change only.
4. Run required build, baseline gates, fleet vitest, and independent review.
5. Run pre-push queue guard, push branch, and open PR to `main` with Mosaic wrapper.
## Progress
- 2026-06-24: Branch `fix/fleet-pane-idle-activity` created from `origin/main` @ `ec8dd7c` after fetching.
- 2026-06-24: Session-start generated local `.mosaic/orchestrator/*` changes on the previous release branch; stashed as `coder1 session-start state before H1b` to keep this branch clean.
- 2026-06-24: Added TDD coverage for the tmux 3.4 production case (`pane_activity` empty, `window_activity` populated), exact new list-panes format, null/future/multiple-source behavior.
- 2026-06-24: Implemented parser fallback without changing readiness classifier thresholds or render shape.
## Verification Evidence
- `pnpm install --store-dir "$HOME/.pnpm-store"` — pass.
- Reproducer before implementation: `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts` — failed as expected (old format, no fallback, negative future idle).
- `npx turbo build --filter=@mosaicstack/mosaic^...` — pass, 12/12 tasks successful.
- `pnpm typecheck` — pass, 41/41 tasks successful.
- `pnpm lint` — pass, 23/23 tasks successful.
- `pnpm format:check` — pass, all matched files use Prettier style.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts` — pass, 176 tests.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — approve, 0 findings (reviewed supplied diff; sandbox file-inspection limitation noted by tool).
## Risks / Blockers
- No current blocker.
docs/archive/work-records/h2-readiness-available.md
+70
View File
@@ -0,0 +1,70 @@
# H2 — readiness semantics: available, not stuck
## Objective
Correct fleet readiness semantics so a healthy long-idle agent is reported as `available` (good/assignable) instead of `stuck` (fault). Reserve `stuck` in the type/JSON value space for future positive block evidence.
## Scope
- `packages/mosaic/src/commands/fleet.ts`
- replace `idle` readiness state with `available`
- keep `stuck` in the union but stop emitting it from idle-only heuristics
- remove stuck threshold helper/env handling
- remove IDLE/STUCK alarm flags from table rendering
- `packages/mosaic/src/commands/fleet.spec.ts`
- update classifier branch/boundary tests
- assert very long idle maps to `available`, not `stuck`
- update table/JSON assertions for available with no alarm flags
- remove stuck threshold helper tests
## Acceptance Criteria
- `classifyReadiness()` remains pure/total/never-throw and maps:
- dead/stale/unknown unchanged
- busy/null/undefined/non-finite idle to `working`
- idle >= activity threshold to `available`
- idle < activity threshold to `working`
- No idle-derived path emits `stuck`.
- `MOSAIC_HEARTBEAT_IDLE_THRESHOLD` remains backward compatible as the working→available activity threshold.
- `MOSAIC_HEARTBEAT_STUCK_THRESHOLD` and helper/default are removed.
- `fleet ps` keeps the idle-seconds column header `IDLE`, renders `available` in HB label, and does not add IDLE/STUCK warning flags.
- Local gates green: build precheck, typecheck, lint, format:check, fleet vitest.
- PR opened against `main`; no merge by worker.
## Constraints / Assumptions
- Source branch: `origin/main` @ `1020cfa`.
- `docs/TASKS.md` is orchestrator-owned; worker will not modify it.
- Documentation impact is captured in this scratchpad and PR description; no user/admin guide behavior beyond CLI readiness label semantics.
## Plan
1. Install dependencies with requested PNPM environment.
2. Inspect current H1/H1b readiness implementation and tests.
3. Update classifier types/helpers/rendering.
4. Update focused tests.
5. Run build precheck + required gates.
6. Run automated code review, remediate any findings.
7. Queue guard, push, open PR.
## Progress
- 2026-06-24: Branch created from `origin/main` @ `1020cfa`.
- 2026-06-24: Replaced idle-derived `idle`/`stuck` outputs with `available`; retained `stuck` in type union for future positive block evidence.
- 2026-06-24: Removed stuck threshold env/helper plumbing and IDLE/STUCK alarm flags.
- 2026-06-24: Updated classifier and table-render tests for available semantics.
## Verification Evidence
- `pnpm install --store-dir "$HOME/.pnpm-store"` — pass.
- `npx turbo build --filter=@mosaicstack/mosaic^...` — pass, 12/12 tasks successful.
- `pnpm typecheck` — pass, 41/41 tasks successful.
- `pnpm lint` — pass, 23/23 tasks successful.
- `pnpm format:check` — pass, all matched files use Prettier style.
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts` — pass, 177 tests.
- `~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted` — approve, 0 findings (reviewed supplied diff; sandbox file-inspection limitation noted by tool).
## Risks / Blockers
- No current blocker.
- Review tool could not inspect repo files directly due sandbox wrapper limitation, but it reviewed the supplied diff and approved with no findings.
docs/archive/work-records/install-ux-hardening-20260405.md
+330
View File
@@ -0,0 +1,330 @@
# Install UX Hardening — IUH-M01 Session Notes
## Session: 2026-04-05 (agent-ad6b6696)
### Plan
**Manifest schema decision:**
- Version 1 JSON at `~/.config/mosaic/.install-manifest.json` (mode 0600)
- Written by `tools/install.sh` after successful install
- Fields: version, installedAt, cliVersion, frameworkVersion, mutations{directories, npmGlobalPackages, npmrcLines, shellProfileEdits, runtimeAssetCopies}
- Uninstall reads it; if missing → heuristic mode (warn user)
**File list:**
- NEW: `packages/mosaic/src/runtime/install-manifest.ts` — read/write helpers + types
- NEW: `packages/mosaic/src/runtime/install-manifest.spec.ts` — unit tests
- NEW: `packages/mosaic/src/commands/uninstall.ts` — command implementation
- NEW: `packages/mosaic/src/commands/uninstall.spec.ts` — unit tests
- MOD: `packages/mosaic/src/cli.ts` — register `uninstall` command
- MOD: `tools/install.sh` — write manifest on success + add `--uninstall` path
**Runtime asset list (from mosaic-link-runtime-assets / framework/install.sh):**
- `~/.claude/CLAUDE.md` (source: `$MOSAIC_HOME/runtime/claude/CLAUDE.md`)
- `~/.claude/settings.json` (source: `$MOSAIC_HOME/runtime/claude/settings.json`)
- `~/.claude/hooks-config.json` (source: `$MOSAIC_HOME/runtime/claude/hooks-config.json`)
- `~/.claude/context7-integration.md` (source: `$MOSAIC_HOME/runtime/claude/context7-integration.md`)
- `~/.config/opencode/AGENTS.md` (source: `$MOSAIC_HOME/runtime/opencode/AGENTS.md`)
- `~/.codex/instructions.md` (source: `$MOSAIC_HOME/runtime/codex/instructions.md`)
**Reversal logic:**
1. If `.mosaic-bak-<stamp>` exists for a file → restore it
2. Else if managed copy exists → remove it
3. Never touch files not in the known list
**npmrc reversal:**
- Only remove line `@mosaicstack:registry=https://git.mosaicstack.dev/api/packages/mosaicstack/npm/`
- If manifest has the line, use that as authoritative; else check heuristically
**PATH reversal:**
- Check install.sh: it does NOT add PATH entries to shell profiles (framework/install.sh migration removes old `$MOSAIC_HOME/bin` PATH entries in v0/v1→v2 migration, but new install does NOT add PATH)
- ASSUMPTION: No PATH edits in current install (v0.0.24+). Shell profiles not modified by current install.
- The `$PREFIX/bin` is mentioned in a warning but NOT added to shell profiles by install.sh.
- shellProfileEdits array will be empty for new installs; heuristic mode also skips it.
**Test strategy:**
- Unit test manifest read/write with temp dir mocking
- Unit test command registration
- Unit test dry-run flag (no actual fs mutations)
- Unit test --keep-data skips protected paths
- Unit test heuristic mode warning
**Implementation order:**
1. install-manifest.ts helpers
2. install-manifest.spec.ts tests
3. uninstall.ts command
4. uninstall.spec.ts tests
5. cli.ts registration
6. tools/install.sh manifest writing + --uninstall path
ASSUMPTION: No PATH modifications in current install.sh (v0.0.24). Framework v0/v1→v2 migration cleaned old PATH entries but current install does not add new ones.
ASSUMPTION: `--uninstall` in install.sh handles framework + cli + npmrc only; gateway teardown deferred to `mosaic gateway uninstall`.
ASSUMPTION: Pi settings.json edits (skills paths) added by framework/install.sh are NOT reversed in this iteration — too risky to touch user Pi config without manifest evidence. Noted as follow-up.
---
## Session 2 — 2026-04-05 (orchestrator resume)
### IUH-M01 completion summary
- **PR:** #429 merged as `25cada77`
- **CI:** green (Woodpecker)
- **Issue:** #425 closed
- **Files:** +1205 lines across 4 new + 2 modified + 1 docs
- **Tests:** 14 new, 170 total passing
### Follow-ups captured from worker report
1. **Pi settings.json reversal deferred** — worker flagged as too risky without manifest evidence. Future IUH task should add manifest entries for Pi settings mutations. Not blocking M02/M03.
2. **Pre-existing `cli-smoke.spec.ts` failure** — `@mosaicstack/brain` package entry resolution fails in Vitest. Unrelated to IUH-M01. Worth a separate issue later.
3. **`pr-create.sh` wrapper bug with multiline bodies** — wrapper evals body args as shell when they contain newlines/paths. Worker fell back to Gitea REST API. Same class of bug I hit earlier with `issue-create.sh`. Worth a tooling-team issue to fix both wrappers.
### Mission doc sync
cli-unification docs that were archived before the M01 subagent ran did not travel into the M01 PR (they were local, stashed before pull). Re-applying now:
- `docs/archive/missions/cli-unification-20260404/` (the old manifest + tasks)
- `docs/MISSION-MANIFEST.md` (new install-ux-hardening content)
- `docs/TASKS.md` (new install-ux-hardening content)
Committing as `docs: scaffold install-ux-hardening mission + archive cli-unification`.
### Next action
Delegate IUH-M02 to a sonnet subagent in an isolated worktree.
---
## Session 3: 2026-04-05 (agent-a6ff34a5) — IUH-M02 Wizard Remediation
### Plan
**AC-3: Password masking + confirmation**
- New `packages/mosaic/src/prompter/masked-prompt.ts` — raw-mode stdin reader that suppresses echo, handles backspace/Ctrl+C/Enter.
- `bootstrapFirstUser` in `packages/mosaic/src/commands/gateway/install.ts`: replace `rl.question('Admin password...')` with `promptMaskedPassword()`, require confirm pass, keep min-8 validation.
- Headless path: when `MOSAIC_ASSUME_YES=1` or `!process.stdin.isTTY`, read `MOSAIC_ADMIN_PASSWORD` env var directly.
**AC-4a: Hooks preview stage**
- New `packages/mosaic/src/stages/hooks-preview.ts` — reads `hooks-config.json` from `state.sourceDir` or `state.mosaicHome`, displays each top-level hook category with name/trigger/command preview, prompts "Install these hooks? [Y/n]", stores result in `state.hooks`.
- `packages/mosaic/src/types.ts` — add `hooks?: { accepted: boolean; acceptedAt?: string }` to `WizardState`.
- `packages/mosaic/src/wizard.ts` — insert `hooksPreviewStage` between `runtimeSetupStage` and `skillsSelectStage`; skip if no claude runtime detected.
**AC-4b: `mosaic config hooks` subcommands**
- Add `hooks` subcommand group to `packages/mosaic/src/commands/config.ts`:
- `list`: reads `~/.claude/hooks-config.json`, shows hook names and enabled/disabled status
- `disable <name>`: prefixes matching hook key with `_disabled_` in the JSON
- `enable <name>`: removes `_disabled_` prefix if present
**AC-5: Headless install path**
- `runConfigWizard`: detect headless mode (`MOSAIC_ASSUME_YES=1` or `!process.stdin.isTTY`), read env vars with defaults, validate required vars, skip prompts entirely.
- `bootstrapFirstUser`: detect headless mode, read `MOSAIC_ADMIN_NAME/EMAIL/PASSWORD`, validate, proceed without prompts.
- Document env vars in `packages/mosaic/README.md` (create if absent).
### File list
NEW:
- `packages/mosaic/src/prompter/masked-prompt.ts`
- `packages/mosaic/src/prompter/masked-prompt.spec.ts`
- `packages/mosaic/src/stages/hooks-preview.ts`
- `packages/mosaic/src/stages/hooks-preview.spec.ts`
MODIFIED:
- `packages/mosaic/src/types.ts` — extend WizardState
- `packages/mosaic/src/wizard.ts` — wire hooksPreviewStage
- `packages/mosaic/src/commands/gateway/install.ts` — masked password + headless path
- `packages/mosaic/src/commands/config.ts` — add hooks subcommands
- `packages/mosaic/src/commands/config.spec.ts` — extend tests
- `packages/mosaic/README.md` — document env vars
### Assumptions
ASSUMPTION: `hooks-config.json` location is `<sourceDir>/framework/runtime/claude/hooks-config.json` during wizard (sourceDir is package root). Fall back to `<mosaicHome>/runtime/claude/hooks-config.json` for installed config.
ASSUMPTION: The `hooks` subcommands under `config` operate on `~/.claude/hooks-config.json` (the installed copy), not the package source.
ASSUMPTION: For the hooks preview stage, the "name" field displayed per hook entry is the top-level event key (e.g. "PostToolUse") plus the matcher from nested hooks array. This is the most user-readable representation given the hooks-config.json structure.
ASSUMPTION: `config hooks list/enable/disable` use `CLAUDE_HOME` env or `~/.claude` as the target directory for hooks files.
ASSUMPTION: The headless TTY detection (`!process.stdin.isTTY`) is sufficient; `MOSAIC_ASSUME_YES=1` is an explicit override for cases where stdin is a TTY but the user still wants non-interactive (e.g., scripted installs with piped terminal).
---
## Session 4 — 2026-04-05 (orchestrator resume) — IUH-M02 closed, delegating IUH-M03
### IUH-M02 completion summary
- **PR:** #431 merged as `cd8b1f66`
- **CI:** green (Woodpecker)
- **Issue:** #426 closed
- **Acceptance criteria:** AC-3 (password mask), AC-4 (hooks visibility — consent recorded), AC-5 (headless path) all satisfied
- **New files:** `prompter/masked-prompt.ts`, `stages/hooks-preview.ts` (+ specs)
- **Modified:** `wizard.ts`, `types.ts` (`state.hooks`), `commands/gateway/install.ts`, `commands/config.ts`
### Follow-up captured from M02 agent
**Hooks consent is recorded but not enforced.** The `hooks-preview` stage sets `state.hooks.accepted` when the user confirms, but the finalize stage still unconditionally runs `mosaic-link-runtime-assets`, which copies `hooks-config.json` into `~/.claude/` regardless of consent. This is a soft gap — the user sees the prompt and can decline, but declining currently has no effect downstream.
Options for addressing:
- Fold into IUH-M03 (since M03 touches the finalize/install convergence path anyway)
- Spin a separate small follow-up issue after M03 lands
Leaning toward folding into M03 — the unified first-run flow naturally reworks the finalize→gateway handoff where this gating belongs.
### IUH-M03 delegation
Now delegating to an **opus** subagent in an isolated worktree. Scope from `/tmp/iuh-m03-body.md`:
- Extract `runConfigWizard` → `stages/gateway-config.ts`
- Extract `bootstrapFirstUser` → `stages/gateway-bootstrap.ts`
- `runWizard` invokes gateway stages as final stages
- Drop the 10-minute `$XDG_RUNTIME_DIR/mosaic-install-state.json` session bridge
- `mosaic gateway install` becomes a thin standalone wrapper for backward-compat
- `tools/install.sh` single auto-launch entry point
- **Bonus if scoped:** honor `state.hooks.accepted` in finalize stage so declining hooks actually skips hook install
Known tooling caveats to pass to worker:
- `issue-create.sh` / `pr-create.sh` wrappers eval multiline bodies as shell — use Gitea REST API fallback with `load_credentials gitea-mosaicstack`
- Protected `main`: PR-only, squash merge
- Must run `ci-queue-wait.sh --purpose push|merge` before push/merge
---
## Session 5: 2026-04-05 (agent-a7875fbd) — IUH-M03 Unified First-Run
### Problem recap
`mosaic wizard` and `mosaic gateway install` currently run as two separate phases bridged by a fragile 10-minute session file at `$XDG_RUNTIME_DIR/mosaic-install-state.json`. `tools/install.sh` auto-launches both sequentially so the user perceives two wizards stitched together; state is not shared, prompts are duplicated, and if the user walks away the bridge expires.
### Design decision — Option A: gateway install becomes terminal stages of `runWizard`
Two options on the table:
- (A) Extract `runConfigWizard` and `bootstrapFirstUser` into `stages/gateway-config.ts` and `stages/gateway-bootstrap.ts`, append them to `runWizard` as final stages, and make `mosaic gateway install` a thin wrapper that runs the same stages with an ephemeral state seeded from existing config.
- (B) Introduce a new top-level orchestrator that composes the wizard and gateway install as siblings.
**Chosen: Option A.** Rationale:
1. The wizard already owns a `WizardState` that threads state across stages — gateway config/bootstrap fit naturally as additional stages without a new orchestration layer.
2. `mosaic gateway install` as standalone entry point stays idempotent by seeding a minimal `WizardState` and running only the gateway stages, reusing the same functions.
3. Avoids a parallel state object and keeps the call graph linear; easier to test and to reason about the "one cohesive flow" UX goal.
4. Option B would leave `runWizard` and the gateway install as siblings that still need to share a state object — equivalent complexity without the narrative simplification.
### Scope
1. Extend `WizardState` with optional `gateway` slice: `{ tier, port, databaseUrl?, valkeyUrl?, anthropicKey?, corsOrigin, admin?: { name, email, password } }`. The admin password is held in memory only — never persisted to disk as part of the state object.
2. New `packages/mosaic/src/stages/gateway-config.ts` — pure stage that:
- Reads existing `.env`/`mosaic.config.json` if present (resume path) and sets state.
- Otherwise prompts via `WizardPrompter` (interactive) or reads env vars (headless).
- Writes `.env` and `mosaic.config.json`, starts the daemon, waits for health.
3. New `packages/mosaic/src/stages/gateway-bootstrap.ts` — pure stage that:
- Checks `/api/bootstrap/status`.
- If needsSetup, prompts for admin name/email/password (uses `promptMaskedConfirmed`) or reads env vars (headless); calls `/api/bootstrap/setup`; persists token in meta.
- If already setup, handles inline token recovery exactly as today.
4. `packages/mosaic/src/wizard.ts` — append gateway-config and gateway-bootstrap as stages 11 and 12. Remove `writeInstallState` and the `INSTALL_STATE_FILE` constant entirely.
5. `packages/mosaic/src/commands/gateway/install.ts` — becomes a thin wrapper that builds a minimal `WizardState` with a `ClackPrompter`, then calls `runGatewayConfigStage(...)` and `runGatewayBootstrapStage(...)` directly. Remove the session-file readers/writers. Headless detection is delegated to the stage itself. The wrapper still exposes the `runInstall({host, port, skipInstall})` API so `gateway.ts` command registration is unchanged.
6. `tools/install.sh` — drop the second `mosaic gateway install` call; `mosaic wizard` now covers end-to-end. Leave `gateway install` guidance for non-auto-launch path so users still know the standalone entry point exists.
7. **Hooks gating (bonus — folded in):** `finalize.ts` already runs `mosaic-link-runtime-assets`. When `state.hooks?.accepted === false`, set `MOSAIC_SKIP_CLAUDE_HOOKS=1` in the env for the subprocess; teach the script to skip copying `hooks-config.json` when that env var is set. Other runtime assets (CLAUDE.md, settings.json, context7) still get linked.
### Files
NEW:
- `packages/mosaic/src/stages/gateway-config.ts` (+ `.spec.ts`)
- `packages/mosaic/src/stages/gateway-bootstrap.ts` (+ `.spec.ts`)
MODIFIED:
- `packages/mosaic/src/types.ts` — extend WizardState with `gateway?:` slice
- `packages/mosaic/src/wizard.ts` — append gateway stages, remove session-file bridge
- `packages/mosaic/src/commands/gateway/install.ts` — thin wrapper over stages, remove 10-min bridge
- `packages/mosaic/src/stages/finalize.ts` — honor `state.hooks.accepted === false` by setting `MOSAIC_SKIP_CLAUDE_HOOKS=1`
- `packages/mosaic/framework/tools/_scripts/mosaic-link-runtime-assets` — honor `MOSAIC_SKIP_CLAUDE_HOOKS=1`
- `tools/install.sh` — single unified auto-launch
### Assumptions
ASSUMPTION: Gateway stages must run **after** `finalizeStage` because finalize writes identity files and links runtime assets that the gateway admin UX may later display — reversed ordering would leave Claude runtime linkage incomplete when the admin token banner prints.
ASSUMPTION: Standalone `mosaic gateway install` uses a `ClackPrompter` (interactive) by default; the headless path is still triggered by `MOSAIC_ASSUME_YES=1` or non-TTY stdin, and the stage functions detect this internally.
ASSUMPTION: When `runWizard` reaches the gateway stages, `state.mosaicHome` is authoritative for GATEWAY_HOME resolution if it differs from the default — we set `process.env.MOSAIC_GATEWAY_HOME` before importing gateway modules so the constants resolve correctly.
ASSUMPTION: Keeping backwards compatibility for `runInstall({host, port, skipInstall})` is enough — no other internal caller exists.
ASSUMPTION: Removing the session file is safe because the old bridge is at most a 10-minute window; there is no on-disk migration to do.
### Test plan
- `gateway-config.spec.ts`: fresh install writes .env + mosaic.config.json (mock fs + prompter); resume path reuses existing BETTER_AUTH_SECRET; headless path respects MOSAIC_STORAGE_TIER/MOSAIC_GATEWAY_PORT/etc.
- `gateway-bootstrap.spec.ts`: calls `/api/bootstrap/setup` with collected creds (mock fetch); handles "already setup" branch; honors headless env vars; persists token via `writeMeta`.
- Extend existing passing tests — no regressions in `login.spec`, `recover-token.spec`, `rotate-token.spec`.
- Unified flow integration is covered at the stage-level; no new e2e test infra required.
### Delivery cycle
plan (this entry) → code → typecheck/lint/format → test → codex review (`~/.config/mosaic/tools/codex/codex-code-review.sh --uncommitted`) → remediate → commit → ci-queue-wait push → push → PR → CI green → merge → close #427.
### Remediation log (codex review rounds)
- **Round 1** — hooks opt-out did not remove an existing managed file; port override ignored on resume; headless errors swallowed. Fixed: hooks cleanup, `portOverride` honored, errors re-thrown.
- **Round 2** — headless stage failures exited 0; port override on decline-rerun mismatched; no default-path integration test. Fixed: `process.exit(1)` in headless, revert portOverride on decline, add `unified-wizard.test.ts`.
- **Round 3** — hooks removal too broad (would touch user-owned files); port override written to meta but not .env (drift); wizard swallowed errors. Fixed: `cmp -s` managed-file check, force regeneration when portOverride differs from saved port, re-throw unexpected errors.
- **Round 4** — port-override regeneration tripped the corrupt-partial-state guard (blocker); headless already-bootstrapped-with-no-local-token path reported failure instead of no-op; hooks byte-equality fragile across template updates. Fixed: introduce `forcePortRegen` flag bypassing the guard (with a dedicated spec test), headless rerun of already-bootstrapped gateway now returns `{ completed: true }` (with spec coverage), hooks cleanup now checks for a stable `"mosaic-managed": true` marker embedded in the template (byte-equality remains as a fallback for legacy installs).
- Round 5 codex review attempted but blocked by upstream usage limit (quota). Rerun after quota refresh if further findings appear; all round-4 findings are code-covered.
---
## Session 6 — 2026-04-05 (orchestrator close-out) — MISSION COMPLETE
### IUH-M03 completion summary (reported by opus delivery agent)
- **PR:** #433 merged as `732f8a49`
- **CI:** Woodpecker green on final rebased commit `f3d5ef8d`
- **Issue:** #427 closed with summary comment
- **Tests:** 219 passing (+15 net new), 24 files
- **Codex review:** 4 rounds applied and remediated; round 5 blocked by upstream quota — no known outstanding findings
### What shipped in M03
- NEW stages: `stages/gateway-config.ts`, `stages/gateway-bootstrap.ts` (extracted from the old monolithic `gateway/install.ts`)
- NEW integration test: `__tests__/integration/unified-wizard.test.ts`
- `runWizard` now has 12 stages — gateway config + bootstrap are terminal stages 11 & 12
- 10-minute `$XDG_RUNTIME_DIR/mosaic-install-state.json` session-file bridge **deleted**
- `mosaic gateway install` rewritten as a thin standalone wrapper invoking the same two stages — backward-compat preserved
- `WizardState.gateway?` slice carries host/port/tier/admin/adminTokenIssued across stages
- `tools/install.sh` single unified `mosaic wizard` call — no more two-phase launch
- **Bonus scoped in:** finalize stage honors `state.hooks.accepted === false` via `MOSAIC_SKIP_CLAUDE_HOOKS=1`; `mosaic-link-runtime-assets` honors the flag; Mosaic-managed detection now uses a stable `"mosaic-managed": true` marker in `hooks-config.json` with byte-equality fallback for legacy installs. **Closes the M02 follow-up.**
### Mission status — ALL DONE
| AC | Status | PR |
| ---- | ------ | ---------------------------------------------------- |
| AC-1 | ✓ | #429 |
| AC-2 | ✓ | #429 |
| AC-3 | ✓ | #431 |
| AC-4 | ✓ | #431 + #433 (gating) |
| AC-5 | ✓ | #431 |
| AC-6 | ✓ | #433 |
| AC-7 | ✓ | #429, #431, #433 all merged, CI green, issues closed |
### Follow-ups for future work (not blocking mission close)
1. **`pr-ci-wait.sh` vs Woodpecker**: wrapper reports `state=unknown` because Woodpecker doesn't publish to Gitea's combined-status endpoint. Worker used `tea pr` CI glyphs as authoritative. Pre-existing tooling gap — worth a separate tooling-team issue.
2. **`issue-create.sh` / `pr-create.sh` wrapper `eval` bug with multiline bodies** — hit by M01, M02, M03 workers. All fell back to Gitea REST API. Needs wrapper fix.
3. **Codex review round 5** — attempted but blocked by upstream quota. Rerun after quota resets to confirm nothing else surfaces.
4. **Pi settings.json reversal** — deferred from M01; install manifest schema should be extended to track Pi settings mutations for reversal.
5. **`cli-smoke.spec.ts` pre-existing failure** — `@mosaicstack/brain` resolution in Vitest. Unrelated. Worth a separate issue.
### Next steps (orchestrator)
1. This scratchpad + MISSION-MANIFEST.md + TASKS.md updates → final docs PR
2. After merge: create release tag per framework rule (milestone/mission completion = release tag + repository release)
3. Archive mission docs under `docs/archive/missions/install-ux-hardening-20260405/` once the tag is published
docs/archive/work-records/installer-next-fast-npm-20260625.md
+40
View File
@@ -0,0 +1,40 @@
# Installer `--next` fast npm lane — 2026-06-25
## Scope
Flip `tools/install.sh --next` from source-build-first to fast npm `@next` first, with source fallback.
## Registry reality check
Gitea npm registry: `https://git.mosaicstack.dev/api/packages/mosaicstack/npm/`
Verified before implementation:
- `@mosaicstack/mosaic@next` resolves to `0.0.49-next.1633`.
- `@mosaicstack/gateway@next` resolves to `0.0.7-next.1633`.
- `@mosaicstack/gateway` dist-tags include `latest: 0.0.6` and `next: 0.0.7-next.1633`.
- `apps/gateway/package.json` is non-private and has Gitea npm `publishConfig`.
Conclusion: the installer can fast-install both CLI and gateway npm packages for `--next`. The gateway Docker `gateway:sha-<short>` remains the deployment/harness artifact; the npm gateway package is valid for the installer global package path.
## Behavior
- `--next` with no explicit ref:
1. framework archive from `next`;
2. resolve `@mosaicstack/gateway@next` and `@mosaicstack/mosaic@next`;
3. require both resolved versions to share the same `next.<pipeline>` suffix;
4. install the exact resolved package versions;
5. set `MOSAIC_GATEWAY_SKIP_NPM_INSTALL=1` so wizard does not overwrite the prerelease gateway;
6. if either package is missing/unreachable/mismatched/fails, fall back to existing source build at `next`.
- `--dev` remains pure source build.
- explicit `--ref` / `MOSAIC_REF` still wins over `--next` and uses the source path for that exact ref.
## Install detail
The installer writes the scoped npmrc mapping (`@mosaicstack:registry=...`) and then runs npm install without overriding npm's default registry. Passing `--registry=<gitea>` to `npm install` forces public transitive dependencies (for example `@anthropic-ai/sdk`) to resolve from Gitea and breaks the fast path; the scoped npmrc mapping is the correct split-registry behavior.
## Verification notes
- Added `tools/install-next-lane.test.sh` with a fake npm/source harness for exact-version fast install, registry failure source fallback, explicit-ref precedence, and mismatched suffix warning.
- Wired the installer harness into `pnpm test` via `pnpm run test:installer`.
- Real temp-prefix fast install succeeded with `@mosaicstack/[email protected]` and `@mosaicstack/[email protected]`.
docs/archive/work-records/installer-next-lane-20260624.md
+35
View File
@@ -0,0 +1,35 @@
# Scratchpad — installer `--next` lane
## Objective
Add a prerelease installer lane for the permanent `next` integration branch.
## Scope
- `tools/install.sh`
- README/install documentation
- Follow-up design note for future npm `@next` prerelease publishing
## Plan
1. Add `--next` and `MOSAIC_NEXT=1` as source-build shorthand for `next`.
2. Preserve explicit ref precedence: `MOSAIC_REF` and `--ref` win over `--next`.
3. Update installer source display/help text.
4. Document three lanes:
- stable npm `@latest`
- prerelease `--next`
- contributor `--dev --ref X`
5. Run shell and repo gates locally, then hold before push/PR until runner serialization greenlight.
## Verification
- `bash -n tools/install.sh` — pass.
- `docker run --rm -v "$PWD:/mnt" -w /mnt koalaman/shellcheck:stable tools/install.sh` — pass.
- `bash tools/install.sh --check --framework --next` — source display shows `ref: next, --next prerelease lane`.
- `bash tools/install.sh --check --cli --next --ref feature-x` — source display shows explicit ref wins.
- `MOSAIC_NEXT=1 MOSAIC_REF=feature-env bash tools/install.sh --check --cli` — source display shows explicit env ref wins.
- `pnpm install --frozen-lockfile --prefer-offline --store-dir /home/jarvis/.local/share/pnpm/store` — pass (local override for repo `.npmrc` CI store path).
- `pnpm typecheck` — pass (41 successful tasks).
- `pnpm lint` — pass (23 successful tasks).
- `pnpm format:check` — pass.
- `bash tools/e2e-install-test.sh` — attempted; current baseline fails during gateway health after stable registry install because Valkey is unavailable in the clean container. The `tools/install.sh --yes --no-auto-launch` stage itself completed before the downstream gateway verification failure.
docs/archive/work-records/issue-766-exact-fleet-comms.md
+80
View File
@@ -0,0 +1,80 @@
# Issue 766 — exact cross-harness fleet comms targeting
- **Issue:** #766
- **Branch:** `fix/766-exact-fleet-comms`
- **Worktree:** `/home/jarvis/src/stack-issue-766`
- **Delivery boundary:** source/tests/docs only; no live tmux, session, or fleet actions; leave uncommitted for independent review.
## Objective
Replace inference-prone fleet onboarding guidance with one roster-resolved contract that gives Claude Code, Codex, OpenCode, and Pi the same authoritative local identity and exact executable command for every known peer.
## Plan
1. Add issue-specific normative requirements to `docs/PRD.md` before source changes; do not modify orchestrator-owned `docs/TASKS.md`.
2. Extract the existing v1 roster parsing/normalization into one lightweight shared resolver used by both fleet commands and runtime comms composition.
3. Write failing contract tests for explicit SSH-only cross-host targeting, global/default socket authority, authoritative identity, unknown-peer failure, no operational metavariables, and four-harness parity.
4. Make source `TOOLS.md` non-operational and marker-versioned; prove fresh installation preserves that exact contract and composition detects a stale installed copy without rewriting it.
5. Render a deterministic comms generation and document comparison/relaunch handling; never rewrite an active session.
6. Run focused Vitest and shell exact-target tests, then package/repository typecheck, lint, format, test, and build gates as relevant.
7. Reconstruct the exact uncommitted tree, including untracked files, for independent review and remediate findings without committing.
## Contract decisions
- `tmux.socket_name` is the one supported socket authority for every local fleet session. A per-agent `socket`, when present for compatibility, must equal that global value; independent per-agent sockets fail closed because the runtime does not provision them. A named socket renders `-L`, while the empty literal default renders no `-L`.
- A peer is same-host only when its resolved host equals the current roster member's resolved host. Every host-omitted member resolves against the stable local fleet-host baseline, never against the viewer's explicit host. Same-host rows never render `-H`.
- A cross-host row requires that peer's explicit roster `ssh`; absence is a contract error. Never substitute `host` as an SSH target.
- The current member's explicit roster `host` wins; otherwise the local machine's short hostname is the baseline for host-omitted local members.
- Unknown members/peers return a deterministic error listing exact known names and an exact self-scoped discovery command. No fuzzy session lookup.
- Exact command fields are structurally constrained to safe targeting grammars and shell-rendered as individual arguments. Unsafe host/SSH/socket values fail roster normalization rather than entering executable guidance.
- Existing installed `TOOLS.md` remains user-owned during ordinary keep-mode updates. Currency requires the expected source and installed marker/version plus bounded SHA-256 byte identity. Explicit `mosaic update --repair-tools` is the supported current-version recovery path: it makes a digest-qualified no-clobber backup, restores the contract and regular executable helper, and does not rewrite active context.
- The v1 resolver preserves and validates `tmux`, `discord`, and `matrix` connector blocks in YAML and JSON; conflicting snake/camel aliases fail closed unless their values are identical.
- JSON roster fallback occurs only when `roster.yaml` is absent. Keep-mode reseed preserves both formats, and relaunch discovery uses the same canonical resolver.
- The helper is inspected without following symlinks and must be a regular executable file. Missing, directory, symlink, and non-executable installations fail closed with deterministic repair guidance.
- Active contexts carry a deterministic comms generation. Operators compare it to `mosaic agent comms-block <exact-agent>` output; mismatch means stale and requires an explicit exact-agent relaunch.
## Risks
- Import cycles if runtime composition imports the command-heavy `fleet.ts`; mitigate with a lightweight shared roster module and re-export compatibility.
- Existing schema prose allowed independent per-agent sockets even though runtime provisioning used one global socket; constrain compatibility declarations to the global value and preserve empty-global default behavior.
- Remote inventory may be incomplete. Fail composition closed for an unreachable cross-host row rather than generating a guessed command.
- `TOOLS.md` is user-seeded and intentionally preserved. Detect/report drift instead of overwriting custom content.
## Planned evidence
- `comms-onboarding.spec.ts`: resolver/renderer/failure/generation contracts.
- `compose-contract.spec.ts`: identical authoritative comms section for all four harnesses and stale installed-contract reporting without mutation.
- `file-adapter.test.ts`: source-to-fresh-install byte equality and preservation of customized installed `TOOLS.md`.
- Existing `agent-send.test.sh`, socket isolation, and tmux runtime transport tests.
- Repository quality gates and independent uncommitted-tree review.
## Evidence log
- Preflight collision scan: no issue-766 local/remote branch, worktree, or open PR collision before branch creation.
- Isolated branch created from fetched `origin/main` at `4990905`; original checkout not edited.
- One strict v1 resolver now serves fleet commands and communications composition; roster writes preserve `host`, `ssh`, and `socket`.
- Exact renderer covers authoritative self identity, global/default socket authority, rejected independent sockets, stable hostless-peer resolution, same-host omission of `-H`, explicit-SSH-only cross-host rows, shell-safe argv rendering, deterministic generations, and fail-closed unknown/missing targets.
- Real framework `defaults/TOOLS.md` is tested byte-equal through a fresh `FileConfigAdapter` install, the installed helper is executable, and the final Pi contract contains the same source contract plus exact generated command; separate parity coverage proves byte-equivalent comms sections for Claude Code, Codex, OpenCode, and Pi.
- Second review remediation adds strict connector/alias coverage, full-semantic generation coverage, ENOENT-only fallback, no-follow helper validation, unconditional current-version repair, digest-qualified no-clobber backups, and marker/version-gated currency.
- The helper, roster, installed TOOLS, and framework source files are read with canonical containment, every existing ancestor and target rejected if symlinked, `O_NOFOLLOW` descriptor reads, inode stability checks, and effective-identity execute access. Read-only TOOLS status treats source/installed symlinks as unavailable without following or rewriting them.
- Explicit repair validates both bundled inputs before destination creation, stages backup/TOOLS/helper plus exact-mode rollback files before any persistent file commit, revalidates destination identity at each commit boundary, installs the digest backup without clobber, and removes or exactly rolls back every committed output on injected failure. `changed: false` is returned only after full cleanup; cleanup/rollback failure is reported as `changed: true`.
- Connector schema and runtime normalization require kind-matching settings and reject inactive connector blocks. Keep-mode installers preserve only exact `roster.yaml`, `roster.json`, `agents/`, and `run/` paths while refreshing framework `roster.schema.json`; shell evidence covers byte preservation and schema refresh.
- Solo contracts render normalized role/class plus explicit no-peer/no-remote authority boundaries; composed-contract evidence keeps role Mandate/Boundaries before Fleet Comms.
- Operational documentation and CLI metavariable now use `mosaic agent comms-block <exact-member>`; historical issue-633 scratchpad text remains historical.
- The latest independent review rejected synthetic tree `556ae4ea04f2715a4e9d381f3cafaf4c8b991b2e` on three mandatory findings: installed `TOOLS.md` could be read through target/ancestor symlinks before unsafe status was reported; ambient class/tool-policy state could split identity authority from the canonical roster member; and the connector schema admitted empty or whitespace-only Discord/Matrix strings rejected by runtime parsing.
- Red-first reproduction proved all three findings with 20 failures and 79 passes. Remediation routes installed `TOOLS.md` through the bounded secure regular-file reader before composition, resolves one exact canonical fleet identity for persona/tool policy/normalized class/Fleet Comms, rejects canonicalized ambient class mismatches, canonicalizes compatibility classes during roster parsing, and aligns parser/schema non-whitespace requirements.
- Four-runtime coverage proves unsafe target and ancestor symlink content is omitted without mutation, while Claude Code, Codex, OpenCode, and Pi all project the same canonical member authority. Connector parser/schema coverage includes empty and whitespace-only Discord `channel_id` and Matrix `homeserver_url`, `user_id`, and `room_id` values.
- Remediated focused gates passed: 99/99 across the two finding-focused suites plus connector schema regression PASS; the six changed-suite matrix passed 341/341; secure-file/transaction coverage remains green, including 28/28 transactional repair tests; installer migration passed 21/21.
- Mosaic package suite passed 906/906. Shell/runtime regressions passed: `agent-send.test.sh` `PASS=11 FAIL=0`; named-socket isolation; matrix/tmux transport 12/12 (Matrix 5/5, tmux 7/7).
- Final repository gates passed: format check; typecheck 42/42 tasks; lint 23/23; tests 42/42 tasks (Mosaic 906/906, gateway 628 passed/12 skipped); build 23/23.
- A subsequent immutable review of tree `aa6414123643a504145fce6ac1d66f0b535feb5e` found one roster-authority blocker: a canonical member with omitted `tool_policy` inherited ambient `MOSAIC_AGENT_TOOL_POLICY`. Red-first four-runtime coverage failed 4/39 specifically on the leaked operator-interaction policy. Composition now branches on canonical membership: fleet launches use only `canonicalMember.toolPolicy` (including canonical absence), while genuinely non-fleet launches retain ambient fallback.
- Final remediated gates passed: four-runtime regression 39/39; six changed-suite matrix 345/345; connector schema regression PASS; Mosaic package 910/910; installer migration 21/21; `agent-send.test.sh` 11/11; named-socket isolation PASS; Matrix/tmux transport 12/12; repository format PASS; typecheck 42/42 tasks; lint 23/23 tasks; tests 42/42 tasks; build 23/23 tasks.
- No live tmux/session/fleet mutation, commit, push, PR mutation, issue mutation, context mutation, or reviewer launch performed.
- Exact synthetic-tree reconstruction and frozen evidence are included in the coordinator handoff.
- Sole-remediation preflight reverified the clean committed checkout at head `0dc47cac92c93a3ffd39ba9dd6685ac4165f6361`, tree `538de6ccce1f8c44ba288a7493286e63a3413e75`, branch `fix/766-exact-fleet-comms`; issue and PR state were read only through Mosaic wrappers.
- Deterministic red-first ancestor substitution swapped validated `root/tools` for an external symlink immediately after `lstat`; current head returned `external marker` (`1 failed, 4 passed`) before implementation.
- Secure reads now hold `/` and every root/descendant directory descriptor, traverse appended components through Linux `/proc/self/fd` with `O_DIRECTORY|O_NOFOLLOW`, and read plus effective-identity execute-check the same final descriptor. Non-Linux or unavailable proc-fd capability fails closed; stable errors redact managed paths while retaining Node `code` compatibility for missing/non-executable repair behavior.
- Added deterministic root-selection, descendant-ancestor, and final-target substitution coverage. All return trusted descriptor-bound bytes after rename/symlink replacement; the race suite passed 50/50 repeated runs.
- Isolated CLI verification drove `mosaic agent --mosaic-home <fixture> comms-block self` while repeatedly swapping `fleet/` with an external symlink: `trusted=2 fail_closed=10 external_marker=0`; a persistent symlink ancestor exited 1 with a redacted unsafe-ancestor error. No live fleet state was used or mutated.
- Remediation gates: focused secure-file/comms/launch/tmux/Matrix `110/110`; full `@mosaicstack/mosaic` `914/914`; package and repository typecheck pass (`42/42` repository tasks); package and repository lint pass (`23/23` repository tasks); repository format check and `git diff --check` pass.
- Independent review found one production hardening blocker (nonblocking final open), one redacted-error blocker, and a deterministic ancestor-test gap. Remediation added `O_NONBLOCK`, normalized execute errors while preserving errno, proved the ancestor hook fires, and added final-target substitution coverage; post-remediation review evidence is clean on the production invariant.
docs/archive/work-records/m3-001-provider-adapter.md
+55
View File
@@ -0,0 +1,55 @@
# M3-001 Provider Adapter Pattern — Scratchpad
## Objective
Refactor ProviderService into an IProviderAdapter pattern without breaking existing Ollama flow.
## Plan
1. Add `IProviderAdapter` interface and supporting types to `@mosaicstack/types` provider package
2. Create `apps/gateway/src/agent/adapters/` directory with:
- `provider-adapter.interface.ts` — IProviderAdapter + ProviderHealth + CompletionParams + CompletionEvent
- `ollama.adapter.ts` — extract existing Ollama logic
3. Refactor ProviderService:
- Accept `IProviderAdapter[]` (injected via DI token)
- `registerAll()` / `listModels()` aggregates from all adapters
- `getAdapter(name)` — lookup by name
- `healthCheckAll()` — check all adapters
- Keep Pi ModelRegistry wiring (required by AgentService)
4. Wire up in AgentModule
## Key Findings
### Pi SDK Compatibility
- Pi SDK uses `ModelRegistry` as central registry; ProviderService wraps it
- `ModelRegistry.registerProvider()` is the integration point — adapters call this
- Pi doesn't have a native "IProviderAdapter" concept — adapters are a Mosaic abstraction on top
- The `createAgentSession()` call in AgentService uses `modelRegistry: this.providerService.getRegistry()`
- OllamaAdapter should call `registry.registerProvider('ollama', {...})` same as today
- CompletionParams/CompletionEvent: Pi SDK streams via `AgentSession.prompt()`, not raw completion
— IProviderAdapter.createCompletion() is for future direct use; for now stub or leave as interface-only
— ASSUMPTION: createCompletion is reserved for future M3+ work; Pi SDK owns the actual streaming
## Implementation Notes
- ESM: use `.js` extensions in all imports
- NestJS: use `@Inject()` explicitly
- Keep RoutingService working — it only uses `providerService.listAvailableModels()`
- Keep AgentService working — it uses `providerService.getRegistry()`, `findModel()`, `getDefaultModel()`, `listAvailableModels()`
## Progress
- [ ] Add types to @mosaicstack/types
- [ ] Create adapters/ directory
- [ ] Create IProviderAdapter interface file
- [ ] Create OllamaAdapter
- [ ] Refactor ProviderService
- [ ] Update AgentModule
- [ ] Run tests
- [ ] Run quality gates
## Risks
- Pi SDK doesn't natively support IProviderAdapter — adapters are a layer on top
- createCompletion() is architecturally sound but requires Pi session bypass (future work)
docs/archive/work-records/macp-oc-bridge-20260330.md
+16
View File
@@ -0,0 +1,16 @@
# Scratchpad: MACP OC Bridge (2026-03-30)
- Objective: Replace the OpenClaw MACP plugin's Pi-direct `runTurn` with the MACP controller queue bridge.
- Scope: `plugins/macp/src/macp-runtime.ts`, `plugins/macp/src/index.ts`, `plugins/macp/openclaw.plugin.json`, `plugins/macp/README.md`.
- Plan:
1. Read controller/dispatcher/plugin docs and confirm queue/result contract.
2. Queue tasks through `.mosaic/orchestrator/tasks.json` using a brief file and controller trigger.
3. Poll result JSON, stream output back to ACP, then validate with typecheck/format checks.
- Risks:
- The repo orchestrator must be enabled in `.mosaic/orchestrator/config.json`.
- Result JSON does not always embed worker output, so the runtime falls back to metadata-linked output files or a formatted result summary.
- Verification:
- `npx tsc --noEmit --target es2022 --module nodenext --moduleResolution nodenext --skipLibCheck plugins/macp/src/macp-runtime.ts plugins/macp/src/index.ts`
- `pnpm prettier --write "plugins/macp/**/*.{ts,json,md}"`
- `pnpm format:check`
- `npx tsc --noEmit -p plugins/macp/tsconfig.json` still fails in this branch because `plugins/macp/tsconfig.json` extends a missing `packages/config/typescript/library.json` file and also pulls in pre-existing external OpenClaw type noise.
docs/archive/work-records/ms-792-fleet-enoent-installer.md
+38
View File
@@ -0,0 +1,38 @@
# ms-792 — Fleet roster error handling and installer heading
## Objective
Make expected missing or malformed fleet roster configuration fail with an actionable message and nonzero exit instead of a raw Node stack trace. Ensure the installer preserves the `@mosaicstack/mosaic` heading.
## Plan
1. Add failing coverage for missing and malformed roster input.
2. Centralize roster-file read and parse error translation; add the CLI async error boundary.
3. Sweep fleet command read paths that bypass the roster loader.
4. Replace the installer heading output with format-safe rendering and test it.
5. Run focused and repository quality checks; request independent review.
## Progress
- 2026-07-16: Confirmed issue #792 and branch base `9745bc3f`.
- 2026-07-16: Installed locked workspace dependencies using a worktree-local pnpm store; no `.mosaic/` files were changed intentionally.
- 2026-07-16: Added a shared roster read/parse guard and routed v1 fleet commands plus v1/v2 selection through Commander’s actionable nonzero error path. V2 command modules already return structured nonzero JSON errors for their guarded reads.
- 2026-07-16: Replaced installer heading `echo` with format-safe `printf`; added a regression check for the scoped package heading.
- 2026-07-16: Rebuilt CLI and manually verified `fleet ps` with no roster prints the initialization hint, exits 1, and has no stack trace.
- 2026-07-17: Rebased #818 onto `origin/main` at `9ddc6fbd` (#791 PR3). The added `fleet regen` command had a canonical roster read in its sibling module; it now uses the same missing-roster guard and Commander exit path. Internal NORTH_STAR, preset, and post-write invariant reads remain intentionally unguarded.
- 2026-07-17: RoR found that semantically invalid v1 documents still escaped as plain `Error` values. `normalizeFleetRosterV1` now preserves each validation message while converting it to `FleetRosterConfigurationError`, so its command callers use the actionable nonzero Commander path.
## Verification
- `pnpm --filter @mosaicstack/mosaic test` — PASS (61 files, 1,046 tests; executed outside sandbox because CLI smoke tests spawn Node)
- `pnpm typecheck` — PASS
- `pnpm lint` — PASS
- `pnpm format:check` — PASS
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts src/commands/install-heading.spec.ts` — PASS (209 tests)
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet-regen-command.spec.ts` — PASS (27 tests, including missing canonical roster)
- `pnpm --filter @mosaicstack/mosaic exec vitest run src/commands/fleet.spec.ts -t "semantically invalid v1 roster"` — RED then PASS; verifies duplicate agent names are reported as `fleet.roster` exit 1 without a stack trace.
- Instrumented Vitest coverage is unavailable because `@vitest/coverage-v8` is not declared in this repository. Each branch added in the roster guard has direct unit coverage.
## Risks / blockers
- Dependency installation is required before executing Vitest, TypeScript, lint, and formatting gates.
docs/archive/work-records/p5-003-telegram-plugin.md
+50
View File
@@ -0,0 +1,50 @@
# Scratchpad — P5-003 Telegram Plugin
## Objective
Implement `@mosaicstack/telegram-plugin` by matching the established Discord plugin pattern with Telegraf + socket.io-client, add package docs, and pass package typecheck/lint.
## Requirements Source
- docs/PRD.md: Phase 5 remote control / Telegram plugin
- docs/TASKS.md: P5-003
- User task brief dated 2026-03-13
## Plan
1. Inspect Discord plugin behavior and package conventions
2. Add Telegram runtime dependencies if missing
3. Implement Telegram plugin with matching gateway event flow
4. Add README usage documentation
5. Run package typecheck and lint
6. Run code review and remediate findings
7. Commit, push, open PR, notify, remove worktree
## TDD Rationale
ASSUMPTION: No existing telegram package test harness or fixture coverage makes package-level TDD
disproportionate for this plugin scaffold task. Validation will rely on typecheck, lint, and
manual structural parity with the Discord plugin.
## Risks
- Telegram API typings may differ from Discord’s event shapes and require narrower guards.
- Socket event payloads may already include `role` in shared gateway expectations.
## Progress Log
- 2026-03-13: Loaded Mosaic/global/repo guidance, mission files, Discord reference implementation, and Telegram package scaffold.
- 2026-03-13: Added `telegraf` and `socket.io-client` to `@mosaicstack/telegram-plugin`.
- 2026-03-13: Implemented Telegram message forwarding, gateway streaming accumulation, response chunking, and package README.
## Verification Evidence
- `pnpm --filter @mosaicstack/telegram-plugin typecheck` → pass
- `pnpm --filter @mosaicstack/telegram-plugin lint` → pass
- `pnpm typecheck` → pass
- `pnpm lint` → pass
## Review
- Automated uncommitted review wrapper was invoked for the current delta.
- Manual review completed against Discord parity, gateway event contracts, and package docs; no additional blockers found.
docs/archive/work-records/p5-004-authentik-sso.md
+44
View File
@@ -0,0 +1,44 @@
# P5-004 Scratchpad
- Objective: Add optional Authentik OIDC SSO adapter via Better Auth genericOAuth.
- Task ref: P5-004
- Issue ref: #96
- Plan:
1. Inspect auth/gateway surfaces and Better Auth plugin shape.
2. Add failing coverage for auth config/startup validation where feasible.
3. Implement adapter, docs, and warnings.
4. Run targeted typechecks, lint, and review.
- TDD note: no low-friction auth plugin or bootstrap-env test seam exists for `packages/auth/src/auth.ts` or `apps/gateway/src/main.ts`. This change is configuration-oriented and does not alter an existing behavioral contract with a current test harness. I skipped new tests for this pass and relied on exact typecheck/lint/test commands plus manual review.
- Changes:
1. Added conditional Better Auth `genericOAuth` plugin registration for the `authentik` provider in `packages/auth/src/auth.ts`.
2. Added a soft startup warning in `apps/gateway/src/main.ts` for incomplete Authentik env configuration.
3. Added `docs/plans/authentik-sso-setup.md` with env, redirect URI, and test-flow guidance.
4. Confirmed `packages/auth/src/index.ts` already exports `AuthConfig`; no change required there.
- Verification:
1. `pnpm --filter @mosaicstack/db build`
2. `pnpm --filter @mosaicstack/auth typecheck`
3. `pnpm --filter @mosaicstack/gateway typecheck`
4. `pnpm lint`
5. `pnpm format:check`
6. `pnpm --filter @mosaicstack/auth test`
7. `pnpm --filter @mosaicstack/gateway test`
- Results:
1. `@mosaicstack/auth` typecheck passed after replacing the non-existent `enabled` field with conditional plugin registration.
2. `@mosaicstack/gateway` typecheck passed.
3. Repo lint passed.
4. Prettier check passed after formatting `apps/gateway/src/main.ts`.
5. `@mosaicstack/auth` tests reported `No test files found, exiting with code 0`.
6. `@mosaicstack/gateway` tests passed: `3` files, `20` tests.
- Review:
1. Manual review of the diff found no blocker issues.
2. External `codex-code-review.sh --uncommitted` was attempted but did not return a usable verdict in-session; no automated review findings were available from that run.
- Situational evidence:
1. Provider activation is env-gated by `AUTHENTIK_CLIENT_ID`.
2. Misconfigured optional SSO surfaces a warning instead of crashing gateway startup.
3. Setup doc records the expected redirect path: `{BETTER_AUTH_URL}/api/auth/callback/authentik`.
docs/archive/work-records/p8-001-sso-providers.md
+65
View File
@@ -0,0 +1,65 @@
# P8-001 — WorkOS + Keycloak SSO Providers
**Branch:** feat/p8-001-sso-providers
**Started:** 2026-03-18
**Mode:** Delivery
## Objective
Add WorkOS and Keycloak as optional SSO providers to the BetterAuth configuration, following the existing Authentik pattern.
## Scope
| Surface | Change |
| ---------------------------------------- | ----------------------------------------------------------------------- |
| `packages/auth/src/auth.ts` | Refactor provider array, add WorkOS + Keycloak conditional registration |
| `apps/web/src/lib/auth-client.ts` | Add `genericOAuthClient()` plugin |
| `apps/web/src/app/(auth)/login/page.tsx` | WorkOS + Keycloak SSO buttons gated by `NEXT_PUBLIC_*` env vars |
| `.env.example` | Document WorkOS + Keycloak env vars |
| `packages/auth/src/auth.test.ts` | Unit tests verifying env-var gating |
## Plan
1. ✅ Refactor `createAuth` to build `oauthProviders[]` conditionally
2. ✅ Add WorkOS provider (explicit URLs, no discovery)
3. ✅ Add Keycloak provider (discoveryUrl pattern)
4. ✅ Add `genericOAuthClient()` to auth-client.ts
5. ✅ Add SSO buttons to login page gated by `NEXT_PUBLIC_WORKOS_ENABLED` / `NEXT_PUBLIC_KEYCLOAK_ENABLED`
6. ✅ Update `.env.example`
7. ⏳ Write `auth.test.ts` with env-var gating tests
8. ⏳ Quality gates: typecheck + lint + format:check + test
9. ⏳ Commit + push + PR
## Decisions
- **WorkOS**: Uses explicit `authorizationUrl`, `tokenUrl`, `userInfoUrl` (no discovery endpoint available)
- **Keycloak**: Uses `discoveryUrl` pattern (`{URL}/realms/{REALM}/.well-known/openid-configuration`)
- **UI gating**: Login page uses `NEXT_PUBLIC_WORKOS_ENABLED` / `NEXT_PUBLIC_KEYCLOAK_ENABLED` feature flags (safer than exposing secret env var names client-side)
- **Refactor**: Authentik moved into same `oauthProviders[]` array pattern — cleaner, more extensible
- **Feature flag design**: `NEXT_PUBLIC_*` flags are opt-in alongside credentials (prevents accidental button render when creds not set)
## Assumptions
- `ASSUMPTION:` WorkOS OIDC discovery URL is not publicly documented; using direct URL pattern from WorkOS SSO docs.
- `ASSUMPTION:` `NEXT_PUBLIC_WORKOS_ENABLED=true` must be explicitly set — this is intentional (credential presence alone doesn't enable the button since NEXT_PUBLIC vars are baked at build time).
## Tests
- `auth.test.ts`: Mocks betterAuth stack, verifies WorkOS included/excluded based on env var
- `auth.test.ts`: Verifies Keycloak discoveryUrl constructed correctly
## Quality Gate Results
| Gate | Status |
| ------------------------ | -------------------------------------------- |
| typecheck | ✅ 32/32 cached green |
| lint | ✅ 18/18 cached green |
| format:check | ✅ All matched files use Prettier code style |
| test (@mosaicstack/auth) | ✅ 8/8 tests passed |
## Verification Evidence
- `pnpm typecheck` — FULL TURBO, 32 tasks successful
- `pnpm lint` — FULL TURBO, 18 tasks successful
- `pnpm format:check` — All matched files use Prettier code style!
- `pnpm --filter=@mosaicstack/auth test` — 8 tests passed, 0 failed
docs/archive/work-records/p8-009-tui-slash-commands.md
+40
View File
@@ -0,0 +1,40 @@
# P8-009: TUI Phase 1 — Slash Command Parsing
## Task Reference
- Issue: #162
- Branch: feat/p8-009-tui-slash-commands
## Scope
- New files: parse.ts, registry.ts, local/help.ts, local/status.ts, commands/index.ts
- Modified files: use-socket.ts, input-bar.tsx, message-list.tsx, app.tsx
## Key Observations
- CommandDef in @mosaicstack/types does NOT have `category` field — will omit from LOCAL_COMMANDS
- CommandDef.args is `CommandArgDef[] | undefined`, not `{ usage: string }` — help.ts args rendering needs adjustment
- Message role union currently: 'user' | 'assistant' | 'thinking' | 'tool' — adding 'system'
- InputBar currently takes `onSubmit: (value: string) => void` — need to add slash command interception
- app.tsx passes `onSubmit={socket.sendMessage}` directly — needs command-aware handler
## Assumptions
- ASSUMPTION: `category` field not in CommandDef type — will skip category grouping in help output, or add it only to registry (not to CommandDef type)
- ASSUMPTION: For the `args` field display in help, will use `CommandArgDef.name` and `CommandArgDef.description`
- ASSUMPTION: `commands:manifest` event type may not be in ServerToClientEvents — will handle via socket.on with casting if needed
## Status
- [ ] Create commands directory structure
- [ ] Implement parse.ts
- [ ] Implement registry.ts
- [ ] Implement local/help.ts
- [ ] Implement local/status.ts
- [ ] Implement commands/index.ts
- [ ] Modify use-socket.ts
- [ ] Modify input-bar.tsx
- [ ] Modify message-list.tsx
- [ ] Modify app.tsx
- [ ] Run quality gates
- [ ] Commit + Push + PR + CI

Some files were not shown because too many files have changed in this diff Show More