chore: consolidate new foundation and archive v1 (#1495)

This commit is contained in:
2026-09-07 12:32:57 -05:00
3511 changed files with 727899 additions and 10 deletions
@@ -0,0 +1,755 @@
# Durable tmux Fleet Installation Plan
> **For Mosaic/Hermes:** This is an implementation plan for making the tmux-backed Mosaic software-factory fleet durable on this server and reusable in generic Mosaic Stack installs. Keep local USC/Mosaic defaults in profiles; keep framework behavior customizable.
**Goal:** Add a supported Mosaic tmux-fleet installation path: holder-owned tmux server, per-agent reusable sessions, reliable send/reset/status tools, local roster customization, and a documented cutover for this server.
**Architecture:** Mosaic should ship generic tmux fleet primitives in the framework, then layer local rosters through configuration. The holder service owns the tmux socket; each agent service joins the holder-owned server and runs `mosaic yolo <runtime>`. The orchestrator addresses agents through `mosaic agent ...` abstractions so tmux can later be replaced by Matrix-backed agent comms without changing mission flow.
**Reference:** AI Guide `playbooks/tmux-fleet.md` at commit `2a0b0b5` documents the organization-neutral holder-service pattern, exact-match `=<name>` stop targets, and coupled-server cutover/verification sequence. The Stack implementation should treat that as the lifecycle model and keep concrete Mosaic unit/tooling details here.
**Tech Stack:** Bash, tmux, user systemd units, Mosaic CLI/framework installer, JSON/YAML roster config, existing `packages/mosaic/framework/tools/tmux/{agent-send.sh,send-message.sh}`.
---
## Current evidence from this server
Checked 2026-06-19:
- Host: `W-jarvis`
- User: `jarvis`
- tmux: `/usr/bin/tmux`, version `3.4`
- user systemd: active
- existing tmux sessions: `ai-bma-0`, `dyor-1`, `melaniewoltje-3`, `sage-2`
- existing Mosaic runtime: `/home/jarvis/.npm-global/bin/mosaic`, version `0.0.31`
- installed `~/.config/mosaic/tools/tmux` was not present even though the stack repo contains `packages/mosaic/framework/tools/tmux/`
Implication: do not kill the current tmux server casually. This server has active ad-hoc/service sessions. The durable fleet cutover must be planned, with either a separate socket first or a scheduled fleet recycle.
## Design decisions
### 1. Generic framework, local profile
The Mosaic framework should ship:
- systemd unit templates;
- tmux fleet CLI wrappers;
- roster schema and examples;
- install/enable/status/reset commands;
- docs and verification scripts.
Local environments should provide:
- agent names;
- runtime per slot (`claude`, `pi`, `codex`, etc.);
- default role class;
- launch directory;
- optional kickstart prompt;
- model/provider hints;
- transport selection (`tmux` now, `matrix` later).
Do not bake the USC roster into generic install code. Ship it as an example profile.
### 2. Durable sessions, disposable task context
Session names are durable operational addresses. Task persona is disposable. Reusable worker slots should be reset with `/clear` or `/new` and then receive a fresh task kickstart.
Persistent/semi-persistent personas:
- lead orchestrator;
- final/adversarial reviewer;
- architecture/enhancement lane.
Disposable slots:
- implementers;
- ordinary reviewers;
- security reviewers unless actively holding a security mission.
### 3. Transport abstraction now
Add commands around tmux instead of calling tmux directly from orchestration:
```bash
mosaic agent send <agent> --message "..."
mosaic agent status [--json]
mosaic agent reset <agent> [--clear|--new]
mosaic agent roster [--json]
mosaic fleet install|start|stop|restart|status|verify
```
Today these call tmux/systemd. Later the same command surface can target Matrix or per-agent gateways.
### 4. Avoid shared-server ownership bug
Use the AI Guide holder pattern:
```text
mosaic-tmux-holder.service owns the tmux server/socket
mosaic-agent@<name>.service joins the existing holder-owned socket
ExecStop kills only session =<name>
```
Use exact tmux targets: `=<session>`.
### 5. Prefer separate named socket for Mosaic factory
To avoid disturbing existing tmux work, the default fleet should use a named socket such as:
```text
$XDG_RUNTIME_DIR/mosaic-factory.tmux
```
or tmux socket name:
```bash
tmux -L mosaic-factory ...
```
This avoids collision with ordinary `tmux ls` sessions. The send tools need socket support.
---
## Target USC-style roster example
Ship as example only, not default:
```yaml
version: 1
transport: tmux
tmux:
socket_name: mosaic-factory
holder_session: _holder
working_directory: ~/src
agents:
- name: mos-claude
runtime: claude
class: orchestrator
model_hint: Claude Opus
persistent_persona: true
- name: coder0
runtime: claude
class: implementer
model_hint: Claude Opus
reset_between_tasks: true
- name: coder1
runtime: claude
class: implementer
model_hint: Claude Opus
reset_between_tasks: true
- name: coder2
runtime: pi
class: implementer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: coder3
runtime: pi
class: implementer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: coder4
runtime: claude
class: implementer
model_hint: Claude Opus
reset_between_tasks: true
- name: coder5
runtime: claude
class: implementer
model_hint: Claude Opus
reset_between_tasks: true
- name: enhance
runtime: claude
class: enhancer
model_hint: Claude Opus
persistent_persona: semi
- name: rev0
runtime: pi
class: reviewer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: rev1
runtime: pi
class: reviewer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: secrev0
runtime: pi
class: security_reviewer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: secrev1
runtime: pi
class: security_reviewer
model_hint: Pi GPT-5.5
reset_between_tasks: true
- name: ultron
runtime: pi
class: final_reviewer
model_hint: Pi GPT-5.5
persistent_persona: semi
```
---
## Phase 0 — Confirm install surfaces
### Task 0.1: Inspect installer copy behavior
**Objective:** Confirm how framework files under `packages/mosaic/framework/` become installed under `~/.config/mosaic/`.
**Files:**
- Read: `tools/install.sh`
- Read: `packages/mosaic/framework/install.sh`
- Read: `packages/mosaic/src/runtime/install-manifest.ts`
**Steps:**
1. Verify `packages/mosaic/framework/install.sh` rsyncs `tools/tmux`.
2. Verify whether npm-packaged installs include `framework/tools/tmux`.
3. Confirm whether installed hosts should run `mosaic update`, `bash tools/install.sh`, or `packages/mosaic/framework/install.sh` to receive new tmux tools.
4. Record exact propagation command in docs.
**Verification:**
```bash
bash packages/mosaic/framework/install.sh --help || true
npm pack --dry-run --json | jq '.[0].files[].path' | grep 'framework/tools/tmux'
```
Expected: tmux tools are included in installable package or packaging fix is identified.
### Task 0.2: Inspect current yolo launch semantics
**Objective:** Confirm `mosaic yolo claude` and `mosaic yolo pi` accept optional initial prompt text and behave well under systemd/tmux.
**Files:**
- Read: `packages/mosaic/src/**`
- Read: `packages/mosaic/framework/runtime/claude/RUNTIME.md`
- Read: `packages/mosaic/framework/runtime/pi/RUNTIME.md`
**Verification commands:**
```bash
mosaic yolo claude --help
mosaic yolo pi --help
```
Expected: a systemd `ExecStart` can launch the runtime either with no prompt or with a kickstart prompt file/string.
---
## Phase 1 — Framework tmux primitives
### Task 1.1: Add socket support to send tools
**Objective:** Allow `agent-send.sh` and `send-message.sh` to target a named Mosaic tmux socket without affecting default tmux sessions.
**Files:**
- Modify: `packages/mosaic/framework/tools/tmux/send-message.sh`
- Modify: `packages/mosaic/framework/tools/tmux/agent-send.sh`
- Modify: `packages/mosaic/framework/tools/tmux/README.md`
- Test: `packages/mosaic/framework/tools/tmux/test-send-message.sh` (new)
**Design:**
Add optional flags:
```bash
-L SOCKET_NAME # tmux -L socket name
-SOCKET PATH # optional later if needed; avoid conflict with existing -S source label in agent-send
```
Because `agent-send.sh` already uses `-S` for source label, prefer `-L` for socket name and `-T` or `--socket-path` only if long-option parsing is added.
**Implementation notes:**
- Build a tmux command array:
```bash
tmux_cmd=(tmux)
if [ -n "$SOCKET_NAME" ]; then tmux_cmd+=( -L "$SOCKET_NAME" ); fi
```
- Replace raw `tmux ...` calls with `"${tmux_cmd[@]}" ...`.
- Pass `-L` through remote ssh invocation.
- Include socket name in verbose output.
**Verification:**
```bash
tmux -L mosaic-test new-session -d -s target 'cat'
packages/mosaic/framework/tools/tmux/send-message.sh -L mosaic-test -t target -m 'hello'
tmux -L mosaic-test capture-pane -t target -p | grep hello
tmux -L mosaic-test kill-server
```
Expected: message lands in the named socket session; default `tmux ls` is untouched.
### Task 1.2: Add exact target validation helper
**Objective:** Prevent accidental prefix targeting in all tmux fleet operations.
**Files:**
- Create: `packages/mosaic/framework/tools/tmux/_lib.sh`
- Modify: `send-message.sh`
- Modify: `agent-send.sh`
**Behavior:**
- For session-only agent names, normalize target to `=<name>` before kill/status/reset operations.
- For explicit pane targets like `session:window.pane`, allow as advanced path but document the risk.
**Verification:**
Create sessions `agent` and `agent0`; verify killing/resetting `agent` does not affect `agent0`.
---
## Phase 2 — systemd unit templates
### Task 2.1: Add holder service template
**Objective:** Ship a user systemd unit template that owns the Mosaic factory tmux server.
**Files:**
- Create: `packages/mosaic/framework/systemd/user/mosaic-tmux-holder.service`
- Create: `packages/mosaic/framework/tools/fleet/install-user-units.sh`
**Unit shape:**
```ini
[Unit]
Description=Mosaic tmux fleet holder
Documentation=https://git.mosaicstack.dev/mosaicstack/aiguide
[Service]
Type=oneshot
RemainAfterExit=yes
Environment=MOSAIC_TMUX_SOCKET=mosaic-factory
ExecStart=/usr/bin/tmux -L ${MOSAIC_TMUX_SOCKET} new-session -d -s _holder 'while true; do sleep 3600; done'
ExecStop=-/usr/bin/tmux -L ${MOSAIC_TMUX_SOCKET} kill-server
[Install]
WantedBy=default.target
```
**Important:** systemd environment expansion in `ExecStart` is limited. Verify syntax; if `%E`/environment expansion is awkward, generate concrete units from config instead of relying on dynamic expansion.
**Verification:**
```bash
systemd-analyze --user verify ~/.config/systemd/user/mosaic-tmux-holder.service
systemctl --user daemon-reload
systemctl --user start mosaic-tmux-holder.service
tmux -L mosaic-factory ls | grep _holder
```
### Task 2.2: Add agent service template
**Objective:** Ship a user systemd template that starts one configured agent slot.
**Files:**
- Create: `packages/mosaic/framework/systemd/user/[email protected]`
- Modify: `packages/mosaic/framework/tools/fleet/install-user-units.sh`
**Unit shape:**
```ini
[Unit]
Description=Mosaic agent session %i
Requires=mosaic-tmux-holder.service
After=mosaic-tmux-holder.service
PartOf=mosaic-tmux-holder.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=%h/src
Environment=MOSAIC_TMUX_SOCKET=mosaic-factory
ExecStart=/bin/bash -lc 'tmux -L "$MOSAIC_TMUX_SOCKET" new-session -d -s "%i" "mosaic yolo $(mosaic fleet runtime %i)"'
ExecStop=-/usr/bin/tmux -L mosaic-factory kill-session -t '=%i'
[Install]
WantedBy=default.target
```
**Design warning:** command substitution in unit files can become brittle. Prefer a generated per-agent EnvironmentFile:
```text
~/.config/mosaic/fleet/agents/coder0.env
```
with:
```bash
MOSAIC_AGENT_NAME=coder0
MOSAIC_AGENT_RUNTIME=claude
MOSAIC_AGENT_WORKDIR=/home/jarvis/src
MOSAIC_TMUX_SOCKET=mosaic-factory
```
Then `ExecStart` calls a wrapper:
```bash
~/.config/mosaic/tools/fleet/start-agent-session.sh
```
**Verification:**
```bash
systemd-analyze --user verify ~/.config/systemd/user/[email protected]
systemctl --user start [email protected]
tmux -L mosaic-factory has-session -t '=coder0'
systemctl --user restart [email protected]
```
Expected: holder server PID remains unchanged; only `coder0` session recycles.
### Task 2.3: Add start-agent wrapper
**Objective:** Keep systemd units simple by moving config lookup and launch command construction into a script.
**Files:**
- Create: `packages/mosaic/framework/tools/fleet/start-agent-session.sh`
**Behavior:**
Inputs:
```bash
start-agent-session.sh <agent-name>
```
Reads:
```text
$MOSAIC_HOME/fleet/agents/<agent-name>.env
```
Starts:
```bash
tmux -L "$MOSAIC_TMUX_SOCKET" new-session -d -s "$MOSAIC_AGENT_NAME" -c "$MOSAIC_AGENT_WORKDIR" "mosaic yolo $MOSAIC_AGENT_RUNTIME"
```
Guardrails:
- fail if runtime is empty;
- fail if workdir does not exist;
- no duplicate sessions unless `--replace` is passed;
- exact session names only.
---
## Phase 3 — roster config and CLI wrappers
### Task 3.1: Add fleet config schema and examples
**Objective:** Define customizable install-time roster without hardcoding USC.
**Files:**
- Create: `packages/mosaic/framework/fleet/roster.schema.json`
- Create: `packages/mosaic/framework/fleet/examples/minimal.yaml`
- Create: `packages/mosaic/framework/fleet/examples/usc-software-factory.yaml`
- Create: `packages/mosaic/framework/fleet/README.md`
**Schema concepts:**
- `transport`: `tmux` now; `matrix` later.
- `tmux.socket_name`
- `tmux.holder_session`
- `defaults.working_directory`
- `agents[].name`
- `agents[].runtime`
- `agents[].class`
- `agents[].model_hint`
- `agents[].persistent_persona`
- `agents[].reset_between_tasks`
- `agents[].kickstart_template`
**Verification:**
Use `jq` for JSON examples or add a small Python/YAML validator if YAML is chosen. If no YAML parser is guaranteed, store examples as JSON or support both with Python stdlib JSON first.
### Task 3.2: Add `mosaic fleet` commands
**Objective:** Provide operator-safe commands for install/status/start/stop/restart/verify.
**Files:**
- Modify: `packages/mosaic/src/cli.ts` or the current commander entrypoint.
- Create scripts under: `packages/mosaic/framework/tools/fleet/`
**Commands:**
```bash
mosaic fleet init --profile minimal|usc --write
mosaic fleet install-systemd
mosaic fleet start [agent]
mosaic fleet stop [agent]
mosaic fleet restart [agent]
mosaic fleet status --json
mosaic fleet verify
```
**Implementation path:**
Start by wrapping framework shell scripts from the TypeScript CLI. Do not overbuild a TypeScript service manager in the first pass.
### Task 3.3: Add `mosaic agent` commands
**Objective:** Provide transport-stable per-agent operations.
**Files:**
- Modify: Mosaic CLI entrypoint.
- Create: `packages/mosaic/framework/tools/agent/` or reuse `tools/tmux` + `tools/fleet`.
**Commands:**
```bash
mosaic agent roster [--json]
mosaic agent status [agent] [--json]
mosaic agent send <agent> --message "..."
mosaic agent reset <agent> --clear|--new
mosaic agent tail <agent> [-n 80]
```
**Reset behavior:**
For tmux transport, `reset --clear` sends `/clear` then Enter through `send-message.sh`.
For Claude/Pi differences, keep reset command configurable per runtime:
```yaml
runtimes:
claude:
reset_command: /clear
pi:
reset_command: /new
```
If a runtime does not support a known reset command, restart the service and send a fresh kickstart.
---
## Phase 4 — this-server rollout strategy
### Task 4.1: Install on separate socket first
**Objective:** Prove the holder pattern without disturbing existing sessions.
**Commands after implementation lands locally:**
```bash
mosaic fleet init --profile minimal --write
mosaic fleet install-systemd
systemctl --user daemon-reload
systemctl --user start mosaic-tmux-holder.service
mosaic fleet verify
```
Expected:
- `tmux -L mosaic-factory ls` shows `_holder`.
- normal `tmux ls` still shows existing sessions unchanged.
### Task 4.2: Start one canary agent
**Objective:** Validate single-agent start/restart isolation.
Use a harmless canary first, not the full fleet.
Example roster addition:
```yaml
- name: canary-pi
runtime: pi
class: canary
working_directory: /home/jarvis/src
```
Commands:
```bash
systemctl --user start [email protected]
SRV=$(tmux -L mosaic-factory display-message -p '#{pid}')
systemctl --user restart [email protected]
test "$SRV" = "$(tmux -L mosaic-factory display-message -p '#{pid}')"
tmux -L mosaic-factory ls
```
Expected: holder PID unchanged; `_holder` remains; `canary-pi` recreated.
### Task 4.3: Configure local Mosaic factory roster
**Objective:** Create the actual local roster for this server after canary passes.
Do not assume USC exact roster is desired here. Create a local profile such as:
```text
~/.config/mosaic/fleet/roster.yaml
```
Initial local recommendation:
- `mos-claude` orchestrator
- `coder0` / `coder1` implementers
- `rev0` reviewer
- `secrev0` security reviewer
- `ultron` final/adversarial reviewer
Scale to full USC-style pool only after resource/budget behavior is understood.
### Task 4.4: Cut over existing ad-hoc tmux sessions only if desired
**Objective:** Avoid data loss.
Existing sessions on this server are not on the proposed `mosaic-factory` socket. They can remain untouched. If we later want them under Mosaic fleet control:
1. list sessions;
2. capture logs/handoffs;
3. stop old processes intentionally;
4. recreate as configured `mosaic-agent@...` services;
5. verify comms and state.
Do not run `tmux kill-server` on the default socket unless Jason explicitly approves that outage.
---
## Phase 5 — docs and AI Guide backfill
### Task 5.1: Stack docs
**Objective:** Document install and customization for Mosaic Stack users.
**Files:**
- Create: `docs/fleet/tmux-fleet.md` or `packages/mosaic/framework/tools/fleet/README.md`
- Modify: top-level `README.md` if appropriate.
Must cover:
- what problem holder service solves;
- install commands;
- customization file;
- example rosters;
- reset/reuse lifecycle;
- exact-target safety;
- separate socket default;
- Matrix migration path.
### Task 5.2: AI Guide docs
**Objective:** Keep generic guidance in AI Guide and implementation details in Stack.
**Files in `mosaicstack/aiguide`:**
- Update: `playbooks/tmux-fleet.md` with named socket, roster/profile, and resettable-slot pattern.
- Add or update: `reference/agent-role-matrix.md` if PR #5 lands.
Do not put Mosaic install commands as the only path in AI Guide. Present them as one implementation profile.
---
## Phase 6 — Matrix migration seam
### Task 6.1: Add transport enum but implement tmux only
**Objective:** Avoid hardcoding tmux into orchestration semantics.
Roster:
```yaml
transport: tmux
```
Future:
```yaml
transport: matrix
matrix:
homeserver: https://matrix.example
room_prefix: mosaic-factory
```
### Task 6.2: Define transport interface docs
**Objective:** Make Matrix plugin work a transport swap, not a rewrite.
Minimum operations:
```text
send(agent, message)
reset(agent, mode)
status(agent)
tail(agent)
listAgents()
```
Any tmux-specific concept must stay below this line.
---
## Acceptance criteria
The implementation is complete when:
- `mosaic fleet init` can write a minimal roster.
- `mosaic fleet install-systemd` installs holder and agent units without hand editing.
- `mosaic fleet start` starts the holder and configured agents on a named tmux socket.
- Restarting one `[email protected]` does not change holder server PID or kill sibling sessions.
- `mosaic agent send` can deliver a message to a named agent with a self-identifying preamble.
- `mosaic agent reset` can clear/new a reusable slot and send a fresh kickstart.
- `mosaic fleet verify` proves holder ownership, exact-target safety, and per-agent restart isolation.
- Existing default tmux sessions on this server are not disturbed by default install.
- Docs explain generic customization and include USC-style roster only as an example.
- AI Guide remains generic; Mosaic Stack docs carry the concrete install path.
## Risks and mitigations
| Risk | Mitigation |
| --------------------------------------------------- | --------------------------------------------------------------------------------- |
| Killing existing tmux sessions | Use named `mosaic-factory` socket; no default `tmux kill-server`. |
| systemd unit quoting/env expansion bugs | Move logic into shell wrappers; verify with `systemd-analyze --user verify`. |
| Runtime reset command mismatch | Make reset command runtime-configurable; fallback to service restart + kickstart. |
| Tool install drift | Ensure npm package includes framework tmux/fleet tools; add packaging test. |
| Mosaic-specific assumptions leak into generic guide | Keep USC roster as example profile; AI Guide documents pattern/options. |
| Matrix migration blocked by tmux coupling | Add `mosaic agent` abstraction now; keep tmux details below transport layer. |
## Suggested first PR split
1. **PR A — tmux tool hardening**
- socket support;
- exact target helpers;
- tests/docs.
2. **PR B — fleet systemd primitives**
- holder unit;
- agent unit;
- start-agent wrapper;
- install-user-units script;
- verify script.
3. **PR C — roster and CLI**
- roster schema/examples;
- `mosaic fleet ...` commands;
- `mosaic agent ...` commands.
4. **PR D — local rollout and docs**
- local roster for this server;
- run canary;
- document verification evidence;
- update AI Guide with generic lessons.
## Immediate next action
Implement PR A first. It is low-risk, improves existing tools, and is required for a safe named-socket rollout on this server.
@@ -0,0 +1,73 @@
# 2026-08-17 — Fleet identity, comms delivery, and the ~/.mosaic tree (continuation record)
> **Status:** active continuation record | **Owner:** Jason (rulings) / fleet (delivery) | **Created:** 2026-08-17, sb-it-1-dt session with Jarvis (jarvis-brain)
> **Audience:** the homelab agents continuing this effort tonight. Read this whole file before acting; it supersedes nothing but preserves structure and decisions that must not be lost.
---
## Why this exists
A session on sb-it-1-dt (2026-08-17) produced three architecture decisions (two awaiting Jason's ruling), one incident postmortem (#1295), interim guardrail edits in the user-owned `~/.mosaic/` contract tree, and one new tool (`ensure-watcher.sh`). The work spans jarvis-brain (P0, not retained) and this repo (the product). **This file is the stack-side anchor so continuation does not depend on jarvis-brain surviving.**
## 1. The `~/.mosaic` tree model — as-built, preserve this structure
Three-tree split (this is design intent, not accident; keep it through all framework work):
| Tree | Owner | Rule |
| ------------------- | --------- | ------------------------------------------------------------------------------------------------------------- |
| `~/.config/mosaic/` | framework | upgrade-managed templates; NEVER user-edited; `mosaic upgrade` may overwrite |
| `~/.mosaic/` | user | working contracts, guides, fleet agents; upgrades reconcile with **deny-wins** (user edits never overwritten) |
| repo satellites | repos | bootstrapped per-repo `.mosaic/` state |
As-built inventory of `~/.mosaic` on sb-it-1-dt (2026-08-17):
- **Contract core:** `CONSTITUTION.md` (L0 law), `AGENTS.md` (dispatcher + guide router + Fleet Comms Watcher requirement), `SOUL.md` (generic base for ALL fleet agents, zero persona — includes the new **Fleet Boundaries** section), `STANDARDS.md` (universal standards — includes new **session identity** + **comms watcher hygiene** sections), `SYSTEM.md` (pure communication contract, byte-identical to jarvis-brain's prompt-testing `sr_opus_5_system_prompt.md`), `USER.md`, `TOOLS.md`.
- **`guides/`** — user-owned working copies (E2E-DELIVERY, ORCHESTRATOR(+PROTOCOL,+LEARNINGS), WAKE-DOCTRINE, VAULT-SECRETS, etc.).
- **`fleet/agents/`** — the per-agent store (this is MOSAIC-D-002's substrate, already in use):
- real agent dirs: `fargo/`, `orchestrator/`, `probe/`, `vision/`, `weekly-update/` — shape: `profile.json` (harness/account/overlay pointer) + `overlay.json` + `SOUL.md` (persona) + `scratch/` `work/` `notes/` subdirs (hygiene rules in root SOUL.md)
- `*.env.generated` launch overlays: `luna` `sol` `terra` (carry `MOSAIC_AGENT_NAME`, `_CLASS`, `_RUNTIME`, `_MODEL`, `_REASONING`, `_TOOL_POLICY`) — these are the mosaic-fleet seat launch envs; `inbox.env`, `itops.env` also present
- `probe/` is the validated layout proof: auth-bundle symlink chain, per-agent sessions, plugin-store symlink (from 2026-08-07)
- **`auth/`, `config/`, `memory/`, `plugins/`, `skills/`, `skills-local/`** — per-tree copies/links for runtime isolation.
- Related but outside the tree: watcher units at `~/.config/systemd/user/<agent>-comms-watcher.service`; watcher seen-state at `~/.local/state/comms-watcher-<agent>/`.
## 2. Decisions register (2026-08-17 session)
Full strict records live in jarvis-brain `docs/decisions/mosaic-stack/` (render on its dashboard); both are **Pending Jason's ruling**. Summaries so the content survives P0:
- **MOSAIC-D-001 — SYSTEM.md as canonical harness system prompt.** Static core (Constitution+AGENTS+USER+overlays) in one tracked file; launcher renders dynamic tail (mission/PRD/fleet/persona). Delivery: `--append-system-prompt` (repeatable) for pi/claude; symlinked core file for codex (`$CODEX_HOME/instructions.md`) and opencode (`AGENTS.md`); their dynamic tail via initial prompt (needs live verification). Static-first order is the cache win. `SYSTEM.md` in `~/.mosaic` today is the communication-contract file — D-001's SYSTEM.md is the broader composition; naming to reconcile at implementation.
- **MOSAIC-D-002 — per-agent harness homes + mechanical profiles.** Launch with targeted config-dir env vars (e.g. `PI_CODING_AGENT_DIR=~/.mosaic/fleet/agents/<name>/pi`), NOT literal HOME. SOUL.md identity mechanically generated from roster (single writer; kills the hand-copy drift measured in `agents/vision/SOUL.md` on jarvis-brain: declared Jarvis, answered Vision). Composes: SYSTEM core → per-agent SOUL → dynamic tail. `MOSAIC_AGENT_NAME` stays load-bearing for comms.
- **Comms delivery tooling belongs in the STACK framework, not jarvis-brain** (decided in discussion; supersedes the interim placement). jarvis-brain keeps only the transport _data_ (`comms/` tree) while it lives. Agents launch from their own repos (terra from `~/src/stack` etc.) — delivery is transport-repo-relative, so this works; but every installed watcher unit's ExecStart currently points into `~/src/jarvis-brain/scripts/` — that dependency is the P0 trap to remove. Migration = move tools + regenerate units, in one step.
- **Watcher provisioning is instantiation duty, never running-agent duty.** Interim landed as jarvis-brain `scripts/comms/ensure-watcher.sh` (idempotent ensure + `--status` boot check + interim identity warnings: missing target session, pane `MOSAIC_AGENT_NAME` mismatch via `/proc/<child>/environ`, bare-runtime NOTE). Framework move: fold into `mosaic agent --new` + fleet launch + `mosaic doctor` drift check.
- **Prose guardrails landed (interim fences until mechanical fixes):** `~/.mosaic/SOUL.md` Fleet Boundaries (wrong-session tripwire; comms ownership; cross-agent investigation requires tasking) · `STANDARDS.md` session identity + watcher hygiene · `AGENTS.md` Fleet Comms Watcher requirement (P0-interim script path marked transitional).
## 3. Incident → #1295 (already tracked here)
`https://git.mosaicstack.dev/mosaicstack/stack/issues/1295` — docs-seat incident: cwd-keyed session files served three lives (dev chat → goals seat → 22 watcher injections into a wedged process); name-based watcher delivery with no identity verification; wedge after pi 0.84.1→0.84.2 update passes every liveness instrument. Proposed fixes enumerated there; provisioning follow-up in comment ID 23027.
## 4. Open work queue (suggested sequence)
1. **Comms tooling migration PR** (lane `next`): move `comms-watcher.sh`, `install-watcher.sh`, `ensure-watcher.sh` into the framework tree → deploy `~/.config/mosaic/tools/comms/`; regenerate existing units' ExecStart to framework paths (one-command sweep); keep `COMMS_WATCH_REPO` per-host config (points at a brain checkout until the queue transport lands). Reference: jarvis-brain commit `701c353b1`.
2. **Ensure-on-instantiation**: `mosaic agent --new` / fleet launch call ensure semantics; `mosaic doctor` gains the drift check (`--status --all` semantics + `fred`'s hand-written unit as the known drift case; also note daphne/docs/happy/pepper/sanity/tiny/fargo currently have no watcher — cover or consciously exempt).
3. **MOSAIC-D-002 implementation** (after ruling): per-agent homes via targeted env vars; roster-generated SOUL.md single-writer; extend the existing `*.env.generated` pattern; launch ledger keeps `config_home` audit.
4. **MOSAIC-D-001 implementation** (after ruling): SYSTEM.md sourcing + per-harness delivery + `compose-contract` becomes render-core+tail with drift check; bench cache-ordering before/after (jarvis-brain `domains/software-dev/mosaic-stack/prompt-testing` has the bench).
5. **Queue transport + forced separation** (longer term): supersedes watcher path; identity-verification and wedge-detection remain valid regardless of transport.
6. **Docs inheritance**: jarvis-brain AGENTS.md's durable comms guidance (E7 pi-glyph delivery gotchas, capture-pane rules, comms protocol) must be inherited into stack docs before P0 retirement.
## 5. Rules for tonight's agents
- Lane: **`next`** only; nothing to `main` without Jason (standing ruling).
- Attribution caveat #1280: Gitea/git identity from this host may misattribute (issue #1295 showed as created by `@mos-dt-0`); prefer per-invocation `git -c user.name=<seat>` and verify what the remote recorded.
- Do not delete `sb-it-1-dt:docs]` (untracked file at repo root) — it is cited fleet-wide as incident evidence.
- Edit user contracts in `~/.mosaic/`, never the templates in `~/.config/mosaic/`.
- Do not restart other fleet seats unilaterally (goals/scrappy/sanity restart decisions are fred's/Jason's per the docs-seat report).
## 6. Artifact map
| Artifact | Where |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| Decision records D-001/D-002 (strict, pending ruling) | jarvis-brain `docs/decisions/mosaic-stack/2026-08-17_mosaic-d-00{1,2}_*.md` |
| Incident issue + provisioning comment | stack #1295 + comment 23027 |
| ensure-watcher.sh (reference implementation) | jarvis-brain `scripts/comms/ensure-watcher.sh` (commit `701c353b1`) |
| Bench for prompt A/B (pi, thinking levels, footer-token semantics) | jarvis-brain `domains/software-dev/mosaic-stack/prompt-testing/` + `docs/reports/2026-08-17-prompt-testing-glm-bench.md` |
| Launcher inspection basis | `@mosaicstack/mosaic` 0.0.49 `dist/commands/launch.js` (composeContract / ensureRuntimeConfig / harness-home isolation) |
| Guardrail edits | `~/.mosaic/{SOUL,AGENTS,STANDARDS}.md` on sb-it-1-dt (2026-08-17 16:5317:04) |
@@ -0,0 +1,66 @@
# CLI/TUI Tools Enhancement Scratchpad
## Objective
Add 5 capability areas to the Mosaic CLI/TUI + gateway agent:
1. Web search tools (multi-provider: Brave, DuckDuckGo, Tavily, SearXNG)
2. File edit tool (`fs_edit_file` with targeted text replacement)
3. MCP management TUI commands (`/mcp status`, `/mcp reconnect`, `/mcp servers`)
4. File reference in chat (`@file` syntax + `/attach` command)
5. Implement `/stop` to cancel streaming
## Plan
### 1. Web Search Tools (gateway agent tools)
- Create `apps/gateway/src/agent/tools/search-tools.ts`
- Providers: Brave Search API, DuckDuckGo (HTML scraping), Tavily API, SearXNG (self-hosted)
- Each provider activated by env var (BRAVE_API_KEY, TAVILY_API_KEY, SEARXNG_URL)
- Tools: `web_search` (unified), `web_search_news` (news-specific)
- Export from `apps/gateway/src/agent/tools/index.ts`
- Wire into `agent.service.ts` buildToolsForSandbox
### 2. File Edit Tool (gateway agent tool)
- Add `fs_edit_file` to `apps/gateway/src/agent/tools/file-tools.ts`
- Parameters: path, edits[{oldText, newText}] — same semantics as pi's Edit tool
- Validates uniqueness of each oldText, applies all edits atomically
### 3. MCP Management Commands (TUI + gateway)
- Add gateway endpoints: GET /api/mcp/status, POST /api/mcp/:name/reconnect
- Add TUI gateway-api.ts functions for MCP
- Add gateway slash commands: /mcp (with subcommands status, reconnect, servers)
- Register in command manifest from gateway
- Handle in TUI via gateway command forwarding (already works)
### 4. File Reference in Chat (@file syntax)
- TUI-side: detect @path/to/file in input, read file contents, inline into message
- Add `/attach <path>` local command as alternative
- gateway-api.ts helper not needed — this is purely client-side pre-processing
- Modify InputBar or sendMessage to expand @file references before sending
### 5. Implement /stop
- Add `abort` event to ClientToServerEvents in @mosaicstack/types
- TUI sends abort event on /stop command
- Gateway chat handler aborts the Pi session prompt
- Update use-socket to support abort
- Wire /stop in app.tsx
## Progress
- [x] 1. Web search tools
- [x] 2. File edit tool
- [x] 3. MCP management commands
- [x] 4. File reference in chat
- [x] 5. Implement /stop
## Risks
- DuckDuckGo has no official API — need HTML scraping or use lite endpoint
- SearXNG needs self-hosted instance
- @file expansion could be large — need size limits
- /stop requires Pi SDK abort support — need to check API
+35
View File
@@ -0,0 +1,35 @@
# Scratchpad — Thin-core prompt diet (#528)
**Branch:** `feat/contract-thin-core` · **Issue:** #528 · **Mode:** Delivery
## Objective
Cut the always-injected contract (`defaults/AGENTS.md` + `defaults/TOOLS.md` + `runtime/claude/RUNTIME.md`, inlined every turn by the launcher) without losing any hard gate. Restore the original "thin core + on-demand guides" intent.
## Change
- `defaults/AGENTS.md` → thin core: 12 hard gates verbatim, 37 operating rules condensed to ~15 bullets (detail already in `guides/E2E-DELIVERY.md`), Superpowers condensed, load order made on-demand (no halt-on-missing for STANDARDS), conditional guide-loading index retained.
- `defaults/TOOLS.md` → index; full catalog moved to new `guides/TOOLS-REFERENCE.md` (read on demand).
- `runtime/claude/RUNTIME.md` → slimmed (dedup tier table, terser pointers).
## Method (autoresearch-style validation)
1. Built a 9-probe role battery (backend/deploy/review/orchestrate/secrets/docs/simple-trap/no-stop-at-PR/agent-work) + a deterministic 18-signature gate-checklist.
2. Headless interactive runs (Claude Max **subscription**, tmux — no API), scored by per-probe rubric.
3. Keep-or-discard hill-climb (token cost gated by per-probe fidelity) proved the method; final design re-derived against THIS repo's content (diet-only, no drifted-deployment content imported).
## Validation evidence
- Gate-checklist: ALL gates + critical rules + mode lines + sequential-thinking + OpenBrain + Superpowers present.
- A/B on real repo content: **thin 7/9 vs monolith 5/9** probes; strictly better on deploy/review/simple-task; composed **8,827 → 4,122 tok (53%)**.
- p11 (don't-stop-at-PR): 3→2/3 on one rubric line — verified a scorer/phrasing artifact (answer correctly cites gates §5/§9 + close-issue; gate verbatim-present). Variance: thin 2/2/3, v0 3/3/3.
## Decisions / risks
- **Diet-only** vs repo content (user decision). Did NOT import web1's Gate 13-15 / federated memory / OpenViking — canonical repo is behind those deployments; flagged for separate reconciliation.
- AGENTS/TOOLS are shared across runtimes → diet benefits codex/pi/opencode too; RUNTIME change is claude-only.
- p11 accepted as-is (user decision) — not gaming the rubric.
## Status
PR open, paused for maintainer merge ratification (fleet-governing change). `mosaic upgrade` will propagate on merge.
@@ -0,0 +1,125 @@
# fix(db): bootstrap migrations on local-tier gateway startup
## Problem
Fresh `mosaic gateway install` (npm-installed) leaves the gateway DB schema empty:
```
relation "users" does not exist
```
Sign-in 500s, `auth users create` says "Not signed in", `admin/bootstrap setup`
also fails — every entry point queries `users` before doing anything else.
## Scope
This PR fixes the **local (PGlite) tier** end-to-end. The postgres-tier path
has additional pre-existing bugs (see "Known issues, out of scope" below) and
needs a separate change with real Postgres validation.
## Root causes addressed (5 stacked bugs on the local-tier path)
1. **`packages/db/package.json` `files: ["dist"]`** — the `drizzle/` SQL
migrations folder is excluded from the published tarball. Even if a
migrate runner existed, it would have nothing to apply.
2. **`packages/db/src/migrate.ts`** only supports `postgres-js`. Local-tier
gateways use embedded PGlite, which can't be reached over a postgres wire
protocol — so `runMigrations()` is unusable for the local tier.
3. **`apps/gateway/src/database/database.module.ts`** never invokes
migrations at startup. The module creates the DB handle and storage
adapter, but no consumer calls `.migrate()` on either. `mosaic storage
migrate` CLI even claims "pglite runs schema setup automatically on first
connection via `adapter.migrate()`" — but `adapter.migrate()` is only
called by tests, never at runtime.
4. **`createPgliteDb` does not load the pgvector extension.** Migration 0001
declares `CREATE EXTENSION IF NOT EXISTS vector;` for the
`insights.embedding` column. Bare PGlite has no pgvector — the migration
fails on extension control file lookup.
5. **Drizzle's PG migrator wraps every migration in one outer transaction.**
Migration 0009 does `ALTER TYPE grant_status ADD VALUE 'pending'` and then
`ALTER TABLE federation_grants ALTER COLUMN status SET DEFAULT 'pending'`.
Postgres' `check_safe_enum_use` rejects the second statement because the
new enum value isn't committed yet. Splitting the migration into two
files doesn't help — drizzle batches all migrations into one outer tx.
## Fix
- `packages/db/package.json` — ship `drizzle/` in `files`.
- `packages/db/src/client-pglite.ts` — load `@electric-sql/pglite/vector`.
- `packages/db/src/migrate.ts` — add `runPgliteMigrations(handle)`. Walks the
Drizzle journal and runs each statement-breakpoint chunk through PGlite's
`client.exec()` (Simple Query protocol → autocommit per statement). Writes
to the standard `drizzle.__drizzle_migrations` ledger so the result is
interoperable with `runMigrations()` on a postgres-backed deployment.
Per-statement try/catch surfaces which statement of which migration failed
and the ledger row is only written on full success.
- `packages/db/src/index.ts` — re-export.
- `apps/gateway/src/database/database.module.ts` — implement `OnModuleInit`:
- Local tier → `runPgliteMigrations(handle)`, then `storageAdapter.migrate()`
(the local storage adapter has its own kv tables in a separate PGlite dir).
- Postgres tier → `storageAdapter.migrate()` only, since
`PostgresAdapter.migrate()` already calls `runMigrations(url)` against
the same DATABASE_URL — we deliberately don't double-call.
NestJS awaits `onModuleInit` before `app.listen()`, so DB-dependent modules
see a populated schema before any HTTP traffic is accepted.
- `packages/storage/src/test-utils/pglite-with-vector.ts`**deleted**.
The "intentionally not exported" rationale is moot now that migration 0001
forces pgvector load anyway. `migrate-tier.integration.test.ts` switched
to `createPgliteDb` + `runPgliteMigrations` from `@mosaicstack/db`.
## Tests
`packages/db/src/migrate.test.ts`:
- Verifies `runPgliteMigrations` creates the BetterAuth tables (the original
failure mode).
- Idempotence (transitively re-runs migration 0009).
- Partial-failure: pre-creates a conflicting `users` table, asserts the
thrown error includes statement context (`hash=… statement #N failed`)
and that no ledger row was written.
## QA evidence
End-to-end on a fresh PGlite install:
- `[DatabaseModule] Applying PGlite schema migrations...` then
`Initializing storage adapter (pglite)...` in startup log.
- `GET /api/bootstrap/status``{"needsSetup":true}` HTTP 200 (was 500
with `relation "users" does not exist`).
- `POST /api/bootstrap/setup` with empty body → HTTP 400 with Zod
validation error (was 500), confirming the request reached the
validator past the table-existence check.
## Known issues, out of scope (file separately)
- **Postgres-tier first install is still broken.** `runMigrations()` uses
Drizzle's `migratePostgres`, which has the same outer-transaction problem
as PGlite's migrator. A fresh standalone-tier install would also fail at
migration 0009. Inline TODO in `migrate.ts:31-35` flags this. Fixing it
needs either (a) a shared per-statement loop reused for both drivers, or
(b) splitting migration 0009.
- **`drizzle/meta/_journal.json` has 0009 ordered before 0008** (`when`
values `1745280000000` < `1776822435828`). `migratePostgres` skips by
`created_at < folderMillis`, so on a postgres deployment that already
applied 0008, 0009 would be skipped forever. Our hash-based skip in the
PGlite path sidesteps this.
- **No advisory lock around the migration loop.** Two gateway processes
pointed at the same DATABASE_URL would race. PGlite is single-process by
file lock so the local tier is fine; postgres-tier deployments should add
`pg_advisory_lock(<deterministic-id>)` around the loop in a follow-up.
- **`mosaic storage migrate` CLI message is misleading** — it claims
"automatic on first connection via adapter.migrate()" but the adapter
doesn't self-migrate. With this PR the gateway invokes it explicitly, but
the CLI message could still be tightened.
- **Crash mid-migration leaves a partial-state PGlite DB without a ledger
row.** Detected loudly on next boot (the replay errors on "already
exists"), but recovery is manual (drop the partially-applied objects or
insert the migration hash into `drizzle.__drizzle_migrations`). A robust
fix would add a "started_at" column to a sidecar table to detect
half-applied state and refuse to start with actionable guidance.
@@ -0,0 +1,45 @@
# RI-1-002 — Publish-gate negative controls (SDLC-D-034 second half)
- Task: RI-1-002 (docs/release-integrity workstream, PRD item RI-N1), issue ref #1275
- Branch: `test/ri-050-publish-gate-negative` (base `origin/next` @ d8e0aec9 = PR #1277, RI-1-001)
- Budget: worker estimate ~45K tokens; keep scoped to the two test files + scratchpad.
## Objective
Checked-in negative-control tests that PROVE the publish gate fails when it must:
1. Broken mandatory check blocks every publish step (structural DAG proof from `.woodpecker/publish.yml`).
2. Bypass shapes fail the checker: missing edge, hidden effect (non-`publish` name), detached verify, always-pass verify (`failure: ignore` / `success` override), conditional verify (`when`).
3. Exact-commit identity: no HEAD-moving step between verify and publish effects; legitimate re-checkout requires verify to re-run after it.
4. `verify-release.mjs` composition control: a SUBSET stage list fails the composition check.
## Plan
- NEW `scripts/publish-gate-structure.test.mjs` — self-contained structural checker (`assertPublishGateBlocksOnVerify`) + positive control on the real pipeline + one negative-control test per bypass shape (S1S6, documented in file header) + positive control for the legitimate re-checkout shape.
- EXTEND `scripts/verify-release.test.mjs` — refactor the stage-mirror test body into `assertStagesMirrorCi(stages, ci)`; add negative control dropping each stage one at a time (subset must throw).
## Conventions confirmed
- Root `test:checkout` = `node --test scripts/*.test.mjs` → new file auto-joins `pnpm test`.
- Test-enumeration guard population is `*test*.sh` under `packages/mosaic/framework/tools/` only → unaffected.
- Root eslint covers only `**/*.{ts,tsx}` → .mjs files need Prettier style only (printWidth 100, singleQuote, semi, trailingComma all).
- Do NOT touch docs/TASKS.md, docs/release-integrity/TASKS.md, docs/scratchpads/.
## Progress log
- [x] Base verified: publish.yml `verify` step + verify-release.mjs present; HEAD contains origin/next.
- [x] Wrote scripts/publish-gate-structure.test.mjs
- [x] Extended scripts/verify-release.test.mjs (mirror fn + subset negative control)
- [x] Gates: node --test scripts (31 tests pass), prettier clean on touched files, pnpm typecheck PASS, pnpm lint PASS, pnpm format:check PASS
- [x] Committed ff585b88 + pushed, PR #1305 → next (no conflicts). Stopped before merge per task instruction.
## Evidence
- `node --test scripts/verify-release.test.mjs scripts/publish-gate-structure.test.mjs` → 31 tests, 0 fail.
- Mutation sanity: temporarily removing the `verify` edge from build-gateway in publish.yml → structure test goes red (verified manually during dev, then reverted).
- Gates run from repo root on this worktree; results in Progress log.
## Risks / notes
- Effect detection (`isPublishCommand`) is deliberately over-broad (any npm/pnpm/yarn command mentioning `publish`, any kaniko/docker-push/`--destination`) — fail-closed: a false positive forces justification, a false negative is the actual hazard.
- `git fetch` flagged as HEAD-moving even though fetch alone doesn't move HEAD — fail-closed on the classic `fetch && reset` pair.
+37
View File
@@ -0,0 +1,37 @@
# Scratchpad — RI-4-001 One transitional PRD authority (RI-N3, #1275)
- Objective: single PrdService authority in `@mosaicstack/prdy`; `mosaic prdy` and
`mission --plan` become thin adapters; mission↔PRD linkage persisted on disk;
Markdown export is a labeled generated view (never read back); import is
validated/conflict-aware with typed refusals.
- Budget: ~35K tokens (card cap). Baselines: prdy build/lint rc=0, 0 tests;
mosaic build rc=0 (after root turbo build), lint rc=0, 1548 tests pass;
root build rc=0.
- Plan: (1) extend store schema (version, missions linkage) (2) PrdService +
typed errors (3) wizard/cli route through service (4) mosaic adapters
(5) contract specs both packages (6) gates (7) sabotage control (8) report
to /var/tmp/ri-050/ri-4-001-report.md.
- Decisions:
- Linkage lives ON the PRD document (`missions` array) — one authority file,
survives restart, no sidecar sync problems.
- `version` = content revision of sections/status (bumped by update/import
accept). Linkage writes bump `updatedAt` only, so ids/versions stay stable
for the card's "stable ids/versions" contract.
- Mission version marker = `mission.updatedAt` (gateway MissionInfo has no
numeric version field).
- Import reads YAML documents only — never the exported Markdown (keeps the
"no code path reads exported Markdown" invariant).
- Import of an existing id with identical core content → `identical` no-op;
divergent → typed `PrdImportConflictError` carrying proposed successor
(existing.version + 1, status draft, linkages preserved). Original bytes
untouched until explicit `acceptSuccessor`.
- `requirementIds` default `[]` at the mission command (no requirement
selection UI yet) — service accepts ids when a caller has them.
- Progress log:
- [16:35] baselines captured (prdy 0 tests; mosaic 1548 after root build; root build rc=0)
- [16:38] store schema v2 + PrdService + wizard/cli rerouted; prdy build/lint green
- [16:40] mosaic adapters done; prdy spec 20/20 (found+fixed: import project-path leak, empty-store typed error, YAML timestamp coercion)
- [16:44] mosaic specs 9/9 (fixed commander from:'user' argv, vi.mock hoisting, restoreAllMocks wiping factory mocks)
- [16:45] all gates green; 4 commits (e291bfb, 2c5d208, a23826c, 540d6f1)
- [16:46] sabotage: linkage write removed → prdy 3 fail / mosaic 2 fail, 1548/1548 pre-existing pass; restored byte-identically; re-green 20/20 + 1557/1557
- [16:47] report written to /var/tmp/ri-050/ri-4-001-report.md — card complete