docs: concept annexation, provider/reference docs, ACT-1 groundwork
Mosaic concepts pages now own the adapted content; source/license metadata under docs/reference/concepts. Adds ACT-1 agent-context planning capture, pinned concept test package + preparation utility, foundation observation notes (durability, evidence, federation, onboarding, workflow), and the #1495 consolidation assessment. TOOLS.md updated for the host-dev launcher.
This commit is contained in:
@@ -0,0 +1,116 @@
|
||||
# ACT-1 concept adaptation tests — Darkwing handoff
|
||||
|
||||
Status: Mosaic concept documentation adopted; synthetic scenarios and preparation utility ready.
|
||||
Behavioral trials NOT RUN. Proposed runtime features NOT IMPLEMENTED by this pack.
|
||||
Owner: Jason. Preparation author: Codex. Trial operator/reviewer: unassigned;
|
||||
Darkwing is the intended collaborator, not automatically retasked.
|
||||
|
||||
Parent: [ACT-1 plan](../2026-09-07_agent-context-templates-and-migration.md).
|
||||
Concepts: [Mosaic Stack concepts](../../concepts/README.md).
|
||||
|
||||
## Authority and demo boundary
|
||||
|
||||
Jason requested pulling the selected references and preparing testing with
|
||||
Darkwing. This authorizes this preparation alongside the existing demo work.
|
||||
It does not authorize changing deployed SOUL/contracts, launching model trials,
|
||||
retasking the active Darkwing/Rocko/Filbert sessions, registering automations,
|
||||
or implementing every documented concept. Read CURRENT.md before scheduling tests.
|
||||
ACT-1's owner demo and migration gates remain in effect.
|
||||
|
||||
Do not use `agents/darkwing/launch.sh --fresh` for an isolated trial: it still
|
||||
uses the repository and Darkwing's native state directory. It is not a test
|
||||
isolation command. This pack creates no replacement launcher or harness.
|
||||
|
||||
## Prepare and inspect now
|
||||
|
||||
From the repository root:
|
||||
|
||||
```sh
|
||||
node scripts/prepare-concept-tests.mjs --check
|
||||
node --test scripts/test-darkwing-launch.mjs
|
||||
node scripts/prepare-concept-tests.mjs
|
||||
```
|
||||
|
||||
The first command validates current concept hashes and the scenario inventory without
|
||||
writes. The second runs the existing launcher regression suite against test
|
||||
doubles in temporary directories; it does not call Docker or a model. The third
|
||||
creates a private, uniquely named workspace under the OS temporary directory
|
||||
and prints its absolute path. No existing directory is reused or cleaned up.
|
||||
|
||||
The prepared workspace contains:
|
||||
|
||||
- `baseline/SOUL.md`: a byte copy of Darkwing's current SOUL, for a baseline trial.
|
||||
- `candidate/SOUL.md`: a separate proposed personality for a candidate trial.
|
||||
- `fixtures/`: synthetic inputs only, including a fictional USER profile.
|
||||
- `concepts/`: Mosaic's conceptual documentation at the recorded content hashes.
|
||||
- `provenance/`: original attribution, license and current/source identity manifest.
|
||||
- `cases.json`: eleven prompt/expectation records, each linked to a concept.
|
||||
- `RESULTS.json`: NOT_RUN rows, evidence fields, provenance, and unset live budget.
|
||||
|
||||
Baseline and candidate are separate experimental configurations. A trial injects
|
||||
exactly one SOUL; never combine them or promote either automatically. The candidate
|
||||
is not the approved bootstrap template and has no effect on active Darkwing.
|
||||
Temporary storage may be cleaned by the OS; preserve any actual evaluation
|
||||
evidence in an assigned durable location before relying on it for acceptance.
|
||||
|
||||
## Review with Darkwing
|
||||
|
||||
Give Darkwing this file's path when the demo work permits. A suggested initial
|
||||
request follows; no message was sent automatically:
|
||||
|
||||
> Review ACT-1's concept test package and Mosaic concept documentation. Inspect the prepared
|
||||
> synthetic fixtures and identify which expectations fit our accepted foundation
|
||||
> contracts. Do not change running sessions, launch inputs, or the demo candidate.
|
||||
> Return a source-backed test-readiness review, gaps, and a proposed bounded trial
|
||||
> using the existing Mosaic test/launch mechanisms in an isolated workspace.
|
||||
> Do not claim the proposed runtime features exist or mark unexecuted cases passed.
|
||||
|
||||
For a later assigned behavioral trial, record the exact harness/version,
|
||||
provider/model, tool allowlist, instruction hashes, fresh test-session identity,
|
||||
output/evidence directory, operator, reviewer, and model-call/token/cost budget.
|
||||
Budget is currently unspecified; the preparation utility makes zero model calls.
|
||||
Authorize an actual evaluation binding before execution. Never borrow a live
|
||||
session or credential file by copying it into the fixture workspace.
|
||||
|
||||
Use a separate test session per case and per variant to avoid prior-answer
|
||||
contamination. Inject one SOUL and synthetic user context, provide only the case
|
||||
prompt and its fixture, and keep the `expected` rubric out of the model's input.
|
||||
The full cases file is for the reviewer. Give the test agent read access to its
|
||||
fixture only where the selected harness supports enforcement; a prompt asking
|
||||
for restricted access is not itself a sandbox. Disable extensions, scheduling,
|
||||
messaging and write tools for these initial reasoning/style cases.
|
||||
|
||||
Record the actual response and judge each listed expectation. A hard correctness
|
||||
or authority error fails the case even if the tone is excellent. Jason separately
|
||||
judges useful brevity, candor and personality; do not automate that judgment away.
|
||||
Repeat ambiguous cases under the recorded budget, retaining every attempt.
|
||||
|
||||
## Coverage and limits
|
||||
|
||||
| Cases | What a model trial can assess | What still needs implementation testing |
|
||||
|---|---|---|
|
||||
| C01 | Correctly reading effective-context evidence | Actual context resolver, single-SOUL rule, truncation refusal |
|
||||
| C02 | Concision and directness | No runtime feature claim |
|
||||
| C03 | Protecting active/unknown-owner work in a recommendation | Real worktree leases, snapshot restore and cleanup refusal |
|
||||
| C04 | Recognizing stale state, scope changes and pauses | Durable events, watchers, history gaps and restart reconciliation |
|
||||
| C05 | Distinguishing queued, running and canceled work | Real harness steering and tool-start boundaries |
|
||||
| C06, C11 | Separating identity/attribution from access/control | Two-client attachment and authorization enforcement |
|
||||
| C07 | Honest interpretation of adapter capability evidence | Pinned adapter conformance tests |
|
||||
| C08 | Honest completion reporting | No substitute for actual product acceptance |
|
||||
| C09 | Memory source skepticism and deletion limits | Memory admission/lineage/deletion implementation |
|
||||
| C10 | Distinguishing events, schedules and aspirations | Authorized wake registration, matching and cancellation |
|
||||
|
||||
Case IDs in `cases.json` are authoritative. Runtime obligations in `runtimeGate`
|
||||
remain DEFERRED until an implementation and assignment exist. Model answers to
|
||||
synthetic fixtures must never be reported as passing those runtime gates.
|
||||
|
||||
Allowed result states: NOT_RUN, PASS, FAIL, BLOCKED, DEFERRED. PASS requires
|
||||
actual output, environment and reviewer evidence. Existing launcher test success
|
||||
does not populate the model-case results. No external sends, real-user profile
|
||||
reads, live source writes, or destructive scenarios are part of this pack.
|
||||
|
||||
## Next handoff
|
||||
|
||||
Darkwing reviews readiness after reconciliation with the current demo. Jason
|
||||
selects the live test timing/scope; actual execution records its own evidence.
|
||||
Any migration implementation remains a separately bounded ACT-1 increment.
|
||||
Reference in New Issue
Block a user