196 lines
30 KiB
Markdown
196 lines
30 KiB
Markdown
# DOCS-IA-002 — Documentation Catalog and Truth Audit
|
|
|
|
- **Task:** DOCS-IA-002
|
|
- **Internal reference:** `TASKS:DOCS-IA-002`
|
|
- **Plan:** `docs/plans/2026-08-10-docs-catalog-audit.md`
|
|
- **Objective:** Catalog all documentation and audit validity/truthfulness before migration.
|
|
- **Mode:** Read-only discovery; coordinator reconciles findings.
|
|
|
|
## Parallel Pi audit lanes — 2026-08-10
|
|
|
|
All lanes used `pi --model openai-codex/gpt-5.6-luna:max` in detached worktrees with only `read` and `bash` tools. No lane edited files, staged changes, committed, or started services.
|
|
|
|
| Lane | Assignment | Outcome |
|
|
| -------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `docs-prd` | Audit `docs/PRD.md`, linked workstream documents, requirements claims, and package/source alignment. | Completed. PRD remains draft/normative; broken fleet/Kanban links and held/contradicted claims require human disposition before promotion. |
|
|
| `docs-api` | Compare `docs/openapi-tess.yaml` with gateway controllers, routes, DTOs, auth guards, and tests. | Completed. Tess contract is scoped and incomplete for the gateway; API migration requires a full route/schema/auth/error inventory and human scope decision. |
|
|
| `docs-control` | Audit `docs/MISSION-MANIFEST.md` and `docs/TASKS.md` without changing orchestrator-owned state. | Completed. Active versus historical MVP/workstream status is unresolved; status, archive/rehome, and control-plane repairs remain orchestrator/human-owned. |
|
|
| `docs-source` | Map live consumers of `docs/fleet`, `docs/federation`, `docs/architecture`, `docs/tess`, and `docs/native-kanban-sot`. | Completed. Legacy roots are absent from current `docs/`; architecture/fleet have source/test consumers, while Tess/Kanban are primarily control-plane/navigation dependencies. Recommended order starts with authority decisions, then architecture security, fleet consumers, and API/Tess migration. |
|
|
|
|
The four isolated worktrees were clean and removed after report collection.
|
|
|
|
## Autonomous finish-line orchestration — 2026-08-10
|
|
|
|
- **Coordinator session:** `019fed44-977c-73b9-b14a-58de29c0cb51`
|
|
- **Recovery anchor:** Resume from this section and `git log`; completed slices are immutable scoped commits, while active lane reports are advisory until reconciled here.
|
|
- **Worker policy:** Isolated worktrees, read-only evidence collection, no services, no database access, no edits to `docs/TASKS.md`, `docs/MISSION-MANIFEST.md`, `docs/PRD.md`, or `.mosaic/orchestrator/*`.
|
|
- **Decision policy:** Migrate only evidence-backed current contracts or clearly historical artifacts. Queue product/control-plane/API-authority decisions for humans.
|
|
|
|
| Lane | Assignment | State |
|
|
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `docs-channel` | Verify the archived channel protocol against shared DTOs, gateway/Discord implementation, and tests; identify safe canonical destination and contradictions. | Complete — rewrite required; shared DTO and Discord baseline are current, while registry, Telegram parity, Matrix, identity linking, and multiplexing claims are draft or contradicted. |
|
|
| `docs-runtime` | Classify the remaining runtime-portability ADR and M1 architecture pages against source/tests and live references. | Complete — egress ADR remains a draft RFC; M1 lease/fencing is an implemented decision requiring precision corrections and a held operations companion. |
|
|
| `docs-reports` | Classify archived reports/checklists into exact evidence/archive destinations and identify references that must move with them. | Complete — 13 evidence artifacts classified; the byte-identical #756 batch was selected and migrated. |
|
|
| `docs-guides` | Identify the next safe user/admin/developer guide slices, verifying commands and held PostgreSQL/runtime boundaries. | Complete after automatic compaction — upgrade/recovery is the next safe rewrite; deployment, tier migration, Gateway/Web activation, and PostgreSQL remain held. |
|
|
|
|
### Implementation lanes
|
|
|
|
| Lane | Branch | Owned slice | State |
|
|
| ------------------- | --------------------------------- | ------------------------------------------------------ | ------------------------------------- |
|
|
| `docs-channel-impl` | `docs/channel-protocol-migration` | Rewritten current channel architecture page only. | Complete; cherry-picked as `00bdf8b`. |
|
|
| `docs-runtime-impl` | `docs/runtime-m1-migration` | M1 decision and held connector-lease operations guide. | Complete; cherry-picked as `631567d`. |
|
|
| `docs-upgrade-impl` | `docs/upgrade-recovery-migration` | Verified local upgrade/recovery administrator guide. | Complete; cherry-picked as `39987a5`. |
|
|
| `docs-reports-impl` | `docs/report-evidence-migration` | Four byte-identical historical evidence moves. | Complete; cherry-picked as `9f74187`. |
|
|
|
|
Shared indexes, navigation, audit records, integration review, and final commits remain coordinator-owned.
|
|
|
|
### Implementation wave 2
|
|
|
|
| Lane | Branch | Owned slice | State |
|
|
| ---------------------- | --------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
|
| `docs-channel-guides` | `docs/channel-audience-guides` | Discord admin, user workflow, and adapter-authoring pages. | Complete; cherry-picked as `f4faa3f`. |
|
|
| `docs-lease-ops` | `docs/lease-operations-migration` | Current/held lease-broker operational boundary. | Complete; cherry-picked as `6b3ebce`, then placed under the canonical testing chapter. |
|
|
| `docs-egress-rfc` | `docs/egress-rfc-migration` | Explicitly draft optional egress gateway RFC. | Complete; cherry-picked as `0692d99`. |
|
|
| `docs-dashboard-guide` | `docs/dashboard-guide-migration` | Verified current web-dashboard user page. | Complete; cherry-picked as `7fa0f65`. |
|
|
|
|
### Final review lanes
|
|
|
|
| Lane | Assignment | State |
|
|
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `docs-nav-review` | Independently audit current-page ownership, reachability, relative links, lifecycle labels, and sitemap accuracy. | Complete — findings remediated: artifact indexes added, quarantine links removed, stale sitemap copy corrected, mission label no longer overrides control state. |
|
|
| `docs-safety-review` | Independently review new current/held pages against source/tests and database/runtime safety constraints. | Complete — Discord durability blocker found; all affected pages now limit claims to live routing/delivery and disclose missing route-to-UUID persistence evidence. |
|
|
| `docs-stop-review` | Confirm that remaining unmigrated groups require authority decisions or coupled source/test migrations, and identify any missed safe slice. | Complete — historical mission bundle migrated; remaining Fleet, Federation, Kanban, Tess/API, and deployment groups are authority- or source/test-coupled. |
|
|
|
|
## Progress
|
|
|
|
- [x] Confirmed external fleet sessions are standing down; no shared agent session was repurposed.
|
|
- [x] Established parallel read-only discovery lanes.
|
|
- [x] Counted current and archived documentation files.
|
|
- [x] Ran initial link audit.
|
|
- [x] Complete corrected code-surface audit.
|
|
- [x] Complete truth triage probes.
|
|
- [x] Write human-readable catalog/audit report.
|
|
- [x] Verify report formatting and scope.
|
|
- [x] Commit audit artifacts without unrelated working-tree changes.
|
|
- [x] Scaffold audience, architecture, and API indexes without moving legacy content.
|
|
- [x] Add current scaffold entry points to `SITEMAP.md` without rewriting its legacy sections.
|
|
- [x] Select the P8-003 performance report as the first low-risk migration slice.
|
|
- [x] Archive the contradicted TUI PRD/task pair with unchanged content and an explicit archive index.
|
|
- [x] Create and promote the verified user quickstart from the empty root placeholder.
|
|
- [x] Rewrite and promote the SSO administrator runbook from current auth/web source evidence.
|
|
- [x] Promote the lease-broker protocol, security, mutator-gate, and compaction-revocation contracts with their navigation and acceptance-test consumers.
|
|
- [x] Remove the blanket `docs/reports/` ignore rule so canonical evidence can be staged and linted normally.
|
|
- [x] Migrate the three-file #756 Discord evidence batch byte-identically and add a reports index.
|
|
- [x] Complete and integrate the four implementation-agent slices after independent scope and evidence review.
|
|
- [x] Complete wave 2 audience, testing, RFC, and dashboard migrations and connect every page to its owning index and sitemap.
|
|
- [x] Replace the stale sitemap link inventory with fully resolvable current navigation and an authority-gated backlog summary.
|
|
- [x] Complete independent navigation, safety, and stop-condition reviews; remediate every autonomous finding.
|
|
- [x] Migrate the bounded historical mission archive and repair its two broken sibling links.
|
|
- [x] Archive the unreferenced five-file monorepo-consolidation planning bundle byte-identically with explicit historical boundaries.
|
|
- [x] Archive ten unreferenced legacy plans/deferred stubs byte-identically; retain the live-test-coupled Agent Reflection PRD and authority-gated WebUI/Fleet draft in quarantine.
|
|
- [x] Archive 94 unreferenced historical scratchpads byte-identically; retain 17 records coupled to control docs, tests/fixtures, mission/evidence records, or KBN-101.
|
|
- [x] Archive the two unreferenced Matrix/MACP RFCs byte-identically as historical drafts, cross-referenced to the canonical unimplemented-Matrix boundary.
|
|
- [x] Archive two unreferenced standalone designs byte-identically; retain the coupled #791 design and normative framework constitution in quarantine.
|
|
- [x] Migrate three #751 Native Kanban review reports byte-identically to their already-referenced canonical evidence paths; retain KBN-101 and Fleet reports with gated workstreams.
|
|
- [x] Reconcile PR #1210 with current `next`: restore the complete Fleet executable book, active Native Kanban/KBN SSOT, KBN-101 reports, and eight protected #1187 hold sites byte-exactly from provider base `216cd722`.
|
|
|
|
## PR #1210 integration reconciliation
|
|
|
|
CI pipeline 2405 proved that `docs/fleet/**` is an executable contract: Fleet tests and CLI projections fail when the book, north-star YAML/Markdown, example roster, or JSON schema is moved. Fred approved restoration and required the active KBN SSOT plus the eight #1187 hold-site documents to remain canonical. Scrappy supplied and will re-review the protected-path census.
|
|
|
|
All protected files were restored from `origin/next@216cd722`, not rewritten. The protected developer guide's Fleet-canary dependency was also restored; its sole stale upgrade-guide link was reconciled to the current administrator-guide destination. Scrappy corrected an initial provenance-pin instruction: `docs/native-kanban-sot/KBN-101-DB-ROLE-SPLIT.md` is provider-base blob `2bb6209909004b74490565d5def9956aba6d83ca`; the void `45e4…` pin belongs to a later unmerged KBN RED-branch preimage and is intentionally absent from PR #1210.
|
|
|
|
No documentation-only relocation of these surfaces is authorized. A future move requires explicit Fleet consumer migration or Fred/Task-18 KBN authority reconciliation.
|
|
|
|
## Initial findings
|
|
|
|
- Baseline inventory: 283 documentation artifacts — 11 current root files, 269 archived files, two prior plans, and one prior scratchpad; current audit artifacts are listed separately.
|
|
- Archived inventory: 269 files under `docs/_old_structure/`, moved unchanged by commit `cd4409a`.
|
|
- Initial link scan: 219 internal relative links/wikilinks, 98 unresolved under the baseline checkout; 83 current legacy links remain after excluding two intentional README blueprint links, and 12 archive links are unresolved.
|
|
- 31 source/test/framework files reference legacy documentation roots such as `docs/fleet/`, `docs/federation/`, or `docs/architecture/`.
|
|
- `docs/QUICKSTART.md` is empty; `docs/SITEMAP.md` has 66 broken current links; `docs/PRD.md`, `docs/TASKS.md`, and `docs/MISSION-MANIFEST.md` retain broken pre-archive references.
|
|
- SSO provider environment names and partial-config behavior align with `packages/auth`, but the documented `NEXT_PUBLIC_*_ENABLED` web flow is stale against dynamic `/api/sso/providers` discovery.
|
|
- TUI documents reference missing `packages/cli` and a missing historical worktree; current TUI code is under `packages/mosaic`.
|
|
- `docs/openapi-tess.yaml` parses as OpenAPI 3.1 with 17 paths, but remains a legacy root-level Tess-scoped contract rather than the full API SSOT.
|
|
- The baseline `.gitignore` ignored `docs/reports/`, conflicting with the new documentation contract and lint-staged. The blanket rule is now removed so canonical evidence can be tracked normally.
|
|
- The scaffold slice adds `USER-GUIDE/README.md`, `ADMIN-GUIDE/README.md`, `ADMIN-GUIDE/security/README.md`, `DEVELOPER-GUIDE/README.md`, `DEVELOPER-GUIDE/architecture/README.md`, and `API/README.md`; legacy pages remain unchanged.
|
|
- `SITEMAP.md` links the protected Fleet and KBN authority surfaces at their canonical paths. Remaining authority-gated Federation, Kanban task-state, Tess/API, and deployment groups are summarized without presenting quarantine paths as current navigation.
|
|
- `docs/PERFORMANCE.md` had no live source/test dependency or internal links; it moved unchanged to `docs/reports/qa/p8-003-performance-optimization.md`. The admin index and current sitemap now classify it as historical evidence, not an operator runbook.
|
|
- `PRD-TUI_Improvements.md` and `TASKS-TUI_Improvements.md` had no live source/test dependencies beyond their internal relative link; both moved unchanged to `docs/archive/tui/`, and `docs/archive/README.md` labels them historical.
|
|
- The empty `docs/QUICKSTART.md` placeholder was replaced by `docs/USER-GUIDE/getting-started/quickstart.md`, based on verified installer, wizard, CLI, and runtime-safety behavior. The page explicitly excludes held PostgreSQL/source-checkout routes.
|
|
- The SSO guide now documents Authentik/WorkOS/Keycloak discovery, OIDC callbacks, Keycloak SAML fallback, partial-config failures, and the absence of frontend feature flags. The root guide and stale `.env.example` flags were removed.
|
|
- The lease-broker acceptance suites directly consumed two quarantined `docs/architecture/` paths; promoting the protocol, security, mutator-gate, and compaction-revocation contracts to `DEVELOPER-GUIDE/architecture/` and updating those consumers restored the intended contract family without changing runtime behavior.
|
|
- The #756 Discord code-review, security-review, and documentation-checklist reports had no source/test consumers and moved byte-identically to their canonical report categories. `docs/reports/README.md` labels their historical evidence boundary.
|
|
- Independent safety review found that Discord external route strings are not proven to map to UUID conversation rows. Current pages now promise live routing/delivery only, disclose that persistence can fail while dispatch continues, and require pre-enrollment for privileged durable-session controls.
|
|
- The monorepo-consolidation brief, board review, and three work-package specifications had no live consumers or internal links. They moved byte-identically to `archive/planning/`; current package existence supports historical classification but does not prove every old acceptance criterion.
|
|
- Ten additional unreferenced implementation plans, setup records, and deferred stubs moved byte-identically to `archive/planning/legacy/`. `agent-reflection-loop-PRD.md` was excluded because a MACP test names its intended canonical path; the WebUI/Fleet bridge draft was excluded because it remains authority-gated.
|
|
- Of 111 quarantined scratchpads, 94 had no current path/name consumers and no coupled local Markdown links; they moved byte-identically to `archive/work-records/`. Seventeen remain quarantined with their control-document, source/test fixture, mission/evidence, or KBN-101 dependencies.
|
|
- The two Matrix/MACP RFCs were explicitly draft, unreferenced, and link-free. They moved byte-identically under `archive/planning/matrix-macp/`; the archive index points to the canonical channel page that records Matrix as unimplemented.
|
|
- The npm prerelease-lane and storage-abstraction designs had no current consumers or local links. They moved byte-identically under `archive/planning/designs/` with explicit source/test verification boundaries.
|
|
- Three #751 Native Kanban review reports moved byte-identically to `reports/native-kanban-sot/`, matching paths already named by the archived #751 scratchpad. Their historical GO verdicts do not decide current authority or KBN-101 execution.
|
|
|
|
## Method
|
|
|
|
- Resolve relative Markdown links from the source page directory.
|
|
- Resolve wikilinks from the `docs/` vault root.
|
|
- Compare document claims to current source, package manifests, tests, configuration, and executable paths.
|
|
- Treat normative PRDs and plans as intent, not shipped behavior.
|
|
- Treat archived material as historical unless current source/tests demonstrate it remains operative.
|
|
|
|
## Risks and blockers
|
|
|
|
- `docs/PRD.md`, `docs/TASKS.md`, and `docs/MISSION-MANIFEST.md` retain authority-owned links that require maintainer/orchestrator disposition. Fleet source/test consumers are satisfied by retaining the executable book at `docs/fleet/`; Federation relocation remains coupled.
|
|
- Historical documents may contain commands or security claims that are unsafe if resurfaced as current guidance.
|
|
- Truth audit is static unless a claim can be verified without starting held services or violating database/runtime safety rules.
|
|
- `docs/TASKS.md` is single-writer orchestrator state and will not be edited by this task.
|
|
- After restoring active Fleet/KBN/#1187 surfaces and the Fleet-canary dependency, 65 quarantine files remain. Every remaining group is authority-, source/test-consumer-, safety-, or path-coupled; the Mos-gated compaction probes additionally hard-code their present repository-relative location and must not be moved or run independently.
|
|
- The pre-existing `9a1cc63` commit removed `docs/GETTING_STARTED.md` and added the `.obsidian` ignore rule before this audit commit; that state is not attributed to this task.
|
|
|
|
## Verification log
|
|
|
|
| Check | Result |
|
|
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| File inventory | PASS — 11 root files, 269 archived files, plus task artifacts |
|
|
| Initial link audit | FINDING — 219 links, 98 unresolved |
|
|
| Code-surface audit | PASS — corrected manifest/script and legacy-reference scans; first attempt was discarded due to a `require()` path bug |
|
|
| Truth probes | PASS — root docs, SSO, performance, TUI, OpenAPI, and source-path evidence triaged |
|
|
| Final report | Written — `docs/reports/documentation/2026-08-10-docs-catalog-audit.md` |
|
|
| Scoped commit | PASS — audit plan, report, and scratchpad committed together; `.mosaic/orchestrator/*` remains outside the commit |
|
|
| Report formatting and scope | PASS — `pnpm exec prettier --ignore-path /dev/null --check` passed; baseline ignored-path conflict was identified and later resolved |
|
|
| Commit hook behavior | RESOLVED — lint-staged exposed the blanket `docs/reports/` ignore conflict; the rule was removed and report paths can now be staged normally |
|
|
| Index scaffold | PASS — six indexes formatted, all new Markdown/wikilinks resolve, and target directories exist; legacy pages were not changed |
|
|
| Sitemap scaffold | PASS — additive current section is formatted, all seven links resolve, and legacy sections are unchanged |
|
|
| First migration selection | PASS — P8-003 report selected as a no-rewrite evidence migration; live references identified before move |
|
|
| P8-003 migration | PASS — destination is byte-identical, live references were updated, formatting and navigation checks pass, and only the scoped migration files are pending commit |
|
|
| TUI archive migration | PASS — both pages are byte-identical, internal links and archive navigation resolve, formatting passes, and only the scoped archive files are pending commit |
|
|
| Quickstart migration | PASS — current page content is source-checked, held routes are labeled, navigation resolves, root placeholder is removed, and formatting passes |
|
|
| SSO migration | PASS — auth/gateway/web SSO tests pass (15+2+2), source claims and navigation resolve, obsolete env flags are removed, and the root guide is retired |
|
|
| Parallel Pi audits | PASS — four requested `openai-codex/gpt-5.6-luna:max` read-only lanes completed in clean isolated worktrees; no agent changes were imported |
|
|
| Lease-broker architecture | PASS — four contract pages promoted, two source/test consumers and navigation updated, 20/20 Vitest acceptance tests passed, and the 3-test Python portability suite passed |
|
|
| #756 evidence migration | PASS — three reports moved byte-identically by SHA-256, no source/test consumers found, reports index and sitemap updated |
|
|
| Implementation agents | PASS — four isolated Luna lanes committed disjoint owned slices; coordinator reviewed and cherry-picked each commit |
|
|
| Second report batch | PASS — four historical evidence files moved byte-identically and added to the reports index |
|
|
| Channel architecture | PASS — verified DTO/Discord behavior is separated from draft Telegram, Matrix, registry, identity, and multiplexing work |
|
|
| Runtime portability M1 | PASS — current decision precision corrected; companion operations page is explicitly held/non-operative |
|
|
| Upgrade/recovery guide | PASS — installed-CLI/local-PGlite route only; PostgreSQL, Compose, migration, and Gateway/Web activation remain held |
|
|
| Implementation wave 2 | PASS — four isolated Luna lanes produced scoped commits; coordinator reviewed, cherry-picked, and integrated each page |
|
|
| Channel audience guides | PASS — Discord admin/user/developer pages align to shared source/tests and explicitly deny Telegram/Matrix parity claims |
|
|
| Lease-broker verification | PASS — safe static/test workflow is current; startup, recovery, mutation, service management, and cleanup remain held |
|
|
| Optional egress RFC | PASS — proposal remains draft/non-operative; model and runtime-provider boundaries are separated and candidates are not integrated |
|
|
| Web dashboard guide | PASS — current routes, views, chat persistence, and absent project/task creation controls are documented from source/tests |
|
|
| Sitemap refresh | PASS — 54/54 relative links resolve after PR #1210 protected-surface reconciliation |
|
|
| Canonical link audit | BLOCKED ONLY BY CONTROL DOCS — 443 links scanned, 10 unresolved; all are in orchestrator-owned `TASKS.md` or `MISSION-MANIFEST.md` |
|
|
| Navigation review | PASS AFTER REMEDIATION — plan/scratchpad indexes added, current quarantine links removed, lifecycle/status copy aligned |
|
|
| Safety review | PASS AFTER REMEDIATION — Discord pages no longer claim unproven ordinary persistence or restart continuity |
|
|
| Stop-condition review | AUTONOMOUS COMPLETE — historical mission bundle migrated; remaining groups require authority or coupled source/test work |
|
|
| Repository pre-push checks | PASS — `pnpm preflight`, `pnpm typecheck`, `pnpm lint`, and `pnpm format:check` with `DATABASE_URL` unset |
|
|
| Root test suite | PR #1210 CI pipeline 2405 failed because the migration removed executable `docs/fleet/` contracts; provider-base restoration is the required amendment. |
|
|
| Fleet contract restoration | PASS LOCALLY — focused north-star, Fleet documentation, and roster-schema suites pass 77/77 after exact provider-base restoration. |
|
|
| Focused final tests | PASS — types 71, Discord 44, web 33, focused gateway 32, and lease-broker/mutator acceptance 57; `DATABASE_URL` unset, no services started |
|
|
| Historical mission archive | PASS — 11 completed/superseded records moved under `archive/missions/`, indexed, and two broken sibling links repaired |
|
|
| Historical planning archive | PASS — five consolidation records plus ten legacy plans/stubs moved byte-identically under `archive/planning/`; hashes and zero-consumer scans verified |
|
|
| Historical work records | PASS — 94 unreferenced scratchpads moved byte-identically under `archive/work-records/`; 17 coupled records deliberately retained in quarantine |
|
|
| Matrix/MACP draft RFCs | PASS — two unreferenced, link-free drafts moved byte-identically under `archive/planning/matrix-macp/`; canonical Matrix limitation linked |
|
|
| Standalone design archive | PASS — prerelease-lane and storage-abstraction designs moved byte-identically under `archive/planning/designs/`; coupled/normative designs excluded |
|
|
| #751 Kanban review evidence | PASS — initial NO-GO, re-review GO, and final GO moved byte-identically under `reports/native-kanban-sot/`; existing evidence paths now resolve |
|