bootstrap/guides/QA-TESTING.md

QA & Testing Guide

Before Starting

1. Check assigned issue: ~/.config/mosaic/rails/git/issue-list.sh -a @me
2. Create scratchpad: docs/scratchpads/{issue-number}-{short-name}.md
3. Review docs/PRD.md or docs/PRD.json as the requirements source.
4. Review acceptance criteria and affected change surfaces.

Testing Policy (Hard Rules)

1. Situational testing is the PRIMARY validation gate.
2. Baseline testing is REQUIRED for all software changes.
3. TDD is risk-based and REQUIRED only for defined high-risk change types.
4. Tests MUST validate requirements and behavior, not only internal implementation details.

Priority Order

1. Situational tests: prove requirements and real behavior on changed surfaces.
2. Baseline tests: lint/type/unit/integration safety checks.
3. TDD discipline: applied where risk justifies test-first workflow.

Risk-Based TDD Requirement

Change Type	TDD Requirement	Required Action
Bug fix	REQUIRED	Write a failing reproducer test first, then fix.
Security/auth/permission logic	REQUIRED	Write failing security/permission-path test first.
Critical business logic or data mutation rules	REQUIRED	Write failing rule/invariant test first.
API behavior regression	REQUIRED	Write failing contract/behavior test first.
Low-risk UI copy/style/layout	OPTIONAL	Add verification tests as appropriate; TDD recommended, not mandatory.
Mechanical refactor with unchanged behavior	OPTIONAL	Ensure regression/smoke coverage and situational evidence.

If TDD is not required and skipped, record rationale in scratchpad. If TDD is required and skipped, task is NOT complete.

Baseline Test Requirements

For all software changes, run baseline checks applicable to the repo:

1. lint/static checks
2. type checks
3. unit tests for changed logic
4. integration tests for changed boundaries

Situational Testing Matrix (Primary Gate)

Change Surface	Required Situational Tests
Authentication/authorization	auth failure-path tests, permission boundary tests, token/session validation
Database schema/migrations	migration up/down validation, rollback safety, data integrity checks
API contract changes	backward compatibility checks, consumer-impact tests, contract tests
Frontend/UI workflow changes	end-to-end flow tests, accessibility sanity checks, state transition checks
CI/CD or deployment changes	pipeline execution validation, artifact integrity checks, rollback path check
Security-sensitive logic	abuse-case tests, input validation fuzzing/sanitization checks
Performance-critical path	baseline comparison, regression threshold checks

Coverage Requirements

Minimum Standards

• Overall Coverage: 85% minimum
• Critical Paths: 95% minimum (auth, payments, data mutations)
• New Code: 90% minimum

Coverage is necessary but NOT sufficient. Passing coverage does not replace situational verification.

Requirements-to-Evidence Mapping (Mandatory)

Before completion, map each acceptance criterion to concrete evidence. Acceptance criteria MUST come from the active PRD.

Template:

| Acceptance Criterion | Verification Method | Evidence |
|---|---|---|
| AC-1: ... | Situational test / baseline test / manual verification | command + result |
| AC-2: ... | ... | ... |

Browser Automation (Hard Rule)

All browser automation (Playwright, Cypress, Puppeteer) MUST run in headless mode. Launching a visible browser collides with the user's display and active session.

• Playwright: use headless: true in config or --headed must NOT be passed
• Cypress: use cypress run (headless by default), never cypress open
• Puppeteer: use headless: true (default)

If a project's playwright.config.ts does not explicitly set headless: true, add it before running tests.

Test Quality Rules

1. Test behavior and outcomes, not private implementation details.
2. Include failure-path and edge-case assertions for changed behavior.
3. Keep tests deterministic; no new flaky tests.
4. Keep tests isolated; no dependency on execution order.

Anti-Gaming Rules

1. Do NOT stop at "tests pass" if acceptance criteria are not verified.
2. Do NOT write narrow tests that only satisfy assertions while missing real workflow behavior.
3. Do NOT claim completion without situational evidence for impacted surfaces.

Reporting

QA report MUST include:

1. baseline tests run and outcomes
2. situational tests run and outcomes
3. TDD usage decision (required/applied or optional/skipped with rationale)
4. acceptance-criteria-to-evidence mapping
5. coverage results
6. residual risk notes

Before Completing

1. Baseline tests pass.
2. Required situational tests pass.
3. TDD obligations met for required change types.
4. Acceptance criteria mapped to evidence.
5. No flaky tests introduced.
6. CI pipeline passes (if available).
7. Scratchpad updated with results.