203 lines
4.6 KiB
Markdown
203 lines
4.6 KiB
Markdown
# QA & Testing Guide
|
|
|
|
## Before Starting
|
|
1. Check assigned issue: `~/.mosaic/rails/git/issue-list.sh -a @me`
|
|
2. Create scratchpad: `docs/scratchpads/{issue-number}-{short-name}.md`
|
|
3. Review existing test structure and patterns
|
|
|
|
## Test-Driven Development (TDD) Process
|
|
|
|
### The TDD Cycle
|
|
1. **Red**: Write a failing test first
|
|
2. **Green**: Write minimal code to pass
|
|
3. **Refactor**: Improve code while keeping tests green
|
|
|
|
### TDD Rules
|
|
- Never write production code without a failing test
|
|
- Write only enough test to fail
|
|
- Write only enough code to pass
|
|
- Refactor continuously
|
|
|
|
## Coverage Requirements
|
|
|
|
### Minimum Standards
|
|
- **Overall Coverage**: 85% minimum
|
|
- **Critical Paths**: 95% minimum (auth, payments, data mutations)
|
|
- **New Code**: 90% minimum
|
|
|
|
### What to Cover
|
|
- All public interfaces
|
|
- Error handling paths
|
|
- Edge cases and boundaries
|
|
- Integration points
|
|
|
|
### What NOT to Count
|
|
- Generated code
|
|
- Configuration files
|
|
- Third-party library wrappers (thin wrappers only)
|
|
|
|
## Test Categories
|
|
|
|
### Unit Tests
|
|
- Test single functions/methods in isolation
|
|
- Mock external dependencies
|
|
- Fast execution (< 100ms per test)
|
|
- No network, database, or filesystem access
|
|
|
|
```python
|
|
def test_calculate_discount_applies_percentage():
|
|
result = calculate_discount(100, 0.20)
|
|
assert result == 80
|
|
```
|
|
|
|
### Integration Tests
|
|
- Test multiple components together
|
|
- Use real databases (test containers)
|
|
- Test API contracts
|
|
- Slower execution acceptable
|
|
|
|
```python
|
|
def test_create_user_persists_to_database(db_session):
|
|
user = create_user(db_session, "test@example.com")
|
|
retrieved = get_user_by_email(db_session, "test@example.com")
|
|
assert retrieved.id == user.id
|
|
```
|
|
|
|
### End-to-End Tests
|
|
- Test complete user workflows
|
|
- Use real browser (Playwright, Cypress)
|
|
- Test critical paths only (expensive to maintain)
|
|
|
|
```javascript
|
|
test('user can complete checkout', async ({ page }) => {
|
|
await page.goto('/products');
|
|
await page.click('[data-testid="add-to-cart"]');
|
|
await page.click('[data-testid="checkout"]');
|
|
await page.fill('#email', 'test@example.com');
|
|
await page.click('[data-testid="submit-order"]');
|
|
await expect(page.locator('.order-confirmation')).toBeVisible();
|
|
});
|
|
```
|
|
|
|
## Test Structure
|
|
|
|
### Naming Convention
|
|
```
|
|
test_{what}_{condition}_{expected_result}
|
|
|
|
Examples:
|
|
- test_login_with_valid_credentials_returns_token
|
|
- test_login_with_invalid_password_returns_401
|
|
- test_get_user_when_not_found_returns_404
|
|
```
|
|
|
|
### Arrange-Act-Assert Pattern
|
|
```python
|
|
def test_add_item_to_cart_increases_count():
|
|
# Arrange
|
|
cart = Cart()
|
|
item = Item(id=1, name="Widget", price=9.99)
|
|
|
|
# Act
|
|
cart.add(item)
|
|
|
|
# Assert
|
|
assert cart.item_count == 1
|
|
assert cart.total == 9.99
|
|
```
|
|
|
|
### Test Isolation
|
|
- Each test should be independent
|
|
- Use setup/teardown for common state
|
|
- Clean up after tests
|
|
- Don't rely on test execution order
|
|
|
|
## Mocking Guidelines
|
|
|
|
### When to Mock
|
|
- External APIs and services
|
|
- Time-dependent operations
|
|
- Random number generation
|
|
- Expensive operations
|
|
|
|
### When NOT to Mock
|
|
- The code under test
|
|
- Simple data structures
|
|
- Database in integration tests
|
|
|
|
### Mock Example
|
|
```python
|
|
def test_send_notification_calls_email_service(mocker):
|
|
mock_email = mocker.patch('services.email.send')
|
|
|
|
send_notification(user_id=1, message="Hello")
|
|
|
|
mock_email.assert_called_once_with(
|
|
to="user@example.com",
|
|
subject="Notification",
|
|
body="Hello"
|
|
)
|
|
```
|
|
|
|
## Test Data Management
|
|
|
|
### Fixtures
|
|
- Use factories for complex objects
|
|
- Keep test data close to tests
|
|
- Use realistic but anonymized data
|
|
|
|
### Database Tests
|
|
- Use transactions with rollback
|
|
- Or use test containers
|
|
- Never test against production data
|
|
|
|
## Reporting
|
|
|
|
### Test Reports Should Include
|
|
- Total tests run
|
|
- Pass/fail counts
|
|
- Coverage percentage
|
|
- Execution time
|
|
- Flaky test identification
|
|
|
|
### QA Report Template
|
|
```markdown
|
|
# QA Report - Issue #{number}
|
|
|
|
## Summary
|
|
- Tests Added: X
|
|
- Tests Modified: Y
|
|
- Coverage: XX%
|
|
|
|
## Test Results
|
|
- Passed: X
|
|
- Failed: X
|
|
- Skipped: X
|
|
|
|
## Coverage Analysis
|
|
- Lines: XX%
|
|
- Branches: XX%
|
|
- Functions: XX%
|
|
|
|
## Notes
|
|
[Any observations or concerns]
|
|
```
|
|
|
|
## Commit Format
|
|
```
|
|
test(#34): Add user registration tests
|
|
|
|
- Unit tests for validation logic
|
|
- Integration tests for /api/users endpoint
|
|
- Coverage increased from 72% to 87%
|
|
|
|
Refs #34
|
|
```
|
|
|
|
## Before Completing
|
|
1. All tests pass locally
|
|
2. Coverage meets 85% threshold
|
|
3. No flaky tests introduced
|
|
4. CI pipeline passes
|
|
5. Update scratchpad with results
|