# A/B Test Templates Reference

Templates for planning, documenting, and analyzing experiments.

## Test Plan Template

```markdown
# A/B Test: [Name]

## Overview

- **Owner**: [Name]
- **Test ID**: [ID in testing tool]
- **Page/Feature**: [What's being tested]
- **Planned dates**: [Start] - [End]

## Hypothesis

Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].

## Test Design

| Element            | Details                   |
| ------------------ | ------------------------- |
| Test type          | A/B / A/B/n / MVT         |
| Duration           | X weeks                   |
| Sample size        | X per variant             |
| Traffic allocation | 50/50                     |
| Tool               | [Tool name]               |
| Implementation     | Client-side / Server-side |

## Variants

### Control (A)

[Screenshot]

- Current experience
- [Key details about current state]

### Variant (B)

[Screenshot or mockup]

- [Specific change #1]
- [Specific change #2]
- Rationale: [Why we think this will win]

## Metrics

### Primary

- **Metric**: [metric name]
- **Definition**: [how it's calculated]
- **Current baseline**: [X%]
- **Minimum detectable effect**: [X%]

### Secondary

- [Metric 1]: [what it tells us]
- [Metric 2]: [what it tells us]
- [Metric 3]: [what it tells us]

### Guardrails

- [Metric that shouldn't get worse]
- [Another safety metric]

## Segment Analysis Plan

- Mobile vs. desktop
- New vs. returning visitors
- Traffic source
- [Other relevant segments]

## Success Criteria

- Winner: [Primary metric improves by X% with 95% confidence]
- Loser: [Primary metric decreases significantly]
- Inconclusive: [What we'll do if no significant result]

## Pre-Launch Checklist

- [ ] Hypothesis documented and reviewed
- [ ] Primary metric defined and trackable
- [ ] Sample size calculated
- [ ] Test duration estimated
- [ ] Variants implemented correctly
- [ ] Tracking verified in all variants
- [ ] QA completed on all variants
- [ ] Stakeholders informed
- [ ] Calendar hold for analysis date
```

---

## Results Documentation Template

```markdown
# A/B Test Results: [Name]

## Summary

| Element  | Value                         |
| -------- | ----------------------------- |
| Test ID  | [ID]                          |
| Dates    | [Start] - [End]               |
| Duration | X days                        |
| Result   | Winner / Loser / Inconclusive |
| Decision | [What we're doing]            |

## Hypothesis (Reminder)

[Copy from test plan]

## Results

### Sample Size

| Variant | Target | Actual | % of target |
| ------- | ------ | ------ | ----------- |
| Control | X      | Y      | Z%          |
| Variant | X      | Y      | Z%          |

### Primary Metric: [Metric Name]

| Variant | Value | 95% CI   | vs. Control |
| ------- | ----- | -------- | ----------- |
| Control | X%    | [X%, Y%] | —           |
| Variant | X%    | [X%, Y%] | +X%         |

**Statistical significance**: p = X.XX (95% = sig / not sig)
**Practical significance**: [Is this lift meaningful for the business?]

### Secondary Metrics

| Metric     | Control | Variant | Change | Significant? |
| ---------- | ------- | ------- | ------ | ------------ |
| [Metric 1] | X       | Y       | +Z%    | Yes/No       |
| [Metric 2] | X       | Y       | +Z%    | Yes/No       |

### Guardrail Metrics

| Metric     | Control | Variant | Change | Concern? |
| ---------- | ------- | ------- | ------ | -------- |
| [Metric 1] | X       | Y       | +Z%    | Yes/No   |

### Segment Analysis

**Mobile vs. Desktop**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| Mobile | X% | Y% | +Z% |
| Desktop | X% | Y% | +Z% |

**New vs. Returning**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| New | X% | Y% | +Z% |
| Returning | X% | Y% | +Z% |

## Interpretation

### What happened?

[Explanation of results in plain language]

### Why do we think this happened?

[Analysis and reasoning]

### Caveats

[Any limitations, external factors, or concerns]

## Decision

**Winner**: [Control / Variant]

**Action**: [Implement variant / Keep control / Re-test]

**Timeline**: [When changes will be implemented]

## Learnings

### What we learned

- [Key insight 1]
- [Key insight 2]

### What to test next

- [Follow-up test idea 1]
- [Follow-up test idea 2]

### Impact

- **Projected lift**: [X% improvement in Y metric]
- **Business impact**: [Revenue, conversions, etc.]
```

---

## Test Repository Entry Template

For tracking all tests in a central location:

```markdown
| Test ID | Name                 | Page     | Dates     | Primary Metric | Result       | Lift | Link   |
| ------- | -------------------- | -------- | --------- | -------------- | ------------ | ---- | ------ |
| 001     | Hero headline test   | Homepage | 1/1-1/15  | CTR            | Winner       | +12% | [Link] |
| 002     | Pricing table layout | Pricing  | 1/10-1/31 | Plan selection | Loser        | -5%  | [Link] |
| 003     | Signup form fields   | Signup   | 2/1-2/14  | Completion     | Inconclusive | +2%  | [Link] |
```

---

## Quick Test Brief Template

For simple tests that don't need full documentation:

```markdown
## [Test Name]

**What**: [One sentence description]
**Why**: [One sentence hypothesis]
**Metric**: [Primary metric]
**Duration**: [X weeks]
**Result**: [TBD / Winner / Loser / Inconclusive]
**Learnings**: [Key takeaway]
```

---

## Stakeholder Update Template

```markdown
## A/B Test Update: [Name]

**Status**: Running / Complete
**Days remaining**: X (or complete)
**Current sample**: X% of target

### Preliminary observations

[What we're seeing - without making decisions yet]

### Next steps

[What happens next]

### Timeline

- [Date]: Analysis complete
- [Date]: Decision and recommendation
- [Date]: Implementation 'if winner'
```

---

## Experiment Prioritization Scorecard

For deciding which tests to run:

| Factor                   | Weight | Test A | Test B | Test C |
| ------------------------ | ------ | ------ | ------ | ------ |
| Potential impact         | 30%    |        |        |        |
| Confidence in hypothesis | 25%    |        |        |        |
| Ease of implementation   | 20%    |        |        |        |
| Risk if wrong            | 15%    |        |        |        |
| Strategic alignment      | 10%    |        |        |        |
| **Total**                |        |        |        |        |

Scoring: 1-5 (5 = best)

---

## Hypothesis Bank Template

For collecting test ideas:

```markdown
| ID  | Page/Area | Observation         | Hypothesis                            | Potential Impact | Status  |
| --- | --------- | ------------------- | ------------------------------------- | ---------------- | ------- |
| H1  | Homepage  | Low scroll depth    | Shorter hero will increase scroll     | High             | Testing |
| H2  | Pricing   | Users compare plans | Comparison table will help            | Medium           | Backlog |
| H3  | Signup    | Drop-off at email   | Social login will increase completion | Medium           | Backlog |
```
