Creative and testing
A/B test
A test that shows two versions of something, such as an ad or a page, to similar groups at random and compares the results.
What it is
Random assignment is what makes the comparison fair: the groups differ in the version they saw and in nothing else on purpose.
Why it matters
For lead generation and e-commerceIt is the standard way to know whether a change caused a difference, rather than coinciding with one.
In practice
Decide the measure and the sample size in advance, run both versions at the same time, and read the result once at the end.
A common mistake
Stopping the test as soon as one version looks ahead, or checking again and again until one does.
Read the evidenceThe winner is smaller than it looks All terms