A/B Test Sample Size and Significance Calculator
Most A/B tests fail in one of two ways: they stop too early, so a random swing looks like a win, or they run on too little traffic to detect the change that matters. Sizing the test before it starts, and testing the result properly after it ends, prevents both.
Everything above runs in your browser and saves to this device only. Plan a test on the left, or enter a finished test’s results on the right.
How it works
The sample size per variant uses the normal approximation for comparing two proportions.
z = (pB − pA) ÷ √(p (1 − p) (1÷nA + 1÷nB)), with p the pooled rate
95% interval for the difference = (pB − pA) ± 1.96 √(pA(1 − pA)÷nA + pB(1 − pB)÷nB)
The p-value is two-sided. Decide the sample size before the test starts and read the result once, at the end. Checking repeatedly and stopping at the first significant result inflates the false-positive rate.
Frequently Asked Questions
How do you calculate sample size for an A/B test?
Set the baseline conversion rate, the smallest lift worth detecting, the significance level, and the power. This calculator applies the standard two-proportion formula and gives visitors per variant and the run time at your traffic.
How do you know if an A/B test is statistically significant?
An A/B test is statistically significant when a two-proportion z-test gives a p-value below your significance level, usually 0.05. The difference is then unlikely to be chance alone.
What is the minimum detectable effect in A/B testing?
The minimum detectable effect is the smallest lift worth acting on, stated relative to the baseline rate. Halving it roughly quadruples the sample a test needs.
Can I stop an A/B test early?
Run an A/B test to its planned sample size and read the result once, at the end. Checking repeatedly and stopping at the first significant result inflates false positives.
Is this A/B Test Calculator free?
Yes. The A/B Test Calculator is free and opens straight away in any modern browser, including on a phone. Everything you enter is processed in your browser and stays on your device.