Power Calculation Calculator

Power Calculation Calculator: run power calculation calculations online. Formula, assumptions, and interpretation guide.

An A/B test calculator determines whether the difference in conversion rates between two variants (control A and treatment B) is statistically significant — i.e., unlikely to be due to random chance. It is the core tool of conversion rate optimisation (CRO) for websites, apps, emails, ads, and product features. Running A/B tests rigorously prevents teams from acting on random noise and confidently identifies genuine improvements.

Used by digital marketing teams, product managers, UX researchers, and growth hackers to make data-driven decisions about design, copy, pricing, and features without guessing.

Enter the number of visitors and conversions for Variant A (control).
Enter the number of visitors and conversions for Variant B (treatment).
Select your significance level (typically 95%) and whether the test is one-tailed or two-tailed.
The calculator shows the conversion rate for each, the relative uplift, p-value, and whether the result is statistically significant.
If significant, implement the winning variant; if not, collect more data or redesign the test.

A/B test statistics formula

Conversion rate: p = conversions / visitors

Pooled proportion: p_pool = (conv_A + conv_B) / (visitors_A + visitors_B)

Z-statistic: z = (p_B − p_A) / √[p_pool × (1−p_pool) × (1/n_A + 1/n_B)]

Minimum detectable effect (MDE): MDE = z_α × √(2 × p × (1−p) / n) — the smallest uplift your test can reliably detect.

Reading A/B test results

Statistical significance and practical uplift

A p-value < 0.05 at 95% confidence means there is less than a 5% chance the observed difference occurred by chance. A relative uplift of +15% with 95% confidence is generally worth implementing if the test ran for a full business cycle. Always report absolute conversion rates alongside relative uplift: a 50% relative uplift from 2% to 3% is very different from a 50% uplift from 20% to 30% in business impact.

Statistics tips and best practices

Run tests for at least one full business cycle (typically 1–2 weeks) to account for day-of-week and seasonal effects.
Never stop a test early just because you see significance — this dramatically inflates false positive rates (peeking problem).
Segment results by device, user cohort, and traffic source — an overall winner may lose for mobile users.
Use a sample size calculator before starting to ensure you have enough traffic to detect your target MDE.

Industry benchmarks suggest that only 1 in 8 A/B tests produces a statistically significant positive result — most ideas do not improve conversion.
A typical e-commerce checkout A/B test requires 10,000–50,000 visitors per variant to detect a 10% relative uplift with 80% power at 95% confidence.
Companies that run 100+ A/B tests per year (Amazon, Booking.com, Netflix) generate most of their CRO gains from cumulative small wins of 1–5% per test.

Common mistakes to avoid

Running multiple simultaneous tests on the same traffic without controlling for interaction effects — results become confounded.
Calling a test significant at p = 0.049 while the confidence interval spans near-zero to large — borderline results need larger samples, not a declaration of victory.
Testing too many variants at once (A/B/C/D testing) without adjusting the significance threshold for multiple comparisons.

A/B test results are probabilistic. Statistical significance does not guarantee sustained real-world improvement due to novelty effects, seasonality, and external factors. Consult a data scientist for high-stakes product decisions.