Statistical Power Calculator

Statistical Power Calculator: run statistical power calculations online. Formula, assumptions, and interpretation guide.

Hypothesis testing is a formal statistical procedure for deciding whether sample data provides enough evidence to reject a null hypothesis (H₀) in favour of an alternative hypothesis (H₁). The process produces a p-value — the probability of observing results as extreme as the data, assuming H₀ is true. A small p-value (typically <0.05) is taken as evidence against H₀.

Hypothesis tests are used to evaluate new drug treatments, test marketing campaigns, assess manufacturing quality, and validate scientific claims across every empirical field.

State your null hypothesis (H₀: no effect / no difference) and alternative hypothesis (H₁: effect exists).
Choose your significance level (α) — 0.05 is standard; use 0.01 for higher-stakes decisions.
Enter your sample data: mean (or proportion), standard deviation, and sample size.
Select the test type: one-sample, two-sample, or paired; one-tailed or two-tailed.
Compare the p-value to α: if p < α, reject H₀; if p ≥ α, fail to reject H₀.

Common test statistics

One-sample t-test: t = (x̄ − μ₀) / (s / √n), degrees of freedom = n−1

Two-sample t-test: t = (x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂)

One-proportion z-test: z = (p̂ − p₀) / √(p₀(1−p₀)/n)

Decision rule: reject H₀ if |t| or |z| exceeds the critical value for chosen α and degrees of freedom.

Interpreting the p-value and test result

Statistical vs practical significance

A p-value of 0.03 means there is a 3% chance of seeing data this extreme if H₀ were true — below the 0.05 threshold, so reject H₀. However, statistical significance does not equal practical importance: a drug might show a statistically significant but clinically trivial improvement. Always report the effect size (Cohen's d, odds ratio, etc.) alongside the p-value to convey the magnitude of the effect.

Statistics tips and best practices

Pre-register your hypothesis and significance threshold before collecting data to prevent p-hacking.
Report exact p-values (e.g. p = 0.023) rather than just "p < 0.05" — this is more informative.
Consider statistical power (1−β): most studies should target 80% power to avoid false negatives.
For multiple comparisons, apply a Bonferroni correction or use false discovery rate (FDR) methods to control Type I error.

The FDA typically requires p < 0.05 in two independent clinical trials before approving a new drug — effectively requiring p < 0.0025 jointly.
Physics uses a 5-sigma (p < 0.0000003) threshold for claiming particle discoveries — used for the Higgs boson announcement in 2012.
Approximately 50% of psychology studies failed to replicate in the 2015 Reproducibility Project, prompting widespread re-evaluation of p < 0.05 as the sole criterion for publication.

Common mistakes to avoid

Interpreting "p > 0.05" as "no effect" — it means insufficient evidence to reject H₀, not proof of no effect.
Running multiple tests on the same data without correction — each test at α=0.05 has a 5% false positive rate; 20 tests gives an expected 1 spurious result.
Confusing one-tailed and two-tailed tests — use two-tailed unless you have a strong prior directional hypothesis stated before data collection.

Hypothesis test results are probabilistic and depend on assumptions of random sampling, independence, and approximate normality. Results should be interpreted by a qualified statistician for regulatory, clinical, or legal applications.