Bayesian A/B Test Sample Size

Size Bayesian A/B tests on conversion rates by combining uplift goals with posterior credibility and beat-probability thresholds.

Observed control rate before the experiment.
Desired relative improvement for the variant.
Posterior credible interval coverage (two-tailed).
Posterior probability that the variant outperforms control.
Optional. Defaults to 1 (uninformative Beta prior).
Optional. Defaults to 1 (uninformative Beta prior).

Planning aid only—validate with your experimentation platform's Bayesian calculator before launch.

Examples

  • 3.2% baseline, 15% uplift, 95% credibility, 80% beat probability ⇒ 22,631 sessions per variant
  • 7.5% baseline, 5% uplift, 95% credibility, 90% beat probability, Beta(1,1) prior ⇒ 106,048 sessions per variant

FAQ

What if my uplift is very small?

Smaller uplifts require larger samples; try reducing the uplift target or relaxing the beat probability to finish tests sooner.

How do priors affect the estimate?

Informative priors contribute pseudo-conversions and pseudo-failures—set alpha and beta to reflect historical data volume so the tool subtracts them from the new sample size.

Can I compare more than two variants?

This tool assumes one control and one variant. For multiple variants, run it per pair or adjust the credibility and beat thresholds downward.

Is the result per variant or total?

Output shows sessions per variant. Multiply by the number of arms to budget total traffic for the experiment.

Additional Information

  • The calculator uses a normal approximation to the Beta-Binomial posterior, matching Bayesian and frequentist results for planning purposes.
  • Credibility level represents the width of the posterior interval—95% is analogous to a two-tailed credible interval.
  • Prior alpha and beta add pseudo-observations; larger values reduce required sample size slightly when past data is informative.

Related calculators