Free tool · A/B testing

A/B test duration calculator

Find out how many visitors your A/B test needs and how many days it will take to reach a statistically significant result.

% How the control performs today, on the metric you will judge the test by.
%
Lift type
The smallest improvement worth acting on. 3% → 3.3% is a 10% relative lift.
Visitors who will see the test, all versions together.
Settings: 95% confidence · 80% power · two-sided
One-sided or two-sided
Checks if the variant is better or worse than the control. The safe default.
Estimated test duration —
Total visitors required ——

—

Duration by lift size

Smaller lifts take much longer to detect. The dot is your setting; hover or tap to compare.

See the numbers
LiftPer versionDuration

Test already running?

Check if your result is significant.

Open the significance calculator →

A/B testing best practices

How to plan an A/B test

Go through these before you set up a test, so the result is clear when it ends.

  1. 01

    Write the hypothesis first

    Name the change, the metric and why you expect it to move. If you can't fill in the sentence, wait before testing.

    If we [change], then [metric] will [move], because [reason].
  2. 02

    Check your traffic before you build

    Use the calculator above. If the test needs more than two months, don't run it as planned. Traffic shifts and other changes muddy the result.

  3. 03

    Short on traffic? Move up the funnel

    Orders may be too rare to test, but clicks or add-to-carts are not. A metric with more volume reaches significance much sooner.

  4. 04

    Change one thing

    Change one element and judge the test on one primary metric. Change three at once and you won't know which one worked.

  5. 05

    Run full weeks on normal days

    Run for at least seven days and in whole weeks. Avoid sales and holidays, or the result won't hold the rest of the year.

  6. 06

    Fix the end date and leave it alone

    Split traffic 50/50 and don't edit the versions or the audience mid-test. Read the result only when the planned sample is in.

Read the full A/B testing guide →

Questions about test duration

What is the difference between a one-sided and a two-sided test?
A two-sided test checks if the variant is better or worse than the control. A one-sided test only checks if it is better. One-sided needs about 20% fewer visitors, but it can't confirm that a change hurts, so two-sided is the safer default.
How long should an A/B test run?
Until it reaches the sample size you planned, and never less than one full week, so weekdays and weekends are both in the data. Enter your traffic above to get the number of days. If it comes out above two months, change the plan rather than waiting.
How many visitors do I need for an A/B test?
It depends on your current conversion rate and the smallest lift you want to detect. Low conversion rates and small lifts need far more visitors: going from 3% to 3.3% needs about 53,000 visitors per version, while 3% to 3.6% needs about 14,000.
What is the minimum detectable effect?
The smallest change in conversion rate your test can reliably pick up. Set it to the smallest lift that would be worth acting on for your business. Halving it roughly quadruples the visitors you need.
What are statistical power and confidence?
Confidence (usually 95%) limits false wins: the chance of calling a winner that isn't one. Power (usually 80%) limits missed wins: the chance of detecting a lift that is really there. The defaults suit most tests.