{"id":111816,"date":"2023-06-05T12:00:50","date_gmt":"2023-06-05T12:00:50","guid":{"rendered":"https:\/\/staging.wingify.com\/blog\/how-long-run-ab-test\/"},"modified":"2023-06-05T12:00:50","modified_gmt":"2023-06-05T12:00:50","slug":"how-long-run-ab-test","status":"publish","type":"post","link":"https:\/\/wingify.com\/blog\/how-long-run-ab-test\/","title":{"rendered":"How Long Should You Run an A\/B Test?"},"content":{"rendered":"

One of the most popular questions when starting with experimentation<\/a> is: How long should an A\/B test run before you can draw conclusions from it?<\/strong><\/p>\n

Determining the ideal A\/B test duration can be a challenge for most businesses. You have to factor in your business cycles, traffic flow, the sample size needed and be aware of other business campaigns.<\/p>\n

Even if you reach your sample size in a few days… is it okay to end your test then? How long should you really<\/em> wait?<\/p>\n

In this article, we will discuss potential mishaps if your testing cycle is too short, give insights into which factors you need to consider and share advice on finding the best duration for your A\/B tests.<\/p>\n

Looking for fast statistical reliability? At Wingify, we provide a free A\/B test duration calculator<\/a>, which also has capabilities for a sample size calculator.<\/p>\n

What happens if you end an A\/B test too soon?<\/h2>\n

The underlying question is a crucial one and can be summed up as follows: At what point can you end a test that appears to be yielding results<\/strong>?<\/p>\n

The answer depends on the relevance of the analysis and on the actual benefits of the test.<\/p>\n

In fact, it\u2019s not all that unusual to see tests yield good results during the trial phase and no longer see those results once the modifications are introduced.<\/p>\n

In most cases, a disappointing observation of this nature comes down to an error during the trial phase: the test was ended too soon and the results at that point were misleading.<\/p>\n

Let\u2019s look at an example that illustrates the nature of the problem.<\/p>\n

\"How<\/p>\n

The graph above shows the change in the conversion rate of two versions of a page that were the subject of a test. The first version appears to break away and perform extremely well. The discrepancy between the two versions is gradually eroded as time goes by \u2013 two weeks after the starting point there’s hardly any observable difference at all.<\/p>\n

This phenomenon where the results converge is a typical situation: the modification made does not have a real impact on conversion<\/strong>.<\/p>\n

There is a simple explanation for the apparent outperformance at the start of the test:\u00a0 it’s unusual for the samples to be representative of your audience when the test starts.<\/strong> You need time for your samples to incorporate all internet user profiles, and therefore, all of their behaviors.<\/p>\n

If you end the test too soon and allow your premature data to be the deciding factor, your results will quickly show discrepancies.<\/p>\n

How to determine the duration of your A\/B test<\/h2>\n

Now that the problem has been aired let\u2019s have a look at how you can avoid falling into this trap.<\/p>\n

The average recommended A\/B testing time is 2 weeks<\/strong>, but you should always identify key factors relevant to your own conversion goals<\/strong> to determine the best length for a test that will meet your goals.<\/p>\n

Let\u2019s discuss several criteria you should use as a foundation to determine when you can trust the results you see in your A\/B testing<\/a>:<\/p>\n