Multivariate testing (MVT) is a form of testing in which changes are made to multiple sections of a webpage on a site, and then variations are created for all the possible combinations of those changes.
Many people seem to get confused about the terms A/B testing, split URL testing, and multivariate testing. Essentially, A/B testing and split URL testing are the same concepts but Multivariate testing is different. So for all practical concepts, consider A/B Testing = Split Testing.
Download Free: Multivariate Testing Guide
The following table will illustrate the difference between Multivariate Testing and A/B testing (or split testing).
What is A/B Testing? |
What is Multivariate Testing? | |
| What is it? | In A/B testing you split traffic amongst two or more completely different versions of a webpage (landing page, home page, etc.) The variations of your original page can differ in any manner. You can either just change the headline; or you can even change the entire design, layout, offer, and whatnot in the variations. What you change in an A/B test is only limited by your creativity. For example, if you are A/B testing on your landing page you may want to create one version with a 15% discount, the other with free shipping, and one with the same offer but a different page design. | In a multivariate test, you identify a few key areas/sections of a page and then create variations for those sections specifically (as opposed to creating variations of the whole page in an A/B split test). So for example, in a multivariate test, you can choose to create different variations for 2 different sections: headline and image. A multivariate testing software will combine all these section-specific variations to generate unique versions of the page to be tested and then simply split traffic amongst those versions. That is, you will get these variations of the page: Headline1_Image1, Headline1_Image2, Headline2_Image1, Headline2_Image2. |
| How it works? |  |  |
| Traffic Required | Relatively Less | Huge |
| Best used for | Testing radically different ideas for conversion rate optimization | Optimizing and refining an existing landing page or homepage without making a significant investment in the redesign |
| Example Case Study | Signups increased by 60% after actually removing the signup form | Hyundai and Wingify Multivariate Test Case Study |
Watch how you can quickly set up a multivariate testing campaign in VWO.
Categories:
I started Wingify in early 2009 to enable businesses to design and deploy great customer experiences for their websites and apps. I have a background in machine learning and am a gold medalist from Delhi College of Engineering. I have been featured twice in the Forbes 30 under 30 list - India and Asia. I'm an entrepreneur by profession and my curiosity is wide-ranging. Follow me at @paraschopra on Twitter. You can email me at paras@wingify.com
