Getting the Most Out of Split Testing Your Website

Quick Navigation

• What Is Split Testing?
• Split Testing vs. AB Testing
• Why Does Split Testing Matter?
• What Are the Benefits of Split Testing?
• What Are Best Practices for Split Testing?

Split tests are one of the most popular and effective methods for gathering useful information about how your website can impact customer behavior. Understanding split testing allows you to learn about the nuances of website design without sacrificing any of your current web traffic.

What Is Split Testing?

Split testing is a popular method of determining which version of your website is most accessible and engaging. It involves directing part of a website’s traffic to one version of a page, then sending the rest of the website’s visitors to a completely different version of the same webpage. The tester measures certain metrics, such as product purchases, clicks, or the time spent on the page, and then compares the results between the two pages.

Split Testing vs. AB Testing

Split testing and AB testing are often used interchangeably, but they each have a unique purpose. Split testing occurs when you compare the performance of two significantly different web designs. AB testing involves changing one small aspect of a web page, such as the test color, to determine if it impacts user engagement. Businesses usually use split tests for significant design changes, like a full website redesign, while they use AB testing to make their current site more effective.

For example, you would use split testing to decide whether you wanted to use a fully interactive page with animations and moving parts or a static, simplified interface. Whichever version is more successful, you would then run through a round of AB testing to refine the site’s individual features.

Image via Flickr by jayninelessons

Why Does Split Testing Matter?

Split testing is important because it measures the real-life interactions of potential customers with your web page. It is a valuable way of gathering data about what visual elements and web features lead to the best engagement. Split testing helps you make intentional decisions about updating your company’s web branding, allowing you to make changes if you do not get the results you anticipated. If an original “control” page performs better than the update, you can avoid a costly relaunch and instead direct your efforts to finding out what other design options would be more successful.

What Are the Benefits of Split Testing?

Split testing has a range of benefits for marketing professionals and optimizers:

• Lower bounce rates based on a direct comparison of what pages customers engage with the most
• Better conversion rates by improving site engagement
• Versatility when assessing multiple website engagement metrics at once
• Using your current website traffic to learn more about your website accessibility
• Mitigating risks by ensuring extensive revisions will be accepted positively before implementing them

What Are Best Practices for Split Testing?

There are several pitfalls that novice marketers fall into when conducting their first split tests. Avoid these challenges by implementing these best practices for split testing your site:

Test Through Multiple Sales Cycles

The first step toward gathering valid data from your split test is to run the tests for a significant amount of time. If you only spend a few days or weeks testing your website’s effectiveness, you may not get the full picture of how visitors interact with your website. Run tests for multiple weeks to make sure that you have a steady stream of customer data.

Wait for Statistical Significance

Don’t rush your tests by assuming that you’ve found the perfect website after seeing a small increase. Wait until you’ve gathered a large sample size of data from a diverse range of website visitors. Measure the statistical significance of your data by analyzing your key performance indicators and look for a p-value of under 0.05 before you decide to end the test.

Identify External Factors

The purpose of split testing is to determine which version of a website will perform better over time, in any circumstances. Identify external factors, such as seasonal sales events or where you’re sourcing your traffic, to ensure that your experiment gathers valuable data.

For example, if you send all previous customers on your email list to test a new website and all organic traffic to the old website, you may skew the results. Current customers who already spend more time on your website and already make regular purchases may make it seem that your new website is more successful than it actually is.

Set a Hypothesis First

Approach split testing like a science experiment and decide on a clear hypothesis before building the new website. Instead of testing random ideas and hoping they work, come up with a theory about what kind of site design will be most impactful on your customers. This will help you identify what is most and least successful about your new website design by comparing actual performance to your expectations.

Gather Full Analytics Reports

As you gather data about both versions of your website, make sure you have access to all of the nuanced details about individual customer behavior. Some programs only deliver averages, which can be a useful guide but can also give too much value to outliers. Google Analytics is a great program for dividing your data up into detailed segments, telling you about specific customer behaviors that caused one website version to perform better than the other.

Make Significant Changes

To get the most out of your current website traffic, only use split testing for significant changes to your website that you are unsure of. If you already know what version of a website will perform better, don’t waste your time on split testing for that factor. Save your testing resources for large website overhauls so that you can focus on high-impact updates.

If your first test doesn’t give you the results you want, keep making changes and running tests. Sometimes it can take several website redesigns to get the lasting results your business is looking for. Ultimately, split testing empowers you to use the power of direct comparison to understand what your potential consumers want and need out of a website, allowing you to tailor your web presence to grow your brand and capture a larger percentage of your target market.

Knowledge Base: Intro to Marketing