What significance does A/B testing have in improving CRO?

CRO isn’t about making sweeping changes overnight, it’s an ongoing process of refining, testing, learning and improving your website through small, purposeful changes that make it easier for users to take action. Over time, these incremental improvements can have a big impact on helping you achieve your business goals.

In this blog, we will cover:

• What CRO is
• What A/B testing is
• The benefits of A/B testing
• And, give you a few examples

via GIPHY

What is CRO

CRO, or Conversion Rate Optimisation, is the process of improving a website to achieve its goals more efficiently. For example, an E-Commerce website’s goal could be to drive more sales whereas an online blog’s goal could be to increase mailing list sign ups. CRO is the umbrella term for any changes made to a website with the aim of making it easier for users to take the action you want them to. CRO is guided by data-driven analysis, or sometimes even a hunch that a change might improve a website’s performance, which is then tested to see if it works. CRO is often a slow process of altering a website and analysing the results before planning the next step.

What is A/B testing?

A/B testing helps to make CRO trackable, often giving reason to CRO improvements through data. A/B testing involves creating variations of your website with small changes to test your hunches, also called hypotheses. You will then serve this variation or multiple variations of your website to users and track the differences found in their actions.

For example you may change a CTA button from black to green, in the hopes that users click it more often. You will then serve 50% of users the original version with the black button and 50% of users the variation with the green button. The amount of users on each variation on the site is then tracked alongside how often the button in question was clicked. This will give you statistics on whether your hypothesis was correct and allows you to make a data-driven decision on what to do next.

Often you will see or interact with something on a website and think to yourself “that doesn’t make sense” or “it’d be better if it did X”. These interactions and thoughts are great starting points for an A/B test. A/B tests allow for a place to test ideas without setting them in stone, giving more chances for creativity, experimentation to learn what resonates with your audience, and continuously improving your website over time.

Benefits of A/B testing

Using A/B testing on your website comes with a whole host of benefits. Depending on the website and strategy A/B testing can be tweaked to work specifically for you. A few ways A/B testing can benefit your website are:

• Increased conversion rates
• Reduced guesswork & hunches
• Better understanding of your website & audience
• Reduced wasted time and costs of development
• Enhanced user experience
• Increased ROI on marketing
• Guiding future site development
• Increased user engagement
• Reduced bounce rates

Example A/B tests

Registration plate quick form

Hypothesis: Shortening the garage forms and redirecting users to the main booking form would increase the amount of form submissions

Goal: Increase form submissions

This A/B test came from findings that a large percentage of users were navigating to a garage page when using the website but not completing the form on that page. The form was working as expected and sat in the header right at the top of the page. The website data confirmed that users were seeing this form but not filling it out. Users are often less likely to fill out a form that requires more information, the shorter the better essentially. This was the motivation behind replacing it with a single field form.

This change allows users to fill in a single piece of information, their registration plate, before redirecting them through to the main site booking form. This form then automatically populated information based off of their registration plate. The hope was that once users had begun the process of filling out this form they were more engaged and therefore more likely to complete the full booking form.

This test showed that by swapping out the form, form conversion rates went from 2.2% up to 7.2%, a 5% increase. This may not seem like a huge amount but when you apply this to the amount of traffic you can see the number of conversions is significantly greater.

Reframing sitewide wording

Hypothesis: Altering the wording across the site would sound less definite, increasing form page visits, form completions and therefore bookings.

Goal: Increase form submissions

Not all tests have to be major changes to the functionality of a website or webpage. If possible the smaller the change the better. This allows you to isolate a very specific change and assess the difference it makes. If you create an A/B test making sweeping changes then can’t pinpoint exactly which part has made a difference to conversion rates.

This test only changed the wording on a website to see if that could improve the likelihood of users submitting a form. This website used the terminology of ‘book now’ across the site but the actual workflow happening upon a form submission was for the user to receive a quote. This generated the idea that if we reframe the terminology of ‘book now’ to ‘get a quote’ then users may be more likely to interact with the form. The feeling was that getting a quote felt less definite than booking and it feels less like the user is committing to anything.

Despite being technically a simple change it alters the meaning of how the users interact with the site. During this A/B test we tracked how many users visited the form submission page along with how many completed the form. There was a very minimal change in how many users visited the form submission page, only increasing by 1%. The major optimisation in this test was seen through how many of those users went on to complete the form. When the wording was reframed to ‘get a quote’ across the site we saw 3.5% more users submitting the form and requesting a quote. From A/B data alone you cannot draw absolute conclusions on why these changes have worked but assumptions can be made that can be tested further in more A/B testing. The assumption here was that users were more comfortable with requesting a quote than they were booking in directly through the form.

Conclusion

A/B testing is a key part of any successful CRO strategy and a great tool to assist in creating a better web experience. Without A/B testing CRO becomes a shot in the dark, making changes to a website based on real user behaviour rather than assumptions. By testing ideas before rolling them out permanently, you can build a better understanding of what your audience responds to and make informed decisions that improve both the user experience and your conversion rates. Sometimes, the smallest tweaks can have the biggest impact. A CRO project with well thought out A/B testing can begin to improve a website’s performance instantly by nudging users down the desired conversion path and improving their experience with your website.