Every A/B Test We’ve Ever Ran at Canny Creative

At Canny Creative, we’ve been A/B testing elements of our website for the past several months.

Rather than writing a blog post outlining “the best practises for A/B testing your website” or “7 things to consider when A/B testing” – we’ve decided to do something slightly different.

We’re going to keep a live diary of every single A/B test we’ve ever ran at Canny. As run new tests, we’ll update the post, so you can see what we’re up to.

The whole idea behind this, is to show what A/B testing looks like in a small business. Keep a record of our findings, and inspire you to try out your own experiments.

A lot of people write about A/B testing. And there is some great testing software out there, including:

  • Visual Website Optimizer
  • Unbounce
  • Optimizely

We’ve used VWO in the past, and it’s a fantastic tool, but we couldn’t make the price point work for us.

Unless you have 3 figures a month to shell out on testing software, you’re likely going to want an alternative solution.

At Canny, we’ve been using Google Optimize, which lets you run 5 experiments at a time, for free.

Hopefully you can learn from some of our failures and try and make them into your own successes!

So, let’s take a look at what we’ve been testing, what the results were, and hopefully, it’ll get you inspired to test your own site.

First though, a little primer.

What is A/B Testing?

If you’re unfamiliar what A/B testing is, then essentially:

A/B testing is a fierce battle between one version of your website, and another, to find out which one converts best for your business.
Or in most cases, it’s tweaking the colour of a button to see if it converts better when it’s red, or green.

VWO have a great primer on A/B testing here.

Essentially, you run your website through Google Optimize, make one or two tweaks, and measure the results.

Multivariate is another version of testing you can utilise.

In multivariate testing, rather than having just 2 versions of your site, you might have 4 or 5.

You can really go down the rabbit hole with testing, and if you’re just getting started. Keep it simple.

Every A/B Test We’ve Ever Ran at Canny Creative

We’ve not been testing our website for long, so far we’ve ran 6 experiments.

What we’ve been trying to achieve with our site, is lead generation.

For all we have around 1300 visitors on a daily basis, our conversion rate is surprisingly low. We’ve identified this is because most of our traffic finds us through our blog.

Blogs have a shockingly high bounce rate, and a very low conversion rate.

We’re going to separate this post out into the areas or funnels on our site that we’re trying to improve.

Area for Improvement 1: Homepage to Start Your Project Funnel

What we decided to focus on at first, was improving the conversion rate of the following funnel:

  • Visitors lands on our homepage
  • They click through to the “Start Your Project” page
  • They fill out our form which drops into our inbox
  • They’re then sent to a dedicated “Thank You” page

It’s a pretty simple funnel, but it’s integral to our lead generation process. So it was the first thing we decided to optimise.

With the following experiments, we were testing click throughs to either our Start Your Project page, or from the Start Your Project page, onto the Thank You page.

Start Your Project Form Arrangement

Before the Experiment: Our enquiry form was on the left hand side of our website. With a column on the right hand side containing our email address, telephone number, and social media links.

Hypothesis: By moving the form from the left hand side, to the right, we’re making the page more logical in terms of reading order. This should encourage people to fill out the form.

Result: We have a winner!

245 people were tested over a 2 week period. The form on the left hand side converted at 2.5%. Our winning form, was the form on the right, which converted people at a rate of 4.05%.

It may seem like a small and insignificant change in the numbers, but in theory, that’s an extra 1.5 leads per 100 site visits.

This result also helped drive a new key decision in our website design process. When we create forms these days, we position them to the right to start with, and test them from there.

A/B Test – Start Your Project

Before the Experiment: The main call to action on our homepage and navigation bar was “start your project” in a purple button.

Hypothesis: The words “start your project” are a little direct, and people will respond better to a softer call to action such as “let’s talk.”

Result: We were wrong!

“Start your project” converted visitors at 5.3% whereas “let’s talk” only converted at 3.95%.

So, we left the page exactly as it was, and vowed to revisit the call to action text at a later date.

Changing Homepage Hero H1

Before the Experiment: The main headline on our homepage read “We create brands that build businesses.”

Hypothesis: Our main headline doesn’t make what we do clear enough. We changed it to “Accelerating business growth with branding and web design.”

Result: We were absolutely right.

The new headline converted at a better rate.

The original converted at 8.29% whereas our new version tested at a conversion rate of 10.02%.

Again, it’s only a small increase, but that’s what this process is all about. If you can’t find huge increases, go for lots of small incremental increases.

Changing “Start Your Project” Text

Before the Experiment: We decided to revisit the “start your project” button text in the hope of finding a winner.

Hypothesis: At the risk of sounding like a shrink, we changed it from “start your project” to “talk to us.”

Result: It failed again!

“Start your project” was still the runaway winner. However, after feedback from the Canny team, and several clients, we went even deeper.

Changing “Start Your Project” Text Multivariate

Before the Experiment: We weren’t happy with “start your project” as a call to action, as it seemed a little too forward, especially for our homepage.

Hypothesis: There has to be a better call to action than “start your project.” We decided to run a multivariate test, with the following options:

  • Start your project
  • Let’s get started
  • Get a free consultation
  • Get in touch

In the hope of finding something slightly more on brand and relevant.

Result: Finally, we found an alternative!

“Get a free consultation” pulled into an early lead. We crossed our fingers tightly, because again, this didn’t feel right for our business.

Over time “Let’s get started” began to emerge as the real winner, converting at 13.29%, which is the biggest win we’ve managed with our testing so far.

This has genuinely helped Canny generate more leads than ever through our website.

Change Text on “Start Your Project” Page Title

Before the Experiment: This experiment is still currently running. However, we’re testing changing the language on our “start your project” page to match our call to action button.

Hypothesis: By changing the language to match our button, we’ll convert more. Again, we ran with several options:

  • Let’s start your project
  • Let’s get started
  • Let’s get your project started

This gives us a nice range of options to test.

Result: As it stands…

“Let’s get your project started” is absolutely running away with it here, converting at a huge 10% compared to 3% and 5% respectively for the other options.

We’ll double back when we have more.

Conclusion: Every A/B Test We’ve Ever Ran at Canny Creative

A/B testing is a difficult skill to master.

You can easily get yourself bogged down with thoughts, ideas, and options.

Essentially, you just have to get started. Then, let the results speak for themselves. Rather than talking your ideas down, plug them in, and get them tested.

At the point of launching your website, you’ve really only made it to ground zero. Sure, it might convert well from the start, but it can always do better!

By testing your ideas, you might end up breaking the floodgates wide open, and before you know it, you’re selling your products and services like hot cakes.

And that’s exactly what you want.

A website that performs and makes a difference to your business. Afterall, it’s the ultimate business marketing tool.

What do you think? Have you managed to make a difference to your conversions through A/B testing? Let us know in the comments below.

Leave a Reply

Your email address will not be published. Required fields are marked *