First steps in A/B testing part 1 of 2: Process

At Myna we’re all about considered, methodical approaches to website optimisation. But for newbies, A/B testing can seem like a lot of hard work: fussy micro-management of page elements and aesthetics in pursuit of minor gains.

How do you know where to begin? How do you find the tests that make a big difference?

To get the most out of A/B testing, you need a plan. Don’t just sit down in front of your website and start testing the colour of a button – start by establishing what you want to improve. For that you need statistics, so sit yourself down, fire up your analytics program of choice, and let’s get started.

This is the first in a two-part series of posts outlining how to build a testing plan. In this post we’ll talk about testing the process you’re guiding your users through. In the next post we’ll talk about testing the positioning of your product in the market.


When testing process, you are assuming that your product or offer is attractive to your users, but something is getting in the way of making a sale.

Look at the paths you want users to take through your web site, find out where you’re losing them, and test around this.

Where are you losing users? Do they hit your landing page and then bounce straight off? Do people fail to complete the sign up process? Or do people get distracted and wander off during checkout? Possible culprits are:

  • your landing page lacks a strong call to action
  • your sign-up process is too long
  • your checkout process is too complicated
  • your product pages don’t offer a clear path to purchase

Your analytics will tell you what you need to know – the key terminology is a funnel or sales process. This is a sequence of steps that you define that lead users to your conversion goal. You can set up funnels using conversion goals in Google Analytics as well as other analytics packages to track the flow of users through the desired steps.

Here is an example sales funnel based on our sign-up process at Myna. Note that the conversion rate figures are fabricated for illustrative purposes.

When people sign up to Myna, we push them through four steps: they choose a plan, enter account details, enter billing information, and arrive at their shiny new Myna dashboard. At each step some percentage of users proceed onwards, while other users leave and go elsewhere. Our goal is to maximise the number of people who get all the way through to the end.

Example sales funnel for Myna
(conversion rates for illustrative purposes only)

The key information provided by the funnel is where we are losing users. In the example there are two weak steps: the landing page (step 1) and the billing page (step 4). We immediately know where to start optimising:

  • The landing page is likely to have a low conversion rate because not all users arrive at our site wanting to sign up. Some users have already signed up and want to log in; other users may want to read our blog or contact us. Nevertheless, we can run a few tests here to try to increase this conversion rate. It’s not hard to come up with several ideas around messaging, design and layout.
  • The final step of the pipeline, billing information, is a great candidate for testing. By the time users get to this step we know they definitely want to sign up to a plan. Why aren’t 60% of them doing so? This is a great candidate for testing ways to minimise friction on the billing form: collect fewer fields, optimise the layout, and provide the user with immediate feedback if they get something wrong.

The conversion rates on a funnel provide useful information going even beyond testing. For example, in Myna we sign people up to our free plan in step 3, before they enter billing information. This way, we allow users to use Myna even if they choose not to give us their credit card details. We also allow users to opt in to our mailing list in step 3, providing us with a way to contact them in the future.


In conclusion, analysing funnels and traffic around your site provides the key to testing process. Once you understand where you can make the biggest gains, it’s much easier to get into the specifics of tweaking layouts, images, and buttons. At every level you should make hypotheses about why people are leaving your funnel and address them by testing new ideas.

It’s important to note that sales funnels are often more complicated than the one we’ve described above. Our sales process at Myna involves many more steps on and off our web site, encouraging users to come and try our product, providing support setting up tests, and helping them get their first set of results. Your web sign-up or checkout process is a great place to start, however. As you get used to testing, you can expand outwards to look at the big picture.

Finally, testing process is only useful if you’re managing to attract users to your site. In order to do that, you’re going to need a marketing strategy. A/B testing can help greatly with marketing as well, as we’ll see in the second post in this series on positioning.

Leave a reply


<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>