The Cheat Sheet to A/B Testing

This article was written by Ohad from Toonimo

In 2015, the biggest challenge for online store owners is that their customers’ attention spans are short-lived and online competition is becoming overwhelming, so Conversion Rate Optimization (CRO) is a prerequisite skill for every marketing guru.

CRO is something quite magical, it helps you to get more out of your existing traffic. When it comes to applying a CRO plan, you’ll find yourself lost in the amount of online resources that tell you to change the colors of your buttons, shorten your web copy or include gamification on your site.

But before you embark on these quick fixes, there’s a secret ingredient that you’ll need to optimize conversion rates, A/B testing.

Most of the A/B tests suggested on the internet are often confusing and conflict each other. Before you embark on the quest of the A’s and B’s, here are the top tricks you should know before carrying out the tests on your site.

Cheat Sheet Answer #1: Assume Nothing, Question Everything

If you have preconceived notions about your customers, forget everything you know and avoid all possible assumptions based on age, gender, location or income. In the past, customer profiling was the best way to target your audience.

But in this digital age, there are so many other options and factors that play a big role in marketing. We have the ability to learn about our customers through many different digital touch points, on a one-to-one basis.

With A/B testing, the way that you find out your results is from testing two different marketing messages or styles, objectively.

If I were to present you with a green and a red ‘call to action’ button saying ‘Buy Now’, you would believe that the green color is always the favored choice as it is more inviting.

Here is the perfect example, where we cannot judge a book by it’s cover and make presumptions. A research was performed on the color variations of two CTA buttons (one red, the other red). The end result, was that the red button succeeded the green by 21%.

But, Just because the color red works for someone it doesn’t mean that it will work for you as well. Don’t go about A/B testing based on assumptions.

Cheat Sheet Answer #2: Walk Before You Run

The first and most important goal is to increase conversion rates, and you may be anxious to get started ASAP.

Before you begin gambling and A/B testing, it is vital to use your time wisely and create a base to measure against. Walk before you run! In other words, you should have a clear idea of what “success” is before you begin testing.

There’s no way to determine your results if you neglect this point.

Ask yourself, what makes one variable more successful than another? Is it more conversions, more opt-ins, more sales or something else?

Cheat Sheet Answer #3: One Size Doesn’t Fit All

Life would be so much easier if we could just use the same CRO process that worked for every website. There would be no need for testing at all and this article would be futile. Unfortunately, there isn't one theory for everything in CRO.

Just because one A/B test worked for someone else, does not mean it will work for you. Each A/B test needs to have it’s own particular content with it’s own audience.

Lets look at an example, if a company X sells shoes and company Y sells computers. Even if both X and Y have the same customers, the way they buy things will be unique and different. Company X, may find that the green CTA button was clicked on more and for company Y the red button was the preferred choice.

This goes back to the point 1, don’t assume anything. Just because one test works for company X does not mean it will for Y.

Cheat Sheet Answer #4: Don’t Start the Victory Party Prematurely

Statistics are an important key in A/B testing. Marketers place confidence that if one test is successful then they expect the same results, if done again at a later stage.

For instance, you perform an A/B test on your shopping cart page, and A is the use of CTA buttons and B is the use of dropdown menus. If B produces a higher conversion rate than A, is B the winner? We cannot take this for granted as there are factors to be considered.

Such as, the size of your audience will never be the same, there are room for errors and if your audience is 250,000 and your sample size is 25, this test will yield low levels of confidence. You can calculate your sample size with Raosoft’s Sample Size Calculator.

Cheat Sheet Answer #5: Understand the User, Without Trickery

Using tricks in A/B testing, gives people the wrong idea about what testing is. It’s true, that sometimes the color affects results, especially when it makes the CTA button stand out better. But green vs red is not the essence of A/B testing. It’s about understanding the target audience.

Doing research into the psychology of your customer can be hard work, but it’s necessary.

If you want to give your conversion rates a serious boost, you need to do conversion research. But by analyzing what your customers really need, the language that resonates with them and how they want to buy it will valuable in the long term.

Final words…

Be patient, be realistic and be open-minded to learn new things. The only way to be sure of what works is to run a set of A/B tests and be willing to make the changes needed based on the data found.