A/B Testing in Marketing– An Overview for a Marketer

This article provides a detailed account of A/B Testing in marketing. It provides more information, detail, and context to the article CHAPTER 1: Conversion Rate Optimization Secrets, one of four articles that lay out the Frictionless Commerce Conversion Rate Optimization philosophy.

If you care at all about optimizing your site, you must understand the role of A/B testing in marketing. If you don’t, you’re lighting money on fire.

But where do you start? How do you decide what is important? What is worthwhile testing? And what is just busy work that won’t make a speck of difference in the long run?

We wholeheartedly believe in the power of A/B testing in marketing. It is one of the core principles here at Frictionless Commerce. Of course, we believe that how and where you perform these tests plays a large role in your testing success.

So, if you’re a marketer, especially a marketer who believes in the power of storytelling, you will love this guide.

A Simple, Clear, Easy-To-Remember Definition of A/B Testing in Marketing

There are heaps of definitions of what A/B Testing in marketing is. If we’re not talking about the same thing, this article won’t help you. 

In my experience, the best definitions are simple, clear, and easy to remember. Here’s my definition of A/B Testing in marketing:

That’s it.

You can make it more complicated, and probably a tiny bit more accurate, but that’s A/B testing in a nutshell.

Let’s move on to the good stuff.

Why Marketers Need to Care About A/B Testing

As we saw in the John Caples example in the history of conversion rate optimization section in the absence of attribution the business will undervalue the marketer’s impact.

And this suppresses our whole industry.

Yes, it’s possible an A/B test may show that your idea was worse, or worse that it had no impact, but it could also decisively prove the value of your work. And even if your idea flops knowing that will lead to ideas that lead to a winner.

In the absence of A/B testing marketing is reduced to the ad agency’s ability to earn the client’s trust. Why spend so much wining and dining the client when you could just let your work do all the talking?

But …

Most Marketers Aren’t Using A/B Testing Correctly

Think about how most marketers do A/B testing. 

If a marketer wanted to optimize the page for the Dyson Supersonic hair dryer, they would typically consider this sort of A/B testing:

Or maybe testing these elements:

I’m not saying these aren’t valid A/B tests, but I am suggesting other tests are more likely to get you your year-end raise.

Why listen to me for A/B testing in marketing ideas? My name is Rishi Rawat, I’m a marketer, and I have spent the last 15 years breaking my head, learning why online shoppers behave the way they do. I know a thing or two because I’ve A/B tested a thing or two.

What Most Marketers Focus On

So, what’s the pattern here? Give most marketers a page to optimize and they’re going to focus on the design.

There are many reasons not to focus on design-based A/B testing. The biggest reason is this: design-based tests get a lot more scrutiny from leadership teams. 

Leadership teams feel they need their fingerprints on design decisions. This slows down the speed of A/B testing, and speed is one of the biggest advantages of A/B testing in marketing.

Design tests are also more expensive and more complicated to develop. 

What About Video?

You may be wondering, “Ok, but what about video? Isn’t video incredibly impactful?” If that’s a question floating in your head, read this post: Words Kick Video’s Ass.

What Should You Focus On?

You’re constructing your pages to convert. To sell. As an eCommerce business, you are looking to convert those first-time visitors into buyers. Conversions are what matters. How do you impact conversion?

Unbounce analyzed 40,000 landing pages. They found that the conversion impact of copy is twice that of design. (source).

Words kick video’s ass. And it’s not even close.

Focus on your sales pitch. Think about the actual words used on the page. 

Focus on your copy.

Conversion Copywriting

Conversion copywriting is the practice of using words to influence shoppers to make a purchase. Don’t make the mistake of thinking copywriting is the domain of people with a degree in English. Knowledge of language has little to do with it. I, for example, am an engineer.

I look at conversion copywriting as a sales function. The marketer uses conversion copywriting to construct a sales pitch designed to get the buyer from interest to purchase.

Once we switched from the throw-spaghetti-at-the-wall-and-see-what-sticks conversion rate optimization strategy to the laser-focused-on-conversion-copywriting strategy, our clients saw their results skyrocket.

You are currently in CHAPTER 1 and conversion copywriting will be covered comprehensively (trust us) in CHAPTER 3: Conversion Copywriting.

Start Your Marketing A/B Tests

Now that you know what to test, here’s a quick overview of how to test.

First, construct your sales pitch. Second, create a new version of an existing product page based on the selling angles you identified. Third, create a test in VWO. Run your new product page against your current product page. Let it run as long as is needed to declare a winner.

Once the test is complete, look at the difference in conversion rates. Did you get a significant bump in conversion rates? Continue to run tests until you are satisfied.

A huge benefit of testing copy is how quickly you can iterate. We routinely create 10, 12, or even 15 different concepts before we find the winner.

Next Steps

Let’s now look at how most agencies do conversion rate optimization.