A/B testing (split testing)

A/B testing is running two versions of an ad or page side by side, with everything else the same, to see which one gets better results.

Reviewed by Calum Peak, CTO Last reviewed

Version A

Sponsored Office coffee delivered weekly | Java Roasters Fresh beans for teams of 10 to 200. Get a quote in a minute.

6,000 impressions · 96 clicks

1.6% CTR

Version B

Sponsored Office coffee, no lease needed | Java Roasters Fresh beans for teams of 10 to 200. Get a quote in a minute.

6,000 impressions · 144 clicks

2.4% CTR Winner

Two versions of Java Roasters' ad, shown the same number of times, and the one that earned more clicks.

What it means for your advertising

You change one thing, such as a headline, an image or the offer, and show both versions to the same kind of audience at the same time. The difference in clickthrough rate or conversion rate tells you which works better, and by how much.

The catch is numbers. With a handful of clicks either version can look like the winner by chance, so a test needs enough impressions and conversions before you act on it. Change one thing at a time, or you won't know which change made the difference.

Platforms build testing in. Google's responsive search ads mix up to 15 headlines and learn which combinations do best, Meta has its own A/B test tool, and a ChatGPT campaign can run up to 10 sponsored cards at once.

An example

Java Roasters tests two headlines for its office coffee ad, each shown 6,000 times. “Office coffee delivered weekly” earns 96 clicks, a 1.6% clickthrough rate; “Office coffee, no lease needed” earns 144, 2.4%. A gap that size over that many impressions is unlikely to be chance, so B becomes the main ad and a new challenger replaces A.

Java Roasters is a fictional business, and its figures are illustrative.

How we handle it

We test a lot of variations, as a matter of course: several headlines and descriptions in each Google ad, more than one card in a ChatGPT campaign, and variations on Facebook and Instagram. As results come in, we retire the weaker versions and keep adding new ones to find the best fit, including wildcards: bolder versions that break from the pattern, backed by what the data is showing. More on Google Search advertising.

Are you ready to grow your business?

No retainers. No lock-ins. Campaigns when you need them.

Book a strategy call Start in the Guru

Strategy calls run for 30 minutes — bring your business and your goal, we will do the rest.