How to Use Email to A/B Test Blog Post Headline


As many long-time readers will know, we’ve maintained an active blog producing content on email marketing for almost 10 years now.

In that time, the discipline of content marketing has evolved significantly, and there are many more organizations publishing content and competing for audience attention.

So how does a marketer like you get your content noticed?

One of the more recent ways marketers are winning audience attention is by A/B testing blog post headlines to identify the most appealing and effective headline for a particular post.

In this article, I wanted to share with you why A/B testing headlines is important, how we use email to do it at Campaign Monitor, and how you can use the same approach to improve the effectiveness of your content.

Why A/B testing headlines is important

According to Copyblogger, 8 out of 10 people will read your blog headline, yet only 2 out of 10 will go on to read the entire post. That’s quite the discrepancy.

Why is that? Why do so many more people read the headline but not the full post?

The main reason is that your headline will be seen in significantly more places than the rest of your blog post will be. Here are just a few examples:your blog post

Considering how many people will see your headline across all these different places, it’s absolutely critical you get your headline right as the difference between a great headline and an average one can literally be thousands of visits.

In fact, when viral content site Upworthy tested two headlines for the exact same video against each other, the improved headline generated 100x the amount of pageviews.

How we A/B test blog post headlines using email

Owing to the multitude of places people see your headline and the results a great headline can bring, we are a big fan of A/B testing blog post headlines.

So how do we use email to A/B test blog post headlines? Let’s start at the beginning.

1. Write multiple headlines for your blog post

Once we’ve written a post about a particular topic, I’ll spend a bit of time writing some alternative headlines for the post. I keep a Headline Swipe File (which is essentially a big list of headline structures I’ve collected over the years) that I look through and use as ideas for alternative headlines.

For example, here are some of the alternative headlines I devised for our recent post on using Google Analytics to measure the success of your email marketing.

According to Adam Moredaci of Upworthy, they will write 25 alternative headlines for each post they write. Now, I usually don’t go that far, as I find I can get a few good ones out of just writing 10 or so. However, I’d recommend writing as many as you feel necessary to get a couple of headlines worth testing.

2. Select two of the best

Once I’ve written out a big list of headlines, I’ll then pick the two I think will work best.

This is a bit of a subjective area, and I certainly don’t have a checklist or concrete guidelines by which I choose the two best, but here are some of the things I think about when narrowing my list down to two:

3. Set up a subject line test

Once we’ve narrowed our list down to two headlines, we set up a subject line test for the email we send out to our blog subscribers.

Not surprisingly, we use Campaign Monitor to send all our email communications, and setting up the subject line test couldn’t be easier.

You simply select “Subject line test” from the testing options, input your two subject lines, and then go about creating the rest of your email as normal.

Campaign Monitor will send one version of the email to one part of the list (we choose to send each version to 10 percent of our list), and the other version to another part. It’ll then measure the results and automatically send the best performing email to the rest of the list.

Aside from adding in the additional subject line, there is literally no extra work in sending a subject line test in comparison to sending a normal email campaign.

4. Measure the results

After the designated amount of time the test runs (we usually choose two hours), we’ll receive an email saying the results are in.

I’ll then go in and look at the reports to better understand how the test went.

Here’s what the A/B test report looked like for our recent post on how to use Google Analytics to measure the success of your email marketing campaign:

We generally base the winner off of Open Rates as the act of opening the email is very similar to the act of clicking through from social media or search engines. As you can see, Version A was the clear winner with a solid 57% increase in opens over Subject Line B.

5. Revise the blog post title

Once the results are in and a winner has been decided, we’ll go back and modify the headline of the article on the blog itself.

That way, when people share the article via social media or when search engines index the article in their results pages, the headline with a 57% increase in opens will be what gets used—driving significantly more people back to the blog than it would have had we not conducted the test and optimized the headline.

Why email is a great way to A/B test blog headlines

I’ve seen a number of different approaches to A/B testing blog headlines. Upworthy has built their own system they refer to as the ‘magic unicorn box’, while the guys over at Buffer will post their different headlines to Twitter and measure the results there (using stats like the number of clicks and retweets to determine which post headline works best).

While no method is completely scientific, the reason I prefer testing blog headlines via email is that the sample size is controlled. When you run a subject line test, it will send each version to the exact same number of people and then select the winner based on how many of those people click the headline and open the email.

The issue with testing headlines via Twitter is you have no idea how many people saw each tweet. Sure, headline A might have 100 clicks while headline B only had 50, but what if headline A had been seen by 10,000 people and headline B was only seen by 500?

You’d end up optimizing the post for headline A based on the number of clicks, yet in reality, headline A had a 1% click-through rate while headline B had a 10% click through rate.

Short of custom developing an A/B testing solution like Upworthy has, using email subject lines to test your blog headlines is the easiest and most accurate way to understand which headline is going to drive the most traffic.

In conclusion

When you’ve worked hard to create great blog content, A/B testing your blog headline via email is an easy way to ensure that content gets seen by as many readers as possible. It’s a simple process that can make a huge impact on the number of reads and shares.

We challenge you to A/B test some blog headlines in email to see which version works best. You know how the process works, so all that’s left to do is to give it a try.


Straight to your inbox

Get the best email and digital marketing content delivered.

Join 250,000 in-the-know marketers and get the latest marketing tips, tactics, and news right in your inbox.


See why 200,000 companies worldwide love Campaign Monitor.

From Australia to Zimbabwe, and everywhere in between, companies count on Campaign Monitor for email campaigns that boost the bottom line.

Request a demo