As many long-time readers will know, we’ve maintained an active blog producing content on email marketing for almost 10 years now.

In that time, the discipline of content marketing has evolved significantly, and there are many more organisations publishing content and competing for audience attention.

So how does a marketer like you get your content noticed?

One of the more recent ways marketers are winning audience attention is by A/B testing blog post headlines to identify the most appealing and effective headline for a particular post.

In this article, I wanted to share with you why A/B testing headlines is important, how we use email to do it at Campaign Monitor, and how you can use the same approach to improve the effectiveness of your content.

Why A/B testing headlines is important

According to Copyblogger, 8 out of 10 people will read your blog headline, yet only 2 out of 10 will go on to read the entire post. That’s quite the discrepancy.

Why is that? Why do so many more people read the headline but not the full post?

The main reason is that your headline will be seen in significantly more places than the rest of your blog post will be. Here’s just a few examples:

Considering how many people will see your headline across all these different places, it’s absolutely critical you get your headline right as the difference between a great headline and an average one can literally be thousands of visits.

In fact, when viral content site Upworthy tested two headlines for the exact same video against each other, the improved headline generated 100x the amount of pageviews.

How we A/B test blog post headlines using email

Owing to the multitude of places people see your headline and the results a great headline can bring, we are a big fan of A/B testing blog post headlines.

So how do we use email to A/B test blog post headlines? Let’s start at the beginning.

1. Write multiple headlines for your blog post

Once we’ve written a post around a particular topic, I’ll spend a bit of time writing some alternative headlines for the post. I keep a Headline Swipe File (which is essentially a big list of headline structures I’ve collected over the years) that I look through and use as ideas for alternative headlines.

For example, here are some of the alternative headlines I devised for our recent post on using Google Analytics to measure the success of your email marketing.

According to Adam Moredaci of Upworthy, they will write 25 alternative headlines for each post they write. Now, I usually don’t go that far, as I find I can get a few good ones out of just writing 10 or so. However, I’d recommend writing as many as you feel necessary to get a couple of headlines worth testing.

2. Select two of the best

Once I’ve written out a big list of headlines, I’ll then pick the two I think will work best.

This is a bit of a subjective area, and I certainly don’t have a checklist or concrete guidelines by which I choose the two best, but here are some of the things I think about when narrowing my list down to two:

3. Set up a subject line test

Once we’ve narrowed our list down to two headlines, we set up a subject line test for the email we send out to our blog subscribers.

Not surprisingly, we use Campaign Monitor to send all our email communications, and setting up the subject line test couldn’t be easier.

You simply select “Subject line test” from the testing options, input your two subject lines, and then go about creating the rest of your email as normal.

Campaign Monitor will send one version of the email to one part of the list (we choose to send each version to 10 percent of our list), and the other version to another part. It’ll then measure the results and automatically send the best performing email to the rest of the list.

Aside from adding in the additional subject line, there is literally no extra work in sending a subject line test in comparison to sending a normal email campaign.

4. Measure the results

After the designated amount of time the test runs (we usually choose two hours), we’ll receive an email saying the results are in.

I’ll then go in and look at the reports to better understand how the test went.

Here’s what the A/B test report looked like for our recent post on how to use Google Analytics to measure the success of your email marketing campaign:

We generally base the winner off of Open Rates as the act of opening the email is very similar to the act of clicking through from social media or search engines. As you can see, Version A was the clear winner with a solid 57% increase in opens over Subject Line B.

5. Revise the blog post title

Once the results are in and a winner has been decided, we’ll go back and modify the headline of the article on the blog itself.

That way, when people share the article via social media or when search engines index the article in their results pages, the headline with a 57% increase in opens will be what gets used—driving significantly more people back to the blog than it would have had we not conducted the test and optimized the headline.

Why email is a great way to A/B test blog headlines

I’ve seen a number of different approaches to A/B testing blog headlines. Upworthy has built their own system they refer to as the ‘magic unicorn box’, while the guys over at Buffer will post their different headlines to Twitter and measure the results there (using stats like the number of clicks and retweets to determine which post headline works best).

While no method is completely scientific, the reason I prefer testing blog headlines via email is that the sample size is controlled. When you run a subject line test, it will send each version to the exact same number of people and then select the winner based on how many of those people click the headline and open the email.

The issue with testing headlines via Twitter is you have no idea how many people saw each tweet. Sure, headline A might have 100 clicks while headline B only had 50, but what if headline A had been seen by 10,000 people and headline B was only seen by 500?

You’d end up optimising the post for headline A based on the number of clicks, yet in reality, headline A had a 1% click-through rate while headline B had a 10% click through rate.

Short of custom developing an A/B testing solution like Upworthy has, using email subject lines to test your blog headlines is the easiest and most accurate way to understand which headline is going to drive the most traffic.

In conclusion

When you’ve worked hard to create great blog content, A/B testing your blog headline via email is an easy way to ensure that content gets seen by as many readers as possible. It’s a simple process that can make a huge impact on the number of reads and shares.

We challenge you to A/B test some blog headlines in email to see which version works best. You know how the process works, so all that’s left to do is to give it a try.

How do you test your blog headlines? Have you found A/B testing via email to be an important part of your publishing process?

Want to improve your email marketing?

Join over 20,000 other marketers & designers who get tips on improving their email marketing delivered directly to their inbox.

Send better email campaigns Subscribe to our email course to learn how to create and send high-performing email campaigns that drive results for your business. GET THE FREE EMAIL COURSE
  • Shannon

    Great post. I wouldn’t have thought to A/B test blog headlines via email. A few things came to mind though.

    Let’s say you did a small A/B test via email and that headline B was the winner, so the blog post was changed to reflect that. If I was one of the people who received the email about blog post A, but I visit the page today and its suddenly a different headline, wouldn’t that cause unwanted results? For example, what if that completely changes the link structure so that the old (headline A) link is no longer valid because its been bookmarked or something similar.

    Like I said before, great idea—especially about trying multiple headlines before picking your top two—but it seems like there would need to be some careful planning before trying to A/B test headlines via email.

  • Aaron Beashel

    Hey Shannon.

    Firstly, really glad to hear you liked the post and would love to hear how you go if/when you try it out.

    It’s a great question you ask and you make two valid points.

    1. You definitely need to keep both the tested headlines within the same vein. For instance, the two headlines we tested for the Google Analytics post were ‘How to measure the success of your email marketing with Google Analytics’ and ‘3 steps to measuring the success of your email marketing with Google Analytics’. Given that there were 3 steps in the post, regardless of which headline you saw in the email the content you saw when you landed on the post was still aligned.

    2. Most CMS systems I know of allow you to edit the URL structure and post title separately, so you should be able to change the post title without breaking links in either of the emails. That being said, it’s a great point you make and definitely something anybody implementing this should be aware of.

    Again, thanks so much for adding your voice to the conversation and I’m glad you brought those two points forward as they are definitely things anybody implementing this technique needs to think about.

  • Reginald

    A very detailed post! I love the part about the A/B headlines stuff and honestly, something I have yet to really try it.

    Thanks for sharing this and gonna try something real soon!

    Have a great weekend!

  • Albert

    You may be interested to know that there is a wordpress plugin (paid) called KingSumo Headlines ( which can A/B test blog headlines. (No, I am not affiliated with them in any way)

  • Paul

    Thats a really great way of using an email list to better you online content

    Just need to build an email list large enough to get accurate results

Want to improve your email marketing? Subscribe to get tips on improving your email marketing delivered to your inbox.

Join 150,000 companies around the world that use Campaign Monitor to run email marketing campaigns that deliver results for their business.

Get started for free