As many long-time readers will know, we’ve maintained an active blog producing content on email marketing for almost 10 years now.
In that time, the discipline of content marketing has evolved significantly, and there are many more organisations publishing content and competing for audience attention.
So how does a marketer like you get your content noticed?
One of the more recent ways marketers are winning audience attention is by A/B testing blog post headlines to identify the most appealing and effective headline for a particular post.
In this article, I wanted to share with you why A/B testing headlines is important, how we use email to do it at Campaign Monitor, and how you can use the same approach to improve the effectiveness of your content.
Why A/B testing headlines is important
According to Copyblogger, 8 out of 10 people will read your blog headline, yet only 2 out of 10 will go on to read the entire post. That’s quite the discrepancy.
Why is that? Why do so many more people read the headline but not the full post?
The main reason is that your headline will be seen in significantly more places than the rest of your blog post will be. Here’s just a few examples:
- On Your Blog Homepage – The homepage is one of the most visited pages of a blog and the majority of blog homepages contain a summary of all the recent posts, normally including the headline, an image, and some summary text. Visitors to your blog homepage will likely see a number of different headlines but only click through to read a select few of the posts.
- On Social Media – Whenever someone shares your blog post, the majority of social networks will automatically grab the headline and use it as part of the share. This means that when your content is shared on social media by someone, many of their followers will see the blog post headline in their feeds, but only a select few will click-through to read the whole post.
- In Search Engine Results – Whenever someone searches for a keyword related to your blog post, the search results will include the headline and a snippet (less than 160 characters) from the blog post itself. Again, many people will see the headline on the search engine results page, but only a few will click-through and read the full post.
Considering how many people will see your headline across all these different places, it’s absolutely critical you get your headline right as the difference between a great headline and an average one can literally be thousands of visits.
In fact, when viral content site Upworthy tested two headlines for the exact same video against each other, the improved headline generated 100x the amount of pageviews.
How we A/B test blog post headlines using email
Owing to the multitude of places people see your headline and the results a great headline can bring, we are a big fan of A/B testing blog post headlines.
So how do we use email to A/B test blog post headlines? Let’s start at the beginning.
1. Write multiple headlines for your blog post
Once we’ve written a post around a particular topic, I’ll spend a bit of time writing some alternative headlines for the post. I keep a Headline Swipe File (which is essentially a big list of headline structures I’ve collected over the years) that I look through and use as ideas for alternative headlines.
For example, here are some of the alternative headlines I devised for our recent post on using Google Analytics to measure the success of your email marketing.
- The ultimate guide to measuring the success of your email marketing
- How to use Google Analytics to measure the success of your email marketing
- 3 steps to measuring the success of your email marketing with Google Analytics
- Here’s a quick way to understand the true impact of your email marketing
- How to setup Google Analytics to measure your email marketing
According to Adam Moredaci of Upworthy, they will write 25 alternative headlines for each post they write. Now, I usually don’t go that far, as I find I can get a few good ones out of just writing 10 or so. However, I’d recommend writing as many as you feel necessary to get a couple of headlines worth testing.
2. Select two of the best
Once I’ve written out a big list of headlines, I’ll then pick the two I think will work best.
This is a bit of a subjective area, and I certainly don’t have a checklist or concrete guidelines by which I choose the two best, but here are some of the things I think about when narrowing my list down to two:
- Likelihood of resonating – When it comes to blog headlines, it’s all about resonating with your target audience. I’m fortunate that I get to write content for people like myself (other marketers), so when picking the two best headlines, I tend to gravitate towards the ones that I’d be most likely to click-through on with the belief that other marketers will feel the same way.
- Fit with post – There is nothing worse than clicking through on a link from social media or search that promises something but doesn’t deliver. I always make sure the headline I choose isn’t overpromising or sensationalizing, as while it might drive a few extra visitors, those visitors will most likely leave with a bad impression of our content as the headline I chose incorrectly set their expectations.
- Fit with brand – I know we could go out and write headlines like “We sent this one email to our list. What happens next will shock you” and it will probably drive more clicks and traffic just like it does for sites like Upworthy—but that doesn’t fit with the Campaign Monitor brand. We’ve built this blog and our business by being sincere and helpful, and while spammy headlines like that may drive traffic, they won’t help our business or our readers in the long run.
3. Set up a subject line test
Once we’ve narrowed our list down to two headlines, we set up a subject line test for the email we send out to our blog subscribers.
Not surprisingly, we use Campaign Monitor to send all our email communications, and setting up the subject line test couldn’t be easier.
You simply select “Subject line test” from the testing options, input your two subject lines, and then go about creating the rest of your email as normal.
Campaign Monitor will send one version of the email to one part of the list (we choose to send each version to 10 percent of our list), and the other version to another part. It’ll then measure the results and automatically send the best performing email to the rest of the list.
Aside from adding in the additional subject line, there is literally no extra work in sending a subject line test in comparison to sending a normal email campaign.
4. Measure the results
After the designated amount of time the test runs (we usually choose two hours), we’ll receive an email saying the results are in.
I’ll then go in and look at the reports to better understand how the test went.
Here’s what the A/B test report looked like for our recent post on how to use Google Analytics to measure the success of your email marketing campaign:
We generally base the winner off of Open Rates as the act of opening the email is very similar to the act of clicking through from social media or search engines. As you can see, Version A was the clear winner with a solid 57% increase in opens over Subject Line B.
5. Revise the blog post title
Once the results are in and a winner has been decided, we’ll go back and modify the headline of the article on the blog itself.
That way, when people share the article via social media or when search engines index the article in their results pages, the headline with a 57% increase in opens will be what gets used—driving significantly more people back to the blog than it would have had we not conducted the test and optimized the headline.
Why email is a great way to A/B test blog headlines
I’ve seen a number of different approaches to A/B testing blog headlines. Upworthy has built their own system they refer to as the ‘magic unicorn box’, while the guys over at Buffer will post their different headlines to Twitter and measure the results there (using stats like the number of clicks and retweets to determine which post headline works best).
While no method is completely scientific, the reason I prefer testing blog headlines via email is that the sample size is controlled. When you run a subject line test, it will send each version to the exact same number of people and then select the winner based on how many of those people click the headline and open the email.
The issue with testing headlines via Twitter is you have no idea how many people saw each tweet. Sure, headline A might have 100 clicks while headline B only had 50, but what if headline A had been seen by 10,000 people and headline B was only seen by 500?
You’d end up optimising the post for headline A based on the number of clicks, yet in reality, headline A had a 1% click-through rate while headline B had a 10% click through rate.
Short of custom developing an A/B testing solution like Upworthy has, using email subject lines to test your blog headlines is the easiest and most accurate way to understand which headline is going to drive the most traffic.
When you’ve worked hard to create great blog content, A/B testing your blog headline via email is an easy way to ensure that content gets seen by as many readers as possible. It’s a simple process that can make a huge impact on the number of reads and shares.
We challenge you to A/B test some blog headlines in email to see which version works best. You know how the process works, so all that’s left to do is to give it a try.
How do you test your blog headlines? Have you found A/B testing via email to be an important part of your publishing process?
Want to improve your email marketing?
Join over 20,000 other marketers & designers who get tips on improving their email marketing delivered directly to their inbox.