Home Resources Blog

It’s been a long time coming, but today I’m excited to announce that our new A/B testing feature we announced a while back is now live and available in all accounts.


Like we mentioned in the original announcement, we’ve worked hard to make sending an A/B campaign just as easy as sending a regular one. You can test different subject lines or sender details in a matter of seconds. If you have the time to create a simple design variation for your campaigns, this is also a breeze to test. I really can’t think of a good reason to send a regular old campaign any more.

The best part, you learn something from every single test you run.

One of my favorite new features that come with the A/B testing update are benefit estimates. Basically, after the test is run and the winning version is sent to the remainder of your subscribers, we’ll look at how much better the winning version performed in your test and extrapolate that across the rest of your campaign results. Check out the screenshot below for a campaign that received an additional 819 clicks by sending the winning version.


Benefit estimates give you a great indication on the total benefit gained from running the test. It’s also a great way to justify testing to your clients or the powers that be.

Now that A/B testing is out, we can roll out a host of other sweet new features we’ve been hard at work on. To help you get the most out of A/B testing, we’ll also be starting a new blog series next week with loads of info on how to plan a test, what you can test and different approaches you and your clients might want to try.

  • Mike Bowzeylo

    Very sweet tool and super slick implementation. You’ve not only done all the hard work…you’ve done _all_ the work. We just get to basically sit back and enjoy all the statistic goodness. Good job guys.

  • Aaron

    Great feature! I can’t wait to test it out on our newsletter.

  • g

    Should be handy.

  • g

    Is there any way to turn the feature off for specific clients?

    (sorry for the split post!)

  • David Greiner

    @Mike and @Aaron Thanks so much guys, looking forward to seeing what you do with it.

    @g Not right now no, this is available for all clients that you’ve given access to create and send their own campaigns. If there is the demand though, we can certainly add this is an optional piece of functionality you can toggle on and off in client settings.

    Anyone else got thoughts on this?

  • David Bell

    Awesome new feature CM, this is going to really help fine tune our campaigns.

  • Derek

    I’d be curious to no the baseline variance if you sent the exact same message to both A and B? It should theoretically be zero but I sort of doubt it is.

  • David Greiner

    Good call Derek, only one way to find out I guess. If the list size was big enough, there really shouldn’t be much deviation.

  • Bjorn van der Neut

    Congratulations with this big breakthru!

    Excitiing to go and test this out…first have to make my template inserted in the campaing monitor envoirement.

  • Paula

    are you able to test for different times of the day as well as different subject lines?

  • Jop Berkhout

    Great functionality, I’m going to test this right away!

  • David Greiner

    @Paula The ability to test different times of day was highly requested when we announced the feature, and it’s definitely something we plan on supporting soon. Right now you can test on subject, sender details and different email content.

  • Phil

    @David – Yes, we would really like this to be optional. We have a wide range of abilities across our clients. Some will want to use this themselves, others will want us to do it for them.

  • David Greiner

    @Phil great, thanks for confirming that. We’ve got some nice improvements to the client permissions screen coming soon, and I’ve added a vote for the ability to set A/B tests as optional for clients.

    The more people we hear this from, the sooner it will likely happen.

  • Eric Oliver

    This is awesome. Additionally, I would just like to chime in and second Paula on the time of day testing — that’s been the most crucial testing we’ve done historically.

    Thanks again!

  • Brace Rosso

    Such awesome timing. My email best practices presentation next week can now include this important feature. Thank you!

  • Tom Davies

    very cool – will this be accessible via the API?

  • Tobias

    At last! Thank you! Really nice feature!

  • Terry Brandon

    Glad to see the functionality. Looks good.

    I see where you declare the winner. Do you also calculate & show the confidence level that the long term response rate differences will sustain as currently shown?. SplitTester.com does something comparable for PPC advertising.

  • Sean Wood

    To back up Terry, I’m also wondering if, amongst all the other fantastic things CM is working on, a statistical significance calculator could be thrown in so that we can see if the difference between A & B is enough to be relevant.

    Thanks so much though, for this. Objectivity needs to be king in the professional world, and features like this really put shine on its crown.

  • zed

    Nice job. I want to add my vote for:
    being able to turn it on and off for clients,
    testing time of day,
    and API access.
    Great so far! Now make it completely irresistible.

  • Jonathan Saipe

    I was very much looking forward to this feature as we are constantly battling over our subject lines. Will have fun with it I’m sure.

  • Eamonn Dunne

    Another vote for the ability to turn on and off for clients. Thanks for a superb feature!

  • Lars Buur

    Voting for API access

  • Robin Morris

    Absolutely brilliant… I love it!

  • Jeremy Glover

    Vote: API Access

  • Ola

    Would be good to turn on/off for clients.

    Abit offtopic:
    I work as a Webmaster for a large group. Im using campaignmonitor for all of the companies we own. I would like a feature to add subaccounts to have somewhat like this: superadmin (have all rights) – client admin (has the rights superadmin set to specific account) – client subuser (has the rights clientadmin set to clientadmins account) Some companies want to send themselves and have different roles in a company, someone is a marketing coordinator and someone is a Product Area Manager. Se what i mean?

  • smallfurrything

    @David Greiner Yes this shoul dbe optional as some of our clients have very small lists and this sin’t appropriate for them. Some clients may stop using this if they can’t switch it off.
    Otherwise it’s a veyr usefull feature and well implemented so thanks CM!

  • Al

    How do I include a photo when creating a campaign ?

  • David Greiner

    Al, if you can code your own HTML, you just do this on your own end and then import the complete campaign into Campaign Monitor.

  • ziki

    that’s one of my wish list and it’s awesome…pretty easy to use though

  • Ziki

    API access please, great job, guys

  • Mark Skeet

    Votes for turn on and off for specific clients and A/b based on time of day.

  • mark

    Awesome – another reason I am glad for choosing CM. Next mail shot will definitely try this out.

  • S Jain

    Very useful feature. I had been waiting for this feature to come to campaign monitor.

  • Shirley Hemphill

    I’m looking forward to the new blog on the A/B testing. This will be a very useful feature and I’m looking forward to using it.

    On another note: I would love to go to Australia next week for the conference, but will not be able to as it’s too far and too much money. Could you send me a shirt?

  • Staffan Landin

    Cool, will be very useful for us.

    Only one thing though, why can I only send each version to 25%+25% to find a winner.

    We would like to test a few functions on the whole group 50%-50% and then analyze the results to learn lessons for the next letters.

  • Claire Diiorio

    Fantastic tool, thanks for improving the service!

  • Ros Hodgekiss

    Our pleasure. We have some great guides in the blog now, as well – https://www.campaignmonitor.com/blog

    If you have any suggestions as to how we can make A/B testing even better, don’t hesitate to drop us a line.

This blog provides general information and discussion about email marketing and related subjects. The content provided in this blog ("Content”), should not be construed as and is not intended to constitute financial, legal or tax advice. You should seek the advice of professionals prior to acting upon any information contained in the Content. All Content is provided strictly “as is” and we make no warranty or representation of any kind regarding the Content.
Straight to your inbox

Get the best email and digital marketing content delivered.

Join 250,000 in-the-know marketers and get the latest marketing tips, tactics, and news right in your inbox.


See why 200,000 companies worldwide love Campaign Monitor.

From Australia to Zimbabwe, and everywhere in between, companies count on Campaign Monitor for email campaigns that boost the bottom line.

Get started for free