Blog

Update: A/B testing has been released and is available in all accounts.

This one has been hard to keep under wraps, but after loads of customer requests and a few months of hard work, I'm happy to announce that we'll shortly be adding support for A/B testing to Campaign Monitor. This is one of those features that we always wanted to add, but were never completely satisfied with the best way to implement it.

Testing is one of the great strengths of email marketing. Almost everything is measurable, making it easy to compare multiple approaches and quickly figure out what works best for your subscribers. It's a guesswork killer.

That's the theory anyway. In reality, most of the A/B testing systems we've seen are bloated and hard to use beasts that require a manual just to get started. That never sat well with us, so we went back to the drawing board. We knew that if running an A/B test wasn't as quick and easy as sending a regular campaign, it was never going to get used.

We plan on releasing this in the next month or so, but wanted to give you all the heads up on what's coming. The testing feature will be available for everyone, including your clients if you've given them access to send their own campaigns.

Choose the test, we'll do the rest

Instead of forcing you to create segments of lists, send multiple campaigns and scour over pages of statistics, our testing tool does all the heavy lifting for you.

All you need to do is choose what to test (subject line, from name or different email content) and how to decide the winner (open rate, link clicks, etc) and we'll take it from there.

Campaign Monitor will then send both versions to a small subset of your recipients, see which version wins and automatically send that winning version to the remainder of your recipients. In a couple of clicks you've guaranteed that the best version of your email is being sent to your subscribers, and you've learned something doing it.

While we'll be sharing more details when A/B testing goes live in the next month or so, here's a sneak peek at how the process will look in your account.

A new type of campaign to send

When you create a new campaign, you'll notice a new tab for running an A/B split campaign. Clicking that will let you choose what test you want to run and get started.

Setting the test parameters

A simple slider makes it easy to choose the size of your list you'd like to test on. Next choose how to pick the winner and how long the test should run for.

Watching a test in progress

For all you stats junkies out there, this sweet report gives you a bird's eye view of a test as it's running. When the winner is decided, we'll send it to the rest of your list automatically. If you like, you can pre-empt this and send any time during the test.

Post-test report with benefit estimates

After the test is run and the winning version sent to the remainder of your recipients, you can access the full results of the test. We'll also extrapolate the performance improvement of the winning version to give you an idea of the total benefit of sending that version to the remainder of your list. It's a great way to justify testing to your client or boss.

When will this be available?

We're putting the finishing touches on this feature now before running it through our usual phase of heavy testing (thanks to our new QA engineer Trish). While we can't give an exact date just yet, we're hoping to have this available in all accounts within a months time. We plan on sharing more as we get closer to launch.

If you have any feedback (positive or negative) or questions about this new feature, we'd love to hear them below.