A/B Testing: Split the whole list?

First of all, I do love A/B testing, it's been very helpful.  But the ultimate goal of testing for us is not just to increase clicks, but to increase sales.  I know CM doesn't track sales, which is fine.  I've got the tracking covered.

Comparing versions is tricky though, with the current limit of only splitting up to half the list - most people don't buy immediately, so if I pick a winner quickly based on sales, it may not have been the winner over a longer period of time.  But I can't wait a week or two to send the winner, because the sale would be nearly over, and an entire 50% of the list would get very short notice!

It would be very helpful if I could do a 50/50 split on my entire list, not just up to half with the rest delayed.  That way, both versions would get a chance to compete for sales over the same amount of time, and at the end of the offer, I could compare them equally and see which did better. 

Is there any chance of that becoming an option?  Does anyone have other suggestions on how to compare email versions over a longer period?

roshodgekiss roshodgekiss, 7 years ago

Hi wtkad, welcome to the forums! I've added your vote internally for this feature. My suggestion is, when setting the period during which to choose the winner of the A/B split, set it to a number of days. If after a day or so, a particular version generates more sales, then manually select it as the winner. All the best with your campaign!

Get in touch with us on Twitter: http://twitter.com/campaignmonitor
We're also on Facebook: http://facebook.com/campaignmonitor

See why 200,000 companies worldwide love Campaign Monitor.

From Australia to Zimbabwe, and everywhere in between, companies count on Campaign Monitor for email campaigns that boost the bottom line.

Get started for free