Style Campaign lifts their click-through rate by 26% with A/B testing

The following post comes courtesy of our guest blogger, Anna Yeaman, from Style Campaign, an email marketing and design agency based in Los Angeles, CA. Through a combination of A/B testing with Campaign Monitor and use of animation, Style Campaign discovered that it could lift an email campaign’s click-through rate by an additional 26%.

{title}

I recently ran an A/B test on an email campaign on behalf of our client, Dressed Up!, to determine if embedding an animated image increases click-throughs. In order to do so, I prepared two emails with a slight variation (click to view emails in a new window):

The animated GIF in Email Version B consists of six frames and plays in the majority of email clients:

{title}

Testing the waters

Initially I stuck to subject line tests. These take seconds to set up and the payoff can be substantial. For example, here are the subject lines we used for our September newsletter:

  • Subject line A: ‘[firstname,fallback=Check out] - Mobile & Animated Emails’
  • Subject Line B: ‘Texting (SMS) & Email :: 360 Product Photo :: Animated GIFs’

My money was on the short, personalized subject line. However, I couldn’t have been more wrong! Subject line B increased our open rate by 250%. Bolstered by the discovery that a five-year-old could set up an A/B test, I started to test email content.

Setting up the A/B test

Under the A/B split campaign tab, I was given three choices:

  1. Subject Line
  2. Email Content
  3. From Name

I selected Email Content for use in this A/B test. At this stage I also entered the subject line, ‘VIDEO: Susan Boyle Makeover :: Fall Tadashi Gowns Modelled’, which would be used for both versions:

{title}

As normal, you are prompted to import your HTML email creative. The only difference is that you do this twice. Then, after selecting my recipients, I moved on to defining the A/B split.

I’ll admit that the segmentation feature intimidates me, so I was relieved that this part of the process was so intuitive. Dragging the bar, I set it up so that 25% of Dressed Up! subscribers received Version A and 25% received Version B. The remaining 50% would receive the winning version.

Underneath I selected, ‘Total clicks on selected link’. This allowed me to determine a winner based on the number of clicks the video graphic received, rather than total unique clicks:

{title}

Finally I set the A/B test to run for 5 hours, after which time the winner would be sent to the remaining subscribers.

It takes only a couple of minutes to set up an A/B content test. I had no problems on my first run though, without reading any documentation!

The Results

Overall, Version B generated a 26% increase in the click-through rate, in comparison to the non-animated version of the email. One explanation is that the animated GIF draws the eye, further acting as a visual cue by crudely simulating video.

My client was thrilled by the results, which I presented to him as a PDF, via the Campaign Snapshot page. 

I’ll be doing more A/B tests to refine this technique, such as testing three animated frames vs. nine frames, to see if more images result in higher conversions. On top of this I’ll be testing a horizontal vs. vertical layouts, after being inspired by this unique Abercrombie & Fitch email.

I’d love to hear from other Campaign Monitor users, what A/B tests have you carried out?

- Anna Yeaman, http://www.stylecampaign.com
Follow Style Campaign on Twitter (@stylecampaign)

To learn more about embedding video into email, visit our resource, ‘The Current State of Video in Email’

Posted by Ros Hodgekiss

4 Comments

  • Mr D
    29th October

    More of a SPAM question. The email has a lot of graphics with a large animated gif (file size), capital letters and not a lot of text. Surely this has a high spam rating?

  • Anna Yeaman
    29th October

    The email passed the CM Spam test at 100%: http://stylecampaign.com/blog/blogimages/spamcheck.png It does have a lot of graphics, but there is HTML text in the pre-header and footer, the caps are HTML as is the copy in the video player. Rather than one big image its broken down into many so it loads fast.

    It’s not perfect, given more time I would have made the bottom bar with the map info HTML, but overall the email worked for Dressed Up! generating a 45.23% click-through rate.

    - Anna

  • Steve M
    25th December

    Any guide out there on how to code the animated gif ? 
    It would be greatly appreciated!

    Thanks,
    Steve

  • Ros Hodgekiss
    30th December

    Hi Steve, you can use software like Adobe Fireworks to create, plus have a lot of control over your animated GIFs, or alternately, use an online tool like Picasion (although I can’t vouch for the end result).

Sign up for free.
Then send campaigns for as little as $9/month

Create an account