Last Thursday was the fifth annual WhichTestWon Email Awards Ceremony. Thanks to their annual award shows, but also recurring site-items such as ‘Test Of The Week’, ‘Behind The Scenes’ and ‘The Live Event’, American company WhichTestWon is an authority in the field of Conversion Rate Optimization. This year, the Netherlands won 2 out of 3 awards, which were coincidentally both won by a lottery. Vriendenloterij won in the category ‘best e-mail blast’, and Staatsloterij and Blue Mango Interactive won ‘best call to action’. The latter case will be examined below.

Test details & hypothesis###

Usually, Staatsloterij will send out an email a few days before an upcoming draw to people who have played before, as a reminder. Because of the timing, a few days before the draw, the focus of the email and the resulting landing page is on single draw tickets. However, subscriptions carry more certainty and are therefore advantageous for Staatsloterij. But pushing subscriptions in the email may have an adverse effect on single ticket sales, so we set up a test in which we added the subscription-link in a subtle way. This led to the hypothesis that adding a secondary CTA for subscriptions (called ‘play automatically’), followed by the incentive of receiving a free ticket upon signup worth €15, will lead to more subscription sales without cannibalizing single draw ticket sales too much.

Control & variant:####

The results###

The variation with the added text link led to an uplift of 335% in subscription orders. As a side effect, single draw orders dropped by 33%, but when calculated to a customer lifetime value, the uplift in subscriptions greatly outweighed the drop in single tickets, crowning the text link variation as the clear winner in this test.

Learnings###

Small changes can lead to big results. Simply adding a text link to the CTA resulted in a dramatic shift in sales of single draw tickets versus subscriptions. This is a reminder that successful tests don’t aways have to be radical, the strength of this case is in it is simplicity.

Verdict###

Winning two of three global awards, the Netherlands is a leader in the field of Email Conversion Optimization. So if you are not structurally A/B-testing within your email campaigns yet, my advice would be to start doing it. Not only is email a channel that converts relatively well, it’s also a channel that can show you a lot of data in a short amount of time, where you’ll often need to have more patience with other media. Further, it turns out the Butterfly Effect also counts for email optimization, because the fact that small changes can generate a big impact, was one of the key takeaways during the WhichTestWon Email Awards 2016.

More inspiring cases and the full award show can be found here.