If I had £1 for every time my team and I are asked about what drives the best results in optimization, I probably wouldn’t need to work at Qubit anymore. While we have always been able to give examples of high-performing experiences and innovative ideas, they’re only that: examples.

That’s why I’m so excited about the fantastic new piece of research we’re releasing, that will give you transparency into the average performance of 29 common customer experiences based on analysis of 2 billion user journeys. This is in many ways a continuation of the very influential work we did in 2013, showing that Most Winning A/B Test Results are Illusory, which transformed the way organizations measure performance.

Our new research is just as bold: we’re now able to prove statistically that some techniques consistently deliver positive revenue uplifts while many others do not. Unsurprisingly, some experiences reliably perform well, some are variable, some are unreliable, and some consistently perform poorly.

I think we all know where we’d prefer to focus our efforts and precious resources!

Our new research uses a methodology (catchily known as the Qubit methodology for the evaluation, classification and aggregation of online experiments in ecommerce) that has been independently assured by PwC. You can read the academic paper to delve into the statistics, and see the findings contextualized in our report: Getting 6% more.

PwC-mock-combines (1)-1.pngWill Browne - one of the Qubit minds behind the study - will be blogging in the next week or so to talk a bit more about the science behind the study, but in the meantime, we thought we’d provide a little more context for you - the people in charge of delivering personalization on your websites.

The headline findings are:

  • Segmentation is important: targeted experiences produce three times greater revenue uplift than blanket ones.
  • Scarcity, social proof, urgency and abandonment recovery are particularly high performers.
  • Cosmetic changes… aren’t.

EMAIL-IMAGE-STAT-graph (2).jpg

So, what does this mean for you?

To get the biggest reward for your efforts, targeting your experiences at the right visitors, based on their preferences and behavior, is crucial. This means making use of user segments.

If you’re short on time or capacity, target the right groups with the options with the better chances of success.

We’ve got lots of experience to help you with scarcity, social proof, urgency and addressing abandonment and recommendations.

And what doesn’t it mean?

These findings don’t mean you should stop all cosmetic changes, or evaluating their performance. Not at all. It just means that cosmetic changes - on average - aren’t as likely to perform as well. This means you need to set expectations accordingly, and be aware that, when it comes to revenue uplift, they’re riskier bets.

In conclusion

This report confirms what I suspect you all knew: that behavioral change drives revenue uplift and that focusing on the right audience multiplies the impact of your site enhancements.

This study should serve as inspiration and help you focus. When you are stretched, you can rely on this data to assist with prioritisation. When you have a great idea, the mean performance figures may help you gather support. And when the results of your experiences come in, you have an average for comparison.


Get your copy of the report here.


Laura Kightlinger

Read More

Subscribe to stay up to date.

Receive the personalization newsletter directly to your inbox.