Most Winning A/B Test Results are Illusory

Most Winning A/B Test Results are Illusory
By Martin Goodson (DPhil)

Marketers have begun to question the value of A/B testing, asking: ‘Where is my 20% uplift? Why doesn’t it ever seem to appear in the bottom line?’ Their A/B test reports an uplift of 20% and yet this increase never seems to translate into increased profits. So what’s going on?

In this article I’ll show that badly performed A/B tests can produce winning results which are more likely to be false than true. At best, this leads to the needless modification of websites; at worst, to modification which damages profits.

Martin Goodson (MPhil)
Research Lead, Qubit

Defy tomorrow’s expectations today

Start personalizing your experiences with rich customer data today