Back to glossary

Bayesian testing

The Bayesian approach to A/B testing asks questions about the world that generated the data. for example, “What are the most probable values of the uplift?” or “What is the probability that the uplift is positive?” This differs from the Frequentist approach which instead asks questions about the data, for example, “What is the probability I’d see something more surprising if I ran my test again 1000 times?”

How is Bayesian testing different from other kinds of A/B testing?

The practical difference in the Bayesian approach to A/B Testing is that you can start looking at your test results on day one, maybe even after the first hour, just by reading the probability of whether A and B is better than the other. Doing so does not affect the false discovery rate, which is automatically controlled by Bayesian techniques. 

That’s not to say only the Bayesian questions matter; Qubit also guarantees the frequentist properties of its Bayesian A/B tests.

Back to glossary

We use cookies and other forms of website navigational information to offer you a better browsing experience, analyze site traffic, personalize content, and serve targeted advertisements.
Read about how we use cookies and how you can control them in our Privacy and Cookie Policy. If you continue to use this site, you consent to our use of cookies.

Accept Cookies