Bayesian analysis

Barlow and Sidebotham, writing in BJA Education, describe Bayesian inference as an alternative statistical framework to classical, or frequentist, inference, which has dominated scientific research for over 100 years. The difference is in what the quantity being estimated is taken to be. Bayesian inference treats the parameter of interest as having an uncertain value and represents that uncertainty as a probability distribution; frequentist methods treat it as having a fixed but unknown value, of which a single study's result is one realisation. A Bayesian model has three parts. The prior distribution represents what is known about the quantity before the data are collected, the likelihood represents the study data, and the posterior distribution is the prior updated by the likelihood, which is to say what is known afterwards. The authors note that the posterior from one study can become the prior for the next, so the uncertainty narrows as consistent results accumulate.

The prior is where the objection usually lands, and the authors meet it directly. Specifying a prior can lead to the idea that the outcome is subjective, and a strong prior can overwhelm the likelihood, skewing the result in favour of prior belief over the study data. Their answer is that a weak prior, one that leans neither way, typically yields results similar to frequentist methods, and that a full analysis uses a range of priors, neutral and enthusiastic, informative and flat, so a reader can see how much the conclusion depends on the starting position rather than on the data. They add that frequentist analysis makes subjective choices too, in the hypotheses, the significance level and the effect size assumed when the sample size was calculated, but makes them implicitly, which can leave the impression that it is purely objective.

Two things follow for anyone reading a result. Because the posterior is a probability distribution, probabilities about the treatment effect can be calculated from it, so a question such as whether an intervention reduces deaths by at least 5 percent gets a direct numerical answer, which a p-value or a confidence interval cannot give. And statistical significance is not a concept Bayesian inference uses: the interval reported is a credible interval, and including no effect inside one carries none of the meaning that the same thing carries in a confidence interval. The NIST and SEMATECH e-Handbook of Statistical Methods draws the same line in its own words, saying that Bayesian analysis considers population parameters to be random rather than fixed, and that old information, or subjective judgment, is used to determine a prior distribution for them.