credible interval

Barlow and Sidebotham, writing in BJA Education, define a credible interval as an interval encompassing most of the posterior distribution, and so an interval estimate for the quantity being estimated. A 95 percent credible interval includes the values that encompass 95 percent of that distribution. The posterior distribution is what a Bayesian analysis produces: a prior distribution, the starting position on how large an effect might be, updated by the study data to give a distribution of how likely each possible value now is. The authors add that reporting 95 percent rather than some other share is merely a convention.

The difference from a confidence interval is not the arithmetic but what may be said afterwards. Bayesian inference treats the quantity of interest as having an uncertain value and represents that uncertainty as a probability distribution, where frequentist methods treat it as having a fixed but unknown value. Because the posterior is a probability distribution, the authors write, probabilities related to the treatment effect can be calculated from it, so a question such as whether an intervention reduces deaths by at least 5 percent gets a direct numerical answer. A frequentist confidence interval cannot be used to make probability statements about the size of a treatment effect.

One consequence changes how a reader should treat an interval that touches no effect. Barlow and Sidebotham note that including a risk difference of zero inside a 95 percent credible interval carries no special meaning, whereas a confidence interval containing the null leads to a result being declared not statistically significant. Statistical significance is not a concept Bayesian inference uses. So an interval running from just below no effect to just above it is a statement about how much of the probability sits on each side, and its width remains the honest measure of how firmly the estimate is pinned down.