Making Sense of Uncertainty: A New Approach to Bayesian Inference

Bayesian statistics is a powerful tool for making sense of data. It helps us understand the uncertainty around our estimates. But what happens when our models are not perfect? This is called model misspecification. In such cases, Bayesian posteriors may not accurately reflect the true uncertainty. Moreover, different datasets from the same true distribution can lead to different posteriors, making the results hard to reproduce. To tackle this issue, researchers have proposed a new criterion. They focus on the probability that two credible sets from independent datasets overlap. This overlap probability helps assess the reproducibility of uncertainty quantification. The standard posterior often fails to meet this criterion, showing inconsistency under misspecification. A solution to this problem is BayesBag. This method involves averaging posterior distributions from bootstrapped datasets. It is motivated by Jeffrey conditionalization and typically satisfies the overlap criterion. Additionally, the bagged posterior has an asymptotic normal distribution, as proven by a Bernstein-Von Mises theorem. The benefits of BayesBag are demonstrated through simulation experiments and an application to crime rate prediction. This approach offers a practical and widely applicable way to improve the reproducibility of Bayesian inference under model misspecification.