The research delves into the challenge of sample-based inference (SBI) for Bayesian neural networks (BNNs) due to the size and structure of the network’s parameter space. The authors propose that successful SBI can be achieved by understanding the relationship between weight and function space. They present an in-depth analysis of the link between overparameterization and the complexity of the sampling problem. Through extensive experiments, they provide practical guidelines for sampling and convergence diagnosis. The paper concludes by presenting a Bayesian deep ensemble approach as an effective solution with competitive performance and robust uncertainty quantification.
Publication date: 2 Feb 2024
Project Page: https://arxiv.org/abs/2402.01484v1
Paper: https://arxiv.org/pdf/2402.01484
