Abstract
Current approaches in approximate inference for Bayesian neural networksminimise the Kullback-Leibler divergence to approximate the true posterior overthe weights. However, this approximation is without knowledge of the finalapplication, and therefore cannot guarantee optimal predictions for a giventask. To make more suitable task-specific approximations, we introduce a newloss-calibrated evidence lower bound for Bayesian neural networks in thecontext of supervised learning, informed by Bayesian decision theory. Byintroducing a lower bound that depends on a utility function, we ensure thatour approximation achieves higher utility than traditional methods forapplications that have asymmetric utility functions. Furthermore, in usingdropout inference, we highlight that our new objective is identical to that ofstandard dropout neural networks, with an additional utility-dependent penaltyterm. We demonstrate our new loss-calibrated model with an illustrative medicalexample and a restricted model capacity experiment, and highlight failure modesof the comparable weighted cross entropy approach. Lastly, we demonstrate thescalability of our method to real world applications with per-pixel semanticsegmentation on an autonomous driving data set.