Abstract

Bayesian Neural Networks (BNNs) have recently received increasing attentionfor their ability to provide well-calibrated posterior uncertainties. However,model selection---even choosing the number of nodes---remains an open question.Recent work has proposed the use of a horseshoe prior over node pre-activationsof a Bayesian neural network, which effectively turns off nodes that do nothelp explain the data. In this work, we propose several modeling and inferenceadvances that consistently improve the compactness of the model learned whilemaintaining predictive performance, especially in smaller-sample settingsincluding reinforcement learning.

 

Quick Read (beta)