Good Initializations of Variational Bayes for Deep Models

  • 2018-10-18 14:35:23
  • Simone Rossi, Pietro Michiardi, Maurizio Filippone
  • 5

Abstract

Stochastic variational inference is an established way to carry outapproximate Bayesian inference for deep models. While there have been effectiveproposals for good initializations for loss minimization in deep learning, farless attention has been devoted to the issue of initialization of stochasticvariational inference. We address this by proposing a novel layer-wiseinitialization strategy based on Bayesian linear models. The proposed method isextensively validated on regression and classification tasks, includingBayesian DeepNets and ConvNets, showing faster convergence compared toalternatives inspired by the literature on initializations for lossminimization.

 

Quick Read (beta)

loading the full paper ...