Abstract
Recent work has demonstrated that Bayesian neural networks (BNN's) trainedwith mean field variational inference (MFVI) can be implemented in analoghardware, promising orders of magnitude energy savings compared to the standarddigital implementations. However, while Gaussians are typically used as thevariational distribution in MFVI, it is difficult to precisely control theshape of the noise distributions produced by sampling analog devices. Thispaper introduces a method for MFVI training using real device noise as thevariational distribution. Furthermore, we demonstrate empirically that thepredictive distributions from BNN's with the same weight means and variancesconverge to the same distribution, regardless of the shape of the variationaldistribution. This result suggests that analog device designers do not need toconsider the shape of the device noise distribution when hardware-implementingBNNs performing MFVI.