Abstract
This paper introduces the variational implicit processes (VIPs), a Bayesiannonparametric method based on a class of highly flexible priors over functions.Similar to Gaussian processes (GPs), in implicit processes (IPs), an implicitmultivariate prior (data simulators, Bayesian neural networks, etc.) is placedover any finite collections of random variables. A novel and efficientvariational inference algorithm for IPs is derived using wake-sleep updates,which gives analytic solutions and allows scalable hyper-parameter learningwith stochastic optimization. Experiments on real-world regression datasetsdemonstrate that VIPs return better uncertainty estimates and superiorperformance over existing inference methods for GPs and Bayesian neuralnetworks. With a Bayesian LSTM as the implicit prior, the proposed approachachieves state-of-the-art results on predicting power conversion efficiency ofmolecules based on raw chemical formulas.