Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation

  • 2020-06-29 17:28:45
  • Steven Kleinegesse, Michael U. Gutmann
  • 0


Implicit stochastic models, where the data-generation distribution isintractable but sampling is possible, are ubiquitous in the natural sciences.The models typically have free parameters that need to be inferred from datacollected in scientific experiments. A fundamental question is how to designthe experiments so that the collected data are most useful. The field ofBayesian experimental design advocates that, ideally, we should choose designsthat maximise the mutual information (MI) between the data and the parameters.For implicit models, however, this approach is severely hampered by the highcomputational cost of computing posteriors and maximising MI, in particularwhen we have more than a handful of design variables to optimise. In thispaper, we propose a new approach to Bayesian experimental design for implicitmodels that leverages recent advances in neural MI estimation to deal withthese issues. We show that training a neural network to maximise a lower boundon MI allows us to jointly determine the optimal design and the posterior.Simulation studies illustrate that this gracefully extends Bayesianexperimental design for implicit models to higher design dimensions.


Quick Read (beta)

loading the full paper ...