Abstract
We introduce a gradient-based learning method to automatically adapt Markovchain Monte Carlo (MCMC) proposal distributions to intractable targets. Wedefine a maximum entropy regularised objective function, referred to asgeneralised speed measure, which can be robustly optimised over the parametersof the proposal distribution by applying stochastic gradient optimisation. Anadvantage of our method compared to traditional adaptive MCMC methods is thatthe adaptation occurs even when candidate state values are rejected. This is ahighly desirable property of any adaptation strategy because the adaptationstarts in early iterations even if the initial proposal distribution is farfrom optimum. We apply the framework for learning multivariate random walkMetropolis and Metropolis-adjusted Langevin proposals with full covariancematrices, and provide empirical evidence that our method can outperform otherMCMC algorithms, including Hamiltonian Monte Carlo schemes.