Abstract
In recent studies on model-based reinforcement learning (MBRL), incorporatinguncertainty in forward dynamics is a state-of-the-art strategy to enhancelearning performance, making MBRLs competitive to cutting-edge model freemethods, especially in simulated robotics tasks. Probabilistic ensembles withtrajectory sampling (PETS) is a leading type of MBRL, which employs Bayesianinference to dynamics modeling and model predictive control (MPC) withstochastic optimization via the cross entropy method (CEM). In this paper, wepropose a novel extension to the uncertainty-aware MBRL. Our main contributionsare twofold: Firstly, we introduce a variational inference MPC, whichreformulates various stochastic methods, including CEM, in a Bayesian fashion.Secondly, we propose a novel instance of the framework, called probabilisticaction ensembles with trajectory sampling (PaETS). As a result, our BayesianMBRL can involve multimodal uncertainties both in dynamics and optimaltrajectories. In comparison to PETS, our method consistently improvesasymptotic performance on several challenging locomotion tasks.