Abstract
Model-based reinforcement learning (MBRL) is believed to have much highersample efficiency compared to model-free algorithms by learning a predictivemodel of the environment. However, the performance of MBRL highly relies on thequality of the learned model, which is usually built in a black-box manner andmay have poor predictive accuracy outside of the data distribution. Thedeficiencies of the learned model may prevent the policy from being fullyoptimized. Although some uncertainty analysis-based remedies have been proposedto alleviate this issue, model bias still poses a great challenge for MBRL. Inthis work, we propose to leverage the prior knowledge of underlying physics ofthe environment, where the governing laws are (partially) known. In particular,we developed a physics-informed MBRL framework, where governing equations andphysical constraints are utilized to inform the model learning and policysearch. By incorporating the prior information of the environment, the qualityof the learned model can be notably improved, while the required interactionswith the environment are significantly reduced, leading to better sampleefficiency and learning performance. The effectiveness and merit have beendemonstrated over a handful of classic control problems, where the environmentsare governed by canonical ordinary/partial differential equations.