Abstract
While model-based deep reinforcement learning (RL) holds great promise forsample efficiency and generalization, learning an accurate dynamics model isoften challenging and requires substantial interaction with the environment. Awide variety of domains have dynamics that share common foundations like thelaws of classical mechanics, which are rarely exploited by existing algorithms.In fact, humans continuously acquire and use such dynamics priors to easilyadapt to operating in new environments. In this work, we propose an approach tolearn task-agnostic dynamics priors from videos and incorporate them into an RLagent. Our method involves pre-training a frame predictor on task-agnosticphysics videos to initialize dynamics models (and fine-tune them) for unseentarget environments. Our frame prediction architecture, SpatialNet, is designedspecifically to capture localized physical phenomena and interactions. Ourapproach allows for both faster policy learning and convergence to betterpolicies, outperforming competitive approaches on several differentenvironments. We also demonstrate that incorporating this prior allows for moreeffective transfer between environments.