Abstract
Deep reinforcement learning (DRL) has shown significant promise foruncovering sophisticated control policies that interact in environments withcomplicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamakfusion reactor or minimizing the drag force exerted on an object in a fluidflow. However, these algorithms require an abundance of training examples andmay become prohibitively expensive for many applications. In addition, thereliance on deep neural networks often results in an uninterpretable, black-boxpolicy that may be too computationally expensive to use with certain embeddedsystems. Recent advances in sparse dictionary learning, such as the sparseidentification of nonlinear dynamics (SINDy), have shown promise for creatingefficient and interpretable data-driven models in the low-data regime. In thiswork we introduce SINDy-RL, a unifying framework for combining SINDy and DRL tocreate efficient, interpretable, and trustworthy representations of thedynamics model, reward function, and control policy. We demonstrate theeffectiveness of our approaches on benchmark control environments andchallenging fluids problems. SINDy-RL achieves comparable performance tostate-of-the-art DRL algorithms using significantly fewer interactions in theenvironment and results in an interpretable control policy orders of magnitudesmaller than a deep neural network policy.