Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning

  • 2022-05-02 00:25:44
  • Kevin Zeng, Alec J. Linot, Michael D. Graham
  • 0

Abstract

Deep reinforcement learning (RL) is a data-driven method capable ofdiscovering complex control strategies for high-dimensional systems, making itpromising for flow control applications. In particular, the present work ismotivated by the goal of reducing energy dissipation in turbulent flows, andthe example considered is the spatiotemporally chaotic dynamics of theKuramoto-Sivashinsky equation (KSE). A major challenge associated with RL isthat substantial training data must be generated by repeatedly interacting withthe target system, making it costly when the system is computationally orexperimentally expensive. We mitigate this challenge in a data-driven manner bycombining dimensionality reduction via an autoencoder with a neural ODEframework to obtain a low-dimensional dynamical model from just a limited dataset. We substitute this data-driven reduced-order model (ROM) in place of thetrue system during RL training to efficiently estimate the optimal policy,which can then be deployed on the true system. For the KSE actuated withlocalized forcing ("jets") at four locations, we demonstrate that we are ableto learn a ROM that accurately captures the actuated dynamics as well as theunderlying natural dynamics just from snapshots of the KSE experiencing randomactuations. Using this ROM and a control objective of minimizing dissipationand power cost, we extract a control policy from it using deep RL. We show thatthe ROM-based control strategy translates well to the true KSE and highlightthat the RL agent discovers and stabilizes an underlying forced equilibriumsolution of the KSE system. We show that this forced equilibrium captured inthe ROM and discovered through RL is related to an existing known equilibriumsolution of the natural KSE.

 

Quick Read (beta)

loading the full paper ...