Abstract
The control of spatio-temporally chaos is challenging because of highdimensionality and unpredictability. Model-free reinforcement learning (RL)discovers optimal control policies by interacting with the system, typicallyrequiring observations of the full physical state. In practice, sensors oftenprovide only partial and noisy measurements (observations) of the system. Theobjective of this paper is to develop a framework that enables the control ofchaotic systems with partial and noisy observability. The proposed method,data-assimilated model-informed reinforcement learning (DA-MIRL), integrates(i) low-order models to approximate high-dimensional dynamics; (ii) sequentialdata assimilation to correct the model prediction when observations becomeavailable; and (iii) an off-policy actor-critic RL algorithm to adaptivelylearn an optimal control strategy based on the corrected state estimates. Wetest DA-MIRL on the spatiotemporally chaotic solutions of theKuramoto-Sivashinsky equation. We estimate the full state of the environmentwith (i) a physics-based model, here, a coarse-grained model; and (ii) adata-driven model, here, the control-aware echo state network, which isproposed in this paper. We show that DA-MIRL successfully estimates andsuppresses the chaotic dynamics of the environment in real time from partialobservations and approximate models. This work opens opportunities for thecontrol of partially observable chaotic systems.