Abstract
This work proposes an approach that integrates reinforcement learning andmodel predictive control (MPC) to solve finite-horizon optimal control problemsin mixed-logical dynamical systems efficiently. Optimization-based control ofsuch systems with discrete and continuous decision variables entails the onlinesolution of mixed-integer linear programs, which suffer from the curse ofdimensionality. Our approach aims to mitigate this issue by decoupling thedecision on the discrete variables from the decision on the continuousvariables. In the proposed approach, reinforcement learning determines thediscrete decision variables and simplifies the online optimization problem ofthe MPC controller from a mixed-integer linear program to a linear program,significantly reducing the computational time. A fundamental contribution ofthis work is the definition of the decoupled Q-function, which plays a crucialrole in making the learning problem tractable in a combinatorial action space.We motivate the use of recurrent neural networks to approximate the decoupledQ-function and show how they can be employed in a reinforcement learningsetting. Simulation experiments on a microgrid system using real-world datademonstrate that the proposed method substantially reduces the onlinecomputation time of MPC while maintaining high feasibility and lowsuboptimality.