Provably Safe Model-Based Meta Reinforcement Learning: An Abstraction-Based Approach

  • 2021-09-03 00:38:05
  • Xiaowu Sun, Wael Fatnassi, Ulices Santa Cruz, Yasser Shoukry
  • 1


While conventional reinforcement learning focuses on designing agents thatcan perform one task, meta-learning aims, instead, to solve the problem ofdesigning agents that can generalize to different tasks (e.g., environments,obstacles, and goals) that were not considered during the design or thetraining of these agents. In this spirit, in this paper, we consider theproblem of training a provably safe Neural Network (NN) controller foruncertain nonlinear dynamical systems that can generalize to new tasks thatwere not present in the training data while preserving strong safetyguarantees. Our approach is to learn a set of NN controllers during thetraining phase. When the task becomes available at runtime, our framework willcarefully select a subset of these NN controllers and compose them to form thefinal NN controller. Critical to our approach is the ability to compute afinite-state abstraction of the nonlinear dynamical system. This abstract modelcaptures the behavior of the closed-loop system under all possible NN weights,and is used to train the NNs and compose them when the task becomes available.We provide theoretical guarantees that govern the correctness of the resultingNN. We evaluated our approach on the problem of controlling a wheeled robot incluttered environments that were not present in the training data.


Quick Read (beta)

loading the full paper ...