Certificated Actor-Critic: Hierarchical Reinforcement Learning with Control Barrier Functions for Safe Navigation

  • 2025-01-29 05:37:47
  • Junjun Xie, Shuhao Zhao, Liang Hu, Huijun Gao
  • 0

Abstract

Control Barrier Functions (CBFs) have emerged as a prominent approach todesigning safe navigation systems of robots. Despite their popularity, currentCBF-based methods exhibit some limitations: optimization-based safe controltechniques tend to be either myopic or computationally intensive, and they relyon simplified system models; conversely, the learning-based methods suffer fromthe lack of quantitative indication in terms of navigation performance andsafety. In this paper, we present a new model-free reinforcement learningalgorithm called Certificated Actor-Critic (CAC), which introduces ahierarchical reinforcement learning framework and well-defined reward functionsderived from CBFs. We carry out theoretical analysis and proof of ouralgorithm, and propose several improvements in algorithm implementation. Ouranalysis is validated by two simulation experiments, showing the effectivenessof our proposed CAC algorithm.

 

Quick Read (beta)

loading the full paper ...