Probably Safe Sim-to-Real Transfer with Lifetime Validity

  • 2026-11-06 11:41:47
  • Rohan Tangri, Jan-Peter Calliess
  • 0

Abstract

We present Dejavu, a method that makes sim-to-real transfer of reinforcement learning policies in constrained Markov decision processes probably safe over an unbounded sequence of policy updates. An anytime valid test using per-decision importance sampling and a betting confidence sequence certifies each candidate trained in simulation before deployment using only trajectories collected by a safe baseline policy operating in production. Simulation quality does not affect the safety guarantee, only the production sample efficiency of the method. Finally, a summable per-test failure probability yields the lifetime valid property. Empirical results across three sim-to-real benchmarks support the theoretical work.

 

Quick Read (beta)

loading the full paper ...