Abstract
The application of reinforcement learning to safety-critical systems islimited by the lack of formal methods for verifying the robustness and safetyof learned policies. This paper introduces a novel framework that addressesthis gap by analyzing the combination of an RL agent and its environment as adiscrete-time autonomous dynamical system. By leveraging tools from dynamicalsystems theory, specifically the Finite-Time Lyapunov Exponent (FTLE), weidentify and visualize Lagrangian Coherent Structures (LCS) that act as thehidden "skeleton" governing the system's behavior. We demonstrate thatrepelling LCS function as safety barriers around unsafe regions, whileattracting LCS reveal the system's convergence properties and potential failuremodes, such as unintended "trap" states. To move beyond qualitativevisualization, we introduce a suite of quantitative metrics, Mean BoundaryRepulsion (MBR), Aggregated Spurious Attractor Strength (ASAS), andTemporally-Aware Spurious Attractor Strength (TASAS), to formally measure apolicy's safety margin and robustness. We further provide a method for derivinglocal stability guarantees and extend the analysis to handle model uncertainty.Through experiments in both discrete and continuous control environments, weshow that this framework provides a comprehensive and interpretable assessmentof policy behavior, successfully identifying critical flaws in policies thatappear successful based on reward alone.