Abstract
Empowering safe exploration of reinforcement learning (RL) agents duringtraining is a critical challenge towards their deployment in many real-worldscenarios. When prior knowledge of the domain or task is unavailable, trainingRL agents in unknown, black-box environments presents an even greater safetyrisk. We introduce ADVICE (Adaptive Shielding with a Contrastive Autoencoder),a novel post-shielding technique that distinguishes safe and unsafe features ofstate-action pairs during training, and uses this knowledge to protect the RLagent from executing actions that yield likely hazardous outcomes. Ourcomprehensive experimental evaluation against state-of-the-art safe RLexploration techniques shows that ADVICE significantly reduces safetyviolations (approx 50%) during training, with a competitive outcome rewardcompared to other techniques.