Abstract
Reinforcement learning (RL) using world models has found significant recentsuccesses. However, when a sudden change to world mechanics or propertiesoccurs then agent performance and reliability can dramatically decline. Werefer to the sudden change in visual properties or state transitions asnovelties. Implementing novelty detection within generated world modelframeworks is a crucial task for protecting the agent when deployed. In thispaper, we propose straightforward bounding approaches to incorporate noveltydetection into world model RL agents, by utilizing the misalignment of theworld model's hallucinated states and the true observed states as an anomalyscore. We provide effective approaches to detecting novelties in a distributionof transitions learned by an agent in a world model. Finally, we show theadvantage of our work in a novel environment compared to traditional machinelearning novelty detection methods as well as currently accepted RL focusednovelty detection algorithms.