Abstract
We propose a learning architecture that allows symbolic control and guidancein reinforcement learning with deep neural networks. We introduce SymDQN, anovel modular approach that augments the existing Dueling Deep Q-Networks(DuelDQN) architecture with modules based on the neuro-symbolic framework ofLogic Tensor Networks (LTNs). The modules guide action policy learning andallow reinforcement learning agents to display behaviour consistent withreasoning about the environment. Our experiment is an ablation study performedon the modules. It is conducted in a reinforcement learning environment of a5x5 grid navigated by an agent that encounters various shapes, each associatedwith a given reward. The underlying DuelDQN attempts to learn the optimalbehaviour of the agent in this environment, while the modules facilitate shaperecognition and reward prediction. We show that our architecture significantlyimproves learning, both in terms of performance and the precision of the agent.The modularity of SymDQN allows reflecting on the intricacies and complexitiesof combining neural and symbolic approaches in reinforcement learning.