Abstract
Reinforcement learning (RL) has seen significant success across variousdomains, but its adoption is often limited by the black-box nature of neuralnetwork policies, making them difficult to interpret. In contrast, symbolicpolicies allow representing decision-making strategies in a compact andinterpretable way. However, learning symbolic policies directly withinon-policy methods remains challenging. In this paper, we introduce SYMPOL, anovel method for SYMbolic tree-based on-POLicy RL. SYMPOL employs a tree-basedmodel integrated with a policy gradient method, enabling the agent to learn andadapt its actions while maintaining a high level of interpretability. Weevaluate SYMPOL on a set of benchmark RL tasks, demonstrating its superiorityover alternative tree-based RL approaches in terms of performance andinterpretability. Unlike existing methods, it enables gradient-based,end-to-end learning of interpretable, axis-aligned decision trees withinstandard on-policy RL algorithms. Therefore, SYMPOL can become the foundationfor a new class of interpretable RL based on decision trees. Our implementationis available under: https://github.com/s-marton/sympol