Abstract
Tabular reinforcement learning methods cannot operate directly on continuousstate spaces. One solution for this problem is to partition the state space. Agood partitioning enables generalization during learning and more efficientexploitation of prior experiences. Consequently, the learning process becomesfaster and produces more reliable policies. However, partitioning introducesapproximation, which is particularly harmful in the presence of nonlinearrelations between state components. An ideal partition should be as coarse aspossible, while capturing the key structure of the state space for the givenproblem. This work extracts partitions from the environment dynamics bysymbolic execution. We show that symbolic partitioning improves state spacecoverage with respect to environmental behavior and allows reinforcementlearning to perform better for sparse rewards. We evaluate symbolic state spacepartitioning with respect to precision, scalability, learning agent performanceand state space coverage for the learnt policies.