Abstract
Research has shown that personalization of health interventions cancontribute to an improved effectiveness. Reinforcement learning algorithms canbe used to perform such tailoring using data that is collected about users.Learning is however very fragile for health interventions as only limited timeis available to learn from the user before disengagement takes place, or beforethe opportunity to intervene passes. In this paper, we present a cluster-basedreinforcement learning approach which learns across groups of users. Such anapproach can speed up the learning process while still giving a level ofpersonalization. The clustering algorithm uses a distance metric over traces ofstates and rewards. We apply both online and batch learning to learn policiesover the clusters and introduce a publicly available simulator which we havedeveloped to evaluate the approach. The results show batch learning clearlyoutperforms online learning. Furthermore, clustering can be beneficial providedthat a proper clustering is found.