Abstract
In this work, we study value function approximation in reinforcement learning(RL) problems with high dimensional state or action spaces via a generalizedversion of representation policy iteration (RPI). We consider the limitationsof proto-value functions (PVFs) at accurately approximating the value functionin low dimensions and we highlight the importance of features learning for animproved low-dimensional value function approximation. Then, we adopt differentrepresentation learning algorithm on graphs to learn the basis functions thatbest represent the value function. We empirically show that node2vec, analgorithm for scalable feature learning in networks, and the Variational GraphAuto-Encoder constantly outperform the commonly used smooth proto-valuefunctions in low-dimensionl feature space.