Abstract

Neural networks allow Q-learning reinforcement learning agents such as deepQ-networks (DQN) to approximate complex mappings from state spaces to valuefunctions. However, this also brings drawbacks when compared to other functionapproximators such as tile coding or their generalisations, radial basisfunctions (RBF) because they introduce instability due to the side effect ofglobalised updates present in neural networks. This instability does not evenvanish in neural networks that do not have any hidden layers. In this paper, weshow that simple modifications to the structure of the neural network canimprove stability of DQN learning when a multi-layer perceptron is used forfunction approximation.

 

Quick Read (beta)