Abstract
The performance of Deep Q-Networks (DQN) is critically dependent on theability of its underlying neural network to accurately approximate theaction-value function. Standard function approximators, such as multi-layerperceptrons, may struggle to efficiently represent the complex value landscapesinherent in many reinforcement learning problems. This paper introduces a novelarchitecture, the Chebyshev-DQN (Ch-DQN), which integrates a Chebyshevpolynomial basis into the DQN framework to create a more effective featurerepresentation. By leveraging the powerful function approximation properties ofChebyshev polynomials, we hypothesize that the Ch-DQN can learn moreefficiently and achieve higher performance. We evaluate our proposed model onthe CartPole-v1 benchmark and compare it against a standard DQN with acomparable number of parameters. Our results demonstrate that the Ch-DQN with amoderate polynomial degree (N=4) achieves significantly better asymptoticperformance, outperforming the baseline by approximately 39\%. However, we alsofind that the choice of polynomial degree is a critical hyperparameter, as ahigh degree (N=8) can be detrimental to learning. This work validates thepotential of using orthogonal polynomial bases in deep reinforcement learningwhile also highlighting the trade-offs involved in model complexity.