Abstract
We introduce Hyperparameter Controller (HyperController), a computationallyefficient algorithm for hyperparameter optimization during training ofreinforcement learning neural networks. HyperController optimizeshyperparameters quickly while also maintaining improvement of the reinforcementlearning neural network, resulting in faster training and deployment. Itachieves this by modeling the hyperparameter optimization problem as an unknownLinear Gaussian Dynamical System, which is a system with a state that linearlychanges. It then learns an efficient representation of the hyperparameterobjective function using the Kalman filter, which is the optimal one-steppredictor for a Linear Gaussian Dynamical System. To demonstrate theperformance of HyperController, it is applied as a hyperparameter optimizerduring training of reinforcement learning neural networks on a variety ofOpenAI Gymnasium environments. In four out of the five Gymnasium environments,HyperController achieves highest median reward during evaluation compared toother algorithms. The results exhibit the potential of HyperController forefficient and stable training of reinforcement learning neural networks.