Abstract
Reinforcement learning (RL) is a subset of artificial intelligence (AI) whereagents learn the best action by interacting with the environment, making itsuitable for tasks that do not require labeled data or direct supervision.Hyperparameters (HP) tuning refers to choosing the best parameter that leads tooptimal solutions in RL algorithms. Manual or random tuning of the HP may be acrucial process because variations in this parameter lead to changes in theoverall learning aspects and different rewards. In this paper, a novel andautomatic HP-tuning method called Q-FOX is proposed. This uses both the FOXoptimizer, a new optimization method inspired by nature that mimics red foxes'hunting behavior, and the commonly used, easy-to-implement RL Q-learningalgorithm to solve the problem of HP tuning. Moreover, a new objective functionis proposed which prioritizes the reward over the mean squared error (MSE) andlearning time (steps). Q-FOX has been evaluated on two OpenAI Gym environmentcontrol tasks: Cart Pole and Frozen Lake. It exposed greater cumulative rewardsthan HP tuning with other optimizers, such as PSO, GA, Bee, or randomlyselected HP. The cumulative reward for the Cart Pole task was 32.08, and forthe Frozen Lake task was 0.95. Despite the robustness of Q-FOX, it haslimitations. It cannot be used directly in real-word problems before choosingthe HP in a simulation environment because its processes work iteratively,making it time-consuming. The results indicate that Q-FOX has played anessential role in HP tuning for RL algorithms to effectively solve differentcontrol tasks.