Abstract
Multi-objective reinforcement learning (MORL) extends traditional RL byseeking policies making different compromises among conflicting objectives. Therecent surge of interest in MORL has led to diverse studies and solvingmethods, often drawing from existing knowledge in multi-objective optimizationbased on decomposition (MOO/D). Yet, a clear categorization based on both RLand MOO/D is lacking in the existing literature. Consequently, MORL researchersface difficulties when trying to classify contributions within a broadercontext due to the absence of a standardized taxonomy. To tackle such an issue,this paper introduces multi-objective reinforcement learning based ondecomposition (MORL/D), a novel methodology bridging the literature of RL andMOO. A comprehensive taxonomy for MORL/D is presented, providing a structuredfoundation for categorizing existing and potential MORL works. The introducedtaxonomy is then used to scrutinize MORL research, enhancing clarity andconciseness through well-defined categorization. Moreover, a flexible frameworkderived from the taxonomy is introduced. This framework accommodates diverseinstantiations using tools from both RL and MOO/D. Its versatility isdemonstrated by implementing it in different configurations and assessing it oncontrasting benchmark problems. Results indicate MORL/D instantiations achievecomparable performance to current state-of-the-art approaches on the studiedproblems. By presenting the taxonomy and framework, this paper offers acomprehensive perspective and a unified vocabulary for MORL. This not onlyfacilitates the identification of algorithmic contributions but also lays thegroundwork for novel research avenues in MORL.