Abstract
In reinforcement learning, conducting task composition by forming cohesive,executable sequences from multiple tasks remains challenging. However, theability to (de)compose tasks is a linchpin in developing robotic systemscapable of learning complex behaviors. Yet, compositional reinforcementlearning is beset with difficulties, including the high dimensionality of theproblem space, scarcity of rewards, and absence of system robustness after taskcomposition. To surmount these challenges, we view task composition through theprism of category theory -- a mathematical discipline exploring structures andtheir compositional relationships. The categorical properties of Markovdecision processes untangle complex tasks into manageable sub-tasks, allowingfor strategical reduction of dimensionality, facilitating more tractable rewardstructures, and bolstering system robustness. Experimental results support thecategorical theory of reinforcement learning by enabling skill reduction,reuse, and recycling when learning complex robotic arm tasks.