Abstract
In multi-agent reinforcement learning (MARL), achieving multi-taskgeneralization to diverse agents and objectives presents significantchallenges. Existing online MARL algorithms primarily focus on single-taskperformance, but their lack of multi-task generalization capabilities typicallyresults in substantial computational waste and limited real-life applicability.Meanwhile, existing offline multi-task MARL approaches are heavily dependent ondata quality, often resulting in poor performance on unseen tasks. In thispaper, we introduce HyGen, a novel hybrid MARL framework, Hybrid Training forEnhanced Multi-Task Generalization, which integrates online and offlinelearning to ensure both multi-task generalization and training efficiency.Specifically, our framework extracts potential general skills from offlinemulti-task datasets. We then train policies to select the optimal skills underthe centralized training and decentralized execution paradigm (CTDE). Duringthis stage, we utilize a replay buffer that integrates both offline data andonline interactions. We empirically demonstrate that our framework effectivelyextracts and refines general skills, yielding impressive generalization tounseen tasks. Comparative analyses on the StarCraft multi-agent challenge showthat HyGen outperforms a wide range of existing solely online and offlinemethods.