Abstract
Multi-agent reinforcement learning has emerged as a powerful framework forenabling agents to learn complex, coordinated behaviors but faces persistentchallenges regarding its generalization, scalability and sample efficiency.Recent advancements have sought to alleviate those issues by embeddingintrinsic symmetries of the systems in the policy. Yet, most dynamical systemsexhibit little to no symmetries to exploit. This paper presents a novelframework for embedding extrinsic symmetries in multi-agent system dynamicsthat enables the use of symmetry-enhanced methods to address systems withinsufficient intrinsic symmetries, expanding the scope of equivariant learningto a wide variety of MARL problems. Central to our framework is the GroupEquivariant Graphormer, a group-modular architecture specifically designed fordistributed swarming tasks. Extensive experiments on a swarm ofsymmetry-breaking quadrotors validate the effectiveness of our approach,showcasing its potential for improved generalization and zero-shot scalability.Our method achieves significant reductions in collision rates and enhances tasksuccess rates across a diverse range of scenarios and varying swarm sizes.