Abstract
Multi-agent reinforcement learning (MARL) systems usually requiresignificantly long training times due to their inherent complexity.Furthermore, deploying them in the real world demands a feature-richenvironment along with multiple embodied agents, which may not be feasible dueto budget or space limitations, not to mention energy consumption and safetyissues. This work tries to address these pain points by presenting asustainable digital twin framework capable of accelerating MARL training byselectively scaling parallelized workloads on-demand, and transferring thetrained policies from simulation to reality using minimal hardware resources.The applicability of the proposed digital twin framework is highlighted throughtwo representative use cases, which cover cooperative as well as competitiveclasses of MARL problems. We study the effect of agent and environmentparallelization on training time and that of systematic domain randomization onzero-shot sim2real transfer across both the case studies. Results indicate upto 76.3% reduction in training time with the proposed parallelization schemeand as low as 2.9% sim2real gap using the suggested deployment method.