Abstract
StarCraft Multi-Agent Challenge (SMAC) has been one of the most commonly usedexperimental environments in multi-agent reinforcement learning (MARL), wherethe specific task is to control a set number of allied units to defeat enemyforces. Traditional MARL algorithms often require interacting with theenvironment for millions of steps to train a parametric model, of which theresulting policies are typically non-interpretable with weak transferability.In this paper, we introduce SMAC-R1 which is based on the Qwen2.5-7B-Base LLMdistilled from DeepSeek-Coder-v2.5-236B. Similar to online reinforcementlearning after behavior cloning in offline learning process, in our pipeline,agents leverage the DeepSeek LLM to generate decision tree code by providingtask descriptions, and the agents are further self-reflected using feedbackfrom the rewards provided by the environment. Based on that, we augment thegenerated scripts to fine-tune a small LLM, Qwen2.5-7B-Base, to distill thedecision-making ability via Supervised Fine-Tuning (SFT) and enhance the scriptgeneration ability by the Group Relative Policy Optimization (GRPO) algorithm.We conduct experiments in the original 23 SMAC tasks and 10 newly-designedtasks to demonstrate that our method can produce high-quality, interpretabledecision trees with minimal environmental exploration. Moreover, these scriptsexhibit strong transferability, successfully applying to homogeneous SMACenvironments without modification. We believe this approach offers a newdirection for solving decision-making tasks and domain-specific LLM trainingpipelines in the future.