Training Software Engineering Agents and Verifiers with SWE-Gym

  • 2025-06-06 08:53:20
  • Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang
  • 0

Abstract

We present SWE-Gym, the first environment for training real-world softwareengineering (SWE) agents. SWE-Gym contains 2,438 real-world Python taskinstances, each comprising a codebase with an executable runtime environment,unit tests, and a task specified in natural language. We use SWE-Gym to trainlanguage model based SWE agents, achieving up to 19% absolute gains in resolverate on the popular SWE-Bench Verified and Lite test sets. We also experimentwith inference-time scaling through verifiers trained on agent trajectoriessampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a newstate-of-the-art for open-weight SWE agents. To facilitate further research, wepublicly release SWE-Gym, models, and agent trajectories.

 

Quick Read (beta)

loading the full paper ...