Abstract
Deep reinforcement learning offers notable benefits in addressingcombinatorial problems over traditional solvers, reducing the reliance ondomain-specific knowledge and expert solutions, and improving computationalefficiency. Despite the recent surge in interest in neural combinatorialoptimization, practitioners often do not have access to a standardized codebase. Moreover, different algorithms are frequently based on fragmentizedimplementations that hinder reproducibility and fair comparison. To addressthese challenges, we introduce RL4CO, a unified Reinforcement Learning (RL) forCombinatorial Optimization (CO) library. We employ state-of-the-art softwareand best practices in implementation, such as modularity and configurationmanagement, to be flexible, easily modifiable, and extensible by researchers.Thanks to our unified codebase, we benchmark baseline RL solvers with differentevaluation schemes on zero-shot performance, generalization, and adaptabilityon diverse tasks. Notably, we find that some recent methods may fall behindtheir predecessors depending on the evaluation settings. We hope RL4CO willencourage the exploration of novel solutions to complex real-world tasks,allowing the community to compare with existing methods through a unifiedframework that decouples the science from software engineering. We open-sourceour library at https://github.com/ai4co/rl4co.