Abstract
Reinforcement learning (RL) has shown promise in solving variouscombinatorial optimization problems. However, conventional RL faces challengeswhen dealing with real-world constraints, especially when action spacefeasibility is explicit and dependent on the corresponding state or trajectory.In this work, we focus on using RL in container shipping, often considered thecornerstone of global trade, by dealing with the critical challenge of masterstowage planning. The main objective is to maximize cargo revenue and minimizeoperational costs while navigating demand uncertainty and various complexoperational constraints, namely vessel capacity and stability, which must bedynamically updated along the vessel's voyage. To address this problem, weimplement a deep reinforcement learning framework with feasibility projectionto solve the master stowage planning problem (MPP) under demand uncertainty.The experimental results show that our architecture efficiently finds adaptive,feasible solutions for this multi-stage stochastic optimization problem,outperforming traditional mixed-integer programming and RL with feasibilityregularization. Our AI-driven decision-support policy enables adaptive andfeasible planning under uncertainty, optimizing operational efficiency andcapacity utilization while contributing to sustainable and resilient globalsupply chains.