Abstract
Deep Reinforcement Learning (DRL) has emerged as a powerful solution formeeting the growing demands for connectivity, reliability, low latency andoperational efficiency in advanced networks. However, most research has focusedon theoretical analysis and simulations, with limited investigation intoreal-world deployment. To bridge the gap and support practical DRL deploymentfor network management, we first present an orchestration framework thatintegrates ETSI Multi-access Edge Computing (MEC) with Open RAN, enablingseamless adoption of DRL-based strategies across different time scales whileenhancing agent lifecycle management. We then identify three criticalchallenges hindering DRL's real-world deployment, including (1) asynchronousrequests from unpredictable or bursty traffic, (2) adaptability andgeneralization across heterogeneous topologies and evolving service demands,and (3) prolonged convergence and service interruptions due to exploration inlive operational environments. To address these challenges, we propose athree-fold solution strategy: (a) advanced time-series integration for handlingasynchronized traffic, (b) flexible architecture design such as multi-agent DRLand incremental learning to support heterogeneous scenarios, and (c)simulation-driven deployment with transfer learning to reduce convergence timeand service disruptions. Lastly, the feasibility of the MEC-O-RAN architectureis validated on an urban-wide testing infrastructure, and two real-world usecases are presented, showcasing the three identified challenges anddemonstrating the effectiveness of the proposed solutions.