Abstract
Smart manufacturing systems increasingly rely on adaptive control mechanismsto optimize complex processes. This research presents a novel approachintegrating Soft Actor-Critic (SAC) reinforcement learning with digital twintechnology to enable real-time process control in robotic additivemanufacturing. We demonstrate our methodology using a Viper X300s robot arm,implementing two distinct control scenarios: static target acquisition anddynamic trajectory following. The system architecture combines Unity'ssimulation environment with ROS2 for seamless digital twin synchronization,while leveraging transfer learning to efficiently adapt trained models acrosstasks. Our hierarchical reward structure addresses common reinforcementlearning challenges including local minima avoidance, convergence acceleration,and training stability. Experimental results show rapid policy convergence androbust task execution in both simulated and physical environments, withperformance metrics including cumulative reward, value prediction accuracy,policy loss, and discrete entropy coefficient demonstrating the effectivenessof our approach. This work advances the integration of reinforcement learningwith digital twins for industrial robotics applications, providing a frameworkfor enhanced adaptive real-time control for smart additive manufacturingprocess.