Digital Twin-Enabled Real-Time Control in Robotic Additive Manufacturing via Soft Actor-Critic Reinforcement Learning

  • 2025-01-29 22:06:53
  • Matsive Ali, Sandesh Giri, Sen Liu, Qin Yang
  • 0

Abstract

Smart manufacturing systems increasingly rely on adaptive control mechanismsto optimize complex processes. This research presents a novel approachintegrating Soft Actor-Critic (SAC) reinforcement learning with digital twintechnology to enable real-time process control in robotic additivemanufacturing. We demonstrate our methodology using a Viper X300s robot arm,implementing two distinct control scenarios: static target acquisition anddynamic trajectory following. The system architecture combines Unity'ssimulation environment with ROS2 for seamless digital twin synchronization,while leveraging transfer learning to efficiently adapt trained models acrosstasks. Our hierarchical reward structure addresses common reinforcementlearning challenges including local minima avoidance, convergence acceleration,and training stability. Experimental results show rapid policy convergence androbust task execution in both simulated and physical environments, withperformance metrics including cumulative reward, value prediction accuracy,policy loss, and discrete entropy coefficient demonstrating the effectivenessof our approach. This work advances the integration of reinforcement learningwith digital twins for industrial robotics applications, providing a frameworkfor enhanced adaptive real-time control for smart additive manufacturingprocess.

 

Quick Read (beta)

loading the full paper ...