Abstract
Swimming microrobots are increasingly developed with complex materials anddynamic shapes and are expected to operate in complex environments in which thesystem dynamics are difficult to model and positional control of the microrobotis not straightforward to achieve. Deep reinforcement learning is a promisingmethod of autonomously developing robust controllers for creating smartmicrorobots, which can adapt their behavior to operate in uncharacterizedenvironments without the need to model the system dynamics. Here, we report thedevelopment of a smart helical magnetic hydrogel microrobot that used the softactor critic reinforcement learning algorithm to autonomously derive a controlpolicy which allowed the microrobot to swim through an uncharacterizedbiomimetic fluidic environment under control of a time varying magnetic fieldgenerated from a three-axis array of electromagnets. The reinforcement learningagent learned successful control policies with fewer than 100,000 trainingsteps, demonstrating sample efficiency for fast learning. We also demonstratethat we can fine tune the control policies learned by the reinforcementlearning agent by fitting mathematical functions to the learned policy's actiondistribution via regression. Deep reinforcement learning applied to microrobotcontrol is likely to significantly expand the capabilities of the nextgeneration of microrobots.