Abstract
Motor control is a set of time-varying muscle excitations which generatedesired motions for a biomechanical system. Muscle excitations cannot bedirectly measured from live subjects. An alternative approach is to estimatemuscle activations using inverse motion-driven simulation. In this article, wepropose a deep reinforcement learning method to estimate the muscle excitationsin simulated biomechanical systems. Here, we introduce a custom-made rewardfunction which incentivizes faster point-to-point tracking of target motion.Moreover, we deploy two new techniques, namely, episode-based hard update anddual buffer experience replay, to avoid feedback training loops. The proposedmethod is tested in four simulated 2D and 3D environments with 6 to 24 axialmuscles. The results show that the models were able to learn muscle excitationsfor given motions after nearly 100,000 simulated steps. Moreover, the root meansquare error in point-to-point reaching of the target across experiments wasless than 1% of the length of the domain of motion. Our reinforcement learningmethod is far from the conventional dynamic approaches as the muscle control isderived functionally by a set of distributed neurons. This can open paths forneural activity interpretation of this phenomenon.