Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning

  • 2021-11-26 21:24:53
  • Isaac J. Sledge, Darshan W. Bryner, Jose C. Principe
  • 0


Reinforcement learning in large-scale environments is challenging due to themany possible actions that can be taken in specific situations. We havepreviously developed a means of constraining, and hence speeding up, the searchprocess through the use of motion primitives; motion primitives are sequencesof pre-specified actions taken across a state series. As a byproduct of thiswork, we have found that if the motion primitives' motions and actions arelabeled, then the search can be sped up further. Since motion primitives mayinitially lack such details, we propose a theoretically viewpoint-insensitiveand speed-insensitive means of automatically annotating the underlying motionsand actions. We do this through a differential-geometric, spatio-temporalkinematics descriptor, which analyzes how the poses of entities in two motionsequences change over time. We use this descriptor in conjunction with aweighted-nearest-neighbor classifier to label the primitives using a limitedset of training examples. In our experiments, we achieve high motion and action annotation rates forhuman-action-derived primitives with as few as one training sample. We alsodemonstrate that reinforcement learning using accurately labeled trajectoriesleads to high-performing policies more quickly than standard reinforcementlearning techniques. This is partly because motion primitives encode priordomain knowledge and preempt the need to re-discover that knowledge duringtraining. It is also because agents can leverage the labels to systematicallyignore action classes that do not facilitate task objectives, thereby reducingthe action space.


Quick Read (beta)

loading the full paper ...