Real-Time Sign Language Detection using Human Pose Estimation

  • 2020-09-13 11:40:20
  • Amit Moryossef, Ioannis Tsochantaridis, Roee Aharoni, Sarah Ebling, Srini Narayanan
  • 0


We propose a lightweight real-time sign language detection model, as weidentify the need for such a case in videoconferencing. We extract optical flowfeatures based on human pose estimation and, using a linear classifier, showthese features are meaningful with an accuracy of 80%, evaluated on the DGSCorpus. Using a recurrent model directly on the input, we see improvements ofup to 91% accuracy, while still working under 4ms. We describe a demoapplication to sign language detection in the browser in order to demonstrateits usage possibility in videoconferencing applications.


Quick Read (beta)

loading the full paper ...