Real-time Pupil Tracking from Monocular Video for Digital Puppetry

  • 2020-06-19 19:39:32
  • Artsiom Ablavatski, Andrey Vakunov, Ivan Grishchenko, Karthik Raveendran, Matsvei Zhdanovich
  • 20

Abstract

We present a simple, real-time approach for pupil tracking from live video onmobile devices. Our method extends a state-of-the-art face mesh detector withtwo new components: a tiny neural network that predicts positions of the pupilsin 2D, and a displacement-based estimation of the pupil blend shapecoefficients. Our technique can be used to accurately control the pupilmovements of a virtual puppet, and lends liveliness and energy to it. Theproposed approach runs at over 50 FPS on modern phones, and enables its usagein any real-time puppeteering pipeline.

 

Quick Read (beta)

loading the full paper ...