Abstract

We present a simple, real-time approach for pupil tracking from live video onmobile devices. Our method extends a state-of-the-art face mesh detector withtwo new components: a tiny neural network that predicts positions of the pupilsin 2D, and a displacement-based estimation of the pupil blend shapecoefficients. Our technique can be used to accurately control the pupilmovements of a virtual puppet, and lends liveliness and energy to it. Theproposed approach runs at over 50 FPS on modern phones, and enables its usagein any real-time puppeteering pipeline.

 

Quick Read (beta)