Multiface: A Dataset for Neural Face Rendering

  • 2022-07-22 18:55:39
  • Cheng-hsin Wuu, Ningyuan Zheng, Scott Ardisson, Rohan Bali, Danielle Belko, Eric Brockmeyer, Lucas Evans, Timothy Godisart, Hyowon Ha, Alexander Hypes, Taylor Koska, Steven Krenn, Stephen Lombardi, Xiaomin Luo, Kevyn McPhail, Laura Millerschoen, Michal Perdoch, Mark Pitts, Alexander Richard, Jason Saragih, Junko Saragih, Takaaki Shiratori, Tomas Simon, Matt Stewart, Autumn Trimble, Xinshuo Weng, David Whitewolf, Chenglei Wu, Shoou-I Yu, Yaser Sheikh
  • 79

Abstract

Photorealistic avatars of human faces have come a long way in recent years,yet research along this area is limited by a lack of publicly available,high-quality datasets covering both, dense multi-view camera captures, and richfacial expressions of the captured subjects. In this work, we presentMultiface, a new multi-view, high-resolution human face dataset collected from13 identities at Reality Labs Research for neural face rendering. We introduceMugsy, a large scale multi-camera apparatus to capture high-resolutionsynchronized videos of a facial performance. The goal of Multiface is to closethe gap in accessibility to high quality data in the academic community and toenable research in VR telepresence. Along with the release of the dataset, weconduct ablation studies on the influence of different model architecturestoward the model's interpolation capacity of novel viewpoint and expressions.With a conditional VAE model serving as our baseline, we found that addingspatial bias, texture warp field, and residual connections improves performanceon novel view synthesis. Our code and data is available at:https://github.com/facebookresearch/multiface

 

Quick Read (beta)

loading the full paper ...