H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

  • 2021-07-26 23:04:18
  • Eduard Ramon, Gil Triginer, Janna Escur, Albert Pumarola, Jaime Garcia, Xavier Giro-i-Nieto, Francesc Moreno-Noguer
  • 96

Abstract

Recent learning approaches that implicitly represent surface geometry usingcoordinate-based neural representations have shown impressive results in theproblem of multi-view 3D reconstruction. The effectiveness of these techniquesis, however, subject to the availability of a large number (several tens) ofinput views of the scene, and computationally demanding optimizations. In thispaper, we tackle these limitations for the specific problem of few-shot full 3Dhead reconstruction, by endowing coordinate-based representations with aprobabilistic shape prior that enables faster convergence and bettergeneralization when using few input images (down to three). First, we learn ashape model of 3D heads from thousands of incomplete raw scans using implicitrepresentations. At test time, we jointly overfit two coordinate-based neuralnetworks to the scene, one modeling the geometry and another estimating thesurface radiance, using implicit differentiable rendering. We devise atwo-stage optimization strategy in which the learned prior is used toinitialize and constrain the geometry during an initial optimization phase.Then, the prior is unfrozen and fine-tuned to the scene. By doing this, weachieve high-fidelity head reconstructions, including hair and shoulders, andwith a high level of detail that consistently outperforms both state-of-the-art3D Morphable Models methods in the few-shot scenario, and non-parametricmethods when large sets of views are available.

 

Quick Read (beta)

loading the full paper ...