InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models

  • 2024-04-10 18:48:37
  • Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, Ying Shan
  • 0

Abstract

We present InstantMesh, a feed-forward framework for instant 3D meshgeneration from a single image, featuring state-of-the-art generation qualityand significant training scalability. By synergizing the strengths of anoff-the-shelf multiview diffusion model and a sparse-view reconstruction modelbased on the LRM architecture, InstantMesh is able to create diverse 3D assetswithin 10 seconds. To enhance the training efficiency and exploit moregeometric supervisions, e.g, depths and normals, we integrate a differentiableiso-surface extraction module into our framework and directly optimize on themesh representation. Experimental results on public datasets demonstrate thatInstantMesh significantly outperforms other latest image-to-3D baselines, bothqualitatively and quantitatively. We release all the code, weights, and demo ofInstantMesh, with the intention that it can make substantial contributions tothe community of 3D generative AI and empower both researchers and contentcreators.

 

Quick Read (beta)

loading the full paper ...