E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

  • 2025-06-09 18:59:01
  • Wenyan Cong, Yiqing Liang, Yancheng Zhang, Ziyi Yang, Yan Wang, Boris Ivanovic, Marco Pavone, Chen Chen, Zhangyang Wang, Zhiwen Fan
  • 0

Abstract

Spatial intelligence, encompassing 3D reconstruction, perception, andreasoning, is fundamental to applications such as robotics, aerial imaging, andextended reality. A key enabler is the real-time, accurate estimation of core3D attributes (camera parameters, point clouds, depth maps, and 3D pointtracks) from unstructured or streaming imagery. Inspired by the success oflarge foundation models in language and 2D vision, a new class of end-to-end 3Dgeometric foundation models (GFMs) has emerged, directly predicting dense 3Drepresentations in a single feed-forward pass, eliminating the need for slow orunavailable precomputed camera parameters. Since late 2023, the field hasexploded with diverse variants, but systematic evaluation is lacking. In thiswork, we present the first comprehensive benchmark for 3D GFMs, covering fivecore tasks: sparse-view depth estimation, video depth estimation, 3Dreconstruction, multi-view pose estimation, novel view synthesis, and spanningboth standard and challenging out-of-distribution datasets. Our standardizedtoolkit automates dataset handling, evaluation protocols, and metriccomputation to ensure fair, reproducible comparisons. We evaluate 16state-of-the-art GFMs, revealing their strengths and limitations across tasksand domains, and derive key insights to guide future model scaling andoptimization. All code, evaluation scripts, and processed data will be publiclyreleased to accelerate research in 3D spatial intelligence.

 

Quick Read (beta)

loading the full paper ...