Abstract
Despite the unprecedented progress in the field of 3D generation, currentsystems still often fail to produce high-quality 3D assets that are visuallyappealing and geometrically and semantically consistent across multipleviewpoints. To effectively assess the quality of the generated 3D data, thereis a need for a reliable 3D evaluation tool. Unfortunately, existing 3Devaluation metrics often overlook the geometric quality of generated assets ormerely rely on black-box multimodal large language models for coarseassessment. In this paper, we introduce Eval3D, a fine-grained, interpretableevaluation tool that can faithfully evaluate the quality of generated 3D assetsbased on various distinct yet complementary criteria. Our key observation isthat many desired properties of 3D generation, such as semantic and geometricconsistency, can be effectively captured by measuring the consistency amongvarious foundation models and tools. We thus leverage a diverse set of modelsand tools as probes to evaluate the inconsistency of generated 3D assets acrossdifferent aspects. Compared to prior work, Eval3D provides pixel-wisemeasurement, enables accurate 3D spatial feedback, and aligns more closely withhuman judgments. We comprehensively evaluate existing 3D generation modelsusing Eval3D and highlight the limitations and challenges of current models.