Abstract
Capturing geometric and material information from images remains afundamental challenge in computer vision and graphics. Traditionaloptimization-based methods often require hours of computational time toreconstruct geometry, material properties, and environmental lighting fromdense multi-view inputs, while still struggling with inherent ambiguitiesbetween lighting and material. On the other hand, learning-based approachesleverage rich material priors from existing 3D object datasets but facechallenges with maintaining multi-view consistency. In this paper, we introduceIDArb, a diffusion-based model designed to perform intrinsic decomposition onan arbitrary number of images under varying illuminations. Our method achievesaccurate and multi-view consistent estimation on surface normals and materialproperties. This is made possible through a novel cross-view, cross-domainattention module and an illumination-augmented, view-adaptive trainingstrategy. Additionally, we introduce ARB-Objaverse, a new dataset that provideslarge-scale multi-view intrinsic data and renderings under diverse lightingconditions, supporting robust training. Extensive experiments demonstrate thatIDArb outperforms state-of-the-art methods both qualitatively andquantitatively. Moreover, our approach facilitates a range of downstream tasks,including single-image relighting, photometric stereo, and 3D reconstruction,highlighting its broad applications in realistic 3D content creation.