Abstract
Realistic color texture generation is an important step in RGB-D surfacereconstruction, but remains challenging in practice due to inaccuracies inreconstructed geometry, misaligned camera poses, and view-dependent imagingartifacts. In this work, we present a novel approach for color texture generation usinga conditional adversarial loss obtained from weakly-supervised views. Specifically, we propose an approach to produce photorealistic textures forapproximate surfaces, even from misaligned images, by learning an objectivefunction that is robust to these errors. The key idea of our approach is to learn a patch-based conditionaldiscriminator which guides the texture optimization to be tolerant tomisalignments. Our discriminator takes a synthesized view and a real image, and evaluateswhether the synthesized one is realistic, under a broadened definition ofrealism. We train the discriminator by providing as `real' examples pairs of inputviews and their misaligned versions -- so that the learned adversarial losswill tolerate errors from the scans. Experiments on synthetic and real data under quantitative or qualitativeevaluation demonstrate the advantage of our approach in comparison to state ofthe art. Our code is publicly available with video demonstration.