Abstract
Visualizing surgical scenes is crucial for revealing internal anatomicalstructures during minimally invasive procedures. Novel View Synthesis is avital technique that offers geometry and appearance reconstruction, enhancingunderstanding, planning, and decision-making in surgical scenes. Despite theimpressive achievements of Neural Radiance Field (NeRF), its direct applicationto surgical scenes produces unsatisfying results due to two challenges:endoscopic sparse views and significant photometric inconsistencies. In thispaper, we propose uncertainty-aware conditional NeRF for novel view synthesisto tackle the severe shape-radiance ambiguity from sparse surgical views. Thecore of UC-NeRF is to incorporate the multi-view uncertainty estimation tocondition the neural radiance field for modeling the severe photometricinconsistencies adaptively. Specifically, our UC-NeRF first builds aconsistency learner in the form of multi-view stereo network, to establish thegeometric correspondence from sparse views and generate uncertainty estimationand feature priors. In neural rendering, we design a base-adaptive NeRF networkto exploit the uncertainty estimation for explicitly handling the photometricinconsistencies. Furthermore, an uncertainty-guided geometry distillation isemployed to enhance geometry learning. Experiments on the SCARED and Hamlyndatasets demonstrate our superior performance in rendering appearance andgeometry, consistently outperforming the current state-of-the-art approaches.Our code will be released at \url{https://github.com/wrld/UC-NeRF}.