Abstract
Rendering dynamic 3D human from monocular videos is crucial for variousapplications such as virtual reality and digital entertainment. Most methodsassume the people is in an unobstructed scene, while various objects may causethe occlusion of body parts in real-life scenarios. Previous method utilizingNeRF for surface rendering to recover the occluded areas, but it requiring morethan one day to train and several seconds to render, failing to meet therequirements of real-time interactive applications. To address these issues, wepropose OccGaussian based on 3D Gaussian Splatting, which can be trained within6 minutes and produces high-quality human renderings up to 160 FPS withoccluded input. OccGaussian initializes 3D Gaussian distributions in thecanonical space, and we perform occlusion feature query at occluded regions,the aggregated pixel-align feature is extracted to compensate for the missinginformation. Then we use Gaussian Feature MLP to further process the featurealong with the occlusion-aware loss functions to better perceive the occludedarea. Extensive experiments both in simulated and real-world occlusions,demonstrate that our method achieves comparable or even superior performancecompared to the state-of-the-art method. And we improving training andinference speeds by 250x and 800x, respectively. Our code will be available forresearch purposes.