Abstract
Neural Radiance Fields (NeRFs) have demonstrated the remarkable potential ofneural networks to capture the intricacies of 3D objects. By encoding the shapeand color information within neural network weights, NeRFs excel at producingstrikingly sharp novel views of 3D objects. Recently, numerous generalizationsof NeRFs utilizing generative models have emerged, expanding its versatility.In contrast, Gaussian Splatting (GS) offers a similar renders quality withfaster training and inference as it does not need neural networks to work. Weencode information about the 3D objects in the set of Gaussian distributionsthat can be rendered in 3D similarly to classical meshes. Unfortunately, GS aredifficult to condition since they usually require circa hundred thousandGaussian components. To mitigate the caveats of both models, we propose ahybrid model that uses GS representation of the 3D object's shape andNeRF-based encoding of color and opacity. Our model uses Gaussian distributionswith trainable positions (i.e. means of Gaussian), shape (i.e. covariance ofGaussian), color and opacity, and neural network, which takes parameters ofGaussian and viewing direction to produce changes in color and opacity.Consequently, our model better describes shadows, light reflections, andtransparency of 3D objects.