Abstract
Implicit neural rendering, especially Neural Radiance Field (NeRF), has showngreat potential in novel view synthesis of a scene. However, current NeRF-basedmethods cannot enable users to perform user-controlled shape deformation in thescene. While existing works have proposed some approaches to modify theradiance field according to the user's constraints, the modification is limitedto color editing or object translation and rotation. In this paper, we proposea method that allows users to perform controllable shape deformation on theimplicit representation of the scene, and synthesizes the novel view images ofthe edited scene without re-training the network. Specifically, we establish acorrespondence between the extracted explicit mesh representation and theimplicit neural representation of the target scene. Users can first utilizewell-developed mesh-based deformation methods to deform the mesh representationof the scene. Our method then utilizes user edits from the mesh representationto bend the camera rays by introducing a tetrahedra mesh as a proxy, obtainingthe rendering results of the edited scene. Extensive experiments demonstratethat our framework can achieve ideal editing results not only on syntheticdata, but also on real scenes captured by users.