KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

  • 2021-04-22 17:59:08
  • Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa
  • 30


We introduce KeypointDeformer, a novel unsupervised method for shape controlthrough automatically discovered 3D keypoints. We cast this as the problem ofaligning a source 3D object to a target 3D object from the same objectcategory. Our method analyzes the difference between the shapes of the twoobjects by comparing their latent representations. This latent representationis in the form of 3D keypoints that are learned in an unsupervised way. Thedifference between the 3D keypoints of the source and the target objects theninforms the shape deformation algorithm that deforms the source object into thetarget object. The whole model is learned end-to-end and simultaneouslydiscovers 3D keypoints while learning to use them for deforming object shapes.Our approach produces intuitive and semantically consistent control of shapedeformations. Moreover, our discovered 3D keypoints are consistent acrossobject category instances despite large shape variations. As our method isunsupervised, it can be readily deployed to new object categories withoutrequiring annotations for 3D keypoints and deformations.


Quick Read (beta)

loading the full paper ...