Abstract
We propose G-HOP, a denoising diffusion based generative prior forhand-object interactions that allows modeling both the 3D object and a humanhand, conditioned on the object category. To learn a 3D spatial diffusion modelthat can capture this joint distribution, we represent the human hand via askeletal distance field to obtain a representation aligned with the (latent)signed distance field for the object. We show that this hand-object prior canthen serve as generic guidance to facilitate other tasks like reconstructionfrom interaction clip and human grasp synthesis. We believe that our model,trained by aggregating seven diverse real-world interaction datasets spanningacross 155 categories, represents a first approach that allows jointlygenerating both hand and object. Our empirical evaluations demonstrate thebenefit of this joint prior in video-based reconstruction and human graspsynthesis, outperforming current task-specific baselines. Project website: https://judyye.github.io/ghop-www