3D Shape Tokenization

  • 2024-12-20 07:22:41
  • Jen-Hao Rick Chang, Yuyang Wang, Miguel Angel Bautista Martin, Jiatao Gu, Josh Susskind, Oncel Tuzel
  • 0

Abstract

We introduce Shape Tokens, a 3D representation that is continuous, compact,and easy to incorporate into machine learning models. Shape Tokens act asconditioning vectors that represent shape information in a 3D flow-matchingmodel. The flow-matching model is trained to approximate probability densityfunctions corresponding to delta functions concentrated on the surfaces ofshapes in 3D. By attaching Shape Tokens to various machine learning models, wecan generate new shapes, convert images to 3D, align 3D shapes with text andimages, and render shapes directly at variable, user specified, resolution.Moreover, Shape Tokens enable a systematic analysis of geometric propertiessuch as normal, density, and deformation field. Across all tasks andexperiments, utilizing Shape Tokens demonstrate strong performance compared toexisting baselines.

 

Quick Read (beta)

loading the full paper ...