Abstract
Being able to learn an effective semantic representation directly on rawpoint clouds has become a central topic in 3D understanding. Despite rapidprogress, state-of-the-art encoders are restrictive to canonicalized pointclouds, and have weaker than necessary performance when encountering geometrictransformation distortions. To overcome this challenge, we propose PointTree, ageneral-purpose point cloud encoder that is robust to transformations based onrelaxed K-D trees. Key to our approach is the design of the division rule inK-D trees by using principal component analysis (PCA). We use the structure ofthe relaxed K-D tree as our computational graph, and model the features asborder descriptors which are merged with pointwise-maximum operation. Inaddition to this novel architecture design, we further improve the robustnessby introducing pre-alignment -- a simple yet effective PCA-based normalizationscheme. Our PointTree encoder combined with pre-alignment consistentlyoutperforms state-of-the-art methods by large margins, for applications fromobject classification to semantic segmentation on various transformed versionsof the widely-benchmarked datasets. Code and pre-trained models are availableat https://github.com/immortalCO/PointTree.