Abstract
We present a Deep Differentiable Simplex Layer (DDSL) for neural networks forgeometric deep learning. The DDSL is a differentiable layer compatible withdeep neural networks for bridging simplex mesh-based geometry representations(point clouds, line mesh, triangular mesh, tetrahedral mesh) with raster images(e.g., 2D/3D grids). The DDSL uses Non-Uniform Fourier Transform (NUFT) toperform differentiable, efficient, anti-aliased rasterization of simplex-basedsignals. We present a complete theoretical framework for the process as well asan efficient backpropagation algorithm. Compared to previous differentiablerenderers and rasterizers, the DDSL generalizes to arbitrary simplex degreesand dimensions. In particular, we explore its applications to 2D shapes andillustrate two applications of this method: (1) mesh editing and optimizationguided by neural network outputs, and (2) using DDSL for a differentiablerasterization loss to facilitate end-to-end training of polygon generators. Weare able to validate the effectiveness of gradient-based shape optimizationwith the example of airfoil optimization, and using the differentiablerasterization loss to facilitate end-to-end training, we surpass state of theart for polygonal image segmentation given ground-truth bounding boxes.