Abstract
In this paper, we introduce a novel analysis of neural networks based ongeometric (Clifford) algebra and convex optimization. We show that optimalweights of deep ReLU neural networks are given by the wedge product of trainingsamples when trained with standard regularized loss. Furthermore, the trainingproblem reduces to convex optimization over wedge product features, whichencode the geometric structure of the training dataset. This structure is givenin terms of signed volumes of triangles and parallelotopes generated by datavectors. The convex problem finds a small subset of samples via $\ell_1$regularization to discover only relevant wedge product features. Our analysisprovides a novel perspective on the inner workings of deep neural networks andsheds light on the role of the hidden layers.