Abstract
In this paper we presented a novel constructive approach for training deepneural networks using geometric approaches. We show that a topological coveringcan be used to define a class of distributed linear matrix inequalities, whichin turn directly specify the shape and depth of a neural network architecture.The key insight is a fundamental relationship between linear matrixinequalities and their ability to bound the shape of data, and the rectifiedlinear unit (ReLU) activation function employed in modern neural networks. Weshow that unit cover geometry and cover porosity are two design variables incover-constructive learning that play a critical role in defining thecomplexity of the model and generalizability of the resulting neural networkclassifier. In the context of cover-constructive learning, these findingsunderscore the age old trade-off between model complexity and overfitting (asquantified by the number of elements in the data cover) and generalizability ontest data. Finally, we benchmark on algorithm on the Iris, MNIST, and Winedataset and show that the constructive algorithm is able to train a deep neuralnetwork classifier in one shot, achieving equal or superior levels of trainingand test classification accuracy with reduced training time.