Abstract
Let F:R^n -> R be a feedforward ReLU neural network. It is well-known thatfor any choice of parameters, F is continuous and piecewise (affine) linear. Welay some foundations for a systematic investigation of how the architecture ofF impacts the geometry and topology of its possible decision regions for binaryclassification tasks. Following the classical progression for smooth functionsin differential topology, we first define the notion of a generic, transversalReLU neural network and show that almost all ReLU networks are generic andtransversal. We then define a partially-oriented linear 1-complex in the domainof F and identify properties of this complex that yield an obstruction to theexistence of bounded connected components of a decision region. We use thisobstruction to prove that a decision region of a generic, transversal ReLUnetwork F: R^n -> R with a single hidden layer of dimension (n + 1) can have nomore than one bounded connected component.