Abstract

In the recent literature the important role of depth in deep learning hasbeen emphasized. In this paper we argue that sufficient width of a feedforwardnetwork is equally important by answering the simple question under whichconditions the decision regions of a neural network are connected. It turns outthat for a class of activation functions including leaky ReLU, neural networkshaving a pyramidal structure, that is no layer has more hidden units than theinput dimension, produce necessarily connected decision regions. This impliesthat a sufficiently wide layer is necessary to produce disconnected decisionregions. We discuss the implications of this result for the construction ofneural networks, in particular the relation to the problem of adversarialmanipulation of classifiers.

 

Introduction (beta)

None

 

Conclusion (beta)

None