Abstract

We demonstrate that a very deep ResNet with stacked modules with one neuronper hidden layer and ReLU activation functions can uniformly approximate anyLebesgue integrable function in $d$ dimensions, i.e. $\ell_1(\mathbb{R}^d)$.Because of the identity mapping inherent to ResNets, our network hasalternating layers of dimension one and $d$. This stands in sharp contrast tofully connected networks, which are not universal approximators if their widthis the input dimension $d$ [Lu et al, 2017]. Hence, our result implies anincrease in representational power for narrow deep networks by the ResNetarchitecture.

 

Quick Read (beta)