Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels

  • 2020-05-21 17:40:38
  • Weinan E, Stephan Wojtowytsch
  • 1

Abstract

We establish a scale separation of Kolmogorov width type between subspaces ofa given Banach space under the condition that a sequence of linear mapsconverges much faster on one of the subspaces. The general technique is thenapplied to show that reproducing kernel Hilbert spaces are poor$L^2$-approximators for the class of two-layer neural networks in highdimension, and that two-layer networks with small path norm are poorapproximators for certain Lipschitz functions, also in the $L^2$-topology.

 

Quick Read (beta)

loading the full paper ...