Weighted variation spaces and approximation by shallow ReLU networks

  • 2024-10-13 16:36:08
  • Ronald DeVore, Robert D. Nowak, Rahul Parhi, Jonathan W. Siegel
  • 0

Abstract

We investigate the approximation of functions $f$ on a bounded domain$\Omega\subset \mathbb{R}^d$ by the outputs of single-hidden-layer ReLU neuralnetworks of width $n$. This form of nonlinear $n$-term dictionary approximationhas been intensely studied since it is the simplest case of neural networkapproximation (NNA). There are several celebrated approximation results forthis form of NNA that introduce novel model classes of functions on $\Omega$whose approximation rates do not grow unbounded with the input dimension. Thesenovel classes include Barron classes, and classes based on sparsity orvariation such as the Radon-domain BV classes. The present paper is concernedwith the definition of these novel model classes on domains $\Omega$. Thecurrent definition of these model classes does not depend on the domain$\Omega$. A new and more proper definition of model classes on domains is givenby introducing the concept of weighted variation spaces. These new modelclasses are intrinsic to the domain itself. The importance of these new modelclasses is that they are strictly larger than the classical(domain-independent) classes. Yet, it is shown that they maintain the same NNArates.

 

Quick Read (beta)

loading the full paper ...