Abstract
While it is widely known that neural networks are universal approximators ofcontinuous functions, a less known and perhaps more powerful result is that aneural network with a single hidden layer can approximate accurately anynonlinear continuous operator. This universal approximation theorem issuggestive of the potential application of neural networks in learningnonlinear operators from data. However, the theorem guarantees only a smallapproximation error for a sufficient large network, and does not consider theimportant optimization and generalization errors. To realize this theorem inpractice, we propose deep operator networks (DeepONets) to learn operatorsaccurately and efficiently from a relatively small dataset. A DeepONet consistsof two sub-networks, one for encoding the input function at a fixed number ofsensors $x_i, i=1,\dots,m$ (branch net), and another for encoding the locationsfor the output functions (trunk net). We perform systematic simulations foridentifying two types of operators, i.e., dynamic systems and partialdifferential equations, and demonstrate that DeepONet significantly reduces thegeneralization error compared to the fully-connected networks. We also derivetheoretically the dependence of the approximation error in terms of the numberof sensors (where the input function is defined) as well as the input functiontype, and we verify the theorem with computational results. More importantly,we observe high-order error convergence in our computational tests, namelypolynomial rates (from half order to fourth order) and even exponentialconvergence with respect to the training dataset size.