Abstract
A wide range of scientific problems, such as those described bycontinuous-time dynamical systems and partial differential equations (PDEs),are naturally formulated on function spaces. While function spaces aretypically infinite-dimensional, deep learning has predominantly advancedthrough applications in computer vision and natural language processing thatfocus on mappings between finite-dimensional spaces. Such fundamentaldisparities in the nature of the data have limited neural networks fromachieving a comparable level of success in scientific applications as seen inother fields. Neural operators are a principled way to generalize neuralnetworks to mappings between function spaces, offering a pathway to replicatedeep learning's transformative impact on scientific problems. For instance,neural operators can learn solution operators for entire classes of PDEs, e.g.,physical systems with different boundary conditions, coefficient functions, andgeometries. A key factor in deep learning's success has been the carefulengineering of neural architectures through extensive empirical testing.Translating these neural architectures into neural operators allows operatorlearning to enjoy these same empirical optimizations. However, prior neuraloperator architectures have often been introduced as standalone models, notdirectly derived as extensions of existing neural network architectures. Inthis paper, we identify and distill the key principles for constructingpractical implementations of mappings between infinite-dimensional functionspaces. Using these principles, we propose a recipe for converting severalpopular neural architectures into neural operators with minimal modifications.This paper aims to guide practitioners through this process and details thesteps to make neural operators work in practice. Our code can be found athttps://github.com/neuraloperator/NNs-to-NOs