Abstract
Deep learning (DL), a new-generation artificial neural network research, hasmade profound strides in recent years. This review paper is intended to providehydrologists with a simple technical overview, trans-disciplinary progressupdate, and potentially inspirations about DL. Novel architectures, large andmore accessible data, and new computing power enabled the success of DL. Thereview reveals that DL is rapidly transforming myriad scientific disciplinesincluding high-energy physics, astronomy, chemistry, genomics and remotesensing, where systematic DL toolkits, innovative customizations, andsub-disciplines have emerged. However, with a few exceptions, its adoption inhydrology has so far been gradual. The literature suggests that novelregularization techniques can effectively prevent high-capacity deep networksfrom overfitting. As a result, in most scientific disciplines, DL modelsdemonstrated superior predictive and generalization performance to conventionalmethods. Meanwhile, less noticed is that DL may also serve as a scientificexploratory tool. A new area termed "AI neuroscience", has been born. Thisbudding sub-discipline is accumulating a significant body of work, e.g.,distilling knowledge obtained in DL networks to interpretable models,attributing decisions to inputs via back-propagation of relevance, orvisualization of activations. These methods are designed to interpret thedecision process of deep networks and derive insights. While scientists so farhave mostly been using customized, ad-hoc methods for interpretation, vastopportunities await for DL to propel science advancement, along with greatlyenhanced predictive capability.