In the context of science, the well-known adage "a picture is worth athousand words" might well be "a model is worth a thousand datasets."Scientific models, such as Newtonian physics or biological gene regulatorynetworks, are human-driven simplifications of complex phenomena that serve assurrogates for the countless experiments that validated the models. Recently,machine learning has been able to overcome the inaccuracies of approximatemodeling by directly learning the entire set of nonlinear interactions fromdata. However, without any predetermined structure from the scientific basisbehind the problem, machine learning approaches are flexible butdata-expensive, requiring large databases of homogeneous labeled training data.A central challenge is reconciling data that is at odds with simplified modelswithout requiring "big data". In this work we develop a new methodology,universal differential equations (UDEs), which augments scientific models withmachine-learnable structures for scientifically-based learning. We show howUDEs can be utilized to discover previously unknown governing equations,accurately extrapolate beyond the original data, and accelerate modelsimulation, all in a time and data-efficient manner. This advance is coupledwith open-source software that allows for training UDEs which incorporatephysical constraints, delayed interactions, implicitly-defined events, andintrinsic stochasticity in the model. Our examples show how a diverse set ofcomputationally-difficult modeling issues across scientific disciplines, fromautomatically discovering biological mechanisms to accelerating climatesimulations by 15,000x, can be handled by training UDEs.