Abstract
The problem of function approximation by neural dynamical systems hastypically been approached in a top-down manner: Any continuous function can beapproximated to an arbitrary accuracy by a sufficiently complex model with agiven architecture. This can lead to high-complexity controls which areimpractical in applications. In this paper, we take the opposite, constructiveapproach: We impose various structural restrictions on system dynamics andconsequently characterize the class of functions that can be realized by such asystem. The systems are implemented as a cascade interconnection of a neuralstochastic differential equation (Neural SDE), a deterministic dynamicalsystem, and a readout map. Both probabilistic and geometric (Lie-theoretic)methods are used to characterize the classes of functions realized by suchsystems.