Abstract
This paper proposes an approach, Spectral Dynamics Embedding Control (SDEC),to optimal control for nonlinear stochastic systems. This method reveals aninfinite-dimensional feature representation induced by the system's nonlinearstochastic dynamics, enabling a linear representation of the state-action valuefunction. For practical implementation, this representation is approximatedusing finite-dimensional trucations, specifically via two prominent kernelapproximation methods: random feature truncation and Nystrom approximation. Tocharacterize the effectiveness of these approximations, we provide an in-depththeoretical analysis to characterize the approximation error arising from thefinite-dimension truncation and statistical error due to finite-sampleapproximation in both policy evaluation and policy optimization. Empirically,our algorithm performs favorably against existing stochastic control algorithmson several benchmark problems.