It has recently been established that the numerical solution of ordinarydifferential equations can be posed as a nonlinear Bayesian inference problem,which can be approximately solved via Gaussian filtering and smoothing,whenever a Gauss--Markov prior is used. In this paper the class of $\nu$ timesdifferentiable linear time invariant Gauss--Markov priors is considered. Ataxonomy of Gaussian estimators is established, with the maximum a posterioriestimate at the top of the hierarchy, which can be computed with the iteratedextended Kalman smoother. The remaining three classes are termed explicit,semi-implicit, and implicit, which are in similarity with the classical notionscorresponding to conditions on the vector field, under which the filter updateproduces a local maximum a posteriori estimate. The maximum a posterioriestimate corresponds to an optimal interpolant in the reproducing Hilbert spaceassociated with the prior, which in the present case is equivalent to a Sobolevspace of smoothness $\nu+1$. Consequently, using methods from scattered dataapproximation and nonlinear analysis in Sobolev spaces, it is shown that themaximum a posteriori estimate converges to the true solution at a polynomialrate in the fill-distance (maximum step size) subject to mild conditions on thevector field. The methodology developed provides a novel and more naturalapproach to study the convergence of these estimators than classical methods ofconvergence analysis. The methods and theoretical results are demonstrated innumerical examples.