Abstract
We argue that the success of reservoir computing lies within the separationcapacity of the reservoirs and show that the expected separation capacity ofrandom linear reservoirs is fully characterised by the spectral decompositionof an associated generalised matrix of moments. Of particular interest arereservoirs with Gaussian matrices that are either symmetric or whose entriesare all independent. In the symmetric case, we prove that the separationcapacity always deteriorates with time; while for short inputs, separation withlarge reservoirs is best achieved when the entries of the matrix are scaledwith a factor $\rho_T/\sqrt{N}$, where $N$ is the dimension of the reservoirand $\rho_T$ depends on the maximum length of the input time series. In thei.i.d. case, we establish that optimal separation with large reservoirs isconsistently achieved when the entries of the reservoir matrix are scaled withthe exact factor $1/\sqrt{N}$. We further give upper bounds on the quality ofseparation in function of the length of the time series. We complement thisanalysis with an investigation of the likelihood of this separation and theimpact of the chosen architecture on separation consistency.