Abstract
In coupled learning rules for PCA (principal component analysis) and SVD(singular value decomposition), the update of the estimates of eigenvectors orsingular vectors is influenced by the estimates of eigenvalues or singularvalues, respectively. This coupled update mitigates the speed-stability problemsince the update equations converge from all directions with approximately thesame speed. A method to derive coupled learning rules from information criteriaby Newton optimization is known. However, these information criteria have to bedesigned, offer no explanatory value, and can only impose Euclidean constraintson the vector estimates. Here we describe an alternative approach where coupledPCA and SVD learning rules can systematically be derived from a Newtonzero-finding framework. The derivation starts from an objective function,combines the equations for its extrema with arbitrary constraints on the vectorestimates, and solves the resulting vector zero-point equation using Newton'szero-finding method. To demonstrate the framework, we derive PCA and SVDlearning rules with constant Euclidean length or constant sum of the vectorestimates.