Abstract
Learning reduced component representations of data using nonlineartransformations is a central problem in unsupervised learning with a richhistory. Recently, a new family of algorithms based on maximum likelihoodoptimization with change of variables has demonstrated an impressive ability tomodel complex nonlinear data distributions. These algorithms learn to map fromarbitrary random variables to independent components using invertible nonlinearfunction approximators. Despite the potential of this framework, the underlyingoptimization objective is ill-posed for a large class of variables, inhibitingaccurate component estimates in many use cases. We present a new Tikhonovregularization technique for nonlinear independent component estimation thatmediates the instability of the algorithm and facilitates robust componentestimates. In addition, we provide a theoretically grounded procedure forfeature extraction that produces PCA-like representations of nonlineardistributions using the learned model. We apply our technique to a handful ofnonlinear data manifolds and show that the resulting representations possessimportant consistencies lacked by unregularized models.