Abstract
Principal component analysis (PCA) can be significantly limited when there istoo few examples of the target data of interest. We propose a transfer learningapproach to PCA (TL-PCA) where knowledge from a related source task is used inaddition to the scarce data of a target task. Our TL-PCA has two versions, onethat uses a pretrained PCA solution of the source task, and another that usesthe source data. Our proposed approach extends the PCA optimization objectivewith a penalty on the proximity of the target subspace and the source subspaceas given by the pretrained source model or the source data. This optimizationis solved by eigendecomposition for which the number of data-dependenteigenvectors (i.e., principal directions of TL-PCA) is not limited to thenumber of target data examples, which is a root cause that limits the standardPCA performance. Accordingly, our results for image datasets show that therepresentation of test data is improved by TL-PCA for dimensionality reductionwhere the learned subspace dimension is lower or higher than the number oftarget data examples.