Abstract
This work considers the problem of computing the canonical polyadicdecomposition (CPD) of large tensors. Prior works mostly leverage data sparsityto handle this problem, which is not suitable for handling dense tensors thatoften arise in applications such as medical imaging, computer vision, andremote sensing. Stochastic optimization is known for its low memory cost andper-iteration complexity when handling dense data. However, exisitingstochastic CPD algorithms are not flexible enough to incorporate a variety ofconstraints/regularizations that are of interest in signal and data analytics.Convergence properties of many such algorithms are also unclear. In this work,we propose a stochastic optimization framework for large-scale CPD withconstraints/regularizations. The framework works under a doubly randomizedfashion, and can be regarded as a judicious combination of randomized blockcoordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithmenjoys lightweight updates and small memory footprint. In addition, thisframework entails considerable flexibility---many frequently used regularizersand constraints can be readily handled under the proposed scheme. The approachis also supported by convergence analysis. Numerical results on large-scaledense tensors are employed to showcase the effectiveness of the proposedapproach.