Abstract
Sparse regularization such as $\ell_1$ regularization is a quite powerful andwidely used strategy for high dimensional learning problems. The effectivenessof sparse regularization has been supported practically and theoretically byseveral studies. However, one of the biggest issues in sparse regularization isthat its performance is quite sensitive to correlations between features.Ordinary $\ell_1$ regularization can select variables correlated with eachother, which results in deterioration of not only its generalization error butalso interpretability. In this paper, we propose a new regularization method,"Independently Interpretable Lasso" (IILasso). Our proposed regularizersuppresses selecting correlated variables, and thus each active variableindependently affects the objective variable in the model. Hence, we caninterpret regression coefficients intuitively and also improve the performanceby avoiding overfitting. We analyze theoretical property of IILasso and showthat the proposed method is much advantageous for its sign recovery andachieves almost minimax optimal convergence rate. Synthetic and real dataanalyses also indicate the effectiveness of IILasso.