Abstract
Simultaneously addressing multiple objectives is becoming increasinglyimportant in modern machine learning. At the same time, data is oftenhigh-dimensional and costly to label. For a single objective such as predictionrisk, conventional regularization techniques are known to improvegeneralization when the data exhibits low-dimensional structure like sparsity.However, it is largely unexplored how to leverage this structure in the contextof multi-objective learning (MOL) with multiple competing objectives. In thiswork, we discuss how the application of vanilla regularization approaches canfail, and propose a two-stage MOL framework that can successfully leveragelow-dimensional structure. We demonstrate its effectiveness experimentally formulti-distribution learning and fairness-risk trade-offs.