Sign-regularized Multi-task Learning

  • 2021-02-22 17:11:15
  • Johnny Torres, Guangji Bai, Junxiang Wang, Liang Zhao, Carmen Vaca, Cristina Abad
  • 0


Multi-task learning is a framework that enforces different learning tasks toshare their knowledge to improve their generalization performance. It is a hotand active domain that strives to handle several core issues; particularly,which tasks are correlated and similar, and how to share the knowledge amongcorrelated tasks. Existing works usually do not distinguish the polarity andmagnitude of feature weights and commonly rely on linear correlation, due tothree major technical challenges in: 1) optimizing the models that regularizefeature weight polarity, 2) deciding whether to regularize sign or magnitude,3) identifying which tasks should share their sign and/or magnitude patterns.To address them, this paper proposes a new multi-task learning framework thatcan regularize feature weight signs across tasks. We innovatively formulate itas a biconvex inequality constrained optimization with slacks and propose a newefficient algorithm for the optimization with theoretical guarantees ongeneralization performance and convergence. Extensive experiments on multipledatasets demonstrate the proposed methods' effectiveness, efficiency, andreasonableness of the regularized feature weighted patterns.


Quick Read (beta)

loading the full paper ...