Meta-learning for mixed linear regression

  • 2020-02-20 18:34:28
  • Weihao Kong, Raghav Somani, Zhao Song, Sham Kakade, Sewoong Oh
  • 2

Abstract

In modern supervised learning, there are a large number of tasks, but many ofthem are associated with only a small amount of labeled data. These includedata from medical image processing and robotic interaction. Even though eachindividual task cannot be meaningfully trained in isolation, one seeks tometa-learn across the tasks from past experiences by exploiting somesimilarities. We study a fundamental question of interest: When can abundanttasks with small data compensate for lack of tasks with big data? We focus on acanonical scenario where each task is drawn from a mixture of $k$ linearregressions, and identify sufficient conditions for such a graceful exchange tohold; The total number of examples necessary with only small data tasks scalessimilarly as when big data tasks are available. To this end, we introduce anovel spectral approach and show that we can efficiently utilize small datatasks with the help of $\tilde\Omega(k^{3/2})$ medium data tasks each with$\tilde\Omega(k^{1/2})$ examples.

 

Quick Read (beta)

loading the full paper ...