Abstract
Deep learning approaches are highly specialized and require training separatemodels for different tasks. Multi-domain learning looks at ways to learn amultitude of different tasks, each coming from a different domain, at once. Themost common approach in multi-domain learning is to form a domain agnosticmodel, the parameters of which are shared among all domains, and learn a smallnumber of extra domain-specific parameters for each individual new domain.However, different domains come with different levels of difficulty;parameterizing the models of all domains using an augmented version of thedomain agnostic model leads to unnecessarily inefficient solutions, especiallyfor easy to solve tasks. We propose an adaptive parameterization approach todeep neural networks for multi-domain learning. The proposed approach performson par with the original approach while reducing by far the number ofparameters, leading to efficient multi-domain learning solutions.