Abstract
Future mobile devices are anticipated to perceive, understand and react tothe world on their own by running multiple correlated deep neural networkson-device. Yet the complexity of these neural networks needs to be trimmed downboth withinmodel and cross-model to fit in mobile storage and memory. Previousstudies focus on squeezing the redundancy within a single neural network. Inthis work, we aim to reduce the redundancy across multiple models. We proposeMulti-Task Zipping (MTZ), a framework to automatically merge correlated,pre-trained deep neural networks for cross-model compression. Central in MTZ isa layer-wise neuron sharing and incoming weight updating scheme that induces aminimal change in the error function. MTZ inherits information from each modeland demands light retraining to re-boost the accuracy of individual tasks.Evaluations show that MTZ is able to fully merge the hidden layers of twoVGG-16 networks with a 3.18% increase in the test error averaged on ImageNetand CelebA, or share 39.61% parameters between the two networks with < 0.5%increase in the test errors for both tasks. The number of iterations to retrainthe combined network is at least 17.8x lower than that of training a singleVGG-16 network.