Abstract

The concept of conditional computation for deep nets has been proposedpreviously to improve model performance by selectively using only parts of themodel conditioned on the sample it is processing. In this paper, we investigateinput-dependent dynamic filter selection in deep convolutional neural networks(CNNs). The problem is interesting because the idea of forcing different partsof the model to learn from different types of samples may help us acquirebetter filters in CNNs, improve the model generalization performance andpotentially increase the interpretability of model behavior. We propose a novelyet simple framework called GaterNet, which involves a backbone and a gaternetwork. The backbone network is a regular CNN that performs the majorcomputation needed for making a prediction, while a global gater network isintroduced to generate binary gates for selectively activating filters in thebackbone network based on each input. Extensive experiments on CIFAR andImageNet datasets show that our models consistently outperform the originalmodels with a large margin. On CIFAR-10, our model also improves uponstate-of-the-art results.

 

Introduction (beta)

None

 

Conclusion (beta)

None