Abstract

Dropout is often used in deep neural networks to prevent over-fitting.Conventionally, dropout training invokes \textit{random drop} of nodes from thehidden layers of a Neural Network. It is our hypothesis that a guided selectionof nodes for intelligent dropout can lead to better generalization as comparedto the traditional dropout. In this research, we propose "guided dropout" fortraining deep neural network which drop nodes by measuring the strength of eachnode. We also demonstrate that conventional dropout is a specific case of theproposed guided dropout. Experimental evaluation on multiple datasets includingMNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet demonstrate the efficacy ofthe proposed guided dropout.

 

Quick Read (beta)