An Improved Trade-off Between Accuracy and Complexity with Progressive Gradient Pruning

  • 2019-08-12 13:46:47
  • Le Thanh Nguyen-Meidine, Eric Granger, Madhu Kiran, Louis-Antoine Blais-Morin
  • 0

Abstract

Although deep neural networks (NNs) have achieved state-of-the-art accuracyin many visual recognition tasks ,the growing computational complexity andenergy consumption of networks remains an issue, especially for applications onplatforms with limited resources and requiring real-time processing. Channelpruning techniques have recently shown promising results for the compression ofconvolutional NNs (CNNs). However, these techniques can result in low accuracyand complex optimisations because some only prune after training CNNs, whileothers prune from scratch during training by integrating sparsity constraintsor modifying the loss function. The progressive soft filter pruning techniqueprovides greater training efficiency, but its soft pruning strategy does nothandle the backward pass which is needed for better optimization. In thispaper, a new Progressive Gradient Pruning (PGP) technique is proposed foriterative channel pruning during training. It relies on a criterion thatmeasures the change in channel weights that improves existing progressivepruning, and on an effective hard and soft pruning strategies to adapt momentumtensors during the backward propagation pass. Experimental results obtainedafter training various CNNs on the MNIST and CIFAR10 datasets indicate that thePGP technique canachieve a better tradeoff between classification accuracy andnetwork (time and memory) complexity than state-of-the-art channel pruningtechniques

 

Quick Read (beta)

loading the full paper ...