Abstract
Convolutional Neural Networks (CNNs) achieve high performance in imageclassification tasks but are challenging to deploy on resource-limited hardwaredue to their large model sizes. To address this issue, we leverage MutualInformation, a metric that provides valuable insights into how deep learningmodels retain and process information through measuring the shared informationbetween input features or output labels and network layers. In this study, wepropose a structured filter-pruning approach for CNNs that identifies andselectively retains the most informative features in each layer. Our approachsuccessively evaluates each layer by ranking the importance of its feature mapsbased on Conditional Mutual Information (CMI) values, computed using amatrix-based Renyi {\alpha}-order entropy numerical method. We propose severalformulations of CMI to capture correlation among features across differentlayers. We then develop various strategies to determine the cutoff point forCMI values to prune unimportant features. This approach allows parallel pruningin both forward and backward directions and significantly reduces model sizewhile preserving accuracy. Tested on the VGG16 architecture with the CIFAR-10dataset, the proposed method reduces the number of filters by more than athird, with only a 0.32% drop in test accuracy.