Abstract
Convolutional neural networks (CNNs) learn filters in order to capture localcorrelation patterns in feature space. We propose to learn these filters ascombinations of preset spectral filters defined by the Discrete CosineTransform (DCT). Our proposed DCT-based harmonic blocks replace conventionalconvolutional layers to produce partially or fully harmonic versions of new orexisting CNN architectures. Using DCT energy compaction properties, wedemonstrate how the harmonic networks can be efficiently compressed bytruncating high-frequency information in harmonic blocks thanks to theredundancies in the spectral domain. We report extensive experimentalvalidation demonstrating benefits of the introduction of harmonic blocks intostate-of-the-art CNN models in image classification, object detection andsemantic segmentation applications.