Pixel-Adaptive Convolutional Neural Networks

  • 2019-04-10 18:02:54
  • Hang Su, Varun Jampani, Deqing Sun, Orazio Gallo, Erik Learned-Miller, Jan Kautz
  • 31


Convolutions are the fundamental building block of CNNs. The fact that theirweights are spatially shared is one of the main reasons for their widespreaduse, but it also is a major limitation, as it makes convolutions contentagnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yeteffective modification of standard convolutions, in which the filter weightsare multiplied with a spatially-varying kernel that depends on learnable, localpixel features. PAC is a generalization of several popular filtering techniquesand thus can be used for a wide range of use cases. Specifically, wedemonstrate state-of-the-art performance when PAC is used for deep joint imageupsampling. PAC also offers an effective alternative to fully-connected CRF(Full-CRF), called PAC-CRF, which performs competitively, while beingconsiderably faster. In addition, we also demonstrate that PAC can be used as adrop-in replacement for convolution layers in pre-trained networks, resultingin consistent performance improvements.


Introduction (beta)



Conclusion (beta)