Deep Neural Networks are Surprisingly Reversible: A Baseline for Zero-Shot Inversion

  • 2021-07-13 18:01:43
  • Xin Dong, Hongxu Yin, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov
  • 21


Understanding the behavior and vulnerability of pre-trained deep neuralnetworks (DNNs) can help to improve them. Analysis can be performed viareversing the network's flow to generate inputs from internal representations.Most existing work relies on priors or data-intensive optimization to invert amodel, yet struggles to scale to deep architectures and complex datasets. Thispaper presents a zero-shot direct model inversion framework that recovers theinput to the trained model given only the internal representation. The crux ofour method is to inverse the DNN in a divide-and-conquer manner whilere-syncing the inverted layers via cycle-consistency guidance with the help ofsynthesized data. As a result, we obtain a single feed-forward model capable ofinversion with a single forward pass without seeing any real data of theoriginal task. With the proposed approach, we scale zero-shot direct inversionto deep architectures and complex datasets. We empirically show that modernclassification models on ImageNet can, surprisingly, be inverted, allowing anapproximate recovery of the original 224x224px images from a representationafter more than 20 layers. Moreover, inversion of generators in GANs unveilslatent code of a given synthesized face image at 128x128px, which can even, inturn, improve defective synthesized images from GANs.


Quick Read (beta)

loading the full paper ...