Noise-adding Methods of Saliency Map as Series of Higher Order Partial Derivative

  • 2018-06-08 07:41:41
  • Junghoon Seo, Jeongyeol Choe, Jamyoung Koo, Seunghyeon Jeon, Beomsu Kim, Taegyun Jeon
  • 222


SmoothGrad and VarGrad are techniques that enhance the empirical quality ofstandard saliency maps by adding noise to input. However, there were few worksthat provide a rigorous theoretical interpretation of those methods. Weanalytically formalize the result of these noise-adding methods. As a result,we observe two interesting results from the existing noise-adding methods.First, SmoothGrad does not make the gradient of the score function smooth.Second, VarGrad is independent of the gradient of the score function. Webelieve that our findings provide a clue to reveal the relationship betweenlocal explanation methods of deep neural networks and higher-order partialderivatives of the score function.


Introduction (beta)



Conclusion (beta)