Concentration inequalities and optimal number of layers for stochastic deep neural networks

  • 2022-06-22 18:42:24
  • Michele Caprio, Sayan Mukherjee
  • 0

Abstract

We state concentration and martingale inequalities for the output of thehidden layers of a stochastic deep neural network (SDNN), as well as for theoutput of the whole SDNN. These results allow us to introduce an expectedclassifier (EC), and to give probabilistic upper bound for the classificationerror of the EC. We also state the optimal number of layers for the SDNN via anoptimal stopping procedure. We apply our analysis to a stochastic version of afeedforward neural network with ReLU activation function.

 

Quick Read (beta)

loading the full paper ...