Abstract

We introduce a novel capacity measure 2sED for statistical models based onthe effective dimension. The new quantity provably bounds the generalizationerror under mild assumptions on the model. Furthermore, simulations on standarddata sets and popular model architectures show that 2sED correlates well withthe training error. For Markovian models, we show how to efficientlyapproximate 2sED from below through a layerwise iterative approach, whichallows us to tackle deep learning models with a large number of parameters.Simulation results suggest that the approximation is good for differentprominent models and data sets.

 

Quick Read (beta)