Abstract

We investigate a stochastic counterpart of majority votes over finiteensembles of classifiers, and study its generalization properties. While ourapproach holds for arbitrary distributions, we instantiate it with Dirichletdistributions: this allows for a closed-form and differentiable expression forthe expected risk, which then turns the generalization bound into a tractabletraining objective. The resulting stochastic majority vote learning algorithmachieves state-of-the-art accuracy and benefits from (non-vacuous) tightgeneralization bounds, in a series of numerical experiments when compared tocompeting algorithms which also minimize PAC-Bayes objectives -- both withuninformed (data-independent) and informed (data-dependent) priors.

 

Quick Read (beta)