Abstract

Gradient tree boosting is a prediction algorithm that sequentially produces amodel in the form of linear combinations of decision trees, by solving aninfinite-dimensional optimization problem. We combine gradient boosting andNesterov's accelerated descent to design a new algorithm, which we call AGB(for Accelerated Gradient Boosting). Substantial numerical evidence is providedon both synthetic and real-life data sets to assess the excellent performanceof the method in a large variety of prediction problems. It is empiricallyshown that AGB is much less sensitive to the shrinkage parameter and outputspredictors that are considerably more sparse in the number of trees, whileretaining the exceptional performance of gradient boosting.

 

Quick Read (beta)