Weighted Random Search for Hyperparameter Optimization

  • 2020-04-03 15:41:22
  • Adrian-Catalin Florea, Razvan Andonie
  • 2

Abstract

We introduce an improved version of Random Search (RS), used here forhyperparameter optimization of machine learning algorithms. Unlike the standardRS, which generates for each trial new values for all hyperparameters, wegenerate new values for each hyperparameter with a probability of change. Theintuition behind our approach is that a value that already triggered a goodresult is a good candidate for the next step, and should be tested in newcombinations of hyperparameter values. Within the same computational budget,our method yields better results than the standard RS. Our theoretical resultsprove this statement. We test our method on a variation of one of the mostcommonly used objective function for this class of problems (the Grievankfunction) and for the hyperparameter optimization of a deep learning CNNarchitecture. Our results can be generalized to any optimization problemdefined on a discrete domain.

 

Quick Read (beta)

loading the full paper ...