Optimizing Posterior Samples for Bayesian Optimization via Rootfinding

  • 2024-10-29 18:57:16
  • Taiwo A. Adebiyi, Bach Do, Ruda Zhang
  • 0

Abstract

Bayesian optimization devolves the global optimization of a costly objectivefunction to the global optimization of a sequence of acquisition functions.This inner-loop optimization can be catastrophically difficult if it involvesposterior samples, especially in higher dimensions. We introduce an efficientglobal optimization strategy for posterior samples based on global rootfinding.It provides gradient-based optimizers with judiciously selected startingpoints, designed to combine exploitation and exploration. The algorithm scalespractically linearly to high dimensions. For posterior sample-based acquisitionfunctions such as Gaussian process Thompson sampling (GP-TS) and variants ofentropy search, we demonstrate remarkable improvement in both inner- andouter-loop optimization, surprisingly outperforming alternatives like EI andGP-UCB in most cases. We also propose a sample-average formulation of GP-TS,which has a parameter to explicitly control exploitation and can be computed atthe cost of one posterior sample. Our implementation is available athttps://github.com/UQUH/TSRoots .

 

Quick Read (beta)

loading the full paper ...