Abstract
Most research in Bayesian optimization (BO) has focused on direct feedbackscenarios, where one has access to exact, or perturbed, values of someexpensive-to-evaluate objective. This direction has been mainly driven by theuse of BO in machine learning hyper-parameter configuration problems. However,in domains such as modelling human preferences, A/B tests or recommendersystems, there is a need of methods that are able to replace direct feedbackwith preferential feedback, obtained via rankings or pairwise comparisons. Inthis work, we present Preferential Batch Bayesian Optimization (PBBO), a newframework that allows to find the optimum of a latent function of interest,given any type of parallel preferential feedback for a group of two or morepoints. We do so by using a Gaussian process model with a likelihood speciallydesigned to enable parallel and efficient data collection mechanisms, which arekey in modern machine learning. We show how the acquisitions developed underthis framework generalize and augment previous approaches in Bayesianoptimization, expanding the use of these techniques to a wider range ofdomains. An extensive simulation study shows the benefits of this approach,both with simulated functions and four real data sets.