Abstract
Over the broad landscape of experimental design, regression has been apowerful tool to accurately predict the outcome metrics of a system or modelgiven a set of parameters, but has been traditionally restricted to methodswhich are only applicable to a specific task. In this paper, we proposeOmniPred, a framework for training language models as universal end-to-endregressors over $(x,y)$ evaluation data from diverse real world experiments.Using data sourced from Google Vizier, one of the largest blackbox optimizationdatabases in the world, our extensive experiments demonstrate that through onlytextual representations of mathematical parameters and values, language modelsare capable of very precise numerical regression, and if given the opportunityto train over multiple tasks, can significantly outperform traditionalregression models.