Abstract
A fundamental question in data-driven decision making is how to quantify theuncertainty of predictions in ways that can usefully inform downstream action.This interface between prediction uncertainty and decision-making is especiallyimportant in risk-sensitive domains, such as medicine. In this paper, wedevelop decision-theoretic foundations that connect uncertainty quantificationusing prediction sets with risk-averse decision-making. Specifically, we answerthree fundamental questions: (1) What is the correct notion of uncertaintyquantification for risk-averse decision makers? We prove that prediction setsare optimal for decision makers who wish to optimize their value at risk. (2)What is the optimal policy that a risk averse decision maker should use to mapprediction sets to actions? We show that a simple max-min decision policy isoptimal for risk-averse decision makers. Finally, (3) How can we deriveprediction sets that are optimal for such decision makers? We provide an exactcharacterization in the population regime and a distribution free finite-sampleconstruction. Answering these questions naturally leads to an algorithm,Risk-Averse Calibration (RAC), which follows a provably optimal design forderiving action policies from predictions. RAC is designed to be bothpractical-capable of leveraging the quality of predictions in a black-boxmanner to enhance downstream utility-and safe-adhering to a user-defined riskthreshold and optimizing the corresponding risk quantile of the user'sdownstream utility. Finally, we experimentally demonstrate the significantadvantages of RAC in applications such as medical diagnosis and recommendationsystems. Specifically, we show that RAC achieves a substantially improvedtrade-off between safety and utility, offering higher utility compared toexisting methods while maintaining the safety guarantee.