End-to-End Conformal Calibration for Optimization Under Uncertainty

  • 2024-09-30 18:38:27
  • Christopher Yeh, Nicolas Christianson, Alan Wu, Adam Wierman, Yisong Yue
  • 0

Abstract

Machine learning can significantly improve performance for decision-makingunder uncertainty in a wide range of domains. However, ensuring robustnessguarantees requires well-calibrated uncertainty estimates, which can bedifficult to achieve in high-capacity prediction models such as deep neuralnetworks. Moreover, in high-dimensional settings, there may be many validuncertainty estimates, each with their own performance profile - i.e., not alluncertainty is equally valuable for downstream decision-making. To address thisproblem, this paper develops an end-to-end framework to learn the uncertaintyestimates for conditional robust optimization, with robustness and calibrationguarantees provided by conformal prediction. In addition, we propose torepresent arbitrary convex uncertainty sets with partially input-convex neuralnetworks, which are learned as part of our framework. Our approach consistentlyimproves upon two-stage estimate-then-optimize baselines on concreteapplications in energy storage arbitrage and portfolio optimization.

 

Quick Read (beta)

loading the full paper ...