Abstract

We extend conformal prediction to control the expected value of any monotoneloss function. The algorithm generalizes split conformal prediction togetherwith its coverage guarantee. Like conformal prediction, the conformal riskcontrol procedure is tight up to an $\mathcal{O}(1/n)$ factor. Worked examplesfrom computer vision and natural language processing demonstrate the usage ofour algorithm to bound the false negative rate, graph distance, and token-levelF1-score.

 

Quick Read (beta)