Abstract

For classification models based on neural networks, the maximum predictedclass probability is often used as a confidence score. This score rarelypredicts well the probability of making a correct prediction and requires apost-processing calibration step. However, many confidence calibration methodsfail for problems with many classes. To address this issue, we transform theproblem of calibrating a multiclass classifier into calibrating a singlesurrogate binary classifier. This approach allows for more efficient use ofstandard calibration methods. We evaluate our approach on numerous neuralnetworks used for image or text classification and show that it significantlyenhances existing calibration methods.

 

Quick Read (beta)