Abstract
Calibrating verbalized probabilities presents a novel approach for reliablyassessing and leveraging outputs from black-box Large Language Models (LLMs).Recent methods have demonstrated improved calibration by applying techniqueslike Platt scaling or temperature scaling to the confidence scores generated byLLMs. In this paper, we explore the calibration of verbalized probabilitydistributions for discriminative tasks. First, we investigate the capability ofLLMs to generate probability distributions over categorical labels. Wetheoretically and empirically identify the issue of re-softmax arising from thescaling of verbalized probabilities, and propose using the invert softmax trickto approximate the "logit" by inverting verbalized probabilities. Throughextensive evaluation on three public datasets, we demonstrate: (1) the robustcapability of LLMs in generating class distributions, and (2) the effectivenessof the invert softmax trick in estimating logits, which, in turn, facilitatespost-calibration adjustments.