Abstract
In this paper, we study the problem of uncertainty estimation and calibrationfor LLMs. We begin by formulating the uncertainty estimation problem, arelevant yet underexplored area in existing literature. We then propose asupervised approach that leverages labeled datasets to estimate the uncertaintyin LLMs' responses. Based on the formulation, we illustrate the differencebetween the uncertainty estimation for LLMs and that for standard ML models andexplain why the hidden neurons of the LLMs may contain uncertainty information.Our designed approach demonstrates the benefits of utilizing hidden activationsto enhance uncertainty estimation across various tasks and shows robusttransferability in out-of-distribution settings. We distinguish the uncertaintyestimation task from the uncertainty calibration task and show that betteruncertainty estimation leads to better calibration performance. Furthermore,our method is easy to implement and adaptable to different levels of modelaccessibility including black box, grey box, and white box.