Abstract
Over the past few years, Large Language Models (LLMs) have developed rapidlyand are widely applied in various domains. However, LLMs face the issue ofhallucinations, generating responses that may be unreliable when the modelslack relevant knowledge. To be aware of potential hallucinations, uncertaintyestimation methods have been introduced, and most of them have confirmed thatreliability lies in critical tokens. However, probability-based methods performpoorly in identifying token reliability, limiting their practical utility. Inthis paper, we reveal that the probability-based method fails to estimate tokenreliability due to the loss of evidence strength information which isaccumulated in the training stage. Therefore, we present Logits-induced tokenuncertainty (LogTokU), a framework for estimating decoupled token uncertaintyin LLMs, enabling real-time uncertainty estimation without requiring multiplesampling processes. We employ evidence modeling to implement LogTokU and usethe estimated uncertainty to guide downstream tasks. The experimental resultsdemonstrate that LogTokU has significant effectiveness and promise.