Abstract
Large Language Models (LLMs) have become powerful, but hallucinations remaina vital obstacle to their trustworthy use. While previous works improved thecapability of hallucination detection by measuring uncertainty, they all lackthe ability to explain the provenance behind why hallucinations occur, i.e.,which part of the inputs tends to trigger hallucinations. Recent works on theprompt attack indicate that uncertainty exists in semantic propagation, whereattention mechanisms gradually fuse local token information into high-levelsemantics across layers. Meanwhile, uncertainty also emerges in languagegeneration, due to its probability-based selection of high-level semantics forsampled generations. Based on that, we propose RePPL to recalibrate uncertaintymeasurement by these two aspects, which dispatches explainable uncertaintyscores to each token and aggregates in Perplexity-style Log-Average form astotal score. Experiments show that our method achieves the best comprehensivedetection performance across various QA datasets on advanced models (averageAUC of 0.833), and our method is capable of producing token-level uncertaintyscores as explanations for the hallucination. Leveraging these scores, wepreliminarily find the chaotic pattern of hallucination and showcase itspromising usage.