Abstract
Large-language models (LLMs) and chatbot agents are known to provide wrongoutputs at times, and it was recently found that this can never be fullyprevented. Hence, uncertainty quantification plays a crucial role, aiming toquantify the level of ambiguity in either one overall number or two numbers foraleatoric and epistemic uncertainty. This position paper argues that thistraditional dichotomy of uncertainties is too limited for the open andinteractive setup that LLM agents operate in when communicating with a user,and that we need to research avenues that enrich uncertainties in this novelscenario. We review the literature and find that popular definitions ofaleatoric and epistemic uncertainties directly contradict each other and losetheir meaning in interactive LLM agent settings. Hence, we propose three novelresearch directions that focus on uncertainties in such human-computerinteractions: Underspecification uncertainties, for when users do not provideall information or define the exact task at the first go, interactive learning,to ask follow-up questions and reduce the uncertainty about the currentcontext, and output uncertainties, to utilize the rich language and speechspace to express uncertainties as more than mere numbers. We expect that thesenew ways of dealing with and communicating uncertainties will lead to LLM agentinteractions that are more transparent, trustworthy, and intuitive.