Know Your Limits: A Survey of Abstention in Large Language Models

  • 2024-08-08 18:39:47
  • Bingbing Wen, Jihan Yao, Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, Lucy Lu Wang
  • 0

Abstract

Abstention, the refusal of large language models (LLMs) to provide an answer,is increasingly recognized for its potential to mitigate hallucinations andenhance safety in LLM systems. In this survey, we introduce a framework toexamine abstention from three perspectives: the query, the model, and humanvalues. We organize the literature on abstention methods, benchmarks, andevaluation metrics using this framework, and discuss merits and limitations ofprior work. We further identify and motivate areas for future work, centeredaround whether abstention can be achieved as a meta-capability that transcendsspecific tasks or domains, while still providing opportunities to optimizeabstention abilities based on context.

 

Quick Read (beta)

loading the full paper ...