From Disagreement to Understanding: The Case for Ambiguity Detection in NLI

  • 2025-07-20 20:27:35
  • Chathuri Jayaweera, Bonnie Dorr
  • 0

Abstract

This position paper argues that annotation disagreement in Natural LanguageInference (NLI) is not mere noise but often reflects meaningful interpretivevariation, especially when triggered by ambiguity in the premise or hypothesis.While underspecified guidelines and annotator behavior can contribute tovariation, content-based ambiguity offers a process-independent signal ofdivergent human perspectives. We call for a shift toward ambiguity-aware NLI bysystematically identifying ambiguous input pairs and classifying ambiguitytypes. To support this, we present a unified framework that integrates existingtaxonomies and illustrate key ambiguity subtypes through concrete examples.These examples reveal how ambiguity shapes annotator decisions and motivate theneed for targeted detection methods that better align models with humaninterpretation. A key limitation is the lack of datasets annotated forambiguity and subtypes. We propose addressing this gap through new annotatedresources and unsupervised approaches to ambiguity detection -- paving the wayfor more robust, explainable, and human-aligned NLI systems.

 

Quick Read (beta)

loading the full paper ...