Abstract
Users often ask dialogue systems ambiguous questions that requireclarification. We show that current language models rarely ask users to clarifyambiguous questions and instead provide incorrect answers. To address this, weintroduce CLAM: a framework for getting language models to selectively ask forclarification about ambiguous user questions. In particular, we show that wecan prompt language models to detect whether a given question is ambiguous,generate an appropriate clarifying question to ask the user, and give a finalanswer after receiving clarification. We also show that we can simulate usersby providing language models with privileged information. This lets usautomatically evaluate multi-turn clarification dialogues. Finally, CLAMsignificantly improves language models' accuracy on mixed ambiguous andunambiguous questions relative to SotA.