Abstract
As Large Language Models (LLMs) are widely deployed in diverse scenarios, theextent to which they could tacitly spread misinformation emerges as a criticalsafety concern. Current research primarily evaluates LLMs on explicit falsestatements, overlooking how misinformation often manifests subtly asunchallenged premises in real-world user interactions. We curated ECHOMIST, thefirst comprehensive benchmark for implicit misinformation, where themisinformed assumptions are embedded in a user query to LLMs. ECHOMIST is basedon rigorous selection criteria and carefully curated data from diverse sources,including real-world human-AI conversations and social media interactions. Wealso introduce a new evaluation metric to measure whether LLMs can recognizeand counter false information rather than amplify users' misconceptions.Through an extensive empirical study on a wide range of LLMs, including GPT-4,Claude, and Llama, we find that current models perform alarmingly poorly onthis task, often failing to detect false premises and generating misleadingexplanations. Our findings underscore the critical need for an increased focuson implicit misinformation in LLM safety research.