Abstract
Multilingual LLMs often have knowledge disparities across languages, withlarger gaps in under-resourced languages. Teaching LLMs to abstain in the faceof knowledge gaps is thus a promising strategy to mitigate hallucinations inmultilingual settings. However, previous studies on LLM abstention primarilyfocus on English; we find that directly applying existing solutions beyondEnglish results in up to 20.5% performance gaps between high and low-resourcelanguages, potentially due to LLMs' drop in calibration and reasoning beyond afew resource-rich languages. To this end, we propose strategies to enhance LLMabstention by learning from multilingual feedback, where LLMs self-reflect onproposed answers in one language by generating multiple feedback items inrelated languages: we show that this helps identifying the knowledge gapsacross diverse languages, cultures, and communities. Extensive experimentsdemonstrate that our multilingual feedback approach outperforms various strongbaselines, achieving up to 9.2% improvement for low-resource languages acrossthree black-box and open models on three datasets, featuring open-book,closed-book, and commonsense QA. Further analysis reveals that multilingualfeedback is both an effective and a more equitable abstain strategy to servediverse language speakers, and cultural factors have great impact on languageselection and LLM abstention behavior, highlighting future directions formultilingual and multi-cultural reliable language modeling.