Abstract
Figurative and metaphorical language are commonplace in discourse, andfigurative expressions play an important role in communication and cognition.However, figurative language has been a relatively under-studied area in NLP,and it remains an open question to what extent modern language models caninterpret nonliteral phrases. To address this question, we introduce Fig-QA, aWinograd-style nonliteral language understanding task consisting of correctlyinterpreting paired figurative phrases with divergent meanings. We evaluate theperformance of several state-of-the-art language models on this task, and findthat although language models achieve performance significantly over chance,they still fall short of human performance, particularly in zero- or few-shotsettings. This suggests that further work is needed to improve the nonliteralreasoning capabilities of language models.