Abstract
Traditionally, natural language processing (NLP) models often use a rich setof features created by linguistic expertise, such as semantic representations.However, in the era of large language models (LLMs), more and more tasks areturned into generic, end-to-end sequence generation problems. In this paper, weinvestigate the question: what is the role of semantic representations in theera of LLMs? Specifically, we investigate the effect of Abstract MeaningRepresentation (AMR) across five diverse NLP tasks. We propose an AMR-drivenchain-of-thought prompting method, which we call AMRCoT, and find that itgenerally hurts performance more than it helps. To investigate what AMR mayhave to offer on these tasks, we conduct a series of analysis experiments. Wefind that it is difficult to predict which input examples AMR may help or hurton, but errors tend to arise with multi-word expressions, named entities, andin the final inference step where the LLM must connect its reasoning over theAMR to its prediction. We recommend focusing on these areas for future work insemantic representations for LLMs. Our code:https://github.com/causalNLP/amr_llm.