Abstract
As interest in economic narratives has grown in recent years, so has thenumber of pipelines dedicated to extracting such narratives from texts.Pipelines often employ a mix of state-of-the-art natural language processingtechniques, such as BERT, to tackle this task. While effective on foundationallinguistic operations essential for narrative extraction, such models lack thedeeper semantic understanding required to distinguish extracting economicnarratives from merely conducting classic tasks like Semantic Role Labeling.Instead of relying on complex model pipelines, we evaluate the benefits ofLarge Language Models (LLMs) by analyzing a corpus of Wall Street Journal andNew York Times newspaper articles about inflation. We apply a rigorousnarrative definition and compare GPT-4o outputs to gold-standard narrativesproduced by expert annotators. Our results suggests that GPT-4o is capable ofextracting valid economic narratives in a structured format, but still fallsshort of expert-level performance when handling complex documents andnarratives. Given the novelty of LLMs in economic research, we also provideguidance for future work in economics and the social sciences that employs LLMsto pursue similar objectives.