Can Large Language Models Address Open-Target Stance Detection?

  • 2024-09-30 18:37:16
  • Abu Ubaida Akash, Ahmed Fahmy, Amine Trabelsi
  • 0

Abstract

Stance detection (SD) identifies a text's position towards a target,typically labeled as favor, against, or none. We introduce Open-Target StanceDetection (OTSD), the most realistic task where targets are neither seen duringtraining nor provided as input. We evaluate Large Language Models (LLMs)GPT-4o, GPT-3.5, Llama-3, and Mistral, comparing their performance to the onlyexisting work, Target-Stance Extraction (TSE), which benefits from predefinedtargets. Unlike TSE, OTSD removes the dependency of a predefined list, makingtarget generation and evaluation more challenging. We also provide a metric forevaluating target quality that correlates well with human judgment. Ourexperiments reveal that LLMs outperform TSE in target generation when the realtarget is explicitly and not explicitly mentioned in the text. Likewise, forstance detection, LLMs excel in explicit cases with comparable performance innon-explicit in general.

 

Quick Read (beta)

loading the full paper ...