ITUNLP at SemEval-2025 Task 8: Question-Answering over Tabular Data: A Zero-Shot Approach using LLM-Driven Code Generation

  • 2025-08-01 16:38:18
  • Atakan Site, Emre Hakan Erdemir, Gülşen Eryiğit
  • 0

Abstract

This paper presents our system for SemEval-2025 Task 8: DataBench,Question-Answering over Tabular Data. The primary objective of this task is toperform question answering on given tabular datasets from diverse domains undertwo subtasks: DataBench QA (Subtask I) and DataBench Lite QA (Subtask II). Totackle both subtasks, we developed a zero-shot solution with a particularemphasis on leveraging Large Language Model (LLM)-based code generation.Specifically, we propose a Python code generation framework utilizingstate-of-the-art open-source LLMs to generate executable Pandas code viaoptimized prompting strategies. Our experiments reveal that different LLMsexhibit varying levels of effectiveness in Python code generation.Additionally, results show that Python code generation achieves superiorperformance in tabular question answering compared to alternative approaches.Although our ranking among zero-shot systems is unknown at the time of thispaper's submission, our system achieved eighth place in Subtask I and sixthplace in Subtask~II among the 30 systems that outperformed the baseline in theopen-source models category.

 

Quick Read (beta)

loading the full paper ...