Abstract

We present HEAD-QA, a multi-choice question answering testbed to encourageresearch on complex reasoning. The questions come from exams to access aspecialized position in the Spanish healthcare system, and are challenging evenfor highly specialized humans. We then consider monolingual (Spanish) andcross-lingual (to English) experiments with information retrieval and neuraltechniques. We show that: (i) HEAD-QA challenges current methods, and (ii) theresults lag well behind human performance, demonstrating its usefulness as abenchmark for future work.

 

Quick Read (beta)