Abstract
Instruction-tuned large language models (LLMs), such as ChatGPT, have led topromising zero-shot performance in discriminative natural languageunderstanding (NLU) tasks. This involves querying the LLM using a promptcontaining the question, and the candidate labels to choose from. Thequestion-answering capabilities of ChatGPT arise from its pre-training on largeamounts of human-written text, as well as its subsequent fine-tuning on humanpreferences, which motivates us to ask: Does ChatGPT also inherits humans'cognitive biases? In this paper, we study the primacy effect of ChatGPT: thetendency of selecting the labels at earlier positions as the answer. We havetwo main findings: i) ChatGPT's decision is sensitive to the order of labels inthe prompt; ii) ChatGPT has a clearly higher chance to select the labels atearlier positions as the answer. We hope that our experiments and analysesprovide additional insights into building more reliable ChatGPT-basedsolutions. We release the source code athttps://github.com/wangywUST/PrimacyEffectGPT.