Abstract
Large Language Models (LLMs) have become essential in many Natural LanguageProcessing (NLP) tasks, leveraging extensive pre-training and fine-tuning toachieve high accuracy. However, like humans, LLMs exhibit biases, particularlypositional biases such as primacy and recency effects, which can influence theaccuracy of the answers. The primacy effect-where items presented first aremore likely to be remembered or selected-plays a key role in Multiple ChoiceQuestion Answering (MCQA), where the order of answer options can affectprediction outcomes. This study focuses on primacy bias in fine-tuned LLMs: Wefirst show that fine-tuning amplifies this bias, probably due to exposure tohuman-like patterns. Hence, we strategically leverage this effect by reorderingresponse options based on semantic similarity to the query, without requiringknowledge of the correct answer. Our experimental results show that thisapproach significantly improves performance in MCQA. More generally, ourfindings underscore the dual nature of biases as both challenges andopportunities, offering insights for bias-aware model design and NLPapplications.