Abstract
Large language models are instruction-finetuned to enhance their ability tofollow user instructions and process the input context. However, evenstate-of-the-art models often struggle to follow the instruction, especiallywhen the input context is not aligned with the model's parametric knowledge.This manifests as various failures, such as hallucinations where the responsesare outdated, biased or contain unverified facts. In this work, we try tounderstand the underlying reason for this poor context reliance, especiallyafter instruction tuning. We observe an intriguing phenomenon: duringinstruction tuning, the context reliance initially increases as expected, butthen gradually decreases as instruction finetuning progresses. We call thisphenomenon context-parametric inversion and observe it across multiple generalpurpose instruction tuning datasets like TULU, Alpaca and Ultrachat, as well asmodel families such as Llama, Mistral and Pythia. In a simple theoreticalsetup, we isolate why context-parametric inversion occurs along the gradientdescent trajectory of instruction finetuning. We tie this phenomena to examplesin the instruction finetuning data mixture where the input context providesinformation that is already present in the model's parametric knowledge. Ouranalysis suggests natural mitigation strategies that provide some limitedgains, while also validating our theoretical insights. We hope that our workserves as a starting point in addressing this failure mode in a staple part ofLLM training.