Abstract
This paper highlights the importance of personalization in large languagemodels and introduces the LaMP benchmark -- a novel benchmark for training andevaluating language models for producing personalized outputs. LaMP offers acomprehensive evaluation framework with diverse language tasks and multipleentries for each user profile. It consists of seven personalized tasks,spanning three text classification and four text generation tasks. Weadditionally propose two retrieval augmentation approaches that retrievepersonal items from each user profile for personalizing language model outputs.To this aim, we study various retrieval models, including term matching,semantic matching, and time-aware methods. Extensive experiments on LaMP forzero-shot and fine-tuned language models demonstrate the efficacy of theproposed retrieval augmentation approach and highlight the impact ofpersonalization in various natural language tasks.