Abstract
Recent studies empirically indicate that language models (LMs) encode richworld knowledge beyond mere semantics, attracting significant attention acrossvarious fields. However, in the recommendation domain, it remains uncertainwhether LMs implicitly encode user preference information. Contrary toprevailing understanding that LMs and traditional recommenders learn twodistinct representation spaces due to the huge gap in language and behaviormodeling objectives, this work re-examines such understanding and exploresextracting a recommendation space directly from the language representationspace. Surprisingly, our findings demonstrate that item representations, whenlinearly mapped from advanced LM representations, yield superior recommendationperformance. This outcome suggests the possible homomorphism between theadvanced language representation space and an effective item representationspace for recommendation, implying that collaborative signals may be implicitlyencoded within LMs. Motivated by these findings, we explore the possibility ofdesigning advanced collaborative filtering (CF) models purely based on languagerepresentations without ID-based embeddings. To be specific, we incorporateseveral crucial components to build a simple yet effective model, with itemtitles as the input. Empirical results show that such a simple model canoutperform leading ID-based CF models, which sheds light on using languagerepresentations for better recommendation. Moreover, we systematically analyzethis simple model and find several key features for using advanced languagerepresentations: a good initialization for item representations, zero-shotrecommendation abilities, and being aware of user intention. Our findingshighlight the connection between language modeling and behavior modeling, whichcan inspire both natural language processing and recommender systemcommunities.