Abstract
Developing a single foundation model with the capability to excel acrossdiverse tasks has been a long-standing objective in the field of artificialintelligence. As the wave of general-purpose foundation models sweeps acrossvarious domains, their influence has significantly extended to the field ofrecommendation systems. While recent efforts have explored recommendationfoundation models for various generative tasks, they often overlook crucialembedding tasks and struggle with the complexities of multi-task learning,including knowledge sharing & conflict resolution, and convergence speedinconsistencies. To address these limitations, we introduce RecFound, agenerative representational learning framework for recommendation foundationmodels. We construct the first comprehensive dataset for recommendationfoundation models covering both generative and embedding tasks across diversescenarios. Based on this dataset, we propose a novel multi-task training schemefeaturing a Task-wise Mixture of Low-rank Experts (TMoLE) to handle knowledgesharing & conflict, a Step-wise Convergence-oriented Sample Scheduler (S2Sched)to address inconsistent convergence, and a Model Merge module to balance theperformance across tasks. Experiments demonstrate that RecFound achievesstate-of-the-art performance across various recommendation tasks, outperformingexisting baselines.