Abstract
Nowadays, real-world applications often face streaming data, which requiresthe learning system to absorb new knowledge as data evolves. Continual Learning(CL) aims to achieve this goal and meanwhile overcome the catastrophicforgetting of former knowledge when learning new ones. Typical CL methods buildthe model from scratch to grow with incoming data. However, the advent of thepre-trained model (PTM) era has sparked immense research interest, particularlyin leveraging PTMs' robust representational capabilities. This paper presents acomprehensive survey of the latest advancements in PTM-based CL. We categorizeexisting methodologies into three distinct groups, providing a comparativeanalysis of their similarities, differences, and respective advantages anddisadvantages. Additionally, we offer an empirical study contrasting variousstate-of-the-art methods to highlight concerns regarding fairness incomparisons. The source code to reproduce these evaluations is available at:https://github.com/sun-hailong/LAMDA-PILOT