Abstract
Gesture recognition is an important research area in the field of computervision. Most gesture recognition efforts focus on close-set scenarios, therebylimiting the capacity to effectively handle unseen or novel gestures. We aim toaddress class-incremental gesture recognition, which entails the ability toaccommodate new and previously unseen gestures over time. Specifically, weintroduce a Prototype-Guided Pseudo Feature Replay (PGPFR) framework fordata-free class-incremental gesture recognition. This framework comprises fourcomponents: Pseudo Feature Generation with Batch Prototypes (PFGBP),Variational Prototype Replay (VPR) for old classes, Truncated Cross-Entropy(TCE) for new classes, and Continual Classifier Re-Training (CCRT). To tacklethe issue of catastrophic forgetting, the PFGBP dynamically generates adiversity of pseudo features in an online manner, leveraging class prototypesof old classes along with batch class prototypes of new classes. Furthermore,the VPR enforces consistency between the classifier's weights and theprototypes of old classes, leveraging class prototypes and covariance matricesto enhance robustness and generalization capabilities. The TCE mitigates theimpact of domain differences of the classifier caused by pseudo features.Finally, the CCRT training strategy is designed to prevent overfitting to newclasses and ensure the stability of features extracted from old classes.Extensive experiments conducted on two widely used gesture recognitiondatasets, namely SHREC 2017 3D and EgoGesture 3D, demonstrate that our approachoutperforms existing state-of-the-art methods by 11.8\% and 12.8\% in terms ofmean global accuracy, respectively. The code is available onhttps://github.com/sunao-101/PGPFR-3/.