Abstract
While electroencephalogram (EEG) has been a crucial tool for monitoring thebrain and diagnosing neurological disorders (e.g., epilepsy), learningmeaningful representations from raw EEG signals remains challenging due tolimited annotations and high signal variability. Recently, EEG foundationmodels (FMs) have shown promising potential by adopting transformerarchitectures and self-supervised pre-training methods from large languagemodels (e.g., masked prediction) to learn representations from diverse EEGdata, followed by fine-tuning on specific EEG tasks. Nonetheless, these largemodels often incurred high computational costs during both training andinference, with only marginal performance improvements as the model sizeincreases. In this work, we proposed an EEG representation learning frameworkbuilding upon Generative Diffusion Model (EEGDM). Specifically, we developed astructured state-space model for diffusion pretraining (SSMDP) to bettercapture the temporal dynamics of EEG signals and trained the model using aDenoising Diffusion Probabilistic Model. Subsequently, the resulting latent EEGrepresentations were then used for downstream classification tasks via ourproposed latent fusion transformer (LFT). To evaluate our method, we usedmulti-event datasets covering both interictal epileptiform discharges andseizure detection, and compared EEGDM with current state-of-the-art approaches,including EEG FMs. Empirical results showed that our method outperformed theexisting methods. These findings suggested that EEGDM offered a promisingalternative to current FMs. Our code is available at:https://github.com/jhpuah/EEGDM.