Abstract
Continuous electroencephalography (EEG) signals are widely used in affectivebrain-computer interface (aBCI) applications. However, not all continuouslycollected EEG signals are relevant or meaningful to the task at hand (e.g.,wondering thoughts). On the other hand, manually labeling the relevant parts isnearly impossible due to varying engagement patterns across different tasks andindividuals. Therefore, effectively and efficiently identifying the importantparts from continuous EEG recordings is crucial for downstream BCI tasks, as itdirectly impacts the accuracy and reliability of the results. In this paper, wepropose a novel unsupervised deep reinforcement learning framework, calledEmotion-Agent, to automatically identify relevant and informative emotionalmoments from continuous EEG signals. Specifically, Emotion-Agent involvesunsupervised deep reinforcement learning combined with a heuristic algorithm.We first use the heuristic algorithm to perform an initial global search andform prototype representations of the EEG signals, which facilitates theefficient exploration of the signal space and identify potential regions ofinterest. Then, we design distribution-prototype reward functions to estimatethe interactions between samples and prototypes, ensuring that the identifiedparts are both relevant and representative of the underlying emotional states.Emotion-Agent is trained using Proximal Policy Optimization (PPO) to achievestable and efficient convergence. Our experiments compare the performance withand without Emotion-Agent. The results demonstrate that selecting relevant andinformative emotional parts before inputting them into downstream tasksenhances the accuracy and reliability of aBCI applications.