EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG

  • 2025-06-17 05:58:31
  • Jacky Tai-Yu Lu, Jung Chiang, Chi-Sheng Chen, Anna Nai-Yun Tung, Hsiang Wei Hu, Yuan Chiao Cheng
  • 0

Abstract

We propose EEG2TEXT-CN, which, to the best of our knowledge, represents oneof the earliest open-vocabulary EEG-to-text generation frameworks tailored forChinese. Built on a biologically grounded EEG encoder (NICE-EEG) and a compactpretrained language model (MiniLM), our architecture aligns multichannel brainsignals with natural language representations via masked pretraining andcontrastive learning. Using a subset of the ChineseEEG dataset, where eachsentence contains approximately ten Chinese characters aligned with 128-channelEEG recorded at 256 Hz, we segment EEG into per-character embeddings andpredict full sentences in a zero-shot setting. The decoder is trained withteacher forcing and padding masks to accommodate variable-length sequences.Evaluation on over 1,500 training-validation sentences and 300 held-out testsamples shows promising lexical alignment, with a best BLEU-1 score of 6.38\%.While syntactic fluency remains a challenge, our findings demonstrate thefeasibility of non-phonetic, cross-modal language decoding from EEG. This workopens a new direction in multilingual brain-to-text research and lays thefoundation for future cognitive-language interfaces in Chinese.

 

Quick Read (beta)

loading the full paper ...