Reconstructing natural language from non-invasive electroencephalography
(EEG) holds great promise as a language decoding technology for brain-computer
interfaces (BCIs). However, EEG-based language decoding is still in its nascent
stages, facing several technical issues such as: 1) Absence of a hybrid
strategy that can effectively integrate cross-modality (between EEG and text)
self-learning with intra-modality self-reconstruction of EEG features or
textual sequences; 2) Under-utilization of large language models (LLMs) to
enhance EEG-based language decoding. To address above issues, we propose the
Contrastive EEG-Text Masked Autoencoder (CET-MAE), a novel model that
orchestrates compound self-supervised learning across and within EEG and text
through a dedicated multi-stream encoder. Furthermore, we develop a framework
called E2T-PTR (EEG-to-Text decoding using Pretrained Transferable
Representations), which leverages pre-trained modules alongside the EEG stream
from CET-MAE and further enables an LLM (specifically BART) to decode text from
EEG sequences. Comprehensive experiments conducted on the popular text-evoked
EEG database, ZuCo, demonstrate the superiority of E2T-PTR, which outperforms
the state-of-the-art in ROUGE-1 F1 and BLEU-4 scores by 8.34% and 32.21%,
respectively. These results indicate significant advancements in the field and
underscores the proposed framework's potential to enable more powerful and
widespread BCI applications.

提出了一种新颖的 Contrastive EEG-Text Masked Autoencoder 模型，并借助预训练模块以及 EEG 流实现了 EEG 到文本的解码，实验结果表明在文本激发的 EEG 数据库上，该模型在 ROUGE-1 F1 和 BLEU-4 得分上分别超过现有技术 8.34％和 32.21％，这显示了该框架在大脑计算界面应用领域的潜力。