Deciphering the intricacies of the human brain has captivated curiosity for
centuries. Recent strides in Brain-Computer Interface (BCI) technology,
particularly using motor imagery, have restored motor functions such as
reaching, grasping, and walking in paralyzed individuals. However, unraveling
natural language from brain signals remains a formidable challenge.
Electroencephalography (EEG) is a non-invasive technique used to record
electrical activity in the brain by placing electrodes on the scalp. Previous
studies of EEG-to-text decoding have achieved high accuracy on small closed
vocabularies, but still fall short of high accuracy when dealing with large
open vocabularies. We propose a novel method, EEG2TEXT, to improve the accuracy
of open vocabulary EEG-to-text decoding. Specifically, EEG2TEXT leverages EEG
pre-training to enhance the learning of semantics from EEG signals and proposes
a multi-view transformer to model the EEG signal processing by different
spatial regions of the brain. Experiments show that EEG2TEXT has superior
performance, outperforming the state-of-the-art baseline methods by a large
margin of up to 5% in absolute BLEU and ROUGE scores. EEG2TEXT shows great
potential for a high-performance open-vocabulary brain-to-text system to
facilitate communication.

通过使用 EEG2TEXT 方法，借助 EEG 预训练和多视图变压器，能够从脑电图信号中提取更准确的开放词汇解码，并在绝对 BLEU 和 ROUGE 得分上超过现有基线方法高达 5%，显示出高性能开放词汇脑机接口与文本系统用于促进交流的巨大潜力。