Deciphering language from brain activity is a crucial task in brain-computer
interface (BCI) research. Non-invasive cerebral signaling techniques including
electroencephalography (EEG) and magnetoencephalography (MEG) are becoming
increasingly popular due to their safety and practicality, avoiding invasive
electrode implantation. However, current works under-investigated three points:
1) a predominant focus on EEG with limited exploration of MEG, which provides
superior signal quality; 2) poor performance on unseen text, indicating the
need for models that can better generalize to diverse linguistic contexts; 3)
insufficient integration of information from other modalities, which could
potentially constrain our capacity to comprehensively understand the intricate
dynamics of brain activity.
This study presents a novel approach for translating MEG signals into text
using a speech-decoding framework with multiple alignments. Our method is the
first to introduce an end-to-end multi-alignment framework for totally unseen
text generation directly from MEG signals. We achieve an impressive BLEU-1
score on the $\textit{GWilliams}$ dataset, significantly outperforming the
baseline from 5.49 to 10.44 on the BLEU-1 metric. This improvement demonstrates
the advancement of our model towards real-world applications and underscores
its potential in advancing BCI research. Code is available at
$\href{https://github.com/NeuSpeech/MAD-MEG2text}{this https URL}$.

本研究提出了一种使用多重对齐框架将脑电信号转换为文本的新方法，通过在 MEG 信号中直接生成全新的文本，实现了令人印象深刻的 BLEU-1 得分，极大地超越了基线模型，显示出实现 BCI 研究的潜力。