Decoding language from brain dynamics is an important open direction in the
realm of brain-computer interface (BCI), especially considering the rapid
growth of large language models. Compared to invasive-based signals which
require electrode implantation surgery, non-invasive neural signals (e.g. EEG,
MEG) have attracted increasing attention considering their safety and
generality. However, the exploration is not adequate in three aspects: 1)
previous methods mainly focus on EEG but none of the previous works address
this problem on MEG with better signal quality; 2) prior works have
predominantly used ``teacher-forcing" during generative decoding, which is
impractical; 3) prior works are mostly ``BART-based" not fully auto-regressive,
which performs better in other sequence tasks. In this paper, we explore the
brain-to-text translation of MEG signals in a speech-decoding formation. Here
we are the first to investigate a cross-attention-based ``whisper" model for
generating text directly from MEG signals without teacher forcing. Our model
achieves impressive BLEU-1 scores of 60.30 and 52.89 without pretraining \&
teacher-forcing on two major datasets (\textit{GWilliams} and
\textit{Schoffelen}). This paper conducts a comprehensive review to understand
how speech decoding formation performs on the neural decoding tasks, including
pretraining initialization, training \& evaluation set splitting, augmentation,
and scaling law.

在该论文中，我们首次探索了基于跨注意力的 “whisper” 模型，通过 MEG 信号直接生成文本，而无需教师强制，实现了令人印象深刻的 60.30 和 52.89 的 BLEU-1 得分，该论文还对有关神经解码任务的语音解码形成进行了全面的回顾。