Deciphering natural language from brain activity through non-invasive devices
remains a formidable challenge. Previous non-invasive decoders either require
multiple experiments with identical stimuli to pinpoint cortical regions and
enhance signal-to-noise ratios in brain activity, or they are limited to
discerning basic linguistic elements such as letters and words. We propose a
novel approach to decoding continuous language from single-trial non-invasive
fMRI recordings, in which a three-dimensional convolutional network augmented
with information bottleneck is developed to automatically identify responsive
voxels to stimuli, and a character-based decoder is designed for the semantic
reconstruction of continuous language characterized by inherent character
structures. The resulting decoder can produce intelligible textual sequences
that faithfully capture the meaning of perceived speech both within and across
subjects, while existing decoders exhibit significantly inferior performance in
cross-subject contexts. The ability to decode continuous language from single
trials across subjects demonstrates the promising applications of non-invasive
language brain-computer interfaces in both healthcare and neuroscience.

通过非侵入式设备从大脑活动中解读自然语言仍然是一个巨大的挑战，本文提出了一种新颖的方法，通过三维卷积网络和信息瓶颈技术对单次非侵入性功能磁共振成像记录进行连续语言解码，该解码器能够生成捕捉到感知语音含义的可理解文本序列，并在跨受试者场景中表现出明显优于现有解码器的性能，展示了非侵入式语言脑机接口在医疗保健和神经科学领域的应用前景。