The past few years have produced a series of spectacular advances in the
decoding of speech from brain activity. The engine of these advances has been
the acquisition of labelled data, with increasingly large datasets acquired
from single subjects. However, participants exhibit anatomical and other
individual differences, and datasets use varied scanners and task designs. As a
result, prior work has struggled to leverage data from multiple subjects,
multiple datasets, multiple tasks, and unlabelled datasets. In turn, the field
has not benefited from the rapidly growing number of open neural data
repositories to exploit large-scale data and deep learning. To address this, we
develop an initial set of neuroscience-inspired self-supervised objectives,
together with a neural architecture, for representation learning from
heterogeneous and unlabelled neural recordings. Experimental results show that
representations learned with these objectives generalise across subjects,
datasets, and tasks, and are also learned faster than using only labelled data.
In addition, we set new benchmarks for two foundational speech decoding tasks.
Taken together, these methods now unlock the potential for training speech
decoding models with orders of magnitude more existing data.

开发一组神经科学启发的自监督目标和神经架构，用于从异构和无标记神经记录中进行表征学习，实验证明这些目标学习到的表征能够普遍适用于不同的被试者、数据集和任务，并比仅使用有标记数据时更快地学习；此外，为两个基础性语音解码任务设置了新的基准，这些方法现在可以利用数量级更多的现有数据来训练语音解码模型。