Deep neural networks (DNNs) used for brain-computer-interface (BCI)
classification are commonly expected to learn general features when trained
across a variety of contexts, such that these features could be fine-tuned to
specific contexts. While some success is found in such an approach, we suggest
that this interpretation is limited and an alternative would better leverage
the newly (publicly) available massive EEG datasets. We consider how to adapt
techniques and architectures used for language modelling (LM), that appear
capable of ingesting awesome amounts of data, towards the development of
encephalography modelling (EM) with DNNs in the same vein. We specifically
adapt an approach effectively used for automatic speech recognition, which
similarly (to LMs) uses a self-supervised training objective to learn
compressed representations of raw data signals. After adaptation to EEG, we
find that a single pre-trained model is capable of modelling completely novel
raw EEG sequences recorded with differing hardware, and different subjects
performing different tasks. Furthermore, both the internal representations of
this model and the entire architecture can be fine-tuned to a variety of
downstream BCI and EEG classification tasks, outperforming prior work in more
task-specific (sleep stage classification) self-supervision.

本文着眼于通过自我监督学习和适应语言建模技术来发展基于深度神经网络的脑电建模技术，建立起一个单一的预训练模型，用于建模不同硬件、不同受试者和不同任务的全新脑电图序列记录，从而取得较好的脑机接口和脑电分类性能。