Biological signals, such as electroencephalograms (EEG), play a crucial role
in numerous clinical applications, exhibiting diverse data formats and quality
profiles. Current deep learning models for biosignals are typically specialized
for specific datasets and clinical settings, limiting their broader
applicability. Motivated by the success of large language models in text
processing, we explore the development of foundational models that are trained
from multiple data sources and can be fine-tuned on different downstream
biosignal tasks.
To overcome the unique challenges associated with biosignals of various
formats, such as mismatched channels, variable sample lengths, and prevalent
missing values, we propose a Biosignal Transformer (\method). The proposed
\method model can enable cross-data learning with mismatched channels, variable
lengths, and missing values by tokenizing diverse biosignals into unified
"biosignal sentences". Specifically, we tokenize each channel into fixed-length
segments containing local signal features, flattening them to form consistent
"sentences". Channel embeddings and {\em relative} position embeddings are
added to preserve spatio-temporal features.
The \method model is versatile and applicable to various biosignal learning
settings across different datasets, including joint pre-training for larger
models. Comprehensive evaluations on EEG, electrocardiogram (ECG), and human
activity sensory signals demonstrate that \method outperforms robust baselines
in common settings and facilitates learning across multiple datasets with
different formats. Use CHB-MIT seizure detection task as an example, our
vanilla \method model shows 3\% improvement over baselines in balanced
accuracy, and the pre-trained \method models (optimized from other data
sources) can further bring up to 4\% improvements.

本文提出了 Biosignal Transformer 模型，通过将不同的生物信号令牌化成一致的 “生物信号句子”，添加通道嵌入和相对位置嵌入等技术，实现跨数据集学习，适用于不同数据格式的多个生物信号学习，并在 EEG、ECG 和人类活动感官信号上进行了综合评估。