Decoding memory content from brain activity during sleep has long been a goal
in neuroscience. While spontaneous reactivation of memories during sleep in
rodents is known to support memory consolidation and offline learning,
capturing memory replay in humans is challenging due to the absence of
well-annotated sleep datasets and the substantial differences in neural
patterns between wakefulness and sleep. To address these challenges, we
designed a novel cognitive neuroscience experiment and collected a
comprehensive, well-annotated electroencephalography (EEG) dataset from 52
subjects during both wakefulness and sleep. Leveraging this benchmark dataset,
we developed the Universal Sleep Decoder (USD) to align neural representations
between wakefulness and sleep across subjects. Our model achieves up to 16.6%
top-1 zero-shot accuracy on unseen subjects, comparable to decoding
performances using individual sleep data. Furthermore, fine-tuning USD on test
subjects enhances decoding accuracy to 25.9% top-1 accuracy, a substantial
improvement over the baseline chance of 6.7%. Model comparison and ablation
analyses reveal that our design choices, including the use of (i) an additional
contrastive objective to integrate awake and sleep neural signals and (ii) the
pretrain-finetune paradigm to incorporate different subjects, significantly
contribute to these performances. Collectively, our findings and methodologies
represent a significant advancement in the field of sleep decoding.

通过收集全面的电脑图谱（EEG）数据集并设计了 Universal Sleep Decoder（USD）模型，成功地在清醒状态和睡眠状态之间建立了神经表征，实现了对记忆重播的解码，提高了解码准确度。