We present a sequential transfer learning framework for transformers on
functional Magnetic Resonance Imaging (fMRI) data and demonstrate its
significant benefits for decoding musical timbre. In the first of two phases,
we pre-train our stacked-encoder transformer architecture on Next Thought
Prediction, a self-supervised task of predicting whether or not one sequence of
fMRI data follows another. This phase imparts a general understanding of the
temporal and spatial dynamics of neural activity, and can be applied to any
fMRI dataset. In the second phase, we fine-tune the pre-trained models and
train additional fresh models on the supervised task of predicting whether or
not two sequences of fMRI data were recorded while listening to the same
musical timbre. The fine-tuned models achieve significantly higher accuracy
with shorter training times than the fresh models, demonstrating the efficacy
of our framework for facilitating transfer learning on fMRI data. Additionally,
our fine-tuning task achieves a level of classification granularity beyond
standard methods. This work contributes to the growing literature on
transformer architectures for sequential transfer learning on fMRI data, and
provides evidence that our framework is an improvement over current methods for
decoding timbre.

使用顺序迁移学习框架的转换器，可以有效地解码音乐音色，形成一种比现有方法更优的 fMRI 数据序列转换框架，用于预测不同 fMRI 数据序列是否记录了相同的音频音色。

基于顺序迁移学习的 fMRI 数据解码听到和想象中的音色

Sequential Transfer Learning to Decode Heard and Imagined Timbre from  fMRI Data

Different flavors of transfer learning have shown tremendous impact in
advancing research and applications of machine learning. In this work we study
the use of a specific family of transfer learning, where the target domain is
mapped to the source domain. Specifically we map Natural Language Understanding
(NLU) problems to QuestionAnswering (QA) problems and we show that in low data
regimes this approach offers significant improvements compared to other
approaches to NLU. Moreover we show that these gains could be increased through
sequential transfer learning across NLU problems from different domains. We
show that our approach could reduce the amount of required data for the same
performance by up to a factor of 10.

该研究针对自然语言理解问题，通过将其映射到问题回答领域，研究了特定的迁移学习方法，并显示出在低数据环境下，与其他方法相比可以带来显著的改进，特别是通过跨不同领域的 NLU 问题进行顺序转移学习可增加其收益，并可将所需数据减少高达 10 倍。

使用语言模型：将自然语言理解视为问答

Language Model is All You Need: Natural Language Understanding as  Question Answering

Deep neural networks are typically trained under a supervised learning
framework where a model learns a single task using labeled data. Instead of
relying solely on labeled data, practitioners can harness unlabeled or related
data to improve model performance, which is often more accessible and
ubiquitous. Self-supervised pre-training for transfer learning is becoming an
increasingly popular technique to improve state-of-the-art results using
unlabeled data. It involves first pre-training a model on a large amount of
unlabeled data, then adapting the model to target tasks of interest. In this
review, we survey self-supervised learning methods and their applications
within the sequential transfer learning framework. We provide an overview of
the taxonomy for self-supervised learning and transfer learning, and highlight
some prominent methods for designing pre-training tasks across different
domains. Finally, we discuss recent trends and suggest areas for future
investigation.

本论文综述了自监督预训练技术及其在序列迁移学习框架中的应用，包括自监督学习和迁移学习的分类体系、设计跨不同领域的预训练任务的突出方法以及近期趋势和未来研究方向。