Individual differences of Electroencephalogram (EEG) could cause the domain
shift which would significantly degrade the performance of cross-subject
strategy. The domain adversarial neural networks (DANN), where the
classification loss and domain loss jointly update the parameters of feature
extractor, are adopted to deal with the domain shift. However, limited EEG data
quantity and strong individual difference are challenges for the DANN with
cumbersome feature extractor. In this work, we propose knowledge distillation
(KD) based lightweight DANN to enhance cross-subject EEG-based emotion
recognition. Specifically, the teacher model with strong context learning
ability is utilized to learn complex temporal dynamics and spatial correlations
of EEG, and robust lightweight student model is guided by the teacher model to
learn more difficult domain-invariant features. In the feature-based KD
framework, a transformer-based hierarchical temporalspatial learning model is
served as the teacher model. The student model, which is composed of Bi-LSTM
units, is a lightweight version of the teacher model. Hence, the student model
could be supervised to mimic the robust feature representations of teacher
model by leveraging complementary latent temporal features and spatial
features. In the DANN-based cross-subject emotion recognition, we combine the
obtained student model and a lightweight temporal-spatial feature interaction
module as the feature extractor. And the feature aggregation is fed to the
emotion classifier and domain classifier for domain-invariant feature learning.
To verify the effectiveness of the proposed method, we conduct the
subject-independent experiments on the public dataset DEAP with arousal and
valence classification. The outstanding performance and t-SNE visualization of
latent features verify the advantage and effectiveness of the proposed method.

本研究提出基于知识蒸馏的轻量级 DANN 模型，引入教师模型来学习 EEG 的复杂时空动态和空间相关性，指导学生模型学习更难以区分的域不变特征，结合轻量级时空特征交互模块，达到跨主体情感识别的优异性能。

基于知识蒸馏的轻量级领域对抗神经网络用于基于 EEG 的跨受试者情绪识别

A Lightweight Domain Adversarial Neural Network Based on Knowledge  Distillation for EEG-based Cross-subject Emotion Recognition

Theoretically, domain adaptation is a well-researched problem. Further, this
theory has been well-used in practice. In particular, we note the bound on
target error given by Ben-David et al. (2010) and the well-known
domain-aligning algorithm based on this work using Domain Adversarial Neural
Networks (DANN) presented by Ganin and Lempitsky (2015). Recently, multiple
variants of DANN have been proposed for the related problem of domain
generalization, but without much discussion of the original motivating bound.
In this paper, we investigate the validity of DANN in domain generalization
from this perspective. We investigate conditions under which application of
DANN makes sense and further consider DANN as a dynamic process during
training. Our investigation suggests that the application of DANN to domain
generalization may not be as straightforward as it seems. To address this, we
design an algorithmic extension to DANN in the domain generalization case. Our
experimentation validates both theory and algorithm.

本研究通过研究 DANN 在域泛化中的应用，讨论其使用的条件并考虑其作为训练时的动态过程。实验证明了我们提出的算法对于域泛化问题的扩展是有效的。