With the concept of teaching being introduced to the machine learning
community, a teacher model start using dynamic loss functions to teach the
training of a student model. The dynamic intends to set adaptive loss functions
to different phases of student model learning. In existing works, the teacher
model 1) merely determines the loss function based on the present states of the
student model, i.e., disregards the experience of the teacher; 2) only utilizes
the states of the student model, e.g., training iteration number and
loss/accuracy from training/validation sets, while ignoring the states of the
loss function. In this paper, we first formulate the loss adjustment as a
temporal task by designing a teacher model with memory units, and, therefore,
enables the student learning to be guided by the experience of the teacher
model. Then, with a dynamic loss network, we can additionally use the states of
the loss to assist the teacher learning in enhancing the interactions between
the teacher and the student model. Extensive experiments demonstrate our
approach can enhance student learning and improve the performance of various
deep models on real-world tasks, including classification, objective detection,
and semantic segmentation scenarios.

此研究论文介绍了使用动态损失函数来指导学生模型训练的教师模型，通过设计带有记忆单元的教师模型，利用教师的经验引导学生学习，并通过动态损失网络利用损失的状态来增强教师和学生模型之间的交互，实验结果证明该方法可以提高学生模型的学习性能和在实际任务中的表现。