The teacher-student framework, prevalent in semi-supervised semantic
segmentation, mainly employs the exponential moving average (EMA) to update a
single teacher's weights based on the student's. However, EMA updates raise a
problem in that the weights of the teacher and student are getting coupled,
causing a potential performance bottleneck. Furthermore, this problem may
become more severe when training with more complicated labels such as
segmentation masks but with few annotated data. This paper introduces Dual
Teacher, a simple yet effective approach that employs dual temporary teachers
aiming to alleviate the coupling problem for the student. The temporary
teachers work in shifts and are progressively improved, so consistently prevent
the teacher and student from becoming excessively close. Specifically, the
temporary teachers periodically take turns generating pseudo-labels to train a
student model and maintain the distinct characteristics of the student model
for each epoch. Consequently, Dual Teacher achieves competitive performance on
the PASCAL VOC, Cityscapes, and ADE20K benchmarks with remarkably shorter
training times than state-of-the-art methods. Moreover, we demonstrate that our
approach is model-agnostic and compatible with both CNN- and Transformer-based
models. Code is available at https://github.com/naver-ai/dual-teacher.

该研究论文介绍了一种名为双教师的简单而有效的方法，利用双临时教师解决了学生和教师权重耦合的问题，通过定期生成伪标签训练学生模型并保持不同特征，以显著缩短训练时间，并证明了该方法适用于各种模型。