Test-time adaptation (TTA) aims to adapt a model, initially trained on
training data, to potential distribution shifts in the test data. Most existing
TTA studies, however, focus on classification tasks, leaving a notable gap in
the exploration of TTA for semantic segmentation. This pronounced emphasis on
classification might lead numerous newcomers and engineers to mistakenly assume
that classic TTA methods designed for classification can be directly applied to
segmentation. Nonetheless, this assumption remains unverified, posing an open
question. To address this, we conduct a systematic, empirical study to disclose
the unique challenges of segmentation TTA, and to determine whether classic TTA
strategies can effectively address this task. Our comprehensive results have
led to three key observations. First, the classic batch norm updating strategy,
commonly used in classification TTA, only brings slight performance
improvement, and in some cases it might even adversely affect the results. Even
with the application of advanced distribution estimation techniques like batch
renormalization, the problem remains unresolved. Second, the teacher-student
scheme does enhance training stability for segmentation TTA in the presence of
noisy pseudo-labels. However, it cannot directly result in performance
improvement compared to the original model without TTA. Third, segmentation TTA
suffers a severe long-tailed imbalance problem, which is substantially more
complex than that in TTA for classification. This long-tailed challenge
significantly affects segmentation TTA performance, even when the accuracy of
pseudo-labels is high.

经过综合研究，我们发现语义分割中经典的测试时适应方法在解决问题上存在一些挑战，批量归一化更新策略对结果只带来轻微改善，师生方案使训练过程稳定但不会直接提高性能，长尾不平衡问题严重影响语义分割性能。