Deep neural networks have achieved remarkable performance across various
tasks when supplied with large-scale labeled data. However, the collection of
labeled data can be time-consuming and labor-intensive. Semi-supervised
learning (SSL), particularly through pseudo-labeling algorithms that
iteratively assign pseudo-labels for self-training, offers a promising solution
to mitigate the dependency of labeled data. Previous research generally applies
a uniform pseudo-labeling strategy across all model layers, assuming that
pseudo-labels exert uniform influence throughout. Contrasting this, our
theoretical analysis and empirical experiment demonstrate feature extraction
layer and linear classification layer have distinct learning behaviors in
response to pseudo-labels. Based on these insights, we develop two
layer-specific pseudo-label strategies, termed Grad-ReLU and Avg-Clustering.
Grad-ReLU mitigates the impact of noisy pseudo-labels by removing the gradient
detrimental effects of pseudo-labels in the linear classification layer.
Avg-Clustering accelerates the convergence of feature extraction layer towards
stable clustering centers by integrating consistent outputs. Our approach,
LayerMatch, which integrates these two strategies, can avoid the severe
interference of noisy pseudo-labels in the linear classification layer while
accelerating the clustering capability of the feature extraction layer. Through
extensive experimentation, our approach consistently demonstrates exceptional
performance on standard semi-supervised learning benchmarks, achieving a
significant improvement of 10.38% over baseline method and a 2.44% increase
compared to state-of-the-art methods.

基于理论分析和经验实验证明，特征提取层和线性分类层在响应伪标签时具有不同的学习行为。我们开发了两种层特定的伪标签策略：Grad-ReLU 可以通过在线性分类层中消除伪标签的梯度不利影响来减轻噪声伪标签的影响，Avg-Clustering 可以通过整合一致输出来加快特征提取层向稳定聚类中心的收敛速度。我们的方法 LayerMatch 集成了这两种策略，可以避免线性分类层中噪声伪标签的严重干扰，同时加速特征提取层的聚类能力。经过大量实验，我们的方法在标准半监督学习基准上始终展现出卓越的性能，相对基准方法提升了 10.38%，相对最先进的方法提升了 2.44%。