Traditional semi-supervised learning (SSL) assumes that the feature
distributions of labeled and unlabeled data are consistent which rarely holds
in realistic scenarios. In this paper, we propose a novel SSL setting, where
unlabeled samples are drawn from a mixed distribution that deviates from the
feature distribution of labeled samples. Under this setting, previous SSL
methods tend to predict wrong pseudo-labels with the model fitted on labeled
data, resulting in noise accumulation. To tackle this issue, we propose
Self-Supervised Feature Adaptation (SSFA), a generic framework for improving
SSL performance when labeled and unlabeled data come from different
distributions. SSFA decouples the prediction of pseudo-labels from the current
model to improve the quality of pseudo-labels. Particularly, SSFA incorporates
a self-supervised task into the SSL framework and uses it to adapt the feature
extractor of the model to the unlabeled data. In this way, the extracted
features better fit the distribution of unlabeled data, thereby generating
high-quality pseudo-labels. Extensive experiments show that our proposed SSFA
is applicable to various pseudo-label-based SSL learners and significantly
improves performance in labeled, unlabeled, and even unseen distributions.

通过提出自监督特征适应（SSFA）方法，解决了传统半监督学习在标记和未标记数据分布不一致时预测错误伪标签的问题，从而提高了半监督学习性能。

自监督特征适应的广义半监督学习

Generalized Semi-Supervised Learning via Self-Supervised Feature  Adaptation

Semi-supervised learning (SSL) can utilize unlabeled data to enhance model
performance. In recent years, with increasingly powerful generative models
becoming available, a large number of synthetic images have been uploaded to
public image sets. Therefore, when collecting unlabeled data from these
sources, the inclusion of synthetic images is inevitable. This prompts us to
consider the impact of unlabeled data mixed with real and synthetic images on
SSL. In this paper, we set up a new task, Real and Synthetic hybrid SSL
(RS-SSL), to investigate this problem. We discover that current SSL methods are
unable to fully utilize synthetic data and are sometimes negatively affected.
Then, by analyzing the issues caused by synthetic images, we propose a new SSL
method, RSMatch, to tackle the RS-SSL problem. Extensive experimental results
show that RSMatch can better utilize the synthetic data in unlabeled images to
improve the SSL performance. The effectiveness is further verified through
ablation studies and visualization.

通过分析合成图像的问题，本文提出了一种新的 SSL 方法 RSMatch 来解决混合真实和合成图像对 SSL 的影响问题，并通过实验证明 RSMatch 能够更好地利用未标记图像中的合成数据来提高 SSL 性能。

从障碍到机遇：通过合成数据增强半监督学习

From Obstacle to Opportunity: Enhancing Semi-supervised Learning with  Synthetic Data

Vision Transformers have been rapidly uprising in computer vision thanks to
their outstanding scaling trends, and gradually replacing convolutional neural
networks (CNNs). Recent works on self-supervised learning (SSL) introduce
siamese pre-training tasks, on which Transformer backbones continue to
demonstrate ever stronger results than CNNs. People come to believe that
Transformers or self-attention modules are inherently more suitable than CNNs
in the context of SSL. However, it is noteworthy that most if not all prior
arts of SSL with CNNs chose the standard ResNets as their backbones, whose
architecture effectiveness is known to already lag behind advanced Vision
Transformers. Therefore, it remains unclear whether the self-attention
operation is crucial for the recent advances in SSL - or CNNs can deliver the
same excellence with more advanced designs, too? Can we close the SSL
performance gap between Transformers and CNNs? To answer these intriguing
questions, we apply self-supervised pre-training to the recently proposed,
stronger lager-kernel CNN architecture and conduct an apple-to-apple comparison
with Transformers, in their SSL performance. Our results show that we are able
to build pure CNN SSL architectures that perform on par with or better than the
best SSL-trained Transformers, by just scaling up convolutional kernel sizes
besides other small tweaks. Impressively, when transferring to the downstream
tasks \texttt{MS COCO} detection and segmentation, our SSL pre-trained CNN
model (trained in 100 epochs) achieves the same good performance as the
300-epoch pre-trained Transformer counterpart. We hope this work can help to
better understand what is essential (or not) for self-supervised learning
backbones.

卷积神经网络与视觉 Transformer 在自我监督学习任务中表现相近，通过增加卷积核大小和其他小调整即可达到与 Transformer 相同的性能水平。