Siamese networks have become a common structure in various recent models for
unsupervised visual representation learning. These models maximize the
similarity between two augmentations of one image, subject to certain
conditions for avoiding collapsing solutions. In this paper, we report
surprising empirical results that simple Siamese networks can learn meaningful
representations even using none of the following: (i) negative sample pairs,
(ii) large batches, (iii) momentum encoders. Our experiments show that
collapsing solutions do exist for the loss and structure, but a stop-gradient
operation plays an essential role in preventing collapsing. We provide a
hypothesis on the implication of stop-gradient, and further show
proof-of-concept experiments verifying it. Our "SimSiam" method achieves
competitive results on ImageNet and downstream tasks. We hope this simple
baseline will motivate people to rethink the roles of Siamese architectures for
unsupervised representation learning. Code will be made available.

本文通过研究，展示出使用 Siamese 网络进行图像无监督表示学习的含义，并阐述了在该情况下级联网络的基本角色，并提出了一个新的基于停止梯度的理论，通过实验证明了其正确性。