Autonomous driving has attracted much attention over the years but turns out
to be harder than expected, probably due to the difficulty of labeled data
collection for model training. Self-supervised learning (SSL), which leverages
unlabeled data only for representation learning, might be a promising way to
improve model performance. Existing SSL methods, however, usually rely on the
single-centric-object guarantee, which may not be applicable for multi-instance
datasets such as street scenes. To alleviate this limitation, we raise two
issues to solve: (1) how to define positive samples for cross-view consistency
and (2) how to measure similarity in multi-instance circumstances. We first
adopt an IoU threshold during random cropping to transfer global-inconsistency
to local-consistency. Then, we propose two feature alignment methods to enable
2D feature maps for multi-instance similarity measurement. Additionally, we
adopt intra-image clustering with self-attention for further mining intra-image
similarity and translation-invariance. Experiments show that, when pre-trained
on Waymo dataset, our method called Multi-instance Siamese Network (MultiSiam)
remarkably improves generalization ability and achieves state-of-the-art
transfer performance on autonomous driving benchmarks, including Cityscapes and
BDD100K, while existing SSL counterparts like MoCo, MoCo-v2, and BYOL show
significant performance drop. By pre-training on SODA10M, a large-scale
autonomous driving dataset, MultiSiam exceeds the ImageNet pre-trained MoCo-v2,
demonstrating the potential of domain-specific pre-training. Code will be
available at this https URL

自主驾驶是一个备受关注的领域，但标记数据的收集过程非常困难，自监督学习可能是提高模型性能的一种有效方式。本文提出了一种名为 MultiSiam 的方法，解决了多示例情况下的自监督学习存在的问题，包括跨视图一致性和相似度度量。该方法在公开数据集上具有很强的性能表现，并显示了特定领域预训练的潜力。