Leveraging nearest neighbor retrieval for self-supervised representation
learning has proven beneficial with object-centric images. However, this
approach faces limitations when applied to scene-centric datasets, where
multiple objects within an image are only implicitly captured in the global
representation. Such global bootstrapping can lead to undesirable entanglement
of object representations. Furthermore, even object-centric datasets stand to
benefit from a finer-grained bootstrapping approach. In response to these
challenges, we introduce a novel Cross-Image Object-Level Bootstrapping method
tailored to enhance dense visual representation learning. By employing
object-level nearest neighbor bootstrapping throughout the training, CrIBo
emerges as a notably strong and adequate candidate for in-context learning,
leveraging nearest neighbor retrieval at test time. CrIBo shows
state-of-the-art performance on the latter task while being highly competitive
in more standard downstream segmentation tasks. Our code and pretrained models
will be publicly available upon acceptance.

通过利用最近邻检索来进行自监督表示学习在以物体为中心的图像中已被证明具有益处。然而，当应用于以场景为中心的数据集时，这种方法面临限制，其中图像中的多个物体仅在全局表示中隐含捕获。这种全局引导可能导致对象表示的不可取缠结。此外，即使以物体为中心的数据集也可以受益于一种细粒度的引导方法。为了应对这些挑战，我们引入了一种新颖的针对增强密集视觉表示学习的跨图像对象级引导方法。通过在训练过程中采用对象级最近邻引导方式，CrIBo 成为一个显著强大且适当的候选算法，用于在上下文中学习，并在测试时利用最近邻检索。CrIBo 在后一任务上表现出了最新的性能，同时在更常见的下游分割任务中具有极高的竞争力。我们的代码和预训练模型将在接受后公开提供。

CrIBo：跨图像对象级自监督学习

CrIBo: Self-Supervised Learning via Cross-Image Object-Level  Bootstrapping

Diffusion models excel at generating photorealistic images from text-queries.
Naturally, many approaches have been proposed to use these generative abilities
to augment training datasets for downstream tasks, such as classification.
However, diffusion models are themselves trained on large noisily supervised,
but nonetheless, annotated datasets. It is an open question whether the
generalization capabilities of diffusion models beyond using the additional
data of the pre-training process for augmentation lead to improved downstream
performance. We perform a systematic evaluation of existing methods to generate
images from diffusion models and study new extensions to assess their benefit
for data augmentation. While we find that personalizing diffusion models
towards the target data outperforms simpler prompting strategies, we also show
that using the training data of the diffusion model alone, via a simple nearest
neighbor retrieval procedure, leads to even stronger downstream performance.
Overall, our study probes the limitations of diffusion models for data
augmentation but also highlights its potential in generating new training data
to improve performance on simple downstream vision tasks.

本文系统评估了扩散模型生成图像的现有方法，并研究了新的扩展方式以评估它们对数据增强的益处。作者发现，将扩散模型个性化到目标数据的方法优于简单的提示策略，但使用扩散模型的训练数据，通过简单的最近邻检索程序，直接提高下游性能。此项研究揭示了扩散模型在数据增强方面的局限性，同时也突显了其在生成新训练数据方面的潜力，以提高在简单的下游视觉任务中的性能。