Self-supervised learning (SSL) has led to important breakthroughs in computer
vision by allowing learning from large amounts of unlabeled data. As such, it
might have a pivotal role to play in biomedicine where annotating data requires
a highly specialized expertise. Yet, there are many healthcare domains for
which SSL has not been extensively explored. One such domain is endoscopy,
minimally invasive procedures which are commonly used to detect and treat
infections, chronic inflammatory diseases or cancer. In this work, we study the
use of a leading SSL framework, namely Masked Siamese Networks (MSNs), for
endoscopic video analysis such as colonoscopy and laparoscopy. To fully exploit
the power of SSL, we create sizable unlabeled endoscopic video datasets for
training MSNs. These strong image representations serve as a foundation for
secondary training with limited annotated datasets, resulting in
state-of-the-art performance in endoscopic benchmarks like surgical phase
recognition during laparoscopy and colonoscopic polyp characterization.
Additionally, we achieve a 50% reduction in annotated data size without
sacrificing performance. Thus, our work provides evidence that SSL can
dramatically reduce the need of annotated data in endoscopy.

自监督学习 (SSl) 在计算机视觉中取得了重要突破，在生物医学中具有重要作用，特别是在需要高度专业知识的数据注释领域。本研究使用先进的 SSL 框架，即 Masked Siamese Networks (MSNs)，对内窥镜视频分析进行研究，以检测和治疗感染、慢性炎症性疾病或癌症。我们构建了大规模的无标签内窥镜视频数据集，用于训练 MSNs，以充分利用 SSL 的优势，并通过有限的注释数据集进行二次训练，从而在内窥镜基准测试中实现了最先进的性能，如腹腔镜和结肠镜手术阶段识别以及结肠镜息肉表征。此外，我们实现了注释数据量的 50％减少，同时不降低性能。因此，我们的工作证明了 SSL 在内窥镜检查中可以大大减少对注释数据的需求。

内镜视频分析的自监督学习

Self-Supervised Learning for Endoscopic Video Analysis

A successful paradigm in representation learning is to perform
self-supervised pretraining using tasks based on mini-batch statistics (e.g.,
SimCLR, VICReg, SwAV, MSN). We show that in the formulation of all these
methods is an overlooked prior to learn features that enable uniform clustering
of the data. While this prior has led to remarkably semantic representations
when pretraining on class-balanced data, such as ImageNet, we demonstrate that
it can hamper performance when pretraining on class-imbalanced data. By moving
away from conventional uniformity priors and instead preferring power-law
distributed feature clusters, we show that one can improve the quality of the
learned representations on real-world class-imbalanced datasets. To demonstrate
this, we develop an extension of the Masked Siamese Networks (MSN) method to
support the use of arbitrary features priors.

通过使用不同于以往的特征先验，可以改善对类别不平衡数据预训练时的表征学习结果，具体应用于 Masked Siamese Networks 方法。

自监督学习中的隐藏统一聚集先验

The Hidden Uniform Cluster Prior in Self-Supervised Learning

We propose Masked Siamese Networks (MSN), a self-supervised learning
framework for learning image representations. Our approach matches the
representation of an image view containing randomly masked patches to the
representation of the original unmasked image. This self-supervised
pre-training strategy is particularly scalable when applied to Vision
Transformers since only the unmasked patches are processed by the network. As a
result, MSNs improve the scalability of joint-embedding architectures, while
producing representations of a high semantic level that perform competitively
on low-shot image classification. For instance, on ImageNet-1K, with only 5,000
annotated images, our base MSN model achieves 72.4% top-1 accuracy, and with 1%
of ImageNet-1K labels, we achieve 75.7% top-1 accuracy, setting a new
state-of-the-art for self-supervised learning on this benchmark. Our code is
publicly available.

提出了一种自监督学习框架 ——Masked Siamese Networks (MSN)，用于学习图像表示。通过将随机遮罩块的图像视图的表征与原始未遮罩图像的表征进行匹配，可以进行自监督预训练，该策略特别适用于 Vision Transformer 模型，在提高联合嵌入结构的可扩展性的同时，产生高语义级别的表示，并在低样本图像分类方面表现竞争性，对 ImageNet-1K 数据集上，使用基础 MSN 模型以仅有 5000 个带注释的图像，达到了 72.4％的 top-1 准确度，并且只有 1％的 ImageNet-1K 标签，就取得了 75.7％的 top-1 准确度，创立了该基准测试的自监督学习的新的最佳成果。