Recently, multi-modal vision-language foundation models have gained
significant attention in the medical field. While these models offer great
opportunities, they still face a number of challenges, such as the requirement
for fine-grained knowledge understanding in computer-aided diagnosis and
capability of utilizing very limited or no task-specific labeled data in
real-world clinical applications. In this study, we present MaCo, a novel
multi-modal medical foundation model that explores masked contrastive learning
to achieve granular alignment and zero-shot learning for a variety of medical
imaging tasks. MaCo incorporates a correlation weighting mechanism to adjust
the correlation between masked image patches and their corresponding reports,
thereby enhancing the representation learning capabilities. We evaluate MaCo on
six well-known open-source X-ray datasets, and the experimental results show it
outperforms seven state-of-the-art approaches for classification, segmentation,
and zero-shot phase grounding, demonstrating its great potential to promote a
wide range of medical image analysis tasks.

本研究介绍了 MaCo，一种新颖的多模态医学基础模型，通过探索遮蔽对比学习实现医学影像任务的细粒度对齐和零样本学习，并结合相关权重机制来增强表示学习能力。我们在六个知名的开源 X 射线数据集上评估 MaCo，实验结果表明其在分类、分割和零样本阶段定位方面优于七种最先进的方法，展示了其在促进各种医学图像分析任务方面的巨大潜力。

基于掩膜对比学习的细粒度对齐算法：提升放射学报告基础模型的表示能力

Enhancing Representation in Radiography-Reports Foundation Model: A  Granular Alignment Algorithm Using Masked Contrastive Learning

We propose a Vision-Language Transformer (VLT) framework for referring
segmentation to facilitate deep interactions among multi-modal information and
enhance the holistic understanding to vision-language features. There are
different ways to understand the dynamic emphasis of a language expression,
especially when interacting with the image. However, the learned queries in
existing transformer works are fixed after training, which cannot cope with the
randomness and huge diversity of the language expressions. To address this
issue, we propose a Query Generation Module, which dynamically produces
multiple sets of input-specific queries to represent the diverse comprehensions
of language expression. To find the best among these diverse comprehensions, so
as to generate a better mask, we propose a Query Balance Module to selectively
fuse the corresponding responses of the set of queries. Furthermore, to enhance
the model's ability in dealing with diverse language expressions, we consider
inter-sample learning to explicitly endow the model with knowledge of
understanding different language expressions to the same object. We introduce
masked contrastive learning to narrow down the features of different
expressions for the same target object while distinguishing the features of
different objects. The proposed approach is lightweight and achieves new
state-of-the-art referring segmentation results consistently on five datasets.

该研究提出了一种轻量级的视觉 - 语言 Transformer 框架来实现引用分割，其中包括 Query Generation Module，Query Balance Module 以及 masked contrastive learning，它们能够动态生成多个特定于输入的查询，以明晰理解语言表达的多样性，同时针对不同的语言表达方式进行了明确的跨样本学习。该框架在五个数据集上都实现了最新的引用分割成果。

VLT: 面向参考分割的视觉语言变换器和查询生成

VLT: Vision-Language Transformer and Query Generation for Referring  Segmentation

Detecting anomalies is one fundamental aspect of a safety-critical software
system, however, it remains a long-standing problem. Numerous branches of works
have been proposed to alleviate the complication and have demonstrated their
efficiencies. In particular, self-supervised learning based methods are
spurring interest due to their capability of learning diverse representations
without additional labels. Among self-supervised learning tactics, contrastive
learning is one specific framework validating their superiority in various
fields, including anomaly detection. However, the primary objective of
contrastive learning is to learn task-agnostic features without any labels,
which is not entirely suited to discern anomalies. In this paper, we propose a
task-specific variant of contrastive learning named masked contrastive
learning, which is more befitted for anomaly detection. Moreover, we propose a
new inference method dubbed self-ensemble inference that further boosts
performance by leveraging the ability learned through auxiliary
self-supervision tasks. By combining our models, we can outperform previous
state-of-the-art methods by a significant margin on various benchmark datasets.

本文提出了一种基于自我监督学习方法的特定任务变体对比学习称为 “遮蔽对比学习”，其更适用于异常检测，并提出了一种自我集成推断方法，通过利用辅助自我监督任务学习的能力进一步提高模型性能，在各种基准数据集上的表现显著超过以往的最先进方法。