Recent advancements in large-scale visual-language pre-trained models have led to significant progress in zero-/few-shot anomaly detection within natural image domains. However, the substantial domain divergence between natural and medical images limits the effectiveness of these methodologies in medical anomaly detection. This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. This multi-level adaptation is guided by multi-level, pixel-wise visual-language feature alignment loss functions, which recalibrate the model's focus from object semantics in natural imagery to anomaly identification in medical images. The adapted features exhibit improved generalization across various medical data types, even in zero-shot scenarios where the model encounters unseen medical modalities and anatomical regions during training. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models, with an average AUC improvement of 6.24% and 7.33% for anomaly classification, 2.03% and 2.37% for anomaly segmentation, under the zero-shot and few-shot settings, respectively. Source code is available at: https://github.com/MediaBrain-SJTU/MVFA-AD

本文介绍了一种用于医学异常检测的轻量级多层次自适应对比框架，通过将多个残差适配器整合到预训练的视觉编码器中，通过多级像素级视觉-语言特征对齐损失函数引导多级适应，使其适用于医学图像。实验结果表明，我们的方法在医学异常检测基准上显著超越了当前最先进的模型，在零样本和少样本情况下分别实现了6.24%和7.33%的平均AUC提升用于异常分类，以及2.03%和2.37%的平均AUC提升用于异常分割。

面向医学图像的通用异常检测的视觉语言模型的适应