Interstitial lung diseases (ILD) present diagnostic challenges due to their varied manifestations and overlapping imaging features. To address this, we propose a machine learning approach that utilizes CLIP, a multimodal (image and text) self-supervised model, for ILD classification. We extensively integrate zero-shot CLIP throughout our workflow, starting from the initial extraction of image patches from volumetric CT scans and proceeding to ILD classification using "patch montages". Furthermore, we investigate how domain adaptive pretraining (DAPT) CLIP with task-specific images (CT "patch montages" extracted with ILD-specific prompts for CLIP) and/or text (lung-specific sections of radiology reports) affects downstream ILD classification performance. By leveraging CLIP-extracted "patch montages" and DAPT, we achieve strong zero-shot ILD classification results, including an AUROC of 0.893, without the need for any labeled training data. This work highlights the versatility and potential of multimodal models like CLIP for medical image classification tasks where labeled data is scarce.

本研究采用机器学习的方法，利用多模态（图像和文本）的自监督模型CLIP，对间质性肺疾病（ILD）进行分类，并探索了如何利用域自适应预训练（DAPT）CLIP实现零样本分类，通过使用CLIP提取的“patch montage”和DAPT，实现了强大的ILD分类结果，包括0.893的AUROC，为医学图像分类任务提供了一种新的解决方案。

探索零样本CLIP在间质性肺疾病分类中的多功能性