Diabetic Retinopathy (DR), a leading cause of vision impairment, requires
early detection and treatment. Developing robust AI models for DR
classification holds substantial potential, but a key challenge is ensuring
their generalization in unfamiliar domains with varying data distributions. To
address this, our paper investigates cross-domain generalization, also known as
domain generalization (DG), within the context of DR classification. DG, a
challenging problem in the medical domain, is complicated by the difficulty of
gathering labeled data across different domains, such as patient demographics
and disease stages. Some recent studies have shown the effectiveness of using
CLIP to handle the DG problem in natural images. In this study, we investigate
CLIP's transfer learning capabilities and its potential for cross-domain
generalization in diabetic retinopathy (DR) classification. We carry out
comprehensive experiments to assess the efficacy and potential of CLIP in
addressing DG for DR classification. Further, we introduce a multi-modal
fine-tuning strategy named Context Optimization with Learnable Visual Tokens
(CoOpLVT), which enhances context optimization by conditioning on visual
features. Our findings demonstrate that the proposed method increases the
F1-score by 1.8% over the baseline, thus underlining its promise for effective
DG in DR classification. Our code is publicly available at
this https URL

糖尿病视网膜病变（DR）是视力障碍的主要原因之一，需要早期检测和治疗。本文研究了糖尿病视网膜病变（DR）分类问题中跨域泛化的挑战，通过使用 CLIP 模型在不同领域和数据分布下进行转化学习，引入了一种名为 CoOpLVT 的多模态微调策略来增强上下文优化，实验结果表明该方法相较于基线模型提高了 1.8% 的 F1 分数，为 DR 分类的有效跨域泛化提供了希望。