Early-stage diabetic retinopathy (DR) presents challenges in clinical
diagnosis due to inconspicuous and minute microangioma lesions, resulting in
limited research in this area. Additionally, the potential of emerging
foundation models, such as the segment anything model (SAM), in medical
scenarios remains rarely explored. In this work, we propose a
human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg,
based on SAM. GlanceSeg enables real-time segmentation of microangioma lesions
as ophthalmologists review fundus images. Our human-in-the-loop framework
integrates the ophthalmologist's gaze map, allowing for rough localization of
minute lesions in fundus images. Subsequently, a saliency map is generated
based on the located region of interest, which provides prompt points to assist
the foundation model in efficiently segmenting microangioma lesions. Finally, a
domain knowledge filter refines the segmentation of minute lesions. We
conducted experiments on two newly-built public datasets, i.e., IDRiD and
Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg
through visualized illustrations and quantitative measures. Additionally, we
demonstrated that GlanceSeg improves annotation efficiency for clinicians and
enhances segmentation performance through fine-tuning using annotations. This
study highlights the potential of GlanceSeg-based annotations for self-model
optimization, leading to enduring performance advancements through continual
learning.

提出了基于 SAM 的 GlanceSeg 人在回合的无标签早期糖尿病视网膜病变诊断框架，该框架通过实时分割显微血管瘤病灶和眼科医生的视线图进行糖尿病视网膜病变的诊断，实验证明了 GlanceSeg 的可行性和优越性。