This paper presents our team's participation in the MEDIQA-ClinicalNLP2024
shared task B. We present a novel approach to diagnosing clinical dermatology
cases by integrating large multimodal models, specifically leveraging the
capabilities of GPT-4V under a retriever and a re-ranker framework. Our
investigation reveals that GPT-4V, when used as a retrieval agent, can
accurately retrieve the correct skin condition 85% of the time using
dermatological images and brief patient histories. Additionally, we empirically
show that Naive Chain-of-Thought (CoT) works well for retrieval while Medical
Guidelines Grounded CoT is required for accurate dermatological diagnosis.
Further, we introduce a Multi-Agent Conversation (MAC) framework and show its
superior performance and potential over the best CoT strategy. The experiments
suggest that using naive CoT for retrieval and multi-agent conversation for
critique-based diagnosis, GPT-4V can lead to an early and accurate diagnosis of
dermatological conditions. The implications of this work extend to improving
diagnostic workflows, supporting dermatological education, and enhancing
patient care by providing a scalable, accessible, and accurate diagnostic tool.

通过整合大型多模型，特别是利用 GPT-4V 在检索者和重新排序框架下的能力，本研究提出了一种诊断临床皮肤病例的新方法，实验证明使用 GPT-4V 作为检索代理在检索皮肤病情时能够准确地检索到正确的皮肤状况，同时通过多智能体对话框架进行诊断可提供更早和准确的诊断。

UMass-BioNLP 在 MEDIQA-M3G 2024 中的 DermPrompt：基于 GPT-4V 的皮肤诊断系统的系统化探索

UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration  of Prompt Engineering with GPT-4V for Dermatological Diagnosis

Reflectance Confocal Microscopy (RCM) is a non-invasive imaging technique
used in biomedical research and clinical dermatology. It provides virtual
high-resolution images of the skin and superficial tissues, reducing the need
for physical biopsies. RCM employs a laser light source to illuminate the
tissue, capturing the reflected light to generate detailed images of
microscopic structures at various depths. Recent studies explored AI and
machine learning, particularly CNNs, for analyzing RCM images. Our study
proposes a segmentation strategy based on textural features to identify
clinically significant regions, empowering dermatologists in effective image
interpretation and boosting diagnostic confidence. This approach promises to
advance dermatological diagnosis and treatment.

反射共聚焦显微镜（RCM）是一种非侵入性成像技术，用于生物医学研究和临床皮肤学。它提供了皮肤和表浅组织的虚拟高分辨率图像，减少了对物理活检的需求。RCM 采用激光光源照射组织，捕捉反射光生成不同深度微观结构的详细图像。最近的研究探索了 AI 和机器学习，特别是 CNN，用于分析 RCM 图像。本研究提出了一种基于纹理特征的分割策略，用于识别具有临床意义的区域，提升皮肤科医生的图像解释能力和诊断信心。这种方法有望推动皮肤病的诊断和治疗。

通过利用人工智能驱动的反射共聚焦显微镜分析提升诊断

Enhancing Diagnosis through AI-driven Analysis of Reflectance Confocal  Microscopy

Clinical dermatology necessitates precision and innovation for efficient
diagnosis and treatment of various skin conditions. This paper introduces the
development of a cutting-edge hyperspectral dermatoscope (the Hyperscope)
tailored for human skin analysis. We detail the requirements to such a device
and the design considerations, from optical configurations to sensor selection,
necessary to capture a wide spectral range with high fidelity. Preliminary
results from 15 individuals and 160 recorded skin images demonstrate the
potential of the Hyperscope in identifying and characterizing various skin
conditions, offering a promising avenue for non-invasive skin evaluation and a
platform for future research in dermatology-related hyperspectral imaging.

该论文介绍了一种先进的高光谱皮肤镜（Hyterscope），旨在为人类皮肤分析提供高精准度和创新性的诊断和治疗。我们详细介绍了这种设备的要求和设计考虑因素，从光学配置到传感器选择，以捕捉宽频谱范围和高保真度。对 15 名个体和 160 个记录的皮肤图像的初步结果表明 Hyterscope 在识别和表征各种皮肤病变方面具有潜力，为非侵入性皮肤评估和皮肤高光谱成像领域的未来研究提供了一个有前景的途径。