The rise of Artificial Intelligence creates great promise in the field of
medical discovery, diagnostics and patient management. However, the vast
complexity of all medical domains require a more complex approach that combines
machine learning algorithms, classifiers, segmentation algorithms and, lately,
large language models. In this paper, we describe, implement and assess an
Artificial Intelligence-empowered system and methodology aimed at assisting the
diagnosis process of skin lesions and other skin conditions within the field of
dermatology that aims to holistically address the diagnostic process in this
domain. The workflow integrates large language, transformer-based vision models
and sophisticated machine learning tools. This holistic approach achieves a
nuanced interpretation of dermatological conditions that simulates and
facilitates a dermatologist's workflow. We assess our proposed methodology
through a thorough cross-model validation technique embedded in an evaluation
pipeline that utilizes publicly available medical case studies of skin
conditions and relevant images. To quantitatively score the system performance,
advanced machine learning and natural language processing tools are employed
which focus on similarity comparison and natural language inference.
Additionally, we incorporate a human expert evaluation process based on a
structured checklist to further validate our results. We implemented the
proposed methodology in a system which achieved approximate (weighted) scores
of 0.87 for both contextual understanding and diagnostic accuracy,
demonstrating the efficacy of our approach in enhancing dermatological
analysis. The proposed methodology is expected to prove useful in the
development of next-generation tele-dermatology applications, enhancing remote
consultation capabilities and access to care, especially in underserved areas.

通过结合机器学习算法、分类器、分割算法和最新的大型语言模型，本文描述、实现和评估了一种基于人工智能的系统和方法，旨在辅助皮肤病变和其他皮肤状况的诊断过程，并通过综合方法全面解决皮肤病理诊断的问题。该方法整合了大型语言模型、基于变形器的视觉模型和复杂的机器学习工具，通过使用公开可用的皮肤病例和相关图像，通过交叉模型验证技术和自然语言处理工具评估了提出的方法。我们实现的系统在上下文理解和诊断准确性两个方面均获得了约为 0.87 的加权得分，证明了我们方法的有效性。该方法预期在发展下一代远程皮肤科咨询应用中发挥重要作用，增强远程会诊能力和卫生保健服务的可及性，特别是在服务不足的地区。