The advancements in the development of Large Language Models have evolved as
a transformative paradigm in conversational tasks which has led to its
integration in the critical domain of healthcare. With LLMs becoming widely
popular and their public access through open-source models, there is a need to
investigate their potential and limitations. One such critical task where LLMs
are applied but require a deeper understanding is that of self-diagnosis of
medical conditions in the interest of public health. The widespread integration
of Gemini with Google search, GPT-4.0 with Bing search, has led to shift in
trend of self-diagnosis from search engine LLMs. In this paper, we prepare a
prompt engineered dataset of 10000 samples and test the performance on the
general task of self-diagnosis. We compare the performance of GPT-4.0 and
Gemini model on the task of self-diagnosis and record accuracies of 63.07% and
6.01% respectively. We also discuss the challenges, limitations, and potential
of both Gemini and GPT-4.0 for the task of self-diagnosis to facilitate future
research and towards the broader impact of general public knowledge.
Furthermore, we demonstrate the potential and improvement in performance for
the task of self-diagnosis using Retrieval Augmented Generation.

该研究探讨了大型语言模型在医疗保健领域的应用，特别是在自我诊断方面，使用 Retrieval Augmented Generation 方法显示了在自我诊断任务中性能的潜力和改善。