The integration of Artificial Intelligence (AI) in healthcare presents a
transformative potential for enhancing operational efficiency and health
outcomes. Large Language Models (LLMs), such as ChatGPT, have shown their
capabilities in supporting medical decision-making. Embedding LLMs in medical
systems is becoming a promising trend in healthcare development. The potential
of ChatGPT to address the triage problem in emergency departments has been
examined, while few studies have explored its application in outpatient
departments. With a focus on streamlining workflows and enhancing efficiency
for outpatient triage, this study specifically aims to evaluate the consistency
of responses provided by ChatGPT in outpatient guidance, including both
within-version response analysis and between-version comparisons. For
within-version, the results indicate that the internal response consistency for
ChatGPT-4.0 is significantly higher than ChatGPT-3.5 (p=0.03) and both have a
moderate consistency (71.2% for 4.0 and 59.6% for 3.5) in their top
recommendation. However, the between-version consistency is relatively low
(mean consistency score=1.43/3, median=1), indicating few recommendations match
between the two versions. Also, only 50% top recommendations match perfectly in
the comparisons. Interestingly, ChatGPT-3.5 responses are more likely to be
complete than those from ChatGPT-4.0 (p=0.02), suggesting possible differences
in information processing and response generation between the two versions. The
findings offer insights into AI-assisted outpatient operations, while also
facilitating the exploration of potentials and limitations of LLMs in
healthcare utilization. Future research may focus on carefully optimizing LLMs
and AI integration in healthcare systems based on ergonomic and human factors
principles, precisely aligning with the specific needs of effective outpatient
triage.

评估 ChatGPT 在门诊指导中提供的回复一致性，包括版本内回复分析和版本间比较，结果表明 ChatGPT-4.0 的内部回复一致性显著高于 ChatGPT-3.5，并且两者的最佳推荐都具有中等一致性。然而，版本间的一致性相对较低，指示两个版本之间几乎没有匹配的推荐。此外，只有 50％的最佳推荐在比较中完全匹配。ChatGPT-3.5 的回复更可能是完整的，相较于 ChatGPT-4.0，这表明两个版本之间存在信息处理和回复生成的可能差异。这些发现提供了关于 AI 辅助门诊运作的见解，同时也促进了对 LLM 在医疗利用中的潜力和限制的探索。未来的研究可能会根据人体工程学和人因原则，精确地根据有效门诊分诊的特定需求，谨慎优化 LLM 和 AI 在医疗系统中的整合。