Automatic diagnosis is a significant application of AI in healthcare, where
diagnoses are generated based on the symptom description of patients. Previous
works have approached this task directly by modeling the relationship between
the normalized symptoms and all possible diseases. However, in the clinical
diagnostic process, patients are initially consulted by a general practitioner
and, if necessary, referred to specialists in specific domains for a more
comprehensive evaluation. The final diagnosis often emerges from a
collaborative consultation among medical specialist groups. Recently, large
language models have shown impressive capabilities in natural language
understanding. In this study, we adopt tuning-free LLM-based agents as medical
practitioners and propose the Agent-derived Multi-Specialist Consultation
(AMSC) framework to model the diagnosis process in the real world by adaptively
fusing probability distributions of agents over potential diseases.
Experimental results demonstrate the superiority of our approach compared with
baselines. Notably, our approach requires significantly less parameter updating
and training time, enhancing efficiency and practical utility. Furthermore, we
delve into a novel perspective on the role of implicit symptoms within the
context of automatic diagnosis.

采用调谐自由的 LLM（大语言模型）代理作为医学从业者，我们提出了基于代理的多专科咨询（AMSC）框架，通过自适应地融合代理对潜在疾病的概率分布来模拟真实世界中的诊断过程。实验结果显示了我们的方法相比基线的优越性。值得注意的是，我们的方法需要极少的参数更新和训练时间，提高了效率和实用性。此外，我们深入探讨了在自动诊断背景下的隐性症状的新角度。

超越直接诊断：基于 LLM 的多专家代理咨询自动诊断

Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation  for Automatic Diagnosis

In clinical scenarios, multi-specialist consultation could significantly
benefit the diagnosis, especially for intricate cases. This inspires us to
explore a "multi-expert joint diagnosis" mechanism to upgrade the existing
"single expert" framework commonly seen in the current literature. To this end,
we propose METransformer, a method to realize this idea with a
transformer-based backbone. The key design of our method is the introduction of
multiple learnable "expert" tokens into both the transformer encoder and
decoder. In the encoder, each expert token interacts with both vision tokens
and other expert tokens to learn to attend different image regions for image
representation. These expert tokens are encouraged to capture complementary
information by an orthogonal loss that minimizes their overlap. In the decoder,
each attended expert token guides the cross-attention between input words and
visual tokens, thus influencing the generated report. A metrics-based expert
voting strategy is further developed to generate the final report. By the
multi-experts concept, our model enjoys the merits of an ensemble-based
approach but through a manner that is computationally more efficient and
supports more sophisticated interactions among experts. Experimental results
demonstrate the promising performance of our proposed model on two widely used
benchmarks. Last but not least, the framework-level innovation makes our work
ready to incorporate advances on existing "single-expert" models to further
improve its performance.

提出了 METransformer，一种基于 transformer 的方法，利用多个可学习的 “专家” 令牌在编码器和解码器之间实现交叉注意力以实现 “多专家联合诊断”。实验结果表明，该模型在两个广泛使用的基准上的性能表现良好。