Large Language Models (LLMs) have achieved remarkable progress, yet their
application in specialized fields, such as medical physics, remains challenging
due to the need for domain-specific knowledge. This study introduces ARCoT
(Adaptable Retrieval-based Chain of Thought), a framework designed to enhance
the domain-specific accuracy of LLMs without requiring fine-tuning or extensive
retraining. ARCoT integrates a retrieval mechanism to access relevant
domain-specific information and employs step-back and chain-of-thought
prompting techniques to guide the LLM's reasoning process, ensuring more
accurate and context-aware responses. Benchmarking on a medical physics
multiple-choice exam, our model outperformed standard LLMs and reported average
human performance, demonstrating improvements of up to 68% and achieving a high
score of 90%. This method reduces hallucinations and increases domain-specific
performance. The versatility and model-agnostic nature of ARCoT make it easily
adaptable to various domains, showcasing its significant potential for
enhancing the accuracy and reliability of LLMs in specialized fields.

ARCoT 是一种专为提高大型语言模型 LLMs 在专业领域（如医学物理学）中的领域特定准确性而设计的框架，通过检索机制获取相关的领域特定信息，并采用回退和思维链提示技术来引导 LLM 的推理过程，确保更准确和上下文感知的回答。在医学物理学多项选择考试的基准测试中，我们的模型优于标准 LLMs 并报道了平均人类表现，实现了高达 90％的高分，改进了多达 68％。ARCoT 的通用性和模型不可知性使其能够轻松适应各个领域，展示了在专门领域中提高 LLMs 准确性和可靠性的重要潜力。