In the ongoing wave of impact driven by large language models (LLMs) like
ChatGPT, the adaptation of LLMs to medical domain has emerged as a crucial
research frontier. Since mainstream LLMs tend to be designed for
general-purpose applications, constructing a medical LLM through domain
adaptation is a huge challenge. While instruction-tuning is used to fine-tune
some LLMs, its precise roles in domain adaptation remain unknown. Here we show
the contribution of LoRA-based instruction-tuning to performance in Japanese
medical question-answering tasks. In doing so, we employ a multifaceted
evaluation for multiple-choice questions, including scoring based on "Exact
match" and "Gestalt distance" in addition to the conventional accuracy. Our
findings suggest that LoRA-based instruction-tuning can partially incorporate
domain-specific knowledge into LLMs, with larger models demonstrating more
pronounced effects. Furthermore, our results underscore the potential of
adapting English-centric models for Japanese applications in domain adaptation,
while also highlighting the persisting limitations of Japanese-centric models.
This initiative represents a pioneering effort in enabling medical institutions
to fine-tune and operate models without relying on external services.

通过 LoRA-based instruction-tuning 来提高日本医学问答任务的性能，在多项选择题的评估中发现其可以部分地将领域特定知识整合到大型语言模型中，较大模型效果更显著，并突出了将英语为中心的模型适应于日本应用的潜力，同时也强调了日本为中心的模型的局限性，这一举措标志着在不依赖外部服务的情况下，医疗机构可以对模型进行精细调整和操作的先驱性努力。