Background: Structured radiology reports remains underdeveloped due to
labor-intensive structuring and narrative-style reporting. Deep learning,
particularly large language models (LLMs) like GPT-3.5, offers promise in
automating the structuring of radiology reports in natural languages. However,
although it has been reported that LLMs are less effective in languages other
than English, their radiological performance has not been extensively studied.
Purpose: This study aimed to investigate the accuracy of TNM classification
based on radiology reports using GPT3.5-turbo (GPT3.5) and the utility of
multilingual LLMs in both Japanese and English. Material and Methods: Utilizing
GPT3.5, we developed a system to automatically generate TNM classifications
from chest CT reports for lung cancer and evaluate its performance. We
statistically analyzed the impact of providing full or partial TNM definitions
in both languages using a Generalized Linear Mixed Model. Results: Highest
accuracy was attained with full TNM definitions and radiology reports in
English (M = 94%, N = 80%, T = 47%, and ALL = 36%). Providing definitions for
each of the T, N, and M factors statistically improved their respective
accuracies (T: odds ratio (OR) = 2.35, p < 0.001; N: OR = 1.94, p < 0.01; M: OR
= 2.50, p < 0.001). Japanese reports exhibited decreased N and M accuracies (N
accuracy: OR = 0.74 and M accuracy: OR = 0.21). Conclusion: This study
underscores the potential of multilingual LLMs for automatic TNM classification
in radiology reports. Even without additional model training, performance
improvements were evident with the provided TNM definitions, indicating LLMs'
relevance in radiology contexts.

本研究旨在调查利用 GPT3.5-turbo（GPT3.5）进行放射学报告的 TNM 分级的准确性及多语种 LLMs 在日语和英语中的实用性。通过分析多语言 TNM 定义对准确性的影响，研究表明提供完整的 TNM 定义可明显提高放射学报告的分类准确性，且多语种 LLMs 在放射学领域具有潜在应用价值。