Influenced by ChatGPT, artificial intelligence (AI) large models have
witnessed a global upsurge in large model research and development. As people
enjoy the convenience by this AI large model, more and more large models in
subdivided fields are gradually being proposed, especially large models in
radiology imaging field. This article first introduces the development history
of large models, technical details, workflow, working principles of multimodal
large models and working principles of video generation large models. Secondly,
we summarize the latest research progress of AI large models in radiology
education, radiology report generation, applications of unimodal and multimodal
radiology. Finally, this paper also summarizes some of the challenges of large
AI models in radiology, with the aim of better promoting the rapid revolution
in the field of radiography.

受 ChatGPT 的影响，人工智能（AI）大型模型在全球范围内迎来了大规模的研究和开发。本文首先介绍了大型模型的发展历史、技术细节、工作流程以及多模态大型模型和视频生成大型模型的工作原理。其次，我们总结了 AI 大型模型在放射学教育、放射学报告生成、单模态和多模态放射学应用方面的最新研究进展。最后，本文还总结了放射学领域中大型 AI 模型面临的一些挑战，旨在更好地推动该领域的快速革命。

大规模人工智能模型在放射学应用中的机遇与挑战

Opportunities and challenges in the application of large artificial  intelligence models in radiology

The scaling laws and extraordinary performance of large foundation models
motivate the development and utilization of such large models in biomedicine.
However, despite early promising results on some biomedical benchmarks, there
are still major challenges that need to be addressed before these models can be
used in real-world applications. Frontier models such as GPT-4V still have
major competency gaps in multimodal capabilities for biomedical applications.
Moreover, pragmatic issues such as access, cost, latency, and compliance make
it hard for clinicians to use privately-hosted state-of-the-art large models
directly on private patient data. In this paper, we explore training
open-source small multimodal models (SMMs) to bridge biomedical competency gaps
for unmet clinical needs. To maximize data efficiency, we adopt a modular
approach by incorporating state-of-the-art pre-trained models for image and
text modalities, and focusing on training a lightweight adapter to ground each
modality to the text embedding space. We conduct a comprehensive study of this
approach on radiology imaging. For training, we assemble a large dataset with
over 1 million image-text pairs. For evaluation, we propose a clinically driven
novel approach using GPT-4 and demonstrate its parity with expert evaluation.
We also study grounding qualitatively using attention. For best practice, we
conduct a systematic ablation study on various choices in data engineering and
multimodal training. The resulting LLaVA-Rad (7B) model attains
state-of-the-art results on radiology tasks such as report generation and
cross-modal retrieval, even outperforming much larger models such as GPT-4V and
Med-PaLM M (84B). LLaVA-Rad is fast and can be run on a single V100 GPU in
private settings, offering a promising state-of-the-art tool for real-world
clinical applications.

利用模块化方法针对医学领域使用开源小型多模态模型来解决大规模基础模型在临床需求中存在的问题，并在放射学成像中展示了 LLaVA-Rad 模型的最新结果和其在报告生成和跨模态检索中的性能优势，成为真实世界临床应用的前景工具。