The extraction of lung lesion information from clinical and medical imaging
reports is crucial for research on and clinical care of lung-related diseases.
Large language models (LLMs) can be effective at interpreting unstructured text
in reports, but they often hallucinate due to a lack of domain-specific
knowledge, leading to reduced accuracy and posing challenges for use in
clinical settings. To address this, we propose a novel framework that aligns
generated internal knowledge with external knowledge through in-context
learning (ICL). Our framework employs a retriever to identify relevant units of
internal or external knowledge and a grader to evaluate the truthfulness and
helpfulness of the retrieved internal-knowledge rules, to align and update the
knowledge bases. Our knowledge-conditioned approach also improves the accuracy
and reliability of LLM outputs by addressing the extraction task in two stages:
(i) lung lesion finding detection and primary structured field parsing,
followed by (ii) further parsing of lesion description text into additional
structured fields. Experiments with expert-curated test datasets demonstrate
that this ICL approach can increase the F1 score for key fields (lesion size,
margin and solidity) by an average of 12.9% over existing ICL methods.

通过内外知识的上下文学习，我们提出了一种新的框架来解决由于缺乏领域特定知识而导致的大型语言模型生成虚构信息的问题，并通过两个阶段的任务处理来提高病灶信息提取的准确性和可靠性。

基于知识条件化 LLMs 的自动临床数据提取

Automated Clinical Data Extraction with Knowledge Conditioned LLMs

The integration of AI into radiology introduces opportunities for improved
clinical care provision and efficiency but it demands a meticulous approach to
mitigate potential risks as with any other new technology. Beginning with
rigorous pre-deployment evaluation and validation, the focus should be on
ensuring models meet the highest standards of safety, effectiveness and
efficacy for their intended applications. Input and output guardrails
implemented during production usage act as an additional layer of protection,
identifying and addressing individual failures as they occur. Continuous
post-deployment monitoring allows for tracking population-level performance
(data drift), fairness, and value delivery over time. Scheduling reviews of
post-deployment model performance and educating radiologists about new
algorithmic-driven findings is critical for AI to be effective in clinical
practice. Recognizing that no single AI solution can provide absolute assurance
even when limited to its intended use, the synergistic application of quality
assurance at multiple levels - regulatory, clinical, technical, and ethical -
is emphasized. Collaborative efforts between stakeholders spanning healthcare
systems, industry, academia, and government are imperative to address the
multifaceted challenges involved. Trust in AI is an earned privilege,
contingent on a broad set of goals, among them transparently demonstrating that
the AI adheres to the same rigorous safety, effectiveness and efficacy
standards as other established medical technologies. By doing so, developers
can instil confidence among providers and patients alike, enabling the
responsible scaling of AI and the realization of its potential benefits. The
roadmap presented herein aims to expedite the achievement of deployable,
reliable, and safe AI in radiology.

通过认真评估和验证以确保 AI 模型达到最高的安全、有效性和功效标准，上线前以及生产使用中实施输入和输出保护措施、持续监测并追踪人群水平性能、公正性和结果传递、定期审查上线后的模型性能、教育放射科医生关于新的算法驱动发现对于 AI 在临床实践中的有效应用至关重要，强调在多个层面上合作的质量保证是重要的，从立法、临床、技术和伦理等方面，通过透明地展示 AI 遵守与其他已建立的医疗技术相同的严格的安全、有效性和功效标准，树立开发者、提供者和患者等的信心，使 AI 可靠扩展并实现其潜在的益处的愿景，本论文提出的路线图旨在加快 AI 在放射学中可部署、可靠和安全的实现。