Instruction-tuned generative Large language models (LLMs) like ChatGPT and
Bloomz possess excellent generalization abilities, but they face limitations in
understanding radiology reports, particularly in the task of generating the
IMPRESSIONS section from the FINDINGS section. They tend to generate either
verbose or incomplete IMPRESSIONS, mainly due to insufficient exposure to
medical text data during training. We present a system which leverages
large-scale medical text data for domain-adaptive pre-training of
instruction-tuned LLMs to enhance its medical knowledge and performance on
specific medical tasks. We show that this system performs better in a zero-shot
setting than a number of pretrain-and-finetune adaptation methods on the
IMPRESSIONS generation task, and ranks 1st among participating systems in Task
1B: Radiology Report Summarization at the BioNLP 2023 workshop.

本文提出了一个使用大规模医疗文本数据对指令调整的生成大语言模型进行领域自适应预训练的系统，以增强其医疗知识和特定任务的表现。该系统在零样本情况下在 IMPRESSIONS 生成任务上表现比其他预训练和微调方法更好，并在 BioNLP 2023 研讨会的 Radiology Report Summarization Task 1B 中排名第一。

RadSum23 的 shs-nlp：面向领域自适应的指令调整 LLMs 预训练，用于放射学报告印象生成

shs-nlp at RadSum23: Domain-Adaptive Pre-training of Instruction-tuned  LLMs for Radiology Report Impression Generation

The digitization of healthcare has facilitated the sharing and re-using of
medical data but has also raised concerns about confidentiality and privacy.
HIPAA (Health Insurance Portability and Accountability Act) mandates removing
re-identifying information before the dissemination of medical records. Thus,
effective and efficient solutions for de-identifying medical data, especially
those in free-text forms, are highly needed. While various computer-assisted
de-identification methods, including both rule-based and learning-based, have
been developed and used in prior practice, such solutions still lack
generalizability or need to be fine-tuned according to different scenarios,
significantly imposing restrictions in wider use. The advancement of large
language models (LLM), such as ChatGPT and GPT-4, have shown great potential in
processing text data in the medical domain with zero-shot in-context learning,
especially in the task of privacy protection, as these models can identify
confidential information by their powerful named entity recognition (NER)
capability. In this work, we developed a novel GPT4-enabled de-identification
framework ("DeID-GPT") to automatically identify and remove the identifying
information. Compared to existing commonly used medical text data
de-identification methods, our developed DeID-GPT showed the highest accuracy
and remarkable reliability in masking private information from the unstructured
medical text while preserving the original structure and meaning of the text.
This study is one of the earliest to utilize ChatGPT and GPT-4 for medical text
data processing and de-identification, which provides insights for further
research and solution development on the use of LLMs such as ChatGPT/GPT-4 in
healthcare. Codes and benchmarking data information are available at
this https URL.

该研究使用 GPT-4 大型语言模型开发了一种名为 DeID-GPT 的新的医学文本数据去识别框架，通过在医学领域使用零 - shot 上下文学习识别模型保护隐私信息，并保留文本的原始结构和含义，比起现有方法 DeID-GPT 的准确性最高，可在更广泛的情境下使用。