Large Language Models (LLMs) have demonstrated their efficacy across a broad
spectrum of tasks in healthcare applications. However, often LLMs need to be
fine-tuned on task-specific expert annotated data to achieve optimal
performance, which can be expensive and time consuming. In this study, we
fine-tune PaLM-2 with parameter efficient fine-tuning (PEFT) using noisy labels
obtained from gemini-pro 1.0 for the detection of Schedule-of-Event (SoE)
tables, which specify care plan in clinical trial protocols. We introduce a
filtering mechanism to select high-confidence labels for this table
classification task, thereby reducing the noise in the auto-generated labels.
We show that fine-tuned PaLM-2 with those labels achieves performance that
exceeds the gemini-pro 1.0 and other LLMs. Furthermore, its performance is
close to a PaLM-2 fine-tuned on labels obtained from non-expert annotators. Our
results show that leveraging LLM-generated labels through powerful models like
gemini-pro can potentially serve as a viable strategy for improving LLM
performance through fine-tuning in specialized tasks, particularly in domains
where expert annotations are scarce, expensive, or time-consuming to obtain.

通过使用 gemini-pro 1.0 生成的噪声标签，使用参数高效的 fine-tuning（PEFT）对 PaLM-2 进行微调，减少自动生成标签的噪声，从而在临床试验方案的 Schedule-of-Event 表的检测中，在超过 gemini-pro 1.0 和其他 LLMs 的性能的基础上，接近于使用非专家注释者获得的标签对 PaLM-2 进行微调的性能。

利用 LLM 标记数据进行有选择的微调可能减少对人工注释的依赖：以事件表检测为例的案例研究

Selective Fine-tuning on LLM-labeled Data May Reduce Reliance on Human  Annotation: A Case Study Using Schedule-of-Event Table Detection

Clinical trials require numerous documents to be written -- protocols,
consent forms, clinical study reports and others. Large language models (LLMs)
offer the potential to rapidly generate first versions of these documents,
however there are concerns about the quality of their output Here we report an
evaluation of LLMs in generating parts of one such document, clinical trial
protocols. We find that an offthe-shelf LLM delivers reasonable results,
especially when assessing content relevance and the correct use of terminology.
However, deficiencies remain: specifically clinical thinking and logic, and
appropriate use of references. To improve performance, we used
retrieval-augmented generation (RAG) to prompt an LLM with accurate up-to-date
information. As a result of using RAG, the writing quality of the LLM improves
substantially, which has implications for the practical useability of LLMs in
clinical trial-related writing.

在临床试验中，通过使用具备内容相关性和正确术语使用的合理结果的现成大型语言模型（LLMs），然而存在于临床思维、逻辑和合适引用的不足，通过使用 RAG（检索增强生成法）提升 LLM 的写作质量，从而提高了 LLMs 在临床试验相关文案中的实际可用性。

从 RAG 到富裕：利用大型语言模型编写临床试验文件

From RAGs to riches: Using large language models to write documents for  clinical trials

We consider the hierarchical representation of documents as graphs and use
geometric deep learning to classify them into different categories. While graph
neural networks can efficiently handle the variable structure of hierarchical
documents using the permutation invariant message passing operations, we show
that we can gain extra performance improvements using our proposed selective
graph pooling operation that arises from the fact that some parts of the
hierarchy are invariable across different documents. We applied our model to
classify clinical trial (CT) protocols into completed and terminated
categories. We use bag-of-words based, as well as pre-trained transformer-based
embeddings to featurize the graph nodes, achieving f1-scores around 0.85 on a
publicly available large scale CT registry of around 360K protocols. We further
demonstrate how the selective pooling can add insights into the CT termination
status prediction. We make the source code and dataset splits accessible.

使用几何深度学习方法将临床试验文件表示为图形，使用选择性图形汇集方法提高了分类准确性，最终在一个包含 360K 份临床试验文件的公共数据库上实现了 0.85 的 F1 分数。