Access to real-world medical instructions is essential for medical research
and healthcare quality improvement. However, access to real medical
instructions is often limited due to the sensitive nature of the information
expressed. Additionally, manually labelling these instructions for training and
fine-tuning Natural Language Processing (NLP) models can be tedious and
expensive. We introduce a novel task-specific model architecture,
Label-To-Text-Transformer (\textbf{LT3}), tailored to generate synthetic
medical instructions based on provided labels, such as a vocabulary list of
medications and their attributes. LT3 is trained on a vast corpus of medical
instructions extracted from the MIMIC-III database, allowing the model to
produce valuable synthetic medical instructions. We evaluate LT3's performance
by contrasting it with a state-of-the-art Pre-trained Language Model (PLM), T5,
analysing the quality and diversity of generated texts. We deploy the generated
synthetic data to train the SpacyNER model for the Named Entity Recognition
(NER) task over the n2c2-2018 dataset. The experiments show that the model
trained on synthetic data can achieve a 96-98\% F1 score at Label Recognition
on Drug, Frequency, Route, Strength, and Form. LT3 codes and data will be
shared at https://github.com/HECTA-UoM/Label-To-Text-Transformer

通过提供标签（例如药物词汇表和属性），我们介绍了一种新的任务特定模型体系结构 Label-To-Text-Transformer（LT3），专门用于生成基于提供的标签的合成医疗指令。我们通过与最先进的预训练语言模型（PLM）T5 进行对比，评估了 LT3 的性能，并分析了生成文本的质量和多样性。我们使用生成的合成数据对 SpacyNER 模型进行了训练，用于在 n2c2-2018 数据集上进行命名实体识别（NER）任务。实验结果表明，使用合成数据训练的模型在药物、频率、途径、强度和形式的标签识别上可以达到 96-98％的 F1 分数。

使用条件 Transformer 生成医疗指导

Generating Medical Instructions with Conditional Transformer

Recently, prompt-based learning has become a very popular solution in many
Natural Language Processing (NLP) tasks by inserting a template into model
input, which converts the task into a cloze-style one to smoothing out
differences between the Pre-trained Language Model (PLM) and the current task.
But in the case of relation classification, it is difficult to map the masked
output to the relation labels because of its abundant semantic information,
e.g. org:founded_by''. Therefore, a pre-trained model still needs enough
labelled data to fit the relations. To mitigate this challenge, in this paper,
we present a novel prompt-based learning method, namely LabelPrompt, for the
relation classification task. It is an extraordinary intuitive approach by a
motivation: ``GIVE MODEL CHOICES!''. First, we define some additional tokens to
represent the relation labels, which regards these tokens as the verbalizer
with semantic initialisation and constructs them with a prompt template method.
Then we revisit the inconsistency of the predicted relation and the given
entities, an entity-aware module with the thought of contrastive learning is
designed to mitigate the problem. At last, we apply an attention query strategy
to self-attention layers to resolve two types of tokens, prompt tokens and
sequence tokens. The proposed strategy effectively improves the adaptation
capability of prompt-based learning in the relation classification task when
only a small labelled data is available. Extensive experimental results
obtained on several bench-marking datasets demonstrate the superiority of the
proposed LabelPrompt method, particularly in the few-shot scenario.

本文提出了一种新的基于提示的学习方法 LabelPrompt，用于关系分类任务，通过定义附加令牌来表示关系标签，并构建提示模板方法，以及设计一个实体感知模块和注意查询策略来解决模型中的一些挑战，极大地提高了在缺乏标注数据情况下利用提示学习的适应能力，特别是在少样本场景中的表现超过了其他几种方法。