We investigate different natural language processing (NLP) approaches based
on contextualised word representations for the problem of early prediction of
lung cancer using free-text patient medical notes of Dutch primary care
physicians. Because lung cancer has a low prevalence in primary care, we also
address the problem of classification under highly imbalanced classes.
Specifically, we use large Transformer-based pretrained language models (PLMs)
and investigate: 1) how \textit{soft prompt-tuning} -- an NLP technique used to
adapt PLMs using small amounts of training data -- compares to standard model
fine-tuning; 2) whether simpler static word embedding models (WEMs) can be more
robust compared to PLMs in highly imbalanced settings; and 3) how models fare
when trained on notes from a small number of patients. We find that 1)
soft-prompt tuning is an efficient alternative to standard model fine-tuning;
2) PLMs show better discrimination but worse calibration compared to simpler
static word embedding models as the classification problem becomes more
imbalanced; and 3) results when training models on small number of patients are
mixed and show no clear differences between PLMs and WEMs. All our code is
available open source in
https://bitbucket.org/aumc-kik/prompt_tuning_cancer_prediction/.

使用基于上下文的词表示法的自然语言处理方法，研究利用荷兰初级保健医生的患者医疗笔记进行早期肺癌预测的问题。使用大型 Transformer 预训练语言模型，调查软提示调整与传统模型微调，WEMs 与 PLMs 的比较，以及在少数患者的笔记上训练模型的效果，发现软提示调整是标准模型微调的有效替代品，PLMs 比 WEMs 表现更好，但当分类问题变得更加不平衡时更容易过拟合，而在小数量的患者笔记上训练模型的结果表现良好。

使用初级保健免费文本荷兰医疗笔记进行软提示调整以预测肺癌

Soft-prompt tuning to predict lung cancer using primary care free-text Dutch medical notes

This paper presents the first unsupervised approach to lexical semantic
change that makes use of contextualised word representations. We propose a
novel method that exploits the BERT neural language model to obtain
representations of word usages, clusters these representations into usage
types, and measures change along time with three proposed metrics. We create a
new evaluation dataset and show that the model representations and the detected
semantic shifts are positively correlated with human judgements. Our extensive
qualitative analysis demonstrates that our method captures a variety of
synchronic and diachronic linguistic phenomena. We expect our work to inspire
further research in this direction.

本文提出了第一种使用上下文词表示进行无监督词汇语义变化的方法。利用 BERT 神经语言模型来获取单词用法的表示，将这些表示聚类到用法类型，并用三种提出的度量方式衡量关于时间的变化。创造了一个新的评估数据集，并表明模型表示和检测到的语义转移与人类判断之间呈正相关。 extensive 的定性分析表明，我们的方法捕捉了各种同步和历时语言现象。我们期望我们的工作将激发进一步的研究。