This study introduces a novel knowledge enhanced tokenisation mechanism,
K-Tokeniser, for clinical text processing. Technically, at initialisation
stage, K-Tokeniser populates global representations of tokens based on semantic
types of domain concepts (such as drugs or diseases) from either a domain
ontology like Unified Medical Language System or the training data of the task
related corpus. At training or inference stage, sentence level localised
context will be utilised for choosing the optimal global token representation
to realise the semantic-based tokenisation. To avoid pretraining using the new
tokeniser, an embedding initialisation approach is proposed to generate
representations for new tokens. Using three transformer-based language models,
a comprehensive set of experiments are conducted on four real-world datasets
for evaluating K-Tokeniser in a wide range of clinical text analytics tasks
including clinical concept and relation extraction, automated clinical coding,
clinical phenotype identification, and clinical research article
classification. Overall, our models demonstrate consistent improvements over
their counterparts in all tasks. In particular, substantial improvements are
observed in the automated clinical coding task with 13\% increase on Micro
$F_1$ score. Furthermore, K-Tokeniser also shows significant capacities in
facilitating quicker converge of language models. Specifically, using
K-Tokeniser, the language models would only require 50\% of the training data
to achieve the best performance of the baseline tokeniser using all training
data in the concept extraction task and less than 20\% of the data for the
automated coding task. It is worth mentioning that all these improvements
require no pre-training process, making the approach generalisable.

本研究介绍了一种新的知识增强的令牌化机制 K-Tokeniser，用于临床文本处理。通过使用语义类型来生成全局表示，并利用句子级上下文选择最佳的全局令牌表示，K-Tokeniser 在临床文本分析任务中展示了一致的改进，特别是在自动临床编码任务中取得了 13% 的微平均 $F_1$ 分数提升，同时对于语言模型的训练也有显著加速作用。

将临床知识融入语言模型的标记器

Infusing clinical knowledge into tokenisers for language models

Clinical text processing has gained more and more attention in recent years.
The access to sensitive patient data, on the other hand, is still a big
challenge, as text cannot be shared without legal hurdles and without removing
personal information. There are many techniques to modify or remove patient
related information, each with different strengths. This paper investigates the
influence of different anonymization techniques on the performance of ML models
using multiple datasets corresponding to five different NLP tasks. Several
learnings and recommendations are presented. This work confirms that
particularly stronger anonymization techniques lead to a significant drop of
performance. In addition to that, most of the presented techniques are not
secure against a re-identification attack based on similarity search.

探究匿名化技术对多种 NLP 任务中 ML 模型性能的影响，旨在寻找更好的匿名化技术，同时发现一些技术并不能有效避免由相似性搜索发起的重新识别攻击。

临床文本处理中哪种匿名技术最好？──这要看任务。一项系统研究

Which anonymization technique is best for which NLP task? -- It depends. A Systematic Study on Clinical Text Processing

Background: In the information extraction and natural language processing
domain, accessible datasets are crucial to reproduce and compare results.
Publicly available implementations and tools can serve as benchmark and
facilitate the development of more complex applications. However, in the
context of clinical text processing the number of accessible datasets is scarce
-- and so is the number of existing tools. One of the main reasons is the
sensitivity of the data. This problem is even more evident for non-English
languages.
Approach: In order to address this situation, we introduce a workbench: a
collection of German clinical text processing models. The models are trained on
a de-identified corpus of German nephrology reports.
Result: The presented models provide promising results on in-domain data.
Moreover, we show that our models can be also successfully applied to other
biomedical text in German. Our workbench is made publicly available so it can
be used out of the box, as a benchmark or transferred to related problems.

为了解决缺乏德语临床文本处理数据集和工具的问题，我们介绍了一个包含临床文本处理模型的工作台，这些模型是使用一个去身份化的德语肾病学报告语料库进行训练的，旨在作为基准或应用于相关问题。