This study introduces a novel knowledge enhanced tokenisation mechanism,
K-Tokeniser, for clinical text processing. Technically, at initialisation
stage, K-Tokeniser populates global representations of tokens based on semantic
types of domain concepts (such as drugs or diseases) from either a domain
ontology like Unified Medical Language System or the training data of the task
related corpus. At training or inference stage, sentence level localised
context will be utilised for choosing the optimal global token representation
to realise the semantic-based tokenisation. To avoid pretraining using the new
tokeniser, an embedding initialisation approach is proposed to generate
representations for new tokens. Using three transformer-based language models,
a comprehensive set of experiments are conducted on four real-world datasets
for evaluating K-Tokeniser in a wide range of clinical text analytics tasks
including clinical concept and relation extraction, automated clinical coding,
clinical phenotype identification, and clinical research article
classification. Overall, our models demonstrate consistent improvements over
their counterparts in all tasks. In particular, substantial improvements are
observed in the automated clinical coding task with 13\% increase on Micro
$F_1$ score. Furthermore, K-Tokeniser also shows significant capacities in
facilitating quicker converge of language models. Specifically, using
K-Tokeniser, the language models would only require 50\% of the training data
to achieve the best performance of the baseline tokeniser using all training
data in the concept extraction task and less than 20\% of the data for the
automated coding task. It is worth mentioning that all these improvements
require no pre-training process, making the approach generalisable.

本研究介绍了一种新的知识增强的令牌化机制 K-Tokeniser，用于临床文本处理。通过使用语义类型来生成全局表示，并利用句子级上下文选择最佳的全局令牌表示，K-Tokeniser 在临床文本分析任务中展示了一致的改进，特别是在自动临床编码任务中取得了 13% 的微平均 $F_1$ 分数提升，同时对于语言模型的训练也有显著加速作用。