Irregular sampling of time series in electronic health records (EHRs) is one
of the main challenges for developing machine learning models. Additionally,
the pattern of missing data in certain clinical variables is not at random but
depends on the decisions of clinicians and the state of the patient. Point
process is a mathematical framework for analyzing event sequence data that is
consistent with irregular sampling patterns. Our model, TEE4EHR, is a
transformer event encoder (TEE) with point process loss that encodes the
pattern of laboratory tests in EHRs. The utility of our TEE has been
investigated in a variety of benchmark event sequence datasets. Additionally,
we conduct experiments on two real-world EHR databases to provide a more
comprehensive evaluation of our model. Firstly, in a self-supervised learning
approach, the TEE is jointly learned with an existing attention-based deep
neural network which gives superior performance in negative log-likelihood and
future event prediction. Besides, we propose an algorithm for aggregating
attention weights that can reveal the interaction between the events. Secondly,
we transfer and freeze the learned TEE to the downstream task for the outcome
prediction, where it outperforms state-of-the-art models for handling
irregularly sampled time series. Furthermore, our results demonstrate that our
approach can improve representation learning in EHRs and can be useful for
clinical prediction tasks.

TEE4EHR 是一个具有点过程损失的 Transformer 事件编码器模型，用于分析电子健康记录中实验室检验模式，通过在各种基准的事件序列数据集上验证其实用性，并在两个真实的电子健康记录数据库上进行实验证明了其能够改善电子健康记录中的表示学习并在临床预测任务中提供帮助。