In research areas with scarce data, representation learning plays a
significant role. This work aims to enhance representation learning for
clinical time series by deriving universal embeddings for clinical features,
such as heart rate and blood pressure. We use self-supervised training
paradigms for language models to learn high-quality clinical feature
embeddings, achieving a finer granularity than existing time-step and
patient-level representation learning. We visualize the learnt embeddings via
unsupervised dimension reduction techniques and observe a high degree of
consistency with prior clinical knowledge. We also evaluate the model
performance on the MIMIC-III benchmark and demonstrate the effectiveness of
using clinical feature embeddings. We publish our code online for replication.

通过自我监督训练模型来学习高质量的临床特征嵌入表示，以增强临床时间序列的表示学习能力，并通过无监督降维技术可视化学习到的嵌入表示，发现其与先前的临床知识具有高度一致性；同时在 MIMIC-III 基准数据上评估模型性能，并证明了使用临床特征嵌入的有效性。

临床特征嵌入的语言模型训练范式

Language Model Training Paradigms for Clinical Feature Embeddings

We introduce a novel contrastive representation learning objective and a
training scheme for clinical time series. Specifically, we project high
dimensional EHR. data to a closed unit ball of low dimension, encoding
geometric priors so that the origin represents an idealized perfect health
state and the Euclidean norm is associated with the patient's mortality risk.
Moreover, using septic patients as an example, we show how we could learn to
associate the angle between two vectors with the different organ system
failures, thereby, learning a compact representation which is indicative of
both mortality risk and specific organ failure. We show how the learned
embedding can be used for online patient monitoring, can supplement clinicians
and improve performance of downstream machine learning tasks. This work was
partially motivated from the desire and the need to introduce a systematic way
of defining intermediate rewards for Reinforcement Learning in critical care
medicine. Hence, we also show how such a design in terms of the learned
embedding can result in qualitatively different policies and value
distributions, as compared with using only terminal rewards.

本文介绍了一种新的对比表示学习目标和训练方案，针对临床时间序列。该方法能将高维的医疗记录数据投影到低维的封闭单位球上，以编码几何先验，从而学习患者的死亡风险和器官功能，该方法可用于在线患者监测，并可提供临床医生的决策支持和改进下游机器学习任务的性能。