In research areas with scarce data, representation learning plays a
significant role. This work aims to enhance representation learning for
clinical time series by deriving universal embeddings for clinical features,
such as heart rate and blood pressure. We use self-supervised training
paradigms for language models to learn high-quality clinical feature
embeddings, achieving a finer granularity than existing time-step and
patient-level representation learning. We visualize the learnt embeddings via
unsupervised dimension reduction techniques and observe a high degree of
consistency with prior clinical knowledge. We also evaluate the model
performance on the MIMIC-III benchmark and demonstrate the effectiveness of
using clinical feature embeddings. We publish our code online for replication.

通过自我监督训练模型来学习高质量的临床特征嵌入表示，以增强临床时间序列的表示学习能力，并通过无监督降维技术可视化学习到的嵌入表示，发现其与先前的临床知识具有高度一致性；同时在 MIMIC-III 基准数据上评估模型性能，并证明了使用临床特征嵌入的有效性。

临床特征嵌入的语言模型训练范式

Language Model Training Paradigms for Clinical Feature Embeddings

A variety of knowledge graph embedding approaches have been developed. Most
of them obtain embeddings by learning the structure of the knowledge graph
within a link prediction setting. As a result, the embeddings reflect only the
semantics of a single knowledge graph, and embeddings for different knowledge
graphs are not aligned, e.g., they cannot be used to find similar entities
across knowledge graphs via nearest neighbor search. However, knowledge graph
embedding applications such as entity disambiguation require a more global
representation, i.e., a representation that is valid across multiple sources.
We propose to learn universal knowledge graph embeddings from large-scale
interlinked knowledge sources. To this end, we fuse large knowledge graphs
based on the owl:sameAs relation such that every entity is represented by a
unique identity. We instantiate our idea by computing universal embeddings
based on DBpedia and Wikidata yielding embeddings for about 180 million
entities, 15 thousand relations, and 1.2 billion triples. Moreover, we develop
a convenient API to provide embeddings as a service. Experiments on link
prediction show that universal knowledge graph embeddings encode better
semantics compared to embeddings computed on a single knowledge graph. For
reproducibility purposes, we provide our source code and datasets open access
at this https URL

从大规模互连的知识源中融合大型知识图谱，以学习通用知识图谱嵌入向量，使得这些嵌入能够在多个知识图谱中进行语义搜索和实体对齐，从而提高链接预测的语义表示能力。