World knowledge exists in both structured (tables, knowledge graphs) and
unstructured forms (texts). Recently, there have been extensive research
efforts in the integration of structured factual knowledge and unstructured
textual knowledge. However, most studies focus on incorporating static factual
knowledge into pre-trained language models, while there is less work on
enhancing temporal knowledge graph embedding using textual knowledge. Existing
integration approaches can not apply to temporal knowledge graphs (tKGs) since
they often assume knowledge embedding is time-invariant. In fact, the entity
embedding in tKG embedding models usually evolves over time, which poses the
challenge of aligning temporally relevant textual information with entities. To
this end, we propose Enhanced Temporal Knowledge Embeddings with Contextualized
Language Representations (ECOLA), which uses tKG quadruple as an implicit
measure to temporally align textual data and the time-evolving entity
representations and uses a novel knowledge-text prediction task to inject
textual information into temporal knowledge embedding. ECOLA jointly optimizes
the knowledge-text prediction objective and the temporal knowledge embedding
objective, and thus, can simultaneously take full advantage of textual and
structured knowledge. Since existing datasets do not provide tKGs with aligned
textual data, we introduce three new datasets for training and evaluating
ECOLA. Experimental results on the temporal knowledge graph completion task
show that ECOLA outperforms state-of-the-art tKG embedding models by a large
margin.

提出了一种增强的在上下文化语言表示下的时间知识嵌入模型 （ECOLA），它使用 tKG 四元组作为隐式测量来暂时对齐文本数据和时光演变的实体表示，并使用新的知识文本预测任务将文本信息注入时间知识嵌入。实验结果显示，ECOLA 在时间知识图嵌入任务中优于现有的 tKG 嵌入模型。