Understanding a long document requires tracking how entities are introduced
and evolve over time. We present a new type of language model, EntityNLM, that
can explicitly model entities, dynamically update their representations, and
contextually generate their mentions. Our model is generative and flexible; it
can model an arbitrary number of entities in context while generating each
entity mention at an arbitrary length. In addition, it can be used for several
different tasks such as language modeling, coreference resolution, and entity
prediction. Experimental results with all these tasks demonstrate that our
model consistently outperforms strong baselines and prior work.

该研究提出了一种名为 EntityNLM 的新型语言模型，可显式建模实体，动态更新其表示，并在特定的上下文中生成其提及。该模型在语言建模、指代消解和实体预测等多个不同任务中表现出色。

神经语言模型中的动态实体表示

Dynamic Entity Representations in Neural Language Models

The objective of knowledge graph embedding is to encode both entities and
relations of knowledge graphs into continuous low-dimensional vector spaces.
Previously, most works focused on symbolic representation of knowledge graph
with structure information, which can not handle new entities or entities with
few facts well. In this paper, we propose a novel deep architecture to utilize
both structural and textual information of entities. Specifically, we introduce
three neural models to encode the valuable information from text description of
entity, among which an attentive model can select related information as
needed. Then, a gating mechanism is applied to integrate representations of
structure and text into a unified architecture. Experiments show that our
models outperform baseline by margin on link prediction and triplet
classification tasks. Source codes of this paper will be available on Github.

本文提出一种新颖的深度架构，以利用实体的结构和文本信息的宝贵信息。其中，注意力模型可以选择所需的相关信息，并应用门控机制将结构和文本的表征集成到一个统一的架构中。实验表明，我们的模型在链接预测和三元组分类任务上的表现优于基线。

联合结构和文本编码的知识图谱表示

Knowledge Graph Representation with Jointly Structural and Textual  Encoding

Knowledge bases are useful resources for many natural language processing
tasks, however, they are far from complete. In this paper, we define a novel
entity representation as a mixture of its neighborhood in the knowledge base
and apply this technique on TransE-a well-known embedding model for knowledge
base completion. Experimental results show that the neighborhood information
significantly helps to improve the results of the TransE model, leading to
better performance than obtained by other state-of-the-art embedding models on
three benchmark datasets for triple classification, entity prediction and
relation prediction tasks.

本文介绍了一种新的实体表示方法，并将其应用于知识库补全中的 TransE 嵌入模型，实验证明这种方法可以明显提高 TransE 模型在三种基准数据集上的性能。

邻域混合模型用于知识库补全

Neighborhood Mixture Model for Knowledge Base Completion

Knowledge bases provide applications with the benefit of easily accessible,
systematic relational knowledge but often suffer in practice from their
incompleteness and lack of knowledge of new entities and relations. Much work
has focused on building or extending them by finding patterns in large
unannotated text corpora. In contrast, here we mainly aim to complete a
knowledge base by predicting additional true relationships between entities,
based on generalizations that can be discerned in the given knowledgebase. We
introduce a neural tensor network (NTN) model which predicts new relationship
entries that can be added to the database. This model can be improved by
initializing entity representations with word vectors learned in an
unsupervised fashion from text, and when doing this, existing relations can
even be queried for entities that were not present in the database. Our model
generalizes and outperforms existing models for this problem, and can classify
unseen relationships in WordNet with an accuracy of 75.8%.

本研究提出了一种基于神经张量网络模型（NTN）的通用数据关系预测方法，通过对给定知识库中的潜在关系进行分析，预测可能存在的新数据关系，初始实体表示使用文本中被无监督学习得到的词向量，该模型准确率可达 75.8%。