Effectively modeling text-rich fresh content such as news articles at
document-level is a challenging problem. To ensure a content-based model
generalize well to a broad range of applications, it is critical to have a
training dataset that is large beyond the scale of human labels while achieving
desired quality. In this work, we address those two challenges by proposing a
novel approach to mine semantically-relevant fresh documents, and their topic
labels, with little human supervision. Meanwhile, we design a multitask model
called NewsEmbed that alternatively trains a contrastive learning with a
multi-label classification to derive a universal document encoder. We show that
the proposed approach can provide billions of high quality organic training
examples and can be naturally extended to multilingual setting where texts in
different languages are encoded in the same semantic space. We experimentally
demonstrate NewsEmbed's competitive performance across multiple natural
language understanding tasks, both supervised and unsupervised.

本文提出了一个新颖的方法，使用少量人为监督来挖掘具有语义相关性的新鲜文档及其主题标签，并设计了一个多任务模型 - NewsEmbed，交替使用对比学习和多标签分类来推导通用文档编码器，在多个自然语言理解任务中实现了出色的性能表现。

NewsEmbed: 预训练文档表示模型对新闻进行建模

NewsEmbed: Modeling News through Pre-trained Document Representations

Relation extraction (RE) aims to identify the semantic relations between
named entities in text. Recent years have witnessed it raised to the document
level, which requires complex reasoning with entities and mentions throughout
an entire document. In this paper, we propose a novel model to document-level
RE, by encoding the document information in terms of entity global and local
representations as well as context relation representations. Entity global
representations model the semantic information of all entities in the document,
entity local representations aggregate the contextual information of multiple
mentions of specific entities, and context relation representations encode the
topic information of other relations. Experimental results demonstrate that our
model achieves superior performance on two public datasets for document-level
RE. It is particularly effective in extracting relations between entities of
long distance and having multiple mentions.

本文提出了一种新的模型对文档级别关系抽取进行建模，该模型能够对全局和局部的实体表示以及上下文关系表示进行编码并提供更好的实验结果。