In this work we propose a novel end-to-end multi-stage Knowledge Graph (KG)
generation system from textual inputs, separating the overall process into two
stages. The graph nodes are generated first using pretrained language model,
followed by a simple edge construction head, enabling efficient KG extraction
from the text. For each stage we consider several architectural choices that
can be used depending on the available training resources. We evaluated the
model on a recent WebNLG 2020 Challenge dataset, matching the state-of-the-art
performance on text-to-RDF generation task, as well as on New York Times (NYT)
and a large-scale TekGen datasets, showing strong overall performance,
outperforming the existing baselines. We believe that the proposed system can
serve as a viable KG construction alternative to the existing linearization or
sampling-based graph generation approaches. Our code can be found at
this https URL

提出了一种通过对文本输入进行多阶段知识图谱生成的系统，并通过使用预训练的语言模型生成图节点，然后使用简单的边构造头部来使得从文本中提取知识图谱效率更高。该模型在 WebNLG 2020 挑战数据集，New York Times（NYT）和大规模 TekGen 数据集上表现不错，超过现有基线。

从文本生成知识图谱

Knowledge Graph Generation From Text

Qualitative causal relationships compactly express the direction, dependency,
temporal constraints, and monotonicity constraints of discrete or continuous
interactions in the world. In everyday or academic language, we may express
interactions between quantities (e.g., sleep decreases stress), between
discrete events or entities (e.g., a protein inhibits another protein's
transcription), or between intentional or functional factors (e.g., hospital
patients pray to relieve their pain). Extracting and representing these diverse
causal relations are critical for cognitive systems that operate in domains
spanning from scientific discovery to social science. This paper presents a
transformer-based NLP architecture that jointly extracts knowledge graphs
including (1) variables or factors described in language, (2) qualitative
causal relationships over these variables, (3) qualifiers and magnitudes that
constrain these causal relationships, and (4) word senses to localize each
extracted node within a large ontology. We do not claim that our
transformer-based architecture is itself a cognitive system; however, we
provide evidence of its accurate knowledge graph extraction in real-world
domains and the practicality of its resulting knowledge graphs for cognitive
systems that perform graph-based reasoning. We demonstrate this approach and
include promising results in two use cases, processing textual inputs from
academic publications, news articles, and social media.

本文介绍了基于 transformer 的 NLP 架构，该架构可以联合提取包括语言描述、定性因果关系、约束限定词、词义在内的知识图谱，以及说明其在真实世界中的准确性和实用性。

基于 Transformer 的方法：从非结构化文本到因果知识图谱

From Unstructured Text to Causal Knowledge Graphs: A Transformer-Based Approach

Many methods now exist for conditioning model outputs on task instructions,
retrieved documents, and user-provided explanations and feedback. Rather than
relying solely on examples of task inputs and outputs, these approaches use
valuable additional data for improving model correctness and aligning learned
models with human priors. Meanwhile, a growing body of evidence suggests that
some language models can (1) store a large amount of knowledge in their
parameters, and (2) perform inference over tasks in textual inputs at test
time. These results raise the possibility that, for some tasks, humans cannot
explain to a model any more about the task than it already knows or could infer
on its own. In this paper, we study the circumstances under which explanations
of individual data points can (or cannot) improve modeling performance. In
order to carefully control important properties of the data and explanations,
we introduce a synthetic dataset for experiments, and we also make use of three
existing datasets with explanations: e-SNLI, TACRED, and SemEval. We first give
a formal framework for the available modeling approaches, in which explanation
data can be used as model inputs, as targets, or as a prior. After arguing that
the most promising role for explanation data is as model inputs, we propose to
use a retrieval-based method and show that it solves our synthetic task with
accuracies upwards of 95%, while baselines without explanation data achieve
below 65% accuracy. We then identify properties of datasets for which
retrieval-based modeling fails. With the three existing datasets, we find no
improvements from explanation retrieval. Drawing on findings from our synthetic
task, we suggest that at least one of six preconditions for successful modeling
fails to hold with these datasets. Our code is publicly available at
this https URL

这篇论文研究了解释数据对于模型效能的影响，利用回收法证明了推理为主的文本输入任务中，解释数据可用作模型输入，且采用检索式的建模方法可以使合成任务的准确率达到 95%，而无解释数据的基线准确率低于 65%。