A key component of fact verification is thevevidence retrieval, often from
multiple documents. Recent approaches use dense representations and condition
the retrieval of each document on the previously retrieved ones. The latter
step is performed over all the documents in the collection, requiring storing
their dense representations in an index, thus incurring a high memory
footprint. An alternative paradigm is retrieve-and-rerank, where documents are
retrieved using methods such as BM25, their sentences are reranked, and further
documents are retrieved conditioned on these sentences, reducing the memory
requirements. However, such approaches can be brittle as they rely on
heuristics and assume hyperlinks between documents. We propose a novel
retrieve-and-rerank method for multi-hop retrieval, that consists of a
retriever that jointly scores documents in the knowledge source and sentences
from previously retrieved documents using an autoregressive formulation and is
guided by a proof system based on natural logic that dynamically terminates the
retrieval process if the evidence is deemed sufficient. This method is
competitive with current state-of-the-art methods on FEVER, HoVer and
FEVEROUS-S, while using $5$ to $10$ times less memory than competing systems.
Evaluation on an adversarial dataset indicates improved stability of our
approach compared to commonly deployed threshold-based methods. Finally, the
proof system helps humans predict model decisions correctly more often than
using the evidence alone.

我们提出了一种基于自回归公式的检索和重新排序方法，该方法用自然逻辑为基础，使用证明系统动态终止检索过程，同时使用比竞争系统少 5 到 10 倍的记忆，对 FEVER、HoVer 和 FEVEROUS-S 等数据集具有与当前最先进方法相当的性能。

基于自然语言逻辑的自回归多跳文档检索方法用于事实核查

Natural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verification

Structured and grounded representation of text is typically formalized by
closed information extraction, the problem of extracting an exhaustive set of
(subject, relation, object) triplets that are consistent with a predefined set
of entities and relations from a knowledge base schema. Most existing works are
pipelines prone to error accumulation, and all approaches are only applicable
to unrealistically small numbers of entities and relations. We introduce GenIE
(generative information extraction), the first end-to-end autoregressive
formulation of closed information extraction. GenIE naturally exploits the
language knowledge from the pre-trained transformer by autoregressively
generating relations and entities in textual form. Thanks to a new bi-level
constrained generation strategy, only triplets consistent with the predefined
knowledge base schema are produced. Our experiments show that GenIE is
state-of-the-art on closed information extraction, generalizes from fewer
training data points than baselines, and scales to a previously unmanageable
number of entities and relations. With this work, closed information extraction
becomes practical in realistic scenarios, providing new opportunities for
downstream tasks. Finally, this work paves the way towards a unified end-to-end
approach to the core tasks of information extraction. Code, data and models
available at this https URL

本文介绍了 GenIE 技术，这是第一个可应用于现实场景的 autoregressive formulation 方法，通过生成文本形式的关系和实体，仅生成与预定义知识库模式一致的三元组。实验表明，GenIE 在关闭信息提取方面表现最佳，并且可以推广到相对较少的训练数据点，尺度可以扩展到以前难于管理的实体和关系数量。

GenIE: 生成式信息提取

GenIE: Generative Information Extraction

Generative approaches have been recently shown to be effective for both
Entity Disambiguation and Entity Linking (i.e., joint mention detection and
disambiguation). However, the previously proposed autoregressive formulation
for EL suffers from i) high computational cost due to a complex (deep) decoder,
ii) non-parallelizable decoding that scales with the source sequence length,
and iii) the need for training on a large amount of data. In this work, we
propose a very efficient approach that parallelizes autoregressive linking
across all potential mentions and relies on a shallow and efficient decoder.
Moreover, we augment the generative objective with an extra discriminative
component, i.e., a correction term which lets us directly optimize the
generator's ranking. When taken together, these techniques tackle all the above
issues: our model is >70 times faster and more accurate than the previous
generative method, outperforming state-of-the-art approaches on the standard
English dataset AIDA-CoNLL. Source code available at
this https URL

本文提出了一种高效的生成方法，采用并行自回归流程和浅层解码器实现实体消岐和链接，并增加了区分度组件，达到了比之前的方法更快（ >70 倍）更准确的效果，代码开源。

具有判别校正的高度并行自回归实体链接

Highly Parallel Autoregressive Entity Linking with Discriminative  Correction

Entities are at the center of how we represent and aggregate knowledge. For
instance, Encyclopedias such as Wikipedia are structured by entities (e.g., one
per Wikipedia article). The ability to retrieve such entities given a query is
fundamental for knowledge-intensive tasks such as entity linking and
open-domain question answering. Current approaches can be understood as
classifiers among atomic labels, one for each entity. Their weight vectors are
dense entity representations produced by encoding entity meta information such
as their descriptions. This approach has several shortcomings: (i) context and
entity affinity is mainly captured through a vector dot product, potentially
missing fine-grained interactions; (ii) a large memory footprint is needed to
store dense representations when considering large entity sets; (iii) an
appropriately hard set of negative data has to be subsampled at training time.
In this work, we propose GENRE, the first system that retrieves entities by
generating their unique names, left to right, token-by-token in an
autoregressive fashion. This mitigates the aforementioned technical issues
since: (i) the autoregressive formulation directly captures relations between
context and entity name, effectively cross encoding both; (ii) the memory
footprint is greatly reduced because the parameters of our encoder-decoder
architecture scale with vocabulary size, not entity count; (iii) the softmax
loss is computed without subsampling negative data. We experiment with more
than 20 datasets on entity disambiguation, end-to-end entity linking and
document retrieval tasks, achieving new state-of-the-art or very competitive
results while using a tiny fraction of the memory footprint of competing
systems. Finally, we demonstrate that new entities can be added by simply
specifying their names. Code and pre-trained models at
this https URL

GENRE 是第一个通过自回归方式逐个 token 生成实体名称以检索实体的系统，此方法可以在自动编码器中直接捕捉上下文和实体名称之间的关系，从而减少内存占用并提高检索效率。