Fact verification has attracted a lot of research attention recently, e.g.,
in journalism, marketing, and policymaking, as misinformation and
disinformation online can sway one's opinion and affect one's actions. While
fact-checking is a hard task in general, in many cases, false statements can be
easily debunked based on analytics over tables with reliable information.
Hence, table-based fact verification has recently emerged as an important and
growing research area. Yet, progress has been limited due to the lack of
datasets that can be used to pre-train language models (LMs) to be aware of
common table operations, such as aggregating a column or comparing tuples. To
bridge this gap, in this paper we introduce PASTA, a novel state-of-the-art
framework for table-based fact verification via pre-training with synthesized
sentence-table cloze questions. In particular, we design six types of common
sentence-table cloze tasks, including Filter, Aggregation, Superlative,
Comparative, Ordinal, and Unique, based on which we synthesize a large corpus
consisting of 1.2 million sentence-table pairs from WikiTables. PASTA uses a
recent pre-trained LM, DeBERTaV3, and further pretrains it on our corpus. Our
experimental results show that PASTA achieves new state-of-the-art performance
on two table-based fact verification benchmarks: TabFact and SEM-TAB-FACTS. In
particular, on the complex set of TabFact, which contains multiple operations,
PASTA largely outperforms the previous state of the art by 4.7 points (85.6%
vs. 80.9%), and the gap between PASTA and human performance on the small
TabFact test set is narrowed to just 1.5 points (90.6% vs. 92.1%).

本文介绍了一种基于数据表的事实验证的新方法 PASTA，使用已经进行了先前训练的语言模型和通过合成句子 - 数据表填空问题而得到的 1.2 百万个数据对进行预训练，PASTA 在 TabFact 和 SEM-TAB-FACTS 两个基准测试中实现了最新的最佳表现

PASTA: 句表填空预训练，适用于表格操作感知的事实验证

PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training

The table-based fact verification task has recently gained widespread
attention and yet remains to be a very challenging problem. It inherently
requires informative reasoning over natural language together with different
numerical and logical reasoning on tables (e.g., count, superlative,
comparative). Considering that, we exploit mixture-of-experts and present in
this paper a new method: Self-adaptive Mixture-of-Experts Network (SaMoE).
Specifically, we have developed a mixture-of-experts neural network to
recognize and execute different types of reasoning -- the network is composed
of multiple experts, each handling a specific part of the semantics for
reasoning, whereas a management module is applied to decide the contribution of
each expert network to the verification result. A self-adaptive method is
developed to teach the management module combining results of different experts
more efficiently without external knowledge. The experimental results
illustrate that our framework achieves 85.1% accuracy on the benchmark dataset
TabFact, comparable with the previous state-of-the-art models. We hope our
framework can serve as a new baseline for table-based verification. Our code is
available at this https URL.

本文提出了一种基于自适应 Mixture-of-Experts 网络的方法用于表格事实验证，该方法有效结合了自然语言和不同类型的数理逻辑推理。实验结果表明，该方法可以取得与之前最先进的模型可比的 85.1% 的准确度。