Despite significant progress having been made in question answering on
tabular data (Table QA), it's unclear whether, and to what extent existing
Table QA models are robust to task-specific perturbations, e.g., replacing key
question entities or shuffling table columns. To systematically study the
robustness of Table QA models, we propose a benchmark called RobuT, which
builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and
includes human-annotated adversarial perturbations in terms of table header,
table content, and question. Our results indicate that both state-of-the-art
Table QA models and large language models (e.g., GPT-3) with few-shot learning
falter in these adversarial sets. We propose to address this problem by using
large language models to generate adversarial examples to enhance training,
which significantly improves the robustness of Table QA models. Our data and
code is publicly available at this https URL

本研究通过构建 RobuT 数据集，含有表头、表内容及问题中的对抗性干扰，对现有表格问答模型的鲁棒性进行了评估，并提出利用大型语言模型生成对抗样本以增强训练，从而显著提高表格问答模型的鲁棒性。

RobuT: 对表格 QA 鲁棒性针对人类注释的对抗扰动的系统研究

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated  Adversarial Perturbations

Recent advances in transformers have enabled Table Question Answering (Table
QA) systems to achieve high accuracy and SOTA results on open domain datasets
like WikiTableQuestions and WikiSQL. Such transformers are frequently
pre-trained on open-domain content such as Wikipedia, where they effectively
encode questions and corresponding tables from Wikipedia as seen in Table QA
dataset. However, web tables in Wikipedia are notably flat in their layout,
with the first row as the sole column header. The layout lends to a relational
view of tables where each row is a tuple. Whereas, tables in domain-specific
business or scientific documents often have a much more complex layout,
including hierarchical row and column headers, in addition to having
specialized vocabulary terms from that domain.
To address this problem, we introduce the domain-specific Table QA dataset
AIT-QA (Airline Industry Table QA). The dataset consists of 515 questions
authored by human annotators on 116 tables extracted from public U.S. SEC
filings (publicly available at: this https URL) of major
airline companies for the fiscal years 2017-2019. We also provide annotations
pertaining to the nature of questions, marking those that require hierarchical
headers, domain-specific terminology, and paraphrased forms. Our zero-shot
baseline evaluation of three transformer-based SOTA Table QA methods - TaPAS
(end-to-end), TaBERT (semantic parsing-based), and RCI (row-column
encoding-based) - clearly exposes the limitation of these methods in this
practical setting, with the best accuracy at just 51.8\% (RCI). We also present
pragmatic table preprocessing steps used to pivot and project these complex
tables into a layout suitable for the SOTA Table QA models.

该研究主要介绍了一个针对航空业公司数据集的领域特定的表格问答数据集 AIT-QA，分析了使用 Transformer 模型进行 Table QA 的现有方法在处理领域特定的表格数据时面临的挑战，并提出了一些实用的表格预处理步骤。

航空行业复杂表格的问答数据集

AIT-QA: Question Answering Dataset over Complex Tables in the Airline  Industry

Transformer based architectures are recently used for the task of answering
questions over tables. In order to improve the accuracy on this task,
specialized pre-training techniques have been developed and applied on millions
of open-domain web tables. In this paper, we propose two novel approaches
demonstrating that one can achieve superior performance on table QA task
without even using any of these specialized pre-training techniques. The first
model, called RCI interaction, leverages a transformer based architecture that
independently classifies rows and columns to identify relevant cells. While
this model yields extremely high accuracy at finding cell values on recent
benchmarks, a second model we propose, called RCI representation, provides a
significant efficiency advantage for online QA systems over tables by
materializing embeddings for existing tables. Experiments on recent benchmarks
prove that the proposed methods can effectively locate cell values on tables
(up to ~98% Hit@1 accuracy on WikiSQL lookup questions). Also, the interaction
model outperforms the state-of-the-art transformer based approaches,
pre-trained on very large table corpora (TAPAS and TaBERT), achieving ~3.4% and
~18.86% additional precision improvement on the standard WikiSQL benchmark.

本文介绍了两种新方法实现 Table QA 任务，一种是利用独立分类来识别相关单元格的 RCI interaction 模型，一种是通过材料化嵌入现有表格来提高在线 QA 系统效率的 RCI representation 模型，实验证明这两种方法在表格上有效，并且交互模型在最近的基准测试中优于最新的基于 Transformer 的方法，获得了约 3.4％和约 18.86％的精度提高。