Recent advances in transformers have enabled Table Question Answering (Table
QA) systems to achieve high accuracy and SOTA results on open domain datasets
like WikiTableQuestions and WikiSQL. Such transformers are frequently
pre-trained on open-domain content such as Wikipedia, where they effectively
encode questions and corresponding tables from Wikipedia as seen in Table QA
dataset. However, web tables in Wikipedia are notably flat in their layout,
with the first row as the sole column header. The layout lends to a relational
view of tables where each row is a tuple. Whereas, tables in domain-specific
business or scientific documents often have a much more complex layout,
including hierarchical row and column headers, in addition to having
specialized vocabulary terms from that domain.
To address this problem, we introduce the domain-specific Table QA dataset
AIT-QA (Airline Industry Table QA). The dataset consists of 515 questions
authored by human annotators on 116 tables extracted from public U.S. SEC
filings (publicly available at: this https URL) of major
airline companies for the fiscal years 2017-2019. We also provide annotations
pertaining to the nature of questions, marking those that require hierarchical
headers, domain-specific terminology, and paraphrased forms. Our zero-shot
baseline evaluation of three transformer-based SOTA Table QA methods - TaPAS
(end-to-end), TaBERT (semantic parsing-based), and RCI (row-column
encoding-based) - clearly exposes the limitation of these methods in this
practical setting, with the best accuracy at just 51.8\% (RCI). We also present
pragmatic table preprocessing steps used to pivot and project these complex
tables into a layout suitable for the SOTA Table QA models.

该研究主要介绍了一个针对航空业公司数据集的领域特定的表格问答数据集 AIT-QA，分析了使用 Transformer 模型进行 Table QA 的现有方法在处理领域特定的表格数据时面临的挑战，并提出了一些实用的表格预处理步骤。