Understanding tables is an important aspect of natural language
understanding. Existing models for table understanding require linearization of
the table structure, where row or column order is encoded as an unwanted bias.
Such spurious biases make the model vulnerable to row and column order
perturbations. Additionally, prior work has not thoroughly modeled the table
structures or table-text alignments, hindering the table-text understanding
ability. In this work, we propose a robust and structurally aware table-text
encoding architecture TableFormer, where tabular structural biases are
incorporated completely through learnable attention biases. TableFormer is (1)
strictly invariant to row and column orders, and, (2) could understand tables
better due to its tabular inductive biases. Our evaluations showed that
TableFormer outperforms strong baselines in all settings on SQA, WTQ and
TabFact table reasoning datasets, and achieves state-of-the-art performance on
SQA, especially when facing answer-invariant row and column order perturbations
(6% improvement over the best baseline), because previous SOTA models'
performance drops by 4% - 6% when facing such perturbations while TableFormer
is not affected.

提出了一种结构完整和鲁棒性强的表格文本编码架构 TableFormer，通过可学习的注意力偏置完全融合了表格结构偏差，进一步强化了对表格的理解能力，在 SQA、WTQ 和 TabFact 等表理解数据集方面优于强基线，尤其在面对具有答案不变的行列顺序扰动时，相对于最佳基线的提升达到 6％。