We present GraPPa, an effective pre-training approach for table semantic
parsing that learns a compositional inductive bias in the joint representations
of textual and tabular data. We construct synthetic question-SQL pairs over
high-quality tables via a synchronous context-free grammar (SCFG) induced from
existing text-to-SQL datasets. We pre-train our model on the synthetic data
using a novel text-schema linking objective that predicts the syntactic role of
a table field in the SQL for each question-SQL pair. To maintain the model's
ability to represent real-world data, we also include masked language modeling
(MLM) over several existing table-and-language datasets to regularize the
pre-training process. On four popular fully supervised and weakly supervised
table semantic parsing benchmarks, GraPPa significantly outperforms
RoBERTa-large as the feature representation layers and establishes new
state-of-the-art results on all of them.

GraPPa 是一种有效的预训练方法，用于表语义分析，学习了文本和表格数据的联合表示中的组合归纳偏置。它通过从现有文本到 SQL 数据集中归纳出的同步上下文无关文法（SCFG）构造高质量表的综合问题 - SQL 对，使用一种新的文本模式链接目标在合成数据上进行模型预训练，并包括遮蔽语言建模以引导预训练过程。GraPPa 在四个流行的全监督和弱监督表格语义解析基准上都显着优于 RoBERTa-large，建立了新的所有最先进的结果。

GraPPa: 表格语义解析的语法增强预训练

GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

Neural language models learn word representations that capture rich
linguistic and conceptual information. Here we investigate the embeddings
learned by neural machine translation models. We show that translation-based
embeddings outperform those learned by cutting-edge monolingual models at
single-language tasks requiring knowledge of conceptual similarity and/or
syntactic role. The findings suggest that, while monolingual models learn
information about how concepts are related, neural-translation models better
capture their true ontological status.

研究神经机器翻译模型和单语模型在表征词汇相似性和语法角色方面的性能，结果表明，神经翻译模型在捕捉概念的本体论状态方面比单语模型更胜一筹。