We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to
achieve human performance in WikiSQL dataset. We revisit and discuss diverse
popular methods in NL2SQL literature, take a full advantage of BERT {Devlin et
al., 2018) through an effective table contextualization method, and coherently
combine them, outperforming the previous state of the art by 8.2% and 2.5% in
logical form and execution accuracy, respectively. We particularly note that
BERT with a seq2seq decoder leads to a poor performance in the task, indicating
the importance of a careful design when using such large pretrained models. We
also provide a comprehensive analysis on the dataset and our model, which can
be helpful for designing future NL2SQL datsets and models. We especially show
that our model's performance is near the upper bound in WikiSQL, where we
observe that a large portion of the evaluation errors are due to wrong
annotations, and our model is already exceeding human performance by 1.3% in
execution accuracy.

SQLova 是第一个在 WikiSQL 数据集中实现人类性能的自然语言到 SQL 模型，利用了 BERT 通过有效的表格语境方法，结合了多种流行的 NL2SQL 方法，在逻辑形式和执行准确度方面分别优于先前的最新技术水平 8.2％和 2.5％。我们特别注意到，在使用这样的大型预训练模型时，BERT 与序列到序列解码器会导致性能不佳，这表明设计的重要性。我们还对数据集和我们的模型进行了全面的分析，这可能有助于设计未来的 NL2SQL 数据集和模型。我们特别展示了我们模型的表现已经接近 WikiSQL 的上限，在其中我们观察到大部分评估错误是由于错误的注释，而我们的模型在执行准确度方面已经超过人类表现 1.3％。