There are many recent advanced developments for the Text-to-SQL task, where
the Picard model is one of the the top performing models as measured by the
Spider dataset competition. However, bringing Text-to-SQL systems to realistic
use-cases through domain adaptation remains a tough challenge. We analyze how
well the base T5 Language Model and Picard perform on query structures
different from the Spider dataset, we fine-tuned the base model on the Spider
data and on independent databases (DB). To avoid accessing the DB content
online during inference, we also present an alternative way to disambiguate the
values in an input question using a rule-based approach that relies on an
intermediate representation of the semantic concepts of an input question. In
our results we show in what cases T5 and Picard can deliver good performance,
we share the lessons learned, and discuss current domain adaptation challenges.

在 Text-to-SQL 任务中，通过领域适应性实现将 Text-to-SQL 系统应用于真实用例仍然是一个艰巨的挑战。通过对基础模型在 Spider 数据集之外的查询结构进行 fine-tuning，并采用基于规则的方法解决输入问题中的值的歧义性，我们展示了 T5 和 Picard 在何种情况下能够提供良好性能，分享了所学到的经验，并讨论了当前领域适应性的挑战。

一种先进的文本到结构化查询语言模型的领域适应：经验教训与挑战

Domain Adaptation of a State of the Art Text-to-SQL Model: Lessons  Learned and Challenges Found

We examine whether large neural language models, trained on very large
collections of varied English text, learn the potentially long-distance
dependency of British versus American spelling conventions, i.e., whether
spelling is consistently one or the other within model-generated strings. In
contrast to long-distance dependencies in non-surface underlying structure
(e.g., syntax), spelling consistency is easier to measure both in LMs and the
text corpora used to train them, which can provide additional insight into
certain observed model behaviors. Using a set of probe words unique to either
British or American English, we first establish that training corpora exhibit
substantial (though not total) consistency. A large T5 language model does
appear to internalize this consistency, though only with respect to observed
lexical items (not nonce words with British/American spelling patterns). We
further experiment with correcting for biases in the training data by
fine-tuning T5 on synthetic data that has been debiased, and find that
finetuned T5 remains only somewhat sensitive to spelling consistency. Further
experiments show GPT2 to be similarly limited.

研究表明，大型神经语言模型在训练时，能够学习英式和美式拼写约定的长距离依赖关系，但是只有对于观察到的词汇项，而非具有英式 / 美式拼写模式的虚构单词。此外，研究还发现 T5 和 GPT2 模型在纠正训练数据偏差方面存在限制。