Long sequences of text are challenging in the context of transformers, due to
quadratic memory increase in the self-attention mechanism. As this issue
directly affects the translation from natural language to SQL queries (as
techniques usually take as input a concatenated text with the question and the
database schema), we present techniques that allow long text sequences to be
handled by transformers with up to 512 input tokens. We propose a training
process with database schema pruning (removal of tables and columns names that
are useless for the query of interest). In addition, we used a multilingual
approach with the mT5-large model fine-tuned with a data-augmented Spider
dataset in four languages simultaneously: English, Portuguese, Spanish, and
French. Our proposed technique used the Spider dataset and increased the exact
set match accuracy results from 0.718 to 0.736 in a validation dataset (Dev).
Source code, evaluations, and checkpoints are available at:
\underline{this https URL}.

本文提出了一种技术，使长文本序列可以通过 transformers 处理，这对于自然语言到 SQL 查询的翻译非常有用，该技术包括训练过程、数据库架构修剪和多语言方法。