Content Warning: This work contains examples that potentially implicate
stereotypes, associations, and other harms that could be offensive to
individuals in certain social groups.} Large pre-trained language models are
acknowledged to carry social biases towards different demographics, which can
further amplify existing stereotypes in our society and cause even more harm.
Text-to-SQL is an important task, models of which are mainly adopted by
administrative industries, where unfair decisions may lead to catastrophic
consequences. However, existing Text-to-SQL models are trained on clean,
neutral datasets, such as Spider and WikiSQL. This, to some extent, cover up
social bias in models under ideal conditions, which nevertheless may emerge in
real application scenarios. In this work, we aim to uncover and categorize
social biases in Text-to-SQL models. We summarize the categories of social
biases that may occur in structured data for Text-to-SQL models. We build test
benchmarks and reveal that models with similar task accuracy can contain social
biases at very different rates. We show how to take advantage of our
methodology to uncover and assess social biases in the downstream Text-to-SQL
task. We will release our code and data.

本研究旨在揭示 Text-to-SQL 模型中的社会偏见，通过建立测试基准和展示如何利用我们的方法来揭示并评估下游 Text-to-SQL 任务中的社会偏见。