Numerous studies have demonstrated the ability of neural language models to
learn various linguistic properties without direct supervision. This work takes
an initial step towards exploring the less researched topic of how neural
models discover linguistic properties of words, such as gender, as well as the
rules governing their usage. We propose to use an artificial corpus generated
by a PCFG based on French to precisely control the gender distribution in the
training data and determine under which conditions a model correctly captures
gender information or, on the contrary, appears gender-biased.

神经语言模型在没有直接监督的情况下学习各种语言属性的能力已被多个研究证明。本研究初步探索神经模型如何发现词语的语言属性（如性别）及其使用规则这一较少研究的主题。我们建议使用基于法语的 PCFG 生成的人工语料库来精确控制训练数据中的性别分布，并确定模型在何种条件下正确捕捉性别信息，或出现性别偏见。

利用人工法语数据理解转换器语言模型中性别偏见的出现

Using Artificial French Data to Understand the Emergence of Gender Bias  in Transformer Language Models

Existing work on controlled text generation (CTG) assumes a control interface
of categorical attributes. In this work, we propose a natural language (NL)
interface, where we craft a PCFG to embed the control attributes into natural
language commands, and propose variants of existing CTG models that take
commands as input. In our experiments, we design tailored setups to test
model's generalization abilities. We find our PCFG-based command generation
approach is effective for handling unseen commands compared to fix-set
templates; our proposed NL models can effectively generalize to unseen
attributes, a new ability enabled by the NL interface, as well as unseen
attribute combinations. Interestingly, we discover that the simple conditional
generation approach, enhanced with our proposed NL interface, is a strong
baseline in those challenging settings.

本论文提出了一种将控制属性嵌入自然语言命令中的基于 PCFG 的生成命令方法，并提出了将命令作为输入的 CTG 模型变体，在实验中发现，这种方法不仅可以有效处理未见过的命令，而且可以有效泛化到未见过的属性及其组合，并通过简单条件生成方法强化了该方法，形成了一个强有力的基线模型。

基于 PCFG 的自然语言接口改进受控文本生成的泛化性能

PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation

Synthesizing data for semantic parsing has gained increasing attention
recently. However, most methods require handcrafted (high-precision) rules in
their generative process, hindering the exploration of diverse unseen data. In
this work, we propose a generative model which features a (non-neural) PCFG
that models the composition of programs (e.g., SQL), and a BART-based
translation model that maps a program to an utterance. Due to the simplicity of
PCFG and pre-trained BART, our generative model can be efficiently learned from
existing data at hand. Moreover, explicitly modeling compositions using PCFG
leads to a better exploration of unseen programs, thus generate more diverse
data. We evaluate our method in both in-domain and out-of-domain settings of
text-to-SQL parsing on the standard benchmarks of GeoQuery and Spider,
respectively. Our empirical results show that the synthesized data generated
from our model can substantially help a semantic parser achieve better
compositional and domain generalization.

本篇论文提出了一种生成模型，该模型具有模拟编程语言的组合结构的 PCFG 模型和将编程转化为语句的 BART 模型，能够有效且高效地从现有的数据中学习，并且能够探索多样化的未知数据，通过在 GeoQuery 和 Spider 标准基准测试中的实验结果表明，该模型合成的数据可以大大帮助语义解析器实现更好的组合性和领域泛化。