Disentangling the encodings of neural models is a fundamental aspect for
improving interpretability, semantic control and downstream task performance in
Natural Language Processing. Currently, most disentanglement methods are
unsupervised or rely on synthetic datasets with known generative factors. We
argue that recurrent syntactic and semantic regularities in textual data can be
used to provide the models with both structural biases and generative factors.
We leverage the semantic structures present in a representative and
semantically dense category of sentence types, definitional sentences, for
training a Variational Autoencoder to learn disentangled representations. Our
experimental results show that the proposed model outperforms unsupervised
baselines on several qualitative and quantitative benchmarks for
disentanglement, and it also improves the results in the downstream task of
definition modeling.

使用可表征语法和语义规律的定义句类别训练变分自编码器，并在多个测试中展现其成功解耦表示效果，以及在定义建模中得到的改善。

学习自然语言定义的解耦表示

Learning Disentangled Representations for Natural Language Definitions

Correlations between factors of variation are prevalent in real-world data.
Exploiting such correlations may increase predictive performance on noisy data;
however, often correlations are not robust (e.g., they may change between
domains, datasets, or applications) and models that exploit them do not
generalize when correlations shift. Disentanglement methods aim to learn
representations which capture different factors of variation in latent
subspaces. A common approach involves minimizing the mutual information between
latent subspaces, such that each encodes a single underlying attribute.
However, this fails when attributes are correlated. We solve this problem by
enforcing independence between subspaces conditioned on the available
attributes, which allows us to remove only dependencies that are not due to the
correlation structure present in the training data. We achieve this via an
adversarial approach to minimize the conditional mutual information (CMI)
between subspaces with respect to categorical variables. We first show
theoretically that CMI minimization is a good objective for robust
disentanglement on linear problems. We then apply our method on real-world
datasets based on MNIST and CelebA, and show that it yields models that are
disentangled and robust under correlation shift, including in weakly supervised
settings.

本文介绍了一种基于条件互信息的对抗学习方法，实现了在考虑属性相关性情况下，对数据因素进行有效分离的目的，提高了模型的鲁棒性。