Using generated data to improve the performance of downstream discriminative
models has recently gained popularity due to the great development of
pre-trained language models. In most previous studies, generative models and
discriminative models are trained separately and thus could not adapt to any
changes in each other. As a result, the generated samples can easily deviate
from the real data distribution, while the improvement of the discriminative
model quickly reaches saturation. Generative adversarial networks (GANs) train
generative models via an adversarial process with discriminative models to
achieve joint training. However, the training of standard GANs is notoriously
unstable and often falls short of convergence. In this paper, to address these
issues, we propose a $\textit{self-consistent learning}$ framework, in which a
discriminator and a generator are cooperatively trained in a closed-loop form.
The discriminator and the generator enhance each other during multiple rounds
of alternating training until a scoring consensus is reached. This framework
proves to be easy to train and free from instabilities such as mode collapse
and non-convergence. Extensive experiments on sentence semantic matching
demonstrate the effectiveness of the proposed framework: the discriminator
achieves 10+ AP of improvement on the zero-shot setting and new
state-of-the-art performance on the full-data setting.

本文提出了一种自洽学习的框架，利用生成对抗网络的生成模型和判别模型的博弈优化，使两者协同训练，避免了 GAN 训练不稳定的问题，并在句子语义匹配问题上实现了较好的性能。