Generative Adversarial Networks (GAN) is a model for data synthesis, which
creates plausible data through the competition of generator and discriminator.
Although GAN application to image synthesis is extensively studied, it has
inherent limitations to natural language generation. Because natural language
is composed of discrete tokens, a generator has difficulty updating its
gradient through backpropagation; therefore, most text-GAN studies generate
sentences starting with a random token based on a reward system. Thus, the
generators of previous studies are pre-trained in an autoregressive way before
adversarial training, causing data memorization that synthesized sentences
reproduce the training data. In this paper, we synthesize sentences using a
framework similar to the original GAN. More specifically, we propose Text
Embedding Space Generative Adversarial Networks (TESGAN) which generate
continuous text embedding spaces instead of discrete tokens to solve the
gradient backpropagation problem. Furthermore, TESGAN conducts unsupervised
learning which does not directly refer to the text of the training data to
overcome the data memorization issue. By adopting this novel method, TESGAN can
synthesize new sentences, showing the potential of unsupervised learning for
text synthesis. We expect to see extended research combining Large Language
Models with a new perspective of viewing text as an continuous space.

本文提出了一种名为 TESGAN 的框架，使用连续文本嵌入空间而非离散标记以解决梯度反向传播的问题，通过无监督学习来克服数据记忆问题，从而能够合成具有潜力的新句子。