We investigate large-scale latent variable models (LVMs) for neural story
generation -- an under-explored application for open-domain long text -- with
objectives in two threads: generation effectiveness and controllability. LVMs,
especially the variational autoencoder (VAE), have achieved both effective and
controllable generation through exploiting flexible distributional latent
representations. Recently, Transformers and its variants have achieved
remarkable effectiveness without explicit latent representation learning, thus
lack satisfying controllability in generation. In this paper, we advocate to
revive latent variable modeling, essentially the power of representation
learning, in the era of Transformers to enhance controllability without hurting
state-of-the-art generation effectiveness. Specifically, we integrate latent
representation vectors with a Transformer-based pre-trained architecture to
build conditional variational autoencoder (CVAE). Model components such as
encoder, decoder and the variational posterior are all built on top of
pre-trained language models -- GPT2 specifically in this paper. Experiments
demonstrate state-of-the-art conditional generation ability of our model, as
well as its excellent representation learning capability and controllability.

本文提出了一种基于转换器和变分自动编码器（VAE）的条件变分自动编码器（CVAE）方法，在保持卓越的生成效果的同时增强了可控性和表征学习能力，实现了对长文本的神经故事生成。