Natural Language Generation (NLG), and more generally generative AI, are
among the currently most impactful research fields. Creative NLG, such as
automatic poetry generation, is a fascinating niche in this area. While most
previous research has focused on forms of the Turing test when evaluating
automatic poetry generation - can humans distinguish between automatic and
human generated poetry - we evaluate the diversity of automatically generated
poetry, by comparing distributions of generated poetry to distributions of
human poetry along structural, lexical, semantic and stylistic dimensions,
assessing different model types (word vs. character-level, general purpose LLMs
vs. poetry-specific models), including the very recent LLaMA3, and types of
fine-tuning (conditioned vs. unconditioned). We find that current automatic
poetry systems are considerably underdiverse along multiple dimensions - they
often do not rhyme sufficiently, are semantically too uniform and even do not
match the length distribution of human poetry. Our experiments reveal, however,
that style-conditioning and character-level modeling clearly increases
diversity across virtually all dimensions we explore. Our identified
limitations may serve as the basis for more genuinely diverse future poetry
generation models.

评估自动生成诗歌的多样性，通过比较自动生成诗歌的结构、词汇、语义和风格等维度的分布与人类诗歌的分布，并考察不同模型类型和微调类型，发现当前的自动生成诗歌系统在多个维度上缺乏多样性，但风格条件和字符级建模明显增加了多样性。这些限制可用于更加多样化的未来诗歌生成模型。

自动诗歌生成中的多样性评估

Evaluating Diversity in Automatic Poetry Generation

As an essential step towards computer creativity, automatic poetry generation
has gained increasing attention these years. Though recent neural models make
prominent progress in some criteria of poetry quality, generated poems still
suffer from the problem of poor diversity. Related literature researches show
that different factors, such as life experience, historical background, etc.,
would influence composition styles of poets, which considerably contributes to
the high diversity of human-authored poetry. Inspired by this, we propose
MixPoet, a novel model that absorbs multiple factors to create various styles
and promote diversity. Based on a semi-supervised variational autoencoder, our
model disentangles the latent space into some subspaces, with each conditioned
on one influence factor by adversarial training. In this way, the model learns
a controllable latent variable to capture and mix generalized factor-related
properties. Different factor mixtures lead to diverse styles and hence further
differentiate generated poems from each other. Experiment results on Chinese
poetry demonstrate that MixPoet improves both diversity and quality against
three state-of-the-art models.

该研究提出了一种新方法 MixPoet，它通过半监督变分自编码器，将潜空间分解成多个子空间，并对每个子空间进行不同的因素训练，从而生成不同的诗歌风格以提高生成诗歌的多样性和质量，该方法在中国诗歌上的实验结果表明其相对于现有的三种最先进的模型具有更高的多样性和质量。