Composing poetry or lyrics involves several creative factors, but a
challenging aspect of generation is the adherence to a more or less strict
metric and rhyming pattern. To address this challenge specifically, previous
work on the task has mainly focused on reverse language modeling, which brings
the critical selection of each rhyming word to the forefront of each verse. On
the other hand, reversing the word order requires that models be trained from
scratch with this task-specific goal and cannot take advantage of transfer
learning from a Pretrained Language Model (PLM). We propose a novel fine-tuning
approach that prepends the rhyming word at the start of each lyric, which
allows the critical rhyming decision to be made before the model commits to the
content of the lyric (as during reverse language modeling), but maintains
compatibility with the word order of regular PLMs as the lyric itself is still
generated in left-to-right order. We conducted extensive experiments to compare
this fine-tuning against the current state-of-the-art strategies for rhyming,
finding that our approach generates more readable text and better rhyming
capabilities. Furthermore, we furnish a high-quality dataset in English and 12
other languages, analyse the approach's feasibility in a multilingual context,
provide extensive experimental results shedding light on good and bad practices
for lyrics generation, and propose metrics to compare methods in the future.

生成诗歌或歌词涉及到几个创作因素，而其中一个具有挑战性的方面是对较严格的韵律和押韵模式的遵循。为了解决这一挑战，之前的工作主要集中在逆向语言建模上，我们提出了一种新的微调方法，通过将押韵的词语放在每个歌词的开头，使关键的押韵决策可以在模型承诺歌词内容之前进行，与常规的预训练语言模型的词序兼容。我们进行了广泛的实验，与当前最先进的押韵策略进行了比较，发现我们的方法生成的文本更可读，韵律能力更好。此外，我们提供了一个高质量的英文和其他 12 种语言的数据集，分析了该方法在多语言环境中的可行性，提供了大量实验结果，阐明了歌词生成的良好和不良实践，并提出了比较方法的度量标准。