Despite previous efforts in melody-to-lyric generation research, there is
still a significant compatibility gap between generated lyrics and melodies,
negatively impacting the singability of the outputs. This paper bridges the
singability gap with a novel approach to generating singable lyrics by jointly
Learning wOrding And Formatting during Melody-to-Lyric training (LOAF-M2L).
After general-domain pretraining, our proposed model acquires length awareness
first from a large text-only lyric corpus. Then, we introduce a new objective
informed by musicological research on the relationship between melody and
lyrics during melody-to-lyric training, which enables the model to learn the
fine-grained format requirements of the melody. Our model achieves 3.75% and
21.44% absolute accuracy gains in the outputs' number-of-line and
syllable-per-line requirements compared to naive fine-tuning, without
sacrificing text fluency. Furthermore, our model demonstrates a 63.92% and
74.18% relative improvement of music-lyric compatibility and overall quality in
the subjective evaluation, compared to the state-of-the-art melody-to-lyric
generation model, highlighting the significance of formatting learning.

本文提出一种新的方法 LOAF-M2L 来生成旋律与歌词之间兼容的歌词，并引入音乐学研究中的信息来帮助模型学习旋律的细粒度格式要求，在主观评估中相对于现有最先进的旋律到歌词生成模型显示出 63.92％和 74.18％的相对改进。

LOAF-M2L: 歌词生成器的歌词和格式化的联合学习

LOAF-M2L: Joint Learning of Wording and Formatting for Singable  Melody-to-Lyric Generation

Automatic melody-to-lyric generation is a task in which song lyrics are
generated to go with a given melody. It is of significant practical interest
and more challenging than unconstrained lyric generation as the music imposes
additional constraints onto the lyrics. The training data is limited as most
songs are copyrighted, resulting in models that underfit the complicated
cross-modal relationship between melody and lyrics. In this work, we propose a
method for generating high-quality lyrics without training on any aligned
melody-lyric data. Specifically, we design a hierarchical lyric generation
framework that first generates a song outline and second the complete lyrics.
The framework enables disentanglement of training (based purely on text) from
inference (melody-guided text generation) to circumvent the shortage of
parallel data.
We leverage the segmentation and rhythm alignment between melody and lyrics
to compile the given melody into decoding constraints as guidance during
inference. The two-step hierarchical design also enables content control via
the lyric outline, a much-desired feature for democratizing collaborative song
creation. Experimental results show that our model can generate high-quality
lyrics that are more on-topic, singable, intelligible, and coherent than strong
baselines, for example SongMASS, a SOTA model trained on a parallel dataset,
with a 24% relative overall quality improvement based on human ratings. O

本研究提出一种层次化歌词生成框架，通过将已知旋律编码为解码约束并获得歌曲大纲和完整歌词的生成，实现未经过任何对齐的歌曲和歌词数据即可生成高质量歌词，并通过歌曲大纲实现内容控制。实验结果表明，本模型相对于 SongMASS 等强基线模型，基于人类评分获得了 24% 的整体质量改善。

无监督旋律 - 歌词生成

Unsupervised Melody-to-Lyric Generation

Automatic song writing aims to compose a song (lyric and/or melody) by
machine, which is an interesting topic in both academia and industry. In
automatic song writing, lyric-to-melody generation and melody-to-lyric
generation are two important tasks, both of which usually suffer from the
following challenges: 1) the paired lyric and melody data are limited, which
affects the generation quality of the two tasks, considering a lot of paired
training data are needed due to the weak correlation between lyric and melody;
2) Strict alignments are required between lyric and melody, which relies on
specific alignment modeling. In this paper, we propose SongMASS to address the
above challenges, which leverages masked sequence to sequence (MASS)
pre-training and attention based alignment modeling for lyric-to-melody and
melody-to-lyric generation. Specifically, 1) we extend the original
sentence-level MASS pre-training to song level to better capture long
contextual information in music, and use a separate encoder and decoder for
each modality (lyric or melody); 2) we leverage sentence-level attention mask
and token-level attention constraint during training to enhance the alignment
between lyric and melody. During inference, we use a dynamic programming
strategy to obtain the alignment between each word/syllable in lyric and note
in melody. We pre-train SongMASS on unpaired lyric and melody datasets, and
both objective and subjective evaluations demonstrate that SongMASS generates
lyric and melody with significantly better quality than the baseline method
without pre-training or alignment constraint.

本文提出了 SongMASS 方法来解决自动生成歌曲时歌词到旋律生成和旋律到歌词生成所面临的限制问题，使用基于掩蔽序列到序列预训练和基于注意力的对齐建模，表现出明显优于基线方法的歌词和旋律生成质量。