Lyrics-to-melody generation is an interesting and challenging topic in AI
music research field. Due to the difficulty of learning the correlations
between lyrics and melody, previous methods suffer from low generation quality
and lack of controllability. Controllability of generative models enables human
interaction with models to generate desired contents, which is especially
important in music generation tasks towards human-centered AI that can
facilitate musicians in creative activities. To address these issues, we
propose a controllable lyrics-to-melody generation network, ConL2M, which is
able to generate realistic melodies from lyrics in user-desired musical style.
Our work contains three main novelties: 1) To model the dependencies of music
attributes cross multiple sequences, inter-branch memory fusion (Memofu) is
proposed to enable information flow between multi-branch stacked LSTM
architecture; 2) Reference style embedding (RSE) is proposed to improve the
quality of generation as well as control the musical style of generated
melodies; 3) Sequence-level statistical loss (SeqLoss) is proposed to help the
model learn sequence-level features of melodies given lyrics. Verified by
evaluation metrics for music quality and controllability, initial study of
controllable lyrics-to-melody generation shows better generation quality and
the feasibility of interacting with users to generate the melodies in desired
musical styles when given lyrics.

我们提出了一种可控的歌词到旋律生成网络，能够以用户期望的音乐风格从歌词中生成逼真的旋律，通过验证各项指标，初始控制生成的旋律的研究表明具有更好的生成质量和与用户的互动性。

可控歌词旋律生成

Controllable Lyrics-to-Melody Generation

Deep generative systems that learn probabilistic models from a corpus of
existing music do not explicitly encode knowledge of a musical style, compared
to traditional rule-based systems. Thus, it can be difficult to determine
whether deep models generate stylistically correct output without expert
evaluation, but this is expensive and time-consuming. Therefore, there is a
need for automatic, interpretable, and musically-motivated evaluation measures
of generated music. In this paper, we introduce a grading function that
evaluates four-part chorales in the style of J.S. Bach along important musical
features. We use the grading function to evaluate the output of a Transformer
model, and show that the function is both interpretable and outperforms human
experts at discriminating Bach chorales from model-generated ones.

本文介绍了一种基于评分函数的评估方法，该评估方法评估四声部赞美诗是否符合巴赫音乐风格，使用该方法评估了 Transformer 模型的输出，证明该函数具有解释性且在区分巴赫和模型生成的赞美诗方面优于人类专家。