Lyrics alignment gained considerable attention in recent years.
State-of-the-art systems either re-use established speech recognition toolkits,
or design end-to-end solutions involving a Connectionist Temporal
Classification (CTC) loss. However, both approaches suffer from specific
weaknesses: toolkits are known for their complexity, and CTC systems use a loss
designed for transcription which can limit alignment accuracy. In this paper,
we use instead a contrastive learning procedure that derives cross-modal
embeddings linking the audio and text domains. This way, we obtain a novel
system that is simple to train end-to-end, can make use of weakly annotated
training data, jointly learns a powerful text model, and is tailored to
alignment. The system is not only the first to yield an average absolute error
below 0.2 seconds on the standard Jamendo dataset but it is also robust to
other languages, even when trained on English data only. Finally, we release
word-level alignments for the JamendoLyrics Multi-Lang dataset.

本文介绍了一种新颖的歌词对齐系统，使用对抗学习导出跨模态嵌入，不仅训练简单、能使用弱标注数据、学习强大的文本模型，而且可用于多语言和获得了标准数据集下平均绝对误差小于 0.2 秒的最佳结果。

基于对比学习的多语言音频与歌词对齐

Contrastive Learning-Based Audio to Lyrics Alignment for Multiple  Languages

Recent work on learning multilingual word representations usually relies on
the use of word-level alignements (e.g. infered with the help of GIZA++)
between translated sentences, in order to align the word embeddings in
different languages. In this workshop paper, we investigate an autoencoder
model for learning multilingual word representations that does without such
word-level alignements. The autoencoder is trained to reconstruct the
bag-of-word representation of given sentence from an encoded representation
extracted from its translation. We evaluate our approach on a multilingual
document classification task, where labeled data is available only for one
language (e.g. English) while classification must be performed in a different
language (e.g. French). In our experiments, we observe that our method compares
favorably with a previously proposed method that exploits word-level alignments
to learn word representations.

本文研究了一种自编码器模型，用于学习多语言单词表示，不需使用单词层面对齐。实验证明，该方法在多语言文档分类方面具有较好的表现，且与利用单词层面对齐的方法相比具有竞争力。