Significant strides have been made in creating voice identity representations
using speech data. However, the same level of progress has not been achieved
for singing voices. To bridge this gap, we suggest a framework for training
singer identity encoders to extract representations suitable for various
singing-related tasks, such as singing voice similarity and synthesis. We
explore different self-supervised learning techniques on a large collection of
isolated vocal tracks and apply data augmentations during training to ensure
that the representations are invariant to pitch and content variations. We
evaluate the quality of the resulting representations on singer similarity and
identification tasks across multiple datasets, with a particular emphasis on
out-of-domain generalization. Our proposed framework produces high-quality
embeddings that outperform both speaker verification and wav2vec 2.0
pre-trained baselines on singing voice while operating at 44.1 kHz. We release
our code and trained models to facilitate further research on singing voice and
related areas.

我们提出了一个框架，通过在大量的孤立音轨上应用不同的自监督学习技术以及数据增强，训练歌手身份编码器以提取适用于各种歌唱相关任务（如歌声相似性和合成）的表示，我们评估了产生的表示在多个数据集上进行歌手相似性和识别任务，并重点关注领域外泛化，我们的框架在 44.1 kHz 下产生高质量的嵌入，优于说话人验证和 wav2vec 2.0 预训练基线，在歌唱声音上，并发布了我们的代码和训练模型，以促进对歌唱声音和相关领域的进一步研究。

使用自监督技术学习歌手身份表示

Singer Identity Representation Learning using Self-Supervised Techniques

Textless speech-to-speech translation systems are rapidly advancing, thanks
to the integration of self-supervised learning techniques. However, existing
state-of-the-art systems fall short when it comes to capturing and transferring
expressivity accurately across different languages. Expressivity plays a vital
role in conveying emotions, nuances, and cultural subtleties, thereby enhancing
communication across diverse languages. To address this issue this study
presents a novel method that operates at the discrete speech unit level and
leverages multilingual emotion embeddings to capture language-agnostic
information. Specifically, we demonstrate how these embeddings can be used to
effectively predict the pitch and duration of speech units in the target
language. Through objective and subjective experiments conducted on a
French-to-English translation task, our findings highlight the superior
expressivity transfer achieved by our approach compared to current
state-of-the-art systems.

通过离散语音单元级别的多语情感嵌入，以及有效地预测目标语言中的语音单元的音高和持续时间，本研究展示了一种能够有效捕捉跨不同语言的表达能力的新方法，通过客观和主观实验，结果显示本方法在提高传达情感和文化细微差别方面优于当前先进系统的表达能力转移。

提升无文字语音翻译中的表达能力转移

Enhancing expressivity transfer in textless speech-to-speech translation

Continuous unsupervised representation learning (CURL) research has greatly
benefited from improvements in self-supervised learning (SSL) techniques. As a
result, existing CURL methods using SSL can learn high-quality representations
without any labels, but with a notable performance drop when learning on a
many-tasks data stream. We hypothesize that this is caused by the
regularization losses that are imposed to prevent forgetting, leading to a
suboptimal plasticity-stability trade-off: they either do not adapt fully to
the incoming data (low plasticity), or incur significant forgetting when
allowed to fully adapt to a new SSL pretext-task (low stability). In this work,
we propose to train an expert network that is relieved of the duty of keeping
the previous knowledge and can focus on performing optimally on the new tasks
(optimizing plasticity). In the second phase, we combine this new knowledge
with the previous network in an adaptation-retrospection phase to avoid
forgetting and initialize a new expert with the knowledge of the old network.
We perform several experiments showing that our proposed approach outperforms
other CURL exemplar-free methods in few- and many-task split settings.
Furthermore, we show how to adapt our approach to semi-supervised continual
learning (Semi-SCL) and show that we surpass the accuracy of other
exemplar-free Semi-SCL methods and reach the results of some others that use
exemplars.

通过训练专家网络和结合自适应回顾相位的方法来解决在多任务数据流上学习时的性能下降问题，该方法在少任务和多任务划分设置中优于其他无示例连续无监督表征学习方法，并且在半监督连续学习方面表现优于其他无示例半监督连续学习方法。