As more and more data is collected in various settings across organizations,
companies, and countries, there has been an increase in the demand of user
privacy. Developing privacy preserving methods for data analytics is thus an
important area of research. In this work we present a model based on generative
adversarial networks (GANs) that learns to obfuscate specific sensitive
attributes in speech data. We train a model that learns to hide sensitive
information in the data, while preserving the meaning in the utterance. The
model is trained in two steps: first to filter sensitive information in the
spectrogram domain, and then to generate new and private information
independent of the filtered one. The model is based on a U-Net CNN that takes
mel-spectrograms as input. A MelGAN is used to invert the spectrograms back to
raw audio waveforms. We show that it is possible to hide sensitive information
such as gender by generating new data, trained adversarially to maintain
utility and realism.

通过生成式对抗网络（GANs）的模型设计，训练一个 U 型卷积神经网络，用于隐藏声音数据中的敏感个人信息，例如性别等。

面向私有语音生成的对抗表示学习

Adversarial representation learning for private speech generation

Recently, Convolutional Neural Network (CNN) and Long short-term memory
(LSTM) based models have been introduced to deep learning-based target speaker
separation. In this paper, we propose an Attention-based neural network
(Atss-Net) in the spectrogram domain for the task. It allows the network to
compute the correlation between each feature parallelly, and using shallower
layers to extract more features, compared with the CNN-LSTM architecture.
Experimental results show that our Atss-Net yields better performance than the
VoiceFilter, although it only contains half of the parameters. Furthermore, our
proposed model also demonstrates promising performance in speech enhancement.

本文提出了一种基于注意力机制的神经网络（Atss-Net）来进行基于深度学习的目标说话人分离和语音增强，实验结果表明 Atss-Net 在频谱图领域比 CNN-LSTM 架构更有效，并且在语音增强方面也具有良好的性能。

基于注意力神经网络的目标说话人分离

Atss-Net: Target Speaker Separation via Attention-based Neural Network

We explore self-supervised models that can be potentially deployed on mobile
devices to learn general purpose audio representations. Specifically, we
propose methods that exploit the temporal context in the spectrogram domain.
One method estimates the temporal gap between two short audio segments
extracted at random from the same audio clip. The other methods are inspired by
Word2Vec, a popular technique used to learn word embeddings, and aim at
reconstructing a temporal spectrogram slice from past and future slices or,
alternatively, at reconstructing the context of surrounding slices from the
current slice. We focus our evaluation on small encoder architectures, which
can be potentially run on mobile devices during both inference (re-using a
common learned representation across multiple downstream tasks) and training
(capturing the true data distribution without compromising users' privacy when
combined with federated learning). We evaluate the quality of the embeddings
produced by the self-supervised learning models, and show that they can be
re-used for a variety of downstream tasks, and for some tasks even approach the
performance of fully supervised models of similar size.

我们探索了可以在移动设备上部署的自监督模型来学习通用音频表示，具体而言，我们提出了利用频谱图域中的时间上下文的方法。一种方法估计从同一音频剪辑中随机提取的两个短音频片段之间的时间间隔。其他两种方法受 Word2Vec 的启发，旨在从过去和未来切片中重建时间频谱图切片，或者从当前切片重建周围切片的上下文。我们关注使用小型编码器体系结构的评估，这些体系结构可以在推理（在多个下游任务中重复使用共同学习的表示）和训练（与联合学习结合使用时捕获真实数据分布而不损害用户隐私）期间潜在运行于移动设备。我们评估了自我监督学习模型产生的嵌入质量，并表明它们可以被重新用于各种下游任务，甚至对一些任务的性能接近相似大小的全监督模型。