Recent innovations in self-supervised representation learning have led to
remarkable advances in natural language processing. That said, in the speech
processing domain, self-supervised representation learning-based systems are
not yet considered state-of-the-art. We propose leveraging recent advances in
self-supervised-based speech processing to create a common speech analysis
engine. Such an engine should be able to handle multiple speech processing
tasks, using a single architecture, to obtain state-of-the-art accuracy. The
engine must also enable support for new tasks with small training datasets.
Beyond that, a common engine should be capable of supporting distributed
training with client in-house private data. We present the architecture for a
common speech analysis engine based on the HuBERT self-supervised speech
representation. Based on experiments, we report our results for language
identification and emotion recognition on the standard evaluations NIST-LRE 07
and IEMOCAP. Our results surpass the state-of-the-art performance reported so
far on these tasks. We also analyzed our engine on the emotion recognition task
using reduced amounts of training data and show how to achieve improved
results.

本文提出了基于 HuBERT 自监督语音表示法的通用语音分析引擎架构，能够处理多个语音处理任务并支持小型数据集的新任务。实验结果表明，在语言识别和情感识别任务上，该引擎的表现超过了现有的最高水平，并且在使用少量训练数据的情感识别任务上也具有改进的能力。