In this paper, we explore the encoding/pooling layer and loss function in the end-to-end speaker and language recognition system. First, a unified and interpretable end-to-end system for both speaker and language recognition is developed. It accepts variable-length input and produces an utterance level result. In the end-to-end system, the encoding layer plays a role in aggregating the variable-length input sequence into an utterance level representation. Besides the basic temporal average pooling, we introduce a self-attentive pooling layer and a learnable dictionary encoding layer to get the utterance level representation. In terms of loss function for open-set speaker verification, to get more discriminative speaker embedding, center loss and angular softmax loss is introduced in the end-to-end system. Experimental results on Voxceleb and NIST LRE 07 datasets show that the performance of end-to-end learning system could be significantly improved by the proposed encoding layer and loss function.

本研究探索了编码层、池化层和损失函数在端到端语音说话人和语言识别系统中的作用，开发了一个用于语音说话人和语言识别系统的端到端统一的、可解释的系统，并对编码层和损失函数进行了改进，实验结果表明改进后的端到端学习系统显著提高了性能。

探索端到端说话人和语言识别系统中的编码层和损失函数