Due to the high performance of multi-channel speech processing, we can use the outputs from a multi-channel model as teacher labels when training a single-channel model with knowledge distillation. To the contrary, it is also known that single-channel speech data can benefit multi-channel models by mixing it with multi-channel speech data during training or by using it for model pretraining. This paper focuses on speaker diarization and proposes to conduct the above bi-directional knowledge transfer alternately. We first introduce an end-to-end neural diarization model that can handle both single- and multi-channel inputs. Using this model, we alternately conduct i) knowledge distillation from a multi-channel model to a single-channel model and ii) finetuning from the distilled single-channel model to a multi-channel model. Experimental results on two-speaker data show that the proposed method mutually improved single- and multi-channel speaker diarization performances.

本文介绍了一种可处理单通道和多通道输入的端对端神经对话模型，并提出了一种双向知识传输的方法，既从多通道模型向单通道模型进行知识蒸馏，又从蒸馏后的单通道模型向多通道模型进行微调，实验结果表明该方法相互提高了单通道和多通道演讲者分离的性能。

单/多通道端到端神经分离的互相学习