Existing multi-channel continuous speech separation (CSS) models are heavily
dependent on supervised data - either simulated data which causes data mismatch
between the training and real-data testing, or the real transcribed overlapping
data, which is difficult to be acquired, hindering further improvements in the
conversational/meeting transcription tasks. In this paper, we propose a
three-stage training scheme for the CSS model that can leverage both supervised
data and extra large-scale unsupervised real-world conversational data. The
scheme consists of two conventional training approaches -- pre-training using
simulated data and ASR-loss-based training using transcribed data -- and a
novel continuous semi-supervised training between the two, in which the CSS
model is further trained by using real data based on the teacher-student
learning framework. We apply this scheme to an array-geometry-agnostic CSS
model, which can use the multi-channel data collected from any microphone
array. Large-scale meeting transcription experiments are carried out on both
Microsoft internal meeting data and the AMI meeting corpus. The steady
improvement by each training stage has been observed, showing the effect of the
proposed method that enables leveraging real conversational data for CSS model
training.

本研究提出一种可同时利用有标注数据和非标注数据进行连续语音分离模型训练的三阶段训练方案，并将其应用于任意麦克风阵列采集的多通道语音数据上取得了较好效果。

利用真实对话数据实现多渠道连续语音分离

Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation

Continuous speech separation plays a vital role in complicated speech related
tasks such as conversation transcription. The separation model extracts a
single speaker signal from a mixed speech. In this paper, we use transformer
and conformer in lieu of recurrent neural networks in the separation system, as
we believe capturing global information with the self-attention based method is
crucial for the speech separation. Evaluating on the LibriCSS dataset, the
conformer separation model achieves state of the art results, with a relative
23.5% word error rate (WER) reduction from bi-directional LSTM (BLSTM) in the
utterance-wise evaluation and a 15.4% WER reduction in the continuous
evaluation.

本文研究使用 Transformer 和 Conformer 代替递归神经网络在语音分离领域中来提取单个说话者的混合语音，以连续语音分离为主要研究方向，通过在 LibriCSS 数据集上的实验，证明所提模型在语音分离任务中具有最优表现。

使用 Conformer 进行连续语音分离

Continuous Speech Separation with Conformer

This paper describes a dataset and protocols for evaluating continuous speech
separation algorithms. Most prior studies on speech separation use
pre-segmented signals of artificially mixed speech utterances which are mostly
\emph{fully} overlapped, and the algorithms are evaluated based on
signal-to-distortion ratio or similar performance metrics. However, in natural
conversations, a speech signal is continuous, containing both overlapped and
overlap-free components. In addition, the signal-based metrics have very weak
correlations with automatic speech recognition (ASR) accuracy. We think that
not only does this make it hard to assess the practical relevance of the tested
algorithms, it also hinders researchers from developing systems that can be
readily applied to real scenarios. In this paper, we define continuous speech
separation (CSS) as a task of generating a set of non-overlapped speech signals
from a \textit{continuous} audio stream that contains multiple utterances that
are \emph{partially} overlapped by a varying degree. A new real recorded
dataset, called LibriCSS, is derived from LibriSpeech by concatenating the
corpus utterances to simulate a conversation and capturing the audio replays
with far-field microphones. A Kaldi-based ASR evaluation protocol is also
established by using a well-trained multi-conditional acoustic model. By using
this dataset, several aspects of a recently proposed speaker-independent CSS
algorithm are investigated. The dataset and evaluation scripts are available to
facilitate the research in this direction.

这篇论文通过建立新的 LibriCSS 数据集并制定一套 Kaldi 基础的 ASR 评估方法，对连续语音分离算法进行了评估和研究，以解决自然对话中语音分离的问题。