Dubbed series are gaining a lot of popularity in recent years with strong
support from major media service providers. Such popularity is fueled by
studies that showed that dubbed versions of TV shows are more popular than
their subtitled equivalents. We propose an unsupervised approach to construct
speech-to-speech corpus, aligned on short segment levels, to produce a parallel
speech corpus in the source- and target- languages. Our methodology exploits
video frames, speech recognition, machine translation, and noisy frames removal
algorithms to match segments in both languages. To verify the performance of
the proposed method, we apply it on long and short dubbed clips. Out of 36
hours TR-AR dubbed series, our pipeline was able to generate 17 hours of paired
segments, which is about 47% of the corpus. We applied our method on another
language pair, EN-AR, to ensure it is robust enough and not tuned for a
specific language or a specific corpus. Regardless of the language pairs, the
accuracy of the paired segments was around 70% when evaluated using human
subjective evaluation. The corpus will be freely available for the research
community.

本研究提出了一种无监督方法来构建语音到语音语料库，并利用视频帧、语音识别、机器翻译和噪声帧去除算法在短片段层面上对齐源语言和目标语言，以生成一种平行语音语料库，并应用于短音频片段和长音频剪辑，效果表现稳健。