Dubbed series are gaining a lot of popularity in recent years with strong
support from major media service providers. Such popularity is fueled by
studies that showed that dubbed versions of TV shows are more popular than
their subtitled equivalents. We propose an unsupervised approach to construct
speech-to-speech corpus, aligned on short segment levels, to produce a parallel
speech corpus in the source- and target- languages. Our methodology exploits
video frames, speech recognition, machine translation, and noisy frames removal
algorithms to match segments in both languages. To verify the performance of
the proposed method, we apply it on long and short dubbed clips. Out of 36
hours TR-AR dubbed series, our pipeline was able to generate 17 hours of paired
segments, which is about 47% of the corpus. We applied our method on another
language pair, EN-AR, to ensure it is robust enough and not tuned for a
specific language or a specific corpus. Regardless of the language pairs, the
accuracy of the paired segments was around 70% when evaluated using human
subjective evaluation. The corpus will be freely available for the research
community.

本研究提出了一种无监督方法来构建语音到语音语料库，并利用视频帧、语音识别、机器翻译和噪声帧去除算法在短片段层面上对齐源语言和目标语言，以生成一种平行语音语料库，并应用于短音频片段和长音频剪辑，效果表现稳健。

从配音系列中创建语音到语音语料库

Creating Speech-to-Speech Corpus from Dubbed Series

User acceptance of artificial intelligence agents might depend on their
ability to explain their reasoning, which requires adding an interpretability
layer that fa- cilitates users to understand their behavior. This paper focuses
on adding an in- terpretable layer on top of Semantic Textual Similarity (STS),
which measures the degree of semantic equivalence between two sentences. The
interpretability layer is formalized as the alignment between pairs of segments
across the two sentences, where the relation between the segments is labeled
with a relation type and a similarity score. We present a publicly available
dataset of sentence pairs annotated following the formalization. We then
develop a system trained on this dataset which, given a sentence pair, explains
what is similar and different, in the form of graded and typed segment
alignments. When evaluated on the dataset, the system performs better than an
informed baseline, showing that the dataset and task are well-defined and
feasible. Most importantly, two user studies show how the system output can be
used to automatically produce explanations in natural language. Users performed
better when having access to the explanations, pro- viding preliminary evidence
that our dataset and method to automatically produce explanations is useful in
real applications.

本文研究了在语义文本相似度（STS）的基础上添加可解释性层，通过分析句对之间的对齐关系，给出了一种可度量关系的方法。同时，作者提供了一个公共的数据集来评估系统性能，研究表明该方法可用于进行自然语言的解释并提高用户的接受度。