Automatically predicting the outcome of subjective listening tests is a
challenging task. Ratings may vary from person to person even if preferences
are consistent across listeners. While previous work has focused on predicting
listeners' ratings (mean opinion scores) of individual stimuli, we focus on the
simpler task of predicting subjective preference given two speech stimuli for
the same text. We propose a model based on anti-symmetric twin neural networks,
trained on pairs of waveforms and their corresponding preference scores. We
explore both attention and recurrent neural nets to account for the fact that
stimuli in a pair are not time aligned. To obtain a large training set we
convert listeners' ratings from MUSHRA tests to values that reflect how often
one stimulus in the pair was rated higher than the other. Specifically, we
evaluate performance on data obtained from twelve MUSHRA evaluations conducted
over five years, containing different TTS systems, built from data of different
speakers. Our results compare favourably to a state-of-the-art model trained to
predict MOS scores.

本文提出基于反对称孪生神经网络的模型来自动预测主观听测试的结果，通过探索关注力和循环神经网络，我们在来自五年的 12 个 MUSHRA 评估数据上进行测试，该数据包含不同的 TTS 系统和说话人，并将听众的评分转换为值以反映一种刺激是否被评为优于另一种刺激，与现有的预测 MOS 得分的最新模型相比，我们的结果明显更好。