In this paper, we propose a new metric for Machine Translation (MT)
evaluation, based on bi-directional entailment. We show that machine generated
translation can be evaluated by determining paraphrasing with a reference
translation provided by a human translator. We hypothesize, and show through
experiments, that paraphrasing can be detected by evaluating entailment
relationship in the forward and backward direction. Unlike conventional
metrics, like BLEU or METEOR, our approach uses deep learning to determine the
semantic similarity between candidate and reference translation for generating
scores rather than relying upon simple n-gram overlap. We use BERT's
pre-trained implementation of transformer networks, fine-tuned on MNLI corpus,
for natural language inferencing. We apply our evaluation metric on WMT'14 and
WMT'17 dataset to evaluate systems participating in the translation task and
find that our metric has a better correlation with the human annotated score
compared to the other traditional metrics at system level.

本文提出了一种基于双向蕴含的机器翻译评估新指标，利用深度学习实现候选和参考翻译之间的语义相似度评分，应用于 WMT'14 和 WMT'17 数据集，与传统指标相比在系统级别上具有更好的与人工注释评分的相关性。