State-of-the-art trainable machine translation evaluation metrics like xCOMET
achieve high correlation with human judgment but rely on large encoders (up to
10.7B parameters), making them computationally expensive and inaccessible to
researchers with limited resources. To address this issue, we investigate
whether the knowledge stored in these large encoders can be compressed while
maintaining quality. We employ distillation, quantization, and pruning
techniques to create efficient xCOMET alternatives and introduce a novel data
collection pipeline for efficient black-box distillation. Our experiments show
that, using quantization, xCOMET can be compressed up to three times with no
quality degradation. Additionally, through distillation, we create an
xCOMET-lite metric, which has only 2.6% of xCOMET-XXL parameters, but retains
92.1% of its quality. Besides, it surpasses strong small-scale metrics like
COMET-22 and BLEURT-20 on the WMT22 metrics challenge dataset by 6.4%, despite
using 50% fewer parameters. All code, dataset, and models are available online.

利用精简和压缩技术，我们创建了高效的 xCOMET 替代方法，并使用黑盒蒸馏数据收集管道，实验证明，经过量化处理后，xCOMET 可以压缩至原体积的三分之一，且质量不降低；通过蒸馏，我们创建了一个 xCOMET-lite 度量方法，它仅含有 xCOMET-XXL 参数的 2.6%，但保持了 92.1% 的质量，并且在 WMT22 度量挑战数据集上比 COMET-22 和 BLEURT-20 等小规模方法提高了 6.4%，即使用的参数量减少了 50%。