Annually, research teams spend large amounts of money to evaluate the quality
of machine translation systems (WMT, inter alia). This is expensive because it
requires detailed human labor. The recently proposed annotation protocol, Error
Span Annotation (ESA), has annotators marking erroneous parts of the
translation. In our work, we help the annotators by pre-filling the span
annotations with automatic quality estimation. With AI assistance, we obtain
more detailed annotations while cutting down the time per span annotation by
half (71s/error span $\rightarrow$ 31s/error span). The biggest advantage of
ESA$^\mathrm{AI}$ protocol is an accurate priming of annotators (pre-filled
error spans) before they assign the final score as opposed to starting from
scratch. In addition, the annotation budget can be reduced by up to 24% with
filtering of examples that the AI deems to be very likely to be correct.

机器翻译系统的质量评估被提出的错误范围标注协议辅助，使用自动质量估计填充范围注释可以提供更详细的注释并将每个范围注释的时间缩短了一半，此外还可以减少最多 24% 的注释预算。