The evaluation of text simplification (TS) systems remains an open challenge.
As the task has common points with machine translation (MT), TS is often
evaluated using MT metrics such as BLEU. However, such metrics require high
quality reference data, which is rarely available for TS. TS has the advantage
over MT of being a monolingual task, which allows for direct comparisons to be
made between the simplified text and its original version. In this paper, we
compare multiple approaches to reference-less quality estimation of
sentence-level text simplification systems, based on the dataset used for the
QATS 2016 shared task. We distinguish three different dimensions:
gram-maticality, meaning preservation and simplicity. We show that n-gram-based
MT metrics such as BLEU and METEOR correlate the most with human judgment of
grammaticality and meaning preservation, whereas simplicity is best evaluated
by basic length-based metrics.

本文探讨了针对在无参考文本的情况下，对简化文本进行质量估计的多种方法，并基于 QATS 2016 的数据集对多个方法进行了比较，最终发现基于 n-gram 的机器翻译度量（如 BLEU 和 METEOR）最能与语法正确性和意义保留的人工评估相匹配，而基于长度的度量方法则最适合衡量简明易懂程度。