Many Natural Language Processing and Computational Linguistics applications involves the generation of new texts based on some existing texts, such as summarization, text simplification and machine translation. However, there has been a serious problem haunting these applications for decades, that is, how to automatically and accurately assess quality of these applications. In this paper, we will present some preliminary results on one especially useful and challenging problem in NLP system evaluation: how to pinpoint content differences of two text passages (especially for large pas-sages such as articles and books). Our idea is intuitive and very different from existing approaches. We treat one text passage as a small knowledge base, and ask it a large number of questions to exhaustively identify all content points in it. By comparing the correctly answered questions from two text passages, we will be able to compare their content precisely. The experiment using 2007 DUC summarization corpus clearly shows promising results.

本文提出了一种基于问题解答的方法，通过将文本视为小型知识库进行大量提问，以精确地比较两个文本之间的内容差异，从而解决NLP系统评估中的一个重要问题。实验结果表明，该方法在分析大型文本语料库方面具有较高的准确性和可靠性。

一种基于语义问答的文本摘要评估方法