Text simplification aims to make technical texts more accessible to laypeople
but often results in deletion of information and vagueness. This work proposes
InfoLossQA, a framework to characterize and recover simplification-induced
information loss in form of question-and-answer (QA) pairs. Building on the
theory of Question Under Discussion, the QA pairs are designed to help readers
deepen their knowledge of a text. We conduct a range of experiments with this
framework. First, we collect a dataset of 1,000 linguist-curated QA pairs
derived from 104 LLM simplifications of scientific abstracts of medical
studies. Our analyses of this data reveal that information loss occurs
frequently, and that the QA pairs give a high-level overview of what
information was lost. Second, we devise two methods for this task: end-to-end
prompting of open-source and commercial language models, and a natural language
inference pipeline. With a novel evaluation framework considering the
correctness of QA pairs and their linguistic suitability, our expert evaluation
reveals that models struggle to reliably identify information loss and applying
similar standards as humans at what constitutes information loss.

文本简化旨在使技术文本对普通读者更易理解，但常常导致信息缺失和模糊。本研究提出了 InfoLossQA 框架，以问答对的形式描述和恢复由于简化而导致的信息缺失，并基于 Question Under Discussion 理论设计问答对，以帮助读者加深对文本的理解。通过一系列实验，我们收集了一千个由语言学家策划的医学研究科学论文摘要的 LLM 简化所生成的问答对数据集。分析结果显示信息缺失频繁发生，而问答对能够提供信息损失的高级概述。我们提出了两种方法来解决这个问题：基于开源和商业语言模型的端到端提示以及自然语言推理流程。通过考虑问答对的正确性和语言适宜性的全新评估框架，我们的专家评估结果显示模型在可靠地识别信息损失上存在困难，并且在确定信息损失的标准方面与人类存在类似的问题。