Recent advancements in Large Language Models (LLMs) have demonstrated
impressive capabilities across a range of natural language processing tasks,
especially in reasoning, a cornerstone for achieving Artificial General
Intelligence (AGI). However, commonly used benchmarks may not fully encapsulate
the inferential abilities of these models in real-world scenarios. To address
this gap, a new form of Question-Answering (QA) task, termed Reasoning with
Redundant Information Provided (RRIP), is introduced. The study designed a
modified version of the grade school math 8K (GSM-8K) dataset which has several
variants focusing on different attributes of redundant information. This
investigation evaluates two popular LLMs, LlaMA2-13B-chat and generative
pre-trained transformer 3.5 (GPT-3.5), contrasting their performance on
traditional QA tasks against the RRIP tasks. Findings indicate that while these
models achieved moderate success on standard QA benchmarks, their performance
notably declines when assessed on RRIP tasks. The study not only highlights the
limitations of current LLMs in handling redundant information but also suggests
that future training of these models should focus on incorporating redundant
information into the training data to increase the performance on RRIP tasks.

最近在大规模语言模型 (LLMs) 的研究中取得了令人印象深刻的进展，尤其在推理方面，对于实现人工智能 (AGI) 至关重要。然而，常用的测试基准可能无法完全展现这些模型在现实场景中的推理能力。为了解决这一问题，引入了一种新形式的问答任务，称为 Reasoning with Redundant Information Provided (RRIP)。该研究设计了改进版本的小学数学 8K (GSM-8K) 数据集，它有几个变体，关注冗余信息的不同属性。在传统的问答任务和 RRIP 任务上，评估了两个流行的 LLMs，分别是 LlaMA2-13B-chat 和生成预训练转换器 3.5（GPT-3.5）。研究结果表明，尽管这些模型在标准问答基准中取得了一定的成功，但它们在 RRIP 任务上的表现明显下降。该研究不仅突出了当前 LLMs 在处理冗余信息方面的局限性，还建议未来训练这些模型时应将冗余信息纳入训练数据，以提高在 RRIP 任务上的性能。