Math Word Problems (MWPs) are crucial for evaluating the capability of Large
Language Models (LLMs), with current research primarily focusing on questions
with concise contexts. However, as real-world math problems often involve
complex circumstances, LLMs' ability to solve long MWPs is vital for their
applications in these scenarios, yet remains under-explored. This study
pioneers the exploration of Context Length Generalizability (CoLeG), the
ability of LLMs to solve long MWPs. We introduce Extended Grade-School Math
(E-GSM), a collection of MWPs with lengthy narratives. Two novel metrics are
proposed to assess the efficacy and resilience of LLMs in solving these
problems. Our examination of existing zero-shot prompting techniques and both
proprietary and open-source LLMs reveals a general deficiency in CoLeG. To
alleviate these challenges, we propose distinct approaches for different
categories of LLMs. For proprietary LLMs, a new instructional prompt is
proposed to mitigate the influence of long context. For open-source LLMs, a new
data augmentation task is developed to improve CoLeG. Our comprehensive results
demonstrate the effectiveness of our proposed methods, showing not only
improved performance on E-GSM but also generalizability across several other
MWP benchmarks. Our findings pave the way for future research in employing LLMs
for complex, real-world applications, offering practical solutions to current
limitations and opening avenues for further exploration of model
generalizability and training methodologies.

本研究探索了大型语言模型（LLMs）解决长篇数学问题的能力，引入了扩展的小学数学（E-GSM）问题集并提出了新的度量方法，旨在改善 LLMs 在解决这类问题上的表现。研究结果表明，所提出的方法不仅在 E-GSM 上取得了改善，还具有一定的推广性，为将 LLMs 应用于复杂的现实应用提供了实际解决方案，并为模型广泛性和训练方法的进一步探索开辟了新的途径。