Large language models (LLMs) have showcased remarkable reasoning
capabilities, yet they remain susceptible to errors, particularly in temporal
reasoning tasks involving complex temporal logic. Existing research has
explored LLM performance on temporal reasoning using diverse datasets and
benchmarks. However, these studies often rely on real-world data that LLMs may
have encountered during pre-training or employ anonymization techniques that
can inadvertently introduce factual inconsistencies. In this work, we address
these limitations by introducing novel synthetic datasets specifically designed
to assess LLM temporal reasoning abilities in various scenarios. The diversity
of question types across these datasets enables systematic investigation into
the impact of the problem structure, size, question type, fact order, and other
factors on LLM performance. Our findings provide valuable insights into the
strengths and weaknesses of current LLMs in temporal reasoning tasks. To foster
further research in this area, we are open-sourcing the datasets and evaluation
framework used in our experiments: this https URL

针对大型语言模型在时间推理任务中的性能，在引入新颖的合成数据集的基础上，对问题结构、尺寸、问题类型、事实顺序等因素对大型语言模型性能的影响进行了系统研究，从而提供了对当前大型语言模型在时间推理任务中优点和不足的有价值洞察。