Large Language Models (LLMs) offer the potential for automatic time series
analysis and reporting, which is a critical task across many domains, spanning
healthcare, finance, climate, energy, and many more. In this paper, we propose
a framework for rigorously evaluating the capabilities of LLMs on time series
understanding, encompassing both univariate and multivariate forms. We
introduce a comprehensive taxonomy of time series features, a critical
framework that delineates various characteristics inherent in time series data.
Leveraging this taxonomy, we have systematically designed and synthesized a
diverse dataset of time series, embodying the different outlined features. This
dataset acts as a solid foundation for assessing the proficiency of LLMs in
comprehending time series. Our experiments shed light on the strengths and
limitations of state-of-the-art LLMs in time series understanding, revealing
which features these models readily comprehend effectively and where they
falter. In addition, we uncover the sensitivity of LLMs to factors including
the formatting of the data, the position of points queried within a series and
the overall time series length.

提出了一个评估大型语言模型（LLM）在时间序列理解方面能力的框架，包括单变量和多变量形式；通过研究时间序列的特征以及设计和合成数据集来评估 LLMs 在时间序列理解方面的能力，并揭示了 LLMs 对数据格式、查询数据的位置和时间序列长度等因素的敏感性。

评估大型语言模型对时间序列特征理解的综合分类和基准

Evaluating Large Language Models on Time Series Feature Understanding: A  Comprehensive Taxonomy and Benchmark

Large language models (LLMs) have strong capabilities in solving diverse
natural language processing tasks. However, the safety and security issues of
LLM systems have become the major obstacle to their widespread application.
Many studies have extensively investigated risks in LLM systems and developed
the corresponding mitigation strategies. Leading-edge enterprises such as
OpenAI, Google, Meta, and Anthropic have also made lots of efforts on
responsible LLMs. Therefore, there is a growing need to organize the existing
studies and establish comprehensive taxonomies for the community. In this
paper, we delve into four essential modules of an LLM system, including an
input module for receiving prompts, a language model trained on extensive
corpora, a toolchain module for development and deployment, and an output
module for exporting LLM-generated content. Based on this, we propose a
comprehensive taxonomy, which systematically analyzes potential risks
associated with each module of an LLM system and discusses the corresponding
mitigation strategies. Furthermore, we review prevalent benchmarks, aiming to
facilitate the risk assessment of LLM systems. We hope that this paper can help
LLM participants embrace a systematic perspective to build their responsible
LLM systems.

通过探索大型语言模型系统的四个关键模块，即接收提示的输入模块，基于丰富语料库的语言模型，用于开发和部署的工具链模块以及生成语言模型内容的输出模块，本文提出了一种综合分类方法，系统分析了每个模块可能涉及的潜在风险，并讨论了相应的缓解策略。此外，我们还回顾了流行基准以促进大型语言模型系统的风险评估。希望本文能帮助大型语言模型参与者从系统角度构建负责任的系统。