From pre-trained language model (PLM) to large language model (LLM), the field of natural language processing (NLP) has witnessed steep performance gains and wide practical uses. The evaluation of a research field guides its direction of improvement. However, LLMs are extremely hard to thoroughly evaluate for two reasons. First of all, traditional NLP tasks become inadequate due to the excellent performance of LLM. Secondly, existing evaluation tasks are difficult to keep up with the wide range of applications in real-world scenarios. To tackle these problems, existing works proposed various benchmarks to better evaluate LLMs. To clarify the numerous evaluation tasks in both academia and industry, we investigate multiple papers concerning LLM evaluations. We summarize 4 core competencies of LLM, including reasoning, knowledge, reliability, and safety. For every competency, we introduce its definition, corresponding benchmarks, and metrics. Under this competency architecture, similar tasks are combined to reflect corresponding ability, while new tasks can also be easily added into the system. Finally, we give our suggestions on the future direction of LLM's evaluation.

从预训练语言模型（PLM）到大型语言模型（LLM），自然语言处理（NLP）领域已经取得了明显的性能提升和广泛的实际应用。为了解决评估LLM的困难，这篇论文调查了关于LLM评估的多篇论文，并总结了LLM的四个核心能力，包括推理、知识、可靠性和安全性。在这个能力结构下，相似的任务被合并以反映相应的能力，而新的任务也可以轻松地添加到系统中。最后，给出了关于LLM评估未来方向的建议。

核心竞争力视角下的大型语言模型评估调查