Recent advancements in Large Language Models (LLMs) have significantly
enhanced their ability to generate and manipulate human language, highlighting
their potential across various applications. Evaluating LLMs in languages other
than English is crucial for ensuring their linguistic versatility, cultural
relevance, and applicability in diverse global contexts, thus broadening their
usability and effectiveness. We tackle this challenge by introducing a
structured benchmark using the INVALSI tests, a set of well-established
assessments designed to measure educational competencies across Italy. Our
study makes three primary contributions: Firstly, we adapt the INVALSI
benchmark for automated LLM evaluation, which involves rigorous adaptation of
the test format to suit automated processing while retaining the essence of the
original tests. Secondly, we provide a detailed assessment of current LLMs,
offering a crucial reference point for the academic community. Finally, we
visually compare the performance of these models against human results.
Additionally, researchers are invited to submit their models for ongoing
evaluation, ensuring the benchmark remains a current and valuable resource.

该研究介绍一种基于 ITALIAN EDUCATIONAL COMPETENCIES 的结构化基准测试来评估大型语言模型（LLMs），并提供它们在 LINGUISTIC VERSATILITY、CULTURAL RELEVANCE 和 BENCHMARK EVALUATION 等方面的性能比较。