We introduce EXAMS-V, a new challenging multi-discipline multimodal
multilingual exam benchmark for evaluating vision language models. It consists
of 20,932 multiple-choice questions across 20 school disciplines covering
natural science, social science, and other miscellaneous studies, e.g.,
religion, fine arts, business, etc. EXAMS-V includes a variety of multimodal
features such as text, images, tables, figures, diagrams, maps, scientific
symbols, and equations. The questions come in 11 languages from 7 language
families. Unlike existing benchmarks, EXAMS-V is uniquely curated by gathering
school exam questions from various countries, with a variety of education
systems. This distinctive approach calls for intricate reasoning across diverse
languages and relies on region-specific knowledge. Solving the problems in the
dataset requires advanced perception and joint reasoning over the text and the
visual content of the image. Our evaluation results demonstrate that this is a
challenging dataset, which is difficult even for advanced vision-text models
such as GPT-4V and Gemini; this underscores the inherent complexity of the
dataset and its significance as a future benchmark.

我们推出了 EXAMS-V，一个新的多学科多模态多语言考试基准，用于评估视觉语言模型。它包含了 20932 个跨越自然科学、社会科学和其他各种学科的多项选择题，如宗教、美术、商务等。EXAMS-V 包含了多种多模态特征，例如文本、图像、表格、图表、图解、地图、科学符号和方程式。这些问题来自于 7 个语系的 11 种语言。与现有的基准不同，EXAMS-V 是通过收集各个国家的校园考试问题，采用多样化的教育体系进行精心策划的。这种独特的方法需要进行复杂的推理，涉及多种语言和地域特定的知识。解决数据集中的问题需要对文本和图像内容进行高级感知和联合推理。我们的评估结果表明，即使对于像 GPT-4V 和 Gemini 这样的先进视觉 - 文本模型，这也是一个具有挑战性的数据集；这凸显了数据集的内在复杂性以及作为未来基准的重要性。

EXAMS-V: 用于评估视觉语言模型的多学科多语言多模态考试基准

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for  Evaluating Vision Language Models

Education systems are dynamically changing to accommodate technological
advances, industrial and societal needs, and to enhance students' learning
journeys. Curriculum specialists and educators constantly revise taught
subjects across educational grades to identify gaps, introduce new learning
topics, and enhance the learning outcomes. This process is usually done within
the same subjects (e.g. math) or across related subjects (e.g. math and
physics) considering the same and different educational levels, leading to
massive multi-layer comparisons. Having nuanced data about subjects, topics,
and learning outcomes structured within a dataset, empowers us to leverage data
science to better understand the progression of various learning topics. In
this paper, Bidirectional Encoder Representations from Transformers (BERT)
topic modeling was used to extract topics from the curriculum, which were then
used to identify relationships between subjects, track their progression, and
identify conceptual gaps. We found that grouping learning outcomes by common
topics helped specialists reduce redundancy and introduce new concepts in the
curriculum. We built a dashboard to avail the methodology to curriculum
specials. Finally, we tested the validity of the approach with subject matter
experts.

使用 Bidirectional Encoder Representations from Transformers (BERT) 主题建模从课程中提取主题，并用于识别学科之间的关系、跟踪它们的进展和识别概念上的差距，研究发现按共同主题对学习成果进行分组有助于减少冗余并在课程中引入新概念。最后，使用主题专家对方法的有效性进行了测试。

透过语义匹配了解教育话题的进展

Understanding the Progression of Educational Topics via Semantic  Matching

This study aims to determine a predictive model to learn students probability
to pass their courses taken at the earliest stage of the semester. To
successfully discover a good predictive model with high acceptability,
accurate, and precision rate which delivers a useful outcome for decision
making in education systems, in improving the processes of conveying knowledge
and uplifting students academic performance, the proponent applies and strictly
followed the CRISP-DM (Cross-Industry Standard Process for Data Mining)
methodology. This study employs classification for data mining techniques, and
decision tree for algorithm. With the utilization of the newly discovered
predictive model, the prediction of students probabilities to pass the current
courses they take gives 0.7619 accuracy, 0.8333 precision, 0.8823 recall, and
0.8571 f1 score, which shows that the model used in the prediction is reliable,
accurate, and recommendable. Considering the indicators and the results, it can
be noted that the prediction model used in this study is highly acceptable. The
data mining techniques provides effective and efficient innovative tools in
analyzing and predicting student performances. The model used in this study
will greatly affect the way educators understand and identify the weakness of
their students in the class, the way they improved the effectiveness of their
learning processes gearing to their students, bring down academic failure
rates, and help institution administrators modify their learning system
outcomes. Further study for the inclusion of some students demographic
information, vast amount of data within the dataset, automated and manual
process of predictive criteria indicators where the students can regulate to
which criteria, they must improve more for them to pass their courses taken at
the end of the semester as early as midterm period are highly needed.

本研究旨在建立一个可预测学生是否能在学期初始阶段通过其所选课程的模型。使用数据挖掘技术和决策树算法的分类方法，我们发现这个预测模型具有高可接受性、准确性和精度，对于学生学业表现的推进和教育体系的改善有重要意义。