This study compared the classification performance of Gemini Pro and GPT-4V
in educational settings. Employing visual question answering (VQA) techniques,
the study examined both models' abilities to read text-based rubrics and then
automatically score student-drawn models in science education. We employed both
quantitative and qualitative analyses using a dataset derived from
student-drawn scientific models and employing NERIF (Notation-Enhanced Rubrics
for Image Feedback) prompting methods. The findings reveal that GPT-4V
significantly outperforms Gemini Pro in terms of scoring accuracy and Quadratic
Weighted Kappa. The qualitative analysis reveals that the differences may be
due to the models' ability to process fine-grained texts in images and overall
image classification performance. Even adapting the NERIF approach by further
de-sizing the input images, Gemini Pro seems not able to perform as well as
GPT-4V. The findings suggest GPT-4V's superior capability in handling complex
multimodal educational tasks. The study concludes that while both models
represent advancements in AI, GPT-4V's higher performance makes it a more
suitable tool for educational applications involving multimodal data
interpretation.

本研究通过使用视觉问答（VQA）技术比较了 Gemini Pro 和 GPT-4V 在教育环境下的分类表现，研究了这两个模型在科学教育中阅读基于文本的评分标准并自动评分学生绘制模型的能力。研究发现，GPT-4V 在评分准确性和二次加权 Kappa 方面显著优于 Gemini Pro。定性分析表明，差异可能源于模型处理图像中细粒度文本以及整体图像分类性能的能力。即使调整 NERIF 方法进一步减小输入图像的大小，Gemini Pro 的表现仍不如 GPT-4V。研究结果表明，GPT-4V 在处理复杂多模态教育任务方面具有优异能力。研究结论指出，虽然两个模型都代表了人工智能的进步，但 GPT-4V 的更高性能使其成为涉及多模态数据解释的教育应用更合适的工具。