Educational scholars have analyzed various image data acquired from teaching
and learning situations, such as photos that shows classroom dynamics,
students' drawings with regard to the learning content, textbook illustrations,
etc. Unquestioningly, most qualitative analysis of and explanation on image
data have been conducted by human researchers, without machine-based
automation. It was partially because most image processing artificial
intelligence models were not accessible to general educational scholars or
explainable due to their complex deep neural network architecture. However, the
recent development of Visual Question Answering (VQA) techniques is
accomplishing usable visual language models, which receive from the user a
question about the given image and returns an answer, both in natural language.
Particularly, GPT-4V released by OpenAI, has wide opened the state-of-the-art
visual langauge model service so that VQA could be used for a variety of
purposes. However, VQA and GPT-4V have not yet been applied to educational
studies much. In this position paper, we suggest that GPT-4V contributes to
realizing VQA for education. By 'realizing' VQA, we denote two meanings: (1)
GPT-4V realizes the utilization of VQA techniques by any educational scholars
without technical/accessibility barrier, and (2) GPT-4V makes educational
scholars realize the usefulness of VQA to educational research. Given these,
this paper aims to introduce VQA for educational studies so that it provides a
milestone for educational research methodology. In this paper, chapter II
reviews the development of VQA techniques, which primes with the release of
GPT-4V. Chapter III reviews the use of image analysis in educational studies.
Chapter IV demonstrates how GPT-4V can be used for each research usage reviewed
in Chapter III, with operating prompts provided. Finally, chapter V discusses
the future implications.

这篇论文介绍了 Visual Question Answering（VQA）技术在教育研究中的应用，特别是 GPT-4V 在促进 VQA 技术的普及和提高教育研究方法方面的作用。

实现教育的视觉问答：GPT-4V 作为一种多模态人工智能

Realizing Visual Question Answering for Education: GPT-4V as a  Multimodal AI

This study explores the application of Large Language Models (LLMs),
specifically GPT-4, in the analysis of classroom dialogue, a crucial research
task for both teaching diagnosis and quality improvement. Recognizing the
knowledge-intensive and labor-intensive nature of traditional qualitative
methods in educational research, this study investigates the potential of LLM
to streamline and enhance the analysis process. The study involves datasets
from a middle school, encompassing classroom dialogues across mathematics and
Chinese classes. These dialogues were manually coded by educational experts and
then analyzed using a customised GPT-4 model. This study focuses on comparing
manual annotations with the outputs of GPT-4 to evaluate its efficacy in
analyzing educational dialogues. Time efficiency, inter-coder agreement, and
inter-coder reliability between human coders and GPT-4 are evaluated. Results
indicate substantial time savings with GPT-4, and a high degree of consistency
in coding between the model and human coders, with some discrepancies in
specific codes. These findings highlight the strong potential of LLM in
teaching evaluation and facilitation.

该研究探讨了大型语言模型（LLM），特别是 GPT-4，在课堂对话分析中的应用，这是教学诊断和质量改进的关键研究任务。研究发现传统定性方法在教育研究中具有知识密集和劳动密集的特点，调查了 LLM 在简化和增强分析过程方面的潜力。通过对中学的数据集进行分析，包括数学和语文课堂的对话，该研究对人工编码的对话进行了评估，并使用定制的 GPT-4 模型进行了分析。该研究重点比较了手动注释和 GPT-4 输出，以评估其在教育对话分析中的有效性。评估了 GPT-4 的时间效率、编码者间一致性和编码者间可靠性。结果表明，使用 GPT-4 能够显著节省时间，并且模型与人工编码者之间具有高度的一致性，尽管在某些代码上存在差异。这些发现凸显了 LLM 在教学评估和促进方面的巨大潜力。