The automatic evaluation for school assignments is an important application
of AI in the education field. In this work, we focus on the task of
personalized multimodal feedback generation, which aims to generate
personalized feedback for various teachers to evaluate students' assignments
involving multimodal inputs such as images, audios, and texts. This task
involves the representation and fusion of multimodal information and natural
language generation, which presents the challenges from three aspects: 1) how
to encode and integrate multimodal inputs; 2) how to generate feedback specific
to each modality; and 3) how to realize personalized feedback generation. In
this paper, we propose a novel Personalized Multimodal Feedback Generation
Network (PMFGN) armed with a modality gate mechanism and a personalized bias
mechanism to address these challenges. The extensive experiments on real-world
K-12 education data show that our model significantly outperforms several
baselines by generating more accurate and diverse feedback. In addition,
detailed ablation experiments are conducted to deepen our understanding of the
proposed framework.

本文介绍了一种个性化多模态反馈生成网络，通过引入模态门机制和个性化偏置机制，实现了自动评估学生作业的功能，并取得了明显的性能提升。模型在实际的 K-12 教育数据上表现出更准确和更多样化的反馈。