Online Judge (OJ) systems are typically considered within programming-related
courses as they yield fast and objective assessments of the code developed by
the students. Such an evaluation generally provides a single decision based on
a rubric, most commonly whether the submission successfully accomplished the
assignment. Nevertheless, since in an educational context such information may
be deemed insufficient, it would be beneficial for both the student and the
instructor to receive additional feedback about the overall development of the
task. This work aims to tackle this limitation by considering the further
exploitation of the information gathered by the OJ and automatically inferring
feedback for both the student and the instructor. More precisely, we consider
the use of learning-based schemes -- particularly, multi-instance learning
(MIL) and classical machine learning formulations -- to model student behavior.
Besides, explainable artificial intelligence (XAI) is contemplated to provide
human-understandable feedback. The proposal has been evaluated considering a
case of study comprising 2500 submissions from roughly 90 different students
from a programming-related course in a computer science degree. The results
obtained validate the proposal: The model is capable of significantly
predicting the user outcome (either passing or failing the assignment) solely
based on the behavioral pattern inferred by the submissions provided to the OJ.
Moreover, the proposal is able to identify prone-to-fail student groups and
profiles as well as other relevant information, which eventually serves as
feedback to both the student and the instructor.

该研究论文通过对在线评测系统收集的信息进行进一步利用，应用基于学习的方案和解释性人工智能提供学生和教师反馈，以预测学生课程作业结果并识别潜在失败的学生群体。

利用可解释人工智能在在线评测系统中识别学生档案

Identifying Student Profiles Within Online Judge Systems Using  Explainable Artificial Intelligence

Referring to the solution programs written by the other users is helpful for
learners in programming education. However, current online judge systems just
list all solution programs submitted by users for references, and the programs
are sorted based on the submission date and time, execution time, or user
rating, ignoring to what extent the program can be a reference. In addition,
users struggle to refer to a variety of solution approaches since there are too
many duplicated and near-duplicated programs. To motivate the learners to refer
to various solutions to learn the better solution approaches, in this paper, we
propose an approach to deduplicate and rank common solution programs in each
programming problem. Based on the hypothesis that the more duplicated programs
adopt a more common approach and can be a reference, we remove the
near-duplicated solution programs and rank the unique programs based on the
duplicate count. The experiments on the solution programs submitted to a
real-world online judge system demonstrate that the number of programs is
reduced by 60.20%, whereas the baseline only reduces by 29.59% after the
deduplication, meaning that the users only need to refer to 39.80% of programs
on average. Furthermore, our analysis shows that top-10 ranked programs cover
29.95% of programs on average, indicating that the users can grasp 29.95% of
solution approaches by referring to only 10 programs. The proposed approach
shows the potential of reducing the learners' burden of referring to too many
solutions and motivating them to learn a variety of better approaches.

本研究通过对程序设计教育中一些常用问题的重复解决方案进行去重和排序，使学习者能够更有效地参考和学习更多不同的解决方式。