This paper describes the results of the first shared task on the generation
of teacher responses in educational dialogues. The goal of the task was to
benchmark the ability of generative language models to act as AI teachers,
replying to a student in a teacher-student dialogue. Eight teams participated
in the competition hosted on CodaLab. They experimented with a wide variety of
state-of-the-art models, including Alpaca, Bloom, DialoGPT, DistilGPT-2,
Flan-T5, GPT-2, GPT-3, GPT- 4, LLaMA, OPT-2.7B, and T5-base. Their submissions
were automatically scored using BERTScore and DialogRPT metrics, and the top
three among them were further manually evaluated in terms of pedagogical
ability based on Tack and Piech (2022). The NAISTeacher system, which ranked
first in both automated and human evaluation, generated responses with GPT-3.5
using an ensemble of prompts and a DialogRPT-based ranking of responses for
given dialogue contexts. Despite the promising achievements of the
participating teams, the results also highlight the need for evaluation metrics
better suited to educational contexts.

本文选举为教育对话中教师回应产生共性任务的首个任务，以基于生成语言模型的人工智能教师为校准目标；在八个参赛者中，NAISTeacher 系统以 GPT-3.5 生成器为基础靠类似于提示流的集合和对话环境上的排名生成，达到自动和人工评估的双重第一名；然而，结果也凸显出现有的关键问题，包括需适合于教育场景的评估指标等。