AI models (including LLM) often rely on narrative question-answering (QA)
datasets to provide customized QA functionalities to support downstream
children education applications; however, existing datasets only include QA
pairs that are grounded within the given storybook content, but children can
learn more when teachers refer the storybook content to real-world knowledge
(e.g., commonsense knowledge). We introduce the FairytaleCQA dataset, which is
annotated by children education experts, to supplement 278 storybook narratives
with educationally appropriate commonsense knowledge. The dataset has 5,868 QA
pairs that not only originate from the storybook narrative but also contain the
commonsense knowledge grounded by an external knowledge graph (i.e.,
ConceptNet). A follow-up experiment shows that a smaller model (T5-large)
fine-tuned with FairytaleCQA reliably outperforms much larger prompt-engineered
LLM (e.g., GPT-4) in this new QA-pair generation task (QAG). This result
suggests that: 1) our dataset brings novel challenges to existing LLMs, and 2)
human experts' data annotation are still critical as they have much nuanced
knowledge that LLMs do not know in the children educational domain.

AI 模型通过使用以叙述为基础的问答数据集来提供定制化问答功能，以支持面向儿童教育应用；然而，现有数据集仅包括基于故事内容的问答对，而当教师将故事内容与现实世界的知识（例如常识知识）进行关联时，儿童可以学到更多。我们引入了由儿童教育专家注释的 FairytaleCQA 数据集，用于补充 278 个故事叙述的教育适用的常识知识。该数据集包含 5,868 个问答对，不仅源自故事叙述，而且包含由外部知识图（即 ConceptNet）支持的常识知识。后续实验表明，使用 FairytaleCQA 对较小模型（T5-large）进行微调，可在新的问答对生成任务（QAG）中可靠地胜过更大的提示工程化语言模型（例如 GPT-4）。这一结果表明：1）我们的数据集给现有语言模型带来了新的挑战，2）人类专家的数据注释仍然至关重要，因为他们在儿童教育领域拥有着更多细微的知识，语言模型不知晓。