In this paper we implement and compare 7 different data augmentation
strategies for the task of automatic scoring of children's ability to
understand others' thoughts, feelings, and desires (or "mindreading").
We recruit in-domain experts to re-annotate augmented samples and determine
to what extent each strategy preserves the original rating. We also carry out
multiple experiments to measure how much each augmentation strategy improves
the performance of automatic scoring systems. To determine the capabilities of
automatic systems to generalize to unseen data, we create UK-MIND-20 - a new
corpus of children's performance on tests of mindreading, consisting of 10,320
question-answer pairs.
We obtain a new state-of-the-art performance on the MIND-CA corpus, improving
macro-F1-score by 6 points. Results indicate that both the number of training
examples and the quality of the augmentation strategies affect the performance
of the systems. The task-specific augmentations generally outperform
task-agnostic augmentations. Automatic augmentations based on vectors (GloVe,
FastText) perform the worst.
We find that systems trained on MIND-CA generalize well to UK-MIND-20. We
demonstrate that data augmentation strategies also improve the performance on
unseen data.

本文对于儿童理解他人思想、感受和欲望的能力自动评分任务，实现和比较了 7 种不同的数据增强策略；除了对每种策略进行性能评估和对自动评分系统性能进行多次实验，还招募行业内专家重新注释增强样本以确定每个策略保留原始分数的程度。此外还创建了 UK-MIND-20 数据集并在 MIND-CA 数据集中获得了最新的最高性能，结果表明，训练样本的数量和增强策略的质量都影响系统的性能。