We used Data Maps to model and characterize the AuTexTification dataset. This
provides insights about the behaviour of individual samples during training
across epochs (training dynamics). We characterized the samples across 3
dimensions: confidence, variability and correctness. This shows the presence of
3 regions: easy-to-learn, ambiguous and hard-to-learn examples. We used a
classic CNN architecture and found out that training the model only on a subset
of ambiguous examples improves the model's out-of-distribution generalization.

使用 Data Maps 对 AuTexTification 数据集进行建模和表征，通过训练过程中个体样本的行为洞察（训练动态）。我们通过自信度、可变性和正确性这三个维度表征样本，发现有三个区域：易学习、模糊和难学习的样本。我们使用经典的 CNN 架构发现仅在模糊样本子集上训练模型可以提高模型的分布外泛化能力。

基于 CNN 和训练动态的自动文本识别

Automated Text Identification Using CNN and Training Dynamics

We propose Cartography Active Learning (CAL), a novel Active Learning (AL)
algorithm that exploits the behavior of the model on individual instances
during training as a proxy to find the most informative instances for labeling.
CAL is inspired by data maps, which were recently proposed to derive insights
into dataset quality (Swayamdipta et al., 2020). We compare our method on
popular text classification tasks to commonly used AL strategies, which instead
rely on post-training behavior. We demonstrate that CAL is competitive to other
common AL methods, showing that training dynamics derived from small seed data
can be successfully used for AL. We provide insights into our new AL method by
analyzing batch-level statistics utilizing the data maps. Our results further
show that CAL results in a more data-efficient learning strategy, achieving
comparable or better results with considerably less training data.

提出了一种名为 “Cartography Active Learning”(CAL) 的主动学习算法，该算法通过利用模型在单个实例上的行为作为代理来查找最具信息价值的实例进行标注。与其他常用的 AL 策略相比，CAL 是一种更有效的学习策略，可以通过利用小种子数据产生的训练动态来成功地进行 AL。

地图学主动学习

Cartography Active Learning

Large datasets have become commonplace in NLP research. However, the
increased emphasis on data quantity has made it challenging to assess the
quality of data. We introduce Data Maps---a model-based tool to characterize
and diagnose datasets. We leverage a largely ignored source of information: the
behavior of the model on individual instances during training (training
dynamics) for building data maps. This yields two intuitive measures for each
example---the model's confidence in the true class, and the variability of this
confidence across epochs---obtained in a single run of training. Experiments
across four datasets show that these model-dependent measures reveal three
distinct regions in the data map, each with pronounced characteristics. First,
our data maps show the presence of "ambiguous" regions with respect to the
model, which contribute the most towards out-of-distribution generalization.
Second, the most populous regions in the data are "easy to learn" for the
model, and play an important role in model optimization. Finally, data maps
uncover a region with instances that the model finds "hard to learn"; these
often correspond to labeling errors. Our results indicate that a shift in focus
from quantity to quality of data could lead to robust models and improved
out-of-distribution generalization.

使用 Data Maps 工具基于模型的训练动态量化和诊断数据集。实验结果表明，数据地图揭示了数据集中的模糊、易于学习和难以学习的区域，这些发现对于构建健壮的模型和提高分布外泛化能力非常重要。