The era of pre-trained models has ushered in a wealth of new insights for the
machine learning community. Among the myriad of questions that arise, one of
paramount importance is: 'Do pre-trained models possess comprehensive
knowledge?' This paper seeks to address this crucial inquiry. In line with our
objective, we have made publicly available a novel dataset comprised of images
from TV series released post-2021. This dataset holds significant potential for
use in various research areas, including the evaluation of incremental
learning, novel class discovery, and long-tailed learning, among others.
Project page: this https URL

预训练模型在机器学习社区带来了丰富的新见解，本文旨在探讨预训练模型是否具备全面的知识，并提供了一个基于 2021 年后发布的电视剧图像的新数据集，该数据集在增量学习评估、新类别发现和长尾学习等多个研究领域具有重要潜力。

TV100：一份 CLIP 未见过的电视剧数据集

TV100: A TV Series Dataset that Pre-Trained CLIP Has Not Seen

This paper presents emerging evidence of using generative large language
models (i.e., GPT-4) to reliably evaluate short-answer reading comprehension
questions. Specifically, we explore how various configurations of generative
(LLMs) are able to evaluate student responses from a new dataset, drawn from a
battery of reading assessments conducted with over 150 students in Ghana. As
this dataset is novel and hence not used in training runs of GPT, it offers an
opportunity to test for domain shift and evaluate the generalizability of
generative LLMs, which are predominantly designed and trained on data from
high-income North American countries. We found that GPT-4, with minimal prompt
engineering performed extremely well on evaluating the novel dataset (Quadratic
Weighted Kappa 0.923, F1 0.88), substantially outperforming transfer-learning
based approaches, and even exceeding expert human raters (Quadratic Weighted
Kappa 0.915, F1 0.87). To the best of our knowledge, our work is the first to
empirically evaluate the performance of generative LLMs on short-answer reading
comprehension questions, using real student data, and suggests that generative
LLMs have the potential to reliably evaluate foundational literacy. Currently
the assessment of formative literacy and numeracy is infrequent in many low and
middle-income countries (LMICs) due to the cost and operational complexities of
conducting them at scale. Automating the grading process for reading assessment
could enable wider usage, and in turn improve decision-making regarding
curricula, school management, and teaching practice at the classroom level.
Importantly, in contrast transfer learning based approaches, generative LLMs
generalize well and the technical barriers to their use are low, making them
more feasible to implement and scale in lower resource educational contexts.

利用生成型大型语言模型（即 GPT-4）有望可靠地评估短答阅读理解问题，并通过使用新的数据集和自动评分过程提供了改进基础扫盲教育的可能性。