The race to train language models on vast, diverse, and inconsistently
documented datasets has raised pressing concerns about the legal and ethical
risks for practitioners. To remedy these practices threatening data
transparency and understanding, we convene a multi-disciplinary effort between
legal and machine learning experts to systematically audit and trace 1800+ text
datasets. We develop tools and standards to trace the lineage of these
datasets, from their source, creators, series of license conditions,
properties, and subsequent use. Our landscape analysis highlights the sharp
divides in composition and focus of commercially open vs closed datasets, with
closed datasets monopolizing important categories: lower resource languages,
more creative tasks, richer topic variety, newer and more synthetic training
data. This points to a deepening divide in the types of data that are made
available under different license conditions, and heightened implications for
jurisdictional legal interpretations of copyright and fair use. We also observe
frequent miscategorization of licenses on widely used dataset hosting sites,
with license omission of 72%+ and error rates of 50%+. This points to a crisis
in misattribution and informed use of the most popular datasets driving many
recent breakthroughs. As a contribution to ongoing improvements in dataset
transparency and responsible use, we release our entire audit, with an
interactive UI, the Data Provenance Explorer, which allows practitioners to
trace and filter on data provenance for the most popular open source finetuning
data collections: www.dataprovenance.org.

我们在语言模型的训练中发现了数据透明性和道德风险的问题，因此我们进行了一项多学科工作，对 1800 多个文本数据集进行了系统审计和追踪。我们发展了工具和标准，以跟踪这些数据集的来源、创作者、许可条件、属性和使用情况。我们的分析突出了商业开放数据集与封闭数据集在构成和重点方面的差异，封闭数据集垄断了重要的类别。此外，我们的研究还发现了广泛使用的数据集托管网站上许可证的错误和遗漏。为了促进数据集的透明性和负责任使用，我们发布了我们的审计结果，并提供了一个交互式界面，数据溯源浏览器，让从业者可以追踪和筛选最流行的开源调整数据集的数据溯源信息：www.dataprovenance.org。