Does the training of large language models potentially infringe upon code
licenses? Furthermore, are there any datasets available that can be safely used
for training these models without violating such licenses? In our study, we
assess the current trends in the field and the importance of incorporating code
into the training of large language models. Additionally, we examine publicly
available datasets to see whether these models can be trained on them without
the risk of legal issues in the future. To accomplish this, we compiled a list
of 53 large language models trained on file-level code. We then extracted their
datasets and analyzed how much they overlap with a dataset we created,
consisting exclusively of strong copyleft code.
Our analysis revealed that every dataset we examined contained license
inconsistencies, despite being selected based on their associated repository
licenses. We analyzed a total of 514 million code files, discovering 38 million
exact duplicates present in our strong copyleft dataset. Additionally, we
examined 171 million file-leading comments, identifying 16 million with strong
copyleft licenses and another 11 million comments that discouraged copying
without explicitly mentioning a license. Based on the findings of our study,
which highlights the pervasive issue of license inconsistencies in large
language models trained on code, our recommendation for both researchers and
the community is to prioritize the development and adoption of best practices
for dataset creation and management.

大语言模型的训练是否会侵犯代码许可证？是否有可以安全使用来训练这些模型的可用数据集而不违反这些许可证呢？通过 53 个训练于文件级代码的大语言模型的研究，我们发现每个数据集都存在许可证一致性问题。因此，我们建议研究人员和社区都应优先发展和采用最佳实践来创建和管理数据集。