Amid rising concerns of reproducibility and generalizability in predictive
modeling, we explore the possibility and potential benefits of introducing
pre-registration to the field. Despite notable advancements in predictive
modeling, spanning core machine learning tasks to various scientific
applications, challenges such as overlooked contextual factors, data-dependent
decision-making, and unintentional re-use of test data have raised questions
about the integrity of results. To address these issues, we propose adapting
pre-registration practices from explanatory modeling to predictive modeling. We
discuss current best practices in predictive modeling and their limitations,
introduce a lightweight pre-registration template, and present a qualitative
study with machine learning researchers to gain insight into the effectiveness
of pre-registration in preventing biased estimates and promoting more reliable
research outcomes. We conclude by exploring the scope of problems that
pre-registration can address in predictive modeling and acknowledging its
limitations within this context.

通过研究预测建模中出现的再现性和泛化性的担忧，我们探讨引入先注册的可能性和潜在好处。虽然预测建模取得了显著进展，涵盖了核心机器学习任务和各种科学应用，但是忽视的背景因素、基于数据的决策和意外重复使用测试数据等挑战引发了对结果完整性的质疑。为解决这些问题，我们建议从解释性建模中借鉴先注册实践到预测建模中。我们讨论了预测建模的当前最佳实践及其局限性，引入了一个轻量级的先注册模板，并进行了一项定性研究，以了解先注册在预防有偏估计和促进更可靠的研究结果方面的有效性。最后，我们探讨了先注册可以解决的预测建模问题的范围，并在此背景下承认其局限性。