Reproducibility is a cornerstone of scientific research, enabling validation,
extension, and progress. However, the rapidly evolving nature of software and
dependencies poses significant challenges to reproducing research results,
particularly in fields like adversarial robustness for deep neural networks,
where complex codebases and specialized toolkits are utilized. This paper
presents a case study of attempting to validate the results on certified
adversarial robustness in "SoK: Certified Robustness for Deep Neural Networks"
using the VeriGauge toolkit. Despite following the documented methodology,
numerous software and hardware compatibility issues were encountered, including
outdated or unavailable dependencies, version conflicts, and driver
incompatibilities. While a subset of the original results could be run, key
findings related to the empirical robust accuracy of various verification
methods proved elusive due to these technical obstacles, as well as slight
discrepancies in the test results. This practical experience sheds light on the
reproducibility crisis afflicting adversarial robustness research, where a lack
of reproducibility threatens scientific integrity and hinders progress. The
paper discusses the broader implications of this crisis, proposing potential
solutions such as containerization, software preservation, and comprehensive
documentation practices. Furthermore, it highlights the need for collaboration
and standardization efforts within the research community to develop robust
frameworks for reproducible research. By addressing the reproducibility crisis
head-on, this work aims to contribute to the ongoing discourse on scientific
reproducibility and advocate for best practices that ensure the reliability and
validity of research findings within not only adversarial robustness, but
security and technology research as a whole.

本文通过案例研究验证了复现性危机对对抗性鲁棒性研究的影响，提出了解决方案，强调了合作和标准化努力的必要性，并倡导了确保研究结果可靠和有效的最佳实践。