Training certifiably robust neural networks is an important but challenging
task. While many algorithms for (deterministic) certified training have been
proposed, they are often evaluated on different training schedules,
certification methods, and systematically under-tuned hyperparameters, making
it difficult to compare their performance. To address this challenge, we
introduce CTBENCH, a unified library and a high-quality benchmark for certified
training that evaluates all algorithms under fair settings and systematically
tuned hyperparameters. We show that (1) almost all algorithms in CTBENCH
surpass the corresponding reported performance in literature in the magnitude
of algorithmic improvements, thus establishing new state-of-the-art, and (2)
the claimed advantage of recent algorithms drops significantly when we enhance
the outdated baselines with a fair training schedule, a fair certification
method and well-tuned hyperparameters. Based on CTBENCH, we provide new
insights into the current state of certified training and suggest future
research directions. We are confident that CTBENCH will serve as a benchmark
and testbed for future research in certified training.

CTBENCH 是一个统一的库和高质量的基准测试，用于证明训练。通过在公平设置和系统调整的超参数下评估所有算法，CTBENCH 展示了几乎所有算法超越了文献中所报道的性能，并建立了新的最先进方法，同时揭示了近期算法的优势在强化基线后明显减弱。基于 CTBENCH，我们为当前的证明训练状态提供了新的见解，并提出了未来的研究方向。我们相信 CTBENCH 将成为证明训练未来研究的基准和测试平台。