A key challenge, supported both theoretically and empirically, is that
robustness demands greater network capacity and more data than standard
training. However, effectively adding capacity under stringent Lipschitz
constraints has proven more difficult than it may seem, evident by the fact
that state-of-the-art approach tend more towards \emph{underfitting} than
overfitting. Moreover, we posit that a lack of careful exploration of the
design space for Lipshitz-based approaches has left potential performance gains
on the table. In this work, we provide a more comprehensive evaluation to
better uncover the potential of Lipschitz-based certification methods. Using a
combination of novel techniques, design optimizations, and synthesis of prior
work, we are able to significantly improve the state-of-the-art \emph{verified
robust accuracy} (VRA) for deterministic certification on a variety of
benchmark datasets, and over a range of perturbation sizes. Of particular note,
we discover that the addition of large "Cholesky-orthogonalized residual dense"
layers to the end of existing state-of-the-art Lipschitz-controlled ResNet
architectures is especially effective for increasing network capacity and
performance. Combined with filtered generative data augmentation, our final
results further the state of the art deterministic VRA by up to 8.5 percentage
points. Code is available at https://github.com/hukkai/liresnet.

通过使用新技术、设计优化和先前工作的综合综合，我们能够显著提高确定性认证方法在各种基准数据集上的最新鲜准确性，并扩展了扰动范围大小，尤其是通过在现有最新雅可比约束控制的 ResNet 架构末端添加大型 “Cholesky - 正交化残差稠密” 层，以增加网络容量和性能，再结合过滤的生成数据增强，最终结果比确定性认证方法的最新准确性提高 8.5 个百分点。