Adversarial training is extensively utilized to improve the adversarial
robustness of deep neural networks. Yet, mitigating the degradation of standard
generalization performance in adversarial-trained models remains an open
problem. This paper attempts to resolve this issue through the lens of model
complexity. First, We leverage the Fisher-Rao norm, a geometrically invariant
metric for model complexity, to establish the non-trivial bounds of the
Cross-Entropy Loss-based Rademacher complexity for a ReLU-activated Multi-Layer
Perceptron. Then we generalize a complexity-related variable, which is
sensitive to the changes in model width and the trade-off factors in
adversarial training. Moreover, intensive empirical evidence validates that
this variable highly correlates with the generalization gap of Cross-Entropy
loss between adversarial-trained and standard-trained models, especially during
the initial and final phases of the training process. Building upon this
observation, we propose a novel regularization framework, called Logit-Oriented
Adversarial Training (LOAT), which can mitigate the trade-off between
robustness and accuracy while imposing only a negligible increase in
computational overhead. Our extensive experiments demonstrate that the proposed
regularization strategy can boost the performance of the prevalent adversarial
training algorithms, including PGD-AT, TRADES, TRADES (LSE), MART, and DM-AT,
across various network architectures. Our code will be available at
this https URL

以模型复杂性为视角解决对抗训练中标准泛化性能的下降问题，通过使用 Logit-Oriented Adversarial Training (LOAT) 提出了一种新的正则化框架，可以在几乎不增加计算开销的情况下平衡鲁棒性与准确性，提高对抗性训练算法的性能。

基于 Fisher-Rao 范式的正则化提升对抗训练

Boosting Adversarial Training via Fisher-Rao Norm-based Regularization

Many fundamental machine learning tasks can be formulated as a problem of
learning with vector-valued functions, where we learn multiple scalar-valued
functions together. Although there is some generalization analysis on different
specific algorithms under the empirical risk minimization principle, a unifying
analysis of vector-valued learning under a regularization framework is still
lacking. In this paper, we initiate the generalization analysis of regularized
vector-valued learning algorithms by presenting bounds with a mild dependency
on the output dimension and a fast rate on the sample size. Our discussions
relax the existing assumptions on the restrictive constraint of hypothesis
spaces, smoothness of loss functions and low-noise condition. To understand the
interaction between optimization and learning, we further use our results to
derive the first generalization bounds for stochastic gradient descent with
vector-valued functions. We apply our general results to multi-class
classification and multi-label classification, which yield the first bounds
with a logarithmic dependency on the output dimension for extreme multi-label
classification with the Frobenius regularization. As a byproduct, we derive a
Rademacher complexity bound for loss function classes defined in terms of a
general strongly convex function.

本文提出了一个正则化框架下的向量值学习算法的一般化分析。该论文扩展了现有假设空间，损失函数平滑性和低噪声条件的限制。此外，作者还将这些结果应用于多类别和多标签分类。

向量值学习的细粒度泛化分析

Fine-grained Generalization Analysis of Vector-valued Learning

Decision-making often requires accurate estimation of treatment effects from
observational data. This is challenging as outcomes of alternative decisions
are not observed and have to be estimated. Previous methods estimate outcomes
based on unconfoundedness but neglect any constraints that unconfoundedness
imposes on the outcomes. In this paper, we propose a novel regularization
framework for estimating average treatment effects that exploits
unconfoundedness. To this end, we formalize unconfoundedness as an
orthogonality constraint, which ensures that the outcomes are orthogonal to the
treatment assignment. This orthogonality constraint is then included in the
loss function via a regularization. Based on our regularization framework, we
develop deep orthogonal networks for unconfounded treatments (DONUT), which
learn outcomes that are orthogonal to the treatment assignment. Using a variety
of benchmark datasets for estimating average treatment effects, we demonstrate
that DONUT outperforms the state-of-the-art substantially.

本文提出一种基于正交性的正则化框架来估计平均治疗效果，并开发了基于此框架的深度正交网络来学习与治疗分配正交的结果，实验结果表明 DONUT 在估计平均治疗效果方面优于现有技术。

通过正交正则化估计平均治疗效应

Estimating Average Treatment Effects via Orthogonal Regularization

Deep generative models have achieved remarkable success in various data
domains, including images, time series, and natural languages. There remain,
however, substantial challenges for combinatorial structures, including graphs.
One of the key challenges lies in the difficulty of ensuring semantic validity
in context. For examples, in molecular graphs, the number of bonding-electron
pairs must not exceed the valence of an atom; whereas in protein interaction
networks, two proteins may be connected only when they belong to the same or
correlated gene ontology terms. These constraints are not easy to be
incorporated into a generative model. In this work, we propose a regularization
framework for variational autoencoders as a step toward semantic validity. We
focus on the matrix representation of graphs and formulate penalty terms that
regularize the output distribution of the decoder to encourage the satisfaction
of validity constraints. Experimental results confirm a much higher likelihood
of sampling valid graphs in our approach, compared with others reported in the
literature.

本文介绍了一种基于矩阵表示的图形正则化框架，利用变分自编码器来实现语义有效性，实验证明和其他方法相比，该方法在采样有效的图形方面具有更高的可能性。