Deep neural networks are often overparameterized and may not easily achieve
model generalization. Adversarial training has shown effectiveness in improving
generalization by regularizing the change of loss on top of adversarially
chosen perturbations. The recently proposed sharpness-aware minimization (SAM)
algorithm conducts adversarial weight perturbation, encouraging the model to
converge to a flat minima. SAM finds a common adversarial weight perturbation
per-batch. Although per-instance adversarial weight perturbations are stronger
adversaries and can potentially lead to better generalization performance,
their computational cost is very high and thus it is impossible to use
per-instance perturbations efficiently in SAM. In this paper, we tackle this
efficiency bottleneck and propose sharpness-aware minimization with dynamic
reweighting (delta-SAM). Our theoretical analysis motivates that it is possible
to approach the stronger, per-instance adversarial weight perturbations using
reweighted per-batch weight perturbations. delta-SAM dynamically reweights
perturbation within each batch according to the theoretically principled
weighting factors, serving as a good approximation to per-instance
perturbation. Experiments on various natural language understanding tasks
demonstrate the effectiveness of delta-SAM.

本研究提出了 delta-SAM 算法来提高深度神经网络的泛化性能，通过动态重新赋权在每个 batch 中实现更强的对抗权重扰动以逼近更强的单实例扰动。

动态调整权重的锐度感知最小化

Sharpness-Aware Minimization with Dynamic Reweighting

Nowadays, we have witnessed the early progress on learning the association
between voice and face automatically, which brings a new wave of studies to the
computer vision community. However, most of the prior arts along this line (a)
merely adopt local information to perform modality alignment and (b) ignore the
diversity of learning difficulty across different subjects. In this paper, we
propose a novel framework to jointly address the above-mentioned issues.
Targeting at (a), we propose a two-level modality alignment loss where both
global and local information are considered. Compared with the existing
methods, we introduce a global loss into the modality alignment process. The
global component of the loss is driven by the identity classification.
Theoretically, we show that minimizing the loss could maximize the distance
between embeddings across different identities while minimizing the distance
between embeddings belonging to the same identity, in a global sense (instead
of a mini-batch). Targeting at (b), we propose a dynamic reweighting scheme to
better explore the hard but valuable identities while filtering out the
unlearnable identities. Experiments show that the proposed method outperforms
the previous methods in multiple settings, including voice-face matching,
verification and retrieval.

本文提出了一个基于全局和局部信息的定位框架，其中引入了全局 loss 来处理学习困难度问题，并提出了动态重新加权机制来探索较难但有价值的身份。通过配对测试，验证和检索任务的实验，表明所提方法优于以往的方法。