Agnostic domain shift is the main reason of model degradation on the unknown
target domains, which brings an urgent need to develop Domain Generalization
(DG). Recent advances at DG use dynamic networks to achieve training-free
adaptation on the unknown target domains, termed Dynamic Domain Generalization
(DDG), which compensates for the lack of self-adaptability in static models
with fixed weights. The parameters of dynamic networks can be decoupled into a
static and a dynamic component, which are designed to learn domain-invariant
and domain-specific features, respectively. Based on the existing arts, in this
work, we try to push the limits of DDG by disentangling the static and dynamic
components more thoroughly from an optimization perspective. Our main
consideration is that we can enable the static component to learn
domain-invariant features more comprehensively by augmenting the
domain-specific information. As a result, the more comprehensive
domain-invariant features learned by the static component can then enforce the
dynamic component to focus more on learning adaptive domain-specific features.
To this end, we propose a simple yet effective Parameter Exchange (PE) method
to perturb the combination between the static and dynamic components. We
optimize the model using the gradients from both the perturbed and
non-perturbed feed-forward jointly to implicitly achieve the aforementioned
disentanglement. In this way, the two components can be optimized in a
mutually-beneficial manner, which can resist the agnostic domain shifts and
improve the self-adaptability on the unknown target domain. Extensive
experiments show that PE can be easily plugged into existing dynamic networks
to improve their generalization ability without bells and whistles.

通过断开静态和动态组件之间的联系，通过参数交换（PE）方法，实现对动态网络的静态组件进行更全面的学习，从而加强对自适应领域特定特征的学习，抵抗不可知领域偏移并提高自适应性能。

鲁棒动态域泛化的参数交换

Parameter Exchange for Robust Dynamic Domain Generalization

Federated Learning (FL) has been successfully adopted for distributed
training and inference of large-scale Deep Neural Networks (DNNs). However,
DNNs are characterized by an extremely large number of parameters, thus,
yielding significant challenges in exchanging these parameters among
distributed nodes and managing the memory. Although recent DNN compression
methods (e.g., sparsification, pruning) tackle such challenges, they do not
holistically consider an adaptively controlled reduction of parameter exchange
while maintaining high accuracy levels. We, therefore, contribute with a novel
FL framework (coined FedDIP), which combines (i) dynamic model pruning with
error feedback to eliminate redundant information exchange, which contributes
to significant performance improvement, with (ii) incremental regularization
that can achieve \textit{extreme} sparsity of models. We provide convergence
analysis of FedDIP and report on a comprehensive performance and comparative
assessment against state-of-the-art methods using benchmark data sets and DNN
models. Our results showcase that FedDIP not only controls the model sparsity
but efficiently achieves similar or better performance compared to other model
pruning methods adopting incremental regularization during distributed model
training. The code is available at: this https URL

本研究提出了一种新的联邦学习框架 (FedDIP)，通过动态模型修剪和错误反馈相结合的方式，来控制参数交换并实现模型稀疏化，以达到与其他模型修剪方法相媲美或更好的性能。