Existing image restoration models are typically designed for specific tasks
and struggle to generalize to out-of-sample degradations not encountered during
training. While zero-shot methods can address this limitation by fine-tuning
model parameters on testing samples, their effectiveness relies on predefined
natural priors and physical models of specific degradations. Nevertheless,
determining out-of-sample degradations faced in real-world scenarios is always
impractical. As a result, it is more desirable to train restoration models with
inherent generalization ability. To this end, this work introduces the
Out-of-Sample Restoration (OSR) task, which aims to develop restoration models
capable of handling out-of-sample degradations. An intuitive solution involves
pre-translating out-of-sample degradations to known degradations of restoration
models. However, directly translating them in the image space could lead to
complex image translation issues. To address this issue, we propose a model
reprogramming framework, which translates out-of-sample degradations by quantum
mechanic and wave functions. Specifically, input images are decoupled as wave
functions of amplitude and phase terms. The translation of out-of-sample
degradation is performed by adapting the phase term. Meanwhile, the image
content is maintained and enhanced in the amplitude term. By taking these two
terms as inputs, restoration models are able to handle out-of-sample
degradations without fine-tuning. Through extensive experiments across multiple
evaluation cases, we demonstrate the effectiveness and flexibility of our
proposed framework. Our codes are available at
\href{this https URL}{Github}.

该研究提出了一种模型重新编程框架，通过量子力学和波函数的方式将样本之外的退化转化为已知的修复模型退化，以提高修复模型对样本之外退化的处理能力。

通过模型重编程实现对样本外退化的泛化

Generalizing to Out-of-Sample Degradations via Model Reprogramming

In this paper, we explore a novel model reusing task tailored for graph
neural networks (GNNs), termed as "deep graph reprogramming". We strive to
reprogram a pre-trained GNN, without amending raw node features nor model
parameters, to handle a bunch of cross-level downstream tasks in various
domains. To this end, we propose an innovative Data Reprogramming paradigm
alongside a Model Reprogramming paradigm. The former one aims to address the
challenge of diversified graph feature dimensions for various tasks on the
input side, while the latter alleviates the dilemma of fixed per-task-per-model
behavior on the model side. For data reprogramming, we specifically devise an
elaborated Meta-FeatPadding method to deal with heterogeneous input dimensions,
and also develop a transductive Edge-Slimming as well as an inductive
Meta-GraPadding approach for diverse homogenous samples. Meanwhile, for model
reprogramming, we propose a novel task-adaptive Reprogrammable-Aggregator, to
endow the frozen model with larger expressive capacities in handling
cross-domain tasks. Experiments on fourteen datasets across node/graph
classification/regression, 3D object recognition, and distributed action
recognition, demonstrate that the proposed methods yield gratifying results, on
par with those by re-training from scratch.

本研究提出一种新的基于深度计算图重编程的无需改变结点特征或者模型参数的任务复用模型，使用数据重编程和模型重编程方法能更好地解决不同领域中的多跨层下游任务，其中元特征填充是解决图像特征维度多样化的挑战的关键方法之一。

深度图重编程

Deep Graph Reprogramming

In data-rich domains such as vision, language, and speech, deep learning
prevails to deliver high-performance task-specific models and can even learn
general task-agnostic representations for efficient finetuning to downstream
tasks. However, deep learning in resource-limited domains still faces the
following challenges including (i) limited data, (ii) constrained model
development cost, and (iii) lack of adequate pre-trained models for effective
finetuning. This paper introduces a new technique called model reprogramming to
bridge this gap. Model reprogramming enables resource-efficient cross-domain
machine learning by repurposing and reusing a well-developed pre-trained model
from a source domain to solve tasks in a target domain without model
finetuning, where the source and target domains can be vastly different. In
many applications, model reprogramming outperforms transfer learning and
training from scratch. This paper elucidates the methodology of model
reprogramming, summarizes existing use cases, provides a theoretical
explanation on the success of model reprogramming, and concludes with a
discussion on open-ended research questions and opportunities. A list of model
reprogramming studies is actively maintained and updated at
this https URL.

本文介绍了一种名为模型再编程的新技术，通过从源域重新利用和重用开发良好的预训练模型来解决目标域中的任务，从而实现资源高效的跨域机器学习，其方法论、现有应用案例和开放性研究问题等均有详细阐述。