This paper presents a simple yet effective approach that improves continual
test-time adaptation (TTA) in a memory-efficient manner. TTA may primarily be
conducted on edge devices with limited memory, so reducing memory is crucial
but has been overlooked in previous TTA studies. In addition, long-term
adaptation often leads to catastrophic forgetting and error accumulation, which
hinders applying TTA in real-world deployments. Our approach consists of two
components to address these issues. First, we present lightweight meta networks
that can adapt the frozen original networks to the target domain. This novel
architecture minimizes memory consumption by decreasing the size of
intermediate activations required for backpropagation. Second, our novel
self-distilled regularization controls the output of the meta networks not to
deviate significantly from the output of the frozen original networks, thereby
preserving well-trained knowledge from the source domain. Without additional
memory, this regularization prevents error accumulation and catastrophic
forgetting, resulting in stable performance even in long-term test-time
adaptation. We demonstrate that our simple yet effective strategy outperforms
other state-of-the-art methods on various benchmarks for image classification
and semantic segmentation tasks. Notably, our proposed method with ResNet-50
and WideResNet-40 takes 86% and 80% less memory than the recent
state-of-the-art method, CoTTA.

本文提出了一个简单而有效的方法，以在内存效率的前提下改善持续的测试时间适应性的执行，该方法主要可在具有有限内存的边缘设备上进行，通过减少内存的使用量来实现。该方法包括两个组件，一个是可以在目标领域中将冻结的原始网络适应的轻量级元网络，第二个是我们的新型自我蒸馏正则化，它可以控制元网络的输出不显著偏离冻结的原始网络的输出，从而保留源域中的良好训练知识，因此即使在长期的测试时间适应过程中，没有额外的存储器，这种正则化也可以防止错误积累和灾难性遗忘，从而实现稳定的性能。我们展示了我们的方法在各种图像分类和语义分割任务的基准测试上优于其他最新方法，其中，我们提出的使用 ResNet-50 和 WideResNet-40 的方法比最新的 CoTTA 方法少使用了 86％和 80％的内存。

EcoTTA: 自我蒸馏正则化实现高效节省内存的持续测试时适应

EcoTTA: Memory-Efficient Continual Test-time Adaptation via  Self-distilled Regularization

The predominant approach to Visual Question Answering (VQA) demands that the
model represents within its weights all of the information required to answer
any question about any image. Learning this information from any real training
set seems unlikely, and representing it in a reasonable number of weights
doubly so. We propose instead to approach VQA as a meta learning task, thus
separating the question answering method from the information required. At test
time, the method is provided with a support set of example questions/answers,
over which it reasons to resolve the given question. The support set is not
fixed and can be extended without retraining, thereby expanding the
capabilities of the model. To exploit this dynamically provided information, we
adapt a state-of-the-art VQA model with two techniques from the recent meta
learning literature, namely prototypical networks and meta networks.
Experiments demonstrate the capability of the system to learn to produce
completely novel answers (i.e. never seen during training) from examples
provided at test time. In comparison to the existing state of the art, the
proposed method produces qualitatively distinct results with higher recall of
rare answers, and a better sample efficiency that allows training with little
initial data. More importantly, it represents an important step towards
vision-and-language methods that can learn and reason on-the-fly.

本文提出了一种元学习的方法来解决图像问题回答（VQA）问题，并采用了原型网络和元网络两种技术，通过给定的例子来推理解决给定的问题，与现有技术相比，该方法能够产生具有更高罕见回答的召回率和更好的样本效率，是学习和推理基于视觉和语言的方法的重要一步。

将视觉问答作为元学习任务

Visual Question Answering as a Meta Learning Task

In this paper we propose a new method to get the specified network parameters
through one time feed-forward propagation of the meta networks and explore the
application to neural style transfer. Recent works on style transfer typically
need to train image transformation networks for every new style, and the style
is encoded in the network parameters by enormous iterations of stochastic
gradient descent. To tackle these issues, we build a meta network which takes
in the style image and produces a corresponding image transformations network
directly. Compared with optimization-based methods for every style, our meta
networks can handle an arbitrary new style within $19ms$ seconds on one modern
GPU card. The fast image transformation network generated by our meta network
is only 449KB, which is capable of real-time executing on a mobile device. We
also investigate the manifold of the style transfer networks by operating the
hidden features from meta networks. Experiments have well validated the
effectiveness of our method. Code and trained models has been released
this https URL

本文提出了一种基于元网络的方法，可以通过一次前向传播来获取指定网络参数，并探索应用于神经风格迁移。相对于每个样式的基于优化的方法，我们的元网络在一台现代 GPU 卡上能够在 $19ms$ 内处理任意新样式，并能生成一个只有 449KB 大小的快速图像转换网络，可在移动设备上实时执行。实验证明了我们方法的有效性。