Most recent works of test-time adaptation (TTA) aim to alleviate domain shift
problems by re-training source classifiers in each domain. On the other hand,
the emergence of the diffusion model provides another solution to TTA, which
directly maps the test data from the target domain to the source domain based
on a diffusion model pre-trained in the source domain. The source classifier
does not need to be fine-tuned. However, 1) the semantic information loss from
test data to the source domain and 2) the model shift between the source
classifier and diffusion model would prevent the diffusion model from mapping
the test data back to the source domain correctly. In this paper, we propose a
novel guidance-based diffusion-driven adaptation (GDDA) to overcome the data
shift and let the diffusion model find a better way to go back to the source.
Concretely, we first propose detail and global guidance to better keep the
common semantics of the test and source data. The two guidance include a
contrastive loss and mean squared error to alleviate the information loss by
fully exploring the diffusion model and the test data. Meanwhile, we propose a
classifier-aware guidance to reduce the bias caused by the model shift, which
can incorporate the source classifier's information into the generation process
of the diffusion model. Extensive experiments on three image datasets with
three classifier backbones demonstrate that GDDA significantly performs better
than the state-of-the-art baselines. On CIFAR-10C, CIFAR-100C, and ImageNetC,
GDDA achieves 11.54\%, 19.05\%, and 11.63\% average accuracy improvements,
respectively. GDDA even achieves equal performance compared with methods of
re-training classifiers. The code is available in the supplementary material.

这篇论文介绍了一种新的基于引导的扩散驱动自适应方法，通过引入详细和全局的引导信息以及考虑分类器的引导信息，解决了测试数据与源域之间的语义信息丢失和模型偏移问题，并在多个实验中取得了显著的改进。

基于导向的扩散模型的测试时适应

Target to Source: Guidance-Based Diffusion Model for Test-Time  Adaptation

Semantic communication aims to transmit meaningful and effective information
rather than focusing on individual symbols or bits, resulting in benefits like
reduced latency, bandwidth usage, and higher throughput compared to traditional
communication. However, semantic communication poses significant challenges due
to the need for universal metrics for benchmarking the joint effects of
semantic information loss and practical energy consumption. This research
presents a novel multi-objective loss function named "Energy-Optimized Semantic
Loss" (EOSL), addressing the challenge of balancing semantic information loss
and energy consumption. Through comprehensive experiments on transformer
models, including CPU and GPU energy usage, it is demonstrated that EOSL-based
encoder model selection can save up to 90\% of energy while achieving a 44\%
improvement in semantic similarity performance during inference in this
experiment. This work paves the way for energy-efficient neural network
selection and the development of greener semantic communication architectures.

语义通信是传输有意义和有效信息的目标，通过 “Energy-Optimized Semantic Loss”（EOSL）这种新型多目标损失函数，旨在解决平衡语义信息丢失和能量消耗的挑战，并通过基于 EOSL 的编码器模型选择在推理过程中实现了 90% 的能量节省和 44% 的语义相似性性能提升，为能源高效的神经网络选择和绿色的语义通信架构发展铺平了道路。