Vision-Language Models (VLMs) such as CLIP have yielded unprecedented
performance for zero-shot image classification, yet their generalization
capability may still be seriously challenged when confronted to domain shifts.
In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a
pioneering approach facilitating full test-time adaptation (TTA) of this VLM.
Our method employs a diverse set of templates for text prompts, augmenting the
existing framework of CLIP. Predictions are utilized as pseudo labels for model
updates, followed by weight averaging to consolidate the learned information
globally. Furthermore, we introduce a text ensemble strategy, enhancing overall
test performance by aggregating diverse textual cues. Our findings underscore
the efficacy of WATT in enhancing performance across diverse datasets,
including CIFAR-10-C, CIFAR-10.1, CIFAR-100-C, VisDA-C, and several other
challenging datasets, effectively covering a wide range of domain shifts.
Notably, these enhancements are achieved without necessitating additional model
transformations or trainable modules. Moreover, compared to other Test-Time
Adaptation methods, our approach can operate effectively with just a single
image. Highlighting the potential of innovative test-time strategies, this
research emphasizes their role in fortifying the adaptability of VLMs. The
implementation is available at:
https://github.com/Mehrdad-Noori/WATT.git.

本研究介绍了 Weight Average Test-Time Adaptation (WATT) 方法，通过使用多样的文本模板和文本整合策略，增强了 CLIP 模型的测试性能，提高对不同领域转变的适应能力。该方法在多个具有挑战性的数据集上取得了显著的增益，而无需额外的模型转换或可训练模块。