Vision-Language Models seamlessly discriminate among arbitrary semantic
categories, yet they still suffer from poor generalization when presented with
challenging examples. For this reason, Episodic Test-Time Adaptation (TTA)
strategies have recently emerged as powerful techniques to adapt VLMs in the
presence of a single unlabeled image. The recent literature on TTA is dominated
by the paradigm of prompt tuning by Marginal Entropy Minimization, which,
relying on online backpropagation, inevitably slows down inference while
increasing memory. In this work, we theoretically investigate the properties of
this approach and unveil that a surprisingly strong TTA method lies dormant and
hidden within it. We term this approach ZERO (TTA with "zero" temperature),
whose design is both incredibly effective and frustratingly simple: augment N
times, predict, retain the most confident predictions, and marginalize after
setting the Softmax temperature to zero. Remarkably, ZERO requires a single
batched forward pass through the vision encoder only and no backward passes. We
thoroughly evaluate our approach following the experimental protocol
established in the literature and show that ZERO largely surpasses or compares
favorably w.r.t. the state-of-the-art while being almost 10x faster and 13x
more memory-friendly than standard Test-Time Prompt Tuning. Thanks to its
simplicity and comparatively negligible computation, ZERO can serve as a strong
baseline for future work in this field. The code is available at
this https URL

研究表明，零温度的 TTA 方法（ZERO）能够在只进行一次前向传播的情况下，准确性大大超过或与现有技术相当，且速度约为 10 倍快，内存占用约为 13 倍少。