Test-time adaptation with pre-trained vision-language models has attracted
increasing attention for tackling distribution shifts during the test time.
Though prior studies have achieved very promising performance, they involve
intensive computation which is severely unaligned with test-time adaptation. We
design TDA, a training-free dynamic adapter that enables effective and
efficient test-time adaptation with vision-language models. TDA works with a
lightweight key-value cache that maintains a dynamic queue with few-shot pseudo
labels as values and the corresponding test-sample features as keys. Leveraging
the key-value cache, TDA allows adapting to test data gradually via progressive
pseudo label refinement which is super-efficient without incurring any
backpropagation. In addition, we introduce negative pseudo labeling that
alleviates the adverse impact of pseudo label noises by assigning pseudo labels
to certain negative classes when the model is uncertain about its pseudo label
predictions. Extensive experiments over two benchmarks demonstrate TDA's
superior effectiveness and efficiency as compared with the state-of-the-art.
The code has been released in https://kdiaaa.github.io/tda/.

TDA 是一种训练 - free 的动态适配器，通过轻量级的键值缓存和逐步的伪标签细化，以及引入负伪标签，实现了有效和高效的测试时自适应。