Existing test-time adaptation (TTA) approaches often adapt models with the
unlabeled testing data stream. A recent attempt relaxed the assumption by
introducing limited human annotation, referred to as Human-In-the-Loop
Test-Time Adaptation (HILTTA) in this study. The focus of existing HILTTA lies
on selecting the most informative samples to label, a.k.a. active learning. In
this work, we are motivated by a pitfall of TTA, i.e. sensitive to
hyper-parameters, and propose to approach HILTTA by synergizing active learning
and model selection. Specifically, we first select samples for human annotation
(active learning) and then use the labeled data to select optimal
hyper-parameters (model selection). A sample selection strategy is tailored for
choosing samples by considering the balance between active learning and model
selection purposes. We demonstrate on 4 TTA datasets that the proposed HILTTA
approach is compatible with off-the-shelf TTA methods which outperform the
state-of-the-art HILTTA methods and stream-based active learning methods.
Importantly, our proposed method can always prevent choosing the worst
hyper-parameters on all off-the-shelf TTA methods. The source code will be
released upon publication.

本研究提出了一种基于主动学习和模型选择的人在环测试时间自适应方法，通过选择具有信息量的样本进行标记并使用已标记的数据选择最优超参数，实验结果表明该方法在 4 个测试时间自适应数据集上优于最先进的方法和基于流的主动学习方法。