Test-Time Adaptation (TTA) is a critical paradigm for tackling distribution
shifts during inference, especially in visual recognition tasks. However, while
acoustic models face similar challenges due to distribution shifts in test-time
speech, TTA techniques specifically designed for acoustic modeling in the
context of open-world data shifts remain scarce. This gap is further
exacerbated when considering the unique characteristics of acoustic foundation
models: 1) they are primarily built on transformer architectures with layer
normalization and 2) they deal with test-time speech data of varying lengths in
a non-stationary manner. These aspects make the direct application of
vision-focused TTA methods, which are mostly reliant on batch normalization and
assume independent samples, infeasible. In this paper, we delve into TTA for
pre-trained acoustic models facing open-world data shifts. We find that noisy,
high-entropy speech frames, often non-silent, carry key semantic content.
Traditional TTA methods might inadvertently filter out this information using
potentially flawed heuristics. In response, we introduce a heuristic-free,
learning-based adaptation enriched by confidence enhancement. Noting that
speech signals' short-term consistency, we also apply consistency
regularization during test-time optimization. Our experiments on synthetic and
real-world datasets affirm our method's superiority over existing baselines.

本论文研究了面对开放世界数据变化的预训练声学模型的测试时间适应方法，通过引入无启发式、基于学习的自适应方法并结合置信度提升以及在测试时间优化中应用一致性规范化，证明了该方法在合成和真实数据集上的优越性。