Automatic Speech Recognition (ASR) systems frequently use a search-based
decoding strategy aiming to find the best attainable transcript by considering
multiple candidates. One prominent speech recognition decoding heuristic is
beam search, which seeks the transcript with the greatest likelihood computed
using the predicted distribution. While showing substantial performance gains
in various tasks, beam search loses some of its effectiveness when the
predicted probabilities are highly confident, i.e., the predicted distribution
is massed for a single or very few classes. We show that recently proposed
Self-Supervised Learning (SSL)-based ASR models tend to yield exceptionally
confident predictions that may hamper beam search from truly considering a
diverse set of candidates. We perform a layer analysis to reveal and visualize
how predictions evolve, and propose a decoding procedure that improves the
performance of fine-tuned ASR models. Our proposed approach does not require
further training beyond the original fine-tuning, nor additional model
parameters. In fact, we find that our proposed method requires significantly
less inference computation than current approaches. We propose aggregating the
top M layers, potentially leveraging useful information encoded in intermediate
layers, and relaxing model confidence. We demonstrate the effectiveness of our
approach by conducting an empirical study on varying amounts of labeled
resources and different model sizes, showing consistent improvements in
particular when applied to low-resource scenarios.

该论文提出了一种解码过程，改进了现有的基于 beam search 的自动语音识别系统中的缺陷，特别是针对最近提出的自监督学习（SSL）模型中高度自信的预测所带来的问题，并且不需要额外的训练以及模型参数，并且在低资源场景中实现了一致的改进。