Automatic Speech Recognition (ASR) systems have attained unprecedented
performance with large speech models pre-trained based on self-supervised
speech representation learning. However, these pre-trained speech models suffer
from representational bias as they tend to better represent those prominent
accents (i.e., native (L1) English accent) in the pre-training speech corpus
than less represented accents, resulting in a deteriorated performance for
non-native (L2) English accents. Although there have been some approaches to
mitigate this issue, all of these methods require updating the pre-trained
model weights. In this paper, we propose Information Theoretic Adversarial
Prompt Tuning (INTapt), which introduces prompts concatenated to the original
input that can re-modulate the attention of the pre-trained model such that the
corresponding input resembles a native (L1) English speech without updating the
backbone weights. INTapt is trained simultaneously in the following two
manners: (1) adversarial training to reduce accent feature dependence between
the original input and the prompt-concatenated input and (2) training to
minimize CTC loss for improving ASR performance to a prompt-concatenated input.
Experimental results show that INTapt improves the performance of L2 English
and increases feature similarity between L2 and L1 accents.

提出 Information Theoretic Adversarial Prompt Tuning (INTapt) 方法来解决 pre-trained speech model 表示偏差的问题，通过引入 prompts 来重新调节预训练模型的注意力，从而改善非母语英语说话者的 ASR 性能，同时增加 L2 英语口音与 L1 英语口音之间的特征相似性。