Improving ASR systems is necessary to make new LLM-based use-cases accessible
to people across the globe. In this paper, we focus on Indian languages, and
make the case that diverse benchmarks are required to evaluate and improve ASR
systems for Indian languages. To address this, we collate Vistaar as a set of
59 benchmarks across various language and domain combinations, on which we
evaluate 3 publicly available ASR systems and 2 commercial systems. We also
train IndicWhisper models by fine-tuning the Whisper models on publicly
available training datasets across 12 Indian languages totalling to 10.7K
hours. We show that IndicWhisper significantly improves on considered ASR
systems on the Vistaar benchmark. Indeed, IndicWhisper has the lowest WER in 39
out of the 59 benchmarks, with an average reduction of 4.1 WER. We open-source
all datasets, code and models.

本文提出了 Vistaar，59 个受试语言和领域组合的基准，用于评估和改进三个公开的 ASR 系统和两个商业系统，同时使用 IndicWhisper 通过在 12 种印度语言上进行微调，在 Vistaar 基准中明显改善了考虑到的 ASR 系统。