Speech recognition applications cover a range of different audio and text
distributions, with different speaking styles, background noise, transcription
punctuation and character casing. However, many speech recognition systems
require dataset-specific tuning (audio filtering, punctuation removal and
normalisation of casing), therefore assuming a-priori knowledge of both the
audio and text distributions. This tuning requirement can lead to systems
failing to generalise to other datasets and domains. To promote the development
of multi-domain speech systems, we introduce the End-to-end Speech Benchmark
(ESB) for evaluating the performance of a single automatic speech recognition
(ASR) system across a broad set of speech datasets. Benchmarked systems must
use the same data pre- and post-processing algorithm across datasets - assuming
the audio and text data distributions are a-priori unknown. We compare a series
of state-of-the-art (SoTA) end-to-end (E2E) systems on this benchmark,
demonstrating how a single speech system can be applied and evaluated on a wide
range of data distributions. We find E2E systems to be effective across
datasets: in a fair comparison, E2E systems achieve within 2.6% of SoTA systems
tuned to a specific dataset. Our analysis reveals that transcription artefacts,
such as punctuation and casing, pose difficulties for ASR systems and should be
included in evaluation. We believe E2E benchmarking over a range of datasets
promotes the research of multi-domain speech recognition systems. ESB is
available at this https URL.

本文介绍了一个最新的基于端到端的语音识别测试平台，探讨了跨多个不同数据集和语音分布时对识别相同语音流的支持，并对不同的系统进行了比较分析分析，发现端到端系统在不同数据集的应用效果良好，并指出了如何改进现有语音识别系统存在的问题。

ESB：多域端到端语音识别基准测试

ESB: A Benchmark For Multi-Domain End-to-End Speech Recognition

In the slot-filling paradigm, where a user can refer back to slots in the
context during a conversation, the goal of the contextual understanding system
is to resolve the referring expressions to the appropriate slots in the
context. In large-scale multi-domain systems, this presents two challenges -
scaling to a very large and potentially unbounded set of slot values, and
dealing with diverse schemas. We present a neural network architecture that
addresses the slot value scalability challenge by reformulating the contextual
interpretation as a decision to carryover a slot from a set of possible
candidates. To deal with heterogenous schemas, we introduce a simple
data-driven method for trans- forming the candidate slots. Our experiments show
that our approach can scale to multiple domains and provides competitive
results over a strong baseline.

本研究提出一种神经网络架构来解决在多域系统中的上下文理解和词槽填充问题，着重解决如何面对大规模和多样化的问题，模型通过候选集合来决定是否填充词槽，并引入了简单但有效的数据驱动方法用于转换候选词槽。实验表明该方法可以在多个领域中提供最优结果。