Human-spoken questions are critical to evaluating the performance of spoken
question answering (SQA) systems that serve several real-world use cases
including digital assistants. We present a new large-scale community-shared SQA
dataset, HeySQuAD that consists of 76k human-spoken questions and 97k
machine-generated questions and corresponding textual answers derived from the
SQuAD QA dataset. The goal of HeySQuAD is to measure the ability of machines to
understand noisy spoken questions and answer the questions accurately. To this
end, we run extensive benchmarks on the human-spoken and machine-generated
questions to quantify the differences in noise from both sources and its
subsequent impact on the model and answering accuracy. Importantly, for the
task of SQA, where we want to answer human-spoken questions, we observe that
training using the transcribed human-spoken and original SQuAD questions leads
to significant improvements (12.51%) over training using only the original
SQuAD textual questions.

本研究主要介绍了一个新的基于 HeySQuAD 数据集的 SQA 系统基准测试方法，该方法使用了包含 76k 个人类口述问题和 97k 个机器生成问题的问答数据集，证明了使用转录的口述问题进行训练可以显著提高系统的性能 (12.51%)，具有很高的实践价值。