In the rapidly evolving landscape of spoken question-answering (SQA), the
integration of large language models (LLMs) has emerged as a transformative
development. Conventional approaches often entail the use of separate models
for question audio transcription and answer selection, resulting in significant
resource utilization and error accumulation. To tackle these challenges, we
explore the effectiveness of end-to-end (E2E) methodologies for SQA in the
medical domain. Our study introduces a novel zero-shot SQA approach, compared
to traditional cascade systems. Through a comprehensive evaluation conducted on
a new open benchmark of 8 medical tasks and 48 hours of synthetic audio, we
demonstrate that our approach requires up to 14.7 times fewer resources than a
combined 1.3B parameters LLM with a 1.55B parameters ASR model while improving
average accuracy by 0.5\%. These findings underscore the potential of E2E
methodologies for SQA in resource-constrained contexts.

通过在医学领域中探索端到端方法，我们引入了一种新颖的零 - shot 问答方法，与传统的级联系统相比，我们的方法在 8 个医学任务和 48 小时合成音频的全面评估中，相对于一个带有 1.3B 参数 LLM 和 1.55B 参数 ASR 模型的组合模型，节约资源高达 14.7 倍，同时提高了 0.5% 的平均准确性，凸显了端到端方法在资源受限环境中进行口述问答的潜力。