Personalization in multi-turn dialogs has been a long standing challenge for
end-to-end automatic speech recognition (E2E ASR) models. Recent work on
contextual adapters has tackled rare word recognition using user catalogs. This
adaptation, however, does not incorporate an important cue, the dialog act,
which is available in a multi-turn dialog scenario. In this work, we propose a
dialog act guided contextual adapter network. Specifically, it leverages dialog
acts to select the most relevant user catalogs and creates queries based on
both -- the audio as well as the semantic relationship between the carrier
phrase and user catalogs to better guide the contextual biasing. On industrial
voice assistant datasets, our model outperforms both the baselines - dialog act
encoder-only model, and the contextual adaptation, leading to the most
improvement over the no-context model: 58% average relative word error rate
reduction (WERR) in the multi-turn dialog scenario, in comparison to the
prior-art contextual adapter, which has achieved 39% WERR over the no-context
model.

本研究提出了一种对话行为引导的上下文适配器网络，改善了多轮对话中自然语言识别的性能，相较于现有技术提高了数量级，具体表现为相对相对单词误差率降低 58%。

对话行为引导的上下文适应器用于个性化语音识别

Dialog act guided contextual adapter for personalized speech recognition

In multi-turn dialogs, natural language understanding models can introduce
obvious errors by being blind to contextual information. To incorporate dialog
history, we present a neural architecture with Speaker-Sensitive Dual Memory
Networks which encode utterances differently depending on the speaker. This
addresses the different extents of information available to the system - the
system knows only the surface form of user utterances while it has the exact
semantics of system output. We performed experiments on real user data from
Microsoft Cortana, a commercial personal assistant. The result showed a
significant performance improvement over the state-of-the-art slot tagging
models using contextual information.

提出了一种基于说话者敏感的双重记忆网络的神经结构，可以编码不同说话者的话语，从而在多轮对话中使用上下文信息进行输入的自然语言理解，从使用 Microsoft Cortana 的真实用户数据上进行的实验表明，与使用上下文信息的最先进的槽标记模型相比，可以明显提高性能。