The incorporation of biasing words obtained through contextual knowledge is
of paramount importance in automatic speech recognition (ASR) applications.
This paper proposes an innovative method for achieving end-to-end contextual
ASR using graph neural network (GNN) encodings based on the tree-constrained
pointer generator method. GNN node encodings facilitate lookahead for future
word pieces in the process of ASR decoding at each tree node by incorporating
information about all word pieces on the tree branches rooted from it. This
results in a more precise prediction of the generation probability of the
biasing words. The study explores three GNN encoding techniques, namely tree
recursive neural networks, graph convolutional network (GCN), and GraphSAGE,
along with different combinations of the complementary GCN and GraphSAGE
structures. The performance of the systems was evaluated using the Librispeech
and AMI corpus, following the visual-grounded contextual ASR pipeline. The
findings indicate that using GNN encodings achieved consistent and significant
reductions in word error rate (WER), particularly for words that are rare or
have not been seen during the training process. Notably, the most effective
combination of GNN encodings obtained more than 60% WER reduction for rare and
unseen words compared to standard end-to-end systems.

本文提出了一种基于图神经网络编码的端到端上下文自动语音识别新方法，通过树限制指针生成方法，将上下文知识中的偏向词整合到识别过程中，显著降低了词误率。