Lattices are compact representations that encode multiple hypotheses, such as
speech recognition results or different word segmentations. It is shown that
encoding lattices as opposed to 1-best results generated by automatic speech
recognizer (ASR) boosts the performance of spoken language understanding (SLU).
Recently, pretrained language models with the transformer architecture have
achieved the state-of-the-art results on natural language understanding, but
their ability of encoding lattices has not been explored. Therefore, this paper
aims at adapting pretrained transformers to lattice inputs in order to perform
understanding tasks specifically for spoken language. Our experiments on the
benchmark ATIS dataset show that fine-tuning pretrained transformers with
lattice inputs yields clear improvement over fine-tuning with 1-best results.
Further evaluation demonstrates the effectiveness of our methods under
different acoustic conditions. Our code is available at
this https URL

本篇论文旨在将 Transformer 预训练模型适应于格输入以执行口语理解任务，并在 ATIS 基准数据集上进行了实验，结果表明，将预训练的 Transformers fine-tuning 用于格输入比用于 1-best 结果有了显著改进，证明了方法的有效性。