In this paper, we propose Shallow Aggressive Decoding (SAD) to improve the online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC). SAD optimizes the online inference efficiency for GEC by two innovations: 1) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism; 2) it uses a shallow decoder instead of the conventional Transformer architecture with balanced encoder-decoder depth to reduce the computational cost during inference. Experiments in both English and Chinese GEC benchmarks show that aggressive decoding could yield the same predictions as greedy decoding but with a significant speedup for online inference. Its combination with the shallow decoder could offer an even higher online inference speedup over the powerful Transformer baseline without quality loss. Not only does our approach allow a single model to achieve the state-of-the-art results in English GEC benchmarks: 66.4 F0.5 in the CoNLL-14 and 72.9 F0.5 in the BEA-19 test set with an almost 10x online inference speedup over the Transformer-big model, but also it is easily adapted to other languages. Our code is available at https://github.com/AutoTemp/Shallow-Aggressive-Decoding.

本文提出Shallow Aggressive Decoding（SAD）以提高Transformer模型在瞬时语法错误纠正（GEC）任务中的在线推断效率，该方法通过两种创新的方式优化在GEC中的在线推断效率：1）积极地并行解码尽可能多的符号以提高计算并行性；2）采用浅解码器，而非传统的具有平衡编码器-解码器深度的Transformer结构以降低推断过程中的计算成本。实验结果表明，积极解码可以产生与贪婪解码相同的预测，但在线推理速度显着提高。与浅解码器相结合，可以在不降低质量的情况下进一步提高在线推理速度。该方法不仅可以使单一模型在英语GEC基准测试中取得最先进的结果，还可以轻松适应其他语言。

浅层激进解码实现即时语法错误纠正