In this work, we empirically confirm that non-autoregressive translation with
an iterative refinement mechanism (IR-NAT) suffers from poor acceleration
robustness because it is more sensitive to decoding batch size and computing
device setting than autoregressive translation (AT). Inspired by it, we attempt
to investigate how to combine the strengths of autoregressive and
non-autoregressive translation paradigms better. To this end, we demonstrate
through synthetic experiments that prompting a small number of AT's predictions
can promote one-shot non-autoregressive translation to achieve the equivalent
performance of IR-NAT. Following this line, we propose a new two-stage
translation prototype called hybrid-regressive translation (HRT). Specifically,
HRT first generates discontinuous sequences via autoregression (e.g., make a
prediction every k tokens, k>1) and then fills in all previously skipped tokens
at once in a non-autoregressive manner. We also propose a bag of techniques to
effectively and efficiently train HRT without adding any model parameters. HRT
achieves the state-of-the-art BLEU score of 28.49 on the WMT En-De task and is
at least 1.5x faster than AT, regardless of batch size and device. In addition,
another bonus of HRT is that it successfully inherits the good characteristics
of AT in the deep-encoder-shallow-decoder architecture. Concretely, compared to
the vanilla HRT with a 6-layer encoder and 6-layer decoder, the inference speed
of HRT with a 12-layer encoder and 1-layer decoder is further doubled on both
GPU and CPU without BLEU loss.

本文提出了一种叫做混合回归翻译的翻译模型，它结合了自回归翻译和非自回归翻译模式的优势，通过在自回归过程中预测部分结果，再通过非自回归方式填充其余结果的方式，开发出了一种基于这两种模式的翻译模型，并对其效率和准确性进行了评估和比较。

混合回归神经机器翻译

Hybrid-Regressive Neural Machine Translation

Node classification on attributed networks is a semi-supervised task that is
crucial for network analysis. By decoupling two critical operations in Graph
Convolutional Networks (GCNs), namely feature transformation and neighborhood
aggregation, some recent works of decoupled GCNs could support the information
to propagate deeper and achieve advanced performance. However, they follow the
traditional structure-aware propagation strategy of GCNs, making it hard to
capture the attribute correlation of nodes and sensitive to the structure noise
described by edges whose two endpoints belong to different categories. To
address these issues, we propose a new method called the itshape Propagation
with Adaptive Mask then Training (PAMT). The key idea is to integrate the
attribute similarity mask into the structure-aware propagation process. In this
way, PAMT could preserve the attribute correlation of adjacent nodes during the
propagation and effectively reduce the influence of structure noise. Moreover,
we develop an iterative refinement mechanism to update the similarity mask
during the training process for improving the training performance. Extensive
experiments on four real-world datasets demonstrate the superior performance
and robustness of PAMT.

该研究提出了一种名为 PAMT 的新方法，利用属性相似性掩码集成了结构感知传播过程，从而能够在传播过程中保留相邻节点的属性相关性并有效减少结构噪声的影响，同时开发了一种迭代细化机制来改善训练性能，该方法在 4 个真实世界数据集上表现出了卓越的性能和鲁棒性。