Conventional neural autoregressive decoding commonly assumes a fixed
left-to-right generation order, which may be sub-optimal. In this work, we
propose a novel decoding algorithm -- InDIGO -- which supports flexible
sequence generation in arbitrary orders through insertion operations. We extend
Transformer, a state-of-the-art sequence generation model, to efficiently
implement the proposed approach, enabling it to be trained with either a
pre-defined generation order or adaptive orders obtained from beam-search.
Experiments on four real-world tasks, including word order recovery, machine
translation, image caption and code generation, demonstrate that our algorithm
can generate sequences following arbitrary orders, while achieving competitive
or even better performance compared to the conventional left-to-right
generation. The generated sequences show that InDIGO adopts adaptive generation
orders based on input information.

该论文提出了一种名为 InDIGO 的新型解码算法，该算法通过插入操作支持任意顺序的序列生成，并使用 Transformer 实现。实验结果显示，InDIGO 能够生成遵循任意顺序的序列，同时与传统的从左到右生成方法相比表现出竞争力甚至更好的性能。