Pretrained using large amount of data, autoregressive language models are able to generate high quality sequences. However, these models do not perform well under hard lexical constraints as they lack fine control of content generation process. Progressive insertion-based transformers can overcome the above limitation and efficiently generate a sequence in parallel given some input tokens as constraint. These transformers however may fail to support hard lexical constraints as their generation process is more likely to terminate prematurely. The paper analyses such early termination problems and proposes the Entity-constrained insertion transformer (ENCONTER), a new insertion transformer that addresses the above pitfall without compromising much generation efficiency. We introduce a new training strategy that considers predefined hard lexical constraints (e.g., entities to be included in the generated sequence). Our experiments show that ENCONTER outperforms other baseline models in several performance metrics rendering it more suitable in practical applications. Our code is available at https://github.com/LARC-CMU-SMU/Enconter

该研究分析自回归语言模型在硬词汇约束下表现不佳的问题，并提出了一种称为Entity-constrained insertion transformer (ENCONTER)的新型插入变压器，该变压器在不损失太多生成效率的情况下解决了这个问题。实验结果显示，ENCONTER在多个性能指标上优于其他基线模型，更适用于实际应用。

ENCONTER: 基于插入式Transformer的实体约束渐进式序列生成