We tackle the problem of semantic image layout manipulation, which aims to
manipulate an input image by editing its semantic label map. A core problem of
this task is how to transfer visual details from the input images to the new
semantic layout while making the resulting image visually realistic. Recent
work on learning cross-domain correspondence has shown promising results for
global layout transfer with dense attention-based warping. However, this method
tends to lose texture details due to the resolution limitation and the lack of
smoothness constraint of correspondence. To adapt this paradigm for the layout
manipulation task, we propose a high-resolution sparse attention module that
effectively transfers visual details to new layouts at a resolution up to
512x512. To further improve visual quality, we introduce a novel generator
architecture consisting of a semantic encoder and a two-stage decoder for
coarse-to-fine synthesis. Experiments on the ADE20k and Places365 datasets
demonstrate that our proposed approach achieves substantial improvements over
the existing inpainting and layout manipulation methods.

该研究提出了一种高分辨率稀疏注意力模块和一个生成器架构，可以在语义标签映射的基础上，有效地实现将输入图像的视觉细节转移至新布局。实验证明，该方法在图像修补和布局操作方面的性能得到了大幅提高。

高分辨率稀疏注意力下的语义布局操作

Semantic Layout Manipulation with High-Resolution Sparse Attention

In this paper, we propose an end-to-end self-driving network featuring a
sparse attention module that learns to automatically attend to important
regions of the input. The attention module specifically targets motion
planning, whereas prior literature only applied attention in perception tasks.
Learning an attention mask directly targeted for motion planning significantly
improves the planner safety by performing more focused computation.
Furthermore, visualizing the attention improves interpretability of end-to-end
self-driving.

本文提出一种全端到端的自动驾驶网络，采用稀疏注意力模块，学习自动关注输入的重要区域。注意力模块专门针对运动规划，而先前的文献只在感知任务中应用注意力。直接学习针对运动规划的注意力掩模显著提高了规划器的安全性，并且可视化注意力提高了全端到端自动驾驶的可解释性。