Given an incomplete image without additional constraint, image inpainting
natively allows for multiple solutions as long as they appear plausible.
Recently, multiplesolution inpainting methods have been proposed and shown the
potential of generating diverse results. However, these methods have difficulty
in ensuring the quality of each solution, e.g. they produce distorted structure
and/or blurry texture. We propose a two-stage model for diverse inpainting,
where the first stage generates multiple coarse results each of which has a
different structure, and the second stage refines each coarse result separately
by augmenting texture. The proposed model is inspired by the hierarchical
vector quantized variational auto-encoder (VQ-VAE), whose hierarchical
architecture isentangles structural and textural information. In addition, the
vector quantization in VQVAE enables autoregressive modeling of the discrete
distribution over the structural information. Sampling from the distribution
can easily generate diverse and high-quality structures, making up the first
stage of our model. In the second stage, we propose a structural attention
module inside the texture generation network, where the module utilizes the
structural information to capture distant correlations. We further reuse the
VQ-VAE to calculate two feature losses, which help improve structure coherence
and texture realism, respectively. Experimental results on CelebA-HQ, Places2,
and ImageNet datasets show that our method not only enhances the diversity of
the inpainting solutions but also improves the visual quality of the generated
multiple images. Code and models are available at:
this https URL

本文提出了一种基于 VQ-VAE 的两阶段模型，在第一阶段生成多个不同结构的粗略结果，第二阶段对于每个粗略结果进行纹理增强，其中纹理生成网络中的结构注意模块利用结构信息捕获远程相关性。这种方法不仅增强了图像修复解的多样性，还提高了生成的多张图像的视觉质量。

采用分层 VQ-VAE 对图像修复生成多样化结构

Generating Diverse Structure for Image Inpainting With Hierarchical  VQ-VAE

Attentional, RNN-based encoder-decoder architectures have achieved impressive
performance on abstractive summarization of news articles. However, these
methods fail to account for long term dependencies within the sentences of a
document. This problem is exacerbated in multi-document summarization tasks
such as summarizing the popular opinion in threads present in community
question answering (CQA) websites such as Yahoo! Answers and Quora. These
threads contain answers which often overlap or contradict each other. In this
work, we present a hierarchical encoder based on structural attention to model
such inter-sentence and inter-document dependencies. We set the popular
pointer-generator architecture and some of the architectures derived from it as
our baselines and show that they fail to generate good summaries in a
multi-document setting. We further illustrate that our proposed model achieves
significant improvement over the baselines in both single and multi-document
summarization settings -- in the former setting, it beats the best baseline by
1.31 and 7.8 ROUGE-1 points on CNN and CQA datasets, respectively; in the
latter setting, the performance is further improved by 1.6 ROUGE-1 points on
the CQA dataset.

本文提出了一种基于结构化注意力的分层编码器，用于建模句内和文档间相互依赖关系，在多文档摘要生成任务中表现出显著的改进，击败了现有的模型。