Recently deep neutral networks have achieved promising performance for
filling large missing regions in image inpainting tasks. They usually adopted
the standard convolutional architecture over the corrupted image, leading to
meaningless contents, such as color discrepancy, blur and artifacts. Moreover,
most inpainting approaches cannot well handle the large continuous missing area
cases. To address these problems, we propose a generic inpainting framework
capable of handling with incomplete images on both continuous and discontinuous
large missing areas, in an adversarial manner. From which, region-wise
convolution is deployed in both generator and discriminator to separately
handle with the different regions, namely existing regions and missing ones.
Moreover, a correlation loss is introduced to capture the non-local
correlations between different patches, and thus guides the generator to obtain
more information during inference. With the help of our proposed framework, we
can restore semantically reasonable and visually realistic images. Extensive
experiments on three widely-used datasets for image inpainting tasks have been
conducted, and both qualitative and quantitative experimental results
demonstrate that the proposed model significantly outperforms the
state-of-the-art approaches, both on the large continuous and discontinuous
missing areas.

我们提出了一个通用的修复损坏图像的网络，能够分开处理连续和不连续区域，使用区域卷积来处理已有区域和缺失区域，并引入互相关损失以提供更多信息。实验结果表明，相对于现有方法，我们的方法在整张图片上能更精确地还原出缺失的区域。

基于区域的生成对抗图像修复技术：针对大面积缺失区域

Region-wise Generative Adversarial ImageInpainting for Large Missing  Areas

Cross-modality generation is an emerging topic that aims to synthesize data
in one modality based on information in a different modality. In this paper, we
consider a task of such: given an arbitrary audio speech and one lip image of
arbitrary target identity, generate synthesized lip movements of the target
identity saying the speech. To perform well in this task, it inevitably
requires a model to not only consider the retention of target identity,
photo-realistic of synthesized images, consistency and smoothness of lip images
in a sequence, but more importantly, learn the correlations between audio
speech and lip movements. To solve the collective problems, we explore the best
modeling of the audio-visual correlations in building and training a
lip-movement generator network. Specifically, we devise a method to fuse audio
and image embeddings to generate multiple lip images at once and propose a
novel correlation loss to synchronize lip changes and speech changes. Our final
model utilizes a combination of four losses for a comprehensive consideration
in generating lip movements; it is trained in an end-to-end fashion and is
robust to lip shapes, view angles and different facial characteristics.
Thoughtful experiments on three datasets ranging from lab-recorded to lips
in-the-wild show that our model significantly outperforms other
state-of-the-art methods extended to this task.

本文研究了跨模态生成的任务，通过融合音频和图像嵌入，提出了一种新的关联损失函数和利用四种损失函数生成唇部运动的端到端模型，并在多个数据集上的实验中验证了该模型对唇形状，视角和面部特征的鲁棒性。