Deep convolutional networks (CNNs) have exhibited their potential in image
inpainting for producing plausible results. However, in most existing methods,
e.g., context encoder, the missing parts are predicted by propagating the
surrounding convolutional features through a fully connected layer, which
intends to produce semantically plausible but blurry result. In this paper, we
introduce a special shift-connection layer to the U-Net architecture, namely
Shift-Net, for filling in missing regions of any shape with sharp structures
and fine-detailed textures. To this end, the encoder feature of the known
region is shifted to serve as an estimation of the missing parts. A guidance
loss is introduced on decoder feature to minimize the distance between the
decoder feature after fully connected layer and the ground-truth encoder
feature of the missing parts. With such constraint, the decoder feature in
missing region can be used to guide the shift of encoder feature in known
region. An end-to-end learning algorithm is further developed to train the
Shift-Net. Experiments on the Paris StreetView and Places datasets demonstrate
the efficiency and effectiveness of our Shift-Net in producing sharper,
fine-detailed, and visually plausible results. The codes and pre-trained models
are available at this https URL

本文提出了一种名为 Shift-Net 的 U-Net 架构的特殊移位连接层，用于使用编码器任务中已知的区域以估算缺失部分，并通过引入辅助损失函数来训练整个模型，实现图像修复中具有更加清晰，精细纹理和逼真视觉效果的结果。