Inpainting arbitrary missing regions is challenging because learning valid
features for various masked regions is nontrivial. Though U-shaped
encoder-decoder frameworks have been witnessed to be successful, most of them
share a common drawback of mask unawareness in feature extraction because all
convolution windows (or regions), including those with various shapes of
missing pixels, are treated equally and filtered with fixed learned kernels. To
this end, we propose our novel mask-aware inpainting solution. Firstly, a
Mask-Aware Dynamic Filtering (MADF) module is designed to effectively learn
multi-scale features for missing regions in the encoding phase. Specifically,
filters for each convolution window are generated from features of the
corresponding region of the mask. The second fold of mask awareness is achieved
by adopting Point-wise Normalization (PN) in our decoding phase, considering
that statistical natures of features at masked points differentiate from those
of unmasked points. The proposed PN can tackle this issue by dynamically
assigning point-wise scaling factor and bias. Lastly, our model is designed to
be an end-to-end cascaded refinement one. Supervision information such as
reconstruction loss, perceptual loss and total variation loss is incrementally
leveraged to boost the inpainting results from coarse to fine. Effectiveness of
the proposed framework is validated both quantitatively and qualitatively via
extensive experiments on three public datasets including Places2, CelebA and
Paris StreetView.

提出了一种新的基于掩模感知的修复算法，使用 Mask-Aware Dynamic Filtering (MADF) 学习缺失区域的多尺度特征，采用 Point-wise Normalization (PN) 动态分配权重，在一个端到端的框架中逐步利用重建损失、感知损失和总变分损失进行逐步优化。实验证明该算法在三个公共数据集上表现得非常出色。