Denoising diffusion probabilistic models for image inpainting aim to add the
noise to the texture of image during the forward process and recover masked
regions with unmasked ones of the texture via the reverse denoising
process.Despite the meaningful semantics generation,the existing arts suffer
from the semantic discrepancy between masked and unmasked regions, since the
semantically dense unmasked texture fails to be completely degraded while the
masked regions turn to the pure noise in diffusion process,leading to the large
discrepancy between them.In this paper,we aim to answer how unmasked semantics
guide texture denoising process;together with how to tackle the semantic
discrepancy,to facilitate the consistent and meaningful semantics generation.To
this end,we propose a novel structure-guided diffusion model named
StrDiffusion,to reformulate the conventional texture denoising process under
structure guidance to derive a simplified denoising objective for image
inpainting,while revealing:1) the semantically sparse structure is beneficial
to tackle semantic discrepancy in early stage, while dense texture generates
reasonable semantics in late stage;2) the semantics from unmasked regions
essentially offer the time-dependent structure guidance for the texture
denoising process,benefiting from the time-dependent sparsity of the structure
semantics.For the denoising process,a structure-guided neural network is
trained to estimate the simplified denoising objective by exploiting the
consistency of the denoised structure between masked and unmasked
regions.Besides,we devise an adaptive resampling strategy as a formal criterion
as whether structure is competent to guide the texture denoising process,while
regulate their semantic correlations.Extensive experiments validate the merits
of StrDiffusion over the state-of-the-arts.Our code is available at
this https URL

基于结构引导的扩散模型在图像修复中通过添加噪音来从前向过程中对图像纹理进行修复，并通过反向去噪过程使用与图像纹理相同的未遮挡区域恢复已遮挡区域，解决了现有技术中遮挡和未遮挡区域之间的语义差异问题。

结构的重要性：解决图像修复中扩散模型的语义差异问题

Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models  for Image Inpainting

There is a broad consensus that news media outlets incorporate ideological
biases in their news articles. However, prior studies on measuring the
discrepancies among media outlets and further dissecting the origins of
semantic differences suffer from small sample sizes and limited scope. In this
study, we collect a large dataset of 1.8 million news headlines from major U.S.
media outlets spanning from 2014 to 2022 to thoroughly track and dissect the
semantic discrepancy in U.S. news media. We employ multiple correspondence
analysis (MCA) to quantify the semantic discrepancy relating to four prominent
topics - domestic politics, economic issues, social issues, and foreign
affairs. Additionally, we compare the most frequent n-grams in media headlines
to provide further qualitative insights into our analysis. Our findings
indicate that on domestic politics and social issues, the discrepancy can be
attributed to a certain degree of media bias. Meanwhile, the discrepancy in
reporting foreign affairs is largely attributed to the diversity in individual
journalistic styles. Finally, U.S. media outlets show consistency and high
similarity in their coverage of economic issues.

通过采用多重对应分析 (MCA) 测量及洞悉 2014 年至 2022 年间 1.8 百万份美国主要媒体新闻头条的语义差异，该研究发现美国国内政治和社会问题的媒体报道存在一定程度的媒体偏见，而外交报道的偏差主要来自个人的新闻风格多样性，而在经济问题的报道上，美国媒体呈现出一致性和高度相似性。