Recent advancements in large-scale pre-trained text-to-image models have led
to remarkable progress in semantic image synthesis. Nevertheless, synthesizing
high-quality images with consistent semantics and layout remains a challenge.
In this paper, we propose the adaPtive LAyout-semantiC fusion modulE (PLACE)
that harnesses pre-trained models to alleviate the aforementioned issues.
Specifically, we first employ the layout control map to faithfully represent
layouts in the feature space. Subsequently, we combine the layout and semantic
features in a timestep-adaptive manner to synthesize images with realistic
details. During fine-tuning, we propose the Semantic Alignment (SA) loss to
further enhance layout alignment. Additionally, we introduce the Layout-Free
Prior Preservation (LFP) loss, which leverages unlabeled data to maintain the
priors of pre-trained models, thereby improving the visual quality and semantic
consistency of synthesized images. Extensive experiments demonstrate that our
approach performs favorably in terms of visual quality, semantic consistency,
and layout alignment. The source code and model are available at
this https URL

提出了 adaPtive LAyout-semantiC fusion modulE (PLACE) 以及附加的 Semantic Alignment (SA) loss，通过利用预训练模型来改善语义图像合成中的布局和语义问题，实现合成图像的真实细节和语义一致性。

PLACE：自适应布局 - 语义融合的语义图像合成

PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis

3D visual grounding aims to find the object within point clouds mentioned by
free-form natural language descriptions with rich semantic cues. However,
existing methods either extract the sentence-level features coupling all words
or focus more on object names, which would lose the word-level information or
neglect other attributes. To alleviate these issues, we present EDA that
Explicitly Decouples the textual attributes in a sentence and conducts Dense
Alignment between such fine-grained language and point cloud objects.
Specifically, we first propose a text decoupling module to produce textual
features for every semantic component. Then, we design two losses to supervise
the dense matching between two modalities: position alignment loss and semantic
alignment loss. On top of that, we further introduce a new visual grounding
task, locating objects without object names, which can thoroughly evaluate the
model's dense alignment capacity. Through experiments, we achieve
state-of-the-art performance on two widely-adopted 3D visual grounding
datasets, ScanRefer and SR3D/NR3D, and obtain absolute leadership on our
newly-proposed task. The source code is available at
this https URL

本研究提出 EDA 方法，通过文本解耦和密集对齐技术实现了 3D 点云中丰富语义描述的物体检索，并在已有数据集上实现了最优结果，并在新提出的任务上取得了绝对领先。