We propose an end-to-end variational generative model for scene layout
synthesis conditioned on scene graphs. Unlike unconditional scene layout
generation, we use scene graphs as an abstract but general representation to
guide the synthesis of diverse scene layouts that satisfy relationships
included in the scene graph. This gives rise to more flexible control over the
synthesis process, allowing various forms of inputs such as scene layouts
extracted from sentences or inferred from a single color image. Using our
conditional layout synthesizer, we can generate various layouts that share the
same structure of the input example. In addition to this conditional generation
design, we also integrate a differentiable rendering module that enables layout
refinement using only 2D projections of the scene. Given a depth and a
semantics map, the differentiable rendering module enables optimizing over the
synthesized layout to fit the given input in an analysis-by-synthesis fashion.
Experiments suggest that our model achieves higher accuracy and diversity in
conditional scene synthesis and allows exemplar-based scene generation from
various input forms.

提出一种基于变分生成模型和场景图条件的端到端场景布局合成方法，并使用可微分渲染模块优化生成的布局。实验表明，该模型可以更精确和多样地进行条件场景合成，并允许从各种输入形式生成样本化场景。

场景布局的端到端优化

End-to-End Optimization of Scene Layout

We present our work on Track 2 in the Dialog System Technology Challenges 7
(DSTC7). The DSTC7-Track 2 aims to evaluate the response generation of fully
data-driven conversation models in knowledge-grounded settings, which provides
the contextual-relevant factual texts. The Sequenceto-Sequence models have been
widely used for end-to-end generative conversation modelling and achieved
impressive results. However, they tend to output dull and repeated responses in
previous studies. Our work aims to promote the diversity for end-to-end
conversation response generation, which follows a two-stage pipeline: 1)
Generate multiple responses. At this stage, two different models are proposed,
i.e., a variational generative (VariGen) model and a retrieval based
(Retrieval) model. 2) Rank and return the most related response by training a
topic coherence discrimination (TCD) model for the ranking process. According
to the official evaluation results, our proposed Retrieval and VariGen systems
ranked first and second respectively on objective diversity metrics, i.e.,
Entropy, among all participant systems. And the VariGen system ranked second on
NIST and METEOR metrics.

本文介绍了在 DSTC7-Track 2 中针对知识基础设定下全数据驱动对话模型的响应生成展开的工作，主要包括利用变分生成模型和检索模型生成多种回复以及通过主题连贯性区别模型确定最相关回复，该工作在官方评估中取得了较好的效果。