Given an object of interest, visual navigation aims to reach the object's
location based on a sequence of partial observations. To this end, an agent
needs to 1) learn a piece of certain knowledge about the relations of object
categories in the world during training and 2) look for the target object based
on the pre-learned object category relations and its moving trajectory in the
current unseen environment. In this paper, we propose a Category Relation Graph
(CRG) to learn the knowledge of object category layout relations and a
Temporal-Spatial-Region (TSR) attention architecture to perceive the long-term
spatial-temporal dependencies of objects helping the navigation. We learn prior
knowledge of object layout, establishing a category relationship graph to
deduce the positions of specific objects. Subsequently, we introduced TSR to
capture the relationships of objects in temporal, spatial, and regions within
the observation trajectories. Specifically, we propose a Temporal attention
module (T) to model the temporal structure of the observation sequence, which
implicitly encodes the historical moving or trajectory information. Then, a
Spatial attention module (S) is used to uncover the spatial context of the
current observation objects based on the category relation graph and past
observations. Last, a Region attention module (R) shifts the attention to the
target-relevant region. Based on the visual representation extracted by our
method, the agent can better perceive the environment and easily learn superior
navigation policy. Experiments on AI2-THOR demonstrate our CRG-TSR method
significantly outperforms existing methods regarding both effectiveness and
efficiency. The code has been included in the supplementary material and will
be publicly available.

提出了一种基于类别关系和时空区域注意力架构的可视导航方法，通过学习对象类别和空间布局的关系以及对象的时空依赖关系，实现对目标对象位置的可靠感知，从而更好地学习并执行导航策略。

使用空间和时间注意力构建分类图表示的视觉导航

Building Category Graphs Representation with Spatial and Temporal  Attention for Visual Navigation

Despite the recent advancement of Generative Adversarial Networks (GANs) in
learning 3D-aware image synthesis from 2D data, existing methods fail to model
indoor scenes due to the large diversity of room layouts and the objects
inside. We argue that indoor scenes do not have a shared intrinsic structure,
and hence only using 2D images cannot adequately guide the model with the 3D
geometry. In this work, we fill in this gap by introducing depth as a 3D prior.
Compared with other 3D data formats, depth better fits the convolution-based
generation mechanism and is more easily accessible in practice. Specifically,
we propose a dual-path generator, where one path is responsible for depth
generation, whose intermediate features are injected into the other path as the
condition for appearance rendering. Such a design eases the 3D-aware synthesis
with explicit geometry information. Meanwhile, we introduce a switchable
discriminator both to differentiate real v.s. fake domains and to predict the
depth from a given input. In this way, the discriminator can take the spatial
arrangement into account and advise the generator to learn an appropriate depth
condition. Extensive experimental results suggest that our approach is capable
of synthesizing indoor scenes with impressively good quality and 3D
consistency, significantly outperforming state-of-the-art alternatives.

该研究提出了一种基于深度的双通道生成器和可切换鉴别器，用于从 2D 数据中合成 3D 感知室内场景的图像，并通过实验表明，该方法可以显著优于现有的最先进的方法。

具深度先验知识的室内场景 3D 合成

3D-Aware Indoor Scene Synthesis with Depth Priors

Furnishing and rendering indoor scenes has been a long-standing task for
interior design, where artists create a conceptual design for the space, build
a 3D model of the space, decorate, and then perform rendering. Although the
task is important, it is tedious and requires tremendous effort. In this paper,
we introduce a new problem of domain-specific indoor scene image synthesis,
namely neural scene decoration. Given a photograph of an empty indoor space and
a list of decorations with layout determined by user, we aim to synthesize a
new image of the same space with desired furnishing and decorations. Neural
scene decoration can be applied to create conceptual interior designs in a
simple yet effective manner. Our attempt to this research problem is a novel
scene generation architecture that transforms an empty scene and an object
layout into a realistic furnished scene photograph. We demonstrate the
performance of our proposed method by comparing it with conditional image
synthesis baselines built upon prevailing image translation approaches both
qualitatively and quantitatively. We conduct extensive experiments to further
validate the plausibility and aesthetics of our generated scenes. Our
implementation is available at
https://github.com/hkust-vgd/neural_scene_decoration.

本研究介绍了一种新的领域特定的室内场景图像合成问题，即神经场景装饰，旨在通过将一个空室内场景和用户确定的布局转换成一个逼真的装饰室内场景照片来创建概念室内设计，并通过与条件图像合成基线的质量和量化比较来证明我们所提出的方法的性能，我们进行了大量实验来进一步验证我们生成场景的合理性和美学。