Despite significant progress in generative image synthesis and full-body
generation in particular, state-of-the-art methods are either
context-independent, overly reliant to text prompts, or bound to the curated
training datasets, such as fashion images with monotonous backgrounds. Here,
our goal is to generate people in clothing that is semantically appropriate for
a given scene. To this end, we present ESP, a novel method for context-aware
full-body generation, that enables photo-realistic inpainting of people into
existing "in-the-wild" photographs. ESP is conditioned on a 2D pose and
contextual cues that are extracted from the environment photograph and
integrated into the generation process. Our models are trained on a dataset
containing a set of in-the-wild photographs of people covering a wide range of
different environments. The method is analyzed quantitatively and
qualitatively, and we show that ESP outperforms state-of-the-art on the task of
contextual full-body generation.

通过在给定场景中融入 2D 姿势和上下文提示，提出了一种用于上下文感知全身生成的新方法 ESP，该方法在现有的 “野外” 照片中实现了真实感人物修复，定量和定性分析表明 ESP 在上下文全身生成的任务上优于现有的方法。

特定环境下的人群

Environment-Specific People

This paper deals with the challenging task of synthesizing novel views for
in-the-wild photographs. Existing methods have shown promising results
leveraging monocular depth estimation and color inpainting with layered depth
representations. However, these methods still have limited capability to handle
scenes with complex 3D geometry. We propose a new method based on the
multiplane image (MPI) representation. To accommodate diverse scene layouts in
the wild and tackle the difficulty in producing high-dimensional MPI contents,
we design a network structure that consists of two novel modules, one for plane
depth adjustment and another for depth-aware color prediction. The former
adjusts the initial plane positions using the RGBD context feature and an
attention mechanism. Given adjusted depth values, the latter predicts the color
and density for each plane separately with proper inter-plane interactions
achieved via a feature masking strategy. To train our method, we construct
large-scale stereo training data using only unconstrained single-view image
collections by a simple yet effective warp-back strategy. The experiments on
both synthetic and real datasets demonstrate that our trained model works
remarkably well and achieves state-of-the-art results.

本文提出一种基于多平面图像（MPI）表示的新方法来合成野外照片的新视角，结合深度估计和颜色填充技术，通过多平面图像表示解决具有复杂 3D 几何的场景，使用大规模立体训练数据并经过实验验证达到最新成果。