Text-to-image generation has achieved astonishing results, yet precise
spatial controllability and prompt fidelity remain highly challenging. This
limitation is typically addressed through cumbersome prompt engineering, scene
layout conditioning, or image editing techniques which often require hand drawn
masks. Nonetheless, pre-existing works struggle to take advantage of the
natural instance-level compositionality of scenes due to the typically flat
nature of rasterized RGB output images. Towards adressing this challenge, we
introduce MuLAn: a novel dataset comprising over 44K MUlti-Layer ANnotations of
RGB images as multilayer, instance-wise RGBA decompositions, and over 100K
instance images. To build MuLAn, we developed a training free pipeline which
decomposes a monocular RGB image into a stack of RGBA layers comprising of
background and isolated instances. We achieve this through the use of
pretrained general-purpose models, and by developing three modules: image
decomposition for instance discovery and extraction, instance completion to
reconstruct occluded areas, and image re-assembly. We use our pipeline to
create MuLAn-COCO and MuLAn-LAION datasets, which contain a variety of image
decompositions in terms of style, composition and complexity. With MuLAn, we
provide the first photorealistic resource providing instance decomposition and
occlusion information for high quality images, opening up new avenues for
text-to-image generative AI research. With this, we aim to encourage the
development of novel generation and editing technology, in particular
layer-wise solutions. MuLAn data resources are available at
this https URL

将一幅单眼 RGB 图像分解成为一个包含背景和独立实例的 RGBA 层叠，并重建遮挡区域，为高质量图像提供实例分解和遮挡信息的第一个照片逼真资源，为文本到图像生成 AI 研究开辟新的可能性。

MULAN：用于可控文本到图像生成的多层注释数据集

MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image  Generation

Existing 2D-to-3D pose lifting networks suffer from poor performance in
cross-dataset benchmarks. Although the use of 2D keypoints joined by
"stick-figure" limbs has shown promise as an intermediate step, stick-figures
do not account for occlusion information that is often inherent in an image. In
this paper, we propose a novel representation using opaque 3D limbs that
preserves occlusion information while implicitly encoding joint locations.
Crucially, when training on data with accurate three-dimensional keypoints and
without part-maps, this representation allows training on abstract synthetic
images, with occlusion, from as many synthetic viewpoints as desired. The
result is a pose defined by limb angles rather than joint positions
$\unicode{x2013}$ because poses are, in the real world, independent of cameras
$\unicode{x2013}$ allowing us to predict poses that are completely independent
of camera viewpoint. The result provides not only an improvement in
same-dataset benchmarks, but a "quantum leap" in cross-dataset benchmarks.

本文提出了一种使用不透明的三维肢体表示的新方法，该方法在保留遮挡信息的同时，隐含地编码关节位置，从而实现对抽象合成图像的训练，并允许预测完全独立于相机视角的姿势，从而在相同和不同数据集基准测试中取得了显著提升。

学习抽象图像中的重要性：关于遮挡在人体姿势最简编码中的作用

Learning from Abstract Images: on the Importance of Occlusion in a  Minimalist Encoding of Human Poses

Learning-based multi-view stereo (MVS) methods have demonstrated promising
results. However, very few existing networks explicitly take the pixel-wise
visibility into consideration, resulting in erroneous cost aggregation from
occluded pixels. In this paper, we explicitly infer and integrate the
pixel-wise occlusion information in the MVS network via the matching
uncertainty estimation. The pair-wise uncertainty map is jointly inferred with
the pair-wise depth map, which is further used as weighting guidance during the
multi-view cost volume fusion. As such, the adverse influence of occluded
pixels is suppressed in the cost fusion. The proposed framework Vis-MVSNet
significantly improves depth accuracies in the scenes with severe occlusion.
Extensive experiments are performed on DTU, BlendedMVS, and Tanks and Temples
datasets to justify the effectiveness of the proposed framework.

本文基于学习的多视角立体 (MVS) 方法，采用匹配不确定性估计，将像素级遮挡信息显式地推理和集成到 MVS 网络中，从而抑制像素遮挡对成本聚合的不利影响，提高了 Vis-MVSNet 算法在严重遮挡场景中的深度精度。在 DTU、BlendedMVS 和 Tanks and Temples 数据集上进行了广泛实验证明了所提出算法的有效性。