We present a 3D shape completion method that recovers the complete geometry
of multiple objects in complex scenes from a single RGB-D image. Despite
notable advancements in single object 3D shape completion, high-quality
reconstructions in highly cluttered real-world multi-object scenes remains a
challenge. To address this issue, we propose OctMAE, an architecture that
leverages an Octree U-Net and a latent 3D MAE to achieve high-quality and near
real-time multi-object shape completion through both local and global geometric
reasoning. Because a na\"ive 3D MAE can be computationally intractable and
memory intensive even in the latent space, we introduce a novel occlusion
masking strategy and adopt 3D rotary embeddings, which significantly improves
the runtime and shape completion quality. To generalize to a wide range of
objects in diverse scenes, we create a large-scale photorealistic dataset,
featuring a diverse set of 12K 3D object models from the Objaverse dataset
which are rendered in multi-object scenes with physics-based positioning. Our
method outperforms the current state-of-the-art on both synthetic and
real-world datasets and demonstrates a strong zero-shot capability.

我们提出了一种 3D 形状补全方法，通过单个 RGB-D 图像从复杂场景中恢复多个对象的完整几何形状。我们的方法 OctMAE 利用八叉树 U-Net 和潜在的 3D MAE，在本地和全局几何推理中实现高质量和接近实时的多对象形状补全，并引入了遮挡掩码策略和 3D 旋转嵌入，以提高性能和形状补全质量。通过创建一个大规模逼真的数据集，我们的方法在合成和现实世界数据集上都优于当前最先进的方法，并展示了强大的零样品学习能力。

零样本多物体形状完成

Zero-Shot Multi-Object Shape Completion

Adversarial attacks aim to perturb images such that a predictor outputs
incorrect results. Due to the limited research in structured attacks, imposing
consistency checks on natural multi-object scenes is a promising yet practical
defense against conventional adversarial attacks. More desired attacks, to this
end, should be able to fool defenses with such consistency checks. Therefore,
we present the first approach GLOW that copes with various attack requests by
generating global layout-aware adversarial attacks, in which both categorical
and geometric layout constraints are explicitly established. Specifically, we
focus on object detection task and given a victim image, GLOW first localizes
victim objects according to target labels. And then it generates multiple
attack plans, together with their context-consistency scores. Our proposed
GLOW, on the one hand, is capable of handling various types of requests,
including single or multiple victim objects, with or without specified victim
objects. On the other hand, it produces a consistency score for each attack
plan, reflecting the overall contextual consistency that both semantic category
and global scene layout are considered. In experiment, we design multiple types
of attack requests and validate our ideas on MS COCO and Pascal. Extensive
experimental results demonstrate that we can achieve about 30$\%$ average
relative improvement compared to state-of-the-art methods in conventional
single object attack request; Moreover, our method outperforms SOTAs
significantly on more generic attack requests by about 20$\%$ in average;
Finally, our method produces superior performance under challenging zero-query
black-box setting, or 20$\%$ better than SOTAs. Our code, model and attack
requests would be made available.

本文提出新的 GLOW 方法对多目标场景下单 / 多个受害者对象产生全局布局感知的分类和几何约束的对抗攻击，并与 SOTAs 和具有挑战性的无查询黑盒攻击的方法进行比较，结果表明在传统攻击请求的单个对象方面比 SOTAs 提高了约 30％，在更广泛的攻击请求方面平均提高约 20％。

全局布局感知物体检测攻击

GLOW: Global Layout Aware Attacks on Object Detection

Learning object-centric representations of multi-object scenes is a promising
approach towards machine intelligence, facilitating high-level reasoning and
control from visual sensory data. However, current approaches for unsupervised
object-centric scene representation are incapable of aggregating information
from multiple observations of a scene. As a result, these "single-view" methods
form their representations of a 3D scene based only on a single 2D observation
(view). Naturally, this leads to several inaccuracies, with these methods
falling victim to single-view spatial ambiguities. To address this, we propose
The Multi-View and Multi-Object Network (MulMON) -- a method for learning
accurate, object-centric representations of multi-object scenes by leveraging
multiple views. In order to sidestep the main technical difficulty of the
multi-object-multi-view scenario -- maintaining object correspondences across
views -- MulMON iteratively updates the latent object representations for a
scene over multiple views. To ensure that these iterative updates do indeed
aggregate spatial information to form a complete 3D scene understanding, MulMON
is asked to predict the appearance of the scene from novel viewpoints during
training. Through experiments, we show that MulMON better-resolves spatial
ambiguities than single-view methods -- learning more accurate and disentangled
object representations -- and also achieves new functionality in predicting
object segmentations for novel viewpoints.

MulMON 基于多视角学习，旨在实现准确和脱耦的多对象场景的目标结构化，克服了单视角方法在物体场景表示中遇到的问题。

从多个视角学习多对象场景的基于对象的表征

Learning Object-Centric Representations of Multi-Object Scenes from  Multiple Views

We present a framework for efficient perceptual inference that explicitly
reasons about the segmentation of its inputs and features. Rather than being
trained for any specific segmentation, our framework learns the grouping
process in an unsupervised manner or alongside any supervised task. By
enriching the representations of a neural network, we enable it to group the
representations of different objects in an iterative manner. By allowing the
system to amortize the iterative inference of the groupings, we achieve very
fast convergence. In contrast to many other recently proposed methods for
addressing multi-object scenes, our system does not assume the inputs to be
images and can therefore directly handle other modalities. For multi-digit
classification of very cluttered images that require texture segmentation, our
method offers improved classification performance over convolutional networks
despite being fully connected. Furthermore, we observe that our system greatly
improves on the semi-supervised result of a baseline Ladder network on our
dataset, indicating that segmentation can also improve sample efficiency.

本文介绍了一种有效的感知推断框架，该框架能够显式地处理输入和特征的分割，并通过增强神经网络的表示能力并对对象表示进行迭代分组，以及对其分组迭代推理进行摊销从而实现了快速收敛，相比其他处理多物体场景的方法，该方法不假设输入为图像，因此可以直接处理其他模态，并在处理具有纹理分割的复杂图像时提供更好的分类性能。