Recent works on open-vocabulary 3D instance segmentation show strong promise,
but at the cost of slow inference speed and high computation requirements. This
high computation cost is typically due to their heavy reliance on 3D clip
features, which require computationally expensive 2D foundation models like
Segment Anything (SAM) and CLIP for multi-view aggregation into 3D. As a
consequence, this hampers their applicability in many real-world applications
that require both fast and accurate predictions. To this end, we propose a fast
yet accurate open-vocabulary 3D instance segmentation approach, named Open-YOLO
3D, that effectively leverages only 2D object detection from multi-view RGB
images for open-vocabulary 3D instance segmentation. We address this task by
generating class-agnostic 3D masks for objects in the scene and associating
them with text prompts. We observe that the projection of class-agnostic 3D
point cloud instances already holds instance information; thus, using SAM might
only result in redundancy that unnecessarily increases the inference time. We
empirically find that a better performance of matching text prompts to 3D masks
can be achieved in a faster fashion with a 2D object detector. We validate our
Open-YOLO 3D on two benchmarks, ScanNet200 and Replica, under two scenarios:
(i) with ground truth masks, where labels are required for given object
proposals, and (ii) with class-agnostic 3D proposals generated from a 3D
proposal network. Our Open-YOLO 3D achieves state-of-the-art performance on
both datasets while obtaining up to $\sim$16$\times$ speedup compared to the
best existing method in literature. On ScanNet200 val. set, our Open-YOLO 3D
achieves mean average precision (mAP) of 24.7\% while operating at 22 seconds
per scene. Code and model are available at github.com/aminebdj/OpenYOLO3D.

我们提出了一种名为 Open-YOLO 3D 的快速而准确的开放词汇 3D 实例分割方法，在多视角 RGB 图像中有效利用仅 2D 对象检测进行开放词汇 3D 实例分割，并通过生成类无关的 3D 掩码并与文本提示相关联来处理此任务。我们的 Open-YOLO 3D 在两个基准测试集 ScanNet200 和 Replica 上实现了最先进的性能，在与文献中最佳方法相比速度提高了多达 16 倍。

Open-YOLO 3D：面向快速准确的开放词汇的 3D 实例分割

Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance  Segmentation

Previous research in $2D$ object detection focuses on various tasks,
including detecting objects in generic and camouflaged images. These works are
regarded as passive works for object detection as they take the input image as
is. However, convergence to global minima is not guaranteed to be optimal in
neural networks; therefore, we argue that the trained weights in the object
detector are not optimal. To rectify this problem, we propose a wrapper based
on proactive schemes, PrObeD, which enhances the performance of these object
detectors by learning a signal. PrObeD consists of an encoder-decoder
architecture, where the encoder network generates an image-dependent signal
termed templates to encrypt the input images, and the decoder recovers this
template from the encrypted images. We propose that learning the optimum
template results in an object detector with an improved detection performance.
The template acts as a mask to the input images to highlight semantics useful
for the object detector. Finetuning the object detector with these encrypted
images enhances the detection performance for both generic and camouflaged. Our
experiments on MS-COCO, CAMO, COD$10$K, and NC$4$K datasets show improvement
over different detectors after applying PrObeD. Our models/codes are available
at this https URL

提出了一种基于主动方案的对象检测包装器 PrObeD，通过学习信号来增强这些对象检测器的性能，包括 2D 对象检测、模板生成和加密图像的解密，实验证明 PrObeD 在多个数据集上对不同检测器的性能有所改进。

PrObeD：主动目标检测封装器

PrObeD: Proactive Object Detection Wrapper

Besides standard cameras, autonomous vehicles typically include multiple
additional sensors, such as lidars and radars, which help acquire richer
information for perceiving the content of the driving scene. While several
recent works focus on fusing certain pairs of sensors - such as camera and
lidar or camera and radar - by using architectural components specific to the
examined setting, a generic and modular sensor fusion architecture is missing
from the literature. In this work, we focus on 2D object detection, a
fundamental high-level task which is defined on the 2D image domain, and
propose HRFuser, a multi-resolution sensor fusion architecture that scales
straightforwardly to an arbitrary number of input modalities. The design of
HRFuser is based on state-of-the-art high-resolution networks for image-only
dense prediction and incorporates a novel multi-window cross-attention block as
the means to perform fusion of multiple modalities at multiple resolutions.
Even though cameras alone provide very informative features for 2D detection,
we demonstrate via extensive experiments on the nuScenes and Seeing Through Fog
datasets that our model effectively leverages complementary features from
additional modalities, substantially improving upon camera-only performance and
consistently outperforming state-of-the-art fusion methods for 2D detection
both in normal and adverse conditions. The source code will be made publicly
available.

本文提出了一种名为 HRFuser 的多分辨率传感器融合架构，用于自动驾驶领域中的 2D 物体检测，可以有效地融合不同传感器的信息，提高 2D 检测的性能。