This paper presents the first significant object detection framework,
NeRF-RPN, which directly operates on NeRF. Given a pre-trained NeRF model,
NeRF-RPN aims to detect all bounding boxes of objects in a scene. By exploiting
a novel voxel representation that incorporates multi-scale 3D neural volumetric
features, we demonstrate it is possible to regress the 3D bounding boxes of
objects in NeRF directly without rendering the NeRF at any viewpoint. NeRF-RPN
is a general framework and can be applied to detect objects without class
labels. We experimented NeRF-RPN with various backbone architectures, RPN head
designs and loss functions. All of them can be trained in an end-to-end manner
to estimate high quality 3D bounding boxes. To facilitate future research in
object detection for NeRF, we built a new benchmark dataset which consists of
both synthetic and real-world data with careful labeling and clean up. Code and
dataset are available at this https URL

NeRF-RPN 是首个在 NeRF 上直接操作的显著物体检测框架。它利用一种新颖的体素表示法，通过回归 NeRF 模型直接得出物体的三维边界框，而无需在任何视点渲染 NeRF。此外，作者还建立了基准数据集，其中包括合成和真实世界数据，可供未来的 NeRF 目标检测研究参考。

NeRF-RPN: 基于 NeRF 的目标检测通用框架

NeRF-RPN: A general framework for object detection in NeRFs

Image-only and pseudo-LiDAR representations are commonly used for monocular
3D object detection. However, methods based on them have shortcomings of either
not well capturing the spatial relationships in neighbored image pixels or
being hard to handle the noisy nature of the monocular pseudo-LiDAR point
cloud. To overcome these issues, in this paper we propose a novel
object-centric voxel representation tailored for monocular 3D object detection.
Specifically, voxels are built on each object proposal, and their sizes are
adaptively determined by the 3D spatial distribution of the points, allowing
the noisy point cloud to be organized effectively within a voxel grid. This
representation is proved to be able to locate the object in 3D space
accurately. Furthermore, prior works would like to estimate the orientation via
deep features extracted from an entire image or a noisy point cloud. By
contrast, we argue that the local RoI information from the object image patch
alone with a proper resizing scheme is a better input as it provides complete
semantic clues meanwhile excludes irrelevant interferences. Besides, we
decompose the confidence mechanism in monocular 3D object detection by
considering the relationship between 3D objects and the associated 2D boxes.
Evaluated on KITTI, our method outperforms state-of-the-art methods by a large
margin. The code will be made publicly available soon.

本文提出了一种适用于单目三维目标检测的新型基于物体的体素表示法，能够有效地组织噪点云在体素网格内，并能够精确定位对象在三维空间中的位置。通过考虑三维物体与相关二维框之间的关系，我们将单目三维目标检测中的置信度机制进行了分解，并在 KITTI 数据集上进行了评估，性能显著优于现有技术。

OCM3D：基于物体的单目三维物体检测

OCM3D: Object-Centric Monocular 3D Object Detection

We present a unified, efficient and effective framework for point-cloud based
3D object detection. Our two-stage approach utilizes both voxel representation
and raw point cloud data to exploit respective advantages. The first stage
network, with voxel representation as input, only consists of light
convolutional operations, producing a small number of high-quality initial
predictions. Coordinate and indexed convolutional feature of each point in
initial prediction are effectively fused with the attention mechanism,
preserving both accurate localization and context information. The second stage
works on interior points with their fused feature for further refining the
prediction. Our method is evaluated on KITTI dataset, in terms of both 3D and
Bird's Eye View (BEV) detection, and achieves state-of-the-arts with a 15FPS
detection rate.

本篇研究提出了一种基于点云的三维物体检测框架，采用两个阶段的方法，利用体素表示和点云原始数据的优势进行特征提取，并在 KITTI 数据集上进行评估，取得了 3D 和 Bird's Eye View（BEV）检测的最新成果，检测速度达到了每秒 15 帧。