Automated vehicles require an accurate perception of their surroundings for
safe and efficient driving. Lidar-based object detection is a widely used
method for environment perception, but its performance is significantly
affected by adverse weather conditions such as rain and fog. In this work, we
investigate various strategies for enhancing the robustness of lidar-based
object detection by processing sequential data samples generated by lidar
sensors. Our approaches leverage temporal information to improve a lidar object
detection model, without the need for additional filtering or pre-processing
steps. We compare $10$ different neural network architectures that process
point cloud sequences including a novel augmentation strategy introducing a
temporal offset between frames of a sequence during training and evaluate the
effectiveness of all strategies on lidar point clouds under adverse weather
conditions through experiments. Our research provides a comprehensive study of
effective methods for mitigating the effects of adverse weather on the
reliability of lidar-based object detection using sequential data that are
evaluated using public datasets such as nuScenes, Dense, and the Canadian
Adverse Driving Conditions Dataset. Our findings demonstrate that our novel
method, involving temporal offset augmentation through randomized frame
skipping in sequences, enhances object detection accuracy compared to both the
baseline model (Pillar-based Object Detection) and no augmentation.

本研究通过处理由激光雷达传感器生成的连续数据样本，研究了增强激光雷达目标检测鲁棒性的各种策略，利用时间信息改进模型，对不利天气下的激光雷达点云进行了实验评估，发现通过引入序列帧之间的时间偏移进行数据增广的新方法，比基准模型（基于柱的目标检测）和无增广的方法提高了目标检测准确性。

利用时间中的偏移序列增强恶劣天气下的激光雷达目标检测

Enhancing Lidar-based Object Detection in Adverse Weather using Offset  Sequences in Time

Traditional LiDAR-based object detection research primarily focuses on
closed-set scenarios, which falls short in complex real-world applications.
Directly transferring existing 2D open-vocabulary models with some known LiDAR
classes for open-vocabulary ability, however, tends to suffer from over-fitting
problems: The obtained model will detect the known objects, even presented with
a novel category. In this paper, we propose OpenSight, a more advanced 2D-3D
modeling framework for LiDAR-based open-vocabulary detection. OpenSight
utilizes 2D-3D geometric priors for the initial discernment and localization of
generic objects, followed by a more specific semantic interpretation of the
detected objects. The process begins by generating 2D boxes for generic objects
from the accompanying camera images of LiDAR. These 2D boxes, together with
LiDAR points, are then lifted back into the LiDAR space to estimate
corresponding 3D boxes. For better generic object perception, our framework
integrates both temporal and spatial-aware constraints. Temporal awareness
correlates the predicted 3D boxes across consecutive timestamps, recalibrating
the missed or inaccurate boxes. The spatial awareness randomly places some
``precisely'' estimated 3D boxes at varying distances, increasing the
visibility of generic objects. To interpret the specific semantics of detected
objects, we develop a cross-modal alignment and fusion module to first align 3D
features with 2D image embeddings and then fuse the aligned 3D-2D features for
semantic decoding. Our experiments indicate that our method establishes
state-of-the-art open-vocabulary performance on widely used 3D detection
benchmarks and effectively identifies objects for new categories of interest.

提出了基于 LiDAR 的开放词汇检测的先进 2D-3D 建模框架 OpenSight，通过生成 2D 边框并将其与 LiDAR 点云转换为对应的 3D 边框，增强了通用对象感知能力，并通过交叉模态对齐和融合进行特定语义解码，从而在广泛应用的 3D 检测基准上建立了最先进的开放词汇性能，并有效识别感兴趣的新类别对象。

OpenSight：一种基于 LiDAR 的目标检测简单开放型词汇框架

OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object  Detection

Efficient point cloud representation is a fundamental element of Lidar-based
3D object detection. Recent grid-based detectors usually divide point clouds
into voxels or pillars and construct single-stream networks in Bird's Eye View.
However, these point cloud encoding paradigms underestimate the point
representation in the vertical direction, which cause the loss of semantic or
fine-grained information, especially for vertical sensitive objects like
pedestrian and cyclists. In this paper, we propose an explicit vertical
multi-scale representation learning framework, VPFusion, to combine the
complementary information from both voxel and pillar streams. Specifically,
VPFusion first builds upon a sparse voxel-pillar-based backbone. The backbone
divides point clouds into voxels and pillars, then encodes features with 3D and
2D sparse convolution simultaneously. Next, we introduce the Sparse Fusion
Layer (SFL), which establishes a bidirectional pathway for sparse voxel and
pillar features to enable the interaction between them. Additionally, we
present the Dense Fusion Neck (DFN) to effectively combine the dense feature
maps from voxel and pillar branches with multi-scale. Extensive experiments on
the large-scale Waymo Open Dataset and nuScenes Dataset demonstrate that
VPFusion surpasses the single-stream baselines by a large margin and achieves
state-of-the-art performance with real-time inference speed.

本文提出了一种名为 VPFusion 的垂直多尺度表示学习框架，它将来自体素和柱状流的互补信息相结合，有效提高了点云表示的精度，实验结果表明 VPFusion 超越了基线，实现了实时推理速度。