Traffic object detection under variable illumination is challenging due to
the information loss caused by the limited dynamic range of conventional
frame-based cameras. To address this issue, we introduce bio-inspired event
cameras and propose a novel Structure-aware Fusion Network (SFNet) that
extracts sharp and complete object structures from the event stream to
compensate for the lost information in images through cross-modality fusion,
enabling the network to obtain illumination-robust representations for traffic
object detection. Specifically, to mitigate the sparsity or blurriness issues
arising from diverse motion states of traffic objects in fixed-interval event
sampling methods, we propose the Reliable Structure Generation Network (RSGNet)
to generate Speed Invariant Frames (SIF), ensuring the integrity and sharpness
of object structures. Next, we design a novel Adaptive Feature Complement
Module (AFCM) which guides the adaptive fusion of two modality features to
compensate for the information loss in the images by perceiving the global
lightness distribution of the images, thereby generating illumination-robust
representations. Finally, considering the lack of large-scale and high-quality
annotations in the existing event-based object detection datasets, we build a
DSEC-Det dataset, which consists of 53 sequences with 63,931 images and more
than 208,000 labels for 8 classes. Extensive experimental results demonstrate
that our proposed SFNet can overcome the perceptual boundaries of conventional
cameras and outperform the frame-based method by 8.0% in mAP50 and 5.9% in
mAP50:95. Our code and dataset will be available at
this https URL

针对传统基于帧的摄像机由于动态范围受限而导致的信息丢失，我们引入了生物启发式事件摄像机，并提出了一种新颖的结构感知融合网络（SFNet），通过跨模态融合从事件流中提取清晰和完整的物体结构，以补偿图像中的信息丢失，从而使网络能够获得对光照鲁棒的交通物体检测表示。

基于 RGB-Event 融合的可变光照下的交通物体检测增强

Enhancing Traffic Object Detection in Variable Illumination with  RGB-Event Fusion

A panoptic driving perception system is an essential part of autonomous
driving. A high-precision and real-time perception system can assist the
vehicle in making the reasonable decision while driving. We present a panoptic
driving perception network (YOLOP) to perform traffic object detection,
drivable area segmentation and lane detection simultaneously. It is composed of
one encoder for feature extraction and three decoders to handle the specific
tasks. Our model performs extremely well on the challenging BDD100K dataset,
achieving state-of-the-art on all three tasks in terms of accuracy and speed.
Besides, we verify the effectiveness of our multi-task learning model for joint
training via ablative studies. To our best knowledge, this is the first work
that can process these three visual perception tasks simultaneously in
real-time on an embedded device Jetson TX2(23 FPS) and maintain excellent
accuracy. To facilitate further research, the source codes and pre-trained
models are released at this https URL

我们提出了一种全景驾驶感知网络（YOLOP），它可以同时执行交通物体检测、可驾驶区域分割和车道检测三个任务，并在嵌入式设备 Jetson TX2 上实现了实时性的高精度表现。