Event camera, a bio-inspired asynchronous triggered camera, offers promising
prospects for fusion with frame-based cameras owing to its low latency and high
dynamic range. However, calibrating stereo vision systems that incorporate both
event and frame-based cameras remains a significant challenge. In this letter,
we present EF-Calib, a spatiotemporal calibration framework for event- and
frame-based cameras using continuous-time trajectories. A novel calibration
pattern applicable to both camera types and the corresponding event recognition
algorithm is proposed. Leveraging the asynchronous nature of events, a
derivable piece-wise B-spline to represent camera pose continuously is
introduced, enabling calibration for intrinsic parameters, extrinsic
parameters, and time offset, with analytical Jacobians provided. Various
experiments are carried out to evaluate the calibration performance of
EF-Calib, including calibration experiments for intrinsic parameters, extrinsic
parameters, and time offset. Experimental results show that EF-Calib achieves
the most accurate intrinsic parameters compared to current SOTA, the close
accuracy of the extrinsic parameters compared to the frame-based results, and
accurate time offset estimation. EF-Calib provides a convenient and accurate
toolbox for calibrating the system that fuses events and frames. The code of
this paper will also be open-sourced at: this https URL

提出了一个名为 EF-Calib 的时空校准框架，用于事件相机和基于帧的相机的校准，通过引入可连续表示相机姿态的 B 样条曲线模型以及分析性的雅可比矩阵，实现了内参、外参和时间偏移的校准，具有较高的准确性和方便性。

EF-Calib: 使用连续时间轨迹对事件和帧分相机进行时空标定

EF-Calib: Spatiotemporal Calibration of Event- and Frame-Based Cameras  Using Continuous-Time Trajectories

Inspired by the complementarity between conventional frame-based and
bio-inspired event-based cameras, we propose a multi-modal based approach to
fuse visual cues from the frame- and event-domain to enhance the single object
tracking performance, especially in degraded conditions (e.g., scenes with high
dynamic range, low light, and fast-motion objects). The proposed approach can
effectively and adaptively combine meaningful information from both domains.
Our approach's effectiveness is enforced by a novel designed cross-domain
attention schemes, which can effectively enhance features based on self- and
cross-domain attention schemes; The adaptiveness is guarded by a specially
designed weighting scheme, which can adaptively balance the contribution of the
two domains. To exploit event-based visual cues in single-object tracking, we
construct a large-scale frame-event-based dataset, which we subsequently employ
to train a novel frame-event fusion based model. Extensive experiments show
that the proposed approach outperforms state-of-the-art frame-based tracking
methods by at least 10.4% and 11.9% in terms of representative success rate and
precision rate, respectively. Besides, the effectiveness of each key component
of our approach is evidenced by our thorough ablation study.

本研究提出了一种多模态方法，可以在单个目标追踪中融合来自帧域和事件域的视觉线索以提高其性能，并通过一种新颖的设计的跨域注意方案有效地和自适应地结合有意义的信息。我们的方法利用自适应平衡方案，其中可以平衡两种领域的贡献。实验证明，所提出的方法的效果比最先进的基于帧的跟踪方法至少提高了 10.4％和 11.9％，并且有效性得到了充分的验证。