Video deblurring relies on leveraging information from other frames in the
video sequence to restore the blurred regions in the current frame. Mainstream
approaches employ bidirectional feature propagation, spatio-temporal
transformers, or a combination of both to extract information from the video
sequence. However, limitations in memory and computational resources
constraints the temporal window length of the spatio-temporal transformer,
preventing the extraction of longer temporal contextual information from the
video sequence. Additionally, bidirectional feature propagation is highly
sensitive to inaccurate optical flow in blurry frames, leading to error
accumulation during the propagation process. To address these issues, we
propose \textbf{BSSTNet}, \textbf{B}lur-aware \textbf{S}patio-temporal
\textbf{S}parse \textbf{T}ransformer Network. It introduces the blur map, which
converts the originally dense attention into a sparse form, enabling a more
extensive utilization of information throughout the entire video sequence.
Specifically, BSSTNet (1) uses a longer temporal window in the transformer,
leveraging information from more distant frames to restore the blurry pixels in
the current frame. (2) introduces bidirectional feature propagation guided by
blur maps, which reduces error accumulation caused by the blur frame. The
experimental results demonstrate the proposed BSSTNet outperforms the
state-of-the-art methods on the GoPro and DVD datasets.

提出了一种名为 BSSTNet 的模型，它通过引入模糊图，将初始的密集注意力转化为稀疏形式，从而更全面地利用整个视频序列中的信息。BSSTNet 在变换器中使用了更长的时间窗口，利用较远帧中的信息来修复当前帧中的模糊像素，同时引入了由模糊图引导的双向特征传播，降低了由模糊帧引起的错误累积。实验结果表明，提出的 BSSTNet 在 GoPro 和 DVD 数据集上优于现有的方法。

模糊感知时空稀疏变压器用于视频去模糊

Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring

Recent works have revealed the superiority of feature-level fusion for
cross-modal 3D object detection, where fine-grained feature propagation from 2D
image pixels to 3D LiDAR points has been widely adopted for performance
improvement. Still, the potential of heterogeneous feature propagation between
2D and 3D domains has not been fully explored. In this paper, in contrast to
existing pixel-to-point feature propagation, we investigate an opposite
point-to-pixel direction, allowing point-wise features to flow inversely into
the 2D image branch. Thus, when jointly optimizing the 2D and 3D streams, the
gradients back-propagated from the 2D image branch can boost the representation
ability of the 3D backbone network working on LiDAR point clouds. Then,
combining pixel-to-point and point-to-pixel information flow mechanisms, we
construct an bidirectional feature propagation framework, dubbed BiProDet. In
addition to the architectural design, we also propose normalized local
coordinates map estimation, a new 2D auxiliary task for the training of the 2D
image branch, which facilitates learning local spatial-aware features from the
image modality and implicitly enhances the overall 3D detection performance.
Extensive experiments and ablation studies validate the effectiveness of our
method. Notably, we rank $\mathbf{1^{\mathrm{st}}}$ on the highly competitive
KITTI benchmark on the cyclist class by the time of submission. The source code
is available at this https URL

本文描述了一种名为 BiProDet 的双向特征传播框架，使用点到像素的信息流和像素到点的信息流等机制优化了 3D 目标检测。同时，提出一种新的 2D 辅助任务，用于训练 2D 分支并间接提高 3D 检测的性能。实验证明，该方法在 KITTI 数据集的自行车类上的排名为第一位。