On a shutter press, modern handheld cameras capture multiple images in rapid
succession and merge them to generate a single image. However, individual
frames in a burst are misaligned due to inevitable motions and contain multiple
degradations. The challenge is to properly align the successive image shots and
merge their complimentary information to achieve high-quality outputs. Towards
this direction, we propose Burstormer: a novel transformer-based architecture
for burst image restoration and enhancement. In comparison to existing works,
our approach exploits multi-scale local and non-local features to achieve
improved alignment and feature fusion. Our key idea is to enable inter-frame
communication in the burst neighborhoods for information aggregation and
progressive fusion while modeling the burst-wide context. However, the input
burst frames need to be properly aligned before fusing their information.
Therefore, we propose an enhanced deformable alignment module for aligning
burst features with regards to the reference frame. Unlike existing methods,
the proposed alignment module not only aligns burst features but also exchanges
feature information and maintains focused communication with the reference
frame through the proposed reference-based feature enrichment mechanism, which
facilitates handling complex motions. After multi-level alignment and
enrichment, we re-emphasize on inter-frame communication within burst using a
cyclic burst sampling module. Finally, the inter-frame information is
aggregated using the proposed burst feature fusion module followed by
progressive upsampling. Our Burstormer outperforms state-of-the-art methods on
burst super-resolution, burst denoising and burst low-light enhancement. Our
codes and pretrained models are available at https://
github.com/akshaydudhane16/Burstormer

提出了一种名为 Burstormer 的新型基于 transformer 的框架，用于爆发图像的恢复和增强。本文采用多尺度局部和非局部特征来实现改进的对齐和特征融合，通过新的变形对齐模块和循环采样模块对连续帧信息进行整合和聚合，并利用爆发特征融合模块进行逐步上采样。Burstormer 在爆发超分辨率、去噪和低光增强方面表现优异。

Burstormer: 爆发图像修复和增强变压器

Burstormer: Burst Image Restoration and Enhancement Transformer

Modern handheld devices can acquire burst image sequence in a quick
succession. However, the individual acquired frames suffer from multiple
degradations and are misaligned due to camera shake and object motions. The
goal of Burst Image Restoration is to effectively combine complimentary cues
across multiple burst frames to generate high-quality outputs. Towards this
goal, we develop a novel approach by solely focusing on the effective
information exchange between burst frames, such that the degradations get
filtered out while the actual scene details are preserved and enhanced. Our
central idea is to create a set of pseudo-burst features that combine
complementary information from all the input burst frames to seamlessly
exchange information. However, the pseudo-burst cannot be successfully created
unless the individual burst frames are properly aligned to discount inter-frame
movements. Therefore, our approach initially extracts pre-processed features
from each burst frame and matches them using an edge-boosting burst alignment
module. The pseudo-burst features are then created and enriched using
multi-scale contextual information. Our final step is to adaptively aggregate
information from the pseudo-burst features to progressively increase resolution
in multiple stages while merging the pseudo-burst features. In comparison to
existing works that usually follow a late fusion scheme with single-stage
upsampling, our approach performs favorably, delivering state-of-the-art
performance on burst superresolution, burst low-light image enhancement, and
burst denoising tasks. The source code and pre-trained models are available at
https://github.com/akshaydudhane16/BIPNet.

该研究提出了一种新颖的 Burst Image Restoration 方法，使用伪爆发特征相互交换信息，通过快速匹配、多尺度的上下文信息和自适应聚合，还原高质量图像。与现有方法相比，该方法表现出最先进的性能，可用于爆发超分辨率、爆发低光图像增强和爆发去噪任务。