Recent research has highlighted improvements in high-quality imaging guided
by event cameras, with most of these efforts concentrating on the RGB domain.
However, these advancements frequently neglect the unique challenges introduced
by the inherent flaws in the sensor design of event cameras in the RAW domain.
Specifically, this sensor design results in the partial loss of pixel values,
posing new challenges for RAW domain processes like demosaicing. The challenge
intensifies as most research in the RAW domain is based on the premise that
each pixel contains a value, making the straightforward adaptation of these
methods to event camera demosaicing problematic. To end this, we present a
Swin-Transformer-based backbone and a pixel-focus loss function for demosaicing
with missing pixel values in RAW domain processing. Our core motivation is to
refine a general and widely applicable foundational model from the RGB domain
for RAW domain processing, thereby broadening the model's applicability within
the entire imaging process. Our method harnesses multi-scale processing and
space-to-depth techniques to ensure efficiency and reduce computing complexity.
We also proposed the Pixel-focus Loss function for network fine-tuning to
improve network convergence based on our discovery of a long-tailed
distribution in training loss. Our method has undergone validation on the MIPI
Demosaic Challenge dataset, with subsequent analytical experimentation
confirming its efficacy. All code and trained models are released here:
this https URL

利用 Swin-Transformer 和像素聚焦损失函数来处理 RAW 领域中带有缺失像素数值的图像，解决了以往在 RGB 领域中的广泛模型适用性问题，并通过 MIPI Demosaic Challenge 数据集验证了方法的有效性。

基于 Swin Transformer 和像素焦点损失的事件相机去马赛克

Event Camera Demosaicing via Swin Transformer and Pixel-focus Loss

Capturing screen contents by smartphone cameras has become a common way for
information sharing. However, these images and videos are often degraded by
moir\'e patterns, which are caused by frequency aliasing between the camera
filter array and digital display grids. We observe that the moir\'e patterns in
raw domain is simpler than those in sRGB domain, and the moir\'e patterns in
raw color channels have different properties. Therefore, we propose an image
and video demoir\'eing network tailored for raw inputs. We introduce a
color-separated feature branch, and it is fused with the traditional
feature-mixed branch via channel and spatial modulations. Specifically, the
channel modulation utilizes modulated color-separated features to enhance the
color-mixed features. The spatial modulation utilizes the feature with large
receptive field to modulate the feature with small receptive field. In
addition, we build the first well-aligned raw video demoir\'eing
(RawVDemoir\'e) dataset and propose an efficient temporal alignment method by
inserting alternating patterns. Experiments demonstrate that our method
achieves state-of-the-art performance for both image and video demori\'eing. We
have released the code and dataset in this https URL

通过利用原始颜色分离特征分支和空间调制，以及构建了第一个对齐良好的原始视频去莫尔（RawVDemoiré）数据集，本研究提出了一种专门针对原始输入的图像和视频去莫尔网络，并且在去莫尔图像和视频方面取得了最先进的性能。