Digital cameras transform sensor RAW readings into RGB images by means of
their Image Signal Processor (ISP). Computational photography tasks such as
image denoising and colour constancy are commonly performed in the RAW domain,
in part due to the inherent hardware design, but also due to the appealing
simplicity of noise statistics that result from the direct sensor readings.
Despite this, the availability of RAW images is limited in comparison with the
abundance and diversity of available RGB data. Recent approaches have attempted
to bridge this gap by estimating the RGB to RAW mapping: handcrafted
model-based methods that are interpretable and controllable usually require
manual parameter fine-tuning, while end-to-end learnable neural networks
require large amounts of training data, at times with complex training
procedures, and generally lack interpretability and parametric control. Towards
addressing these existing limitations, we present a novel hybrid model-based
and data-driven ISP that builds on canonical ISP operations and is both
learnable and interpretable. Our proposed invertible model, capable of
bidirectional mapping between RAW and RGB domains, employs end-to-end learning
of rich parameter representations, i.e. dictionaries, that are free from direct
parametric supervision and additionally enable simple and plausible data
augmentation. We evidence the value of our data generation process by extensive
experiments under both RAW image reconstruction and RAW image denoising tasks,
obtaining state-of-the-art performance in both. Additionally, we show that our
ISP can learn meaningful mappings from few data samples, and that denoising
models trained with our dictionary-based data augmentation are competitive
despite having only few or zero ground-truth labels.

该研究提出了一种混合模型和数据驱动的 ISP 技术，实现了 RAW 和 RGB 域的双向映射和参数学习，通过基于字典的数据增强方法，在 RAW 图像重建和去噪任务中实现了最先进的性能。

基于可学习字典的模型化图像信号处理器

Model-Based Image Signal Processors via Learnable Dictionaries

This presentation addresses the problem of reconstructing a high-resolution
image from multiple lower-resolution snapshots captured from slightly different
viewpoints in space and time. Key challenges for solving this problem include
(i) aligning the input pictures with sub-pixel accuracy, (ii) handling raw
(noisy) images for maximal faithfulness to native camera data, and (iii)
designing/learning an image prior (regularizer) well suited to the task. We
address these three challenges with a hybrid algorithm building on the insight
from Wronski et al. that aliasing is an ally in this setting, with parameters
that can be learned end to end, while retaining the interpretability of
classical approaches to inverse problems. The effectiveness of our approach is
demonstrated on synthetic and real image bursts, setting a new state of the art
on several benchmarks and delivering excellent qualitative results on real raw
bursts captured by smartphones and prosumer cameras.

本文提出了一种混合算法来解决从不同空间和时间视角拍摄的多个低分辨率快照重建高分辨率图像的问题，并证明其在合成和真实图像集上均优于其他方法。