Modern end-to-end image signal processors (ISPs) can learn complex mappings
from RAW/XYZ data to sRGB (or inverse), opening new possibilities in image
processing. However, as the diversity of camera models continues to expand,
developing and maintaining individual ISPs is not sustainable in the long term,
which inherently lacks versatility, hindering the adaptability to multiple
camera models. In this paper, we propose a novel pipeline, Uni-ISP, which
unifies the learning of ISPs from multiple cameras, offering an accurate and
versatile processor to multiple camera models. The core of Uni-ISP is
leveraging device-aware embeddings through learning inverse/forward ISPs and
its special training scheme. By doing so, Uni-ISP not only improves the
performance of inverse/forward ISPs but also unlocks a variety of new
applications inaccessible to existing learned ISPs. Moreover, since there is no
dataset synchronously captured by multiple cameras for training, we construct a
real-world 4K dataset, FiveCam, comprising more than 2,400 pairs of sRGB-RAW
images synchronously captured by five smartphones. We conducted extensive
experiments demonstrating Uni-ISP's accuracy in inverse/forward ISPs (with
improvements of +1.5dB/2.4dB PSNR), its versatility in enabling new
applications, and its adaptability to new camera models.

通过统一多种相机模型的学习，提出了一种新颖的管线 Uni-ISP，利用设备感知的嵌入技术和特殊的训练方案，提高了逆向 / 正向 ISP 的性能，并解锁了多种新应用。同时，通过构建一个真实的 4K 数据集，证明了 Uni-ISP 在逆向 / 正向 ISP 方面的精确性以及其对新相机模型的适应性。

Uni-ISP：融合多个摄像头学习 ISP

Uni-ISP: Unifying the Learning of ISPs from Multiple Cameras

Reconstructing accurate 3D scenes from images is a long-standing vision task.
Due to the ill-posedness of the single-image reconstruction problem, most
well-established methods are built upon multi-view geometry. State-of-the-art
(SOTA) monocular metric depth estimation methods can only handle a single
camera model and are unable to perform mixed-data training due to the metric
ambiguity. Meanwhile, SOTA monocular methods trained on large mixed datasets
achieve zero-shot generalization by learning affine-invariant depths, which
cannot recover real-world metrics. In this work, we show that the key to a
zero-shot single-view metric depth model lies in the combination of large-scale
data training and resolving the metric ambiguity from various camera models. We
propose a canonical camera space transformation module, which explicitly
addresses the ambiguity problems and can be effortlessly plugged into existing
monocular models. Equipped with our module, monocular models can be stably
trained with over 8 million images with thousands of camera models, resulting
in zero-shot generalization to in-the-wild images with unseen camera settings.
Experiments demonstrate SOTA performance of our method on 7 zero-shot
benchmarks. Notably, our method won the championship in the 2nd Monocular Depth
Estimation Challenge. Our method enables the accurate recovery of metric 3D
structures on randomly collected internet images, paving the way for plausible
single-image metrology. The potential benefits extend to downstream tasks,
which can be significantly improved by simply plugging in our model. For
example, our model relieves the scale drift issues of monocular-SLAM (Fig. 1),
leading to high-quality metric scale dense mapping. The code is available at
this https URL

基于大规模数据训练和解决来自不同摄像机模型的度量歧义，实现了在野外图像中无需训练即可广义泛化的单视图度量深度模型，并在 7 个零样本基准测试上展现了领先表现。