Recovering sharp images from dual-pixel (DP) pairs with disparity-dependent
blur is a challenging task.~Existing blur map-based deblurring methods have
demonstrated promising results. In this paper, we propose, to the best of our
knowledge, the first framework to introduce the contrastive language-image
pre-training framework (CLIP) to achieve accurate blur map estimation from DP
pairs unsupervisedly. To this end, we first carefully design text prompts to
enable CLIP to understand blur-related geometric prior knowledge from the DP
pair. Then, we propose a format to input stereo DP pair to the CLIP without any
fine-tuning, where the CLIP is pre-trained on monocular images. Given the
estimated blur map, we introduce a blur-prior attention block, a blur-weighting
loss and a blur-aware loss to recover the all-in-focus image. Our method
achieves state-of-the-art performance in extensive experiments.

从双像素（DP）对中恢复带有视差相关模糊的清晰图像是一项具有挑战性的任务。我们提出了第一个引入对比语言 - 图像自训练框架（CLIP）实现的无监督 DP 对模糊图估计的框架，通过精心设计的文本提示使 CLIP 能够理解 DP 对中与模糊相关的几何先验知识。在获得估计的模糊图之后，我们引入了一个模糊先验注意块、一个模糊加权损失和一个模糊感知损失来恢复全焦点图像。我们的方法在广泛的实验中达到了最先进的性能。

LDP: 基于语言的双像素图像离焦去模糊网络

LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network

We present a method that takes as input a single dual-pixel image, and
simultaneously estimates the image's defocus map -- the amount of defocus blur
at each pixel -- and recovers an all-in-focus image. Our method is inspired
from recent works that leverage the dual-pixel sensors available in many
consumer cameras to assist with autofocus, and use them for recovery of defocus
maps or all-in-focus images. These prior works have solved the two recovery
problems independently of each other, and often require large labeled datasets
for supervised training. By contrast, we show that it is beneficial to treat
these two closely-connected problems simultaneously. To this end, we set up an
optimization problem that, by carefully modeling the optics of dual-pixel
images, jointly solves both problems. We use data captured with a consumer
smartphone camera to demonstrate that, after a one-time calibration step, our
approach improves upon prior works for both defocus map estimation and blur
removal, despite being entirely unsupervised.

该研究提出了一种基于双像素图像的无监督学习方法，通过同时估计图像的焦散图和所有点的聚焦图来减轻焦散的影响并生成全聚焦图。