In this paper we propose a novel modification of Contrastive Language-Image
Pre-Training (CLIP) guidance for the task of unsupervised backlit image
enhancement. Our work builds on the state-of-the-art CLIP-LIT approach, which
learns a prompt pair by constraining the text-image similarity between a prompt
(negative/positive sample) and a corresponding image (backlit image/well-lit
image) in the CLIP embedding space. Learned prompts then guide an image
enhancement network. Based on the CLIP-LIT framework, we propose two novel
methods for CLIP guidance. First, we show that instead of tuning prompts in the
space of text embeddings, it is possible to directly tune their embeddings in
the latent space without any loss in quality. This accelerates training and
potentially enables the use of additional encoders that do not have a text
encoder. Second, we propose a novel approach that does not require any prompt
tuning. Instead, based on CLIP embeddings of backlit and well-lit images from
training data, we compute the residual vector in the embedding space as a
simple difference between the mean embeddings of the well-lit and backlit
images. This vector then guides the enhancement network during training,
pushing a backlit image towards the space of well-lit images. This approach
further dramatically reduces training time, stabilizes training and produces
high quality enhanced images without artifacts, both in supervised and
unsupervised training regimes. Additionally, we show that residual vectors can
be interpreted, revealing biases in training data, and thereby enabling
potential bias correction.

本研究提出了对对比语言 - 图像预训练（CLIP）指导的一种新型修改，用于无监督背光图像增强任务，通过学习背光图像和良好照明图像之间的文本 - 图像相似度来引导图像增强网络，并通过计算嵌入空间中的残差向量来指导增强网络的训练，以实现背光图像向良好照明图像的转化，并揭示了训练数据中的偏见，从而使潜在偏差修正成为可能。