Image generators are gaining vast amount of popularity and have rapidly
changed how digital content is created. With the latest AI technology, millions
of high quality images are being generated by the public, which are constantly
motivating the research community to push the limits of generative models to
create more complex and realistic images. This paper focuses on Cross-Domain
Image Retrieval (CDIR) which can be used as an additional tool to inspect
collections of generated images by determining the level of similarity between
images in a dataset. An ideal retrieval system would be able to generalize to
unseen complex images from multiple domains (e.g., photos, drawings and
paintings). To address this goal, we propose a novel caption-matching approach
that leverages multimodal language-vision architectures pre-trained on large
datasets. The method is tested on DomainNet and Office-Home datasets and
consistently achieves state-of-the-art performance over the latest approaches
in the literature for cross-domain image retrieval. In order to verify the
effectiveness with AI-generated images, the method was also put to test with a
database composed by samples collected from Midjourney, which is a widely used
generative platform for content creation.

使用多模态语言 - 视觉架构的新颖标题匹配方法能够在跨域图像检索中实现最先进的性能表现，尤其在与人工智能生成的图像一起进行测试时。

跨域图像检索的多模态方法

A Multimodal Approach for Cross-Domain Image Retrieval

Cross-domain image retrieval aims at retrieving images across different
domains to excavate cross-domain classificatory or correspondence
relationships. This paper studies a less-touched problem of cross-domain image
retrieval, i.e., unsupervised cross-domain image retrieval, considering the
following practical assumptions: (i) no correspondence relationship, and (ii)
no category annotations. It is challenging to align and bridge distinct domains
without cross-domain correspondence. To tackle the challenge, we present a
novel Correspondence-free Domain Alignment (CoDA) method to effectively
eliminate the cross-domain gap through In-domain Self-matching Supervision
(ISS) and Cross-domain Classifier Alignment (CCA). To be specific, ISS is
presented to encapsulate discriminative information into the latent common
space by elaborating a novel self-matching supervision mechanism. To alleviate
the cross-domain discrepancy, CCA is proposed to align distinct domain-specific
classifiers. Thanks to the ISS and CCA, our method could encode the
discrimination into the domain-invariant embedding space for unsupervised
cross-domain image retrieval. To verify the effectiveness of the proposed
method, extensive experiments are conducted on four benchmark datasets compared
with six state-of-the-art methods.

本文研究跨域图像检索的无监督情况，提出了一种新的无对应关系的跨域对齐方法，即不考虑对应关系和类别注释的情况下，利用 In-domain 自匹配监督机制和跨域分类器对齐消除跨域差异，实现了无监督跨域图像检索。

无需对应关系的领域对齐用于无监督跨领域图像检索

Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image  Retrieval

With the proliferation of e-commerce websites and the ubiquitousness of smart
phones, cross-domain image retrieval using images taken by smart phones as
queries to search products on e-commerce websites is emerging as a popular
application. One challenge of this task is to locate the attention of both the
query and database images. In particular, database images, e.g. of fashion
products, on e-commerce websites are typically displayed with other
accessories, and the images taken by users contain noisy background and large
variations in orientation and lighting. Consequently, their attention is
difficult to locate. In this paper, we exploit the rich tag information
available on the e-commerce websites to locate the attention of database
images. For query images, we use each candidate image in the database as the
context to locate the query attention. Novel deep convolutional neural network
architectures, namely TagYNet and CtxYNet, are proposed to learn the attention
weights and then extract effective representations of the images. Experimental
results on public datasets confirm that our approaches have significant
improvement over the existing methods in terms of the retrieval accuracy and
efficiency.

本文旨在通过在电商网站上的标签信息和候选图像上下文信息，利用了 TagYNet 和 CtxYNet 等深度卷积神经网络架构，以实现对基于智能手机拍摄的图像进行检索的 Attention weights 计算和图像特征提取，可显著提高检索的准确率和效率。

基于注意力模型的跨领域图像检索

Cross-Domain Image Retrieval with Attention Modeling

We address the problem of cross-domain image retrieval, considering the
following practical application: given a user photo depicting a clothing image,
our goal is to retrieve the same or attribute-similar clothing items from
online shopping stores. This is a challenging problem due to the large
discrepancy between online shopping images, usually taken in ideal
lighting/pose/background conditions, and user photos captured in uncontrolled
conditions. To address this problem, we propose a Dual Attribute-aware Ranking
Network (DARN) for retrieval feature learning. More specifically, DARN consists
of two sub-networks, one for each domain, whose retrieval feature
representations are driven by semantic attribute learning. We show that this
attribute-guided learning is a key factor for retrieval accuracy improvement.
In addition, to further align with the nature of the retrieval problem, we
impose a triplet visual similarity constraint for learning to rank across the
two sub-networks. Another contribution of our work is a large-scale dataset
which makes the network learning feasible. We exploit customer review websites
to crawl a large set of online shopping images and corresponding offline user
photos with fine-grained clothing attributes, i.e., around 450,000 online
shopping images and about 90,000 exact offline counterpart images of those
online ones. All these images are collected from real-world consumer websites
reflecting the diversity of the data modality, which makes this dataset unique
and rare in the academic community. We extensively evaluate the retrieval
performance of networks in different configurations. The top-20 retrieval
accuracy is doubled when using the proposed DARN other than the current popular
solution using pre-trained CNN features only (0.570 vs. 0.268).

本文提出了一种基于双属性感知排序网络的跨域图像检索方法，解决了实际应用中用户拍摄的非理想服装图像与在线购物图片之间巨大差距的问题，并通过语义属性学习、视觉相似性约束以及大规模数据集的使用极大地提高了图像检索的性能。