Existing information retrieval (IR) models often assume a homogeneous format,
limiting their applicability to diverse user needs, such as searching for
images with text descriptions, searching for a news article with a headline
image, or finding a similar photo with a query image. To approach such
different information-seeking demands, we introduce UniIR, a unified
instruction-guided multimodal retriever capable of handling eight distinct
retrieval tasks across modalities. UniIR, a single retrieval system jointly
trained on ten diverse multimodal-IR datasets, interprets user instructions to
execute various retrieval tasks, demonstrating robust performance across
existing datasets and zero-shot generalization to new tasks. Our experiments
highlight that multi-task training and instruction tuning are keys to UniIR's
generalization ability. Additionally, we construct the M-BEIR, a multimodal
retrieval benchmark with comprehensive results, to standardize the evaluation
of universal multimodal information retrieval.

对于信息检索 (IR) 模型通常假设为同质化格式，限制了其适用于各种用户需求，如使用文本描述搜索图片，使用标题图搜索新闻文章或使用查询图搜索相似照片。为了满足这种不同的信息需求，我们引入了 UniIR，一种统一的指令引导多模态检索器，可以处理八个不同的跨模态检索任务。UniIR 是一个单一的检索系统，同时在十个不同的多模态 - IR 数据集上进行训练，解释用户指令来执行各种检索任务，展示了对现有数据集的鲁棒性能以及对新任务的零 - shot 泛化能力。我们的实验突出了多任务训练和指令调整是 UniIR 泛化能力的关键。此外，我们构建了 M-BEIR，一个综合结果的多模态检索基准，用于标准化通用多模态信息检索的评估。