In this paper, we primarily address the issue of dialogue-form context query
within the interactive text-to-image retrieval task. Our methodology, PlugIR,
actively utilizes the general instruction-following capability of LLMs in two
ways. First, by reformulating the dialogue-form context, we eliminate the
necessity of fine-tuning a retrieval model on existing visual dialogue data,
thereby enabling the use of any arbitrary black-box model. Second, we construct
the LLM questioner to generate non-redundant questions about the attributes of
the target image, based on the information of retrieval candidate images in the
current context. This approach mitigates the issues of noisiness and redundancy
in the generated questions. Beyond our methodology, we propose a novel
evaluation metric, Best log Rank Integral (BRI), for a comprehensive assessment
of the interactive retrieval system. PlugIR demonstrates superior performance
compared to both zero-shot and fine-tuned baselines in various benchmarks.
Additionally, the two methodologies comprising PlugIR can be flexibly applied
together or separately in various situations. Our codes are available at
this https URL

该论文主要关注互动式文本图像检索任务中对话形式的上下文查询问题。PlugIR 方法利用 LLMs 的一般指令遵循能力以两种方式解决了这个问题。该方法不仅提出了一种新颖的评估指标 BRI，还展示了在各种基准测试中优于零射击和微调基准线的优越性能。