Demand for image editing has been increasing as users' desire for expression
is also increasing. However, for most users, image editing tools are not easy
to use since the tools require certain expertise in photo effects and have
complex interfaces. Hence, users might need someone to help edit their images,
but having a personal dedicated human assistant for every user is impossible to
scale. For that reason, an automated assistant system for image editing is
desirable. Additionally, users want more image sources for diverse image
editing works, and integrating an image search functionality into the editing
tool is a potential remedy for this demand. Thus, we propose a dataset of an
automated Conversational Agent for Image Search and Editing (CAISE). To our
knowledge, this is the first dataset that provides conversational image search
and editing annotations, where the agent holds a grounded conversation with
users and helps them to search and edit images according to their requests. To
build such a system, we first collect image search and editing conversations
between pairs of annotators. The assistant-annotators are equipped with a
customized image search and editing tool to address the requests from the
user-annotators. The functions that the assistant-annotators conduct with the
tool are recorded as executable commands, allowing the trained system to be
useful for real-world application execution. We also introduce a
generator-extractor baseline model for this task, which can adaptively select
the source of the next token (i.e., from the vocabulary or from textual/visual
contexts) for the executable command. This serves as a strong starting point
while still leaving a large human-machine performance gap for useful future
work. Our code and dataset are publicly available at:
this https URL

提出了一个基于自动化对话代理的图像搜索和编辑数据集（CAISE），该数据集提供了包括代理机器人和用户之间的对话、编辑的过程和结果记录、和执行命令等。使用自定义搜索和编辑工具，辅助标注工人与用户进行搜索和编辑对话，并记录这些工具产生的功能。我们引入了生成器提取器基线模型，并使用公开可用的代码和数据集进行了实现，这是一个有用的起点。