Adjusting the photo color to associate with some design elements is an
essential way for a graphic design to effectively deliver its message and make
it aesthetically pleasing. However, existing tools and previous works face a
dilemma between the ease of use and level of expressiveness. To this end, we
introduce an interactive language-based approach for photo recoloring, which
provides an intuitive system that can assist both experts and novices on
graphic design. Given a graphic design containing a photo that needs to be
recolored, our model can predict the source colors and the target regions, and
then recolor the target regions with the source colors based on the given
language-based instruction. The multi-granularity of the instruction allows
diverse user intentions. The proposed novel task faces several unique
challenges, including: 1) color accuracy for recoloring with exactly the same
color from the target design element as specified by the user; 2)
multi-granularity instructions for parsing instructions correctly to generate a
specific result or multiple plausible ones; and 3) locality for recoloring in
semantically meaningful local regions to preserve original image semantics. To
address these challenges, we propose a model called LangRecol with two main
components: the language-based source color prediction module and the
semantic-palette-based photo recoloring module. We also introduce an approach
for generating a synthetic graphic design dataset with instructions to enable
model training. We evaluate our model via extensive experiments and user
studies. We also discuss several practical applications, showing the
effectiveness and practicality of our approach. Code and data for this paper
are at: this https URL

通过基于语言的方法，我们提供了一个直观的系统，可以帮助图形设计专家和新手进行照片重新着色，以解决现有工具和之前的作品在易用性和表现力之间的困境。