The recent large-scale generative modeling has attained unprecedented
performance especially in producing high-fidelity images driven by text
prompts. Text inversion (TI), alongside the text-to-image model backbones, is
proposed as an effective technique in personalizing the generation when the
prompts contain user-defined, unseen or long-tail concept tokens. Despite that,
we find and show that the deployment of TI remains full of "dark-magics" -- to
name a few, the harsh requirement of additional datasets, arduous human efforts
in the loop and lack of robustness. In this work, we propose a much-enhanced
version of TI, dubbed Controllable Textual Inversion (COTI), in resolving all
the aforementioned problems and in turn delivering a robust, data-efficient and
easy-to-use framework. The core to COTI is a theoretically-guided loss
objective instantiated with a comprehensive and novel weighted scoring
mechanism, encapsulated by an active-learning paradigm. The extensive results
show that COTI significantly outperforms the prior TI-related approaches with a
26.05 decrease in the FID score and a 23.00% boost in the R-precision.

本文提出了一种名为 COTI 的新方法，应用在大规模生成模型中，通过一个综合的、新颖的加权评分机制指导的理论损失目标，实现了具有鲁棒性、数据高效性和易于使用等特点，能够显著提升 FID 得分并增加 R-precision 的生成模型，一定程度上解决了过往生成模型中仍存在的问题。