Text-to-image generation models have seen considerable advancement, catering
to the increasing interest in personalized image creation. Current
customization techniques often necessitate users to provide multiple images
(typically 3-5) for each customized object, along with the classification of
these objects and descriptive textual prompts for scenes. This paper questions
whether the process can be made more user-friendly and the customization more
intricate. We propose a method where users need only provide images along with
text for each customization topic, and necessitates only a single image per
visual concept. We introduce the concept of a ``multi-modal prompt'', a novel
integration of text and images tailored to each customization concept, which
simplifies user interaction and facilitates precise customization of both
objects and scenes. Our proposed paradigm for customized text-to-image
generation surpasses existing finetune-based methods in user-friendliness and
the ability to customize complex objects with user-friendly inputs. Our code is
available at
$\href{https://github.com/zhongzero/Multi-Modal-Prompt}{this https URL}$.

本文提出了一种方法，用户只需提供每个定制主题的图像和文本，而不需要提供多张图像，通过引入 “多模态提示” 概念，将文本和图像相结合，简化用户交互，实现对对象和场景的精确定制。该方法在用户友好性和使用友好输入定制复杂对象方面超过了现有的基于微调方法。

多模式提示下的用户友好定制生成

User-Friendly Customized Generation with Multi-Modal Prompts

Customization techniques for text-to-image models have paved the way for a
wide range of previously unattainable applications, enabling the generation of
specific concepts across diverse contexts and styles. While existing methods
facilitate high-fidelity customization for individual concepts or a limited,
pre-defined set of them, they fall short of achieving scalability, where a
single model can seamlessly render countless concepts. In this paper, we
address a new problem called Modular Customization, with the goal of
efficiently merging customized models that were fine-tuned independently for
individual concepts. This allows the merged model to jointly synthesize
concepts in one image without compromising fidelity or incurring any additional
computational costs.
To address this problem, we introduce Orthogonal Adaptation, a method
designed to encourage the customized models, which do not have access to each
other during fine-tuning, to have orthogonal residual weights. This ensures
that during inference time, the customized models can be summed with minimal
interference.
Our proposed method is both simple and versatile, applicable to nearly all
optimizable weights in the model architecture. Through an extensive set of
quantitative and qualitative evaluations, our method consistently outperforms
relevant baselines in terms of efficiency and identity preservation,
demonstrating a significant leap toward scalable customization of diffusion
models.

通过引入正交适应方法，我们解决了一个名为模块化定制的新问题，旨在高效合并独立为个别概念进行微调的定制模型，从而在一幅图像中联合合成概念，同时保持保真度且不增加计算开销，这一方法在效率和身份保持方面始终优于相关基线。

扩散模型模块定制的正交适应

Orthogonal Adaptation for Modular Customization of Diffusion Models

This paper presents an evaluation of edge detectors using receiver operating
characteristic (ROC) analysis on the BIPED dataset. Our study examines the
benefits and drawbacks of applying this technique in Matlab. We observed that
while ROC analysis is suitable for certain edge filters, but for filters such
as Laplacian, Laplacian of Gaussian, and Canny, it presents challenges when
accurately measuring their performance using ROC metrics. To address this
issue, we introduce customization techniques to enhance the performance of
these filters, enabling more accurate evaluation. Through our customization
efforts, we achieved improved results, ultimately facilitating a comprehensive
assessment of the edge detectors.

该论文使用 BIPED 数据集，使用接收器操作特征（ROC）分析方法来评估边缘检测器，探讨了在 Matlab 中应用该技术的优缺点。作者通过定制技术提高了 Laplacian，Laplacian of Gaussian 和 Canny 等过滤器的性能，从而更准确地评估边缘检测器的综合表现。