Personalized text-to-image generation has attracted unprecedented attention
in the recent few years due to its unique capability of generating
highly-personalized images via using the input concept dataset and novel
textual prompt. However, previous methods solely focus on the performance of
the reconstruction task, degrading its ability to combine with different
textual prompt. Besides, optimizing in the high-dimensional embedding space
usually leads to unnecessary time-consuming training process and slow
convergence. To address these issues, we propose an efficient method to explore
the target embedding in a textual subspace, drawing inspiration from the
self-expressiveness property. Additionally, we propose an efficient selection
strategy for determining the basis vectors of the textual subspace. The
experimental evaluations demonstrate that the learned embedding can not only
faithfully reconstruct input image, but also significantly improves its
alignment with novel input textual prompt. Furthermore, we observe that
optimizing in the textual subspace leads to an significant improvement of the
robustness to the initial word, relaxing the constraint that requires users to
input the most relevant initial word. Our method opens the door to more
efficient representation learning for personalized text-to-image generation.

我们提出了一种有效的方法来在文本子空间中探索目标嵌入，借鉴了自我表现性特性，并提出了一种有效的选择策略来确定文本子空间的基向量。实验评估表明，所学习的嵌入不仅能够忠实地重构输入图像，还能显著改善其与新的输入文本提示的对齐性。此外，我们观察到在文本子空间中优化能够显著提高对初始词的鲁棒性，从而放宽了用户需要输入最相关初始词的约束。我们的方法为个性化文本到图像生成的更高效的表示学习打开了大门。

通过利用文本子空间提高高效个性化文本到图像生成

Efficient Personalized Text-to-image Generation by Leveraging Textual  Subspace

Image restoration has always been a cutting-edge topic in the academic and
industrial fields of computer vision. Since degradation signals are often
random and diverse, "all-in-one" models that can do blind image restoration
have been concerned in recent years. Early works require training specialized
headers and tails to handle each degradation of concern, which are manually
cumbersome. Recent works focus on learning visual prompts from data
distribution to identify degradation type. However, the prompts employed in
most of models are non-text, lacking sufficient emphasis on the importance of
human-in-the-loop. In this paper, an effective textual prompt guided image
restoration model has been proposed. In this model, task-specific BERT is
fine-tuned to accurately understand user's instructions and generating textual
prompt guidance. Depth-wise multi-head transposed attentions and gated
convolution modules are designed to bridge the gap between textual prompts and
visual features. The proposed model has innovatively introduced semantic
prompts into low-level visual domain. It highlights the potential to provide a
natural, precise, and controllable way to perform image restoration tasks.
Extensive experiments have been done on public denoising, dehazing and
deraining datasets. The experiment results demonstrate that, compared with
popular state-of-the-art methods, the proposed model can obtain much more
superior performance, achieving accurate recognition and removal of degradation
without increasing model's complexity. Related source codes and data will be
publicly available on github site
this https URL

本研究提出了一种有效的基于文本提示的图像修复模型，通过任务特定的 BERT 进行精确理解用户指令并生成文本提示，设计了深度多头转置注意力和门控卷积模块来弥合文本提示和视觉特征之间的差距，创新地将语义提示引入到低层视觉域中，实验证明该模型在公共去噪、去雾和去雨数据集上获得了显著优越的性能，能够准确识别和去除图像的退化，无需增加模型的复杂性。

文本提示引导的图像恢复

Textual Prompt Guided Image Restoration

We introduce the new setting of open-vocabulary object 6D pose estimation, in
which a textual prompt is used to specify the object of interest. In contrast
to existing approaches, in our setting (i) the object of interest is specified
solely through the textual prompt, (ii) no object model (e.g. CAD or video
sequence) is required at inference, (iii) the object is imaged from two
different viewpoints of two different scenes, and (iv) the object was not
observed during the training phase. To operate in this setting, we introduce a
novel approach that leverages a Vision-Language Model to segment the object of
interest from two distinct scenes and to estimate its relative 6D pose. The key
of our approach is a carefully devised strategy to fuse object-level
information provided by the prompt with local image features, resulting in a
feature space that can generalize to novel concepts. We validate our approach
on a new benchmark based on two popular datasets, REAL275 and Toyota-Light,
which collectively encompass 39 object instances appearing in four thousand
image pairs. The results demonstrate that our approach outperforms both a
well-established hand-crafted method and a recent deep learning-based baseline
in estimating the relative 6D pose of objects in different scenes. Project
website: this https URL

我们在研究中引入了开放词汇的物体 6D 姿态估计新设置，其中使用文本提示来指定感兴趣的物体，并提出了一种新方法来通过视觉 - 语言模型从两个不同场景的图像中分割和估计物体的相对 6D 姿态。