The enormous success of diffusion models in text-to-image synthesis has made
them promising candidates for the next generation of end-user applications for
image generation and editing. Previous works have focused on improving the
usability of diffusion models by reducing the inference time or increasing user
interactivity by allowing new, fine-grained controls such as region-based text
prompts. However, we empirically find that integrating both branches of works
is nontrivial, limiting the potential of diffusion models. To solve this
incompatibility, we present StreamMultiDiffusion, the first real-time
region-based text-to-image generation framework. By stabilizing fast inference
techniques and restructuring the model into a newly proposed multi-prompt
stream batch architecture, we achieve $\times 10$ faster panorama generation
than existing solutions, and the generation speed of 1.57 FPS in region-based
text-to-image synthesis on a single RTX 2080 Ti GPU. Our solution opens up a
new paradigm for interactive image generation named semantic palette, where
high-quality images are generated in real-time from given multiple hand-drawn
regions, encoding prescribed semantic meanings (e.g., eagle, girl). Our code
and demo application are available at
this https URL

通过稳定快速推理技术和重构模型为新提出的多提示流批处理架构，我们提出了 StreamMultiDiffusion，这是第一个基于区域的实时文本到图像生成框架，在单个 RTX 2080 Ti GPU 上实现了 1.57 FPS 的区域文本到图像合成生成速度，比现有解决方案快 10 倍。

流式多扩散：基于区域的语义控制下的实时交互生成

StreamMultiDiffusion: Real-Time Interactive Generation with Region-Based  Semantic Control

Effective resistance (ER) is an attractive way to interrogate the structure
of graphs. It is an alternative to computing the eigenvectors of the graph
Laplacian.
One attractive application of ER is to point clouds, i.e. graphs whose
vertices correspond to IID samples from a distribution over a metric space.
Unfortunately, it was shown that the ER between any two points converges to a
trivial quantity that holds no information about the graph's structure as the
size of the sample increases to infinity.
In this study, we show that this trivial solution can be circumvented by
considering a region-based ER between pairs of small regions rather than pairs
of points and by scaling the edge weights appropriately with respect to the
underlying density in each region. By keeping the regions fixed, we show
analytically that the region-based ER converges to a non-trivial limit as the
number of points increases to infinity. Namely the ER on a metric space. We
support our theoretical findings with numerical experiments.

本文介绍一种基于区域的有效电阻方法来研究图表结构，在度量空间中应用该方法可以获得一个与区域相关的非平凡解。

度量空间中的有效阻抗

Effective resistance in metric spaces

Referring Expression Segmentation (RES) aims to generate a segmentation mask
for the object described by a given language expression. Existing classic RES
datasets and methods commonly support single-target expressions only, i.e., one
expression refers to one target object. Multi-target and no-target expressions
are not considered. This limits the usage of RES in practice. In this paper, we
introduce a new benchmark called Generalized Referring Expression Segmentation
(GRES), which extends the classic RES to allow expressions to refer to an
arbitrary number of target objects. Towards this, we construct the first
large-scale GRES dataset called gRefCOCO that contains multi-target, no-target,
and single-target expressions. GRES and gRefCOCO are designed to be
well-compatible with RES, facilitating extensive experiments to study the
performance gap of the existing RES methods on the GRES task. In the
experimental study, we find that one of the big challenges of GRES is complex
relationship modeling. Based on this, we propose a region-based GRES baseline
ReLA that adaptively divides the image into regions with sub-instance clues,
and explicitly models the region-region and region-language dependencies. The
proposed approach ReLA achieves new state-of-the-art performance on the both
newly proposed GRES and classic RES tasks. The proposed gRefCOCO dataset and
method are available at this https URL

本文介绍了一个新的基准测试 Generalized Referring Expression Segmentation (GRES)，它允许表达式引用任意数量的目标对象，为此我们构建了第一个大规模数据集 gRefCOCO，并提出一种基于区域的 GRES 基线 ReLA，它适应性地将图像划分为具有子实例线索的区域，并明确建模区域 - 区域和区域 - 语言之间的依赖关系，在实验研究中，发现 GRES 的一个重要挑战是复杂的关系建模，而基于区域的方法 ReLA 在新提出的 GRES 和经典的 RES 任务上实现了最新的性能水平。