Existing text-to-image models still struggle to generate images of multiple
objects, especially in handling their spatial positions, relative sizes,
overlapping, and attribute bindings. In this paper, we develop a training-free
Multimodal-LLM agent (MuLan) to address these challenges by progressive
multi-object generation with planning and feedback control, like a human
painter. MuLan harnesses a large language model (LLM) to decompose a prompt to
a sequence of sub-tasks, each generating only one object conditioned on
previously generated objects by stable diffusion. Unlike existing LLM-grounded
methods, MuLan only produces a high-level plan at the beginning while the exact
size and location of each object are determined by an LLM and attention
guidance upon each sub-task. Moreover, MuLan adopts a vision-language model
(VLM) to provide feedback to the image generated in each sub-task and control
the diffusion model to re-generate the image if it violates the original
prompt. Hence, each model in every step of MuLan only needs to address an easy
sub-task it is specialized for. We collect 200 prompts containing multi-objects
with spatial relationships and attribute bindings from different benchmarks to
evaluate MuLan. The results demonstrate the superiority of MuLan in generating
multiple objects over baselines. The code is available on
this https URL

通过渐进式多对象生成、规划和反馈控制，我们开发了一种无需训练的多模态语言模型代理（MuLan），以解决现有文本到图像模型在处理多对象、对象空间位置、相对大小、重叠和属性绑定方面的困难。

MuLan: 多模态 - LLM 进化多对象扩散智能体

MuLan: Multimodal-LLM Agent for Progressive Multi-Object Diffusion

Forecasting motion and spatial positions of objects is of fundamental
importance, especially in safety-critical settings such as autonomous driving.
In this work, we address the issue by forecasting two different modalities that
carry complementary information, namely optical flow and depth. To this end we
propose FLODCAST a flow and depth forecasting model that leverages a multitask
recurrent architecture, trained to jointly forecast both modalities at once. We
stress the importance of training using flows and depth maps together,
demonstrating that both tasks improve when the model is informed of the other
modality. We train the proposed model to also perform predictions for several
timesteps in the future. This provides better supervision and leads to more
precise predictions, retaining the capability of the model to yield outputs
autoregressively for any future time horizon. We test our model on the
challenging Cityscapes dataset, obtaining state of the art results for both
flow and depth forecasting. Thanks to the high quality of the generated flows,
we also report benefits on the downstream task of segmentation forecasting,
injecting our predictions in a flow-based mask-warping framework.

本研究提出了一种多任务循环架构的流和深度预测模型 FLODCAST，通过训练使用流和深度地图的多个时刻一起进行预测，改善了两个任务的性能，并在 Cityscapes 数据集上获得了最先进的结果。

FLODCAST：多模态递归架构中的流量和深度预测

FLODCAST: Flow and Depth Forecasting via Multimodal Recurrent  Architectures

We propose a novel approach to reduce the computational cost of evaluation of
convolutional neural networks, a factor that has hindered their deployment in
low-power devices such as mobile phones. Inspired by the loop perforation
technique from source code optimization, we speed up the bottleneck
convolutional layers by skipping their evaluation in some of the spatial
positions. We propose and analyze several strategies of choosing these
positions. We demonstrate that perforation can accelerate modern convolutional
networks such as AlexNet and VGG-16 by a factor of 2x - 4x. Additionally, we
show that perforation is complementary to the recently proposed acceleration
method of Zhang et al.

通过循环穿孔技术，我们提出一种降低卷积神经网络在低功耗设备如手机上计算成本的新方法，可以将现代卷积网络的速度提高 2 倍至 4 倍，这种方法还可以补充最近由 Zhang 等人提出的加速方法。