Achieving machine autonomy and human control often represent divergent
objectives in the design of interactive AI systems. Visual generative
foundation models such as Stable Diffusion show promise in navigating these
goals, especially when prompted with arbitrary languages. However, they often
fall short in generating images with spatial, structural, or geometric
controls. The integration of such controls, which can accommodate various
visual conditions in a single unified model, remains an unaddressed challenge.
In response, we introduce UniControl, a new generative foundation model that
consolidates a wide array of controllable condition-to-image (C2I) tasks within
a singular framework, while still allowing for arbitrary language prompts.
UniControl enables pixel-level-precise image generation, where visual
conditions primarily influence the generated structures and language prompts
guide the style and context. To equip UniControl with the capacity to handle
diverse visual conditions, we augment pretrained text-to-image diffusion models
and introduce a task-aware HyperNet to modulate the diffusion models, enabling
the adaptation to different C2I tasks simultaneously. Trained on nine unique
C2I tasks, UniControl demonstrates impressive zero-shot generation abilities
with unseen visual conditions. Experimental results show that UniControl often
surpasses the performance of single-task-controlled methods of comparable model
sizes. This control versatility positions UniControl as a significant
advancement in the realm of controllable visual generation.

UniControl 是一种新的可控生成基础模型，集成了多种可控因素，实现了基于任意语言提示的像素级精确图像生成，并通过多任务学习，使其具有适应不同可控条件的能力，实验证明其性能优于其他同类型模型。

UniControl：一种用于自然环境中可控视觉生成的统一扩散模型

UniControl: A Unified Diffusion Model for Controllable Visual Generation  In the Wild

Driving safely requires multiple capabilities from human and intelligent
agents, such as the generalizability to unseen environments, the safety
awareness of the surrounding traffic, and the decision-making in complex
multi-agent settings. Despite the great success of Reinforcement Learning (RL),
most of the RL research works investigate each capability separately due to the
lack of integrated environments. In this work, we develop a new driving
simulation platform called MetaDrive to support the research of generalizable
reinforcement learning algorithms for machine autonomy. MetaDrive is highly
compositional, which can generate an infinite number of diverse driving
scenarios from both the procedural generation and the real data importing.
Based on MetaDrive, we construct a variety of RL tasks and baselines in both
single-agent and multi-agent settings, including benchmarking generalizability
across unseen scenes, safe exploration, and learning multi-agent traffic. The
generalization experiments conducted on both procedurally generated scenarios
and real-world scenarios show that increasing the diversity and the size of the
training set leads to the improvement of the RL agent's generalizability. We
further evaluate various safe reinforcement learning and multi-agent
reinforcement learning algorithms in MetaDrive environments and provide the
benchmarks. Source code, documentation, and demo video are available at \url{
this https URL}.

本文通过开发一个名为 MetaDrive 的驾驶仿真平台来支持机器自我行驶可推广的强化学习算法的研究，并构建了各种单智能体和多智能体的强化学习任务和基准测试，评估了在不同场景下机器的行为以及提出了安全强化学习算法和多智能体强化学习算法的基准测试，训练循环增加可以提高机器的推广能力。