The field of image synthesis has made tremendous strides forward in the last
years. Besides defining the desired output image with text-prompts, an
intuitive approach is to additionally use spatial guidance in form of an image,
such as a depth map. For this, a recent and highly popular approach is to use a
controlling network, such as ControlNet, in combination with a pre-trained
image generation model, such as Stable Diffusion. When evaluating the design of
existing controlling networks, we observe that they all suffer from the same
problem of a delay in information flowing between the generation and
controlling process. This, in turn, means that the controlling network must
have generative capabilities. In this work we propose a new controlling
architecture, called ControlNet-XS, which does not suffer from this problem,
and hence can focus on the given task of learning to control. In contrast to
ControlNet, our model needs only a fraction of parameters, and hence is about
twice as fast during inference and training time. Furthermore, the generated
images are of higher quality and the control is of higher fidelity. All code
and pre-trained models will be made publicly available.

图像合成领域取得了巨大的进展，在本文中，我们提出了一种新的控制架构 ControlNet-XS，它不受信息流动延迟的问题困扰，因此可以专注于学习控制任务。与现有的控制网络相比，我们的模型参数量少，推理和训练时间大约快两倍，生成的图像质量更高，控制精度更高。