U-Nets are among the most widely used architectures in computer vision,
renowned for their exceptional performance in applications such as image
segmentation, denoising, and diffusion modeling. However, a theoretical
explanation of the U-Net architecture design has not yet been fully
established.
This paper introduces a novel interpretation of the U-Net architecture by
studying certain generative hierarchical models, which are tree-structured
graphical models extensively utilized in both language and image domains. With
their encoder-decoder structure, long skip connections, and pooling and
up-sampling layers, we demonstrate how U-Nets can naturally implement the
belief propagation denoising algorithm in such generative hierarchical models,
thereby efficiently approximating the denoising functions. This leads to an
efficient sample complexity bound for learning the denoising function using
U-Nets within these models. Additionally, we discuss the broader implications
of these findings for diffusion models in generative hierarchical models. We
also demonstrate that the conventional architecture of convolutional neural
networks (ConvNets) is ideally suited for classification tasks within these
models. This offers a unified view of the roles of ConvNets and U-Nets,
highlighting the versatility of generative hierarchical models in modeling
complex data distributions across language and image domains.

通过研究某些生成式分层模型，本文引入了 U-Net 架构的新解释，它是一种在语言和图像领域广泛使用的树状结构图模型。我们演示了 U-Net 如何自然地在这些生成式分层模型中实现置信传播去噪算法，并以此高效逼近去噪函数。此外，我们讨论了这些发现对生成式分层模型中扩散模型的更广泛影响，还证明了卷积神经网络 (ConvNets) 的传统架构在这些模型中非常适合分类任务，从而突显出生成式分层模型在语言和图像领域中建模复杂数据分布的多功能性。