U-Net has become a cornerstone in various visual applications such as image
segmentation and diffusion probability models. While numerous innovative
designs and improvements have been introduced by incorporating transformers or
MLPs, the networks are still limited to linearly modeling patterns as well as
the deficient interpretability. To address these challenges, our intuition is
inspired by the impressive results of the Kolmogorov-Arnold Networks (KANs) in
terms of accuracy and interpretability, which reshape the neural network
learning via the stack of non-linear learnable activation functions derived
from the Kolmogorov-Anold representation theorem. Specifically, in this paper,
we explore the untapped potential of KANs in improving backbones for vision
tasks. We investigate, modify and re-design the established U-Net pipeline by
integrating the dedicated KAN layers on the tokenized intermediate
representation, termed U-KAN. Rigorous medical image segmentation benchmarks
verify the superiority of U-KAN by higher accuracy even with less computation
cost. We further delved into the potential of U-KAN as an alternative U-Net
noise predictor in diffusion models, demonstrating its applicability in
generating task-oriented model architectures. These endeavours unveil valuable
insights and sheds light on the prospect that with U-KAN, you can make strong
backbone for medical image segmentation and generation. Project page:
this https URL

我们通过将 KAN 层整合到中间表示中，探索了 Kolmogorov-Arnold 网络（KANs）在改进视觉任务的骨干网络中的潜力，并证明了 U-KAN 在医学图像分割中的优越性。

U-KAN 为医学图像分割和生成提供了强大的骨干

U-KAN Makes Strong Backbone for Medical Image Segmentation and  Generation

Diffusion Probability Models (DPMs) have made impressive advancements in
various machine learning domains. However, achieving high-quality synthetic
samples typically involves performing a large number of sampling steps, which
impedes the possibility of real-time sample synthesis. Traditional accelerated
sampling algorithms via knowledge distillation rely on pre-trained model
weights and discrete time step scenarios, necessitating additional training
sessions to achieve their goals. To address these issues, we propose the
Catch-Up Distillation (CUD), which encourages the current moment output of the
velocity estimation model ``catch up'' with its previous moment output.
Specifically, CUD adjusts the original Ordinary Differential Equation (ODE)
training objective to align the current moment output with both the ground
truth label and the previous moment output, utilizing Runge-Kutta-based
multi-step alignment distillation for precise ODE estimation while preventing
asynchronous updates. Furthermore, we investigate the design space for CUDs
under continuous time-step scenarios and analyze how to determine the suitable
strategies. To demonstrate CUD's effectiveness, we conduct thorough ablation
and comparison experiments on CIFAR-10, MNIST, and ImageNet-64. On CIFAR-10, we
obtain a FID of 2.80 by sampling in 15 steps under one-session training and the
new state-of-the-art FID of 3.37 by sampling in one step with additional
training. This latter result necessitated only 62w iterations with a batch size
of 128, in contrast to Consistency Distillation, which demanded 210w iterations
with a larger batch size of 256.

为了解决 Diffusion Probability Models 生成样本所需的采样步骤大量增加的问题，文章提出了一种基于 Catch-Up Distillation (CUD) 的算法，通过多步对齐蒙古采样输出来实现精确 ODE 估计，成功地在 CIFAR-10，MNIST 和 ImageNet-64 数据集上取得了比较优异的效果。