Training Artificial Intelligence (AI) models on three-dimensional image data
presents unique challenges compared to the two-dimensional case: Firstly, the
computational resources are significantly higher, and secondly, the
availability of large pretraining datasets is often limited, impeding training
success. In this study, we propose a simple approach of adapting 2D networks
with an intermediate feature representation for processing 3D volumes. Our
method involves sequentially applying these networks to slices of a 3D volume
from all orientations. Subsequently, a feature reduction module combines the
extracted slice features into a single representation, which is then used for
classification. We evaluate our approach on medical classification benchmarks
and a real-world clinical dataset, demonstrating comparable results to existing
methods. Furthermore, by employing attention pooling as a feature reduction
module we obtain weighted importance values for each slice during the forward
pass. We show that slices deemed important by our approach allow the inspection
of the basis of a model's prediction.

本研究提出了一种简单的方法，使用 2D 网络和中间特征表示来处理 3D 数据，并使用特征减少模块进行分类，以解决在医学分类的基准测试和实际临床数据中训练成功受限的问题。

可解释的二维视觉模型用于三维医学数据

Explainable 2D Vision Models for 3D Medical Data

3D convolutional neural networks have revealed superior performance in
processing volumetric data such as video and medical imaging. However, the
competitive performance by leveraging 3D networks results in huge computational
costs, which are far beyond that of 2D networks. In this paper, we propose a
novel Hilbert curve-based cross-dimensionality distillation approach that
facilitates the knowledge of 3D networks to improve the performance of 2D
networks. The proposed Hilbert Distillation (HD) method preserves the
structural information via the Hilbert curve, which maps high-dimensional (>=2)
representations to one-dimensional continuous space-filling curves. Since the
distilled 2D networks are supervised by the curves converted from dimensionally
heterogeneous 3D features, the 2D networks are given an informative view in
terms of learning structural information embedded in well-trained
high-dimensional representations. We further propose a Variable-length Hilbert
Distillation (VHD) method to dynamically shorten the walking stride of the
Hilbert curve in activation feature areas and lengthen the stride in context
feature areas, forcing the 2D networks to pay more attention to learning from
activation features. The proposed algorithm outperforms the current
state-of-the-art distillation techniques adapted to cross-dimensionality
distillation on two classification tasks. Moreover, the distilled 2D networks
by the proposed method achieve competitive performance with the original 3D
networks, indicating the lightweight distilled 2D networks could potentially be
the substitution of cumbersome 3D networks in the real-world scenario.

本文提出了一种全新的基于希尔伯特曲线的跨维度蒸馏方法 (HD)，通过将 3D 特征转换为一维连续的空间填充曲线，将 3D 网络的知识迁移到提高 2D 网络性能，进一步提出了一种动态调整步幅的变长度希尔伯特蒸馏方法 (VHD)，可以更好地学习激活特征，相比现有的跨维度蒸馏技术，实验表明本文提出的算法在两个分类任务上性能更优，而且所蒸馏的 2D 网络具有与 3D 网络相媲美的性能水平，证明轻量级的蒸馏 2D 网络可以成为麻烦的 3D 网络的替代品之一

跨维度网络的希尔伯特蒸馏

Hilbert Distillation for Cross-Dimensionality Networks

Deep learning is showing an increasing number of audience in medical imaging
research. In the segmentation task of medical images, we oftentimes rely on
volumetric data, and thus require the use of 3D architectures which are praised
for their ability to capture more features from the depth dimension. Yet, these
architectures are generally more ineffective in time and compute compared to
their 2D counterpart on account of 3D convolutions, max pooling,
up-convolutions, and other operations used in these networks. Moreover, there
are limited to no 3D pretrained model weights, and pretraining is generally
challenging. To alleviate these issues, we propose to cast volumetric data to
2D super images and use 2D networks for the segmentation task. The method
processes the 3D image by stitching slices side-by-side to generate a super
resolution image. While the depth information is lost, we expect that deep
neural networks can still capture and learn these features. Our goal in this
work is to introduce a new perspective when dealing with volumetric data, and
test our hypothesis using vanilla networks. We hope that this approach, while
achieving close enough results to 3D networks using only 2D counterparts, can
attract more related research in the future, especially in medical image
analysis since volumetric data is comparably limited.

为了解决 3D 卷积神经网络在医学影像识别中时间和计算效率低的问题，本文提出将体数据转化为 2D 超级图像，使用 2D 网络用于分割任务，具有实用价值和吸引更多相关研究的意义。

基于超级图像的分割: 3D 医学图像分析中的新 2D 视角

Segmentation with Super Images: A New 2D Perspective on 3D Medical Image Analysis

Most 3D neural networks are trained from scratch owing to the lack of
large-scale labeled 3D datasets. In this paper, we present a novel 3D
pretraining method by leveraging 2D networks learned from rich 2D datasets. We
propose the contrastive pixel-to-point knowledge transfer to effectively
utilize the 2D information by mapping the pixel-level and point-level features
into the same embedding space. Due to the heterogeneous nature between 2D and
3D networks, we introduce the back-projection function to align the features
between 2D and 3D to make the transfer possible. Additionally, we devise an
upsampling feature projection layer to increase the spatial resolution of
high-level 2D feature maps, which enables learning fine-grained 3D
representations. With a pretrained 2D network, the proposed pretraining process
requires no additional 2D or 3D labeled data, further alleviating the expensive
3D data annotation cost. To the best of our knowledge, we are the first to
exploit existing 2D trained weights to pretrain 3D deep neural networks. Our
intensive experiments show that the 3D models pretrained with 2D knowledge
boost the performances of 3D networks across various real-world 3D downstream
tasks.

本文提出了一种使用 2D 网络预训练 3D 深度神经网络的新方法，包括像素到点知识转移、反向投影函数、上采样特征投影层等，成功提升了各种真实世界 3D 网络下游任务的模型性能。