Analyzing volumetric data with rotational invariance or equivariance is an
active topic in current research. Existing deep-learning approaches utilize
either group convolutional networks limited to discrete rotations or steerable
convolutional networks with constrained filter structures. This work proposes a
novel equivariant neural network architecture that achieves analytical
Equivariance to Local Pattern Orientation on the continuous SO(3) group while
allowing unconstrained trainable filters - EquiLoPO Network. Our key
innovations are a group convolutional operation leveraging irreducible
representations as the Fourier basis and a local activation function in the
SO(3) space that provides a well-defined mapping from input to output
functions, preserving equivariance. By integrating these operations into a
ResNet-style architecture, we propose a model that overcomes the limitations of
prior methods. A comprehensive evaluation on diverse 3D medical imaging
datasets from MedMNIST3D demonstrates the effectiveness of our approach, which
consistently outperforms state of the art. This work suggests the benefits of
true rotational equivariance on SO(3) and flexible unconstrained filters
enabled by the local activation function, providing a flexible framework for
equivariant deep learning on volumetric data with potential applications across
domains. Our code is publicly available at
https://gricad-gitlab.univ-grenoble-alpes.fr/GruLab/ILPO/-/tree/main/EquiLoPO.

通过局部模式方向在连续 SO (3) 群上实现分析 3D 体积数据的等变神经网络，提供了对流形旋转不变性的创新方法，并显示出在各个领域具有潜在应用的灵活性。

关于 SO (3) 空间中的傅里叶分析：EquiLoPO 网络

On the Fourier analysis in the SO(3) space : EquiLoPO Network

We analyze the role of rotational equivariance in convolutional neural
networks (CNNs) applied to spherical images. We compare the performance of the
group equivariant networks known as S2CNNs and standard non-equivariant CNNs
trained with an increasing amount of data augmentation. The chosen
architectures can be considered baseline references for the respective design
paradigms. Our models are trained and evaluated on single or multiple items
from the MNIST or FashionMNIST dataset projected onto the sphere. For the task
of image classification, which is inherently rotationally invariant, we find
that by considerably increasing the amount of data augmentation and the size of
the networks, it is possible for the standard CNNs to reach at least the same
performance as the equivariant network. In contrast, for the inherently
equivariant task of semantic segmentation, the non-equivariant networks are
consistently outperformed by the equivariant networks with significantly fewer
parameters. We also analyze and compare the inference latency and training
times of the different networks, enabling detailed tradeoff considerations
between equivariant architectures and data augmentation for practical problems.
The equivariant spherical networks used in the experiments are available at
this https URL .

本研究分析了卷积神经网络（CNN）在球形图像分类和语义分割中旋转等价性的角色，并以 MNIST 和 FashionMNIST 数据集为基础，比较了 S2CNNs 和标准非等变 CNNs 的性能与推理时间等因素的权衡考虑。研究发现，对于固有的旋转不变的任务，通过大幅增加数据扩增和网络规模，标准 CNNs 可以达到与等变网络至少同样的性能，而对于固有的等变任务，非等变网络始终无法达到等变网络的性能水平。

球面图像的等变性与增强对比

Equivariance versus Augmentation for Spherical Images

Convolutional neural networks (CNNs) constructed natively on the sphere have
been developed recently and shown to be highly effective for the analysis of
spherical data. While an efficient framework has been formulated, spherical
CNNs are nevertheless highly computationally demanding; typically they cannot
scale beyond spherical signals of thousands of pixels. We develop scattering
networks constructed natively on the sphere that provide a powerful
representational space for spherical data. Spherical scattering networks are
computationally scalable and exhibit rotational equivariance, while their
representational space is invariant to isometries and provides efficient and
stable signal representations. By integrating scattering networks as an
additional type of layer in the generalized spherical CNN framework, we show
how they can be leveraged to scale spherical CNNs to the high-resolution data
typical of many practical applications, with spherical signals of many tens of
megapixels and beyond.

本研究提出了一种基于球面构建的扩散网络来处理球面数据，该网络具有旋转等变性和计算可扩展性，并且可以与卷积神经网络（CNNs）相结合，使其可以适用于具有高分辨率球面信号的实际应用中。

用于可扩展和旋转等变球形 CNN 的球面散射网络

Scattering Networks on the Sphere for Scalable and Rotationally  Equivariant Spherical CNNs

We propose contrastive coding to learn shared, dense image representations,
referred to as CoMIRs (Contrastive Multimodal Image Representations). CoMIRs
enable the registration of multimodal images where existing registration
methods often fail due to a lack of sufficiently similar image structures.
CoMIRs reduce the multimodal registration problem to a monomodal one, in which
general intensity-based, as well as feature-based, registration algorithms can
be applied. The method involves training one neural network per modality on
aligned images, using a contrastive loss based on noise-contrastive estimation
(InfoNCE). Unlike other contrastive coding methods, used for, e.g.,
classification, our approach generates image-like representations that contain
the information shared between modalities. We introduce a novel,
hyperparameter-free modification to InfoNCE, to enforce rotational equivariance
of the learnt representations, a property essential to the registration task.
We assess the extent of achieved rotational equivariance and the stability of
the representations with respect to weight initialization, training set, and
hyperparameter settings, on a remote sensing dataset of RGB and near-infrared
images. We evaluate the learnt representations through registration of a
biomedical dataset of bright-field and second-harmonic generation microscopy
images; two modalities with very little apparent correlation. The proposed
approach based on CoMIRs significantly outperforms registration of
representations created by GAN-based image-to-image translation, as well as a
state-of-the-art, application-specific method which takes additional knowledge
about the data into account. Code is available at:
this https URL

提出了对比编码方法来学习共享的、稠密的图像表示，称为 CoMIRs (Contrastive Multimodal Image Representations)，通过在对齐的图像上使用基于噪声对比估计的对比损失训练每种模态的一个神经网络，使用超参数 - free InfoNCE 修改确保所学表示的旋转等变性，实现了多模式图像注册，超越了现有的注册方法。