Neural operations that rely on neighborhood information are much more
expensive when deployed on point clouds than on grid data due to the irregular
distances between points in a point cloud. In a grid, on the other hand, we can
compute the kernel only once and reuse it for all query positions. As a result,
operations that rely on neighborhood information scale much worse for point
clouds than for grid data, specially for large inputs and large neighborhoods.
In this work, we address the scalability issue of point cloud methods by
tackling its root cause: the irregularity of the data. We propose learnable
gridification as the first step in a point cloud processing pipeline to
transform the point cloud into a compact, regular grid. Thanks to
gridification, subsequent layers can use operations defined on regular grids,
e.g., Conv3D, which scale much better than native point cloud methods. We then
extend gridification to point cloud to point cloud tasks, e.g., segmentation,
by adding a learnable de-gridification step at the end of the point cloud
processing pipeline to map the compact, regular grid back to its original point
cloud form. Through theoretical and empirical analysis, we show that gridified
networks scale better in terms of memory and time than networks directly
applied on raw point cloud data, while being able to achieve competitive
results. Our code is publicly available at
this https URL

通过可学习的网格化处理将点云数据转化为紧凑、规则的网格，解决了点云方法在规模可扩展性上的问题，使得后续层能够使用更好扩展的操作，在时间和内存消耗上表现更佳且能获得有竞争力的结果。

高效点云处理的学习网格化

Learned Gridification for Efficient Point Cloud Processing

Invariance and equivariance to the rotation group have been widely discussed
in the 3D deep learning community for pointclouds. Yet most proposed methods
either use complex mathematical tools that may limit their accessibility, or
are tied to specific input data types and network architectures. In this paper,
we introduce a general framework built on top of what we call Vector Neuron
representations for creating SO(3)-equivariant neural networks for pointcloud
processing. Extending neurons from 1D scalars to 3D vectors, our vector neurons
enable a simple mapping of SO(3) actions to latent spaces thereby providing a
framework for building equivariance in common neural operations -- including
linear layers, non-linearities, pooling, and normalizations. Due to their
simplicity, vector neurons are versatile and, as we demonstrate, can be
incorporated into diverse network architecture backbones, allowing them to
process geometry inputs in arbitrary poses. Despite its simplicity, our method
performs comparably well in accuracy and generalization with other more complex
and specialized state-of-the-art methods on classification and segmentation
tasks. We also show for the first time a rotation equivariant reconstruction
network.

本文介绍了一种基于向量神经元的通用框架，用于创建点云处理的 SO（3）- 等变神经网络。通过扩展神经元从 1D 标量到 3D 向量，向量神经元使得我们可以将 SO（3）操作简单地映射到潜在空间中，从而提供了构建常见神经操作的等变性的框架，尽管方法简单，但在准确性和泛化性方面表现出色，是处理任意姿态下的几何输入的一种有效方法。

矢量神经元：一个针对 SO (3)- 等变网络的通用框架

Vector Neurons: A General Framework for SO(3)-Equivariant Networks

Neural operations as convolutions, self-attention, and vector aggregation are
the go-to choices for recognizing short-range actions. However, they have three
limitations in modeling long-range activities. This paper presents PIC,
Permutation Invariant Convolution, a novel neural layer to model the temporal
structure of long-range activities. It has three desirable properties. i.
Unlike standard convolution, PIC is invariant to the temporal permutations of
features within its receptive field, qualifying it to model the weak temporal
structures. ii. Different from vector aggregation, PIC respects local
connectivity, enabling it to learn long-range temporal abstractions using
cascaded layers. iii. In contrast to self-attention, PIC uses shared weights,
making it more capable of detecting the most discriminant visual evidence
across long and noisy videos. We study the three properties of PIC and
demonstrate its effectiveness in recognizing the long-range activities of
Charades, Breakfast, and MultiThumos.

介绍了一种新的神经网络层 ——Permutation Invariant Convolution（PIC），它具有三种可取的属性：在其感受野内部特征的时间排列不变，各个区域之间存在本地连接，使用共享权重，这使得它更能够检测噪声较大的视频中的最有鉴别性的视觉证据，并且该神经网络层能够有效地识别 Charades，Breakfast 和 MultiThumos 等长时间的活动。