Geometric deep learning is increasingly important thanks to the popularity of
3D sensors. Inspired by the recent advances in NLP domain, the self-attention
transformer is introduced to consume the point clouds. We develop Point
Attention Transformers (PATs), using a parameter-efficient Group Shuffle
Attention (GSA) to replace the costly Multi-Head Attention. We demonstrate its
ability to process size-varying inputs, and prove its permutation equivariance.
Besides, prior work uses heuristics dependence on the input data (e.g.,
Furthest Point Sampling) to hierarchically select subsets of input points.
Thereby, we for the first time propose an end-to-end learnable and
task-agnostic sampling operation, named Gumbel Subset Sampling (GSS), to select
a representative subset of input points. Equipped with Gumbel-Softmax, it
produces a "soft" continuous subset in training phase, and a "hard" discrete
subset in test phase. By selecting representative subsets in a hierarchical
fashion, the networks learn a stronger representation of the input sets with
lower computation cost. Experiments on classification and segmentation
benchmarks show the effectiveness and efficiency of our methods. Furthermore,
we propose a novel application, to process event camera stream as point clouds,
and achieve a state-of-the-art performance on DVS128 Gesture Dataset.

本研究提出了一种基于自注意力变换器和 Gumbel 子集采样的特征提取方法，可用于处理大小变化的输入，并能够用于处理事件相机流作为点云的新领域。