Transformers have gained much attention by outperforming convolutional neural
networks in many 2D vision tasks. However, they are known to have
generalization problems and rely on massive-scale pre-training and
sophisticated training techniques. When applying to 3D tasks, the irregular
data structure and limited data scale add to the difficulty of transformer's
application. We propose CodedVTR (Codebook-based Voxel TRansformer), which
improves data efficiency and generalization ability for 3D sparse voxel
transformers. On the one hand, we propose the codebook-based attention that
projects an attention space into its subspace represented by the combination of
"prototypes" in a learnable codebook. It regularizes attention learning and
improves generalization. On the other hand, we propose geometry-aware
self-attention that utilizes geometric information (geometric pattern, density)
to guide attention learning. CodedVTR could be embedded into existing sparse
convolution-based methods, and bring consistent performance improvements for
indoor and outdoor 3D semantic segmentation tasks

提出基于码本的体素 Transformer（CodedVTR）以提高 3D 稀疏体素 Transformer 的数据效率和泛化能力，其中包括通过学习可学习码本中的原型来规范化注意力学习和改善泛化能力，并利用几何信息（几何模式、密度）指导注意力学习。CodedVTR 可嵌入现有的基于稀疏卷积的方法，并为室内和室外 3D 语义分割任务带来一致的性能提升。

CodedVTR：基于码本的稀疏体素变换器以及几何引导

CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric  Guidance

Facial action unit (AU) detection and face alignment are two highly
correlated tasks, since facial landmarks can provide precise AU locations to
facilitate the extraction of meaningful local features for AU detection.
However, most existing AU detection works handle the two tasks independently by
treating face alignment as a preprocessing, and often use landmarks to
predefine a fixed region or attention for each AU. In this paper, we propose a
novel end-to-end deep learning framework for joint AU detection and face
alignment, which has not been explored before. In particular, multi-scale
shared feature is learned firstly, and high-level feature of face alignment is
fed into AU detection. Moreover, to extract precise local features, we propose
an adaptive attention learning module to refine the attention map of each AU
adaptively. Finally, the assembled local features are integrated with face
alignment feature and global feature for AU detection. Extensive experiments
demonstrate that our framework (i) significantly outperforms the
state-of-the-art AU detection methods on the challenging BP4D, DISFA, GFT and
BP4D+ benchmarks, (ii) can adaptively capture the irregular region of each AU,
(iii) achieves competitive performance for face alignment, and (iv) also works
well under partial occlusions and non-frontal poses. The code for our method is
available at this https URL

本文提出了一种用于联合面部动作单元检测和面部对齐的深度学习框架，通过学习多尺度共享特征，将面部对齐特征和全局特征与调整后的局部特征相结合，采用自适应注意力学习模块来调整 AU 的注意力图，实现了更好的 AU 检测表现，可适应不规则的 AU 区域，有望在面部识别领域有更广泛的应用。