Point cloud processing methods leverage local and global point features %at
the feature level to cater to downstream tasks, yet they often overlook the
task-level context inherent in point clouds during the encoding stage. We argue
that integrating task-level information into the encoding stage significantly
enhances performance. To that end, we propose SMTransformer which incorporates
task-level information into a vector-based transformer by utilizing a soft mask
generated from task-level queries and keys to learn the attention weights.
Additionally, to facilitate effective communication between features from the
encoding and decoding layers in high-level tasks such as segmentation, we
introduce a skip-attention-based up-sampling block. This block dynamically
fuses features from various resolution points across the encoding and decoding
layers. To mitigate the increase in network parameters and training time
resulting from the complexity of the aforementioned blocks, we propose a novel
shared position encoding strategy. This strategy allows various transformer
blocks to share the same position information over the same resolution points,
thereby reducing network parameters and training time without compromising
accuracy.Experimental comparisons with existing methods on multiple datasets
demonstrate the efficacy of SMTransformer and skip-attention-based up-sampling
for point cloud processing tasks, including semantic segmentation and
classification. In particular, we achieve state-of-the-art semantic
segmentation results of 73.4% mIoU on S3DIS Area 5 and 62.4% mIoU on SWAN
dataset

将任务级信息整合到编码阶段，采用软掩码并使用任务级查询和关键字生成的注意力权重学习来增强 SMTransformer 的性能，同时通过跳跃注意力上采样块动态融合来自编码和解码层的各种分辨率点的特征，以在点云处理任务中实现最先进的语义分割结果。