This paper explores various learning strategies for 3D building type
classification and part segmentation on the BuildingNet dataset. ULIP with
PointNeXt and PointNeXt segmentation are extended for the classification and
segmentation task on BuildingNet dataset. The best multi-task PointNeXt-s model
with multi-modal pretraining achieves 59.36 overall accuracy for 3D building
type classification, and 31.68 PartIoU for 3D building part segmentation on
validation split. The final PointNeXt XL model achieves 31.33 PartIoU and 22.78
ShapeIoU on test split for BuildingNet-Points segmentation, which significantly
improved over PointNet++ model reported from BuildingNet paper, and it won the
1st place in the BuildingNet challenge at CVPR23 StruCo3D workshop.

本文旨在探究基于 BuildingNet 数据集的 3D 建筑类型分类和部件分割的多种学习策略，提出了使用扩展的 ULIP、PointNeXt 和 PointNeXt 分割方法进行分类和分割任务的解决方案。在多模态预训练的前提下，最佳的多任务 PointNeXt-s 模型在验证集上达到了 59.36% 的准确率和 31.68 的 PartIoU，PointNeXt XL 模型在 BuildingNet-Points 分割测试集上达到了 31.33 的 PartIoU 和 22.78 的 ShapeIoU，显著提高了建筑网络论文中所述的 PointNet ++ 模型的性能，并在 CVPR23 StruCo3D 工作室的 BuildingNet 挑战中获得了第一名。

多模态预训练下的多任务 3D 建筑理解

Multi-task 3D building understanding with multi-modal pretraining

In point cloud analysis, point-based methods have rapidly developed in recent
years. These methods have recently focused on concise MLP structures, such as
PointNeXt, which have demonstrated competitiveness with Convolutional and
Transformer structures. However, standard MLPs are limited in their ability to
extract local features effectively. To address this limitation, we propose a
Vector-oriented Point Set Abstraction that can aggregate neighboring features
through higher-dimensional vectors. To facilitate network optimization, we
construct a transformation from scalar to vector using independent angles based
on 3D vector rotations. Finally, we develop a PointVector model that follows
the structure of PointNeXt. Our experimental results demonstrate that
PointVector achieves state-of-the-art performance $\textbf{72.3\% mIOU}$ on the
S3DIS Area 5 and $\textbf{78.4\% mIOU}$ on the S3DIS (6-fold cross-validation)
with only $\textbf{58\%}$ model parameters of PointNeXt. We hope our work will
help the exploration of concise and effective feature representations. The code
will be released soon.

本文探讨了基于点云的方法在点云分析中的应用，提出了一种聚合邻近特征的高维向量的 “Vector-oriented Point Set Abstraction” 方法，并开发了一种 PointNeXt 结构的 PointVector 模型以提高特征提取的效率，此方法在 S3DIS Area 5 上和 S3DIS (6 折交叉验证) 上取得了 72.3% 和 78.4% 的 mIOU 的优异表现，在研究中探索了更加紧凑和有效的特征表示方法。