Recent advances have demonstrated that Language Vision Models (LVMs) surpass
the existing State-of-the-Art (SOTA) in two-dimensional (2D) computer vision
tasks, motivating attempts to apply LVMs to three-dimensional (3D) data. While
LVMs are efficient and effective in addressing various downstream 2D vision
tasks without training, they face significant challenges when it comes to point
clouds, a representative format for representing 3D data. It is more difficult
to extract features from 3D data and there are challenges due to large data
sizes and the cost of the collection and labelling, resulting in a notably
limited availability of datasets. Moreover, constructing LVMs for point clouds
is even more challenging due to the requirements for large amounts of data and
training time. To address these issues, our research aims to 1) apply the
Grounded SAM through Spherical Projection to transfer 3D to 2D, and 2)
experiment with synthetic data to evaluate its effectiveness in bridging the
gap between synthetic and real-world data domains. Our approach exhibited high
performance with an accuracy of 0.96, an IoU of 0.85, precision of 0.92, recall
of 0.91, and an F1 score of 0.92, confirming its potential. However, challenges
such as occlusion problems and pixel-level overlaps of multi-label points
during spherical image generation remain to be addressed in future studies.

最近的研究进展表明，语言视觉模型在二维计算机视觉任务方面已经超过了现有的最先进技术 (SOTA)，激发了将语言视觉模型应用于三维数据的尝试。然而，应用于点云的 LVM 面临着从三维数据提取特征的困难以及由于数据量大、采集和标注成本高导致数据集的可用性明显有限等挑战。为了解决这些问题，我们的研究旨在通过球面投影将 3D 数据转化为 2D，并尝试使用合成数据评估其在合成和真实数据之间弥合差距方面的效果。我们的方法表现出了高性能，准确率为 0.96，IoU 为 0.85，精确率为 0.92，召回率为 0.91，F1 分数为 0.92，验证了其潜力。然而，在未来的研究中仍需要解决遮挡问题和球面图像生成过程中多标签点的像素级重叠等挑战。