In this work, we present a new visual prompting method called 3DAxiesPrompts (3DAP) to unleash the capabilities of GPT-4V in performing 3D spatial tasks. Our investigation reveals that while GPT-4V exhibits proficiency in discerning the position and interrelations of 2D entities through current visual prompting techniques, its abilities in handling 3D spatial tasks have yet to be explored. In our approach, we create a 3D coordinate system tailored to 3D imagery, complete with annotated scale information. By presenting images infused with the 3DAP visual prompt as inputs, we empower GPT-4V to ascertain the spatial positioning information of the given 3D target image with a high degree of precision. Through experiments, We identified three tasks that could be stably completed using the 3DAP method, namely, 2D to 3D Point Reconstruction, 2D to 3D point matching, and 3D Object Detection. We perform experiments on our proposed dataset 3DAP-Data, the results from these experiments validate the efficacy of 3DAP-enhanced GPT-4V inputs, marking a significant stride in 3D spatial task execution.

通过使用一种名为3DAxiesPrompts（3DAP）的新型视觉提示方法，我们解放了GPT-4V在执行3D空间任务方面的能力。我们的研究发现，尽管GPT-4V在通过现有的视觉提示技术识别2D实体的位置和相互关系方面表现出熟练度，但其在处理3D空间任务方面的能力尚待探索。我们的方法是创建一个适用于3D图像的3D坐标系统，并带有注释的比例信息。通过将注入了3DAP视觉提示的图像作为输入呈现，我们使GPT-4V能够以高度精确性确定给定3D目标图像的空间位置信息。通过实验证明了使用3DAP方法可以稳定完成的三个任务，即2D到3D点重建、2D到3D点匹配和3D物体检测。我们在我们提出的数据集3DAP-Data上进行了实验证明了3DAP增强的GPT-4V输入的有效性，标志着3D空间任务执行方面的重要进展。

3DAxiesPrompts：释放GPT-4V的三维空间任务能力