We introduce the novel task of Language-Guided Object Placement in Real 3D Scenes. Our model is given a 3D scene's point cloud, a 3D asset, and a textual prompt broadly describing where the 3D asset should be placed. The task here is to find a valid placement for the 3D asset that respects the prompt. Compared with other language-guided localization tasks in 3D scenes such as grounding, this task has specific challenges: it is ambiguous because it has multiple valid solutions, and it requires reasoning about 3D geometric relationships and free space. We inaugurate this task by proposing a new benchmark and evaluation protocol. We also introduce a new dataset for training 3D LLMs on this task, as well as the first method to serve as a non-trivial baseline. We believe that this challenging task and our new benchmark could become part of the suite of benchmarks used to evaluate and compare generalist 3D LLM models.

本研究提出了实时3D场景中语言指导物体放置的新任务，填补了现有3D场景定位任务的空白。创新地建立了基准和评估协议，并提供了用于训练3D大语言模型的数据集，首次提出一个非平凡的基线方法。该任务和新的基准有望成为评估和比较通用3D大语言模型的重要工具。

PlaceIt3D：基于语言的真实3D场景物体放置