Recent conditional 3D completion works have mainly relied on CLIP or BERT to encode textual information, which cannot support complex instruction. Meanwhile, large language models (LLMs) have shown great potential in multi-modal understanding and generation tasks. Inspired by the recent advancements of LLM, we present Volume Patch LLM (VP-LLM), which leverages LLMs to perform conditional 3D completion in a single-forward pass. To integrate a 3D model into the LLM tokenization configuration, the incomplete 3D object is first divided into small patches that can be encoded independently. These encoded patches are then fed into an LLM along with the text prompt, instructing the LLM to capture the relations between these patches as well as injecting semantic meanings into the 3D object. Our results demonstrate a strong ability of LLMs to interpret complex text instructions and understand 3D objects, surpassing state-of-the-art diffusion-based 3D completion models in generation quality.

借助大型语言模型，在单个前向传递中，我们提出了一种Volume Patch LLM (VP-LLM)方法，用于对3D模型进行条件完成。我们的结果表明，LLMs在理解复杂的文本指令和3D物体方面具有很强的能力，并且在生成质量方面超过了最先进的基于扩散的3D完成模型。

VP-LLM: 基于大型语言模型的文本驱动3D体积补全通过贴片处理