Creating 3D textured meshes using generative artificial intelligence has
garnered significant attention recently. While existing methods support
text-based generative texture generation or editing on 3D meshes, they often
struggle to precisely control pixels of texture images through more intuitive
interaction. While 2D images can be edited generatively using drag interaction,
applying this type of methods directly to 3D mesh textures still leads to
issues such as the lack of local consistency among multiple views, error
accumulation and long training times. To address these challenges, we propose a
generative point-based 3D mesh texture editing method called DragTex. This
method utilizes a diffusion model to blend locally inconsistent textures in the
region near the deformed silhouette between different views, enabling locally
consistent texture editing. Besides, we fine-tune a decoder to reduce
reconstruction errors in the non-drag region, thereby mitigating overall error
accumulation. Moreover, we train LoRA using multi-view images instead of
training each view individually, which significantly shortens the training
time. The experimental results show that our method effectively achieves
dragging textures on 3D meshes and generates plausible textures that align with
the desired intent of drag interaction.

通过使用扩展人工智能技术创建三维纹理网格近期引起了重要关注。我们提出了一种名为 DragTex 的生成点型三维网格纹理编辑方法，该方法利用扩散模型在不同视角之间变形剪影附近的区域中混合局部不一致的纹理，实现了局部一致的纹理编辑。此外，我们通过微调解码器来减少非拖动区域的重构错误，从而减轻了总体的错误积累。此外，我们使用多视图图像训练 LoRA，而不是对每个视图进行单独训练，从而显著缩短了训练时间。实验结果表明，我们的方法可有效地在三维网格上拖动纹理，并生成与拖动交互的预期意图相一致的合理纹理。