Learning to generate textures for a novel 3D mesh given a collection of 3D
meshes and real-world 2D images is an important problem with applications in
various domains such as 3D simulation, augmented and virtual reality, gaming,
architecture, and design. Existing solutions either do not produce high-quality
textures or deform the original high-resolution input mesh topology into a
regular grid to make this generation easier but also lose the original mesh
topology. In this paper, we present a novel framework called the
3DTextureTransformer that enables us to generate high-quality textures without
deforming the original, high-resolution input mesh. Our solution, a hybrid of
geometric deep learning and StyleGAN-like architecture, is flexible enough to
work on arbitrary mesh topologies and also easily extensible to texture
generation for point cloud representations. Our solution employs a
message-passing framework in 3D in conjunction with a StyleGAN-like
architecture for 3D texture generation. The architecture achieves
state-of-the-art performance among a class of solutions that can learn from a
collection of 3D geometry and real-world 2D images while working with any
arbitrary mesh topology.

使用名为 3DTextureTransformer 的新颖框架，结合几何深度学习和 StyleGAN 类似的架构，能够在不改变原始高分辨率输入网格拓扑的情况下生成高质量的纹理。该架构在学习来自 3D 几何体和现实世界 2D 图像的情况下，与任意网格拓扑一起工作，取得了该类解决方案中的最新性能。