Modeling and synthesizing textures are essential for enhancing the realism of
virtual environments. Methods that directly synthesize textures in 3D offer
distinct advantages to the UV-mapping-based methods as they can create seamless
textures and align more closely with the ways textures form in nature. We
propose Mesh Neural Cellular Automata (MeshNCA), a method for directly
synthesizing dynamic textures on 3D meshes without requiring any UV maps.
MeshNCA is a generalized type of cellular automata that can operate on a set of
cells arranged on a non-grid structure such as vertices of a 3D mesh. While
only being trained on an Icosphere mesh, MeshNCA shows remarkable
generalization and can synthesize textures on any mesh in real time after the
training. Additionally, it accommodates multi-modal supervision and can be
trained using different targets such as images, text prompts, and motion vector
fields. Moreover, we conceptualize a way of grafting trained MeshNCA instances,
enabling texture interpolation. Our MeshNCA model enables real-time 3D texture
synthesis on meshes and allows several user interactions including texture
density/orientation control, a grafting brush, and motion speed/direction
control. Finally, we implement the forward pass of our MeshNCA model using the
WebGL shading language and showcase our trained models in an online interactive
demo which is accessible on personal computers and smartphones. Our demo and
the high resolution version of this PDF are available at
this https URL

通过 Mesh Neural Cellular Automata (MeshNCA) 方法，在 3D 网格上直接合成动态纹理，无需任何 UV 贴图，实现了实时的 3D 纹理合成。