Textures in natural images can be characterized by color, shape, periodicity of elements within them, and other attributes that can be described using natural language. In this paper, we study the problem of describing visual attributes of texture on a novel dataset containing rich descriptions of textures, and conduct a systematic study of current generative and discriminative models for grounding language to images on this dataset. We find that while these models capture some properties of texture, they fail to capture several compositional properties, such as the colors of dots. We provide critical analysis of existing models by generating synthetic but realistic textures with different descriptions. Our dataset also allows us to train interpretable models and generate language-based explanations of what discriminative features are learned by deep networks for fine-grained categorization where texture plays a key role. We present visualizations of several fine-grained domains and show that texture attributes learned on our dataset offer improvements over expert-designed attributes on the Caltech-UCSD Birds dataset.

本文研究了纹理的视觉属性及对不同模型的影响，发现现有的生成和判别模型不能很好地捕捉纹理的组成属性。我们以证据支持了此观点，并提供了可解释性的模型来生成基于语言的解释，以加深对纹理分类的理解。我们通过在Caltech-UCSD鸟类数据集上进行精细分类，证明了该数据集上所学习到的纹理属性相比专家设计的属性能够实现显著改进。

使用自然语言描述纹理