Human language users can generate descriptions of perceptual concepts beyond instance-level representations and also use such descriptions to learn provisional class-level representations. However, the ability of computational models to learn and operate with class representations is under-investigated in the language-and-vision field. In this paper, we train separate neural networks to generate and interpret class-level descriptions. We then use the zero-shot classification performance of the interpretation model as a measure of communicative success and class-level conceptual grounding. We investigate the performance of prototype- and exemplar-based neural representations grounded category description. Finally, we show that communicative success reveals performance issues in the generation model that are not captured by traditional intrinsic NLG evaluation metrics, and argue that these issues can be traced to a failure to properly ground language in vision at the class level. We observe that the interpretation model performs better with descriptions that are low in diversity on the class level, possibly indicating a strong reliance on frequently occurring features.

本文探究了计算模型学习和运用类别表征的能力，使用零样本分类推断来度量交际成功和类别概念基础，并通过两种不同的神经表征（基于原型和范例）研究了类别描述的植入性。结果发现属性重复度高的描述更容易达到交际成功，而这也暴露了类别描述生成模型中存在的性能问题。

描述奥凯特：生成基于感知的类别描述