People typically learn through exposure to visual concepts associated with linguistic descriptions. For instance, teaching visual object categories to children is often accompanied by descriptions in text or speech. In a machine learning context, these observations motivates us to ask whether this learning process could be computationally modeled to learn visual classifiers. More specifically, the main question of this work is how to utilize purely textual description of visual classes with no training images, to learn explicit visual classifiers for them. We propose and investigate two baseline formulations, based on regression and domain transfer that predict a linear classifier. Then, we propose a new constrained optimization formulation that combines a regression function and a knowledge transfer function with additional constraints to predict the linear classifier parameters for new classes. We also propose a generic kernelized models where a kernel classifier, in the form defined by the representer theorem, is predicted. The kernelized models allow defining any two RKHS kernel functions in the visual space and text space, respectively, and could be useful for other applications. We finally propose a kernel function between unstructured text descriptions that builds on distributional semantics, which shows an advantage in our setting and could be useful for other applications. We applied all the studied models to predict visual classifiers for two fine-grained categorization datasets, and the results indicate successful predictions of our final model against several baselines that we designed.

本篇研究提出并探究了四种基于回归和领域转移算法的方法以及在分布语义下基于内核函数的文本描述预测可视化分级任务，并在CU Birds和 Flower数据集上成功预测了可视化分类器。

编写分类器：从非结构化文本预测视觉分类器