In this work, we propose a new technique that combines appearance and text in a Convolutional Neural Network (CNN), with the aim of detecting regions of different textual categories. We define a novel visual representation of the semantic meaning of text that allows a seamless integration in a standard CNN architecture. This representation, referred to as text-map, is integrated with the actual image to provide a much richer input to the network. Text-maps are colored with different intensities depending on the relevance of the words recognized over the image. Concretely, these words are previously extracted using Optical Character Recognition (OCR) and they are colored according to the probability of belonging to a textual category of interest. In this sense, this solution is especially relevant in the context of item coding for supermarket products, where different types of textual categories must be identified, such as ingredients or nutritional facts. We evaluated our solution in the proprietary item coding dataset of Nielsen Brandbank, which contains more than 10,000 images for train and 2,000 images for test. The reported results demonstrate that our approach focused on visual and textual data outperforms state-of-the-art algorithms only based on appearance, such as standard Faster R-CNN. These enhancements are reflected in precision and recall, which are improved in 42 and 33 points respectively.

提出了一种新技术，将外观和文本结合在卷积神经网络(CNN)中，以便检测不同文本类别的区域。该技术通过一种名为text-map的新的文本语义视觉表示与实际图像相集成，针对商品编码的文本类别如配料或营养成分的多种类型的标识具有优异性能，采用OCR识别技术为预处理。针对Nielsen Brandbank的自有数据集，实验结果表明，该方法优于仅基于外观的标准Faster R-CNN等最先进的算法，精确率和召回率均得到了显著提高。

卷积神经网络中的文本映射集成，用于不同文本类别的区域检测