Despite the abundance of multi-modal data, such as image-text pairs, there
has been little effort in understanding the individual entities and their
different roles in the construction of these data instances. In this work, we
endeavour to discover the entities and their corresponding importance in
cooking recipes automaticall} as a visual-linguistic association problem. More
specifically, we introduce a novel cross-modal learning framework to jointly
model the latent representations of images and text in the food image-recipe
association and retrieval tasks. This model allows one to discover complex
functional and hierarchical relationships between images and text, and among
textual parts of a recipe including title, ingredients and cooking
instructions. Our experiments show that by making use of efficient
tree-structured Long Short-Term Memory as the text encoder in our computational
cross-modal retrieval framework, we are not only able to identify the main
ingredients and cooking actions in the recipe descriptions without explicit
supervision, but we can also learn more meaningful feature representations of
food recipes, appropriate for challenging cross-modal retrieval and recipe
adaption tasks.

本文提出了一种新颖的跨模态学习框架来自动发现食谱中文本和图像的实体及其相应的重要性，并能在没有明确监督的情况下识别出食谱描述中的主要成分和烹饪动作，为跨模态检索和食谱适应任务提供更有意义的特征表示。

CHEF: 跨模态层次嵌入用于食品领域检索

CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval

Previous approaches to training syntax-based sentiment classification models
required phrase-level annotated corpora, which are not readily available in
many languages other than English. Thus, we propose the use of tree-structured
Long Short-Term Memory with an attention mechanism that pays attention to each
subtree of the parse tree. Experimental results indicate that our model
achieves the state-of-the-art performance in a Japanese sentiment
classification task.

使用树形 LSTM 和注意力机制，无需短语级标注语料库，在日本情感分类任务中达到了最先进的性能。