Learning effective recipe representations is essential in food studies.
Unlike what has been developed for image-based recipe retrieval or learning
structural text embeddings, the combined effect of multi-modal information
(i.e., recipe images, text, and relation data) receives less attention. In this
paper, we formalize the problem of multi-modal recipe representation learning
to integrate the visual, textual, and relational information into recipe
embeddings. In particular, we first present Large-RG, a new recipe graph data
with over half a million nodes, making it the largest recipe graph to date. We
then propose Recipe2Vec, a novel graph neural network based recipe embedding
model to capture multi-modal information. Additionally, we introduce an
adversarial attack strategy to ensure stable learning and improve performance.
Finally, we design a joint objective function of node classification and
adversarial learning to optimize the model. Extensive experiments demonstrate
that Recipe2Vec outperforms state-of-the-art baselines on two classic food
study tasks, i.e., cuisine category classification and region prediction.
Dataset and codes are available at this https URL.

本文讨论了将视觉，文本和关系信息集成到食谱嵌入中的问题，并介绍了一种基于图神经网络（GNN）的全新食谱嵌入模型 Recipe2Vec。通过对大量实验的验证，证明该模型在两项食品学研究任务中表现出色 - 风味分类和地区预测。

Recipe2Vec: 基于图神经网络的多模态食谱表示学习

Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks

Food is not only a basic human necessity but also a key factor driving a
society's health and economic well-being. As a result, the cooking domain is a
popular use-case to demonstrate decision-support (AI) capabilities in service
of benefits like precision health with tools ranging from information retrieval
interfaces to task-oriented chatbots. An AI here should understand concepts in
the food domain (e.g., recipes, ingredients), be tolerant to failures
encountered while cooking (e.g., browning of butter), handle allergy-based
substitutions, and work with multiple data modalities (e.g. text and images).
However, the recipes today are handled as textual documents which makes it
difficult for machines to read, reason and handle ambiguity. This demands a
need for better representation of the recipes, overcoming the ambiguity and
sparseness that exists in the current textual documents. In this paper, we
discuss the construction of a machine-understandable rich recipe representation
(R3), in the form of plans, from the recipes available in natural language. R3
is infused with additional knowledge such as information about allergens and
images of ingredients, possible failures and tips for each atomic cooking step.
To show the benefits of R3, we also present TREAT, a tool for recipe retrieval
which uses R3 to perform multi-modal reasoning on the recipe's content (plan
objects - ingredients and cooking tools), food preparation process (plan
actions and time), and media type (image, text). R3 leads to improved retrieval
efficiency and new capabilities that were hither-to not possible in textual
representation.

本文探讨在食品领域运用 AI 进行决策支持的构建，需要更好地表达和处理食谱信息，为此构建了一个基于计划的可机器理解食谱表示和 TREAT，这种方法可以提高智能化制作食品效率。