Many aging individuals encounter challenges in effectively tracking their
dietary intake, exacerbating their susceptibility to nutrition-related health
complications. Self-reporting methods are often inaccurate and suffer from
substantial bias; however, leveraging intelligent prediction methods can
automate and enhance precision in this process. Recent work has explored using
computer vision prediction systems to predict nutritional information from food
images. Still, these methods are often tailored to specific situations, require
other inputs in addition to a food image, or do not provide comprehensive
nutritional information.
This paper aims to enhance the efficacy of dietary intake estimation by
leveraging various neural network architectures to directly predict a meal's
nutritional content from its image. Through comprehensive experimentation and
evaluation, we present NutritionVerse-Direct, a model utilizing a vision
transformer base architecture with three fully connected layers that lead to
five regression heads predicting calories (kcal), mass (g), protein (g), fat
(g), and carbohydrates (g) present in a meal. NutritionVerse-Direct yields a
combined mean average error score on the NutritionVerse-Real dataset of 412.6,
an improvement of 25.5% over the Inception-ResNet model, demonstrating its
potential for improving dietary intake estimation accuracy.

该论文通过利用各种神经网络结构来预测餐食图像的营养含量，提高了膳食摄入估计的效力，并展示了其在改善膳食摄入估计准确性方面的潜力。

利用深度神经网络从食物图像中进行多任务营养预测的研究

NutritionVerse-Direct: Exploring Deep Neural Networks for Multitask  Nutrition Prediction from Food Images

Dietary intake estimation plays a crucial role in understanding the
nutritional habits of individuals and populations, aiding in the prevention and
management of diet-related health issues. Accurate estimation requires
comprehensive datasets of food scenes, including images, segmentation masks,
and accompanying dietary intake metadata. In this paper, we introduce
NutritionVerse-Real, an open access manually collected 2D food scene dataset
for dietary intake estimation with 889 images of 251 distinct dishes and 45
unique food types. The NutritionVerse-Real dataset was created by manually
collecting images of food scenes in real life, measuring the weight of every
ingredient and computing the associated dietary content of each dish using the
ingredient weights and nutritional information from the food packaging or the
Canada Nutrient File. Segmentation masks were then generated through human
labelling of the images. We provide further analysis on the data diversity to
highlight potential biases when using this data to develop models for dietary
intake estimation. NutritionVerse-Real is publicly available at
this https URL as part of
an open initiative to accelerate machine learning for dietary sensing.

营养素摄入估计的重要性和面向机器学习的营养感知的开放倡议在这篇论文中介绍了 NutritionVerse-Real 数据集的创建、分析和可用性。

NutritionVerse-Real：一项用于饮食摄入估计的开放获取的手工精选 2D 食物场景数据集

NutritionVerse-Real: An Open Access Manually Collected 2D Food Scene  Dataset for Dietary Intake Estimation

Accurate dietary intake estimation is critical for informing policies and
programs to support healthy eating, as malnutrition has been directly linked to
decreased quality of life. However self-reporting methods such as food diaries
suffer from substantial bias. Other conventional dietary assessment techniques
and emerging alternative approaches such as mobile applications incur high time
costs and may necessitate trained personnel. Recent work has focused on using
computer vision and machine learning to automatically estimate dietary intake
from food images, but the lack of comprehensive datasets with diverse
viewpoints, modalities and food annotations hinders the accuracy and realism of
such methods. To address this limitation, we introduce NutritionVerse-Synth,
the first large-scale dataset of 84,984 photorealistic synthetic 2D food images
with associated dietary information and multimodal annotations (including depth
images, instance masks, and semantic masks). Additionally, we collect a real
image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to
evaluate realism. Leveraging these novel datasets, we develop and benchmark
NutritionVerse, an empirical study of various dietary intake estimation
approaches, including indirect segmentation-based and direct prediction
networks. We further fine-tune models pretrained on synthetic data with real
images to provide insights into the fusion of synthetic and real data. Finally,
we release both datasets (NutritionVerse-Synth, NutritionVerse-Real) on
this https URL as part of an open initiative to
accelerate machine learning for dietary sensing.

准确的膳食摄入估计对于支持健康饮食的政策和计划至关重要，然而，自我报告方法存在严重的偏倚。为了解决这个问题，本文介绍了 NutritionVerse-Synth，这是第一个大规模的数据集，包含 84,984 张真实感合成的 2D 食物图像，以及相关的膳食信息和多模态注释。通过利用这些新颖的数据集，开发和评估了 NutritionVerse，包括间接基于分割和直接预测网络的各种膳食摄入估计方法。最后，发布了这两个数据集（NutritionVerse-Synth、NutritionVerse-Real），作为加速机器学习在膳食感知方面的开放倡议的一部分。