Accurate dietary intake estimation is critical for informing policies and programs to support healthy eating, as malnutrition has been directly linked to decreased quality of life. However self-reporting methods such as food diaries suffer from substantial bias. Other conventional dietary assessment techniques and emerging alternative approaches such as mobile applications incur high time costs and may necessitate trained personnel. Recent work has focused on using computer vision and machine learning to automatically estimate dietary intake from food images, but the lack of comprehensive datasets with diverse viewpoints, modalities and food annotations hinders the accuracy and realism of such methods. To address this limitation, we introduce NutritionVerse-Synth, the first large-scale dataset of 84,984 photorealistic synthetic 2D food images with associated dietary information and multimodal annotations (including depth images, instance masks, and semantic masks). Additionally, we collect a real image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to evaluate realism. Leveraging these novel datasets, we develop and benchmark NutritionVerse, an empirical study of various dietary intake estimation approaches, including indirect segmentation-based and direct prediction networks. We further fine-tune models pretrained on synthetic data with real images to provide insights into the fusion of synthetic and real data. Finally, we release both datasets (NutritionVerse-Synth, NutritionVerse-Real) on https://www.kaggle.com/nutritionverse/datasets as part of an open initiative to accelerate machine learning for dietary sensing.

准确的膳食摄入估计对于支持健康饮食的政策和计划至关重要，然而，自我报告方法存在严重的偏倚。为了解决这个问题，本文介绍了NutritionVerse-Synth，这是第一个大规模的数据集，包含84,984张真实感合成的2D食物图像，以及相关的膳食信息和多模态注释。通过利用这些新颖的数据集，开发和评估了NutritionVerse，包括间接基于分割和直接预测网络的各种膳食摄入估计方法。最后，发布了这两个数据集（NutritionVerse-Synth、NutritionVerse-Real），作为加速机器学习在膳食感知方面的开放倡议的一部分。

营养宇宙: 不同饮食摄入估计方法的实证研究