Image-based methods to analyze food images have alleviated the user burden and biases associated with traditional methods. However, accurate portion estimation remains a major challenge due to the loss of 3D information in the 2D representation of foods captured by smartphone cameras or wearable devices. In this paper, we propose a new framework to estimate both food volume and energy from 2D images by leveraging the power of 3D food models and physical reference in the eating scene. Our method estimates the pose of the camera and the food object in the input image and recreates the eating occasion by rendering an image of a 3D model of the food with the estimated poses. We also introduce a new dataset, SimpleFood45, which contains 2D images of 45 food items and associated annotations including food volume, weight, and energy. Our method achieves an average error of 31.10 kCal (17.67%) on this dataset, outperforming existing portion estimation methods.

本研究提出了一种新的框架，通过利用3D食物模型和场景中的物理参考，从2D图像中估计食物的容量和能量。该方法通过重现具有估计姿态的食物的3D模型图像，来估计输入图像中摄像头和食物物体的姿态。我们还介绍了一个新的数据集，SimpleFood45，其中包含45种食物的2D图像和相应的注释，包括食物的容量、重量和能量。我们的方法在这个数据集上的平均误差为31.10千卡（17.67%），表现优于现有的分量估计方法。

通过三维物体缩放进行食物容量估计