Accurate food volume estimation is essential for dietary assessment,
nutritional tracking, and portion control applications. We present VolETA, a
sophisticated methodology for estimating food volume using 3D generative
techniques. Our approach creates a scaled 3D mesh of food objects using one- or
few-RGBD images. We start by selecting keyframes based on the RGB images and
then segmenting the reference object in the RGB images using XMem++.
Simultaneously, camera positions are estimated and refined using the PixSfM
technique. The segmented food images, reference objects, and camera poses are
combined to form a data model suitable for NeuS2. Independent mesh
reconstructions for reference and food objects are carried out, with scaling
factors determined using MeshLab based on the reference object. Moreover, depth
information is used to fine-tune the scaling factors by estimating the
potential volume range. The fine-tuned scaling factors are then applied to the
cleaned food meshes for accurate volume measurements. Similarly, we enter a
segmented RGB image to the One-2-3-45 model for one-shot food volume
estimation, resulting in a mesh. We then leverage the obtained scaling factors
to the cleaned food mesh for accurate volume measurements. Our experiments show
that our method effectively addresses occlusions, varying lighting conditions,
and complex food geometries, achieving robust and accurate volume estimations
with 10.97% MAPE using the MTF dataset. This innovative approach enhances the
precision of volume assessments and significantly contributes to computational
nutrition and dietary monitoring advancements.

使用 3D 生成技术进行食物容量估算，通过选择关键帧、分割参考物体、估计相机位置、独立构建参考和食物对象的网格，并通过精调缩放因子进行准确容量测量，实现了对食物容量的鲁棒准确估计，对计算营养和膳食监测的进展具有重要贡献。

VolETA：一次和少量样本的食物体积估计

VolETA: One- and Few-shot Food Volume Estimation

Researches have shown that diet recording can help people increase awareness
of food intake and improve nutrition management, and thereby maintain a
healthier life. Recently, researchers have been working on smartphone-based
diet recording methods and applications that help users accomplish two tasks:
record what they eat and how much they eat. Although the former task has made
great progress through adopting image recognition technology, it is still a
challenge to estimate the volume of foods accurately and conveniently. In this
paper, we propose a novel method, named MUSEFood, for food volume estimation.
MUSEFood uses the camera to capture photos of the food, but unlike existing
volume measurement methods, MUSEFood requires neither training images with
volume information nor placing a reference object of known size while taking
photos. In addition, considering the impact of different containers on the
contour shape of foods, MUSEFood uses a multi-task learning framework to
improve the accuracy of food segmentation, and uses a differential model
applicable for various containers to further reduce the negative impact of
container differences on volume estimation accuracy. Furthermore, MUSEFood uses
the microphone and the speaker to accurately measure the vertical distance from
the camera to the food in a noisy environment, thus scaling the size of food in
the image to its actual size. The experiments on real foods indicate that
MUSEFood outperforms state-of-the-art approaches, and highly improves the speed
of food volume estimation.

本文提出了一种名为 MUSEFood 的新方法，利用智能手机的摄像头、麦克风和扬声器结合多任务学习框架，旨在解决估算食物容量准确度差的问题。实验证明，MUSEFood 可以高效地估算食物容量并具有比现有方法更好的性能。