Accurate food volume estimation is essential for dietary assessment,
nutritional tracking, and portion control applications. We present VolETA, a
sophisticated methodology for estimating food volume using 3D generative
techniques. Our approach creates a scaled 3D mesh of food objects using one- or
few-RGBD images. We start by selecting keyframes based on the RGB images and
then segmenting the reference object in the RGB images using XMem++.
Simultaneously, camera positions are estimated and refined using the PixSfM
technique. The segmented food images, reference objects, and camera poses are
combined to form a data model suitable for NeuS2. Independent mesh
reconstructions for reference and food objects are carried out, with scaling
factors determined using MeshLab based on the reference object. Moreover, depth
information is used to fine-tune the scaling factors by estimating the
potential volume range. The fine-tuned scaling factors are then applied to the
cleaned food meshes for accurate volume measurements. Similarly, we enter a
segmented RGB image to the One-2-3-45 model for one-shot food volume
estimation, resulting in a mesh. We then leverage the obtained scaling factors
to the cleaned food mesh for accurate volume measurements. Our experiments show
that our method effectively addresses occlusions, varying lighting conditions,
and complex food geometries, achieving robust and accurate volume estimations
with 10.97% MAPE using the MTF dataset. This innovative approach enhances the
precision of volume assessments and significantly contributes to computational
nutrition and dietary monitoring advancements.

使用 3D 生成技术进行食物容量估算，通过选择关键帧、分割参考物体、估计相机位置、独立构建参考和食物对象的网格，并通过精调缩放因子进行准确容量测量，实现了对食物容量的鲁棒准确估计，对计算营养和膳食监测的进展具有重要贡献。

VolETA：一次和少量样本的食物体积估计

VolETA: One- and Few-shot Food Volume Estimation

Recent methods for neural surface representation and rendering, for example
NeuS, have demonstrated remarkably high-quality reconstruction of static
scenes. However, the training of NeuS takes an extremely long time (8 hours),
which makes it almost impossible to apply them to dynamic scenes with thousands
of frames. We propose a fast neural surface reconstruction approach, called
NeuS2, which achieves two orders of magnitude improvement in terms of
acceleration without compromising reconstruction quality. To accelerate the
training process, we integrate multi-resolution hash encodings into a neural
surface representation and implement our whole algorithm in CUDA. We also
present a lightweight calculation of second-order derivatives tailored to our
networks (i.e., ReLU-based MLPs), which achieves a factor two speed up. To
further stabilize training, a progressive learning strategy is proposed to
optimize multi-resolution hash encodings from coarse to fine. In addition, we
extend our method for reconstructing dynamic scenes with an incremental
training strategy. Our experiments on various datasets demonstrate that NeuS2
significantly outperforms the state-of-the-arts in both surface reconstruction
accuracy and training speed. The video is available at
this https URL .

提出了一种快速的神经表面重建方法，称为 NeuS2，通过将多分辨率哈希编码集成到神经表面表示中，实现了两个数量级的加速，并通过 CUDA 实现了整个算法。该方法不仅提高了表面重建的准确性，还能有效地解决神经表面表示的训练时间长的问题，特别是在动态场景下。