Automating the checkout process is important in smart retail, where users
effortlessly pass products by hand through a camera, triggering automatic
product detection, tracking, and counting. In this emerging area, due to the
lack of annotated training data, we introduce a dataset comprised of product 3D
models, which allows for fast, flexible, and large-scale training data
generation through graphic engine rendering. Within this context, we discern an
intriguing facet, because of the user "hands-on" approach, bias in user
behavior leads to distinct patterns in the real checkout process. The existence
of such patterns would compromise training effectiveness if training data fail
to reflect the same. To address this user bias problem, we propose a training
data optimization framework, i.e., training with digital twins (DtTrain).
Specifically, we leverage the product 3D models and optimize their rendering
viewpoint and illumination to generate "digital twins" that visually resemble
representative user images. These digital twins, inherit product labels and,
when augmented, form the Digital Twin training set (DT set). Because the
digital twins individually mimic user bias, the resulting DT training set
better reflects the characteristics of the target scenario and allows us to
train more effective product detection and tracking models. In our experiment,
we show that DT set outperforms training sets created by existing dataset
synthesis methods in terms of counting accuracy. Moreover, by combining DT set
with pseudo-labeled real checkout data, further improvement is observed. The
code is available at this https URL

通过优化训练数据集并结合数字双胞胎的方法，实现了在智能零售中自动化结账过程的改善和准确性提高。