Multi-class product counting and recognition identifies product items from
images or videos for automated retail checkout. The task is challenging due to
the real-world scenario of occlusions where product items overlap, fast
movement in the conveyor belt, large similarity in overall appearance of the
items being scanned, novel products, and the negative impact of misidentifying
items. Further, there is a domain bias between training and test sets,
specifically, the provided training dataset consists of synthetic images and
the test set videos consist of foreign objects such as hands and tray. To
address these aforementioned issues, we propose to segment and classify
individual frames from a video sequence. The segmentation method consists of a
unified single product item- and hand-segmentation followed by entropy masking
to address the domain bias problem. The multi-class classification method is
based on Vision Transformers (ViT). To identify the frames with target objects,
we utilize several image processing methods and propose a custom metric to
discard frames not having any product items. Combining all these mechanisms,
our best system achieves 3rd place in the AI City Challenge 2022 Track 4 with
an F1 score of 0.4545. Code will be available at

针对自动零售结账场景中存在的物品重叠、物品外观相似等问题，本文提出了一种基于多分类的物品计数和识别方法，采用 ViT 进行分类，采用图像分割和熵掩蔽解决域偏差问题，并使用多种图像处理方法找到目标对象的帧，最终在 AI 城市挑战赛 2022 第四赛道中获得了第三名。