Retail product or packaged grocery goods images need to classified in various
computer vision applications like self checkout stores, supply chain automation
and retail execution evaluation. Previous works explore ways to finetune deep
models for this purpose. But because of the fact that finetuning a large model
or even linear layer for a pretrained backbone requires to run at least a few
epochs of gradient descent for every new retail product added in classification
range, frequent retrainings are needed in a real world scenario. In this work,
we propose finetuning the vision encoder of a CLIP model in a way that its
embeddings can be easily used for nearest neighbor based classification, while
also getting accuracy close to or exceeding full finetuning. A nearest neighbor
based classifier needs no incremental training for new products, thus saving
resources and wait time.

该研究主要探索如何用最近邻分类的方法对零售产品或打包食品进行图像分类，并使用 CLIP 模型的视觉编码器进行微调以实现准确性与完全微调相当甚至超过完全微调的结果，从而节省资源和等待时间。