Food computing brings various perspectives to computer vision like
vision-based food analysis for nutrition and health. As a fundamental task in
food computing, food detection needs Zero-Shot Detection (ZSD) on novel unseen
food objects to support real-world scenarios, such as intelligent kitchens and
smart restaurants. Therefore, we first benchmark the task of Zero-Shot Food
Detection (ZSFD) by introducing FOWA dataset with rich attribute annotations.
Unlike ZSD, fine-grained problems in ZSFD like inter-class similarity make
synthesized features inseparable. The complexity of food semantic attributes
further makes it more difficult for current ZSD methods to distinguish various
food categories. To address these problems, we propose a novel framework ZSFDet
to tackle fine-grained problems by exploiting the interaction between complex
attributes. Specifically, we model the correlation between food categories and
attributes in ZSFDet by multi-source graphs to provide prior knowledge for
distinguishing fine-grained features. Within ZSFDet, Knowledge-Enhanced Feature
Synthesizer (KEFS) learns knowledge representation from multiple sources (e.g.,
ingredients correlation from knowledge graph) via the multi-source graph
fusion. Conditioned on the fusion of semantic knowledge representation, the
region feature diffusion model in KEFS can generate fine-grained features for
training the effective zero-shot detector. Extensive evaluations demonstrate
the superior performance of our method ZSFDet on FOWA and the widely-used food
dataset UECFOOD-256, with significant improvements by 1.8% and 3.7% ZSD mAP
compared with the strong baseline RRFS. Further experiments on PASCAL VOC and
MS COCO prove that enhancement of the semantic knowledge can also improve the
performance on general ZSD. Code and dataset are available at
this https URL

提出了一种新的框架 ZSFDet 来解决零样本食物检测中的细粒度问题，并通过知识表示增强性能。

合成强化知识特征的现实世界零样本食物检测

Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food  Detection

Generally, image-to-image translation (i2i) methods aim at learning mappings
across domains with the assumption that the images used for translation share
content (e.g., pose) but have their own domain-specific information (a.k.a.
style). Conditioned on a target image, such methods extract the target style
and combine it with the source image content, keeping coherence between the
domains. In our proposal, we depart from this traditional view and instead
consider the scenario where the target domain is represented by a very
low-resolution (LR) image, proposing a domain-agnostic i2i method for
fine-grained problems, where the domains are related. More specifically, our
domain-agnostic approach aims at generating an image that combines visual
features from the source image with low-frequency information (e.g. pose,
color) of the LR target image. To do so, we present a novel approach that
relies on training the generative model to produce images that both share
distinctive information of the associated source image and correctly match the
LR target image when downscaled. We validate our method on the CelebA-HQ and
AFHQ datasets by demonstrating improvements in terms of visual quality.
Qualitative and quantitative results show that when dealing with intra-domain
image translation, our method generates realistic samples compared to
state-of-the-art methods such as StarGAN v2. Ablation studies also reveal that
our method is robust to changes in color, it can be applied to
out-of-distribution images, and it allows for manual control over the final
results.

本论文提出了一种面向细粒度问题的无域图像转换方法，其使用一种全新的方法生成图像，这种方法集中在从源图像提取视觉特征并与低频信息相结合，以生成符合期望的输出图像，该方法在处理面向细粒度问题的图像转换任务时能够生成更真实的图像样本。