Advancements in large pre-trained generative models have expanded their
potential as effective data generators in visual recognition. This work delves
into the impact of generative images, primarily comparing paradigms that
harness external data (\ie generative \vs retrieval \vs original).
Our key contributions are: \textbf{1) GenBench Construction:} We devise
\textbf{GenBench}, a broad benchmark comprising 22 datasets with 2548
categories, to appraise generative data across various visual recognition
tasks. \textbf{2) CLER Score:} To address the insufficient correlation of
existing metrics (\eg, FID, CLIP score) with downstream recognition
performance, we propose \textbf{CLER}, a training-free metric indicating
generative data's efficiency for recognition tasks prior to training.
\textbf{3) New Baselines:} Comparisons of generative data with retrieved data
from the same external pool help to elucidate the unique traits of generative
data. \textbf{4) External Knowledge Injection:} By fine-tuning special token
embeddings for each category via Textual Inversion, performance improves across
17 datasets, except when dealing with low-resolution reference images.
Our exhaustive benchmark and analysis spotlight generative data's promise in
visual recognition, while identifying key challenges for future investigation.

本研究探讨了生成图像的影响，通过比较利用外部数据的范例（生成数据 vs 检索数据 vs 原始数据），并提出 GenBench 构建、CLER 评分、新的基准、外部知识注入等关键贡献，突出生成数据在视觉识别中的潜力和未来研究所面临的主要挑战。

基于生成数据的视觉识别基准评估与分析

Benchmarking and Analyzing Generative Data for Visual Recognition

Recent research demonstrates that external knowledge injection can advance
pre-trained language models (PLMs) in a variety of downstream NLP tasks.
However, existing knowledge injection methods are either applicable to
structured knowledge or unstructured knowledge, lacking a unified usage. In
this paper, we propose a UNified knowledge inTERface, UNTER, to provide a
unified perspective to exploit both structured knowledge and unstructured
knowledge. In UNTER, we adopt the decoder as a unified knowledge interface,
aligning span representations obtained from the encoder with their
corresponding knowledge. This approach enables the encoder to uniformly invoke
span-related knowledge from its parameters for downstream applications.
Experimental results show that, with both forms of knowledge injected, UNTER
gains continuous improvements on a series of knowledge-driven NLP tasks,
including entity typing, named entity recognition and relation extraction,
especially in low-resource scenarios.

本研究提出了一种名为 UNTER 的统一知识接口，它能够将结构化知识和非结构化知识结合在一起，从而进一步提升预训练语言模型在自然语言处理任务中的性能，特别是在低资源情况下实体类型识别等方面。

UNTER: 增强预训练语言模型的统一知识接口

UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language  Models

Procedural text describes dynamic state changes during a step-by-step natural
process (e.g., photosynthesis). In this work, we focus on the task of
procedural text understanding, which aims to comprehend such documents and
track entities' states and locations during a process. Although recent
approaches have achieved substantial progress, their results are far behind
human performance. Two challenges, the difficulty of commonsense reasoning and
data insufficiency, still remain unsolved, which require the incorporation of
external knowledge bases. Previous works on external knowledge injection
usually rely on noisy web mining tools and heuristic rules with limited
applicable scenarios. In this paper, we propose a novel KnOwledge-Aware
proceduraL text understAnding (KOALA) model, which effectively leverages
multiple forms of external knowledge in this task. Specifically, we retrieve
informative knowledge triples from ConceptNet and perform knowledge-aware
reasoning while tracking the entities. Besides, we employ a multi-stage
training schema which fine-tunes the BERT model over unlabeled data collected
from Wikipedia before further fine-tuning it on the final model. Experimental
results on two procedural text datasets, ProPara and Recipes, verify the
effectiveness of the proposed methods, in which our model achieves
state-of-the-art performance in comparison to various baselines.

研究了手续性文本理解的挑战，发现外部知识注入和数据不足的问题，提出了一个多阶段训练的知识感知 KOALA 模型，最终在 ProPara 和 Recipes 数据集上达到了最先进的性能表现。