Natural Language Explanations (NLE) aim at supplementing the prediction of a
model with human-friendly natural text. Existing NLE approaches involve
training separate models for each downstream task. In this work, we propose
Uni-NLX, a unified framework that consolidates all NLE tasks into a single and
compact multi-task model using a unified training objective of text generation.
Additionally, we introduce two new NLE datasets: 1) ImageNetX, a dataset of
144K samples for explaining ImageNet categories, and 2) VQA-ParaX, a dataset of
123K samples for explaining the task of Visual Question Answering (VQA). Both
datasets are derived leveraging large language models (LLMs). By training on
the 1M combined NLE samples, our single unified framework is capable of
simultaneously performing seven NLE tasks including VQA, visual recognition and
visual reasoning tasks with 7X fewer parameters, demonstrating comparable
performance to the independent task-specific models in previous approaches, and
in certain tasks even outperforming them. Code is at
this https URL

提出了 Uni-NLX，一个统一的框架，将所有自然语言解释任务整合到一个紧凑的多任务模型中，使用文本生成的统一训练目标。引入了两个新的 NLE 数据集：ImageNetX，一个包含 144K 个样本的数据集，用于解释 ImageNet 分类；VQA-ParaX，一个包含 123K 个样本的数据集，用于解释视觉问答（VQA）任务。通过训练这 1M 个组合的 NLE 样本，在参数数量减少 7 倍的情况下，我们的统一框架能够同时执行七个 NLE 任务，包括 VQA、视觉识别和视觉推理任务，在某些任务中甚至表现出色于以前方法中的独立任务特定模型。