We introduce a method to provide vectorial representations of visual
classification tasks which can be used to reason about the nature of those
tasks and their relations. Given a dataset with ground-truth labels and a loss
function defined over those labels, we process images through a "probe network"
and compute an embedding based on estimates of the Fisher information matrix
associated with the probe network parameters. This provides a fixed-dimensional
embedding of the task that is independent of details such as the number of
classes and does not require any understanding of the class label semantics. We
demonstrate that this embedding is capable of predicting task similarities that
match our intuition about semantic and taxonomic relations between different
visual tasks (e.g., tasks based on classifying different types of plants are
similar) We also demonstrate the practical value of this framework for the
meta-task of selecting a pre-trained feature extractor for a new task. We
present a simple meta-learning framework for learning a metric on embeddings
that is capable of predicting which feature extractors will perform well.
Selecting a feature extractor with task embedding obtains a performance close
to the best available feature extractor, while costing substantially less than
exhaustively training and evaluating on all available feature extractors.

本文介绍了一种提供视觉分类任务向量表示的方法，该表示可用于推断任务及其关系的性质，并提供了独立于细节（如类标签语义的理解等）的任务的固定维度嵌入。我们还展示了该框架的实用价值以及通过学习嵌入度量来选择预训练特征提取器的简单元学习框架。选择具有任务嵌入的特征提取器可获得接近最佳特征提取器的性能，而成本则显著低于对所有可用特征提取器进行详尽的训练和评估。