Existing model evaluation tools mainly focus on evaluating classification
models, leaving a gap in evaluating more complex models, such as object
detection. In this paper, we develop an open-source visual analysis tool,
Uni-Evaluator, to support a unified model evaluation for classification, object
detection, and instance segmentation in computer vision. The key idea behind
our method is to formulate both discrete and continuous predictions in
different tasks as unified probability distributions. Based on these
distributions, we develop 1) a matrix-based visualization to provide an
overview of model performance; 2) a table visualization to identify the
problematic data subsets where the model performs poorly; 3) a grid
visualization to display the samples of interest. These visualizations work
together to facilitate the model evaluation from a global overview to
individual samples. Two case studies demonstrate the effectiveness of
Uni-Evaluator in evaluating model performance and making informed improvements.

该研究开发了一个开源的视觉分析工具 Uni-Evaluator，用于支持计算机视觉中分类、对象检测和实例分割等任务的统一模型评估。通过将离散和连续预测转化为统一的概率分布，研究提供了矩阵、表格和网格三种可视化方式，以全面评估模型性能，并通过两个案例研究证明了 Uni-Evaluator 在评估和改进模型性能方面的有效性。

计算机视觉中分类、目标检测和实例分割的统一交互式模型评估

A Unified Interactive Model Evaluation for Classification, Object  Detection, and Instance Segmentation in Computer Vision

The rapid development of deep natural language processing (NLP) models for
text classification has led to an urgent need for a unified understanding of
these models proposed individually. Existing methods cannot meet the need for
understanding different models in one framework due to the lack of a unified
measure for explaining both low-level (e.g., words) and high-level (e.g.,
phrases) features. We have developed a visual analysis tool, DeepNLPVis, to
enable a unified understanding of NLP models for text classification. The key
idea is a mutual information-based measure, which provides quantitative
explanations on how each layer of a model maintains the information of input
words in a sample. We model the intra- and inter-word information at each layer
measuring the importance of a word to the final prediction as well as the
relationships between words, such as the formation of phrases. A multi-level
visualization, which consists of a corpus-level, a sample-level, and a
word-level visualization, supports the analysis from the overall training set
to individual samples. Two case studies on classification tasks and comparison
between models demonstrate that DeepNLPVis can help users effectively identify
potential problems caused by samples and model architectures and then make
informed improvements.

本文提出一个可视化分析工具 DeepNLPVis，采用基于互信息的量化方法对深度自然语言处理模型进行统一理解，通过多层次的可视化（语料库级别、样本级别和单词级别）支持从整体到个别的分析，通过两个分类任务的案例研究和模型比较，证明了 DeepNLPVis 能够帮助用户识别潜在问题并做出知情决策。

深度 NLP 模型在文本分类中的统一理解

A Unified Understanding of Deep NLP Models for Text Classification

Neural Sequence-to-Sequence models have proven to be accurate and robust for
many sequence prediction tasks, and have become the standard approach for
automatic translation of text. The models work in a five stage blackbox process
that involves encoding a source sequence to a vector space and then decoding
out to a new target sequence. This process is now standard, but like many deep
learning methods remains quite difficult to understand or debug. In this work,
we present a visual analysis tool that allows interaction with a trained
sequence-to-sequence model through each stage of the translation process. The
aim is to identify which patterns have been learned and to detect model errors.
We demonstrate the utility of our tool through several real-world large-scale
sequence-to-sequence use cases.

本文提出了一种视觉分析工具，通过交互式的方式使得我们可以了解神经序列到序列模型训练过程中学习到的模式和检测模型错误，并通过多个实际大规模序列 - to - 序列应用程序验证了该工具的实用性。