Despite significant progress, evaluation of explainable artificial
intelligence remains elusive and challenging. In this paper we propose a
fine-grained validation framework that is not overly reliant on any one facet
of these sociotechnical systems, and that recognises their inherent modular
structure: technical building blocks, user-facing explanatory artefacts and
social communication protocols. While we concur that user studies are
invaluable in assessing the quality and effectiveness of explanation
presentation and delivery strategies from the explainees' perspective in a
particular deployment context, the underlying explanation generation mechanisms
require a separate, predominantly algorithmic validation strategy that accounts
for the technical and human-centred desiderata of their (numerical) outputs.
Such a comprehensive sociotechnical utility-based evaluation framework could
allow to systematically reason about the properties and downstream influence of
different building blocks from which explainable artificial intelligence
systems are composed -- accounting for a diverse range of their engineering and
social aspects -- in view of the anticipated use case.

该论文提出了一种基于精细验证框架的解释人工智能，该框架不过分依赖于这些社会技术系统的任何一个方面，并承认它们固有的模块化结构：技术构建模块、面向用户的解释工件和社交通信协议。

可解释人工智能评估实际告诉我们什么？对 XAI 构建模块进行组合与上下文验证的案例

What Does Evaluation of Explainable Artificial Intelligence Actually  Tell Us? A Case for Compositional and Contextual Validation of XAI Building  Blocks

Guidance on how to validate computational text-based measures of social
science constructs is fragmented. Whereas scholars are generally acknowledging
the importance of validating their text-based measures, they often lack common
terminology and a unified framework to do so. This paper introduces a new
validation framework called ValiTex, designed to assist scholars to measure
social science constructs based on textual data. The framework draws on a
long-established tradition within psychometrics while extending the framework
for the purpose of computational text analysis. ValiTex consists of two
components, a conceptual model, and a dynamic checklist. Whereas the conceptual
model provides a general structure along distinct phases on how to approach
validation, the dynamic checklist defines specific validation steps and
provides guidance on which steps might be considered recommendable (i.e.,
providing relevant and necessary validation evidence) or optional (i.e., useful
for providing additional supporting validation evidence. The utility of the
framework is demonstrated by applying it to a use case of detecting sexism from
social media data.

本研究提出了一种名为 ValiTex 的新的验证框架，旨在帮助学者基于文本数据对社会科学构造进行测量，并通过将心理测量学的框架扩展到计算文本分析领域，提供通用的结构和具体的验证步骤。该框架在社交媒体数据中检测性别歧视的案例中展示了其实用性。