Predicting the collaboration likelihood and measuring cognitive trust to AI
systems is more important than ever. To do that, previous research mostly focus
solely on the model features (e.g., accuracy, confidence) and ignore the human
factor. To address that, we propose several decision-making similarity measures
based on divergence metrics (e.g., KL, JSD) calculated over the labels acquired
from humans and a wide range of models. We conduct a user study on a textual
entailment task, where the users are provided with soft labels from various
models and asked to pick the closest option to them. The users are then shown
the similarities/differences to their most similar model and are surveyed for
their likelihood of collaboration and cognitive trust to the selected system.
Finally, we qualitatively and quantitatively analyze the relation between the
proposed decision-making similarity measures and the survey results. We find
that people tend to collaborate with their most similar models -- measured via
JSD -- yet this collaboration does not necessarily imply a similar level of
cognitive trust. We release all resources related to the user study (e.g.,
design, outputs), models, and metrics at our repo.

预测协作可能性和衡量对人工智能系统的认知信任比以往任何时候都更加重要。为了解决这个问题，我们提出了几种基于分歧度量（如 KL，JSD）的决策相似度度量方法，这些方法通过对人类获取的标签和各种模型所得到的标签进行计算。我们在一个文本蕴涵任务上进行了用户研究，用户被提供了来自各种模型的软标签，并被要求选择与他们最接近的选项。然后，用户看到了与他们最相似的模型的相似性 / 差异，并被询问他们对所选择系统的协作可能性和认知信任的看法。最后，我们对提出的决策相似度度量方法与调查结果之间的关系进行了定性和定量分析。我们发现人们倾向于与他们最相似的模型进行协作（通过 JSD 来衡量），然而这种协作并不一定意味着相似的认知信任水平。我们在我们的资源库中发布了与用户研究（如设计，输出）、模型和度量有关的所有资源。

人工智能与人类协作及认知信任的差异量化

Quantifying Divergence for Human-AI Collaboration and Cognitive Trust

Automated vehicles (AVs) are social robots that can potentially benefit our
society. According to the existing literature, AV explanations can promote
passengers' trust by reducing the uncertainty associated with the AV's
reasoning and actions. However, the literature on AV explanations and trust has
failed to consider how the type of trust
- cognitive versus affective - might alter this relationship. Yet, the
existing literature has shown that the implications associated with trust vary
widely depending on whether it is cognitive or affective. To address this
shortcoming and better understand the impacts of explanations on trust in AVs,
we designed a study to investigate the effectiveness of explanations on both
cognitive and affective trust. We expect these results to be of great
significance in designing AV explanations to promote AV trust.

研究发现，AV 解释对乘客的信任有促进作用，不过这一效果在认知信任和情感信任上可能会有所不同。因此，该研究旨在探究 AV 解释对认知和情感信任的影响，以更好地设计 AV 解释，并促进人们对 AV 的信任。