While research on explaining predictions of open-domain QA systems (ODQA) to
users is gaining momentum, most works have failed to evaluate the extent to
which explanations improve user trust. While few works evaluate explanations
using user studies, they employ settings that may deviate from the end-user's
usage in-the-wild: ODQA is most ubiquitous in voice-assistants, yet current
research only evaluates explanations using a visual display, and may
erroneously extrapolate conclusions about the most performant explanations to
other modalities. To alleviate these issues, we conduct user studies that
measure whether explanations help users correctly decide when to accept or
reject an ODQA system's answer. Unlike prior work, we control for explanation
modality, e.g., whether they are communicated to users through a spoken or
visual interface, and contrast effectiveness across modalities. Our results
show that explanations derived from retrieved evidence passages can outperform
strong baselines (calibrated confidence) across modalities but the best
explanation strategy in fact changes with the modality. We show common failure
cases of current explanations, emphasize end-to-end evaluation of explanations,
and caution against evaluating them in proxy modalities that are different from
deployment.

本文研究了如何解释开放域 QA 系统（ODQA）的预测结果，结果表明解释可以帮助用户正确决定何时接受或拒绝 ODQA 系统的答案，而最佳的解释策略则随着使用的交互方式而异。