Human-Computer Interaction has been shown to lead to improvements in machine
learning systems by boosting model performance, accelerating learning and
building user confidence. In this work, we aim to alleviate the expectation
that human annotators adapt to the constraints imposed by traditional labels by
allowing for extra flexibility in the form that supervision information is
collected. For this, we propose a human-machine learning interface for binary
classification tasks which enables human annotators to utilise counterfactual
examples to complement standard binary labels as annotations for a dataset.
Finally we discuss the challenges in future extensions of this work.

本文旨在通过为二分类任务提供人机学习界面，使人类注释者能够利用反事实例来补充标准的二分类标签，以缓解人类注释者适应传统标签所施加的限制。

高级交互式人机学习界面用于收集和学习复杂的注释

An Interactive Human-Machine Learning Interface for Collecting and  Learning from Complex Annotations

Recent work shows issues of consistency with explanations, with methods
generating local explanations that seem reasonable instance-wise, but that are
inconsistent across instances. This suggests not only that instance-wise
explanations can be unreliable, but mainly that, when interacting with a system
via multiple inputs, a user may actually lose confidence in the system. To
better analyse this issue, in this work we treat explanations as objects that
can be subject to reasoning and present a formal model of the interactive
scenario between user and system, via sequences of inputs, outputs, and
explanations. We argue that explanations can be thought of as committing to
some model behaviour (even if only prima facie), suggesting a form of
entailment, which, we argue, should be thought of as non-monotonic. This
allows: 1) to solve some considered inconsistencies in explanation, such as via
a specificity relation; 2) to consider properties from the non-monotonic
reasoning literature and discuss their desirability, gaining more insight on
the interactive explanation scenario.

本文提出了关于解释对象的一种形式模型，用于分析用户与系统之间的交互情景，通过输入，输出和解释的顺序。作者论证了解释可以被认为是承诺某种行为模式的对象，并认为其应被视为非单调性。这样做可以通过确定性关系解决一些解释中考虑到的不一致性，也可以考虑非单调推理文献中的属性并讨论其可取性，从而更深入地了解交互式解释场景。