While the role of humans is increasingly recognized in machine learning
community, representation of and interaction with models in current
human-in-the-loop machine learning (HITL-ML) approaches are too low-level and
far-removed from human's conceptual models. We demonstrate HEIDL, a prototype
HITL-ML system that exposes the machine-learned model through high-level,
explainable linguistic expressions formed of predicates representing semantic
structure of text. In HEIDL, human's role is elevated from simply evaluating
model predictions to interpreting and even updating the model logic directly by
enabling interaction with rule predicates themselves. Raising the currency of
interaction to such semantic levels calls for new interaction paradigms between
humans and machines that result in improved productivity for text analytics
model development process. Moreover, by involving humans in the process, the
human-machine co-created models generalize better to unseen data as domain
experts are able to instill their expertise by extrapolating from what has been
learned by automated algorithms from few labelled data.

本文介绍了一种名为 HEIDL 的新型人机交互机器学习系统，可以通过高水平的、可解释的语言表达式与人交互，从而提高人类在模型预测、理解以及更新方面的参与度，并且可以使得机器学习模型能够更好地应用于实际场景中。

HEIDL: 用深度学习和人工参与学习语言表达

HEIDL: Learning Linguistic Expressions with Deep Learning and  Human-in-the-Loop

In human-in-the-loop machine learning, the user provides information beyond
that in the training data. Many algorithms and user interfaces have been
designed to optimize and facilitate this human--machine interaction; however,
fewer studies have addressed the potential defects the designs can cause.
Effective interaction often requires exposing the user to the training data or
its statistics. The design of the system is then critical, as this can lead to
double use of data and overfitting, if the user reinforces noisy patterns in
the data. We propose a user modelling methodology, by assuming simple rational
behaviour, to correct the problem. We show, in a user study with 48
participants, that the method improves predictive performance in a sparse
linear regression sentiment analysis task, where graded user knowledge on
feature relevance is elicited. We believe that the key idea of inferring user
knowledge with probabilistic user models has general applicability in guarding
against overfitting and improving interactive machine learning.

本研究提出了一种用户建模方法，通过概率用户模型来推断用户的知识，从而提高交互式机器学习的性能，并避免过度拟合的问题。