Gradient boosting of prediction rules is an efficient approach to learn
potentially interpretable yet accurate probabilistic models. However, actual
interpretability requires to limit the number and size of the generated rules,
and existing boosting variants are not designed for this purpose. Though
corrective boosting refits all rule weights in each iteration to minimise
prediction risk, the included rule conditions tend to be sub-optimal, because
commonly used objective functions fail to anticipate this refitting. Here, we
address this issue by a new objective function that measures the angle between
the risk gradient vector and the projection of the condition output vector onto
the orthogonal complement of the already selected conditions. This approach
correctly approximate the ideal update of adding the risk gradient itself to
the model and favours the inclusion of more general and thus shorter rules. As
we demonstrate using a wide range of prediction tasks, this significantly
improves the comprehensibility/accuracy trade-off of the fitted ensemble.
Additionally, we show how objective values for related rule conditions can be
computed incrementally to avoid any substantial computational overhead of the
new method.

梯度提升预测规则是学习可能解释性又准确的概率模型的有效方法，本文通过引入新的目标函数来优化规则条件的选择，从而在理解性与准确性之间取得了显著的平衡。

正交梯度提升用于简化的加法规则集合

Orthogonal Gradient Boosting for Simpler Additive Rule Ensembles

In this article I propose an approach for defining replicability for
prediction rules. Motivated by a recent NAS report, I start from the
perspective that replicability is obtaining consistent results across studies
suitable to address the same prediction question, each of which has obtained
its own data. I then discuss concept and issues in defining key elements of
this statement. I focus specifically on the meaning of "consistent results" in
typical utilization contexts, and propose a multi-agent framework for defining
replicability, in which agents are neither partners nor adversaries. I recover
some of the prevalent practical approaches as special cases. I hope to provide
guidance for a more systematic assessment of replicability in machine learning.

本文提出了一种基于多智能体框架的方法来定义机器学习中预测规则的可复制性，旨在提供更系统的评估复制性的指导，关注一致结果在典型利用情境中的含义。

预测规则的可重复性定义

Defining Replicability of Prediction Rules

We present a methodology for modifying the behavior of a classifier by
directly rewriting its prediction rules. Our approach requires virtually no
additional data collection and can be applied to a variety of settings,
including adapting a model to new environments, and modifying it to ignore
spurious features. Our code is available at
this https URL .

本文提出了一种通过直接重写预测规则来修改分类器行为的方法，该方法几乎不需要额外的数据收集，并可应用于各种设置，包括适应模型到新环境和修改模型以忽略杂散特征。

通过重写预测规则来编辑分类器

Editing a classifier by rewriting its prediction rules

Implicit deep learning prediction rules generalize the recursive rules of
feedforward neural networks. Such rules are based on the solution of a
fixed-point equation involving a single vector of hidden features, which is
thus only implicitly defined. The implicit framework greatly simplifies the
notation of deep learning, and opens up many new possibilities, in terms of
novel architectures and algorithms, robustness analysis and design,
interpretability, sparsity, and network architecture optimization.

隐式深度学习预测规则推广了前馈神经网络的递归规则。这种规则基于涉及单个隐含特征向量的不动点方程的解，因此只是隐式定义的。隐式框架极大地简化了深度学习的符号表示，并在新体系结构和算法、稳健性分析和设计、可解释性、稀疏性和网络架构优化等方面开辟了许多新的可能性。