In machine learning ensembles predictions from multiple models are
aggregated. Despite widespread use and strong performance of ensembles in
applied problems little is known about the mathematical properties of
aggregating models and associated consequences for safe, explainable use of
such models. In this paper we prove a theorem that shows that any ensemble will
exhibit at least one of the following forms of prediction instability. It will
either ignore agreement among all underlying models, change its mind when none
of the underlying models have done so, or be manipulable through inclusion or
exclusion of options it would never actually predict. As a consequence,
ensemble aggregation procedures will always need to balance the benefits of
information use against the risk of these prediction instabilities. This
analysis also sheds light on what specific forms of prediction instability to
expect from particular ensemble algorithms; for example popular tree ensembles
like random forest, or xgboost will violate basic, intuitive monotonicity and
fairness properties.

在机器学习集成中，多个模型的预测被汇总。本文证明了一个定理，表明任何集成模型都会展现出至少一种形式的预测不稳定性，从而需要在信息利用的好处和预测不稳定性的风险之间取得平衡。该分析还揭示了特定集成算法所产生的预测不稳定性的形式，例如像随机森林或 xgboost 这样的流行树集成算法将违反基本的、直观的单调性和公平性属性。

机器学习集成中的预测不稳定性

Prediction Instability in Machine Learning Ensembles

Machine learning ensembles combine multiple base models to produce a more
accurate output. They can be applied to a range of machine learning problems,
including anomaly detection. In this paper, we investigate how to maximize the
composability and scalability of an FPGA-based streaming ensemble anomaly
detector (fSEAD). To achieve this, we propose a flexible computing architecture
consisting of multiple partially reconfigurable regions, pblocks, which each
implement anomaly detectors. Our proof-of-concept design supports three
state-of-the-art anomaly detection algorithms: Loda, RS-Hash and xStream. Each
algorithm is scalable, meaning multiple instances can be placed within a pblock
to improve performance. Moreover, fSEAD is implemented using High-level
synthesis (HLS), meaning further custom anomaly detectors can be supported.
Pblocks are interconnected via an AXI-switch, enabling them to be composed in
an arbitrary fashion before combining and merging results at run-time to create
an ensemble that maximizes the use of FPGA resources and accuracy. Through
utilizing reconfigurable Dynamic Function eXchange (DFX), the detector can be
modified at run-time to adapt to changing environmental conditions. We compare
fSEAD to an equivalent central processing unit (CPU) implementation using four
standard datasets, with speed-ups ranging from $3\times$ to $8\times$.

提出了一种机器学习集合体的异常检测器，它基于 FPGA，采用多重可重构区域的柔性计算架构，并支持多种异常检测算法，通过最大程度利用 FPGA 资源以及通过运行时的组合和合并结果来优化精度。