SecureBoost is a tree-boosting algorithm that leverages homomorphic
encryption (HE) to protect data privacy in vertical federated learning.
SecureBoost and its variants have been widely adopted in fields such as finance
and healthcare. However, the hyperparameters of SecureBoost are typically
configured heuristically for optimizing model performance (i.e., utility)
solely, assuming that privacy is secured. Our study found that SecureBoost and
some of its variants are still vulnerable to label leakage. This vulnerability
may lead the current heuristic hyperparameter configuration of SecureBoost to a
suboptimal trade-off between utility, privacy, and efficiency, which are
pivotal elements toward a trustworthy federated learning system. To address
this issue, we propose the Constrained Multi-Objective SecureBoost (CMOSB)
algorithm, which aims to approximate Pareto optimal solutions that each
solution is a set of hyperparameters achieving an optimal trade-off between
utility loss, training cost, and privacy leakage. We design measurements of the
three objectives, including a novel label inference attack named instance
clustering attack (ICA) to measure the privacy leakage of SecureBoost.
Additionally, we provide two countermeasures against ICA. The experimental
results demonstrate that the CMOSB yields superior hyperparameters over those
optimized by grid search and Bayesian optimization regarding the trade-off
between utility loss, training cost, and privacy leakage.

为了解决 SecureBoost 及其变种在隐私泄露方面的漏洞问题，我们提出了一种名为 Constrained Multi-Objective SecureBoost (CMOSB) 的算法，旨在实现在模型效果、训练成本和隐私泄露之间取得最佳平衡的一组超参数集合。

通过约束多目标联邦学习优化 SecureBoost 的超参数

Hyperparameter Optimization for SecureBoost via Constrained  Multi-Objective Federated Learning

SecureBoost is a tree-boosting algorithm leveraging homomorphic encryption to
protect data privacy in vertical federated learning setting. It is widely used
in fields such as finance and healthcare due to its interpretability,
effectiveness, and privacy-preserving capability. However, SecureBoost suffers
from high computational complexity and risk of label leakage. To harness the
full potential of SecureBoost, hyperparameters of SecureBoost should be
carefully chosen to strike an optimal balance between utility, efficiency, and
privacy. Existing methods either set hyperparameters empirically or
heuristically, which are far from optimal. To fill this gap, we propose a
Constrained Multi-Objective SecureBoost (CMOSB) algorithm to find Pareto
optimal solutions that each solution is a set of hyperparameters achieving
optimal tradeoff between utility loss, training cost, and privacy leakage. We
design measurements of the three objectives. In particular, the privacy leakage
is measured using our proposed instance clustering attack. Experimental results
demonstrate that the CMOSB yields not only hyperparameters superior to the
baseline but also optimal sets of hyperparameters that can support the flexible
requirements of FL participants.

SecureBoost 是一种利用同态加密保护数据隐私的树增强算法，广泛应用于金融和医疗保健等领域，具有解释性、有效性和保护隐私的能力。然而，SecureBoost 存在计算复杂度高和标签泄漏风险的问题。为了充分发挥 SecureBoost 的潜力，应该仔细选择其超参数，以在效用、效率和隐私之间取得最佳平衡。现有方法要么以经验方式设置超参数，要么是经验性启发式方法，远未达到最优。为了填补这一空白，我们提出了一种约束多目标 SecureBoost（CMOSB）算法，以找到 Pareto 最优解，其中每个解都是一组超参数，实现了效用损失、训练成本和隐私泄漏的最佳权衡。我们设计了三个目标的度量方法，特别是使用我们提出的实例聚类攻击来衡量隐私泄漏。实验结果表明，CMOSB 不仅产生了优于基线的超参数，而且还能够支持 FL 参与者的灵活要求的最佳超参数集合。

SecureBoost 超参数调整的多目标联邦学习

SecureBoost Hyperparameter Tuning via Multi-Objective Federated Learning

The protection of user privacy is an important concern in machine learning,
as evidenced by the rolling out of the General Data Protection Regulation
(GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give
users more control over their personal data, which motivates us to explore
machine learning frameworks for data sharing that do not violate user privacy.
To meet this goal, in this paper, we propose a novel lossless
privacy-preserving tree-boosting system known as SecureBoost in the setting of
federated learning. SecureBoost first conducts entity alignment under a
privacy-preserving protocol and then constructs boosting trees across multiple
parties with a carefully designed encryption strategy. This federated learning
system allows the learning process to be jointly conducted over multiple
parties with common user samples but different feature sets, which corresponds
to a vertically partitioned data set. An advantage of SecureBoost is that it
provides the same level of accuracy as the non-privacy-preserving approach
while at the same time, reveals no information of each private data provider.
We show that the SecureBoost framework is as accurate as other non-federated
gradient tree-boosting algorithms that require centralized data and thus it is
highly scalable and practical for industrial applications such as credit risk
analysis. To this end, we discuss information leakage during the protocol
execution and propose ways to provably reduce it.

本文提出了一种名为 SecureBoost 的隐私保护决策树增强系统，旨在使用联邦学习框架实现数据共享，该框架在不遵守用户隐私的前提下，通过实体对齐和加密来对多方进行决策树增强，并且 SecureBoost 可与其他受欢迎的增强算法实现同样准确率，同时保护每个数据提供者的隐私。