As machine learning models are increasingly deployed in dynamic environments,
it becomes paramount to assess and quantify uncertainties associated with
distribution shifts. A distribution shift occurs when the underlying
data-generating process changes, leading to a deviation in the model's
performance. The prediction interval, which captures the range of likely
outcomes for a given prediction, serves as a crucial tool for characterizing
uncertainties induced by their underlying distribution. In this paper, we
propose methodologies for aggregating prediction intervals to obtain one with
minimal width and adequate coverage on the target domain under unsupervised
domain shift, under which we have labeled samples from a related source domain
and unlabeled covariates from the target domain. Our analysis encompasses
scenarios where the source and the target domain are related via i) a bounded
density ratio, and ii) a measure-preserving transformation. Our proposed
methodologies are computationally efficient and easy to implement. Beyond
illustrating the performance of our method through a real-world dataset, we
also delve into the theoretical details. This includes establishing rigorous
theoretical guarantees, coupled with finite sample bounds, regarding the
coverage and width of our prediction intervals. Our approach excels in
practical applications and is underpinned by a solid theoretical framework,
ensuring its reliability and effectiveness across diverse contexts.

我们提出了一种在无监督域偏移下聚合预测区间的方法，该方法能够获得宽度较小且覆盖目标域的预测区间，其计算效率高并易于实现。

无监督领域转移下的最优预测区间聚合

Optimal Aggregation of Prediction Intervals under Unsupervised Domain  Shift

Precise measurements of the black hole mass are essential to gain insight on
the black hole and host galaxy co-evolution. A direct measure of the black hole
mass is often restricted to nearest galaxies and instead, an indirect method
using the single-epoch virial black hole mass estimation is used for objects at
high redshifts. However, this method is subjected to biases and uncertainties
as it is reliant on the scaling relation from a small sample of local active
galactic nuclei. In this study, we propose the application of conformalised
quantile regression (CQR) to quantify the uncertainties of the black hole
predictions in a machine learning setting. We compare CQR with various
prediction interval techniques and demonstrated that CQR can provide a more
useful prediction interval indicator. In contrast to baseline approaches for
prediction interval estimation, we show that the CQR method provides prediction
intervals that adjust to the black hole mass and its related properties. That
is it yields a tighter constraint on the prediction interval (hence more
certain) for a larger black hole mass, and accordingly, bright and broad
spectral line width source. Using a combination of neural network model and CQR
framework, the recovered virial black hole mass predictions and uncertainties
are comparable to those measured from the Sloan Digital Sky Survey. The code is
publicly available at this https URL

本研究提出采用变量量化回归方法（CQR）来量化机器学习中黑洞质量预测的不确定性，并比较 CQR 与其他预测区间技术的表现。研究表明，CQR 方法可以更好地适应黑洞质量及其相关特性，为较大黑洞预测提供更紧密的预测区间的界定。

利用共形预测对虚拟黑洞质量进行不确定性量化

Uncertainty Quantification of the Virial Black Hole Mass with Conformal  Prediction

The quantitative structure-activity relationship (QSAR) regression model is a
commonly used technique for predicting biological activities of compounds using
their molecular descriptors. Predictions from QSAR models can help, for
example, to optimize molecular structure; prioritize compounds for further
experimental testing; and estimate their toxicity. In addition to the accurate
estimation of the activity, it is highly desirable to obtain some estimate of
the uncertainty associated with the prediction, e.g., calculate a prediction
interval (PI) containing the true molecular activity with a pre-specified
probability, say 70%, 90% or 95%. The challenge is that most machine learning
(ML) algorithms that achieve superior predictive performance require some
add-on methods for estimating uncertainty of their prediction. The development
of these algorithms is an active area of research by statistical and ML
communities but their implementation for QSAR modeling remains limited.
Conformal prediction (CP) is a promising approach. It is agnostic to the
prediction algorithm and can produce valid prediction intervals under some weak
assumptions on the data distribution. We proposed computationally efficient CP
algorithms tailored to the most advanced ML models, including Deep Neural
Networks and Gradient Boosting Machines. The validity and efficiency of
proposed conformal predictors are demonstrated on a diverse collection of QSAR
datasets as well as simulation studies.

提出了针对最先进的机器学习模型，包括深度神经网络和梯度提升机的计算有效的 CP 算法，可在一些数据分布的弱前提下生成有效预测间隔，为 QSAR 建模提供了新的工具。

QSAR 拟合的一致性预测方法的开发与评估

Development and Evaluation of Conformal Prediction Methods for QSAR

The recent decade has seen an enormous rise in the popularity of deep
learning and neural networks. These algorithms have broken many previous
records and achieved remarkable results. Their outstanding performance has
significantly sped up the progress of AI, and so far various milestones have
been achieved earlier than expected. However, in the case of relatively small
datasets, the performance of Deep Neural Networks (DNN) may suffer from reduced
accuracy compared to other Machine Learning models. Furthermore, it is
difficult to construct prediction intervals or evaluate the uncertainty of
predictions when dealing with regression tasks. In this paper, we propose an
ensemble method that attempts to estimate the uncertainty of predictions,
increase their accuracy and provide an interval for the expected variation.
Compared with traditional DNNs that only provide a prediction, our proposed
method can output a prediction interval by combining DNNs, extreme gradient
boosting (XGBoost) and dissimilarity computation techniques. Albeit the simple
design, this approach significantly increases accuracy on small datasets and
does not introduce much complexity to the architecture of the neural network.
The proposed method is tested on various datasets, and a significant
improvement in the performance of the neural network model is seen. The model's
prediction interval can include the ground truth value at an average rate of
71% and 78% across training sizes of 90% and 55%, respectively. Finally, we
highlight other aspects and applications of the approach in experimental error
estimation, and the application of transfer learning.

本文介绍了一种组合 DNN、XGBoost 和差异性计算技术的集成方法，用于估计预测的不确定性、提高准确性并提供预期变化的区间，其计算简单，适用于小数据集且性能提升显著，且模型的预测区间可平均包括实际值的 71％和 78％。