The recent decade has seen an enormous rise in the popularity of deep learning and neural networks. These algorithms have broken many previous records and achieved remarkable results. Their outstanding performance has significantly sped up the progress of AI, and so far various milestones have been achieved earlier than expected. However, in the case of relatively small datasets, the performance of Deep Neural Networks (DNN) may suffer from reduced accuracy compared to other Machine Learning models. Furthermore, it is difficult to construct prediction intervals or evaluate the uncertainty of predictions when dealing with regression tasks. In this paper, we propose an ensemble method that attempts to estimate the uncertainty of predictions, increase their accuracy and provide an interval for the expected variation. Compared with traditional DNNs that only provide a prediction, our proposed method can output a prediction interval by combining DNNs, extreme gradient boosting (XGBoost) and dissimilarity computation techniques. Albeit the simple design, this approach significantly increases accuracy on small datasets and does not introduce much complexity to the architecture of the neural network. The proposed method is tested on various datasets, and a significant improvement in the performance of the neural network model is seen. The model's prediction interval can include the ground truth value at an average rate of 71% and 78% across training sizes of 90% and 55%, respectively. Finally, we highlight other aspects and applications of the approach in experimental error estimation, and the application of transfer learning.

本文介绍了一种组合DNN、XGBoost和差异性计算技术的集成方法，用于估计预测的不确定性、提高准确性并提供预期变化的区间，其计算简单，适用于小数据集且性能提升显著，且模型的预测区间可平均包括实际值的71％和78％。

Confidence-Nets: 面向小数据集回归神经网络更好预测区间的一步