K-nearest neighbors (KNN) is one of the earliest and most established
algorithms in machine learning. For regression tasks, KNN averages the targets
within a neighborhood which poses a number of challenges: the neighborhood
definition is crucial for the predictive performance as neighbors might be
selected based on uninformative features, and averaging does not account for
how the function changes locally. We propose a novel method called Differential
Nearest Neighbors Regression (DNNR) that addresses both issues simultaneously:
during training, DNNR estimates local gradients to scale the features; during
inference, it performs an n-th order Taylor approximation using estimated
gradients. In a large-scale evaluation on over 250 datasets, we find that DNNR
performs comparably to state-of-the-art gradient boosting methods and MLPs
while maintaining the simplicity and transparency of KNN. This allows us to
derive theoretical error bounds and inspect failures. In times that call for
transparency of ML models, DNNR provides a good balance between performance and
interpretability.

本研究提出一种名为 Differential Nearest Neighbors Regression (DNNR) 的新方法，通过在训练期间估计局部梯度并在推断期间使用估计的梯度执行 n 阶泰勒逼近。在超过 250 个数据集的大规模评估中，我们发现，DNNR 在保持 KNN 的简单性和透明度的同时，表现与最先进的梯度提升方法和 MLP 相当，允许我们导出理论误差界并检查失败，从而在需要 ML 模型透明度的时代提供了绩效和可解释性之间的良好平衡。