Recent geospatial machine learning studies have shown that the results of
model evaluation via cross-validation (CV) are strongly affected by the
dissimilarity between the sample data and the prediction locations. In this
paper, we propose a method to quantify such a dissimilarity in the interval 0
to 100%, and from the perspective of the data feature space. The proposed
method is based on adversarial validation, which is an approach that can check
whether sample data and prediction locations can be separated with a binary
classifier. To study the effectiveness and generality of our method, we tested
it on a series of experiments based on both synthetic and real datasets and
with gradually increasing dissimilarities. Results show that the proposed
method can successfully quantify dissimilarity across the entire range of
values. Next to this, we studied how dissimilarity affects CV evaluations by
comparing the results of random CV and of two spatial CV methods, namely block
and spatial+ CV. Our results showed that CV evaluations follow similar patterns
in all datasets and predictions: when dissimilarity is low (usually lower than
30%), random CV provides the most accurate evaluation results. As dissimilarity
increases, spatial CV methods, especially spatial+ CV, become more and more
accurate and even outperforming random CV. When dissimilarity is high (>=90%),
no CV method provides accurate evaluations. These results show the importance
of considering feature space dissimilarity when working with geospatial machine
learning predictions, and can help researchers and practitioners to select more
suitable CV methods for evaluating their predictions.

通过对地理空间机器学习预测结果和样本数据之间的差异度进行量化，在 0 到 100% 的区间内，基于对抗验证的方法可以准确度量差异度。此方法在合成和真实的数据集上测试，并逐渐增加差异度来研究其效果和普适性。测试结果表明该方法可以成功地量化整个数值范围内的差异度，并揭示出在地理空间机器学习预测中考虑特征空间差异度的重要性，以便选择更适合的交叉验证方法进行预测评估。

利用对抗验证量化地理空间机器学习预测的差异性

On the use of adversarial validation for quantifying dissimilarity in  geospatial machine learning prediction

The ACM RecSys Challenge 2023, organized by ShareChat, aims to predict the
probability of the app being installed. This paper describes the lightweight
solution to this challenge. We formulate the task as a user response prediction
task. For rapid prototyping for the task, we propose a lightweight solution
including the following steps: 1) using adversarial validation, we effectively
eliminate uninformative features from a dataset; 2) to address noisy continuous
features and categorical features with a large number of unique values, we
employ feature engineering techniques.; 3) we leverage Gradient Boosted
Decision Trees (GBDT) for their exceptional performance and scalability. The
experiments show that a single LightGBM model, without additional ensembling,
performs quite well. Our team achieved ninth place in the challenge with the
final leaderboard score of 6.059065. Code for our approach can be found here:
this https URL

本文描述了 ACM RecSys Challenge 2023 的轻量级解决方案，将任务形式化为用户响应预测任务，并通过敌对验证、特征工程技术和梯度提升决策树等方法有效地解决了问题。实验证明单个 LightGBM 模型表现出色，无需额外集成。