Benefiting from high-quality datasets and standardized evaluation metrics,
machine learning (ML) has achieved sustained progress and widespread
applications. However, while applying machine learning to relational databases
(RDBs), the absence of a well-established benchmark remains a significant
obstacle to the development of ML. To address this issue, we introduce ML
Benchmark For Relational Databases (RDBench), a standardized benchmark that
aims to promote reproducible ML research on RDBs that include multiple tables.
RDBench offers diverse RDB datasets of varying scales, domains, and relational
structures, organized into 4 levels. Notably, to simplify the adoption of
RDBench for diverse ML domains, for any given database, RDBench exposes three
types of interfaces including tabular data, homogeneous graphs, and
heterogeneous graphs, sharing the same underlying task definition. For the
first time, RDBench enables meaningful comparisons between ML methods from
diverse domains, ranging from XGBoost to Graph Neural Networks, under RDB
prediction tasks. We design multiple classification and regression tasks for
each RDB dataset and report averaged results over the same dataset, further
enhancing the robustness of the experimental findings. RDBench is implemented
with DBGym, a user-friendly platform for ML research and application on
databases, enabling benchmarking new ML methods with RDBench at ease.

通过引入 ML Benchmark For Relational Databases (RDBench) 标准化基准测试，旨在促进多表关系数据库上可复现的机器学习研究，RDBench 为不同的机器学习领域提供多样的关系数据库数据集，使得从 XGBoost 到图神经网络等不同领域的 ML 方法可以进行有意义的比较和评估。