In recent years, genetic programming (GP)-based evolutionary feature
construction has achieved significant success. However, a primary challenge
with evolutionary feature construction is its tendency to overfit the training
data, resulting in poor generalization on unseen data. In this research, we
draw inspiration from PAC-Bayesian theory and propose using sharpness-aware
minimization in function space to discover symbolic features that exhibit
robust performance within a smooth loss landscape in the semantic space. By
optimizing sharpness in conjunction with cross-validation loss, as well as
designing a sharpness reduction layer, the proposed method effectively
mitigates the overfitting problem of GP, especially when dealing with a limited
number of instances or in the presence of label noise. Experimental results on
58 real-world regression datasets show that our approach outperforms standard
GP as well as six state-of-the-art complexity measurement methods for GP in
controlling overfitting. Furthermore, the ensemble version of GP with
sharpness-aware minimization demonstrates superior performance compared to nine
fine-tuned machine learning and symbolic regression algorithms, including
XGBoost and LightGBM.

通过使用 PAC-Bayesian 理论中的 sharpness-aware 最小化方法，在语义空间中寻找表现稳健的符号特征，该方法能够有效减轻遗传规划的过拟合问题，并在限定实例数量和标签噪声存在的情况下表现优于标准遗传规划和其他先进的复杂度测量方法。