Click-Through Rate (CTR) prediction holds paramount significance in online advertising and recommendation scenarios. Despite the proliferation of recent CTR prediction models, the improvements in performance have remained limited, as evidenced by open-source benchmark assessments. Current researchers tend to focus on developing new models for various datasets and settings, often neglecting a crucial question: What is the key challenge that truly makes CTR prediction so demanding? In this paper, we approach the problem of CTR prediction from an optimization perspective. We explore the typical data characteristics and optimization statistics of CTR prediction, revealing a strong positive correlation between the top hessian eigenvalue and feature frequency. This correlation implies that frequently occurring features tend to converge towards sharp local minima, ultimately leading to suboptimal performance. Motivated by the recent advancements in sharpness-aware minimization (SAM), which considers the geometric aspects of the loss landscape during optimization, we present a dedicated optimizer crafted for CTR prediction, named Helen. Helen incorporates frequency-wise Hessian eigenvalue regularization, achieved through adaptive perturbations based on normalized feature frequencies. Empirical results under the open-source benchmark framework underscore Helen's effectiveness. It successfully constrains the top eigenvalue of the Hessian matrix and demonstrates a clear advantage over widely used optimization algorithms when applied to seven popular models across three public benchmark datasets on BARS. Our code locates at github.com/NUS-HPC-AI-Lab/Helen.

通过优化的角度来解决CTR预测问题，研究发现CTR预测中最大的挑战在于特征频率与最大海森矩阵特征值之间的正相关关系，即频繁出现的特征倾向于收敛于尖锐的局部最小值，从而导致亚优化性能。基于锐度感知最小化的最新进展，我们提出了一种专门为CTR预测设计的优化器，命名为Helen。Helen通过基于归一化特征频率的自适应扰动来实现特征频率相关的海森矩阵特征值正则化。在开源基准框架下的实证结果突显了Helen的有效性。它成功地约束了海森矩阵的最大特征值，并在三个公共基准数据集上应用于七种流行模型时，与广泛使用的优化算法相比具有明显优势。我们的代码位于github.com/NUS-HPC-AI-Lab/Helen。

Helen: 频率导向Hessian特征值正则化优化CTR预测模型