Alphas are pivotal in providing signals for quantitative trading. The
industry highly values the discovery of formulaic alphas for their
interpretability and ease of analysis, compared with the expressive yet
overfitting-prone black-box alphas. In this work, we focus on discovering
formulaic alphas. Prior studies on automatically generating a collection of
formulaic alphas were mostly based on genetic programming (GP), which is known
to suffer from the problems of being sensitive to the initial population,
converting to local optima, and slow computation speed. Recent efforts
employing deep reinforcement learning (DRL) for alpha discovery have not fully
addressed key practical considerations such as alpha correlations and validity,
which are crucial for their effectiveness. In this work, we propose a novel
framework for alpha discovery using DRL by formulating the alpha discovery
process as program construction. Our agent, $\text{Alpha}^2$, assembles an
alpha program optimized for an evaluation metric. A search algorithm guided by
DRL navigates through the search space based on value estimates for potential
alpha outcomes. The evaluation metric encourages both the performance and the
diversity of alphas for a better final trading strategy. Our formulation of
searching alphas also brings the advantage of pre-calculation dimensional
analysis, ensuring the logical soundness of alphas, and pruning the vast search
space to a large extent. Empirical experiments on real-world stock markets
demonstrates $\text{Alpha}^2$'s capability to identify a diverse set of logical
and effective alphas, which significantly improves the performance of the final
trading strategy. The code of our method is available at
this https URL

我们提出了一种使用深度强化学习进行 alpha 发现的新框架，通过将 alpha 发现过程形式化为程序构建。我们的代理人引导的搜索算法根据潜在 alpha 结果的价值估计在搜索空间中导航。我们的实证实验表明，我们的方法能够识别出一组多样化、逻辑正确且有效的 alpha，显著提高了最终交易策略的性能。