Portfolio management (PM) is a fundamental financial planning task that aims
to achieve investment goals such as maximal profits or minimal risks. Its
decision process involves continuous derivation of valuable information from
various data sources and sequential decision optimization, which is a
prospective research direction for reinforcement learning (RL). In this paper,
we propose SARL, a novel State-Augmented RL framework for PM. Our framework
aims to address two unique challenges in financial PM: (1) data heterogeneity
-- the collected information for each asset is usually diverse, noisy and
imbalanced (e.g., news articles); and (2) environment uncertainty -- the
financial market is versatile and non-stationary. To incorporate heterogeneous
data and enhance robustness against environment uncertainty, our SARL augments
the asset information with their price movement prediction as additional
states, where the prediction can be solely based on financial data (e.g., asset
prices) or derived from alternative sources such as news. Experiments on two
real-world datasets, (i) Bitcoin market and (ii) HighTech stock market with
7-year Reuters news articles, validate the effectiveness of SARL over existing
PM approaches, both in terms of accumulated profits and risk-adjusted profits.
Moreover, extensive simulations are conducted to demonstrate the importance of
our proposed state augmentation, providing new insights and boosting
performance significantly over standard RL-based PM method and other baselines.

提出了一种基于强化学习的框架（SARL）来应对金融投资组合管理中数据异构性和环境不确定性的问题，实验结果表明，该框架可以提高投资组合的累积收益和风险调整后的利润。