Dynamic portfolio optimization is the process of sequentially allocating
wealth to a collection of assets in some consecutive trading periods, based on
investors' return-risk profile. Automating this process with machine learning
remains a challenging problem. Here, we design a deep reinforcement learning
(RL) architecture with an autonomous trading agent such that, investment
decisions and actions are made periodically, based on a global objective, with
autonomy. In particular, without relying on a purely model-free RL agent, we
train our trading agent using a novel RL architecture consisting of an infused
prediction module (IPM), a generative adversarial data augmentation module
(DAM) and a behavior cloning module (BCM). Our model-based approach works with
both on-policy or off-policy RL algorithms. We further design the back-testing
and execution engine which interact with the RL agent in real time. Using
historical {\em real} financial market data, we simulate trading with practical
constraints, and demonstrate that our proposed model is robust, profitable and
risk-sensitive, as compared to baseline trading strategies and model-free RL
agents from prior work.

使用深层强化学习的自主交易代理设计动态投资组合优化模型，通过模型无关机器学习模块、生成式对抗数据增强模块以及行为克隆模块实现训练，对历史金融市场数据进行模拟交易，证明该模型相较于先前的交易策略是稳健的、盈利的和风险敏感的。