Deep reinforcement learning (DRL) has been envisioned to have a competitive
edge in quantitative finance. However, there is a steep development curve for
quantitative traders to obtain an agent that automatically positions to win in
the market, namely \textit{to decide where to trade, at what price} and
\textit{what quantity}, due to the error-prone programming and arduous
debugging. In this paper, we present the first open-source framework
\textit{FinRL} as a full pipeline to help quantitative traders overcome the
steep learning curve. FinRL is featured with simplicity, applicability and
extensibility under the key principles, \textit{full-stack framework,
customization, reproducibility} and \textit{hands-on tutoring}.
Embodied as a three-layer architecture with modular structures, FinRL
implements fine-tuned state-of-the-art DRL algorithms and common reward
functions, while alleviating the debugging workloads. Thus, we help users
pipeline the strategy design at a high turnover rate. At multiple levels of
time granularity, FinRL simulates various markets as training environments
using historical data and live trading APIs. Being highly extensible, FinRL
reserves a set of user-import interfaces and incorporates trading constraints
such as market friction, market liquidity and investor's risk-aversion.
Moreover, serving as practitioners' stepping stones, typical trading tasks are
provided as step-by-step tutorials, e.g., stock trading, portfolio allocation,
cryptocurrency trading, etc.

本文介绍了一种名为 FinRL 的开源框架，它利用深度强化学习算法实现智能交易，精细调整了当前最先进的 DRL 算法和通用奖励函数，并提供了多个精细化的教程，方便用户进行自定义操作。

FinRL：量化交易自动化的深度强化学习框架

FinRL: Deep Reinforcement Learning Framework to Automate Trading in  Quantitative Finance

Financial portfolio management is the process of constant redistribution of a
fund into different financial products. This paper presents a
financial-model-free Reinforcement Learning framework to provide a deep machine
learning solution to the portfolio management problem. The framework consists
of the Ensemble of Identical Independent Evaluators (EIIE) topology, a
Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL)
scheme, and a fully exploiting and explicit reward function. This framework is
realized in three instants in this work with a Convolutional Neural Network
(CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory
(LSTM). They are, along with a number of recently reviewed or published
portfolio-selection strategies, examined in three back-test experiments with a
trading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are
electronic and decentralized alternatives to government-issued money, with
Bitcoin as the best-known example of a cryptocurrency. All three instances of
the framework monopolize the top three positions in all experiments,
outdistancing other compared trading algorithms. Although with a high
commission rate of 0.25% in the backtests, the framework is able to achieve at
least 4-fold returns in 50 days.

本文提出了一个具有 EIIE 拓扑结构、PVM、OSBL 方案和完全利用和显式奖励函数的基于深度强化学习的组合策略金融投资组合管理框架，该框架需要使用 CNN、RNN 或 LSTM，通过在加密货币市场上进行三次回测实验，与其他相比，实现了最好的投资回报。