Experimental studies are a cornerstone of machine learning (ML) research. A
common, but often implicit, assumption is that the results of a study will
generalize beyond the study itself, e.g. to new data. That is, there is a high
probability that repeating the study under different conditions will yield
similar results. Despite the importance of the concept, the problem of
measuring generalizability remains open. This is probably due to the lack of a
mathematical formalization of experimental studies. In this paper, we propose
such a formalization and develop a quantifiable notion of generalizability.
This notion allows to explore the generalizability of existing studies and to
estimate the number of experiments needed to achieve the generalizability of
new studies. To demonstrate its usefulness, we apply it to two recently
published benchmarks to discern generalizable and non-generalizable results. We
also publish a Python module that allows our analysis to be repeated for other
experimental studies.

本文提出了实验研究的数学形式化方法，并发展了一个可量化的泛化概念，旨在探索现有研究的泛化性，并估计实现新研究泛化性所需的实验数量，为此我们应用两个最近发表的基准测试来区分可泛化和不可泛化的结果，并发布了一个 Python 模块，可以用于其他实验研究的分析。

实验研究的普遍适用性

Generalizability of experimental studies

Within the mathematical finance literature there is a rich catalogue of
mathematical models for studying algorithmic trading problems -- such as
market-making and optimal execution -- in limit order books. This paper
introduces \mbtgym, a Python module that provides a suite of gym environments
for training reinforcement learning (RL) agents to solve such model-based
trading problems. The module is set up in an extensible way to allow the
combination of different aspects of different models. It supports highly
efficient implementations of vectorized environments to allow faster training
of RL agents. In this paper, we motivate the challenge of using RL to solve
such model-based limit order book problems in mathematical finance, we explain
the design of our gym environment, and then demonstrate its use in solving
standard and non-standard problems from the literature. Finally, we lay out a
roadmap for further development of our module, which we provide as an open
source repository on GitHub so that it can serve as a focal point for RL
research in model-based algorithmic trading.

本文介绍了一种名为 mbtgym 的 Python 模块，其提供了一套 gym 环境，可用于训练强化学习代理解决基于模型的交易问题，该模块支持高效实现向量化环境以加快强化学习代理的训练速度。