We introduce the "cram" method, a general and efficient approach to
simultaneous learning and evaluation using a generic machine learning (ML)
algorithm. In a single pass of batched data, the proposed method repeatedly
trains an ML algorithm and tests its empirical performance. Because it utilizes
the entire sample for both learning and evaluation, cramming is significantly
more data-efficient than sample-splitting. The cram method also naturally
accommodates online learning algorithms, making its implementation
computationally efficient. To demonstrate the power of the cram method, we
consider the standard policy learning setting where cramming is applied to the
same data to both develop an individualized treatment rule (ITR) and estimate
the average outcome that would result if the learned ITR were to be deployed.
We show that under a minimal set of assumptions, the resulting crammed
evaluation estimator is consistent and asymptotically normal. While our
asymptotic results require a relatively weak stabilization condition of ML
algorithm, we develop a simple, generic method that can be used with any policy
learning algorithm to satisfy this condition. Our extensive simulation studies
show that, when compared to sample-splitting, cramming reduces the evaluation
standard error by more than 40% while improving the performance of learned
policy. We also apply the cram method to a randomized clinical trial to
demonstrate its applicability to real-world problems. Finally, we briefly
discuss future extensions of the cram method to other learning and evaluation
settings.

引入了 “cram” 方法，一种通用且高效的方法，用于使用通用的机器学习算法进行同时学习和评估。我们展示了该方法在政策学习环境中的应用，证明了该方法的一致性和渐进正态性。还通过广泛的模拟研究和针对实际问题的随机临床试验应用了该方法，并讨论了对其他学习和评估设置的未来扩展。