Tensor rematerialization allows the training of deep neural networks (DNNs)
under limited memory budgets by checkpointing the models and recomputing the
evicted tensors as needed. However, the existing tensor rematerialization
techniques overlook the memory system in deep learning frameworks and
implicitly assume that free memory blocks at different addresses are identical.
Under this flawed assumption, discontiguous tensors are evicted, among which
some are not used to allocate the new tensor. This leads to severe memory
fragmentation and increases the cost of potential rematerializations. To
address this issue, we propose to evict tensors within a sliding window to
ensure all evictions are contiguous and are immediately used. Furthermore, we
proposed cheap tensor partitioning and recomputable in-place to further reduce
the rematerialization cost by optimizing the tensor allocation. We named our
method Coop as it is a co-optimization of tensor allocation and tensor
rematerialization. We evaluated Coop on eight representative DNNs. The
experimental results demonstrate that Coop achieves up to $2\times$ memory
saving and hugely reduces compute overhead, search latency, and memory
fragmentation compared to the state-of-the-art baselines.

深度学习框架中的张量再材料化方法 Coop 通过优化张量分配和再材料化，实现了多达 2 倍的内存节省，大大减少了计算开销、搜索延迟和内存碎片化。

合作：记忆不是一种商品

Coop: Memory is not a Commodity

The deployment and training of neural networks on edge computing devices pose
many challenges. The low memory nature of edge devices is often one of the
biggest limiting factors encountered in the deployment of large neural network
models. Tensor rematerialization or recompute is a way to address high memory
requirements for neural network training and inference. In this paper we
consider the problem of execution time minimization of compute graphs subject
to a memory budget. In particular, we develop a new constraint programming
formulation called \textsc{Moccasin} with only $O(n)$ integer variables, where
$n$ is the number of nodes in the compute graph. This is a significant
improvement over the works in the recent literature that propose formulations
with $O(n^2)$ Boolean variables. We present numerical studies that show that
our approach is up to an order of magnitude faster than recent work especially
for large-scale graphs.

本文讨论了在边缘计算设备上部署和训练神经网络所面临的诸多挑战，提出了一种名为 Moccasin 的新约束编程公式来最小化计算图的执行时间，达到内存预算的目的，并在大规模图中比最近的工作快一个数量级。