Deep neural networks (DNNs) have proven to be effective models for accurate
Memory Access Prediction (MAP), a critical task in mitigating memory latency
through data prefetching. However, existing DNN-based MAP models suffer from
the challenges such as significant physical storage space and poor inference
latency, primarily due to their large number of parameters. These limitations
render them impractical for deployment in real-world scenarios. In this paper,
we propose PaCKD, a Pattern-Clustered Knowledge Distillation approach to
compress MAP models while maintaining the prediction performance. The PaCKD
approach encompasses three steps: clustering memory access sequences into
distinct partitions involving similar patterns, training large pattern-specific
teacher models for memory access prediction for each partition, and training a
single lightweight student model by distilling the knowledge from the trained
pattern-specific teachers. We evaluate our approach on LSTM, MLP-Mixer, and
ResNet models, as they exhibit diverse structures and are widely used for image
classification tasks in order to test their effectiveness in four widely used
graph applications. Compared to the teacher models with 5.406M parameters and
an F1-score of 0.4626, our student models achieve a 552$\times$ model size
compression while maintaining an F1-score of 0.4538 (with a 1.92% performance
drop). Our approach yields an 8.70% higher result compared to student models
trained with standard knowledge distillation and an 8.88% higher result
compared to student models trained without any form of knowledge distillation.

使用模式聚类知识蒸馏方法，我们提出了 PaCKD，这是一种压缩内存访问预测模型的方法，通过为每个分区训练大型模式特定的教师模型，然后从这些训练好的模式特定教师中提取知识训练一个单一轻量级的学生模型，以在保持预测性能的同时实现模型大小的 552 倍压缩。

PaCKD: 模式聚类知识蒸馏用于压缩内存访问预测模型

PaCKD: Pattern-Clustered Knowledge Distillation for Compressing Memory  Access Prediction Models

Attention-based Neural Networks (NN) have demonstrated their effectiveness in
accurate memory access prediction, an essential step in data prefetching.
However, the substantial computational overheads associated with these models
result in high inference latency, limiting their feasibility as practical
prefetchers. To close the gap, we propose a new approach based on
tabularization that significantly reduces model complexity and inference
latency without sacrificing prediction accuracy. Our novel tabularization
methodology takes as input a distilled, yet highly accurate attention-based
model for memory access prediction and efficiently converts its expensive
matrix multiplications into a hierarchy of fast table lookups. As an exemplar
of the above approach, we develop DART, a prefetcher comprised of a simple
hierarchy of tables. With a modest 0.09 drop in F1-score, DART reduces 99.99%
of arithmetic operations from the large attention-based model and 91.83% from
the distilled model. DART accelerates the large model inference by 170x and the
distilled model by 9.4x. DART has comparable latency and storage costs as
state-of-the-art rule-based prefetcher BO but surpasses it by 6.1% in IPC
improvement, resulting in a 37.6% speed-up. DART outperforms state-of-the-art
NN-based prefetchers TransFetch by 33.1% and Voyager by 37.2% in terms of IPC
improvement, primarily due to its low prefetching latency.

提出了一种基于表格化的新方法，通过将注意力机制模型的昂贵矩阵乘法转化为快速查找表的层次结构，显著降低了模型复杂性和推理延迟，并在保持预测准确性的情况下进行了内存访问预测。该方法在性能方面超过了基于规则的最先进预取器，且与基于神经网络的最先进预取器相比，具有更低的预取延迟，从而实现了速度的提升。