In recent years, machine learning, and in particular generative adversarial
neural networks (GANs) and attention-based neural networks (transformers), have
been successfully used to compose and generate music, both melodies and
polyphonic pieces. Current research focuses foremost on style replication (eg.
generating a Bach-style chorale) or style transfer (eg. classical to jazz)
based on large amounts of recorded or transcribed music, which in turn also
allows for fairly straight-forward "performance" evaluation. However, most of
these models are not suitable for human-machine co-creation through live
interaction, neither is clear, how such models and resulting creations would be
evaluated. This article presents a thorough review of music representation,
feature analysis, heuristic algorithms, statistical and parametric modelling,
and human and automatic evaluation measures, along with a discussion of which
approaches and models seem most suitable for live interaction.

近年来，机器学习，特别是生成对抗神经网络（GANs）和基于注意力的神经网络（transformers），已成功用于作曲和生成音乐，包括旋律和多声部作品。然而，现有研究主要集中在风格复制和转换的问题上，并未涉及到人机共同创作和评估。本文综述了音乐表征、特征分析、启发式算法、统计和参数建模，人类和自动化评估措施，并讨论了哪种方法和模型最适合于实时互动。

互动背景下的音乐生成调查

A Survey of Music Generation in the Context of Interaction

Attention-based Neural Networks (NN) have demonstrated their effectiveness in
accurate memory access prediction, an essential step in data prefetching.
However, the substantial computational overheads associated with these models
result in high inference latency, limiting their feasibility as practical
prefetchers. To close the gap, we propose a new approach based on
tabularization that significantly reduces model complexity and inference
latency without sacrificing prediction accuracy. Our novel tabularization
methodology takes as input a distilled, yet highly accurate attention-based
model for memory access prediction and efficiently converts its expensive
matrix multiplications into a hierarchy of fast table lookups. As an exemplar
of the above approach, we develop DART, a prefetcher comprised of a simple
hierarchy of tables. With a modest 0.09 drop in F1-score, DART reduces 99.99%
of arithmetic operations from the large attention-based model and 91.83% from
the distilled model. DART accelerates the large model inference by 170x and the
distilled model by 9.4x. DART has comparable latency and storage costs as
state-of-the-art rule-based prefetcher BO but surpasses it by 6.1% in IPC
improvement, resulting in a 37.6% speed-up. DART outperforms state-of-the-art
NN-based prefetchers TransFetch by 33.1% and Voyager by 37.2% in terms of IPC
improvement, primarily due to its low prefetching latency.

提出了一种基于表格化的新方法，通过将注意力机制模型的昂贵矩阵乘法转化为快速查找表的层次结构，显著降低了模型复杂性和推理延迟，并在保持预测准确性的情况下进行了内存访问预测。该方法在性能方面超过了基于规则的最先进预取器，且与基于神经网络的最先进预取器相比，具有更低的预取延迟，从而实现了速度的提升。

关注、蒸馏和表格化：面向实用的基于神经网络的预取

Attention, Distillation, and Tabularization: Towards Practical Neural  Network-Based Prefetching

Recently, neural networks purely based on attention were shown to address
image understanding tasks such as image classification. However, these visual
transformers are pre-trained with hundreds of millions of images using an
expensive infrastructure, thereby limiting their adoption.
In this work, we produce a competitive convolution-free transformer by
training on Imagenet only. We train them on a single computer in less than 3
days. Our reference vision transformer (86M parameters) achieves top-1 accuracy
of 83.1% (single-crop evaluation) on ImageNet with no external data.
More importantly, we introduce a teacher-student strategy specific to
transformers. It relies on a distillation token ensuring that the student
learns from the teacher through attention. We show the interest of this
token-based distillation, especially when using a convnet as a teacher. This
leads us to report results competitive with convnets for both Imagenet (where
we obtain up to 85.2% accuracy) and when transferring to other tasks. We share
our code and models.

通过使用注意力机制的神经网络，我们以卷积神经网络为教师，在 ImageNet 上训练单个计算机下的优秀可比拟的转换器，并引入了基于蒸馏令牌的教师 - 学生策略，以获得竞争性结果。