Capturing intricate and subtle variations in human expressiveness in music
performance using computational approaches is challenging. In this paper, we
propose a novel approach for reconstructing human expressiveness in piano
performance with a multi-layer bi-directional Transformer encoder. To address
the needs for large amounts of accurately captured and score-aligned
performance data in training neural networks, we use transcribed scores
obtained from an existing transcription model to train our model. We integrate
pianist identities to control the sampling process and explore the ability of
our system to model variations in expressiveness for different pianists. The
system is evaluated through statistical analysis of generated expressive
performances and a listening test. Overall, the results suggest that our method
achieves state-of-the-art in generating human-like piano performances from
transcribed scores, while fully and consistently reconstructing human
expressiveness poses further challenges.

本文提出了一种使用多层双向 Transformer 编码器重建钢琴演奏人类表现力的新方法，并使用现有的转录模型得到的转录乐谱来训练我们的模型，在此基础上通过控制采样过程中的钢琴家身份探索我们系统模拟不同钢琴家表现力差异的能力，在统计生成的表现力强的演奏和听力测试中对该系统进行评估，结果表明我们的方法实现了从转录分数生成类似于人类钢琴演奏的结果的最新技术水平，同时全面且一致地重建了人类的表现力，这进一步提出了挑战。

利用 Transformer 网络重建钢琴演奏中的人类表现力

Reconstructing Human Expressiveness in Piano Performances with a  Transformer Network

Motivated by the state-of-art psychological research, we note that a piano
performance transcribed with existing Automatic Music Transcription (AMT)
methods cannot be successfully resynthesized without affecting the artistic
content of the performance. This is due to 1) the different mappings between
MIDI parameters used by different instruments, and 2) the fact that musicians
adapt their way of playing to the surrounding acoustic environment. To face
this issue, we propose a methodology to build acoustics-specific AMT systems
that are able to model the adaptations that musicians apply to convey their
interpretation. Specifically, we train models tailored for virtual instruments
in a modular architecture that takes as input an audio recording and the
relative aligned music score, and outputs the acoustics-specific velocities of
each note. We test different model shapes and show that the proposed
methodology generally outperforms the usual AMT pipeline which does not
consider specificities of the instrument and of the acoustic environment.
Interestingly, such a methodology is extensible in a straightforward way since
only slight efforts are required to train models for the inference of other
piano parameters, such as pedaling.

通过建立具有特定声学特性的 AMT 系统，该系统以音频记录和相对齐奏乐谱为输入，并输出每个音符的声学特性速度，以应对现有 AMT 方法无法完整还原钢琴演奏的问题。