Action Quality Assessment(AQA) is important for action understanding and
resolving the task poses unique challenges due to subtle visual differences.
Existing state-of-the-art methods typically rely on the holistic video
representations for score regression or ranking, which limits the
generalization to capture fine-grained intra-class variation. To overcome the
above limitation, we propose a temporal parsing transformer to decompose the
holistic feature into temporal part-level representations. Specifically, we
utilize a set of learnable queries to represent the atomic temporal patterns
for a specific action. Our decoding process converts the frame representations
to a fixed number of temporally ordered part representations. To obtain the
quality score, we adopt the state-of-the-art contrastive regression based on
the part representations. Since existing AQA datasets do not provide temporal
part-level labels or partitions, we propose two novel loss functions on the
cross attention responses of the decoder: a ranking loss to ensure the
learnable queries to satisfy the temporal order in cross attention and a
sparsity loss to encourage the part representations to be more discriminative.
Extensive experiments show that our proposed method outperforms prior work on
three public AQA benchmarks by a considerable margin.

通过时间解析变压器将运动分解为时间部分级表示，采用对比回归与分级损失函数用于动作质量评估，提高了当前方法在多个 AQA 基准测试中的性能。