Evaluation measures have a crucial impact on the direction of research. Therefore, it is of utmost importance to develop appropriate and reliable evaluation measures for new applications where conventional measures are not well suited. Video Moment Retrieval (VMR) is one such application, and the current practice is to use R@$K,\theta$ for evaluating VMR systems. However, this measure has two disadvantages. First, it is rank-insensitive: It ignores the rank positions of successfully localised moments in the top-$K$ ranked list by treating the list as a set. Second, it binarizes the Intersection over Union (IoU) of each retrieved video moment using the threshold $\theta$ and thereby ignoring fine-grained localisation quality of ranked moments. We propose an alternative measure for evaluating VMR, called Average Max IoU (AxIoU), which is free from the above two problems. We show that AxIoU satisfies two important axioms for VMR evaluation, namely, \textbf{Invariance against Redundant Moments} and \textbf{Monotonicity with respect to the Best Moment}, and also that R@$K,\theta$ satisfies the first axiom only. We also empirically examine how AxIoU agrees with R@$K,\theta$, as well as its stability with respect to change in the test data and human-annotated temporal boundaries.

本文提出一种新的 VMR（Video Moment Retrieval）评价指标 -- Average Max IoU，该指标不再受制于旧评价方法 R@$K,	heta$ 存在的问题，并满足了两个重要的 VMR 评价公理，即“Redundant Moments 不变性”和“Best Moment 单调性”。同时，作者也通过实验给出了新评价指标和旧评价方法在不同数据和时间边界下的表现比较。

AxIoU：一种公理上证明的视频时刻检索度量