High-speed research networks are built to meet the ever-increasing needs of
data-intensive distributed workflows. However, data transfers in these networks
often fail to attain the promised transfer rates for several reasons, including
I/O and network interference, server misconfigurations, and network anomalies.
Although understanding the root causes of performance issues is critical to
mitigating them and increasing the utilization of expensive network
infrastructures, there is currently no available mechanism to monitor data
transfers in these networks. In this paper, we present a scalable, end-to-end
monitoring framework to gather and store key performance metrics for file
transfers to shed light on the performance of transfers. The evaluation results
show that the proposed framework can monitor up to 400 transfers per host and
more than 40, 000 transfers in total while collecting performance statistics at
one-second precision. We also introduce a heuristic method to automatically
process the gathered performance metrics and identify the root causes of
performance anomalies with an F-score of 87 - 98%.

构建用于满足数据密集型分布式工作流需求的高速研究网络，但是由于多种原因，包括 I/O 和网络干扰、服务器配置错误和网络异常，这些网络中的数据传输经常无法达到承诺的传输速率。本文提出了一个可扩展的端到端监测框架，用于收集和存储文件传输的关键性能指标，以明确传输性能。评估结果表明，该提议的框架可以监测每个主机多达 400 个传输和总共超过 40,000 个传输，同时以一秒的精度收集性能统计数据。我们还引入了一种启发式方法，能够自动处理收集到的性能指标，并以 87-98% 的 F-score 识别性能异常的根本原因。