In this paper, we report on work performed for the MLCommons Science Working
Group on the cloud masking benchmark. MLCommons is a consortium that develops
and maintains several scientific benchmarks that aim to benefit developments in
AI. The benchmarks are conducted on the High Performance Computing (HPC)
Clusters of New York University and University of Virginia, as well as a
commodity desktop. We provide a description of the cloud masking benchmark, as
well as a summary of our submission to MLCommons on the benchmark experiment we
conducted. It includes a modification to the reference implementation of the
cloud masking benchmark enabling early stopping. This benchmark is executed on
the NYU HPC through a custom batch script that runs the various experiments
through the batch queuing system while allowing for variation on the number of
epochs trained. Our submission includes the modified code, a custom batch
script to modify epochs, documentation, and the benchmark results. We report
the highest accuracy (scientific metric) and the average time taken
(performance metric) for training and inference that was achieved on NYU HPC
Greene. We also provide a comparison of the compute capabilities between
different systems by running the benchmark for one epoch. Our submission can be
found in a Globus repository that is accessible to MLCommons Science Working
Group.

MLCommons 科学工作组提交了关于云掩膜基准测试的研究报告，其中包括对基准试验进行了早停机制的改进，并在 NYU HPC 上获得了最高的准确度和平均训练与推断时间。