In this paper, we report the performance benchmarking results of deep
learning models on MLCommons' Science cloud-masking benchmark using a
high-performance computing cluster at New York University (NYU): NYU Greene.
MLCommons is a consortium that develops and maintains several scientific
benchmarks that can benefit from developments in AI. We provide a description
of the cloud-masking benchmark task, updated code, and the best model for this
benchmark when using our selected hyperparameter settings. Our benchmarking
results include the highest accuracy achieved on the NYU system as well as the
average time taken for both training and inference on the benchmark across
several runs/seeds. Our code can be found on GitHub. MLCommons team has been
kept informed about our progress and may use the developed code for their
future work.

使用纽约大学（NYU）的高性能计算集群 NYU Greene，在 MLCommons 的科学云掩膜基准测试中，报告了深度学习模型的性能基准测试结果：包括基准测试任务的描述、更新的代码以及在我们选定的超参数设置下，该基准测试的最佳模型。基准测试结果包括在 NYU 系统上实现的最高准确率以及训练和推理的平均耗时。我们的代码可在 GitHub 上找到，MLCommons 团队已知悉我们的进展并可能在未来的工作中使用我们开发的代码。

MLCommons CloudMask 基准测试的改进与评估

Improvements & Evaluations on the MLCommons CloudMask Benchmark

In this paper, we report on work performed for the MLCommons Science Working
Group on the cloud masking benchmark. MLCommons is a consortium that develops
and maintains several scientific benchmarks that aim to benefit developments in
AI. The benchmarks are conducted on the High Performance Computing (HPC)
Clusters of New York University and University of Virginia, as well as a
commodity desktop. We provide a description of the cloud masking benchmark, as
well as a summary of our submission to MLCommons on the benchmark experiment we
conducted. It includes a modification to the reference implementation of the
cloud masking benchmark enabling early stopping. This benchmark is executed on
the NYU HPC through a custom batch script that runs the various experiments
through the batch queuing system while allowing for variation on the number of
epochs trained. Our submission includes the modified code, a custom batch
script to modify epochs, documentation, and the benchmark results. We report
the highest accuracy (scientific metric) and the average time taken
(performance metric) for training and inference that was achieved on NYU HPC
Greene. We also provide a comparison of the compute capabilities between
different systems by running the benchmark for one epoch. Our submission can be
found in a Globus repository that is accessible to MLCommons Science Working
Group.

MLCommons 科学工作组提交了关于云掩膜基准测试的研究报告，其中包括对基准试验进行了早停机制的改进，并在 NYU HPC 上获得了最高的准确度和平均训练与推断时间。