The rapid spread of the COVID-19 pandemic has resulted in an unprecedented
amount of sequence data of the SARS-CoV-2 genome -- millions of sequences and
counting. This amount of data, while being orders of magnitude beyond the
capacity of traditional approaches to understanding the diversity, dynamics,
and evolution of viruses is nonetheless a rich resource for machine learning
(ML) approaches as alternatives for extracting such important information from
these data. It is of hence utmost importance to design a framework for testing
and benchmarking the robustness of these ML models.
This paper makes the first effort (to our knowledge) to benchmark the
robustness of ML models by simulating biological sequences with errors. In this
paper, we introduce several ways to perturb SARS-CoV-2 genome sequences to
mimic the error profiles of common sequencing platforms such as Illumina and
PacBio. We show from experiments on a wide array of ML models that some
simulation-based approaches are more robust (and accurate) than others for
specific embedding methods to certain adversarial attacks to the input
sequences. Our benchmarking framework may assist researchers in properly
assessing different ML models and help them understand the behavior of the
SARS-CoV-2 virus or avoid possible future pandemics.

本文首次尝试通过模拟具有错误的生物序列来测试机器学习模型的鲁棒性，并介绍了几种扰动 SARS-CoV-2 基因组序列以模拟 Illumina、PacBio 等常见测序平台的错误数据的方法。实验表明，针对特定嵌入方法，某些基于模拟的方法对输入序列的某些对抗攻击更为鲁棒（且准确）。引入的基准测试框架可帮助研究者正确评估不同的机器学习模型，并帮助他们更好地理解 SARS-CoV-2 病毒的行为或避免可能的未来大流行病。