In recent decades, challenges have become very popular in scientific research
as these are crowdsourcing schemes. In particular, challenges are essential for
developing machine learning algorithms. For the challenges settings, it is
vital to establish the scientific question, the dataset (with adequate quality,
quantity, diversity, and complexity), performance metrics, as well as a way to
authenticate the participants' results (Gold Standard). This paper addresses
the problem of evaluating the performance of different competitors (algorithms)
under the restrictions imposed by the challenge scheme, such as the comparison
of multiple competitors with a unique dataset (with fixed size), a minimal
number of submissions and, a set of metrics chosen to assess performance. The
algorithms are sorted according to the performance metric. Still, it is common
to observe performance differences among competitors as small as hundredths or
even thousandths, so the question is whether the differences are significant.
This paper analyzes the results of the MeOffendEs@IberLEF 2021 competition and
proposes to make inference through resampling techniques (bootstrap) to support
Challenge organizers' decision-making.

研究了如何在挑战设置中评估不同竞争者（算法）的表现，分析了 MeOffendEs @ IberLEF 2021 比赛的结果，并提议通过重采样技术（引导）进行推理，以支持挑战组织者的决策。

挑战方案中分类器的比较

Comparison of classifiers in challenge scheme

This work compares supervised machine learning methods using reliable data
from constructive simulations to estimate the most effective moment for
launching missiles during air combat. We employed resampling techniques to
improve the predictive model, analyzing accuracy, precision, recall, and
f1-score. Indeed, we could identify the remarkable performance of the models
based on decision trees and the significant sensitivity of other algorithms to
resampling techniques. The models with the best f1-score brought values of
0.379 and 0.465 without and with the resampling technique, respectively, which
is an increase of 22.69%. Thus, if desirable, resampling techniques can improve
the model's recall and f1-score with a slight decline in accuracy and
precision. Therefore, through data obtained through constructive simulations,
it is possible to develop decision support tools based on machine learning
models, which may improve the flight quality in BVR air combat, increasing the
effectiveness of offensive missions to hit a particular target.

本研究基于可靠的建模模拟数据，比较了监督机器学习方法在估算在空中战斗期间发射导弹的最有效时刻的能力，并采用重新采样技术提高预测模型，发现基于决策树的模型性能显着更优越，而其他算法对重新采样技术具有显著灵敏度。获得最佳 f1-score 的模型在没有和使用重新采样技术时的值分别为 0.379 和 0.465，后者增长了 22.69%。因此，通过构建模拟数据可以开发基于机器学习模型的决策支持工具，从而提高空战中的飞行质量，增加攻击特定目标的有效性。

基于超视距空战模拟的导弹发射有效监督机器学习

Supervised Machine Learning for Effective Missile Launch Based on Beyond Visual Range Air Combat Simulations

A number of classification problems need to deal with data imbalance between
classes. Often it is desired to have a high recall on the minority class while
maintaining a high precision on the majority class. In this paper, we review a
number of resampling techniques proposed in literature to handle unbalanced
datasets and study their effect on classification performance.

研究分类问题中遇到的数据不平衡的情况，讨论了文献中提出的一些重新采样技术，并研究了它们对分类性能的影响。