We discuss problems the null hypothesis significance testing (NHST) paradigm
poses for replication and more broadly in the biomedical and social sciences as
well as how these problems remain unresolved by proposals involving modified
p-value thresholds, confidence intervals, and Bayes factors. We then discuss
our own proposal, which is to abandon statistical significance. We recommend
dropping the NHST paradigm--and the p-value thresholds intrinsic to it--as the
default statistical paradigm for research, publication, and discovery in the
biomedical and social sciences. Specifically, we propose that the p-value be
demoted from its threshold screening role and instead, treated continuously, be
considered along with currently subordinate factors (e.g., related prior
evidence, plausibility of mechanism, study design and data quality, real world
costs and benefits, novelty of finding, and other factors that vary by research
domain) as just one among many pieces of evidence. We have no desire to "ban"
p-values or other purely statistical measures. Rather, we believe that such
measures should not be thresholded and that, thresholded or not, they should
not take priority over the currently subordinate factors. We also argue that it
seldom makes sense to calibrate evidence as a function of p-values or other
purely statistical measures. We offer recommendations for how our proposal can
be implemented in the scientific publication process as well as in statistical
decision making more broadly.

建议弃用 NHST 范式和 p-value 替代其地位，作为一种证据，应该与其它因素一起考虑，包括先验证据、机制合理性、研究设计和数据质量、真实世界的成本和收益、发现的新颖性等。同时不应以统计显著性作为基本标准进行强行验证和报告。

弃用统计显著性

Abandon Statistical Significance

The machine learning community adopted the use of null hypothesis
significance testing (NHST) in order to ensure the statistical validity of
results. Many scientific fields however realized the shortcomings of
frequentist reasoning and in the most radical cases even banned its use in
publications. We should do the same: just as we have embraced the Bayesian
paradigm in the development of new machine learning methods, so we should also
use it in the analysis of our own results. We argue for abandonment of NHST by
exposing its fallacies and, more importantly, offer better - more sound and
useful - alternatives for it.

该研究证明了经典的 “空假设显著性检验” 在机器学习领域的不足，提出应该转向贝叶斯统计的方法来更好地分析机器学习结果。