Empirical research in Natural Language Processing (NLP) has adopted a narrow
set of principles for assessing hypotheses, relying mainly on p-value
computation, which suffers from several known issues. While alternative
proposals have been well-debated and adopted in other fields, they remain
rarely discussed or used within the NLP community. We address this gap by
contrasting various hypothesis assessment techniques, especially those not
commonly used in the field (such as evaluations based on Bayesian inference).
Since these statistical techniques differ in the hypotheses they can support,
we argue that practitioners should first decide their target hypothesis before
choosing an assessment method. This is crucial because common fallacies,
misconceptions, and misinterpretation surrounding hypothesis assessment methods
often stem from a discrepancy between what one would like to claim versus what
the method used actually assesses. Our survey reveals that these issues are
omnipresent in the NLP research community. As a step forward, we provide best
practices and guidelines tailored to NLP research, as well as an easy-to-use
package called 'HyBayes' for Bayesian assessment of hypotheses, complementing
existing tools.

该论文探讨了自然语言处理中假设评估的问题，提出使用贝叶斯方法评估假设，并提供了针对该领域的最佳实践和指南。