Question answering (QA) can only make progress if we know if an answer is
correct, but for many of the most challenging and interesting QA examples,
current evaluation metrics to determine answer equivalence (AE) often do not
align with human judgments, particularly more verbose, free-form answers from
large language models (LLM). There are two challenges: a lack of data and that
models are too big: LLM-based scorers can correlate better with human judges,
but this task has only been tested on limited QA datasets, and even when
available, update of the model is limited because LLMs are large and often
expensive. We rectify both of these issues by providing clear and consistent
guidelines for evaluating AE in machine QA adopted from professional human QA
contests. We also introduce a combination of standard evaluation and a more
efficient, robust, and lightweight discriminate AE classifier-based matching
method (CFMatch, smaller than 1 MB), trained and validated to more accurately
evaluate answer correctness in accordance with adopted expert AE rules that are
more aligned with human judgments.

通过采用职业人员 QA 竞赛来评估机器 QA 中答案的正确性，结合标准评估和一种更高效、稳健、轻量级的基于偏差 AE 分类器的匹配方法（CFMatch，小于 1 MB），培训和验证以更准确地评估答案的正确性，使得 QA 能够进一步发展。